What are the responsibilities and job description for the Sr Software Engineer position at Positron AI?
About Us :
Positron.ai specializes in developing custom hardware systems to accelerate AI inference. These inference systems offer significant performance and efficiency gains over traditional GPU-based systems, delivering advantages in both performance per dollar and performance per watt. Positron exists to create the world's best AI inference systems.
Senior Software Engineer - Machine Learning Systems & High-Performance LLM Inference
We are seeking a Senior Software Engineer to contribute to the development of high-performance software that powers execution of open-source large language models (LLMs) on our custom appliance . This appliance leverages a combination of FPGAs and x86 CPUs to accelerate transformer-based models . The software stack is written primarily in modern C (C 17 / 20) and heavily relies on templates, SIMD optimizations, and efficient parallel computing techniques .
Key Areas of Focus & Responsibilities
- Design and implement high-performance inference software for LLMs on custom hardware.
- Develop and optimize C -based libraries that efficiently utilize SIMD instructions, threading, and memory hierarchy .
- Work closely with FPGA and systems engineers to ensure efficient data movement and computational offloading between x86 CPUs and FPGAs.
- Optimize model execution via low-level optimizations , including vectorization, cache efficiency, and hardware-aware scheduling.
- Contribute to performance profiling tools and methodologies to analyze execution bottlenecks at the instruction and data flow levels.
- Apply NUMA-aware memory management techniques to optimize memory access patterns for large-scale inference workloads.
- Implement ML system-level optimizations such as token streaming, KV cache optimizations, and efficient batching for transformer execution.
- Collaborate with ML researchers and software engineers to integrate model quantization techniques, sparsity optimizations, and mixed-precision execution .
- Ensure all code contributions include unit, performance, acceptance, and regression tests as part of a continuous integration-based development process .
Required Skills & Experience
Preferred Skills (Nice to Have)
Why Join Us?
