What are the responsibilities and job description for the Software Engineer - ML Inference position at Predibase?
We're looking to hire a software engineer working at the intersection of AI / ML and systems programming to develop our next generation LLM Inference Engine. As an engineer working on our ML Inference team, you will work to integrate new LLM inference techniques from the research to improve latency and throughput of LLM serving systems, you'll work closely with customers to optimize performance on specific use cases, and go deep into performance optimizations at multiple levels of the stack, including : PyTorch, C , and CUDA. As part of the role, you will have significant technical leadership responsibilities to define the roadmap and technical vision of our inference stack, and work closely with partner teams to build scalable, multi-replica serving infrastructure.
