Distributed Computing Engineer

Job Summary: We are seeking a highly skilled Distributed Computing Engineer to join our team. The ideal candidate will be responsible for designing and implementing a platform that allows large AI models to be executed across a fleet of many small, distributed processors and reassembling the results into a single response. This role involves providing architectural guidance to a larger team that will manage and distribute these jobs across a large fleet of computing agents.
Key Responsibilities:
Design and Implementation: Design and implement scalable and reliable distributed systems for deploying, managing, and updating AI models specifically for inferencing on edge devices. This includes considering factors like device heterogeneity, limited resources (CPU, memory, bandwidth), and intermittent connectivity. The focus will be on optimizing the inferencing pipeline.
Model Deployment and Optimization: Work with AI/ML engineers to optimize AI models specifically for inferencing on resource-constrained edge devices. This may involve model compression, quantization, pruning, and other optimization techniques tailored for inferencing. Develop and maintain the deployment pipeline for pushing models to edge devices.
Communication Protocols: Implement efficient and secure communication protocols between edge devices, the cloud, and other components of the distributed system. This will involve selecting and optimizing protocols for various data types (model updates, configuration parameters, and potentially limited amounts of input data for inferencing if required). Crucially, this will involve designing and implementing strategies for efficient and reliable model updates and potentially small data payloads for inferencing, considering network constraints.
Inferencing Pipeline Optimization: Design and implement the inferencing pipeline on edge devices, focusing on minimizing latency and maximizing throughput. This may involve optimizing data pre-processing, model loading, and inferencing execution.
Hardware Abstraction: Develop and maintain abstractions for different hardware platforms of varying capabilities to ensure portability and ease of deployment of AI models.
Scalability and Reliability: Ensure the platform can scale to support many edge devices of varying capabilities. Implement mechanisms for fault tolerance, redundancy,
and disaster recovery, particularly concerning model availability and inferencing reliability.
Performance Monitoring and Tuning: Monitor the performance of the distributed system and identify bottlenecks, specifically related to the inferencing pipeline. Implement optimizations to improve inferencing latency, throughput, and resource utilization.
Qualifications:

Master's degree or PhD in Computer Science, Engineering, or a related field or relevant work experience with these technologies at scale.

Proven experience in distributed computing (10 years), AI model deployment (2 years), and system architecture.

Strong understanding of distributed systems, parallel computing, and AI model inferencing at scale.

Experience with cloud computing platforms and tools.

Experience identifying and leveraging appropriate open source software.

Excellent problem-solving skills and attention to detail.

Strong communication and teamwork skills.
Preferred Qualifications:

Experience with AI model slicing and reassembly.

Knowledge of cloud-based distributed computing frameworks.

Familiarity with containerization and orchestration tools such as Docker and Kubernetes.

Experience with performance monitoring and optimization of distributed systems.
About Us: We are a forward-thinking company dedicated to leveraging cutting-edge technology to solve complex problems. Our team is composed of talented professionals who are passionate about innovation and excellence. Join us and be a part of a dynamic and collaborative environment where your contributions will make a significant impact.