Cluster Deployment Operations Engineer

Cerebras Systems has pioneered a groundbreaking chip and system that revolutionizes deep learning applications. Our system empowers ML researchers to achieve unprecedented speeds in training and inference workloads, propelling AI innovation to new horizons. Condor Galaxy 1 (CG-1), a supercomputer set to revolutionize the world of artificial intelligence. With an astounding processing power of 4 ExaFLOPs, 54 million cores, and a cutting-edge 64-node architecture, the CG-1 is the first milestone of a larger project that will redefine the possibilities of AI. The successful completion and deployment of the CG-1, the first of nine powerful supercomputers, is a significant achievement for Cerebras. As we enter phase 2 of the project with CG2, we are taking a bold step towards creating a network of interconnected supercomputers that will collectively deliver a mind-boggling 36 ExaFLOPs of AI compute power upon completion.  About the JobIn this role, you will be responsible for deploying and managing clusters in highly distributed and complex environments, ranging from a few nodes to thousands. You will work on a critical cluster deployment tool that manages deployments, with an emphasis on scalability, reliability, and incremental build-outs in production environments. This will include planning, building, troubleshooting, and maintaining clusters in live data centers where portions of the infrastructure may already be in production use. Key areas of responsibility will include running and troubleshooting cluster deployments, and addressing networking challenges that may arise during the deployment process. ResponsibilitiesPlan and execute cluster deployments, from small-scale to massive distributed systems. Manage hands-on aspects of the deployments, coordinating with data center staff for hardware configurations and necessary maintenance. Troubleshoot issues related to networking (e.g., BGP, cluster creation hurdles, or cabling errors) and hardware (e.g., hardware DOA). Monitor and maintain systems to ensure uptime, performance, and reliability. Collaborate with cross-functional teams including hardware vendors, data center operations, and network engineers to manage the entire lifecycle of deployment. Ensure comprehensive documentation is created and maintained for deployments, configurations, and operational processes. Develop tools, scripts, or playbooks to automate routine tasks and deployment processes. QualificationsProficiency in scripting and practical coding, particularly in Shell and Python (Go is a plus). Strong experience troubleshooting, analyzing, and administering large-scale, distributed systems. 5 years of experience in data center operations and Linux system administration. Knowledge and hands-on experience with network configuration and operations. Expertise in hardware operations including networking components (e.g., cabling, switches, routers). Preferred QualificationsExperience with Kubernetes and the Prometheus monitoring stack. Experience with CI / CD tools (e.g., Git, Jenkins, etc.). Familiarity with BGP and other networking protocols, including troubleshooting at Layer 1,2,3. Experience with automation tools for deployments, monitoring, and operational efficiency (such as creating playbooks or automated scripts). Why Join CerebrasPeople who are serious about software make their own hardware. At Cerebras we have built a breakthrough architecture that is unlocking new opportunities for the AI industry. With dozens of model releases and rapid growth, we’ve reached an inflection  point in our business. Members of our team tell us there are five main reasons they joined Cerebras : Build a breakthrough AI platform beyond the constraints of the GPUPublish and open source their cutting-edge AI researchWork on one of the fastest AI supercomputers in the worldEnjoy job stability with startup vitalityOur simple, non-corporate work culture that respects individual beliefsRead our blog :  Five Reasons to Join Cerebras in 2024.Apply today and become part of the forefront of groundbreaking advancements in AI.Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.This website or its third-party tools process personal data. For more details, click here to review our CCPA disclosure notice.