What are the responsibilities and job description for the Lead Agentic Data Engineer position at TalentBurst, an Inc 5000 company?
Position: Lead Agentic Data Engineer
Location: Richmond, VA (Hybrid)
Duration: 7 Month Renewable Contract
Answer Question
Client is seeking a highly skilled professional to design, develop, and deploy data pipelines that leverage agentic AI to solve real-world problems. The ideal candidate will have experience in designing data processes to support agentic systems, ensuring data quality, and facilitating interaction between agents and data.
Responsibilities
Location: Richmond, VA (Hybrid)
Duration: 7 Month Renewable Contract
Answer Question
- Understanding the Big data Technologies Required
- Experience developing ETL and ELT pipelines Required
- Experience with Spark, GraphDB, Azure Databricks Required
- Expertise in Data Partitioning Required
- Experience with Data conflation Required
- Experience developing Python Scripts Required
- Experience training LLMs with structured and unstructured data sets Required
- Experience with GIS spatial data Required
Client is seeking a highly skilled professional to design, develop, and deploy data pipelines that leverage agentic AI to solve real-world problems. The ideal candidate will have experience in designing data processes to support agentic systems, ensuring data quality, and facilitating interaction between agents and data.
Responsibilities
- Guide and mentor AI engineers, helping them develop their skills and knowledge in the field.
- Lead and manage AI projects, ensuring they stay on track, meet deadlines, and the findings are actionable and relevant.
- Contribute to the creation and implementation of AI strategies that align with the organization's goals and objectives.
- Design and develop data pipelines for agentic systems, and develop robust data flows to handle complex interactions between AI agents and data sources.
- Use advanced mathematical modeling, statistical analysis, and optimization techniques to gather and analyze data, identifying problems and developing solutions to improve efficiency in prompts.
- Train and fine-tune large language models and design and build the data architecture, including databases, data warehouses, and data lakes, to support various data engineering tasks.
- Develop and manage Extract, Load, Transform (ELT) processes to ensure data is accurately and efficiently moved from source systems to analytical platforms used in data science.
- Implement data pipelines that facilitate feedback loops, allowing human input to improve system performance in human-in-the-loop systems.
- Work with vector databases to store and retrieve embeddings efficiently.
- Collaborate with data scientists and engineers to preprocess data, train models, and integrate AI into applications.
- Optimize data storage and retrieval with high performance.
- Strong data engineering fundamentals.
- Utilize big data frameworks like Spark/Databricks.
- Training LLMs with structured and unstructured data sets.
- Understanding of Graph DB.
- Experience with Azure Blob Storage, Azure Data Lakes, Azure Databricks.
- Experience implementing Azure Machine Learning, Azure Computer Vision, Azure Video Indexer, Azure OpenAI models, Azure Media Services, Azure AI Search.
- Determine effective data partitioning criteria.
- Utilize data storage system Spark to implement partition schemes.
- Understanding core machine learning concepts and algorithms.
- Familiarity with cloud computing skills.
- Strong programming skills in Python and experience with AI/ML frameworks.
- Proficiency in vector databases and embedding models for retrieval tasks.
- Expertise in integrating with AI agent frameworks.
- Experience with cloud AI services (Azure AI).
- Experience with GIS spatial data.
- Strong leadership, excellent problem-solving, and communication skills.
- Proven experience in leading projects and teams, including the mentorship of AI engineers.
- The ability to engage in critical evaluation of information, hypothesis testing, and scenario analysis.
- Flexibility in learning and adopting new technologies, methodologies, and tools to stay at the forefront of AI trends.
- Experience with Department of Transportation data domains developing an AI Composite Agentic Solution designed to identify and analyze data models, connect & correlate information to validate hypotheses, forecast, predict and recommend potential strategies, and conduct What-if analysis.
- Bachelor's or master's degree in computer science, AI, Data Science, or a related field.
