What are the responsibilities and job description for the Data Engineer position at IT Avalon?
Job Details
SUMMARY : Responsible for the development of all phases in SQL, T-SQL, relational database programming/design, APIs, SSIS, and SSRS reporting.
BENEFITS:
ESSENTIAL DUTIES & RESPONSIBILITIES:
EDUCATION & EXPERIENCE:
WORK ENVIRONMENT: Individuals must be team-quality players in a fast-paced environment. The candidate must work in an environment with 50% development and 50% production support.
BENEFITS:
- Medical, Dental, Vision coverage
- Paid time off
- 401k savings plans
ESSENTIAL DUTIES & RESPONSIBILITIES:
- SQL Development:
- Write and optimize stored procedures, functions, table alterations, and indexes in SQL Server 2016 and 2019.
- Design and implement efficient queries using inner/outer joins and set-based logic.
- Performance tuning of stored procedures, set-based code, and indexing strategies.
- Leverage execution plans, statistics, and plan diagrams for troubleshooting and optimization.
- Data Integration & Pipeline Development:
- Develop and manage data pipelines using SSIS and Azure Data Factory.
- Consume and process data from APIs, handling JSON data efficiently.
- Configure and promote objects through development to production environments using version control and coding standards.
- ElasticSearch Expertise:
- Design and manage indexes and lifecycle management for optimal search performance.
- Develop ElasticSearch queries, including keyword, text, and vector search, and manage document ingestion.
- Leverage ML inference models within ElasticSearch pipelines to enhance retrieval and ranking mechanisms.
- Machine Learning Integration:
- Build and integrate machine learning models into search pipelines using Python.
- Experience with libraries such as TensorFlow, PyTorch, or scikit-learn.
- Familiarity with natural language processing (NLP) and large language models (LLMs).
- Advanced Search and Retrieval Systems:
- Integrate large language models (LLMs) for retrieval-augmented generation (RAG) solutions.
- Design and manage scalable, high-availability systems for efficient query processing and document retrieval.
- Utilize knowledge graphs and dynamic query generation to enhance search functionality.
- Agentic AI Systems:
- Build multi-agent systems for advanced search and recommendation engines.
- Use frameworks like LangChain or LangGraph for orchestrating agent reasoning and decision-making processes.
- Optimize multi-turn conversations, state management, and task collaboration across distributed agents.
- Collaboration and Ownership:
- Own the design and improvement of database schema and architecture.
- Translate business requirements into scalable system designs.
- Interact with cross-functional teams and act as a subject matter expert for data engineering and search technologies.
EDUCATION & EXPERIENCE:
- Bachelor's degree in Computer Science required Master's degree preferred
- Minimum 5 years of experience as an MS SQL programmer in a fast-paced, dynamic environment using a structured software development methodology
- Experience designing and implementing large-scale database solutions that are reliable, scalable, and perform at high levels
- Working knowledge of security and privacy issues, SQL 2019, and SQL 2022
WORK ENVIRONMENT: Individuals must be team-quality players in a fast-paced environment. The candidate must work in an environment with 50% development and 50% production support.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
