What are the responsibilities and job description for the Data Engineer position at Aureon Consulting?
Aureon Consulting is currently seeking a skilled Data Engineer. The ideal candidate will have extensive experience integrating data from multiple sources, including legacy systems, cloud applications, structured databases, proprietary file formats, APIs, and event streams. This role requires proficiency in both relational and dimensional data models, particularly within Azure SQL Server, and expertise in ETL/ELT pipelines using tools such as SSIS, Azure Data Factory, Function Apps, Logic Apps, Databricks, SPARK, and Azure Synapse Analytics.
Responsibilities:
- Integrate data from various sources, including legacy systems, cloud applications, structured databases, proprietary file formats, APIs, and event streams.
- Develop and maintain ETL/ELT pipelines using tools like SSIS, Azure Data Factory, Function Apps, Logic Apps, Databricks, SPARK, and Azure Synapse Analytics.
- Implement and manage data mesh and event streaming architectures, such as Azure Event Hub and Kafka.
- Work with structured, semi-structured, and unstructured data forms, including SQL, JSON, and YAML.
- Utilize Microsoft Visual Studio and Azure DevOps for development and deployment (preferred).
- Follow the software development life cycle within an Agile framework.
- Administer and manage the organization's data infrastructure.
- Monitor and optimize database performance to ensure efficiency.
- Ensure database performance and security are maintained.
Required Qualifications:
- Proven experience integrating data from multiple sources
- Proficiency in both relational and dimensional data models, particularly within Azure SQL Server.
- Extensive experience with ETL/ELT pipelines
- Experience with data mesh and event streaming architectures, such as Azure Event Hub and Kafka.
- Proficiency with structured, semi-structured, and unstructured data forms, including SQL, JSON, and YAML.
- Familiarity with Microsoft Visual Studio and Azure DevOps (preferred).
- Understanding of the software development life cycle in an Agile framework.
- Experience in administrating and managing data infrastructure.
- Ability to monitor and optimize database performance and ensure security.
Visa sponsorship is not available for this position. Candidates must be able to work without the need of sponsorship now or in the future. Not open to third party applicants.
