Senior Data Engineer Job Description

A large equipment manufacturing company is seeking a Senior Data Engineer to develop and execute the data analytics strategy encompassing the acquisition, processing, and machine learning of engineering or service data. This will primarily be applied to preventive maintenance and intelligent control of on or off-road vehicles.

Responsibilities:

• Develop scalable ETL solutions (structured and unstructured data from source systems to streaming analytics platforms)
• Test and validate to support the accuracy of data transformations and data verification used with company-wide analytics
• Assist with data quality and security, ensuring proper data governance within the enterprise data warehouse systems and data lake across all segments of the business
• Assist with the improvement of data processes, troubleshoot data processing issues
• Develop design processes and prototyping for fast solution delivery
• Maintain documentation and reference architecture for application roadmap and architectural governance
• Utilize best practices for database design, data modeling, ETL processes, monitoring, job scheduling, etc.
• Lead or assist feasibility studies and project estimates (budget development, manpower, project timelines, etc.) on proposed development projects and research

Requirements:

• Bachelor's degree in data science, computer science, or related field
• 5 years of working as a data engineer, ETL developer, and/or data warehouse DBA
• Experience with big data tools and architectures, such as Hadoop, Hive, Spark, and HDFS
• Experience with Databricks, Azure cloud services, Synapse, or similar technologies
• Experience with Azure DevOps
• Experience with developing highly responsive data structures, metadata capture strategies, ontologies, and data dictionaries
• Experience with Git (version control)
• Working knowledge of telematics interfaces and streaming solutions (Kafka, MQTT, NiFi, etc.)
• Ability to handle multiple tasks in a fast-paced environment, both independently and as part of a team
• Automotive or heavy-duty on or off-road vehicle, digital data bus, including Ethernet or Controller Area Network (CAN) experience
• Effectively present information and respond to questions from leadership and peers
• Strong proficiency with SQL, Bash, and Python (and experience with the Anaconda distribution)