Job Purpose

The Sr. Data Engineer will be responsible for building and optimizing the data platform. The Engineer will provide the development and automation of computing processes to detect, predict and respond to opportunities in business operations. The Engineer will work with a variety of disparate datasets that encompass many disciplines and business units. They will strive to transform and implement true business integration, leveraging top-notch data integration best practices. Merging and securing data in a way that reduces the cost to maintain and increases the utilization of enterprise-wide data as an asset and developing business intelligence.

Essential Responsibilities

• Designs and engineers solutions that align to architecture patterns and security guidelines for data, compute and technology platforms, meeting business needs while evaluating tradeoffs between cost, performance, maintainability, and scalability.

• Works in partnership with business product owners, and DevSecOps to design, engineer and deliver mature scalable and robust business capabilities.

• Collaborates with IT architecture and IT operations to identify and implement optimal engineering solutions for the company.

• Leads engineering and planning initiatives related to capabilities, future roadmaps, operations, and strategic planning.

• Designs and engineers business and technology innovation that drives the organization's top and bottom lines.

• Stays current with industry trends, making recommendations of new technologies that deliver strategic business value and reduce costs.

• Designs and engineers architecture solutions that maximize reuse and are efficient, maintainable, scalable, stable, highly available, portable, secure, and implemented correctly.

• Strong communication competency across all levels of stakeholders, providing engineering guidance and insight into best practices and guardrails.

• Follows and enforces organizational policies and goals for IT security, change management, and operational risk.

• Contextualizes information for the correct audience level into clear and concise materials with thoughtful attention to detail and quality.

• Provides engineering consulting services within domain to help achieve desired business and operational outcomes across Jackson.

• Designs and engineers the service management processes for maintaining the technology platforms in production.

• Acquires and maintains a working knowledge of Jackson processes and procedures across various departments and ensures that proper controls are put in place.

• Promotes quality by performing code reviews, developing and enforcing coding standards, recognizing system deficiencies, and implementing effective solutions coaching other team members in this domain.

• Actively seeks and participates in personal and team development opportunities and maintains knowledge in this domain.

• Researches, plans, and executes the migration of on-prem data infrastructure to Azure, including evaluating tradeoffs between cost, performance, and maintainability.

• Delivers solutions that align to architecture patterns for data platforms and movement.

• Develops and maintains data pipelines for complex business use cases.

• Develops, designs, and builds solutions on a platform dedicated to large-scale processing of various data sets.

• Collaborates with IT architecture and IT operations to identify and implement the most optimal data engineering solutions for the company.

• Collaborates with team to execute and iterate on application development, including interfacing with legacy databases, parsing raw structured and unstructured data, documenting the data warehouse, debugging, and deploying to production environments.

• Collaborates with other teams to clarify and answer complex business questions using statistical and graphical approaches.

• Employs exceptional problem-solving skills, with the ability to see and solve issues before applications are moved to production systems.

• Sets, communicates, and reinforces technical standards.

• Leads and develops best practices for larger Data Engineer Community of Practice (Data CoP).

• Stays current with industry trends, makes recommendations of new technologies that deliver strategic business value and reduce costs.

Other Duties

• May provide training to users on new/updated systems.

• Provides trending and related support for production issues as needed.

• Ability to work extended hours as needed.

• Mentors junior engineers on practices within Jackson.

• Performs other duties and/or projects as assigned.

Knowledge, Skills and Abilities

• Knowledge of industry standards, emerging technologies; and security and system best practices.

• Excellent verbal and written communication skills including presentation creation and delivery.

• Experience with the Microsoft Cloud ecosystem.

• Demonstrated scripting skills in a system automation language.

• Strong organizational skills; ability to guide task prioritization to meet project deadlines.

• Strong collaboration skills with the ability to build high performance teams, influencing across all levels in the organization.

• Ability to learn and maintain a comprehensive understanding of Jackson s business and technology.

• Demonstrated record of successful, complex project completion with minimal supervision including skills in design, requirement gathering, project planning, prioritization, coordination, multi-tasking, and problem solving.

• High level understanding of Project Management tools, techniques and methodologies including Lean, Agile, and/or SAFe.

• Ability to explain and communicate technical concepts clearly to a Senior level audience.

• Ability to conduct gap analysis and identify possible solutions for continuous improvements.

• Fluency in Python, R, Powershell and Java with experience parsing, manipulating, and converting data to and from a wide range of formats (CSV, json, XML, html, SQL tables, etc.).

• Deep understanding of modern database concepts and SQL syntax, including experience with DB2, MongoDB, SQL Server, CosmosDB, Data Lake, Hadoop, etc.

• Deep understanding of modern data migration/messaging platforms including experience with Kafka, SAP Data Service, IBM InfoSphere, etc.

• Familiarity with the Azure ecosystem, including tradeoffs between services that meet overlapping needs.

• Hands-on experience with a batch data processing framework/technology (Spark, Data Bricks, Hive, etc.).

• Basic understanding of modern software development practices and application frameworks.

• Experience building and optimizing big data data pipelines, architectures and data sets.

• Strong analytic skills related to working with unstructured datasets.

• Build processes supporting data transformation, data structures, metadata, dependency and workload management.

Qualifications

• Bachelor's Degree in Computer Science/Engineering and/or equivalent education or experience required.

• 5 years related IT experience required.

• 5 years in Data Engineering required.

• 3 years in software development or related position required.

• 5 years of SQL, relational database experience and unstructured datasets required.

• 3 years experience of Big Data, Spark, Python, Java, SDLC, Docker, Terraform, Config Management and Monitoring required.

• 2 years experience using Agile methodologies, data pipelines, data streaming, data as a service (REST APIs), CI/CD pipelines, cloud technologies, IBM Infosphere (DataStage, Quality Stage), Parquet, JSON, AVRO required.

• 2 years experience working with Jupyter notebooks, Google Colab, R, SAS Data Visualization, microservices and data modeling preferred.

• 2 years in IT infrastructure/operations role preferred.

• Certification in Azure Fundamentals & Data Engineer upon hire preferred.