What are the responsibilities and job description for the Senior Data Test Engineer position at FARM CREDIT FINANCIAL PARTNERS INC?
JOB SUMMARY: The Senior Data Test Engineer is responsible for ensuring the accuracy, reliability, and integrity of data pipelines, ETL processes, and data lakes within a cloud-based enterprise environment. This role involves developing and executing data validation strategies, automating data quality checks, and ensuring compliance with data governance standards. The ideal candidate has a strong background in SQL, Python, ETL testing, and data automation frameworks.
Key Responsibilities:
- Design, develop, and execute data validation and ETL test plans for large-scale data processing systems.
- Perform data reconciliation, integrity checks, and schema validation across multiple data sources.
- Implement automated data testing frameworks using Great Expectations, DataGaps, or QuerySurge.
- Validate data pipelines, transformations, and aggregations within Azure Data Lake, Synapse, and Snowflake.
- Develop and maintain SQL queries, Python scripts, and test automation for data quality assurance.
- Work closely with Data Engineers, BI teams, and stakeholders to define data quality KPIs and monitoring strategies.
- Troubleshoot data inconsistencies, ETL failures, and data migration issues in cloud environments.
- Support the implementation of CI/CD pipelines for automated data testing within DevOps frameworks.
- Monitor and enforce data governance, compliance, and best practices.
- Document test plans, test cases, and results, ensuring clarity and reproducibility.
ADDITIONAL FUNCTIONS:
- Maintain a closed-loop discrepancy reporting system.
- Prepare and maintain knowledge transfer documentation for onboarding new analysts and staff transitions.
- Mentor junior test engineers, guiding them in best practices and technical problem-solving.
- Promote active participation in quality practice groups.
- Own and enhance best practices in test strategy, planning, and execution.
- Advocate for product integration improvements and a stronger Voice of the Customer
OTHER DUTIES: This job description is not exhaustive and may evolve based on business needs. Duties, responsibilities, and activities may change at any time with or without notice.