What are the responsibilities and job description for the Sr. Data Engineer position at Lensa?
Lensa is the leading career site for job seekers at every stage of their career. Our client, Insight Global, is seeking professionals in New Albany, OH. Apply via Lensa today!
Job Description
Day to day:
Insight Global is looking for a remote Data Engineer. Candidates will team up with the engineering teams and enterprise architecture (EA) to define standards, design patterns, accelerators, development practices, DevOps and CI/CD automation while creating and maintaining the data ingestion, quality testing and audit framework. Candidates will conduct complex data analysis to answer the queries from Business Users or Technology team partners either directly from Analysts or stemmed from one of the Reporting tools such as PowerBI, Tableau, OBIEE and build and automate the data ingestion, transformation and aggregation pipelines using Azure Data Factory, Databricks / Spark, Snowflake, Kafka as well as Enterprise Scheduler tools such as CA Workload automation or Control M. Candidates will setup and evangelize the metadata driven approach to data pipelines to promote self-service and continuously improve the data quality and audit monitoring as well as alerting while constantly evaluate the process automation options and collaborate with engineering as well as architecture to review the proposed design.
Additional Responsibilities
Demonstrate mastery of build and release engineering principles and methodologies including source control, branch management, build and smoke testing, archiving and retention practices
Adhere to and enhance and document the design principles, best practices by collaborating with Solution and in some cases Enterprise Architects
Participate in and support the Data Academy and Data Literacy program to train the Business Users and Technology teams on Data
Respond SLA driven production data quality or pipeline issues
Document the Data Flow Diagrams, Data Models, Technical Data Mapping and Production Support Information for Data Pipelines
Pay Rate: $50-55/hr USD
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills And Requirements
experience in an Enterprise Data Management or Data Engineering role
3 of hands-on experience in building metadata driven data pipelines using Azure Data Factory, Databricks / Spark for Cloud Datalake
5 years hands on experience with using one or more of the following for data analysis and wrangling Databricks, Python / PySpark, Jupyter Notebooks
Expert level SQL knowledge on databases such as but not limited to Snowflake, Netezza, Oracle, Sql Server, MySQL, Teradata
Experience working in a multi developer environment and hands on experience in using either azure devops or gitlab
Experience or strong understanding of the traditional enterprise ETL platforms such as IBM Datastage, Informatica, Pentaho, Ab Initio etc. experience on one or more of big data technologies such as Cloudera Hadoop, Pivotal, Vertica, MapR
Functional knowledge of some of the following technologies - Terraform, Azure CLI, PowerShell, Containerization (Kubernetes, Docker)
Functional knowledge of one or more Reporting tools such as PowerBI, Tableau, OBIEE null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Job Description
Day to day:
Insight Global is looking for a remote Data Engineer. Candidates will team up with the engineering teams and enterprise architecture (EA) to define standards, design patterns, accelerators, development practices, DevOps and CI/CD automation while creating and maintaining the data ingestion, quality testing and audit framework. Candidates will conduct complex data analysis to answer the queries from Business Users or Technology team partners either directly from Analysts or stemmed from one of the Reporting tools such as PowerBI, Tableau, OBIEE and build and automate the data ingestion, transformation and aggregation pipelines using Azure Data Factory, Databricks / Spark, Snowflake, Kafka as well as Enterprise Scheduler tools such as CA Workload automation or Control M. Candidates will setup and evangelize the metadata driven approach to data pipelines to promote self-service and continuously improve the data quality and audit monitoring as well as alerting while constantly evaluate the process automation options and collaborate with engineering as well as architecture to review the proposed design.
Additional Responsibilities
Demonstrate mastery of build and release engineering principles and methodologies including source control, branch management, build and smoke testing, archiving and retention practices
Adhere to and enhance and document the design principles, best practices by collaborating with Solution and in some cases Enterprise Architects
Participate in and support the Data Academy and Data Literacy program to train the Business Users and Technology teams on Data
Respond SLA driven production data quality or pipeline issues
Document the Data Flow Diagrams, Data Models, Technical Data Mapping and Production Support Information for Data Pipelines
Pay Rate: $50-55/hr USD
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills And Requirements
experience in an Enterprise Data Management or Data Engineering role
3 of hands-on experience in building metadata driven data pipelines using Azure Data Factory, Databricks / Spark for Cloud Datalake
5 years hands on experience with using one or more of the following for data analysis and wrangling Databricks, Python / PySpark, Jupyter Notebooks
Expert level SQL knowledge on databases such as but not limited to Snowflake, Netezza, Oracle, Sql Server, MySQL, Teradata
Experience working in a multi developer environment and hands on experience in using either azure devops or gitlab
Experience or strong understanding of the traditional enterprise ETL platforms such as IBM Datastage, Informatica, Pentaho, Ab Initio etc. experience on one or more of big data technologies such as Cloudera Hadoop, Pivotal, Vertica, MapR
Functional knowledge of some of the following technologies - Terraform, Azure CLI, PowerShell, Containerization (Kubernetes, Docker)
Functional knowledge of one or more Reporting tools such as PowerBI, Tableau, OBIEE null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.
Salary : $50 - $55
