What are the responsibilities and job description for the Senior Data Engineer position at Ztek Consulting?
Job Details
Job Title: Senior Data Engineer
Location: Wilmington, DE
Experience Required: 10 years
Note: Delta Lake & Azure Data Lake Storage is Mandatory.
Job Description:
- Knowledge of distributed systems (e.g., Apache Spark or Databricks).
- Knowledge of Delta Lake and Lakehouse architecture.
- Experience with multiple data programming languages (Spark-SQL, Pyspark, and Pandas).
- Experience with On-premises databases such as SQL server, Oracle, etc.
- Experience with version control (e.g., Git), DevOps, CI/CD.
- Design and build data pipelines using Spark-SQL and Pyspark in Azure Databricks.
- Build and maintain Lakehouse architecture in ADLS / Databricks.
- Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion, etc.
- Work with the DevOps team to deploy solutions in production environments.
- Collaborate with Data Science and Business Intelligence teams to share key learnings, leverage ideas and solutions, and propagate best practices.
- Apply change management tools including training, communication, and documentation to manage upgrades, changes, and data migrations.
ABOUT Ztek:
Website:
Ztek Consulting Inc is a minority- and woman-owned business enterprise that leverages a unique blend of human talent, machine learning algorithms, and artificial intelligence to provide customized talent management solutions for clients. Ztek s diversity-focused hiring models have proven successful, particularly in hiring veterans. The company's veteran-focused hiring and deployment program, Z-V.E.T, matches skilled veterans with clients' needs in various areas.
Ztek Consulting | Talent Matters
Ztek is a minority and woman-owned business enterprise that provides customized talent management solutions for clients.
