Job Description
Job Description : Data Engineer
Role Overview :
We are looking for a highly skilled Data Engineer to design, develop, and optimize data pipelines and infrastructure that enable data-driven decision-making.
Key Responsibilities : Data Pipeline Development
- Design, build, and maintain ETL / ELT pipelines for data ingestion, transformation, and loading.
- Automate data workflows to ensure efficient data processing and availability.
- Integrate data from multiple sources such as APIs, streaming platforms, and databases.
Data Infrastructure Management
Develop and optimize data storage solutions , including data lakes , data warehouses , and databases .Collaborate with DevOps teams to manage cloud-based infrastructure for data pipelines.Ensure scalability, reliability, and performance of data systems to support business needs.Data Integration and Transformation
Implement real-time and batch data processing solutions using tools like Apache Kafka , Spark , or Flink .Transform raw data into structured formats to support analytics and reporting.Implement schema management, data partitioning, and version control.Data Quality and Governance
Monitor and improve data quality , ensuring consistency and accuracy.Implement and enforce data governance practices, including metadata management, data lineage, and cataloging.Collaborate with stakeholders to establish standards for data validation and transformation.Cloud-Based Solutions
Leverage cloud platforms like AWS , Azure , or GCP for building and deploying data infrastructure.Work with cloud-native tools (e.g., Azure Data Factory , AWS Glue , Google Cloud Dataflow ) for data integration and processing.Optimize cloud costs while ensuring high availability and performance.Collaboration and Documentation
Collaborate with data scientists, analysts, and business teams to understand data requirements.Document data workflows, pipeline designs, and infrastructure architecture.Provide technical support for data-related issues in production environments.Optimization and Performance
Identify and resolve bottlenecks in data pipelines and storage systems.Implement caching, indexing, and partitioning strategies to improve performance.Continuously monitor and optimize data workflows for scalability and efficiency.Key Qualifications : Technical Expertise
Proficiency in programming languages such as Python , Java , or Scala for data engineering tasks.Strong experience with SQL and database systems (e.g., PostgreSQL , MySQL , MongoDB ).Expertise in data processing tools like Apache Spark , Hadoop , or Flink .Familiarity with data orchestration tools like Apache Airflow , Luigi , or Prefect .Cloud and Big Data
Hands-on experience with cloud platforms ( AWS , Azure , or GCP ) and their data services (e.g., Redshift , BigQuery , Azure Synapse ).Experience with streaming technologies like Apache Kafka , RabbitMQ , or Kinesis .Knowledge of data storage solutions such as S3 , Azure Blob Storage , or HDFS .Preferred Skills
Experience with data visualization tools like Tableau , Power BI , or Looker is a plus.Knowledge of machine learning workflows and tools is desirable.Understanding of CI / CD pipelines for deploying data pipelines in production environments.Soft Skills
Strong problem-solving skills and attention to detail.Excellent communication and collaboration skills to work with cross-functional teams.Ability to prioritize tasks and manage time effectively in a fast-paced environment.Education and Experience :
Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.3 years of experience in data engineering, big data, or a related field.Relevant certifications like AWS Certified Data Analytics , Google Professional Data Engineer , or Azure Data Engineer Associate are preferred.Compensation, Benefits and Duration
Minimum Compensation : USD 56,000
Maximum Compensation : USD 224,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay / incentives, paid time off, and paid holidays are available for full time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Salary : $224,000
