What are the responsibilities and job description for the Jr Data Engineer position at Flexon Technologies, INC.?
Job Description
Job Description
We are seeking a highly skilled Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing data pipelines, databases, and ETL processes. You will collaborate with data scientists, analysts, and software engineers to ensure efficient data flow, storage, and accessibility across various systems.
Key Responsibilities :
Design, develop, and maintain data pipelines using tools like Apache Airflow, Apache NiFi, or Prefect.
Build and optimize ETL processes for data extraction, transformation, and loading.
Manage and optimize data warehouses (Snowflake, Redshift, BigQuery) and databases (PostgreSQL, MySQL, MongoDB).
Work with cloud platforms (AWS, Azure, GCP) to manage data storage and processing solutions.
Implement data modeling techniques to optimize storage and retrieval.
Ensure data integrity, quality, and governance across systems.
Develop and maintain real-time data streaming solutions using Kafka, Spark Streaming, or Flink.
Write and optimize SQL queries for data analysis and reporting.
Automate data workflows using Python, Scala, or Java .
Develop and maintain big data processing using Apache Spark .
Work with Databricks for large-scale data processing and analytics.
Collaborate with cross-functional teams to define data requirements and business needs.
Monitor and improve data security and compliance with industry standards.
Required Skills & Qualifications :
Bachelor’s / Master’s degree in Computer Science, Data Engineering, or related field.
3 years of experience in data engineering or related roles.
Hands-on experience with data pipeline orchestration tools (Airflow, Luigi, Prefect).
Strong experience in SQL and NoSQL databases (PostgreSQL, MySQL, MongoDB, Cassandra).
Expertise in big data processing frameworks (Apache Spark, Hadoop, Flink).
Experience with cloud data services (AWS Redshift, Google BigQuery, Azure Synapse, Snowflake).
Experience with Databricks for large-scale data processing.
Strong programming skills in Python, Scala, or Java.
Experience with real-time data streaming (Kafka, Kinesis, Pulsar).
Familiarity with containerization and orchestration (Docker, Kubernetes).
Strong understanding of data governance, security, and compliance .
Knowledge of data warehouse design, OLAP, and dimensional modeling .
