What are the responsibilities and job description for the Data Engineer position at Careers Integrated Resources Inc?
Job Description: MUST HAVE SKILLS:
- 5 years of experience designing, developing and supporting complex data
pipelines.
5 years of Spark experience in batch and streaming mode
5 years of advanced SQL experience for analyzing and interacting with data
5 years experience in Big Data stack environments like Databricks, AWS EMR etc.
3 years of experience in scripting using Python
3 years of experience working on clound environment like AWS.
Strong understanding of solution and technical design.
Experience building cloud scalable high-performance data lake solutions
Experience with relational SQL & tools like Snowflake
Aware of Datawarehouse concepts
Performance tuning with large datasets
Experience with source control tools such as GitHub and related dev processes
Experience with workflow scheduling tools like Airflow or Databricks Workflow
Strong problem solving and analytical mindset
Able to influence and communicate effectively, both verbally and written, with
team members and business stakeholders
Comments for Suppliers: Must have skillset - Spark, SQL, Python. Strong good to have - Databricks and streaming.
- 5 years of experience designing, developing and supporting complex data
pipelines.
5 years of Spark experience in batch and streaming mode
5 years of advanced SQL experience for analyzing and interacting with data
5 years experience in Big Data stack environments like Databricks, AWS EMR etc.
3 years of experience in scripting using Python
3 years of experience working on clound environment like AWS.
Strong understanding of solution and technical design.
Experience building cloud scalable high-performance data lake solutions
Experience with relational SQL & tools like Snowflake
Aware of Datawarehouse concepts
Performance tuning with large datasets
Experience with source control tools such as GitHub and related dev processes
Experience with workflow scheduling tools like Airflow or Databricks Workflow
Strong problem solving and analytical mindset
Able to influence and communicate effectively, both verbally and written, with
team members and business stakeholders
Comments for Suppliers: Must have skillset - Spark, SQL, Python. Strong good to have - Databricks and streaming.
