What are the responsibilities and job description for the Big Data Engineer with python position at Cloud Bigdata?
Job Details
Key Responsibilities:
- Collaborate with diverse teams including business, platform, technology, and analytic teams to design, build, and maintain scalable, high-performance data solutions across a variety of environments such as traditional data warehouses, Big Data platforms, and cloud-based solutions.
- Design and implement data pipelines for seamless ingestion, transformation, and extraction processes that support data curation, cleansing, and loading into conformed, fit-for-purpose data structures.
- Ensure compliance with security, data governance, and data quality standards, contributing to metadata management, data quality management, and the application of business rules throughout the data lifecycle.
- Work closely with internal and external systems to manage data sourcing, flow, structure, and expertise. Act as a bridge between business stakeholders and technical teams to implement and support both operational and analytic platforms.
- Leverage Google Cloud Platform, Hadoop, Hive, Kafka, Spark, Python, Linux shell scripting, SAS, Teradata, Oracle, and Informatica to develop and maintain high-performance data pipelines and data engineering frameworks.
- Take ownership of MLOps processes for machine learning and AI deployments in cloud or big data environments, ensuring effective integration and scalability.
- Provide leadership and guidance to junior team members, and assist in setting standards for data ingestion, transformation, and delivery. Collaborate across teams to maintain best practices and drive continuous improvement in data engineering methodologies.
Required Qualifications:
- Bachelor's Degree in Software Engineering, Information Systems, Computer Science, Data Science, or a related field.
- At least 3 years of experience in data platform development, data engineering, software development, or data science.
- A minimum of 1 year of hands-on experience with Big Data or cloud data platforms.
- Proficiency in SQL for querying and managing large datasets.
- Solid experience in Data Warehousing and Data Modeling.
- Strong problem-solving abilities and analytical thinking.
- Excellent communication skills, both verbal and written.
- FHIR-based healthcare data solutions, ensuring compliance with HIPAA, SNOMED CT, and LOINC standards.
Equal Opportunity Employer We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
