What are the responsibilities and job description for the Principal Data Engineer position at Blue Health Intelligence(BHI)?
About Us :
BHI is a data and analytics company and helping health plans, researchers, healthcare providers, employers, and government agencies use data to improve the health outcomes, promote value-based care, and address health equity and increase the value of care provided. We leverage the industry's most comprehensive and highest quality commercial data assets combined with advanced analytics, AI / ML, and deep healthcare expertise to collaborate with our customers to produce measurable outcomes and value.
Based on our continuing growth, and the increasing value our team is producing in partnership with our customers, we are growing our data engineering team and are looking for someone to lead this effort with us.
Summary :
We are seeking a highly experienced Principal Data Engineer with a deep understanding of PySpark, Databricks, AWS or Azure, and cloud-based databases such as Snowflake. Proficiency in workflow management tools like Airflow and container orchestration systems such as Kubernetes is essential. Healthcare industry experience is a significant advantage. The ideal candidate will be responsible for designing, implementing, and maintaining data pipelines, while ensuring the highest levels of performance, security, and data quality.
Responsibilities :
- Design, develop, and maintain scalable, reliable, and secure data pipelines to process large volumes of structured and unstructured healthcare data using PySpark, Databricks, and cloud-based databases.
- Collaborate with data architects, data scientists, and analysts to understand data requirements and implement solutions that meet business and technical objectives.
- Leverage AWS or Azure cloud services for data storage, processing, and analytics, optimizing cost and performance.
- Utilize tools like Airflow for workflow management and Kubernetes for container orchestration to ensure seamless deployment, scaling, and management of data processing applications.
- Develop and implement data ingestion, transformation, and validation processes to ensure data quality, consistency, and reliability across various healthcare datasets.
- Monitor and troubleshoot data pipelines, proactively identifying and resolving issues to minimize downtime and ensure optimal performance.
- Establish and enforce data engineering best practices, ensuring compliance with data privacy and security regulations specific to the healthcare industry.
- Continuously evaluate and adopt new tools, technologies, and frameworks to improve the data infrastructure and drive innovation.
- Mentor and guide junior data engineers, fostering a culture of collaboration, learning, and growth within the team.
- Collaborate with cross-functional teams to align data engineering efforts with broader organizational goals and strategies.
- Is familiar with SOC 2 compliance and its impact on company policies and processes.
- Understands importance of adhering to SOC 2 requirements and maintains an effort to do so.
- Reviews and understands the Employee Handbook, and internal policies that define individual security responsibilities, and maintains segregation of duties in accordance to their role requirements
Requirements :
We offer competitive compensation and benefits packages, along with opportunities for career growth and development. Join our team of passionate and talented data professionals as we drive innovation in the healthcare industry.
Equal Employment Opportunity It is the policy of BHI to provide equal employment opportunity and advancement opportunities to all colleagues and qualified applicants for employment without regard to race, color, religion, national origin, sex, age, disability, sexual orientation, gender identity, or any other classification protected by the federal, state, or local laws
