What are the responsibilities and job description for the Data Engineering Leader position at LatentView Analytics?
LatentView Analytics is revolutionizing the way businesses make decisions by providing cutting-edge analytics and decision sciences solutions. Our team is dedicated to delivering innovative data-driven solutions that help companies achieve their goals.
Job Responsibilities
The successful candidate will be responsible for:
Developing Scalable Data Pipelines
Build and maintain efficient data pipelines using Databricks, Apache Spark, and other cloud-based tools to ensure seamless data flow across systems.
Designing Cloud-Native Solutions
Collaborate with cloud providers (Azure, AWS) to design and implement cloud-native solutions for data storage, processing, and analytics using Databricks.
Working with Data Scientists/Analysts
Partner with data scientists, analysts, and business stakeholders to transform business requirements into actionable data solutions.
Optimizing Performance
Continuously optimize Spark and Databricks workflows for improved performance and cost efficiency.
Key Requirements
- Proven experience with Databricks and Apache Spark.
- Strong understanding of cloud platforms (Azure, AWS).
- Expertise in SQL, Python, Scala, or Java.
- Familiarity with data storage solutions.
