What are the responsibilities and job description for the Data Engineer with Databricks Expertise position at Next Ventures?
Contract Position : Onsite in Edison, NJ (4x per week)
Long-term Contract : Commitment for 3 Years
Hourly Rate : $90-$100 USD
Previous experience in Finance, especially in Capital Markets, is essential.
We are seeking a talented and experienced Data Engineer to join our client's dynamic team and play a key role in the development of innovative data solutions. If you have a genuine enthusiasm for optimizing big data pipelines, utilizing cloud platforms, and generating impactful insights, we encourage you to apply!
Your Responsibilities Will Include :
- Designing and constructing scalable data pipelines for efficient ingestion, transformation, and integration from various sources.
- Developing and enhancing Spark applications in Databricks to effectively process and analyze extensive datasets.
- Implementing Delta Lake , alongside data modeling and cloud-based warehousing solutions.
- Writing clean, efficient code using Python, SQL, and PySpark to fulfill data processing requirements.
- Working with structured, semi-structured, and unstructured data in both event-driven and streaming contexts.
- Ensuring high performance and scalability of Databricks workloads , while troubleshooting and optimizing jobs.
- Upholding best practices in data governance, security, and compliance across cloud platforms.
- Conducting code reviews to ensure optimal performance and compliance with industry standards.
Essential Skills :
Preferred Qualifications :
Salary : $90 - $100
