Role: Data Engineer with Databricks and PySpark
Location: New York, NY - Remote to start

Job Description:

• 5 years working experience as Databricks Developer.
• 3 years working experience on Python, PySpark and AWS.
• 5 years relevant and progressive data engineering experience.
• Deep Technical knowledge and experience in Databricks, Python, Scala, Microsoft Azure architecture and platform including Synapse, ADF (Azure Data Factory) pipelines and Synapse stored procedures.
• Hands-on experience working with data pipelines using a variety of source and target locations (e.g., Databricks, Synapse, SQL Server, Data Lake, file-based, SQL and No-SQL database).
• Experience in engineering practices such as development, code refactoring, and leveraging design patterns, CI/CD, and building highly scalable data applications and processes.
• Experience developing batch ETL pipelines; real-time pipelines are a plus.
• Knowledge of advanced data engineering concepts such as dimensional modeling, ETL, data governance, data warehousing involving structured and unstructured data.
• Thorough knowledge of Synapse and SQL Server including T-SQL and stored procedures.
• Experience working with and supporting cross-functional teams in a dynamic environment.
• successful history of manipulating, processing and extracting value from large disconnected datasets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Advise, consult, mentor and coach other data and analytic professionals on data standards and practices.
• Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytic solutions.
• Develop and deliver documentation on data engineering capabilities, standards, and processes; participate in coaching, mentoring, design reviews and code reviews.
• Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives.
• Solve complex data problems to deliver insights that helps the organization achieve its goals.
• Knowledge and understanding of Boomi is a plus.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.