What are the responsibilities and job description for the Databricks Engineer position at Techstra Solutions?
Databricks Engineer
Techstra Solutions is seeking a highly skilled and experienced Databricks Engineer to join our dynamic team. The ideal candidate will have expertise in cloud technologies, data warehousing, ETL processes, and big data analytics. This role requires proficiency in Azure cloud services, Databricks technologies, and various data integration tools.
Key Responsibilities:
Design, develop, and optimize cloud-based data solutions leveraging Azure and Databricks.
Develop and maintain ETL pipelines, ensuring data accuracy and reliability.
Manage and transform structured and unstructured data for business intelligence and analytics.
Implement best practices for data processing, storage, and retrieval.
Optimize data formats including SCD1, SCD2, Parquet, GIS, and Timeseries.
Write and maintain technical design specifications for data integration and transformation.
Collaborate with cross-functional teams to capture, analyze, and translate business needs into functional IT requirements.
Ensure high availability, disaster recovery, and performance tuning of data applications.
Lead small IT projects or sub-teams and support Agile/Scrum and waterfall methodologies.
Preferred Qualifications:
Expertise in cloud technologies, including Databricks and Azure.
Strong knowledge of Azure cloud platform and its applications.
Experience with Databricks warehouse and big data technologies like Spark.
Proficiency in data integration tools including ODI, IDMC, ADF, and Synapse.
Familiarity with Oracle Golden Gate, Cosmos DB, and Hyperscale DB.
Experience in SQL (DML, DCL, DDL) with Oracle (11g - 12c) and Microsoft SQL Server (2008r2 - 2016).
Proficiency with scheduling tools like Automic UC4 and various BI tools such as Oracle BI, Informatica, Business Objects, and SAS.
Strong understanding of data modeling, application re-engineering, and performance tuning.
Experience managing IT projects using Agile/Scrum and waterfall methodologies.
Bachelor's degree in computer science or related field with 4-7 years of experience, or an equivalent combination of education and work experience (6-9 years).
Strong problem-solving, analytical skills, and excellent communication abilities.
Equal Employment Opportunity Statement
Techstra Solutions is an equal opportunity employer. The Company makes its decisions on merit, and its policy of equal opportunity prohibits discrimination in all phases of the employment process, including, but not limited to, recruitment, hiring, promotion, selection, transfer, demotion, layoff, termination, compensation, benefits, and other terms and conditions of employment. The policy of equal opportunity applies without regard to race, color, creed, religion, gender,, sexual orientation, gender identification, pregnancy, marital status, national origin, ancestry, age, disability that can reasonably be accommodated without undue hardship, military status, veteran status, genetic predisposition or carrier status, alienage or citizenship, domestic partnership status, arrest or conviction record, status as a victim of domestic violence, or any other protected categories under federal, state, or local law. The Company also prohibits discrimination or harassment based upon the perception that a person has, or is associated with a person who has, any of these characteristics.
