JD For Databricks architect: - Architect scalable and efficient data pipelines and workflows on Databricks. - Integrate Databricks with other data services and tools, such as data lakes, warehouses, and machine learning platforms. -Must have snowflake and databricks experience - Oversee data ingestion, storage, processing, and retrieval in Databricks. - Ensure data quality, security, and compliance within the platform. - Optimize Databricks clusters for performance and cost-efficiency. - Monitor and troubleshoot performance issues. - Work closely with data engineers, data scientists, and business stakeholders to ensure the architecture aligns with business needs. - Provide guidance on best practices for data engineering and analytics on Databricks. - Implement automation for data pipeline deployment and maintenance. - Integrate Databricks with CI/CD pipelines for continuous development and deployment. - Document architectural decisions, processes, and best practices. - Train team members and stakeholders on using Databricks effectively. Skills - - Strong knowledge of Apache Spark and its implementation on Databricks. - Proficiency in programming languages such as Python, Scala, or SQL. - Familiarity with cloud platforms (e.g., Snowflake,AWS, Azure, Google Cloud) where Databricks is deployed. - Experience in designing ETL processes and data pipelines. - Understanding of data modeling, warehousing, and big data technologies. - Ability to translate business requirements into technical solutions. - Strong problem-solving skills, especially related to data processing and performance optimization. - Knowledge of data security best practices, including encryption, access control, and data governance. - Excellent communication skills for collaborating with technical and non-technical stakeholders. - Abil