Role: Azure Data Architect

Location: Louisville, KY (Day 1 Onsite)

Duration: Long Term

Job Description:

Azure Databricks, Python, Spark, ETL, Big Data/Hadoop/ Splunk Architect

About the Role:

We are seeking a highly skilled Azure Data Architect to design and implement scalable, robust data architecture solutions. The ideal candidate will have deep expertise in Azure Databricks, Python, Apache Spark, ETL pipelines, and Big Data/Hadoop platforms, with hands-on experience in Splunk Architecture. You will work closely with data engineers, analysts, and business stakeholders to ensure seamless data integration, processing, and analysis in a cloud-based environment.

Key Responsibilities:

Architect, design, and implement large-scale data solutions on Microsoft Azure, using Azure Databricks, Azure Data Lake, and related services.

Develop ETL pipelines for data integration from various sources into cloud environments, ensuring data quality, accuracy, and performance.

Lead initiatives in designing and developing big data solutions using Hadoop, Spark, and Azure Synapse Analytics.

Design and optimize distributed data processing frameworks, leveraging Apache Spark for real-time and batch processing.

Collaborate with business stakeholders to understand data requirements and translate them into technical specifications.

Architect and maintain Splunk environments, ensuring integration with data platforms, providing real-time analytics, and handling large-scale log and event data.

Ensure end-to-end data security, governance, and compliance on cloud platforms.

Lead, mentor, and collaborate with engineering teams in the implementation of scalable, fault-tolerant, and cost-effective cloud-based data solutions.

Continuously monitor and optimize the performance of cloud-based data solutions, ensuring minimal downtime and high availability.

Mandatory Skills:

Azure Databricks expertise, including cluster management, notebook development, and pipeline automation.

Proficiency in Python and Apache Spark for data engineering and processing tasks.

Strong hands-on experience with ETL processes using Azure services (e.g., Azure Data Factory, Logic Apps).

Expertise in Big Data technologies such as Hadoop, HDFS, Hive, or similar.

Experience in Splunk architecture for large-scale data ingestion, real-time monitoring, and analytics.

Deep understanding of Azure cloud architecture, including security, networking, and governance in cloud environments.

Experience with SQL and NoSQL databases for structured and unstructured data processing.

Preferred Qualifications:

Certifications in Microsoft Azure, such as Azure Solutions Architect or Azure Data Engineer.

Experience with additional data processing tools like Kafka, NiFi, or Flume.

Familiarity with DataOps practices and CI/CD pipelines for data engineering.

Knowledge of machine learning frameworks and libraries.