Senior Data Engineer

Primary Purpose:

This role will be responsible for the design, development, and maintenance of robust data pipelines that transform raw data into reliable datasets enabling downstream analysis, decision-making, and advanced AI initiatives. This role will collaborate closely with data analysts, data scientists, and business stakeholders to ensure data accessibility, integrity, and optimal performance for a wide range of use cases. The role will implement scalable solutions for data storage and retrieval, including data warehouses or data lakes, while leveraging containerization for streamlined deployment. A focus on data quality, automation, and continuous improvement are essential. This role should apply modern technologies and best practices to deliver an exceptional data infrastructure that drives business value, supports AI-powered solutions, and integrates cutting-edge LLM capabilities.

Essential Responsibilities:

• Develop and implement robust ELT (Extract, Load, Transform) processes to move data from diverse sources to target systems, ensuring data integrity and efficiency.
• Actively lead/participate in evaluation of new data sources, acquisition of data, and delivery of data.
• Design and maintain scalable data infrastructure, including data warehouses, data lakes, and databases, to optimize data accessibility and performance for analysis and AI models.
• Implement data cleansing, validation, and quality control procedures to maintain the accuracy, completeness, and reliability of datasets.
• Partner with data analysts, data scientists, and business teams to understand data requirements, translate them into technical solutions, and support AI initiatives.
• Serve as a highly collaborative partner to ensure cross functional data governance, adoption/adherence to data quality standards/processes.
• Automate processes to increase efficiency and reduce manual intervention. Continuously evaluate and improve for optimal performance.
• Explore and implement modern data engineering tools, techniques, and cloud-based platforms to enhance the company's data capabilities. Be obsessed with improving the business data capabilities and business performance.
• Promote use of industry leading trends and new data management technologies.
• Adheres to applicable federal laws, rules, and regulations including those related to Anti-Money Laundering (AML) and the Bank Secrecy Act (BSA).
• Other duties as assigned.

Minimum Required Skills & Competencies:

• 5 years of data engineering and reporting experience.
• Proficiency in one or more programming languages (Python, GoLang, Scala).
• Solid understanding of relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
• Databricks experience
• Experience with big data processing frameworks such as Apache Spark, Kafka, or similar technologies.
• Knowledge of data warehousing concepts, modeling techniques (star schema, snowflake schema), and ELT processes.
• Experience working with containerization technologies (e.g., Docker, Kubernetes) to package, deploy, and manage applications in a consistent and scalable way.
• Experience with Power BI for data modeling, visualization, dashboard development, and reporting. Ability to understand business requirements and translate them into insightful visualizations.
• Understanding of CI/CD (continuous integration/continuous delivery), infrastructure as code (IaC), and configuration management tools (e.g., Terraform, Ansible, Jenkins).
• Familiarity with Azure.
• Attention to detail.
• Self-motivated.
• High level of organization and follow through.
• Strong analytical and problem-solving skills to diagnose data issues and design optimal solutions.
• Understanding of multiple disparate architectures.
• Excellent communication.
• High degree of personal integrity.

Desired Skills & Competencies:

• Proficiency in common machine learning frameworks such as TensorFlow or PyTorch. Ability to build, train, and deploy machine learning models for various AI use cases.
• Experience in working with Large Language Models (LLMs). Understands techniques like fine-tuning, prompt engineering, and evaluation of LLM outputs to tailor them for specific applications.
• Financial Services background
• Strong interpersonal and leadership skills.
• Advanced degree or Industry designations.

Physical Requirements:

• Express or exchange ideas by means of the spoken word via email and verbally.
• Exert up to 10 pounds of force occasionally, use your arms and legs, and sit most of the time.
• Have close visual acuity to perform activities such as analyzing data, viewing a computer terminal, reading, and preparing documentation.
• Not substantially exposed to adverse environmental conditions.
• The physical demands described here are representative of those that must be met by an employee to successfully perform the essential responsibilities of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform essential responsibilities.