What are the responsibilities and job description for the Senior Data Engineer - PySpark position at DATAECONOMY?
Job Details
DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.
We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.
Senior Data Engineer - PySpark
Tampa, FL 33602
Full-time
Job Summary:
We are seeking a highly skilled and experienced Senior Data Engineer to lead the end-to-end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud-based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient solutions.
Key Responsibilities:
• Model Development: Lead the development of advanced models using AWS services such as EMR, Glue, and Glue Notebooks.
• Cloud Infrastructure: Design, build, and optimize scalable cloud infrastructure solutions with a minimum of 5 years of experience.
• ETL Pipeline Development: Create, manage, and optimize ETL pipelines using PySpark for large-scale data processing.
• CI/CD Implementation: Build and maintain CI/CD pipelines for deploying and maintaining cloud-based applications.
• Data Analysis: Perform detailed data analysis and deliver actionable insights to stakeholders.
• Collaboration: Work closely with cross-functional teams to understand requirements, present solutions, and ensure alignment with business goals.
• Agile Methodology: Operate effectively in agile or hybrid agile environments, delivering high-quality results within tight deadlines.
• Framework Development: Enhance and expand existing frameworks and capabilities to support evolving business needs.
• Documentation and Communication: Create clear documentation and present technical solutions to both technical and non-technical audiences.
Required Qualifications:
• 10 years of experience with Python programming.
• 5 years of experience in cloud infrastructure, particularly AWS.
• 3 years of experience with PySpark, including usage with EMR or Glue Notebooks.
• 3 years of experience with Apache Airflow for workflow orchestration.
• Solid experience with data analysis in fast-paced environments.
Domain Expertise:
• Strong understanding of capital markets, financial systems, or prior experience in the financial domain is a must.
Technical Skills:
• Proficiency with cloud-native technologies and frameworks.
• Familiarity with CI/CD practices and tools like Jenkins, GitLab CI/CD, or AWS CodePipeline.
• Experience with notebooks (e.g., Jupyter, Glue Notebooks) for interactive development.
Soft Skills:
• Excellent problem-solving skills and ability to handle complex technical challenges.
• Strong communication and interpersonal skills for collaboration across teams and presenting solutions to diverse audiences.
• Ability to thrive in a fast-paced, dynamic environment.
Preferred Qualifications:
• Experience with data governance and compliance frameworks.
• Familiarity with machine learning frameworks and techniques for financial modeling.
• Certifications in AWS or other cloud technologies.
If you are interested in the opportunity listed above, please share your resume at
If you aren t interested, perhaps you know an excellent referral?
