4 hiring for lead data engineer Jobs in scottsdale, az
E-solutions Inc is Hiring a Lead Data Engineer Near Scottsdale, AZ
Role : Lead Data Engineer
Location : Scottsdale AZ (100% onsite)
Hire Type : FTE and CTH
Must have skill set: Spark, S3, Glue, AWS Redshift , python and stream set exp
6-8 years of IT experience focusing on enterprise data architecture and management.
• Experience in Conceptual/Logical/Physical Data Modelling & expertise in Relational and Dimensional Data Modelling
• Experience with Databricks & on Prem , Structured Streaming, Delta Lake concepts, and Delta Live Tables required
• Experience with Spark scala
• Data Lake concepts such as time travel and schema evolution and optimization
• Structured Streaming and Delta Live Tables with Databricks a bonus
• Experience leading and architecting enterprise-wide initiatives specifically system integration, data migration, transformation, data warehouse build, data mart build, and data lakes implementation / support
• Advanced level understanding of streaming data pipelines and how they differ from batch systems
• Formalize concepts of how to handle late data, defining windows, and data freshness
• Advanced understanding of ETL and ELT and ETL/ELT tools such as Data Migration Service etc
• Understanding of concepts and implementation strategies for different incremental data loads such as tumbling window, sliding window, high watermark, etc.
• Familiarity and/or expertise with Great Expectations or other data quality/data validation frameworks a bonus
• Familiarity with concepts such as late data, defining windows, and how window definitions impact data freshness
• Advanced level SQL experience (Joins, Aggregation, Windowing functions, Common Table Expressions, RDBMS schema design performance optimization)
• Indexing and partitioning strategy experience
• Debug, troubleshoot, design and implement solutions to complex technical issues
• Experience with large-scale, high-performance enterprise big data application deployment and solution
• Architecture experience in AWS environment a bonus
• Familiarity working with Lambda specifically with how to push and pull data, how to use AWS tools to view data for processing massive data at scale a bonus
• Experience with Gitlabs and CloudWatch and ability to write and maintain gitlabs for supporting CI/CD pipelines
• Experience working with AWS Lambdas for configuration and optimization and experience with S3
• Familiarity with Schema Registry, message formats such as Avro, ORC, etc.
• Ability to thrive in a team-based environment
• Experience briefing the benefits and constraints of technology solutions to technology partners, stakeholders, team members, and senior level of management
Job Summary
Contractor
Business Services
$122k-149k (estimate)
06/10/2024
07/06/2024
e-solutionsinc.com
ALBANY, NY
1,000 - 3,000
2003
MICHAEL MULLIN
$50M - $200M
Business Services
Related Companies
About E-solutions Inc
"Empowering Technology Services Founded in 2003, we are in our 19th year of structured growth. We are a talent-driven, talent acquisition company providing individual consultants, project teams, and strategic outsourcing services to clients in a wide range of industries. E-Solutions, Inc. is a certified minority-owned staffing firm spearheaded by industry experts. We offer award-winning solutions for the Global Talent Cloud. Our services include: Staff Augmentation IT Outsourcing Business Process Outsourcing We leverage our recruiting expertise to deliver high-end consulting services for a ...
variety of engagements. E-Solutions, Inc. has developed a business structure with no boundaries, which means we work as one collective entity. Our MSP partners and end customers have recognized our commitment to a partnering supplier model enabling us to expand to our present feat: 8 Nations 32 Operational Centers 1800+ Workforce Revenue ~$100+ Million Top 10 vendor for $10 Billion System Integrator Our clients extend from Fortune 500s to IT Companies, and we're experts at finding the right combination of solutions & solutions to fit any company's unique environment. We place professionals in contract, contract-to-hire, permanent hire, and statement-of-work-based positions with industry forerunners and innovative start-ups. We partner with talent ranging from IT professionals to engineering to skilled trades and many more. Our vast experience has provided us with a depth of knowledge that spans across industries and technologies. The following are the industry verticals, which we serve:- BFSI ITES Healthcare Energy & Utilities Light Engineering Manufacturing Warehouse & Logistics Administrative & Support Services Disclaimer E-Solutions Inc. Provides Equal Employment Opportunities (EEO) To All Employees And Applicants For Employment Without Regard To Race, Color, Religion,
More
Show Less
The job skills required for Lead Data Engineer include SQL, Python, Initiative, Data Quality, Data Warehouse, etc. Having related job skills and expertise will give you an advantage when applying to be a Lead Data Engineer. That makes you unique and can impact how much salary you can get paid. Below are job openings related to skills required by Lead Data Engineer. Select any job title you are interested in and start to search job requirements.
The following is the career advancement route for Lead Data Engineer positions, which can be used as a reference in future career path planning. As a Lead Data Engineer, it can be promoted into senior positions as a Database Engineer V that are expected to handle more key tasks, people in this role will get a higher salary paid than an ordinary Lead Data Engineer. You can explore the career advancement for a Lead Data Engineer below and select your interested title to get hiring information.
Similar Jobs
Popular Articles
