What are the responsibilities and job description for the Data Engineer position at National Computer Systems?
Job Title: Data Engineer (PySpark, AWS Glue, AWS Lambda)
Location: Plano, TX(Hybrid)
Required : W2 Only H1 Transfer Work
Key Responsibilities:
Design, develop, and maintain data pipelines using PySpark, AWS Glue, and AWS Lambda.
Work with large-scale datasets, ensuring data accuracy, quality, and availability.
Implement data processing workflows to automate data transformation, cleaning, and integration.
Collaborate with cross-functional teams to understand data needs and optimize data architecture.
Optimize and troubleshoot data processing workflows for performance and cost-efficiency in AWS.
Ensure data security, privacy, and compliance standards are met.
Required Skills & Qualifications:
Strong experience in PySpark and data processing.
Expertise in AWS Glue and AWS Lambda for building scalable data pipelines.
Solid understanding of data modeling, ETL processes, and data storage solutions in AWS.
Proficiency in Python and SQL.
Experience with cloud data storage solutions like S3, Redshift, or RDS.
