What are the responsibilities and job description for the Data Integration Specialist position at Powersolv, Inc.?
Our client values inclusion, integrity, innovation, empowerment, and offers a vibrant work environment. Department is the central IT agency oversees all major information and communications technology initiatives increasing the effectiveness of the information technology infrastructure, where the services provided are advanced, optimized, and responsive to the needs of the agency’s businesses, residents, and visitors.
Client is known for its rich history, vibrant culture, and diverse neighborhoods. Experience iconic landmarks such as Independence Hall, the Liberty Bell, and the Philadelphia Museum of Art. Enjoy a thriving food scene, lively music venues, and world-class museums and theaters. Benefit from affordable living costs, excellent universities, and a strong sense of community in the City of Brotherly Love.
Position Overview / Statement Of Work
We are seeking a skilled and experienced Data Integration Specialist for a temporary assignment to join our dynamic team. This position will be responsible for the migration, integration, and maintenance of data within Amazon S3 and Amazon Redshift environments. This position will also be responsible for ensuring data integrity and security throughout the entire process, from initial migration to continuous data operations. This position requires 3 or more years of experience working as a Data Integration Specialist.
Work Activities
Skills/experience of the assigned staff:
Client is known for its rich history, vibrant culture, and diverse neighborhoods. Experience iconic landmarks such as Independence Hall, the Liberty Bell, and the Philadelphia Museum of Art. Enjoy a thriving food scene, lively music venues, and world-class museums and theaters. Benefit from affordable living costs, excellent universities, and a strong sense of community in the City of Brotherly Love.
Position Overview / Statement Of Work
We are seeking a skilled and experienced Data Integration Specialist for a temporary assignment to join our dynamic team. This position will be responsible for the migration, integration, and maintenance of data within Amazon S3 and Amazon Redshift environments. This position will also be responsible for ensuring data integrity and security throughout the entire process, from initial migration to continuous data operations. This position requires 3 or more years of experience working as a Data Integration Specialist.
Work Activities
- Design and implement data pipelines to extract, transform and load data (ETL and/or ELT) data into Amazon S3 using Azure Data Factory and other tools
- Set up automated workflows to ensure continuous data migration
- Configure Amazon S3 buckets for data storage
- Set up data schemas and tables in Amazon Redshift
- Migrate cleaned and preprocessed data into Amazon S3 and Redshift
- Ensure data integrity and security during migration
- Monitor data pipelines for performance and reliability
- Perform regular maintenance to address any issues
- Improve data integration processes for increased efficiency and scalability
- Document data integration processes and procedures
Skills/experience of the assigned staff:
- Experience with various cloud platforms
- Proficient with Azure Data Factory
- Strong knowledge of Amazon AWS S3, AWS Redshift and AWS Lambda
- Proficient in SQL and ETL tools
- Effective communication and documentation abilities
- Expertise in data security and integrity measures
- Excellent problem-solving and analytical skills
- 3 years of experience creating, documenting, implementing, and maintaining data pipelines
- Experience using AWS Glue.
- Proficient with Python, C# and sparkSQL.
