What are the responsibilities and job description for the Data Engineer Lead_Pittsburgh, PA_Only on W2_No 1099/C2C position at Chelsoft Solutions Co.?
The ideal candidate is an experienced Data Engineering Lead with a strong background in Snowflake, Python, PySpark, SQL, and Azure cloud-based data solutions. They are a self-motivated, independent problem-solver who is eager to learn new skills and adapt to changing technologies. Collaboration, performance optimization, and a commitment to maintaining data security and compliance are critical for success in this role.
Day-to-Day
Day-to-Day
- Serve as the subject matter expert in technologies used for our data strategy in the cloud.
- Design, develop, optimize, and maintain data architecture and pipelines that adhere to ETL principles and business goals.
- Solve complex data problems to deliver insights that helps the organization's business to achieve their goals.
- Create data products for analytics and data scientist team members to improve their productivity.
- Advise, consult, mentor and coach other data and analytic professionals on data standards and practices.
- Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions.
- Lead the evaluation, implementation and deployment of emerging tools and process for analytic data engineering to improve the organization's productivity as a team.
- Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives.
- Utilize programming & scripting languages like Python, Spark (PySpark) and Open-Source RDBMS and NoSQL databases and Cloud based data warehousing services such as and Snowflake
- Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community
- Bachelor's degree in computer science engineering or a related discipline, or equivalent work experience required.
- 7-10 years of experience in data management services such as the following required.
- Data Handling: Managing, storing, and organizing large volumes of data.
- Data Integration: Combining data from different sources to create a unified view.
- Data Quality: Ensuring the accuracy, consistency, and reliability of data.
- Data Security: Protecting data from unauthorized access and breaches.
- Data Analysis: Analyzing data to extract meaningful insights and support decision-making.
- Database Management: Working with various database systems to store and retrieve data efficiently.
- Big Data Technologies: Utilizing tools and technologies like Hadoop, Spark, and others to process and analyze large datasets.
- 5 years of advanced Python programming experience is required.
- 5 years of advanced SQL querying skill is required.
- 2 years of Big Data technologies like HDFS , Spark (PySpark) and various data store technologies from on-prem to cloud required.
- 2 years of Azure Cloud experience required.
- 2 years’ experience with enterprise BI Reporting Tools such as PowerBI, Tableau or Qlikview (PowerBI preferred).
- 1-2 years of Snowflake Cloud Database required.
- Experience in creating data pipelines to move and transform data on a large scale in an enterprise organization.
- Strong experience in SDLC, DevOps processes – CI/CD tools, Git, etc.
- Strong analytical skills and attention to detail
- Improve data quality and performance.
