What are the responsibilities and job description for the Big Data Engineer- Jr position at ConsultNet Technology Services and Solutions?
Consultnet is looking for a Big Data resource to join our team in Rockville, MD. This is a hybrid position at a minimum two days a week.
Responsibilities:
• Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala).
• Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable.
• Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Optimize and enhance existing data pipelines for performance, scalability, and reliability.
• Develop automated testing frameworks and implement continuous testing for data quality assurance.
• Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines.
• Work with data scientists and analysts to support data-driven decision-making across the organization.
• Ability to write and maintain automated unit, integration, and end-to-end tests
• Monitor and troubleshoot data pipelines in production environments to identify and resolve issues.
Preferred:
• Experience with big data tools such as Hadoop, Spark, and Presto/Trino
• Experience with production data pipelines/ETL systems
• Comfortable in AWS console and using AWS cli tools
• AWS certifications
Welcome to ConsultNet and the family of companies, Tekne, SaltClick, TechBridge, and OmniMedia. As a premier national provider of technology talent and solutions, our expertise spans across project services, contract-to-hire, direct placement, and managed services both onshore and nearshore.
Celebrating more than 25 years of partnership with a diverse client base, we've crafted rewarding opportunities for our consultants, fostering high-performing teams that deliver impactful results.
Over the last few years thousands of consultants have found their calling with us in roles that have made a meaningful impact on their lives, enhanced their career, challenged them, and propelled them towards achieving their personal and professional goals. At the ConsultNet family of companies, we believe effective communication is crucial in aligning the right job with your unique skills and professional aspirations. To us, it's all about the personal approach we take and the values we uphold.
Our comprehensive service offerings cover a wide range of technology positions across key markets nationwide. Client more at www.consultnet.com .
We champion equality and inclusivity, proudly supporting an Equal Opportunity Employer policy. We welcome applicants regardless of Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other status protected by law.
