What are the responsibilities and job description for the BigData Engineer position at Diverse Lynx?
Job Title : Bigdata Engineer
Location : Sunrise, FL (Onsite)
Type : Contract
Job Description / Responsibilities :
We are looking for a Senior Big Data engineer to drive Data and AI / ML development for our enterprise data platform.
This role requires a deep understanding of enterprise data needs to create seamless data experiences for business users.
Key Responsibilities :
Hands on development experience of Big Data initiatives.
Big Data Engineer with technical expertise with Spark, Python, Hive and Shell Scripting
Focused on Designing, developing and maintaining scalable data processing systems
Hands on experience of handling large data sets using Apache Spark and other big data technologies
Develop Data transformation and aggregation logic using Python, and Spark SQL for efficient querying and processing of data.
Work with Hive for writing complex HiveQL queries for data storage, transformation and analysis.
Implement Shell Scripting to automate data processing, System Monitoring and data integration tasks.
Collaborate with Data Scientist, Analysts and other stakeholders to ensure data availability and integrity.
Monitor and troubleshoot production data pipelines, ensuring timely data delivery and minimal downtime.
Drive A / B testing and user research to optimize data platform interfaces.
Qualifications :
Bachelors or master's degree in Computer Science, Engineering or related field.
Experience with Cloud-based Platforms (GCP etc.)
Experience in designing and launching enterprise-grade data platforms.
Familiarity with Kubernetes or containerized environments.
Strong analytical, problem-solving, and communication skills.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Keep a pulse on the job market with advanced job matching technology.
If your compensation planning software is too rigid to deploy winning incentive strategies, it’s time to find an adaptable solution.
Compensation Planning
Enhance your organization's compensation strategy with salary data sets that HR and team managers can use to pay your staff right.
Surveys & Data Sets
What is the career path for a BigData Engineer?
Sign up to receive alerts about other jobs on the BigData Engineer career path by checking the boxes next to the positions that interest you.
