What are the responsibilities and job description for the Senior Data Engineer position at BrickRed Systems?
We are looking for a Senior Data Engineer with expertise in Big Data, Spark, and Cloud technologies to join our team. If you have a strong background in designing, developing, and optimizing complex data pipelines , this role is for you!
Key Responsibilities :
- Design, develop, and support high-performance data pipelines for large-scale data processing.
- Work with Apache Spark (batch & streaming) to transform and analyze massive datasets.
- Utilize advanced SQL for data analysis and interaction.
- Build and optimize cloud-scale data lake solutions in Databricks, AWS EMR, Snowflake , and other Big Data environments.
- Implement Python-based scripting for automation and data processing.
- Ensure performance tuning and optimization of large datasets .
- Leverage GitHub for source control and adhere to best practices in development.
- Work with workflow scheduling tools like Airflow or Databricks Workflow .
- Collaborate with cross-functional teams, influencing stakeholders through effective communication.
Must-Have Skills :
Nice-to-Have Skills :
About BrickRed Systems :
BrickRed Systems is a global leader in next-generation technology, consulting, and business process service companies. We enable clients to navigate their digital transformation. BrickRed Systems delivers a range of consulting services to our clients across multiple industries around the world. Our practices employ highly skilled and experienced individuals with a client-centric passion for innovation and delivery excellence.
With ISO 27001 and ISO 9001 certification and over a decade of experience in managing the systems and workings of global enterprises, we harness the power of cognitive computing hyper-automation, robotics, cloud, analytics, and emerging technologies to help our clients adapt to the digital world and make them successful. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem.
