What are the responsibilities and job description for the Data Engineer, Kafka position at Shamrock Trading Corporation?
Responsibilities
Shamrock Trading Corporation is looking for a Data Engineer who wants to utilize their expertise in data warehousing, data pipeline creation / support and analytical reporting skills by joining our Data Services team. This role is responsible for gathering and analyzing data from several internal and external sources, designing a cloud-focused data platform for analytics and business intelligence, reliably providing data to our analysts. This role requires significant understanding of data mining and analytical techniques. An ideal candidate will have business acumen, the ability to work effectively with cross-functional teams and strong technical capabilities, including experience in Apache Kafka. Responsibilies include but are not limited to :
- Work with Data architects to understand current data models, to build pipelines for data ingestion and transformation.
- Design, build, and maintain a framework for pipeline observation and monitoring, focusing on reliability and performance of jobs.
- Surface data integration errors to the proper teams focusing on :
- Ensuring timely processing of new data
- Performance of data pipelines
- Integrity and quality of source data
- Hands-on experience building data-lake style infrastructures using streaming data set technologies (particularly with Apache Kafka)
Qualifications
Preferred Qualifications
Benefits Package At Shamrock we hire bright, ambitious people and give them the tools they need to be successful. By investing in training and development, we hope to become a long-term career for employees, where there are always opportunities for advancement. Shamrock also offers a premier set of benefits for employees and their families :
