What are the responsibilities and job description for the Data Engineer - Investment Data; Asset Management; Fixed Income; Equities position at Blue Bridge People?
****MUST BE ABLE TO WORK ON SITE 1-3 DAYS A WEEK IN THIS CLIENTS BOSTON OFFICE****
***MUST HAVE PREVIOUS EXPERIENCE IN INVESTMENT MANAGEMENT***
**MUST HAVE HANDS ON EXPERIENCE USING AZURE DATABRICKS & PYTHON**
Position Overview
A leading investment management firm has undertaken a strategic, firm-wide initiative to develop a next-generation research and investment platform. This initiative integrates traditional RDBMS with Big Data technologies to provide a collaborative environment that supports investment teams in delivering strong performance for clients.
We are seeking an experienced Data Engineer to join the Research Data Engineering team. This role will be responsible for maintaining the existing on-premises platform while contributing to the development of a cloud-based future platform with cutting-edge capabilities.
Required Skills and Experience
- 5 years of experience in data warehouse engineering (SQL Server, T-SQL).
- 5 years of hands-on experience with Azure Databricks
- Strong knowledge of data warehousing concepts, including persistent staging, Slowly Changing Dimensions (SCD), dimensional modeling, and time-series analysis.
- Proficiency in programming languages and tools such as Scala, Python, C#, .NET, unit test frameworks, and CI/CD pipelines.
- Proven experience implementing efficient ETL and ELT processes for large datasets, preferably with financial markets and trading data.
- Hands-on experience with workflow orchestration tools, particularly Apache Airflow, including designing, implementing, and maintaining DAGs to automate complex data pipelines.
- Expertise in developing and optimizing containerized applications using Docker, with knowledge of designing and deploying Kubernetes resources (e.g., deployments, services) to support scalable and efficient application workflows.
- Demonstrated ability to build Big Data solutions using Python and Spark or equivalent scale-out technologies.
- Production experience working with public cloud platforms, particularly Azure (preferred) or AWS.
- Familiarity with at least one NoSQL database (e.g., MongoDB, Cassandra, HBase, CouchDB, BigTable, DynamoDB, CosmosDB).
- Strong attention to detail with the ability to prioritize tasks effectively and manage multiple deadlines in a fast-paced environment.
