Senior Data Engineer

Job responsibilities

• Create and manage a single master record for each business entity, ensuring data consistency, accuracy, and reliability.
• Implement data governance processes, including data quality management, data profiling, data remediation, and automated data lineage.
• Create and maintain multiple robust and high-performance data processing pipelines within Cloud, Private Data Centre, and Hybrid data ecosystems.
• Assemble large, complex data sets from a wide variety of data sources.
• Collaborate with Data Scientists, Machine Learning Engineers, Business Analysts, and Business users to derive actionable insights and reliable foresights into customer acquisition, operational efficiency, and other key business performance metrics.
• Develop, deploy, and maintain multiple microservices, REST APIs, and reporting services.
• Design and implement internal processes to automate manual workflows, optimize data delivery, and re-design infrastructure for greater scalability.
• Establish expertise in designing, analyzing, and troubleshooting large-scale distributed systems.
• Support and work with cross-functional teams in a dynamic environment.

Job Requirements

• Experience building and operating large-scale data lakes and data warehouses.
• Experience with Hadoop ecosystem and big data tools, including Spark and Kafka.
• Experience with Master Data Management (MDM) tools and platforms such as Informatica MDM, Talend Data Catalog, Semarchy xDM, IBM PIM & IKC, or Profisee.
• Familiarity with MDM processes such as golden record creation, survivorship,reconciliation, enrichment, and quality.
• Experience in data governance, including data quality management, data profiling, data remediation, and automated data lineage.
• Experience with stream-processing systems including Spark-Streaming.
• Experience working with Cloud services using one or more Cloud providers such as Azure, GCP, or AWS.
• Experience with Delta Lake and Databricks.
• Advanced working experience with relational SQL and NoSQL databases, including Hive, HBase, and Postgres.
• Deep understanding of SQL and the ability to optimize data queries.

If the requirement matches with your profile, kindly share your updated CV/resume to Aparna at aparna@wshexperts.com.sg