Data Solution Engineer

RESPONSIBILITY
• Responsible for the end-to-end architecture for a system and ensures the detailed design and development is aligned with the needs of data users
• Translate business requirements, including application resiliency requirements, into application architectural specifications
• Propose products and solutions based on Enterprise Architecture and standards
• Design the architecture blueprint of the organisation's business, information and ICT assets, by using multiple architectural models, syncing together a front-end data visualisation platform, with a back-end cloud data lakehouse infastructure.
• Use, govern and enhance the architecture blueprint to ensure that the business and ICT are in alignment.
• Review source codes and solutions as part of the software development lifecycle process
EXPERIENCE AND SKILLS NEEDED
Educational Requirements:
• Bachelor's degree in Computer Science, Software Engineering, Information Technology or related technical field; Master's degree preferred
• Professional certifications in cloud platforms (AWS/Azure) and data engineering certifications (e.g., Databricks, Snowflake) are advantageous
Technical Experience:
• Minimum 7 years of enterprise software development experience, including 3 years leading technical teams and architecting data-intensive solutions
• Strong expertise in designing and implementing cloud-native data platforms and pipelines using modern tools like Apache Spark, Airflow, or Azure Data Factory
• Proven experience with cloud-based data platforms, including data warehouses (Redshift, BigQuery) and lakehouse solutions (Databricks, Delta Lake).
• Advanced knowledge of both traditional and cloud-native ETL/ELT processes
• Deep understanding of real-time and batch data processing architectures
• Experience working with geospatial data, including spatial databases (PostGIS, Azure Cosmos DB with spatial features) and GIS tools.
• Experience integrating geospatial APIs (e.g., Azure Maps, ESRI ArcGIS) into cloud-based data architectures.
• Knowledge of 3D data structures and processing pipelines, including formats such as CityGML, 3D Tiles, LAS/LAZ.
Data Architecture & Governance:
• Experience implementing comprehensive Role-Based Access Control (RBAC) systems across multi-tenant data platforms
• Proven track record in designing and implementing data governance frameworks including data lineage, metadata management, and data quality controls
• Experience with data masking, encryption, and other data protection mechanisms
• Knowledge of implementing column-level security and row-level security in database systems
• Understanding of regulatory compliance requirements (PDPA, GDPR) and their implementation in data architectures
• Experience implementing end-to-end encryption strategies (e.g., Azure Key Vault, TDE, Always Encrypted) for cloud data lakes and databases.
• Strong understanding of data security for SFTP, including TLS, SSH key management, and auditing.
• Ability to implement secure data sharing mechanisms while maintaining governance (e.g., Databricks Unity Catalog, Azure Purview).
System Design & Integration:
• Expertise in designing scalable data integration patterns and implementing data mesh architectures
• Experience with streaming data platforms (Kafka, Event Hubs) and real-time analytics
• Strong background in API security and authentication mechanisms for data access
• Proven ability to design and implement data cataloguing and discovery solutions
• Experience with modern data observability tools and practices
Cloud & Database Expertise:
• Deep expertise in cloud-native database solutions including:
o Relational: Azure SQL Managed Instance" (SQL MI), Azure SQL, Cloud SQL
o NoSQL: MongoDB, Cosmos DB, DynamoDB
o Analytics: Redshift, BigQuery, Synapse
• Experience optimising database performance and implementing auto-scaling solutions
• Knowledge of database backup, disaster recovery, and high availability configurations
• Understanding of data migration strategies and tools
• Experience integrating Azure Data Factory (ADF) with Databricks and ADLS for end-to-end ETL workflows.
• Experience optimizing Azure Synapse Analytics for large-scale data processing.
Project Leadership:
• Successfully delivered at least 3 enterprise-scale data platform projects
• Experience leading data architecture modernisation initiatives
• Proven track record in stakeholder management across data owners, engineers, and business users
• Ability to develop and maintain data architecture roadmaps
Technical Leadership:
• Experience designing microservices architectures with focus on data security and governance
• Strong understanding of DevSecOps practices for data platforms
• Expertise in implementing CI/CD pipelines for database changes and data pipeline deployments
• Knowledge of infrastructure as code (Terraform, CloudFormation) for data platform components
Soft Skills:
• Excellent communication skills in explaining complex data architectures to various stakeholders
• Strong analytical mindset with ability to balance security, governance, and usability requirements
• Experience in building and leading data engineering teams
• Proven ability to collaborate with data scientists, analysts, and business intelligence teams
Additional Desirable Skills:
• Experience with data quality monitoring and validation frameworks
• Knowledge of ML Ops and AI model deployment in production
• Understanding of data mesh principles and domain-driven design
• Experience with data contract management and API versioning
• Familiarity with modern data stack tools and emerging technologies in the data space