What are the responsibilities and job description for the Software Engineer position at National Engineering?
Design & optimize scalable and efficient data processing pipelines and architectures within Azure Data Lake and Databricks.
Manage complex ETL (Extract, Transform, Load) processes to seamlessly integrate data from diverse sources (e.g., databases, APIs, streaming platforms) into Azure Data Lake, ensuring data quality and consistency.
Develop interactive dashboards and reports using SAP Business Objects and Power BI.
Leverage Azure Data Factory for data orchestration, workflow automation, and scheduling, ensuring reliable and timely data delivery.
Implement and maintain Azure Security & Governance policies, including access control, data encryption, & compliance frameworks.
Optimize data storage & retrieval mechanisms within Azure, including performance tuning of Databricks clusters and Azure SQL databases.
Bachelor’s degree in computer science.
Must be a U.S. Citizen.
4 years in data engineering, data warehousing, and cloud-based data platforms.
Expertise in Azure Data Lake, Azure Data Factory, Azure Security & Governance, Databricks, and SAP Business Objects.
Proficiency in SQL
Power BI dashboards & reports.
Azure services such as Azure Synapse Analytics, Azure SQL Database, and Azure Blob Storage.
Data modeling concepts, ETL processes, and big data frameworks (e.g., Spark).
Optimizing and managing large-scale datasets in cloud environments.
Developing and maintaining ETL packages using SSIS and reports using SSRS.
Familiarity with machine learning models & data science concepts.
Understanding of DevOps practices and CI / CD pipelines for data applications.
Data governance tools & frameworks.
Experience with other cloud platforms (e.g., AWS, GCP).
