What are the responsibilities and job description for the Data Architect position at Finding Candidate?
We are hiring for our client - Mega Cloud Lab.
Responsibilities
Has worked in the past as an Architect. This is Onsite Role from Day 1.
Professional experience in Data Engineering using Google Big Query (GBQ) and Google Cloud Platform (GCP) data sets and building data pipelines.
Hands on and deep experience working with Google Data Products (e.g. BigQuery, Dataflow, Dataproc, Dataprep, Cloud Composer, Airflow, DAG etc.).
Python Programming Expert PySpark, Pandas
Experience in Airflow (Create DAG, Configure the variables in Airflow, Scheduling)
Big Data technologies and solutions (Spark, Hadoop, Hive, MapReduce) and multiple scripting and languages (YAML, Python).
Experience in DBT to create the lineage in GCP. – Optional
Worked in Dev-Sec-Ops (CICD) environment.
Design and develop the ETL ELT framework using BigQuery, Expertise in Big Query concepts like Nested Queries, Clustering, Partitioning, etc.
Experience in Data Integration, Data Transformation, Data Quality and Data Lineage tools.
Should be able to automate the data load from Big Query using APIs or scripting language.
E2E Data Engineering and Lifecycle (including non-functional requirements and operations) management.
E2E Solution Design skills – Prototyping, Usability testing and data visualization literacy.
Experience with SQL and NoSQL modern data stores.
Skills: yaml,dbt,cloud composer,airflow,google bigquery,apis,elt framework,nosql,gcp,google big query,partitioning,google,scripting language,data quality,apache spark,spark,data pipelines,pyspark,nested queries,data lineage,prototyping,mapreduce,solution design,dataprep,etl,clustering,dataflow,data,pandas,hive,google cloud platform,bigquery,dataproc,data integration,elt,data engineering,data visualization,big data technologies,python,hadoop,data transformation,usability testing,sql
Responsibilities
Has worked in the past as an Architect. This is Onsite Role from Day 1.
Professional experience in Data Engineering using Google Big Query (GBQ) and Google Cloud Platform (GCP) data sets and building data pipelines.
Hands on and deep experience working with Google Data Products (e.g. BigQuery, Dataflow, Dataproc, Dataprep, Cloud Composer, Airflow, DAG etc.).
Python Programming Expert PySpark, Pandas
Experience in Airflow (Create DAG, Configure the variables in Airflow, Scheduling)
Big Data technologies and solutions (Spark, Hadoop, Hive, MapReduce) and multiple scripting and languages (YAML, Python).
Experience in DBT to create the lineage in GCP. – Optional
Worked in Dev-Sec-Ops (CICD) environment.
Design and develop the ETL ELT framework using BigQuery, Expertise in Big Query concepts like Nested Queries, Clustering, Partitioning, etc.
Experience in Data Integration, Data Transformation, Data Quality and Data Lineage tools.
Should be able to automate the data load from Big Query using APIs or scripting language.
E2E Data Engineering and Lifecycle (including non-functional requirements and operations) management.
E2E Solution Design skills – Prototyping, Usability testing and data visualization literacy.
Experience with SQL and NoSQL modern data stores.
Skills: yaml,dbt,cloud composer,airflow,google bigquery,apis,elt framework,nosql,gcp,google big query,partitioning,google,scripting language,data quality,apache spark,spark,data pipelines,pyspark,nested queries,data lineage,prototyping,mapreduce,solution design,dataprep,etl,clustering,dataflow,data,pandas,hive,google cloud platform,bigquery,dataproc,data integration,elt,data engineering,data visualization,big data technologies,python,hadoop,data transformation,usability testing,sql
