What are the responsibilities and job description for the Spark Developer with Iceberg position at ST Global LLC?
Job Details
3 years of experience in Data Lake Architectures, data warehousing technologies and data engineering.
Strong understanding of Apache Iceberg, including schema evolution, partition management, and data versioning
Proficient in Apache Spark, PySpark and other big data processing frameworks like Presto or Trino, Flink
Proficient in Snowflake Database and Architecture, SQL development and data modeling
Experience with cloud platforms and technologies like EMR, AWS IAM, AWS S3, including object storage services and compute services
Hands-on experience of working with Git, Jupyter notebooks, Splunk and Sandbox environments
Exceptional communication and collaboration skills, with the ability to work effectively across multiple teams and functions.