Big Data Developer

About the role

A developer role in an Agile setting, building and evolving data lake solutions for market risk use cases with large scale data processing pipelines.

What you will do

• Create scalable processing solutions for very large data volumes
• Schedule batch workflows and dependencies with tools such as Control-M and Airflow
• Handle ingestion, transformation, validation and publication of data for market risk
• Optimise the performance, reliability and scalability of Spark workloads
• Support production incidents, run root cause analysis and improve operational processes

What they look for

• Understanding of data lake architecture, data quality controls and data lineage
• Strong analytical and problem solving abilities
• Good communication and teamwork skills for collaborating across squads