Lead Data Engineer – Databricks Implementation

Our Data and Artificial Intelligence technology team builds and supports enterprise‑wide data platforms that enable smarter decision‑making across Macquarie. Working closely with our Commodities and Global Markets business, the team delivers modern data solutions using Macquarie’s cloud‑based data technology stack. This is a 12‑month contract.

What role will you play?

In this role, you will help design and deliver new data solutions in a greenfield environment. You will play a lead role in driving our Databricks implementation, building and enhancing data pipelines that support real‑time and near real‑time business decisions. You will take end‑to‑end ownership of key data engineering components, make architectural decisions, and work closely with senior technical and business stakeholders to deliver scalable, high‑quality data solutions. You will also help improve the reliability, performance, and usability of data products while collaborating across teams and functions.

What You Offer

• Demonstrated experience building, implementing, and supporting data solutions using Databricks
• Strong experience working with modern data technologies, including Python‑based data processing, structured query languages, real‑time data streaming tools, data transformation frameworks, and cloud data warehouses (including PySpark, Spark SQL, Kafka, DBT and Redshift)
• Proven ability to lead hands‑on data engineering delivery, including making architectural and technical design decisions
• Strong stakeholder management skills, with the ability to communicate effectively with senior technical and non‑technical stakeholders
• A strong understanding of data and analytics concepts, such as data lakes, data pipelines, data visualisation, and reporting, with a focus on delivering clear business value
• An appreciation of good data engineering practices, including data architecture, data modelling, and the management of data definitions and quality
• Strong analytical and problem‑solving skills, with the confidence to investigate complex issues, take ownership, and see them through to resolution while working effectively with others

We encourage people from all backgrounds to apply and welcome all identities, including race, ethnicity, cultural identity, nationality, gender (including gender identity or expression), age, sexual orientation, marital or partnership status, parental, caregiving or family status, family and domestic violence status, neurodiversity, religion or belief, disability, or socio‑economic background.