Senior Databricks Data Engineer

Hileya - Management Consulting (Sandton GT, South Africa) Follow 9 hours ago

Work Location:

Cape Town CBD – onsite for 3 days per week

Job Description / Responsibilities:

Looking for strong Data Engineers to join our Data, Information and Analytics team as client-facing, permanently employed consultants. Someone who is passionate about data and implementing technology solutions and leading project teams to achieve business outcomes. There will be exposure to a wide range of industries, so the ability to quickly understand new business environments is essential.

Responsibilities include:

Engage with clients to conceptualize data solutions
Modernize existing data environments
Assist our sales team with Pre‑Sales and POC’s
Create and maintain optimal data pipeline architecture; assemble large, complex data sets that meet functional and non‑functional business requirements
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re‑designing infrastructure for greater scalability
Build the infrastructure required for optimal extraction, transformation and loading of data from a wide variety of data sources using Microsoft, Azure, AWS, GCP, technologies etc.
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
Work with stakeholders including the Executive, Product, Data and Design teams to assist with data‑related technical issues and support their data infrastructure needs
Work with data and analytics experts to strive for greater functionality in our data systems
Assist support desk where applicable

Background and experience required:

Minimum 3 years working experience in data engineering
Advanced working SQL knowledge and experience with relational databases, query authoring (SQL) as well as familiarity with a variety of databases
Experience building and optimizing data pipelines, architectures, and data structures
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
Strong analytic skills related to working with unstructured datasets
Build processes supporting data transformation, data structures, metadata, dependency and workload management
A successful history of manipulating, processing and extracting value from large, disconnected datasets
Strong project management and organizational skills
Experience supporting and working with cross‑functional teams in a dynamic environment

Microsoft Qualifications