Location: Kai Tak Office
Employment period: 1-year contract
(For internal application, only short-term employment contract’s terms and condition will be offered for this position regardless of job applicant’s current job category)
You will work closely with Data Architects, Platform Engineers, AI Engineers, and Business Stakeholders to design, build, and operate scalable, secure, and intelligent data platforms on Azure. You will build pipelines to gather data from across the enterprise and integrate into databases and other platforms from where it can be used by other end-users to generate meaningful insights into our business.
Reporting to the Senior Manager – Business Delivery, Data Services, the appointee will be responsible for the following duties:
Data Engineering & Platform Development
Design, build, and maintain end‑to‑end data pipelines (batch, streaming, and CDC) using Azure‑native and Databricks technologies
Develop scalable Lakehouse architectures leveraging Databricks, Delta Lake, and medallion design patterns (Bronze/Silver/Gold)
Integrate data from enterprise systems, SaaS platforms, IoT sources, APIs, and external providers
Optimize data pipelines for performance, reliability, cost, and security
Implement data quality checks, reconciliation, and observability using automated frameworks
Build solutions using Azure Data Factory, Azure Databricks, Azure Synapse, Azure SQL, Azure Data Lake Storage Gen2
Implement CI/CD pipelines for data solutions using Azure DevOps / GitHub, Infrastructure as Code (Terraform)
Apply DevOps and DataOps best practices for release management, deployment, and monitoring
Partner with Platform and Security teams to align with cloud governance, IAM, and enterprise security standards
Leverage AI coding agents and copilots (e.g., for Spark, SQL, Python) to:
Accelerate pipeline development and debugging
Auto‑generate transformation logic and documentation
Improve test coverage and code quality
Metadata‑driven pipelines
Requirements
Academic Qualification
Bachelor or Masters degree in a related field (e.g., computer science, information technology, etc.)
Professional Experience
At least 8-year experience in data engineering or analytics engineering
Experience with Azure Cloud Services, Databricks & Spark (PySpark / Python), SQL scripting languages, relational databases (e.g., SQL DB/DW), NoSQL platforms (e.g., HBase, MongoDB, Cassandra)
Proven experience building large‑scale, production‑grade data pipelines using on‑premises or cloud‑based data platforms
Experience with structured, semi‑structured, and unstructured data
Experience with Data lakes, Lakehouse, or modern data warehouse platforms
Experience with BI and analytics integration (Power BI, Tableau, or equivalent)
Experience in coding in data management, data warehousing or unstructured data environments
Less experience with good attitude will be considered as Senior Data Engineer
Competencies Technical (Functional)
Programming: Python, PySpark, SQL, Python
Data Platforms: Databricks Lakehouse, Delta Lake
Cloud: Azure (mandatory); AWS experience is a plus
DevOps/DataOps: CI/CD, Git, Infrastructure as Code
Databases: Relational (SQL DW), NoSQL (Cosmos DB, MongoDB, Cassandra)
Streaming (nice to have): Kafka, Event Hubs, Spark Structured Streaming
#J-18808-Ljbffr
or continue with email
Employment period: 1-year contract
(For internal application, only short-term employment contract’s terms and condition will be offered for this position regardless of job applicant’s current job category)
You will work closely with Data Architects, Platform Engineers, AI Engineers, and Business Stakeholders to design, build, and operate scalable, secure, and intelligent data platforms on Azure. You will build pipelines to gather data from across the enterprise and integrate into databases and other platforms from where it can be used by other end-users to generate meaningful insights into our business.
Reporting to the Senior Manager – Business Delivery, Data Services, the appointee will be responsible for the following duties:
Data Engineering & Platform Development
Design, build, and maintain end‑to‑end data pipelines (batch, streaming, and CDC) using Azure‑native and Databricks technologies
Develop scalable Lakehouse architectures leveraging Databricks, Delta Lake, and medallion design patterns (Bronze/Silver/Gold)
Integrate data from enterprise systems, SaaS platforms, IoT sources, APIs, and external providers
Optimize data pipelines for performance, reliability, cost, and security
Implement data quality checks, reconciliation, and observability using automated frameworks
Build solutions using Azure Data Factory, Azure Databricks, Azure Synapse, Azure SQL, Azure Data Lake Storage Gen2
Implement CI/CD pipelines for data solutions using Azure DevOps / GitHub, Infrastructure as Code (Terraform)
Apply DevOps and DataOps best practices for release management, deployment, and monitoring
Partner with Platform and Security teams to align with cloud governance, IAM, and enterprise security standards
Leverage AI coding agents and copilots (e.g., for Spark, SQL, Python) to:
Accelerate pipeline development and debugging
Auto‑generate transformation logic and documentation
Improve test coverage and code quality
Metadata‑driven pipelines
Requirements
Academic Qualification
Bachelor or Masters degree in a related field (e.g., computer science, information technology, etc.)
Professional Experience
At least 8-year experience in data engineering or analytics engineering
Experience with Azure Cloud Services, Databricks & Spark (PySpark / Python), SQL scripting languages, relational databases (e.g., SQL DB/DW), NoSQL platforms (e.g., HBase, MongoDB, Cassandra)
Proven experience building large‑scale, production‑grade data pipelines using on‑premises or cloud‑based data platforms
Experience with structured, semi‑structured, and unstructured data
Experience with Data lakes, Lakehouse, or modern data warehouse platforms
Experience with BI and analytics integration (Power BI, Tableau, or equivalent)
Experience in coding in data management, data warehousing or unstructured data environments
Less experience with good attitude will be considered as Senior Data Engineer
Competencies Technical (Functional)
Programming: Python, PySpark, SQL, Python
Data Platforms: Databricks Lakehouse, Delta Lake
Cloud: Azure (mandatory); AWS experience is a plus
DevOps/DataOps: CI/CD, Git, Infrastructure as Code
Databases: Relational (SQL DW), NoSQL (Cosmos DB, MongoDB, Cassandra)
Streaming (nice to have): Kafka, Event Hubs, Spark Structured Streaming
#J-18808-Ljbffr
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more