Skills : Data engineer
Location : Remote
Experience : 4+ years
Notice : Immediate only
Key Skills
Data Engineering Expertise : Bring 3+ years of experience in building data pipelines and managing a secure, modern data stack. This includes CDC streaming ingestion using tools like Debezium into a Hudi data lake that supports AI/ML workloads and a curated Redshift data warehouse.
AWS Cloud Proficiency : At least 3 years of experience working with AWS cloud infrastructure, including Kafka (MSK), Spark / AWS Glue, and infrastructure as code (IaC) using Terraform.
Strong Coding Skills : Write and review high-quality, maintainable code that enhances the reliability and scalability of our data platform. We use Python, SQL, and dbt extensively, and you should be comfortable leveraging third-party frameworks to accelerate development.
Data Lake Development : Prior experience building data lakes on S3 using Apache Hudi with Parquet, Avro, JSON, and CSV file formats.
Workflow Automation : Build and manage multi-stage workflows using serverless Lambdas and AWS Step Functions to automate and orchestrate data processing pipelines.
Data Governance Knowledge : Familiarity with data governance practices, including data quality, lineage, and privacy, as well as experience using cataloging tools to enhance discoverability and compliance.
CI/CD Best Practices : Experience developing and deploying data pipeline solutions using CI/CD best practices to ensure reliability and scalability.
Data Integration Tools : Working knowledge of tools such as Stitch and Segment CDP for integrating diverse data sources into a cohesive ecosystem.
Analytical and ML Tools Expertise : Knowledge and practical experience with Athena, Redshift, or Sagemaker Feature Store to support analytical and machine learning workflows is a definite bonus!
or continue with email
Location : Remote
Experience : 4+ years
Notice : Immediate only
Key Skills
Data Engineering Expertise : Bring 3+ years of experience in building data pipelines and managing a secure, modern data stack. This includes CDC streaming ingestion using tools like Debezium into a Hudi data lake that supports AI/ML workloads and a curated Redshift data warehouse.
AWS Cloud Proficiency : At least 3 years of experience working with AWS cloud infrastructure, including Kafka (MSK), Spark / AWS Glue, and infrastructure as code (IaC) using Terraform.
Strong Coding Skills : Write and review high-quality, maintainable code that enhances the reliability and scalability of our data platform. We use Python, SQL, and dbt extensively, and you should be comfortable leveraging third-party frameworks to accelerate development.
Data Lake Development : Prior experience building data lakes on S3 using Apache Hudi with Parquet, Avro, JSON, and CSV file formats.
Workflow Automation : Build and manage multi-stage workflows using serverless Lambdas and AWS Step Functions to automate and orchestrate data processing pipelines.
Data Governance Knowledge : Familiarity with data governance practices, including data quality, lineage, and privacy, as well as experience using cataloging tools to enhance discoverability and compliance.
CI/CD Best Practices : Experience developing and deploying data pipeline solutions using CI/CD best practices to ensure reliability and scalability.
Data Integration Tools : Working knowledge of tools such as Stitch and Segment CDP for integrating diverse data sources into a cohesive ecosystem.
Analytical and ML Tools Expertise : Knowledge and practical experience with Athena, Redshift, or Sagemaker Feature Store to support analytical and machine learning workflows is a definite bonus!
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more
More from Xpetize Technology Solutions Private Limited
