Databricks with Python and Cloud

Your Role

7+ years of experience in data engineering, data platforms & analytics, 10+ years of consulting experience
Minimum 6-8+ projects delivered with hands-on experience in development on Databricks
Strong hands-on experience in Python programming. Expertise in:

Design, develop, and optimize scalable data pipelines on Databricks using Apache Spark (PySpark/Scala), ensuring high performance and reliability for batch and real-time processing.
Implement data engineering best practices including Delta Lake, data modeling, partitioning, and performance tuning; manage data workflows using Databricks Workflows or orchestration tools.
Collaborate with data scientists and analysts to build and deploy machine learning models and analytics solutions, leveraging Databricks notebooks, MLflow, and Unity Catalog for governance.

Your Profile

Ensure data quality, security, and compliance by applying monitoring, logging, access controls, and CI/CD pipelines (Azure DevOps/Git), supporting end-to-end data lifecycle management
Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one
Deep experience with distributed computing with Spark with knowledge of Spark runtime internals
Familiarity with CI/CD for production deployments
Working knowledge of MLOps
Current knowledge across the breadth of Databricks product and platform features
Familiarity with optimizations for performance and scalability
Databricks certification is an added advantage

What will you love working at Capgemini

You will have the opportunity to learn on one of the industry's largest digital learning platforms, with access to 250,000+ courses and numerous certifications.
We’re committed to ensure that people of all backgrounds feel encouraged and have a sense of belonging at Capgemini. You are valued for who you are, and you can bring your original self to work.
At Capgemini, you can work on cutting-edge projects in tech and engineering with industry leaders or create solutions to overcome societal and environmental challenges.
Capgemini office campuses in India are green and run on 100% renewable electricity. We have installed Solar plants across India locations and ‘Battery Energy Storage Solution’ (BESS) in the Noida and Mumbai campuses. You will have chance to make a difference everyday.