We are looking for an experienced Senior Data Engineer with strong expertise in PySpark and Python to join our Data Engineering team supporting enterprise-scale Data & Analytics initiatives. The ideal candidate will have hands-on experience in building scalable ETL pipelines, data marts, and production-grade data engineering solutions across structured, semi-structured, and unstructured datasets.
The role requires strong technical capabilities in Big Data technologies, data warehousing, data analysis, software engineering best practices, and end-to-end SDLC ownership. Candidates with banking or financial services domain experience will be highly preferred.
Requirements
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines and data marts using PySpark and Python.
- Build robust, maintainable, and production-ready data engineering solutions.
- Perform end-to-end SDLC activities including development, UAT support, bug fixes, production deployments, and post-production support.
- Work with large-scale structured, semi-structured, and unstructured datasets.
- Perform data analysis, cleansing, transformation, and feature engineering activities.
- Debug and optimize PySpark code and complex SQL queries for performance and scalability.
- Develop and maintain production-grade data pipelines using modern data engineering best practices.
- Collaborate with cross-functional teams to resolve dependencies and ensure timely project delivery.
- Participate in CI/CD implementation, testing, validation, and deployment activities.
- Ensure data quality, integrity, and consistency across enterprise data platforms.
- Work closely with technical and business stakeholders to understand data requirements and deliver scalable solutions.
- Contribute to technical documentation, engineering standards, and process improvements.
Required Technical Skills
Programming & Data Engineering
- Python (Expert level)
- PySpark (Expert level)
- ETL Pipeline Development
- Data Mart Development
- Data Warehousing Concepts
- End-to-End SDLC Experience
Big Data Technologies
- Apache Spark (PySpark)
- Hadoop
- MapReduce
- Hive
- Pandas
Database Technologies
- SQL
- NoSQL Databases
- Oracle SQL
- Oracle Query Optimization & Data Analysis
Data Engineering & Analytics
- Data Analysis
- Data Cleansing
- Data Linking
- Data Transformation
- Feature Engineering
- Imputation Techniques
- Data Validation
Workflow & Orchestration Tools
- Apache Airflow
- Oozie
- Jenkins Pipelines
Software Engineering & DevOps
- Git Version Control
- CI/CD Pipelines
- Testing & Validation of Data Pipelines
- Production Deployment & Support
- Software Engineering Best Practices
Development Tools
- Jupyter Notebook
- Git
Required Experience
- 5+ years of commercial experience in Data Engineering or related data-driven roles.
- Strong hands-on experience in building ETL pipelines and Data Marts.
- Proven experience in developing production-grade PySpark and Python solutions.
- Strong understanding of software engineering concepts and best practices.
- Experience working with large-scale data processing frameworks.
- Hands-on experience with production support, UAT activities, and deployment processes.
- Strong analytical and debugging capabilities for PySpark and SQL-based data solutions.
- Experience working within Agile delivery environments is preferred.
Preferred Domain Experience
- Banking & Financial Services (Highly Preferred)
- Digital Products
- Data & Analytics Platforms
Soft Skills & Competencies
- Strong analytical and problem-solving skills.
- Excellent communication and interpersonal skills.
- Ability to communicate effectively with both technical and non-technical stakeholders.
- Strong ownership mindset and accountability for deliverables.
- Ability to work under pressure and effectively prioritize tasks.
- Strong collaboration skills with cross-functional teams.
- Ability to lead technical initiatives and drive delivery outcomes.
- Excellent verbal and written communication skills in English.
Nice to Have
- Banking domain experience.
- Experience working with enterprise-scale Data & Analytics platforms.
- Exposure to Agile methodologies and modern data engineering practices.
- Knowledge of production-grade data pipeline monitoring and optimization.
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
