Position Description:
We are seeking an experienced ETL/SQL/Python Testing Engineer to design, develop, and execute test cases for data pipelines, ETL processes, and database systems. The ideal candidate will have strong expertise in SQL querying, ETL validation, Python scripting, and test automation to ensure data accuracy, completeness, and optimal performance.
Your future duties and responsibilities:
• Develop and execute ETL test cases to validate data extraction, transformation, and loading processes.
• Perform ETL pipeline testing in Azure Databricks (ADB) environments.
• Write complex SQL queries to verify data integrity, consistency, and correctness across source and target systems.
• Automate ETL testing workflows using Python, PyTest, or similar frameworks.
• Conduct data reconciliation, schema validation, and comprehensive data quality checks.
• Identify and report data anomalies, performance issues, and defects.
• Collaborate with Data Engineers, Analysts, and Business Teams to understand data requirements.
• Design and maintain test datasets for validation activities.
• Implement CI/CD pipelines for automated ETL testing (e.g., Jenkins, GitLab CI).
• Document test results, defects, and detailed validation reports.
#LI-AD11
Required qualifications to be successful in this role:
• ETL Tools Expertise: Hands-on experience with Informatica, Talend, SSIS, Databricks or similar ETL platforms.
• SQL Mastery: Advanced SQL skills including complex joins, aggregations, subqueries, stored procedures, and performance tuning.
• Python Automation: Strong proficiency in Python with libraries/frameworks such as Pandas, PySpark, PyTest for automation and data validation.
• Database Knowledge: Experience with both RDBMS (Oracle, SQL Server, PostgreSQL) and NoSQL databases (MongoDB, Cassandra).
• CI/CD & DevOps: Familiarity with Jenkins, GitLab CI/CD, Azure DevOps for automated testing pipelines.
• Version Control: Proficient in Git, GitHub/GitLab for collaborative development and code management.
• Big Data Ecosystem (Good to Have): Exposure to Hadoop, Hive, Spark, Kafka for large-scale data testing.
• Testing Tools (Good to Have): Knowledge of Airflow, Great Expectations, Selenium or similar frameworks for workflow orchestration and validation.
• Agile Practices: Experience working in agile environments with sprint-based delivery and in-sprint automation.
• Communication & Leadership: Excellent communication skills to interact with business analysts, line-of-business stakeholders, and ability to lead testing initiatives and drive innovation.
Must-Have Skills:
• ETL Testing: Strong hands-on experience with tools such as Informatica, Talend, SSIS, Databricks.
• SQL Expertise: Advanced SQL skills (joins, aggregations, subqueries, stored procedures).
• Python Automation: Proficiency in Python with libraries/frameworks like Pandas, PySpark, PyTest.
• Databases: Experience with RDBMS (Oracle, SQL Server, PostgreSQL) and NoSQL (MongoDB, Cassandra).
• CI/CD: Practical experience with Jenkins, Azure DevOps, GitLab CI/CD.
• Version Control: Strong knowledge of Git, GitHub/GitLab.
Good-to-Have Skills:
• Big Data Ecosystem: Exposure to Hadoop, Hive, Spark, Kafka.
• Testing Tools: Familiarity with Selenium, Airflow, Great Expectations or similar frameworks.
• Cloud & Data Engineering: Understanding of cloud-based data platforms and orchestration tools.
• Agile Practices: Experience working in agile testing environments with sprint-based delivery.
• Communication & Collaboration: Ability to interact effectively with business analysts and cross-functional teams.
Skills:
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
