Job Summary (List Format):
- Lead the automation and orchestration of complex data workflows using Python.
- Design, develop, and maintain robust, fault-tolerant, and auditable data pipelines across on-prem Oracle 19c systems.
- Replace legacy Perl/PLSQL-based scheduling logic with modern Python-based orchestration using Apache Airflow.
- Build, deploy, and manage Airflow DAGs for complex, interdependent ETL workflows.
- Migrate job logic from Perl, RunMyJob, and PL/SQL scheduling into modular Airflow DAGs.
- Create custom Airflow operators and sensors for integration with Oracle databases, REST APIs, SFTP/FTP file drops, and external triggers.
- Implement error handling, alerting, and retry mechanisms to ensure data quality, traceability, and recoverability.
- Collaborate with DBAs and application teams to understand dependencies, data lineage, and critical workflow paths.
- Develop and maintain job execution logs, audit trails, and SLA monitoring dashboards.
- Participate in code reviews, documentation efforts, and onboarding of new jobs into the orchestration platform.
- Ensure strong debugging capabilities across logs, databases, and filesystems for troubleshooting failed or partial job runs.
- Integrate automation with REST APIs and parameter-driven workflows.
- Preferably, contribute to modernization of legacy workflows and support code-controlled job rollouts using Git/Bitbucket and Jenkins.
- Optionally, bring experience in financial data models and an understanding of data governance, audit, and operational risk in financial systems.
or continue with email
- Lead the automation and orchestration of complex data workflows using Python.
- Design, develop, and maintain robust, fault-tolerant, and auditable data pipelines across on-prem Oracle 19c systems.
- Replace legacy Perl/PLSQL-based scheduling logic with modern Python-based orchestration using Apache Airflow.
- Build, deploy, and manage Airflow DAGs for complex, interdependent ETL workflows.
- Migrate job logic from Perl, RunMyJob, and PL/SQL scheduling into modular Airflow DAGs.
- Create custom Airflow operators and sensors for integration with Oracle databases, REST APIs, SFTP/FTP file drops, and external triggers.
- Implement error handling, alerting, and retry mechanisms to ensure data quality, traceability, and recoverability.
- Collaborate with DBAs and application teams to understand dependencies, data lineage, and critical workflow paths.
- Develop and maintain job execution logs, audit trails, and SLA monitoring dashboards.
- Participate in code reviews, documentation efforts, and onboarding of new jobs into the orchestration platform.
- Ensure strong debugging capabilities across logs, databases, and filesystems for troubleshooting failed or partial job runs.
- Integrate automation with REST APIs and parameter-driven workflows.
- Preferably, contribute to modernization of legacy workflows and support code-controlled job rollouts using Git/Bitbucket and Jenkins.
- Optionally, bring experience in financial data models and an understanding of data governance, audit, and operational risk in financial systems.
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more