AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!
ABOUT THE ROLE
We are looking for a
Senior Data Engineer with deep Airflow and dbt expertise to support Medallion Architecture implementation, harden pipeline reliability through automated testing and failure alerting, and migrate scheduled ETL jobs into orchestrated Airflow DAGs within a modern Snowflake data stack. You will build complex DAG architectures using Python, audit dbt codebases for performance and cost optimization, maintain integration points between Paradime and Airflow, and ensure zero downtime for downstream BI tools during ongoing infrastructure changes. Eastern Time Zone overlap preferred for daily team standups.
Senior Data Engineer with deep Airflow and dbt expertise to support Medallion Architecture implementation, harden pipeline reliability through automated testing and failure alerting, and migrate scheduled ETL jobs into orchestrated Airflow DAGs within a modern Snowflake data stack. You will build complex DAG architectures using Python, audit dbt codebases for performance and cost optimization, maintain integration points between Paradime and Airflow, and ensure zero downtime for downstream BI tools during ongoing infrastructure changes. Eastern Time Zone overlap preferred for daily team standups.
WHAT YOU WILL DO
- Support implementation of bronze/silver/gold layering across the Snowflake warehouse, working with the Lead Data Engineer on architecture and modeling decisions.
- Build automated testing and failure alerting across Paradime (ETL) and Airflow (orchestration), including reworking select existing jobs so failures are clearly attributable.
- Migrate select schedule-based Paradime jobs into orchestrated Airflow DAGs.
- Work within established data governance and access controls when building or modifying pipelines touching PII data.
- Translate dbt transform schedules and dependencies into scalable, dynamic Airflow DAGs using modern Python design patterns (@task decorators, dynamic task mapping).
- Maintain and evolve integration points between Paradime and Airflow as both tools continue to be used in production.
- Audit dbt codebases and pipeline execution paths to identify bottlenecks, reducing compute runtime and cost.
- Partner with the Lead Data Engineer to adjust the modern data stack architecture to support future AI semantic layers and downstream consumption.
- Establish and maintain automated error handling, alerting, and failure recovery routines across the pipeline stack.
- Ensure no downtime or data loss for downstream BI (Sigma) and analytics tools during ongoing infrastructure changes.
- Take on additional data engineering priorities as assigned, based on evolving business needs.
MUST HAVES
-
4+ years of experience as a Data Engineer .
4+ years of experience as a Data Engineer .
-
Apache Airflow / Astronomer : Substantial hands-on production experience building, debugging, and managing complex DAG architectures using Python.
Apache Airflow / Astronomer : Substantial hands-on production experience building, debugging, and managing complex DAG architectures using Python.
-
dbt (Data Build Tool) expertise : Strong proficiency with dbt Core architecture, macros, packages, custom materializations, CI/CD patterns (state deferral), and environment configurations.
dbt (Data Build Tool) expertise : Strong proficiency with dbt Core architecture, macros, packages, custom materializations, CI/CD patterns (state deferral), and environment configurations.
-
Advanced Git & Data CI/CD : Track record building deployment pipelines and automated regression testing for data teams using GitHub Actions or similar.
Advanced Git & Data CI/CD : Track record building deployment pipelines and automated regression testing for data teams using GitHub Actions or similar.
-
Production Data Engineering : Solid general software/data engineering background writing clean, maintainable, modular Python and SQL.
Production Data Engineering : Solid general software/data engineering background writing clean, maintainable, modular Python and SQL.
- Upper-intermediate English level.
NICE TO HAVES
- Snowflake expertise: tuning compute (warehouse sizing, query profiling, clustering), RBAC/security architecture.
- Infrastructure as Code: Terraform, Docker, or CloudFormation.
- AWS S3 experience as part of a data pipeline (e.g., staging/raw ingestion layers).
- Enterprise retail, supply chain, or inventory systems data background.
PERKS AND BENEFITS
-
Growth without limits : build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
Growth without limits : build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget
-
Competitive compensation : get recognition that reflects your skills and impact, with regular performance and compensation reviews
Competitive compensation : get recognition that reflects your skills and impact, with regular performance and compensation reviews
-
Flexibility : work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
Flexibility : work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm
-
Meaningful, modern projects : build impactful products using modern technologies alongside global teams and leading brands
Meaningful, modern projects : build impactful products using modern technologies alongside global teams and leading brands
-
Collaborative culture : join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
Collaborative culture : join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized
-
Well-being & support : access local well-being programs and people-focused support tailored to your location
Well-being & support : access local well-being programs and people-focused support tailored to your location
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more
