Data Engineering (DE) Lead
Location: Rochester, Minnesota(Open for Remote).
Role Family
Data Engineering
Experience
8+ years
Platform Focus
GCP DataFlow
Role Description
Own data-engineering delivery on GCP/DataFlow, including ingestion, transformation, standardisation, reconciliation, publication and operational engineering standards.
Key Responsibilities
Define reusable DataFlow/GCP ingestion and transformation patterns.
- Implement source ingestion from Epic, ELMS, Oracle, terminology and other approved sources.
- Build raw/source-preserving, standardised/cleansed and curated/mastered processing layers.
- Implement standardisation, validation, transformation and reference-data enrichment.
- Build reconciliation controls between source, MDM/RDM and downstream publication.
- Build publication pipelines to GCP data products, APIs, events and tables/files.
- Implement replay, restart, idempotency, audit and monitoring; optimise performance and cost.
- Establish CI/CD, code quality and production support procedures.
Primary Deliverables
- Data engineering architecture / design
- Ingestion and transformation pipelines
- Reconciliation framework
- Publication pipelines
- Operational monitoring and runbooks
- CI/CD and engineering standards
Required Experience & Skills
- 8+ years in data engineering, hands-on with GCP and DataFlow (or equivalent).
- Large-scale batch, incremental and event-driven pipeline experience.
- Experience feeding and reconciling MDM/RDM data.
- Strong on operationalisation: monitoring, replay, CI/CD.
Good to Have
- BigQuery, Pub/Sub and Dataflow depth.
- Healthcare data engineering.
- Integration with Informatica MDM loads.
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more
More from American IT Systems