Sr. Data Quality Engineer
Eagan, Minnesota
Eagan, Minnesota
Candidates MUST currently live in Minnesota (NOT relocate). Candidates need be able to interview onsite at of MN in Eagan and be able to work onsite 2-3 days/week.
Sr. level is needed:
SQL
Python
Key AWS data services:
AWS Glue
AWS Redshift
Can have Databricks or AWS Databrew but not required
Healthcare experience is preferred
SQL
Python
Key AWS data services:
AWS Glue
AWS Redshift
Can have Databricks or AWS Databrew but not required
Healthcare experience is preferred
Key Responsibilities
I. Data Quality Assurance Strategy, Frameworks, and Automation
Design, develop, and maintain scalable data quality assurance frameworks to validate data pipelines, transformations, and warehouse tables across the end-to-end data lifecycle.
Build and automate data validation, reconciliation, and testing processes using Python and SQL to detect defects before data reaches downstream consumers.
Define and implement reusable data quality rules and controls focused on key dimensions such as completeness, accuracy, consistency, uniqueness, validity, and timeliness.
Establish automated checks for schema changes, null handling, duplicate detection, referential integrity, threshold breaches, and business rule compliance.
Integrate data quality tests into CI/CD and deployment workflows to ensure quality gates are enforced prior to production release.
Develop dashboards, scorecards, and reporting mechanisms to measure and communicate data quality KPIs and SLA adherence to both technical and business stakeholders.
Support continuous monitoring and alerting to rapidly identify, escalate, and resolve data quality issues.
II. Data Quality Implementation Across the AWS Data Platform
Build and automate data validation, reconciliation, and testing processes using Python and SQL to detect defects before data reaches downstream consumers.
Define and implement reusable data quality rules and controls focused on key dimensions such as completeness, accuracy, consistency, uniqueness, validity, and timeliness.
Establish automated checks for schema changes, null handling, duplicate detection, referential integrity, threshold breaches, and business rule compliance.
Integrate data quality tests into CI/CD and deployment workflows to ensure quality gates are enforced prior to production release.
Develop dashboards, scorecards, and reporting mechanisms to measure and communicate data quality KPIs and SLA adherence to both technical and business stakeholders.
Support continuous monitoring and alerting to rapidly identify, escalate, and resolve data quality issues.
II. Data Quality Implementation Across the AWS Data Platform
Apply strong expertise in AWS Glue to embed data quality checks within ETL/ELT jobs and ensure data is validated during ingestion and transformation.
Use AWS Databricks to profile, standardize, cleanse, and prepare datasets for downstream consumption while improving overall quality and usability.
Design and execute high-performance validation queries and reconciliation checks within Amazon Redshift to verify data integrity at scale.
Partner with engineering teams to implement resilient and scalable quality controls using AWS services such as S3, Lambda, CloudWatch, and MWAA / Airflow.
Ensure monitoring, logging, and traceability are in place to support auditability and operational visibility of data quality processes.
Contribute to the design of data quality solutions that scale across complex, high-volume, and business-critical datasets.
III. Data Validation, Analysis, and Issue Resolution
Use AWS Databricks to profile, standardize, cleanse, and prepare datasets for downstream consumption while improving overall quality and usability.
Design and execute high-performance validation queries and reconciliation checks within Amazon Redshift to verify data integrity at scale.
Partner with engineering teams to implement resilient and scalable quality controls using AWS services such as S3, Lambda, CloudWatch, and MWAA / Airflow.
Ensure monitoring, logging, and traceability are in place to support auditability and operational visibility of data quality processes.
Contribute to the design of data quality solutions that scale across complex, high-volume, and business-critical datasets.
III. Data Validation, Analysis, and Issue Resolution
Write advanced, efficient SQL queries to perform data profiling, validation, reconciliation, defect detection, and root cause analysis.
Investigate recurring data quality issues to identify systemic causes across source systems, transformations, and downstream reporting layers.
Analyze patterns, anomalies, and trends in data defects to recommend preventive and corrective actions.
Collaborate with Data Engineers, Data Architects, Analysts, and Governance teams to define and enforce enterprise data quality standards.
Translate business requirements into measurable validation rules and acceptance criteria.
Document data quality checks, testing logic, exception handling, profiling outcomes, and remediation processes in a clear and maintainable manner.
Participate in defect triage, issue resolution, and continuous improvement efforts to enhance data reliability and reduce recurring quality issues.
Investigate recurring data quality issues to identify systemic causes across source systems, transformations, and downstream reporting layers.
Analyze patterns, anomalies, and trends in data defects to recommend preventive and corrective actions.
Collaborate with Data Engineers, Data Architects, Analysts, and Governance teams to define and enforce enterprise data quality standards.
Translate business requirements into measurable validation rules and acceptance criteria.
Document data quality checks, testing logic, exception handling, profiling outcomes, and remediation processes in a clear and maintainable manner.
Participate in defect triage, issue resolution, and continuous improvement efforts to enhance data reliability and reduce recurring quality issues.
Required Skills and Experience
9+ years of experience in Data Quality Assurance, Data Engineering, Data Validation, or Data Testing roles.
Strong experience building and executing data quality assurance processes, including automated testing, reconciliation, profiling, and defect management.
Expert-level proficiency in SQL with demonstrated ability to write complex queries for validation, analysis, reconciliation, and root cause investigation.
Strong programming experience in Python or a similar language for developing test automation frameworks, validation scripts, and data analysis tools.
Deep hands-on knowledge of AWS data technologies, including:
AWS Glue - job development, transformation logic, embedded quality checks, Data Catalog
Amazon Redshift - querying, validation, performance tuning, large-scale warehouse quality checks
AWS Databricks - data profiling, cleansing, normalization
MWAA / Airflow - workflow orchestration and automation
Proven ability to define, measure, monitor, and report on core data quality metrics, including completeness, consistency, accuracy, validity, uniqueness, and timeliness.
Experience implementing data quality controls, test strategies, and monitoring solutions within modern cloud data platforms.
Solid understanding of data modeling concepts, including star schema and snowflake schema, and how model design impacts data quality and testing strategy.
Experience supporting data governance, auditability, and compliance-oriented validation practices is highly preferred.
Strong analytical, troubleshooting, and communication skills with the ability to work cross-functionally across technical and business teams.
Healthcare domain experience preferred.
Strong experience building and executing data quality assurance processes, including automated testing, reconciliation, profiling, and defect management.
Expert-level proficiency in SQL with demonstrated ability to write complex queries for validation, analysis, reconciliation, and root cause investigation.
Strong programming experience in Python or a similar language for developing test automation frameworks, validation scripts, and data analysis tools.
Deep hands-on knowledge of AWS data technologies, including:
AWS Glue - job development, transformation logic, embedded quality checks, Data Catalog
Amazon Redshift - querying, validation, performance tuning, large-scale warehouse quality checks
AWS Databricks - data profiling, cleansing, normalization
MWAA / Airflow - workflow orchestration and automation
Proven ability to define, measure, monitor, and report on core data quality metrics, including completeness, consistency, accuracy, validity, uniqueness, and timeliness.
Experience implementing data quality controls, test strategies, and monitoring solutions within modern cloud data platforms.
Solid understanding of data modeling concepts, including star schema and snowflake schema, and how model design impacts data quality and testing strategy.
Experience supporting data governance, auditability, and compliance-oriented validation practices is highly preferred.
Strong analytical, troubleshooting, and communication skills with the ability to work cross-functionally across technical and business teams.
Healthcare domain experience preferred.
Preferred Candidate Profile
The ideal candidate is a hands-on data quality leader who combines strong technical depth with a quality assurance mindset. They are passionate about preventing defects, improving trust in enterprise data, and building scalable controls that make data quality measurable, visible, and actionable across the organization.
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
