Senior Data Engineer ((Data Replication & UAT Platform)

Environment Provisioning & Data Replication

Develop automated processes for cloning and provisioning environments, including:

• Production UAT
• Production Development
• Production QA
• Production Demo
• Production Production
• Future environment combinations

Support capabilities including:

• Full environment cloning
• Customer-specific cloning
• Multi-tenant data selection
• Incremental refresh
• Partial refresh
• Environment synchronization
• Automated validation

Build repeatable workflows that allow engineering teams to provision environments on demand rather than through one-time manual processes.

Data Synchronization

Implement and optimize synchronization mechanisms between:

• Snowflake
• Azure SQL
• Azure Cosmos DB
• Azure Data Factory
• Azure Functions
• Application databases

Develop processes for:

• Data hydration
• Incremental synchronization
• Change tracking
• Data consistency validation
• Error detection and recovery
• Performance optimization

Data De-identification

Partner with Solution Architecture, Security, and Data Privacy teams to implement enterprise data protection processes.

Responsibilities include:

• Integrate Delphix-based masking workflows.
• Implement policy-driven de-identification.
• Support database masking strategies.
• Validate masked datasets.
• Ensure appropriate masking based on target environment.
• Support HIPAA-compliant handling of PHI.

Focus will be on structured database environments rather than raw files.

Automation & DevOps

Develop automation supporting:

• Environment provisioning
• Database refresh
• Data synchronization
• Pipeline execution
• Validation workflows
• Operational monitoring

Contribute to Infrastructure as Code and CI/CD practices in collaboration with Platform Engineering and DevSecOps teams.

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
• 7+ years of Data Engineering experience.
• Strong expertise in enterprise data engineering on Microsoft Azure.
• Hands-on experience with:

• • Snowflake
  • Azure Data Factory
  • Azure SQL Database
  • Azure Cosmos DB

• Experience building enterprise ETL/ELT pipelines.
• Experience with large-scale database replication and synchronization.
• Strong SQL development skills.
• Experience with Python or similar scripting languages for data automation.
• Familiarity with Git-based development workflows.

Preferred Qualifications

Experience with:

• Delphix
• Data masking and de-identification
• Healthcare data platforms
• HIPAA-regulated environments
• Azure Functions
• Kubernetes
• Terraform
• CI/CD pipelines
• Event-driven architectures
• CDC (Change Data Capture)
• Data quality and validation frameworks
• Multi-tenant SaaS applications

Technical Skills

Data Platforms

• Snowflake
• Azure SQL Database
• Azure Cosmos DB
• Azure Data Factory
• Azure Storage
• SQL Server

Programming

• SQL
• Python
• PowerShell (preferred)
• JSON
• REST APIs

Cloud

• Microsoft Azure
• Azure Functions
• Azure Key Vault
• Azure Monitor

Data Engineering

• ETL / ELT
• Data replication
• Incremental refresh
• Data synchronization
• CDC
• Data quality
• Performance tuning

DevOps

• Git
• Azure DevOps
• CI/CD
• Terraform (preferred)
• Infrastructure as Code

Security

• HIPAA
• PHI handling
• Data masking
• Delphix integration
• Secure data movement

At the end of the engagement, the Data Engineer will have successfully:

• Built and validated scalable pipelines supporting Snowflake, Azure SQL, Cosmos DB, and Azure Data Factory.
• Implemented automated workflows for cloning and refreshing environments across multiple source and target combinations, including full-environment and customer-specific cloning.
• Developed reliable synchronization processes ensuring consistency across operational and analytical data stores.
• Integrated data de-identification processes using Delphix in accordance with enterprise privacy policies and HIPAA requirements.
• Delivered repeatable, on-demand automation for environment provisioning and refresh, reducing manual effort and improving operational efficiency.
• Contributed to technical documentation, implementation estimates, and engineering assets that enable a smooth transition from discovery to implementation.