Technology Stack Reference: Development activities will primarily leverage Snowflake, Matillion, AWS Services (S3, Lambda, DynamoDB, CloudWatch), GitHub, Cortex, and Snowflake Intelligence.
Tech stack & core skills: Aligned with the existing CDW stack, including Snowflake, Matillion, AWS services (S3, Lambda, DynamoDB, CloudWatch), along with GitHub.
Role expectation: We are looking for an expert Data Engineer with good English speaking and strong communication skills and a proactive mindset to effectively probe and gather requirements as well from the sales team.
Stakeholder interaction: The resource gets involved in discussions with Sales Compensation stakeholders along with Architect and other onshore developers, engaging in discussions to capture/gather compensation-related requirements.
Requirements Gathering: Collaborating with business stakeholders to map, refine, and translate business logic into scalable technical workflows.
Data Source Analysis: Analyze target data structures and underlying systems.
Pipeline Design: Design data processing pipelines using the defined technology stack.
ETL/ELT Development: Development of data ingestion, processing, and Extract, Transform, Load (ETL) jobs.
Data Cleaning & Validation: Implementing validations, data cleaning steps, and quality checkpoints.
Deployment & Testing: Deployment and thorough testing of ETL jobs within the development and staging environments.