TekWissen is a global workforce management provider headquartered in Ann Arbor, Michigan that offers strategic talent solutions to our clients world-wide. Our client provider of digital technology and transformation, information technology and services
Job Description:
Job Summary:
- We are seeking a Senior Data Engineer to design, build, and operate highly scalable batch and streaming data pipelines supporting T Mobile's Finance and Intelligence platforms.
- This role requires deep expertise in modern cloud data stacks (Snowflake, Databricks, dbt), strong SQL/Python skills, and solid understanding of finance data domains including billing, revenue, GL, and OPEX.
- The ideal candidate owns complex pipelines end to end, mentors junior engineers, and helps drive platform standards and best practices.
Key Responsibilities:
Data Pipeline Development:
- Design and build scalable, reliable ELT/ETL pipelines for finance data (billing, revenue, GL, OPEX).
- Implement batch and incremental ingestion patterns (full load, CDC, watermark-based).
- Build idempotent, rerunnable pipelines with robust error handling, retry logic, and dead-letter queue patterns.
Platform & Tooling:
- Develop and optimize pipelines using Snowflake (Snowpipe, Streams, Tasks, Dynamic Tables, performance tuning).
- Build data processing workflows in Databricks (PySpark, Delta Live Tables, Unity Catalog, job clusters).
- Create and maintain dbt models, tests, snapshots, macros, and packages with CI integration.
- Orchestrate data workflows using Airflow or Azure Data Factory (DAG design, dependencies, scheduling, alerts).
Cloud Infrastructure:
- Work within Azure (ADLS Gen2, Event Hub, ADF, Azure Functions, Key Vault) and/or AWS (S3, Glue, Lambda, Secrets Manager).
- Apply Infrastructure as Code fundamentals (Terraform, Bicep) for pipeline and resource provisioning.
- Apply cloud cost awareness including compute sizing, partitioning strategies, and storage optimization.
Languages & Frameworks:
- Write advanced SQL (CTEs, window functions, query tuning, execution plan analysis).
- Develop in Python (pandas, PySpark, requests, pytest, logging).
- Read and modify existing Scala/Spark jobs as needed.
- Use shell scripting for automation and operational tasks.
Streaming & Real Time Processing:
- Build near real time pipelines using Apache Kafka / Azure Event Hub.
- Implement Spark Structured Streaming with stateful aggregations, watermarking, and checkpointing.
- Support finance use cases such as revenue reconciliation and fraud signal feeds.
Data Quality & Testing:
- Implement unit and integration testing for pipelines (pytest, dbt tests).
- Create data quality checks (row counts, nulls, duplicates, referential integrity).
- Use Great Expectations or custom frameworks for validation.
- Monitor SLAs for pipeline latency and data freshness with alerting.
Data Modeling Support:
- Implement architected schemas (star, snowflake, data vault).
- Manage Slowly Changing Dimensions (SCD Type 1 & 2) for finance entities.
- Define partitioning and clustering strategies for large-scale finance tables.
- Support semantic layer definitions (metrics and dimensions).
DevOps & Engineering Practices:
- Participate in CI/CD for data pipelines using GitHub Actions or Azure DevOps.
- Follow Git branching strategies (trunk-based, feature branches).
- Perform code reviews and enforce engineering standards.
- Support environment promotion patterns (dev QA prod).
Security & Governance:
- Implement RBAC and row/column-level security in Snowflake and Databricks.
- Ensure PII and CPNI handling per T Mobile TISS 310 policy.
- Manage secrets securely (Key Vault, environment variables, no hardcoded credentials).
- Implement data lineage and audit instrumentation for compliance.
Collaboration & Communication:
- Partner with Data Architects to translate design specs into production-ready pipelines.
- Work closely with Data Analysts to optimize downstream consumption performance.
- Communicate pipeline incidents and data issues clearly to business stakeholders.
- Participate in on-call rotation to support production pipelines.
Senior-Level Expectations:
- Own delivery of complex, multi-source pipelines with minimal direction.
- Mentor junior and mid-level data engineers through pairing and code reviews.
- Identify and drive technical debt reduction alongside feature delivery.
- Contribute to and shape team standards, templates, and reusable components.
- Influence tooling, framework, and platform decisions across the team.
Required Qualifications:
- 8+ years of experience in data engineering or platform engineering roles.
- Strong experience with Snowflake, Databricks, and dbt in production environments.
- Advanced SQL and Python skills.
- Experience building finance or regulated data pipelines at scale.
- Preferred Qualifications
- Telecom industry experience (ARPU, churn, prepaid/postpaid metrics).
- Experience with both Azure and AWS cloud platforms.
- Prior experience supporting financial reporting and period-end close cycles.
TekWissen Group is an equal opportunity employer supporting workforce diversity.