Sr. Data Engineer/Lead Data Engineer
About the Role:
Xtelify Ltd is looking for a Data Engineer to spearhead our Data Platform team. In
this role, you will architect, scale, and oversee the deployment of mission-critical data
pipelines and data-products handling a massive scale of ~5B daily events and a concurrency
of 5M active users. Our platform is built on a cloud-native modern data stack (AWS/GCP),
enabling real-time reporting and deep data exploration from first principles.
As a Lead, you will not only write high-performance code but also define technical
roadmaps, mentor engineers, and ensure our infrastructure supports Xtelify's aggressive
growth.
• Experience: 4+ Years (with 3+ years dedicated to data engineering, software
development, or distributed systems)
• Job Location: Gurgaon
Responsibilities:
• Architecture & Scaling: Architect, build, and optimize robust, fault-tolerant, and
high-performance data pipelines capable of processing multi-terabyte streaming data
daily across Xtelify's video streaming platforms.
• SLA Ownership: Define and maintain the infrastructure required to extract,
consume, and analyze live streaming data within a high-availability, 99.999% SLA
Big Data environment.
• Data Strategy & Optimization: Enhance data models, workflows, and processing
frameworks to improve performance, scalability, and cost-efficiency across both
structured and unstructured datasets.
• Data Governance: Drive data governance, data security, and compliance best
practices across large-scale, complex data environments.
• Cross-Functional Collaboration: Partner with data scientists, product teams, and
business stakeholders to translate complex requirements into scalable, data-driven
solutions.
• Technical Leadership & Mentorship: Serve as the technical anchor for the
engineering team. Provide mentorship, promote engineering best practices in pipeline
and API design, and manage the full lifecycle of data services.
• Innovation: Stay ahead of emerging technologies to drive innovation in analytics
pipelines, real-time data serving, and infrastructure for machine learning applications.
Desired Profile:
• Education: B.E / B.Tech / M.E / M.Tech / M.S in Computer Science or Software
Engineering from a premier institute.
• Core Expertise: Deep, hands-on experience with Spark and Scala/Java with a
proven track record of deploying production-grade streaming and batch applications
at scale.• Streaming & Real-Time: Advanced, production-level experience with real-time and
streaming data architectures such as Kafka, Flink, or Spark Streaming for low-
latency data processing.
• System Design & Modeling: Strong grasp of advanced data structures, algorithms,
distributed system design, ETL pipeline design, and dimensional modeling.
• Big Data & Cloud Ecosystem: Deep proficiency in the Hadoop ecosystem and cloud
platform ecosystems (AWS/GCP) including tools like S3/GCS, EMR/Dataproc,
Cloud SQL, Nifi, Trino and query optimization.
• Delivery & Execution: Proven ability to work in complex, unstructured
environments and take end-to-end technical ownership of complex, enterprise-wide
data modules.
Good to Have:
• API Development: Hands-on experience building APIs using REST, GraphQL, or
gRPC to serve real-time and batch data.
• OLAP Engines: Experience with real-time OLAP engines like Apache Druid,
Apache Pinot, or ClickHouse.
• NoSQL Solutions: Exposure to high-throughput NoSQL solutions like Cassandra,
MongoDB, Redis, or Elasticsearch at scale.
• Cloud Infrastructure: Familiarity with Infrastructure as Code (Terraform) and
containerization (Docker, Kubernetes).
• AI Agents for Data Ops: Experience building intelligent AI agents or LLM-driven
tools to automate BAU data operations (e.g., automated root-cause analysis for
pipeline failures, self-healing workflows, semantic parsing of logs, or intelligent
optimization of query costs and resource allocation).
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
