9 to 15 years, Excellent Academics (B.Tech/M.Tech – NIT/IIT/BITs or Top Colleges), Very Hands on with career aspiration aligned core technical ladder.
Principal Data Engineer
at the Data Analytics Centre of Excellence, you shall be very hands-on, and shall be responsible for designing, developing and maintaining a scalable and robust Data Platform involving development of complex data processing pipelines for both streaming and batching applications, ETL, data integration, developing high throughput data collection services , designing efficient data access with data marts.
_Cultivate a customer-centric, results-driven, high quality data engineering solutions.
_Partner with internal and external teams to ingest, store and enrich data throughout the organization within the enterprise data lake and underlying warehouses/data marts/etc
_Operate as a thought leader with the ability to guide, influence, and inspire peak performance across the data architecture design, data services and ETL functions
_Collaborate with Business leaders, Data Science, backend services and Client Development teams on implementing analytics and machine learning algorithms at scale to facilitate audience intelligence, segmentation and personalization initiatives
_Collaborate with Business Intelligence and extended analytics groups on data mart optimizations, query tuning, and data model designs with a focus on delivering actionable insights
_Work strategically to align data assets, storage, and computation/query layers to support long-term data platform goals, help define data models, publish metadata, guide best practice ETL standards
_Make sound architectural decisions across relational, multi-dimensional, distributed and object-based storage platforms
Knowledge and Area of Expertise:
_Hands-on Experience in Designing, Developing and Rolling out Large Scale Big Data Solutions involving both batch and real time stream processing Apache Kafka, Streaming Solutions in Spark/Flink and Elasticsearch.
_Experience in designing and developing data warehouse and data marts with experience in star schema, Apache Kylin and Apache Druid Experience will be an added advanatage.
_Experience in designing and developing solutions for on-Premise and Cloud (AWS/GCP)
_Thorough understanding of internals of Big Data Technology Stack including Spark/Flink, Kafka EMR, distributed solutions using efficient MapReduce Algorithms
_Experience in Troubleshooting and Performance Tuning of Kafka, Spark/Flink based solutions. Performance tuning of Go, Java, Python, Scala based applications.
_Hands on experience in developing large scale high throughput services in Go/Java based services with expertise in tuning Go/Java applications with understanding of parallelism, multithreading, memory management, performance, etc.
_Experience in creating Relational and NoSQL data models to fit the needs of a diverse set of data consumers ( Data Analysts, Data Scientists and Business Analysts).
_Experience in designing solutions for data collections for real time streaming applications from mobile devices/IOTs/, REST based HTTP/Websocket/gRPC Services
_Experience in designing large scale high through user event processing systems from mobile applications and IOTs with focus on payloads optimizations and efficient data collection.