Principal Data Engineer

Experience:
9 to 15 years, Excellent Academics (B.Tech/M.Tech – NIT/IIT/BITs or Top Colleges), Very Hands on with career aspiration aligned core technical ladder.
As a
Principal Data Engineer
at the Data Analytics Centre of Excellence, you shall be very hands-on, and shall be responsible for designing, developing and maintaining a scalable and robust Data Platform involving development of complex data processing pipelines for both streaming and batching applications, ETL, data integration, developing high throughput data collection services , designing efficient data access with data marts.
Key responsibilities:
 _Cultivate a customer-centric, results-driven, high quality data engineering solutions.
 _Partner with internal and external teams to ingest, store and enrich data throughout the organization within the enterprise data lake and underlying warehouses/data marts/etc
 _Operate as a thought leader with the ability to guide, influence, and inspire peak performance across the data architecture design, data services and ETL functions
 _Collaborate with Business leaders, Data Science, backend services and Client Development teams on implementing analytics and machine learning algorithms at scale to facilitate audience intelligence, segmentation and personalization initiatives
 _Collaborate with Business Intelligence and extended analytics groups on data mart optimizations, query tuning, and data model designs with a focus on delivering actionable insights
 _Work strategically to align data assets, storage, and computation/query layers to support long-term data platform goals, help define data models, publish metadata, guide best practice ETL standards
 _Make sound architectural decisions across relational, multi-dimensional, distributed and object-based storage platforms
 
Knowledge and Area of Expertise:
 _Hands-on Experience in Designing, Developing and Rolling out Large Scale Big Data Solutions involving both batch and real time stream processing Apache Kafka, Streaming Solutions in Spark/Flink and Elasticsearch.
 _Experience in designing and developing data warehouse and data marts with experience in star schema, Apache Kylin and Apache Druid Experience will be an added advanatage.
 _Experience in designing and developing solutions for on-Premise and Cloud (AWS/GCP)
 _Thorough understanding of internals of Big Data Technology Stack including Spark/Flink, Kafka EMR, distributed solutions using efficient MapReduce Algorithms
 _Experience in Troubleshooting and Performance Tuning of Kafka, Spark/Flink based solutions. Performance tuning of Go, Java, Python, Scala based applications.
 _Hands on experience in developing large scale high throughput services in Go/Java based services with expertise in tuning Go/Java applications with understanding of parallelism, multithreading, memory management, performance, etc.
 _Experience in creating Relational and NoSQL data models to fit the needs of a diverse set of data consumers ( Data Analysts, Data Scientists and Business Analysts).
 _Experience in designing solutions for data collections for real time streaming applications from mobile devices/IOTs/, REST based HTTP/Websocket/gRPC Services
 _Experience in designing large scale high through user event processing systems from mobile applications and IOTs with focus on payloads optimizations and efficient data collection.