Technical Lead - Data Engineering

TO THE NEW (Noida, UP, India) 4 days ago


  • Define client needs and oversee project milestones to ensure expectations, timelines, and budgets are met

  • Define data platform architecture and design, have hands on capability to review code and make required changes

  • Leverage extensive knowledge of distributed systems, process flows and procedures to aid analyses and recommendations for solution offerings

  • Responsible for the overall quality of project deliverables and the successful implementation of defined solution for the customer

  • Build long-term, superior client relationships and proactively manage client expectations, and ensure that change control is used when scope boundaries are exceeded

  • Communicate a compelling and inspired vision to all customer levels, from CTOs/CIOs, to engineering managers and programmers

  • Maintain a strong network and promote the organization at various meetings, forums, panels, publications, and conferences to establish thought leadership in the industry

  • Responsible for POCs on any tech stacks introduced, or possible version upgrades

Must Have:

  • At least 8+ years of experience in design and development using various database technologies with recent 4+ years associated Hadoop technology stack and programming languages

  • Hands­-on and technical lead experience in 2 or more areas:

    • Hadoop, HDFS, MR

    • High Availability architecture and DR setup

    • Spark Streaming, Spark SQL, Spark ML

    • Kafka/Flume

    • Apache NiFi

    • Worked with Hortonworks Data Platform as Architect CDH (Cloudera Distribution for Hadoop) as developer/administrator

    • Hive / Pig / Sqoop

    • NoSQL Databases ­ HBase/Cassandra/Neo4j/MongoDB

    • Visualisation & Reporting frameworks like D3.js, Zeppelin, Grafana, Kibana Tableau, Pentaho

    • Scrapy for crawling websites

    • Good to have knowledge of Elastic Search

    • Good to have understanding on Google Analytics data streaming.

    • Data security (Kerberos/OpenLDAP/Knox/Ranger)

  • Should have a very good overview of the current landscape and ability to visualise technology and industry trends

  • Ability to successfully lead a team of big data engineers and big data developers through all phases of the development life cycle, including requirements definition, architecture, design, development, implementation, and testing

  • Working knowledge of Big Data Integration with Third party / in house built Metadata Management, Data Quality, Master Data Management Solutions, Structured/Unstructured data

  • Have been active in the community in terms of articles / blogs / speaking engagements at conferences


Technical Lead - Data Engineering

Apply On Company Site
Back to search page