Introduction At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, lets talk. Your Role and Responsibilities The Data Engineer is responsible for building and supporting the big data platform. As a Data Engineer - Platform your responsibilities include –
Develop, test and support future-ready data solutions for customers across industry verticals
Develop, test and support end-to-end batch and near real-time data flows/pipelines
Demonstrate understanding of data architectures, modern data platforms, big data, ML/AI, analytics, cloud platforms, data governance and information management and associated technologies
Develop and demonstrate Proof of Concepts and Working Demos
Lead or collaborate with other internal/external consultants in consulting, workshops and delivery engagements
Mentor junior IBM consultants in the practice and delivery engagements
To ensure success in the role you will possess the following skills –
The ideal candidate(s) should have 6-10 years of hands-on data engineering expertise and experience in building at least one python/spark / scala based data ingestion, transformation, and egress framework
Working experience on Data Lake Implementations on Azure cloud / BigData technologies
Cloud experience in one or more Azure, AWS or GCP environments
Expertise with modern data warehouse design patterns and tools
Good understanding of data architecture principles including data modelling
Should be able to tune Spark and Scala jobs for performance
Deep understanding of distributed computing systems
Should be familiar with Data Warehousing concepts and physical Data modelling techniques
Able to stitch together a Real-time, Streaming, kappa and lambda solution
Proficient with the latest Big data tools/technologies like Hive, Hadoop, Yarn, Kafka, Spark Streaming.
Good knowledge of AZURE cloud data technologies (Spark, ADLS2, CosmosDB, AKS, AEH). Experience not mandatory
Good temperament - buildings relationships and stakeholder management skills with an ability to interact with a diverse group of stakeholders and proactively provide actionable data
Familiar working with DevSecOps tools, methodologies - CI/CD (i.e., Jenkins, bitbucket, GitHub), Azure DevOps i.e best practices for code deployment
Working knowledge in job Orchestration (Control-M, Airflow)
Work with Architects, Technical Leads and Business Teams and contributes to the development of technical designs.
Strong knowledge of various data structures and the ability to extract data from various data sources.
Provides technical database consultation on application development, global infrastructure, and other database administration efforts related to specific DBMS.
Experience in writing complex SQL queries preferred
Ability to conduct data profiling, cataloguing, and mapping for technical design and construction of technical data flows.
Required Technical and Professional Expertise - Preferred Technical and Professional Expertise -
About Business Unit
IBM Consulting is IBM’s consulting and global professional services business, with market leading capabilities in business and technology transformation. With deep expertise in many industries, we offer strategy, experience, technology, and operations services to many of the most innovative and valuable companies in the world. Our people are focused on accelerating our clients’ businesses through the power of collaboration. We believe in the power of technology responsibly used to help people, partners and the planet.
The successful applicant will need to be fully vaccinated with an approved COVID-19 vaccination, or otherwise be granted an exemption from this requirement by IBM. This is to ensure that IBM can meet its work, health and safety obligations and also its customers' requirements.
Your Life @ IBM
Are you craving to learn more? Prepared to solve some of the world's most unique challenges? And ready to shape the future for millions of people? If so, then it's time to join us, express your individuality, unleash your curiosity and discover new possibilities. Every IBMer, and potential ones like yourself, has a voice, carves their own path, and uses their expertise to help co-create and add to our story. Together, we have the power to make meaningful change – to alter the fabric of our clients, of society and IBM itself, to create a truly positive impact and make the world work better for everyone. It's time to define your career. IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
IBM’s greatest invention is the IBMer. We believe that through the application of intelligence, reason and science, we can improve business, society and the human condition, bringing the power of an open hybrid cloud and AI strategy to life for our clients and partners around the world. Restlessly reinventing since 1911, we are not only one of the largest corporate organizations in the world, we’re also one of the biggest technology and consulting employers, with many of the Fortune 50 companies relying on the IBM Cloud to run their business. At IBM, we pride ourselves on being an early adopter of artificial intelligence, quantum computing and blockchain. Now it’s time for you to join us on our journey to being a responsible technology innovator and a force for good in the world.
For additional information about location requirements, please discuss with the recruiter following submission of your application.
Being You @ IBM
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, pregnancy, disability, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.