Location: Berkley Heights, NJ
Work Mode: 5-days in office (flexible to support weekend)
Job Description:
We are seeking a highly skilled resource to design and implement high-performance, event-driven data pipelines, ensuring low-latency data processing and high availability system for the large credit card processing system. The ideal candidate will work with the Ab Initio ecosystem (GDE, EME, Conduct>It) to build stateful services that ingest, filter, and transform data from sources like Kafka or message queues, pushing updates to dashboards or downstream databases in near-real-time.
Responsibilities
Design, develop, and maintain complex ETL graphs using Ab Initio GDE (Graphical Development Environment)
Architect end-to-end data integration solutions spanning multiple source/target systems (RDBMS, mainframe, flat files, message queues)
Optimize graph performance - parallelism, partitioning (broadcast, hash, range), layout tuning, checkpointing
Lead development using Ab Initio components: Co>Operating System, EME (Enterprise Meta>Environment), Conduct>It, Data Profiler, Continuous Flows
Own metadata management and data lineage within EME
Design and implement CDC (Change Data Capture) and near-real-time data pipelines
Troubleshoot production job failures, perform root-cause analysis, and implement fixes with minimal downtime
Define coding standards, review peer-developed graphs, and enforce best practices
Mentor junior/mid-level Ab Initio developers
Collaborate with data architects, DBAs, and business analysts to translate requirements into technical design
Required Qualifications
Bachelor's degree in Computer Science, Engineering, or related field
7+ years of ETL development experience, with 5+ years specifically in Ab Initio
Strong hands-on expertise with Ab Initio GDE, Co>Op, EME, and Conduct>It
Deep understanding of parallel processing concepts and Ab Initio partitioning/multi-file system (MFS)
Strong SQL skills and experience with major RDBMS (DB2)
Experience with Unix/Linux shell scripting
Solid understanding of data warehousing concepts (dimensional modeling, SCDs, star/snowflake schemas)
Experience with Kafka
Strong debugging and performance-tuning skills for large-volume data processing
Preferred Qualifications
Experience with mainframe data integration (COBOL copybooks, VSAM, flat files)
Exposure to Kubernetes, openshift
Familiarity with Hadoop/Spark ecosystems and hybrid ETL architectures
Knowledge of API-based or streaming integration (Kafka)
Prior experience as a technical lead or in a client-facing architecture role
Soft Skills / Leadership
Strong analytical and problem-solving skills for complex data issues
Ability to lead design discussions and drive technical consensus
Effective mentorship and knowledge-sharing across the team
Clear communication with both technical and business stakeholders
I
T Recruiter
: 901-466-4708
www.conchtech.com
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
