Location: Berkley Heights, NJ
Work Mode: 5-days in office (flexible to support weekend)
Responsibilities:
Architect and build scalable, fault-tolerant data pipelines using Apache Spark (Java)
Lead design of batch and streaming ETL/ELT systems handling large data volumes
Deep-dive performance tuning: partitioning strategy, memory management, shuffle/skew optimization, job cost reduction
Set coding standards and lead code/design reviews across the team
Drive technical decisions on data architecture, storage formats, and pipeline orchestration
Mentor mid-level and junior engineers; act as a technical escalation point
Partner with product, analytics, and platform teams to translate requirements into scalable systems
Own production reliability - on-call ownership, incident response, root-cause analysis for pipeline failures
Evaluate and introduce new tools/frameworks where they improve the system
Contribute to capacity planning and cost optimization for cluster infrastructure
Required Qualifications
Bachelor's or Master's degree in Computer Science, Engineering, or related field
7+ years of professional Java development experience
5+ years hands-on experience with Apache Spark in production environments
Expert-level understanding of distributed systems: fault tolerance, data locality, shuffle mechanics, resource management
Proven track record designing systems processing terabyte+ scale data
Strong SQL skills and deep familiarity with columnar storage formats (Parquet, ORC, Avro, Delta Lake/Iceberg)
Experience with cluster managers (YARN, Kubernetes) and cloud-managed Spark
Proficiency with Kafka
Strong grasp of CI/CD, containerization, and infrastructure-as-code practices
Preferred Qualifications
Experience with Flink or other stream-processing frameworks
Familiarity with data governance, lineage, and quality frameworks
Experience with workflow orchestration at scale
Background in system design for multi-tenant or multi-region data platforms
Prior experience leading a team or acting as a technical lead
Soft Skills / Leadership
Excellent communication - able to explain technical tradeoffs to non-technical stakeholders
Strong mentorship and coaching ability
Comfortable driving ambiguous, cross-team technical initiatives
Behavioral Skills:
- Good Communication skills
- 5 days Work from Office at Berkley Heights, NJ
- Team Player
- Ability to work in a changing environment
- Strong problem solving and analytical skills
- Ability to work independently or within a team
- Manage day-to-day challenges and communicate developmental risks with the technical team
Recruiter
|
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
