Location: Lincoln, NE Salary: $60.00 USD Hourly - $65.00 USD Hourly
Description: ONLY W2 Job Title: Senior Data Engineer - Encounter Processing & Data Location: Lincoln, NE (Remote) Duration: 12+ Months
About the Role We are seeking a highly experienced Senior Data Engineer to serve as the primary technical owner of a large-scale enterprise data processing platform responsible for ingesting, transforming, validating, and processing healthcare and operational data. This mission-critical environment supports complex batch processing workflows, business rule execution, and downstream reporting and analytics systems.
The successful candidate will be responsible for the ongoing maintenance, enhancement, operational support, and optimization of a distributed data processing framework built on Scala, Apache Spark, Hive, and Drools, operating within a Cloudera Data Platform (CDP) environment.
This role requires deep technical expertise, strong production support capabilities, and the ability to independently manage the full lifecycle of enterprise data processing solutions in a highly regulated environment.
What You'll Do Technical Ownership & Platform Management:
- Own and maintain the enterprise Scala/Spark data processing framework.
- Manage and enhance business rule implementations using the Drools Rules Engine.
- Oversee enterprise data ingestion processes for reference, master, and source-system data.
- Support and optimize Spark and Hive batch processing workflows.
- Lead production issue investigation, root cause analysis, and defect resolution.
- Coordinate application releases, deployments, and version management activities.
- Drive application performance tuning, scalability improvements, and operational efficiency.
- Maintain technical documentation, operational procedures, and knowledge-sharing resources.
- Provide platform monitoring and basic administrative support within the Cloudera ecosystem while collaborating with infrastructure and cloud operations teams.
Production Support & Continuous Improvement:
- Troubleshoot and resolve issues across distributed data processing applications.
- Monitor system health, job execution, and data pipeline performance.
- Implement business, regulatory, and operational enhancements within existing data processing frameworks.
- Ensure platform reliability, maintainability, and operational continuity.
- Collaborate with cross-functional teams to deliver high-quality, scalable data solutions.
Minimum Qualifications Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience. 7+ years of experience designing and developing enterprise-scale distributed data processing applications.
Strong hands-on programming experience with Scala. Experience building and maintaining Apache Spark (2.x and/or 3.x) applications. Experience working with Cloudera Data Platform (CDP) or comparable enterprise Hadoop ecosystems.
Strong expertise with Apache Hive for data processing, data modeling, and table management. Experience implementing and maintaining business rules using the Drools Rules Engine.
Advanced SQL development, query tuning, and performance optimization skills. Experience supporting Linux-based production environments. Proven experience troubleshooting and optimizing distributed Spark workloads.
Experience with Git and modern software development and version control practices.
Preferred Qualifications Experience in healthcare, payer, or regulated industry environments. Familiarity with large-scale encounter, claims, or transactional data processing systems.
Experience with Cloudera Manager, HDFS, and YARN administration. Experience integrating with enterprise ETL platforms such as IBM DataStage.
Familiarity with cloud environments supporting Hadoop and Cloudera workloads, including AWS. Experience working within Agile development methodologies and tools such as Jira and Confluence
By providing your phone number, you consent to: (1) receive automated text messages and calls from OpenKyber, Inc. and its affiliates (collectively "OpenKyber") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with OpenKyber's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from OpenKyber and HELP for help. Contact: This job and many more are available through OpenKyber. Please apply with us today!
For applications and inquiries, contact:[email protected]
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
