Mid-Level Data Engineer
We are seeking a skilled Data Engineer / Backend Developer to design, build, and operate cloud-native, API-driven Big Data systems. The role centers on AWS EMR and Spark-based batch processing, with strong emphasis on Python development, workflow orchestration, and production reliability.
Key Responsibilities
- Design and develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications.
- Build scalable backend services and data engineering solutions supporting data processing and operational workflows.
- Develop and maintain data transformation processes using Spark, SQL, Hive, Python, Scala, and related technologies.
- Build cloud-native data solutions using AWS services including EMR, S3, EC2, Lambda, DynamoDB, and API Gateway.
- Design and enhance Apache Airflow workflows, including complex DAGs, scheduling, dependency management, monitoring, and failure handling.
- Optimize AWS EMR performance and cost efficiency across large-scale data workloads.
- Participate in requirements gathering, technical research, and solution design.
- Contribute to architecture reviews, code reviews, performance tuning, and operational readiness.
- Support testing, validation, and integration across applications and data pipelines.
- Collaborate with product owners, data engineers, backend engineers, QA, and DevOps teams in an Agile/Scrum environment.
- Troubleshoot production issues and drive automation to improve system reliability.
- All other duties as assigned.
Required Qualifications
- 6+ years of hands-on experience building enterprise-scale applications, data platforms, or distributed systems.
- 6+ years of experience developing and operating Big Data platforms in the cloud, preferably using AWS EMR and the Hadoop ecosystem.
- Strong hands-on experience with AWS, Spark, Python and/or Scala, Airflow, SQL, and Hive.
- Advanced Python or Scala development skills with experience building production-grade data pipelines.
- Strong experience with Apache Airflow orchestration for complex, production workflows.
- Solid understanding of data engineering concepts including batch processing, data quality, performance optimization, and reliability.
- Experience with Git, Jenkins, and CI/CD workflows.
- Strong analytical, problem-solving, and communication skills.
- Ability to work effectively in cross-functional Agile/Scrum teams.
Preferred Qualifications
- Experience with LLMs, Generative AI, Agentic AI, or AI-assisted engineering workflows.
- Experience with API design, microservices, event-driven, or serverless architectures.
- Experience with infrastructure as code, automated testing, and production deployment.
- Exposure to data governance, metadata management, lineage, or data observability platforms.
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more