Role: Data Engineer
Location: Phoenix, AZ
Can do Only W2, No C2C
Job Description:
We are looking for a highly experienced Data Engineer with strong expertise in GCP, Scala, Spark, and Python. The ideal candidate should have hands-on experience building scalable data pipelines, distributed data processing systems, and cloud-based analytics platforms in enterprise environments.
Key Responsibilities:
- Design, develop, and maintain scalable batch and real-time data pipelines using Spark and Scala.
- Build and optimize large-scale data processing solutions on Google Cloud Platform (GCP).
- Develop ETL/ELT workflows for ingesting, transforming, and loading structured and semi-structured dataWork with BigQuery, Dataflow, Dataproc, Cloud Storage, and Pub/Sub services within GCP.
- Develop robust data engineering solutions using Python and Scala programming languages.
- Collaborate with architects, data scientists, analysts, and business teams to support analytics and reporting needs.
- Optimize Spark jobs for performance, scalability, and fault tolerance.
- Implement data quality checks, monitoring, and validation processes.
- Support CI/CD deployment processes and automation for data engineering workflows.
- Participate in code reviews, Agile ceremonies, and production support activities.
- Troubleshoot data pipeline issues and ensure high availability of enterprise data platforms.
Required Skills:
- 12+ years of overall IT experience.
- Strong hands-on experience with GCP services such as BigQuery, Dataproc, Dataflow, Pub/Sub, and Cloud Storage.
- Expertise in Scala, Apache Spark, and Python.
- Strong experience in building distributed data processing and ETL pipelines.
- Experience with Spark SQL, Spark Streaming, and performance tuning.
- Knowledge of relational and NoSQL databases.
- Experience working with large-scale enterprise data environments.
- Strong understanding of data warehousing and data lake concepts.
- Familiarity with Git, Jenkins, CI/CD pipelines, and Agile methodologies.
- Excellent analytical, troubleshooting, and communication skills.
Best Regards:
Lucy Rose
Phone: +1-469-612-3133
Email: [email protected]
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more