Description As a GCP Data Engineer, you will be instrumental in modernizing our data infrastructure by building robust, automated ETL/ELT pipelines and leveraging Google Cloud Platform's powerful suite of big data tools. Collaborating closely with Enterprise Architects, you will spearhead the migration from legacy Hadoop/CDP systems to BigQuery. Your daily focus will involve architecting historical and incremental data loads, orchestrating tasks with Apache Airflow, and creating scalable, data-driven solutions that empower the organization with clean, accessible, and organized data.
Must Have:
Certification: Professional GCP Data Engineer Certification or equivalent. Development Experience: 2+ years of coding experience in Java/Python and Infrastructure as Code using Terraform. GCP Expertise: 2+ years of experience working in GCP-based Big Data deployments (both Batch and Real-Time), specifically leveraging BigQuery, Bigtable, Google Cloud Storage, Pub/Sub, Data Fusion, Dataflow, Dataproc, and Airflow. Data Design: Proven history of designing and delivering comprehensive data lake and data warehousing solutions. Data Processing: Strong hands-on experience in extracting, loading, transforming (ETL/ELT), cleaning, and validating data, as well as designing pipelines and architectures for data processing. Database Skills: Proficiency in at least one SQL language. Big Data Tools: Practical experience working with Spark services. Methodologies: Experience working within Agile and Lean methodologies. Good to Have (Preferred):
Data Visualization: Experience using visualization tools such as Qlik or Looker Studio. Migration Experience: Proven experience in migrating legacy systems (preferably Big Data Cloudera Data Platform) into GCP technologies. Large-Scale Systems: Experience working with either a MapReduce or an MPP (Massively Parallel Processing) system at any size or scale. undefined
