Big Data Developer

Job Title: Big Data Developer (Spark, AWS, Airflow, Snowflake, Big Data & ETL)
Company: Cogency Inc.
Location: Greater Toronto Area (Hybrid - 4 days office)
Job SummaryWe are seeking a skilled and detail-oriented Data Engineer with strong expertise in Snowflake, Big Data technologies, and ETL processes. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and ensuring high data quality, performance, and security across the will work in an Agile environment, collaborating with cross-functional teams to deliver robust data solutions that support analytics, reporting, and business intelligence initiatives.
Key ResponsibilitiesDesign, develop, and maintain data ingestion pipelines and ETL workflowsBuild and optimize data pipelines, frameworks, and transformation processesEnsure data quality, integrity, security, and performance across systemsDevelop and optimize SQL queries, stored procedures, and data modelsIntegrate Snowflake with various data sources and BI/reporting toolsMonitor, troubleshoot, and resolve data pipeline and platform issuesCollaborate with cross-functional teams in an Agile delivery environmentMaintain comprehensive documentation for pipelines, transformations, and data modelsImplement best practices in DevSecOps, CI/CD, and data engineering standards
Required Skills & Experience5+ years of experience in Data Engineering / ETL developmentStrong hands-on experience with Big Data technologies:HadoopSparkHiveSolid experience working with Snowflake data platformStrong expertise in SQL and data modeling conceptsProgramming experience in Scala or Java, including API developmentExperience with ETL tools such as:InformaticaTalendApache AirflowFamiliarity with cloud platforms (preferably AWS)Understanding of DevSecOps practices, CI/CD pipelines, and Infrastructure-as-CodeKnowledge of GenAI tools for code generation and developer productivity enhancementExcellent problem-solving, analytical, and communication skills
Preferred / Nice-to-Have SkillsExperience with AWS services (e.g., Glue, S3, Lambda)Hands-on experience with Apache Airflow or similar workflow orchestration toolsExperience with CI/CD tools (GitHub Actions, Git, automated testing tools)Exposure to containerization technologies (Docker, Kubernetes, OpenShift)Knowledge of Python or scripting languagesExperience with shell scripting
EducationBachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field
Key CompetenciesStrong analytical and problem-solving mindsetAbility to work independently and in team environmentsExcellent communication and stakeholder collaboration skillsAttention to detail and commitment to data quality