Senior Databricks Data Engineer
Location: Chicago-IL - Hybrid 3 days at On-site
Duration: 6 months
Role Descriptions:
Key Responsibilities
Pipeline Development Design, implement, and optimize scalable ETL/ELT pipelines using Databricks (PySpark, SQL, Delta Lake) to ingest structured and semi structured data from multiple sources (APIs, databases, streaming).
Data Warehousing Build and maintain cloud data warehouse solutions (Snowflake / Azure Synapse) design star schemas, fact/dimension tables, and aggregate tables for high performance reporting.
Data Modeling Create logical and physical data models for operational and analytical use cases; implement SCD Type 2, slowly changing dimensions, and data vault methodologies where appropriate.
Performance Tuning Optimize Spark jobs, SQL queries, and data partitioning strategies to handle petabyte scale data with low latency.
Governance & Quality Implement data quality checks, monitoring, and lineage using tools like Great Expectations or custom frameworks; enforce data governance policies (GDPR/CCPA).
Collaboration Partner with data analysts, product managers, and engineers to translate business requirements into technical data solutions.
CI/CD & Automation Automate deployment of data pipelines using Azure DevOps or GitHub Actions; maintain infrastructure as code (Terraform) for data resources.
Required Skills & Experience
Total Experience: 10+ years in data engineering or related roles.
Cloud Data Platforms: Deep hands on experience with Databricks (notebooks, jobs, clusters, Delta Lake, Unity Catalog) must have production level work.
Data Warehousing: Proven experience with cloud data warehouses (Snowflake, Azure Synapse, or Redshift) design, optimisation, and administration.
Data Modeling: Strong knowledge of dimensional modeling (Kimball/Inmon), relational database design, and experience with tools like ER/Studio or dbt.
Programming: Expert in Python and SQL ability to write maintainable, production grade code.
Big Data: Hands on with Apache Spark (PySpark), distributed computing, and performance tuning.
Orchestration: Experience with workflow tools (Airflow, Azure Data Factory, or Prefect) for scheduling and monitoring pipelines.
Version Control: Proficient with Git and collaborative development workflows.
Preferred Qualifications
Experience with streaming technologies (Kafka, Event Hubs, or Kinesis).
Knowledge of data mesh or data fabric architectures.
Familiarity with BI tools (Power BI, Tableau, Looker).
Databricks certification (e.g., Associate or Professional Data Engineer).
Experience with dbt (data build tool) and transformation testing.
Exposure to MLflow or MLOps practices.
Education & Soft Skills
Bachelor's or Master's degree in Computer Science, Information Systems, or a related field (or equivalent practical experience).
Strong communication skills
Self starter with a problem solving mindset and ability to work independently in a hybrid environment.
Keyword:
Skills: Digital : Snowflake~Digital : Databricks
Experience Required: 10 & Above
Donato Technologies Inc. is a trusted IT staffing, consulting, and software development partner headquartered in Dallas, Texas. We support clients across industries by understanding their unique business needs and delivering tailored technology and workforce solutions. Our focus is on connecting the right talent with the right opportunity-ensuring clients receive dependable, skilled professionals and candidates receive meaningful career growth and support. We work closely with small to mid-sized organizations to provide flexible, high-quality services that drive performance, innovation, and long-term success.
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more
More from Donato Technologies, Inc