Location: Hartford, CT (Hybrid)
Duration: Contract to hire
Summary:
Seeking a senior Databricks Architect/Admin to design, implement, and optimize enterprise data platforms. This role will lead architecture, governance, performance, and cost optimization while supporting scalable data pipelines and cloud infrastructure.
Key Responsibilities:
- Architect and manage Databricks platform (Unity Catalog, Delta Lake, Workflows)
- Design scalable data pipelines (DLT, Spark/PySpark, streaming)
- Optimize performance, cost (DBUs), and cluster configurations
- Manage Unix/Linux environments, scripting, and automation
- Ensure governance, security, and compliance standards
- Integrate with tools like Fivetran, Ab Initio, and cloud platforms
- Support AI/ML workflows (MLflow, AutoML)
Requirements:
- 7+ years in data engineering; 4+ years Databricks
- Strong Python, PySpark, SQL
- Expertise in Databricks (Unity Catalog, Delta Lake, Workflows)
- Unix/Linux & scripting (Bash)
- Experience with AWS/Azure/GCP
- Knowledge of CI/CD, Git, Jira/ServiceNow
Nice to Have:
- MLflow, AutoML, AI/LLM exposure
- Airflow or workflow orchestration tools
- Data governance tools (Collibra, Alation)