Create Alert
Email me similar jobs

Data Engineer with Python Spark Databricks and AIML

Data Engineer Python, Spark, Databricks & GenAI

Location: Dallas, TX
Duration: 6+ Months Contract

Preferred: Candidates with Telecom industry experience


Job Summary

We are seeking an experienced Data Engineer with expertise in Python, Spark, Databricks, and AI/ML technologies to build scalable data platforms and Generative AI solutions. The ideal candidate should have experience developing data pipelines, LLM applications, AI agent workflows, and cloud-based data engineering solutions.


Key Responsibilities
  • Design and build scalable data pipelines using Python, Spark, Databricks, and Delta Lake.

  • Develop and manage data solutions on Azure, Snowflake, and PostgreSQL.

  • Build and optimize LLM/GenAI pipelines, including data ingestion, prompt engineering, fine-tuning, and deployment.

  • Develop AI workflows using LangChain, LangGraph, AI Agents, and RAG architectures.

  • Integrate with OpenAI, Azure OpenAI, Hugging Face, and other LLM APIs.

  • Collaborate with data scientists, AI engineers, and business teams.

  • Ensure data quality, governance, security, and performance across enterprise data platforms.


Required Skills
  • 7 12 years of Data Engineering experience.

  • Strong expertise in Python, PySpark, and Apache Spark.

  • Hands-on experience with Databricks and Delta Lake.

  • Experience with Snowflake and PostgreSQL.

  • Experience building LLM/GenAI applications and pipelines.

  • Strong knowledge of LangChain, LangGraph, AI Agents, and RAG (Retrieval-Augmented Generation).

  • Experience with OpenAI, Azure OpenAI, or Hugging Face APIs.

  • Good understanding of Azure Cloud architecture and scalable data solutions.

  • Strong analytical, problem-solving, and communication skills.


Preferred Skills
  • Telecom domain experience.

  • Experience with Model Context Protocol (MCP).

  • AI Agent orchestration.

  • Docker.

  • CI/CD pipelines.

  • Azure or GCP cloud platforms.


Top Skills
  1. Python & PySpark

  2. Databricks & Apache Spark

  3. LLM/GenAI (LangChain, LangGraph, RAG, OpenAI/Azure OpenAI)

  4. Snowflake & Delta Lake

  5. Azure Cloud



Nice to Have
  • Telecom industry experience

  • MCP (Model Context Protocol)

  • Docker & CI/CD

  • GCP Cloud Experience


Data Engineer with Python Spark Databricks and AIML

Apply Now
Back to search page