We are seeking a highly skilled and experienced Lead Data Software Engineer to lead our data engineering team in the design, implementation, and maintenance of our data platform.
RESPONSIBILITIES
- Collaborate with Solution Architects to design and build a configuration/metadata-driven framework for data transformation and orchestration
- Develop and implement a Data Ingestion Framework, identifying patterns for initial and incremental data loads
- Guide the data engineering team in adopting best practices for data processing, ensuring scalability and maintainability of the data platform
- Oversee the documentation of all data engineering processes and frameworks for knowledge sharing and future reference
- Collaborate with DevOps Team to deploy & configure required infrastructure components, connect to Data Source, enable CI/CD, and deliver pipelines implementation (framework)
REQUIREMENTS :
- Data Architectural mindset
- Hands-on coding experience of Databricks Jobs with Python, PySpark
- Hands-on experience with Data Lake or Data Platform implementation
- Good understanding of relational and NoSQL data stores, methods, and approaches (star and snowflake, dimensional modeling)
- Experience in Azure cloud data platform, namely Azure Databricks, Delta Lake, ADSL2, Azure Data Factory, Azure SQL Database, Azure Functions
- Proven experience in leading data engineering teams
- Background or Excessive Knowledge in designing and implementing metadata-driven frameworks for data transformation and orchestration
- Expertise in developing data ingestion strategies for handling both initial and incremental data loads. Various integration patterns (REST, Files, DB Connect)
- Solid understanding of data engineering best practices and the ability to implement these in a team setting
- Excellent problem-solving skills and the ability to lead complex data engineering projects
- Strong communication and collaboration skills to work effectively with both technical and non-technical stakeholders
NICE TO HAVE
- Relevant certifications in Azure, Databricks, and data engineering methodologies
WE OFFER
- Career plan and real growth opportunities
- Unlimited access to LinkedIn learning solutions
- International Mobility Plan within 25 countries
- Constant training, mentoring, online corporate courses, eLearning and more
- English classes with a certified teacher
- Support for employee’s initiatives (Algorithms club, toastmasters, agile club and more)
- Enjoyable working environment (Gaming room, napping area, amenities, events, sport teams and more)
- Flexible work schedule and dress code
- Collaborate in a multicultural environment and share best practices from around the globe
- Hired directly by EPAM & 100% under payroll
- Law benefits (IMSS, INFONAVIT, 25% vacation bonus)
- Major medical expenses insurance: Life, Major medical expenses with dental & visual coverage (for the employee and direct family members)
- 13 % employee savings fund, capped to the law limit
- Grocery coupons
- 30 days December bonus
- Employee Stock Purchase Plan
- 12 vacations days plus 4 floating days
- Official Mexican holidays, plus 5 extra holidays (Maundry Thursday and Friday, November 2nd, December 24th & 31st)
- Monthly non-taxable amount for the electricity and internet bills
or continue with email
RESPONSIBILITIES
- Collaborate with Solution Architects to design and build a configuration/metadata-driven framework for data transformation and orchestration
- Develop and implement a Data Ingestion Framework, identifying patterns for initial and incremental data loads
- Guide the data engineering team in adopting best practices for data processing, ensuring scalability and maintainability of the data platform
- Oversee the documentation of all data engineering processes and frameworks for knowledge sharing and future reference
- Collaborate with DevOps Team to deploy & configure required infrastructure components, connect to Data Source, enable CI/CD, and deliver pipelines implementation (framework)
REQUIREMENTS :
- Data Architectural mindset
- Hands-on coding experience of Databricks Jobs with Python, PySpark
- Hands-on experience with Data Lake or Data Platform implementation
- Good understanding of relational and NoSQL data stores, methods, and approaches (star and snowflake, dimensional modeling)
- Experience in Azure cloud data platform, namely Azure Databricks, Delta Lake, ADSL2, Azure Data Factory, Azure SQL Database, Azure Functions
- Proven experience in leading data engineering teams
- Background or Excessive Knowledge in designing and implementing metadata-driven frameworks for data transformation and orchestration
- Expertise in developing data ingestion strategies for handling both initial and incremental data loads. Various integration patterns (REST, Files, DB Connect)
- Solid understanding of data engineering best practices and the ability to implement these in a team setting
- Excellent problem-solving skills and the ability to lead complex data engineering projects
- Strong communication and collaboration skills to work effectively with both technical and non-technical stakeholders
NICE TO HAVE
- Relevant certifications in Azure, Databricks, and data engineering methodologies
WE OFFER
- Career plan and real growth opportunities
- Unlimited access to LinkedIn learning solutions
- International Mobility Plan within 25 countries
- Constant training, mentoring, online corporate courses, eLearning and more
- English classes with a certified teacher
- Support for employee’s initiatives (Algorithms club, toastmasters, agile club and more)
- Enjoyable working environment (Gaming room, napping area, amenities, events, sport teams and more)
- Flexible work schedule and dress code
- Collaborate in a multicultural environment and share best practices from around the globe
- Hired directly by EPAM & 100% under payroll
- Law benefits (IMSS, INFONAVIT, 25% vacation bonus)
- Major medical expenses insurance: Life, Major medical expenses with dental & visual coverage (for the employee and direct family members)
- 13 % employee savings fund, capped to the law limit
- Grocery coupons
- 30 days December bonus
- Employee Stock Purchase Plan
- 12 vacations days plus 4 floating days
- Official Mexican holidays, plus 5 extra holidays (Maundry Thursday and Friday, November 2nd, December 24th & 31st)
- Monthly non-taxable amount for the electricity and internet bills
Sign in
To continue your applicationor continue with email
By continuing you agree to our Terms & Privacy Policy.
Similar jobs
Show more