DP-3028-A: Implement Generative AI engineering with Azure Databricks
Course Overview
The DP-3028-A: Implement Generative AI Engineering with Azure Databricks course is designed for data engineers, AI developers, and solution architects who aim to leverage the full potential of Generative AI within the Azure Databricks ecosystem. Participants will gain advanced expertise in designing, building, and deploying generative AI models and pipelines for enterprise-grade applications.
Through hands-on labs, real-world case studies, and advanced engineering techniques, learners will explore the integration of large language models (LLMs), natural language processing (NLP), and machine learning workflows within Databricks. The course emphasizes scalable, secure, and efficient AI solutions that drive business insights, automation, and innovation.
In this course, you will:
- Gain hands-on experience implementing Retrieval-Augmented Generation (RAG) and fine-tuning large language models (LLMs).
- Explore multi-stage reasoning techniques using LangChain, LlamaIndex, Haystack, and DSPy.
- Understand and apply LLMOps practices for model deployment, monitoring, and governance with MLflow and Unity Catalog.
- Incorporate responsible AI principles, including risk mitigation and ethical considerations.
- Build and operationalize generative AI solutions using Azure Databricks and Apache Spark.
- Acquire in-demand generative AI skills in a focused, one-day training format.
Prerequisites:
Before starting this module, you should be familiar with fundamental Azure Databricks concepts.
Target Audience
Schedule Dates
Course Content
- Understand Generative AI
- Understand Large Language Models (LLMs)
- Identify key components of LLM applications
- Use LLMs for Natural Language Processing (NLP) tasks
- Exercise – Explore language models
- Explore the main concepts of a RAG workflow
- Prepare your data for RAG
- Find relevant data with vector search
- Rerank your retrieved results
- Exercise – Set up RAG
- What are multi-stage reasoning systems?
- Explore LangChain
- Explore LlamaIndex
- Explore Haystack
- Explore the DSPy framework
- Exercise – Implement multi-stage reasoning with LangChain
- What is fine-tuning?
- Prepare your data for fine-tuning
- Fine-tune an Azure OpenAI model
- Exercise – Fine-tune an Azure OpenAI model
- Explore LLM evaluation
- Evaluate LLMs and AI systems
- Evaluate LLMs with standard metrics
- Describe LLM-as-a-judge for evaluation
- Exercise – Evaluate an Azure OpenAI model
- What is responsible AI?
- Identify risks
- Mitigate issues
- Use key security tooling to protect your AI systems
- Exercise – Implement responsible A
- Transition from traditional MLOps to LLMOps
- Understand model deployments
- Describe MLflow deployment capabilities
- Use Unity Catalog to manage models
- Exercise – Implement LLMOps
FAQs
This course is designed for data engineers and AI practitioners looking to build, deploy, and manage generative AI solutions. It specifically focuses on leveraging the Databricks Data Intelligence Platform to handle the full lifecycle of Large Language Models (LLMs), from data preparation to RAG (Retrieval-Augmented Generation) architectures.
Yes. A core pillar of DP-3028-A is implementing RAG workflows. You will learn how to:
- Chunk and vectorize proprietary data.
- Store data in Databricks Vector Search.
- Connect these vectors to LLMs to provide contextually accurate, company-specific responses.
Absolutely. Building an AI is only half the battle; the course covers MLflow for LLMs, tracking experiments, and using Databricks Model Serving to deploy your models with low latency and high reliability.
While AI-102 focuses on pre-built Azure AI Services (like Vision or Language APIs), DP-3028-A is an engineering-heavy deep dive into the Databricks ecosystem. It is tailored for those who need to build custom, scalable AI architectures rather than just consuming “off-the-shelf” APIs.
Yes. The curriculum is regularly updated to include the latest advancements in Mosaic AI and the integration of the latest models (like Llama 3 or GPT-4o) via Azure OpenAI.
Flexible Training Options to Meet Your Needs
We know the importance of flexibility in the success of learning and professional development. This is the reason CounselTrain provides a variety of delivery options for all IT training offered in the UAE to ensure access and convenience for each learner.
Select the method that best suits your needs.
