Databricks Certified Data Engineer Zero to Hero
Contact us

2026 edition · Pre-recorded lessons with support

Databricks Zero to Hero

One path from SQL basics to production pipelines, governance, CI/CD and AI on Databricks, with certification prep and interview practice running alongside, not tacked on at the end.

Go from data enthusiast to Databricks Certified Data Engineer extraordinaire with our comprehensive Zero to Hero course - your journey to mastery starts here!

Files & APIs Bronze Silver Gold
22
in-depth modules with labs
120–150
hours of content and labs
3 + 1
projects and a capstone
2
certification tracks

starstarstarstarstar_half 4.6 (11 ratings)

Learners enrolled: 652

Created by Sagar Prajapati

Language: English

About the course

Most students can write a PySpark transformation. Far fewer can tell you why their job got slower last week, how to restart a streaming pipeline without losing data, or how to get code into production without copying notebooks by hand. The course is built around those questions.

Built for
Freshers, career switchers and working engineers
Format
Pre-recorded lessons with support; about 70% hands-on engineering
Certification
Associate and Professional tracks, mapped to the official exam guides
Full course
Complete by end of November 2026

Who it's for

Freshers & career switchers

  • Starting from zero, or close to it, with SQL, Python and data concepts
  • Want a clear learning order instead of jumping between playlists
  • Need projects you can explain in an interview

Working engineers

  • Use SQL or Spark at work and want production-level depth
  • Moving toward senior, lead or architect-style roles
  • Want to get comfortable with Lakeflow, Unity Catalog, CI/CD and performance tuning

A note before you start

Hi, I'm Sagar. Here's why I built this course.

Sagar Prajapati

Sagar Prajapati

Lead Data Engineer · Founder & instructor, GeekCoders

I've been teaching Databricks and data engineering through GeekCoders since 2022. Since 2024 I've also been on the founding team of an early-stage startup building a data platform that runs natively on Databricks, so most of my week is spent on exactly the things this course covers.

Most students can write a PySpark transformation. Far fewer can tell you why their job got slower last week, how to restart a streaming pipeline without losing data, or how to get code into production without copying notebooks by hand. So the course is one path from SQL basics to production pipelines, governance, CI/CD and AI on Databricks, with certification prep and interview practice running alongside.

You'll write a lot of code. You'll also break things on purpose and fix them, because that's the part most courses skip and the part that makes you useful on a team.

"If you put in the hours, I'll make sure every one of them counts. See you in Module 01."

Less watching. More building.

Roughly 70% of the course is hands-on engineering. Every lesson follows the same four steps, so you always know what's coming and can keep going on your own.

  1. Understand itWhat the feature is, why it exists, how it works under the hood, and when you'd reach for it.
  2. Build it with meImplemented step by step on realistic data, with every important decision explained.
  3. Build it aloneA similar but different problem. No copying the instructor's code. This is where it sticks.
  4. Prove itConcept checks, output-prediction questions, scenarios, common mistakes and a short assessment.

A real example

In the Delta Lake module, MERGE INTO is not the end. You implement an upsert, push duplicate and late-arriving records through it, check the results, dig into a failure, and explain when a different incremental pattern would have been the better choice.

22 modules, four phases

Foundations come first, then core Databricks engineering, then production-grade delivery, then AI and data applications. Every module includes concept lessons with the internals explained, an instructor-led build, an independent challenge, a hands-on lab on realistic data, interview questions and certification-style scenarios.

Foundations

For anyone starting from zero. These four modules are the base everything else sits on.

01Data engineering and the lakehouse3 hrs
  • What data engineers do on real projects; OLTP vs OLAP
  • Warehouse vs lake vs lakehouse; ETL vs ELT, batch vs streaming
  • Medallion architecture: Bronze, Silver, Gold
  • Workspace, account, metastore, catalogs, schemas, tables, volumes
  • How storage, compute, Delta Lake and Unity Catalog fit together

Lab: Design the full data architecture for an e-commerce company and identify every component.

02SQL for data engineers3–4 hrs
  • SELECT, filtering, CASE, NULL handling; inner, outer, semi, anti and cross joins
  • Aggregations, HAVING, subqueries, CTEs
  • Window functions: ranking, running totals, dedup
  • Dates, strings, arrays, maps and JSON
  • Interview classics: gaps and islands, top-N per group, sessionization

Lab: Solve 25 SQL problems on sales, customer, order and transaction data.

03Python for data engineering3–4 hrs
  • Variables, collections, conditions, loops, functions
  • Comprehensions, lambdas, iterators, generators
  • Exceptions, logging, modules, packages, virtual envs
  • Reading JSON, CSV, Parquet and API responses
  • REST APIs: pagination, auth concepts, retries; testing and debugging

Lab: Build a reusable API ingestion utility with pagination, logging and error handling.

04Apache Spark and PySpark fundamentals5–6 hrs
  • Driver, executors, jobs, stages, tasks, partitions
  • Lazy evaluation; transformations vs actions; DataFrames and Spark SQL
  • Narrow vs wide transformations and shuffles; repartition, coalesce, caching
  • Broadcast joins, skew, shuffle partitions, Catalyst, physical plans
  • UDFs vs built-ins; reading the Spark UI and execution plans

Lab: Compare join and aggregation strategies, inspect the plans, and diagnose a skewed workload.

Core Databricks engineering

The heart of the job: Delta Lake, ingestion, modelling, Lakeflow pipelines and orchestration.

05Databricks platform and compute3 hrs
  • Workspace architecture and account-level concepts
  • All-purpose vs job compute, SQL warehouses, serverless
  • Compute policies, autoscaling, auto-termination, access modes, Photon
  • Notebooks, Git folders, SQL editor, Catalog Explorer
  • Legacy DBFS vs Unity Catalog volumes; cost drivers and security boundaries

Lab: Pick the right compute for an ETL job, a SQL dashboard, a streaming pipeline and interactive development.

06Delta Lake, in depth5 hrs
  • Transaction log and ACID guarantees; schema enforcement and evolution
  • MERGE INTO, upserts, deletes, updates
  • Time travel, history, restore and recovery
  • OPTIMIZE, VACUUM, data skipping, file sizing
  • Change Data Feed, deletion vectors, concurrency; partitioning vs Z-ORDER vs liquid clustering

Lab: Build an incremental Delta table, merge late updates, recover an old version and optimise it.

07Ingestion: files, APIs and databases5 hrs
  • Batch ingestion with Spark readers and SQL; CSV, JSON, Parquet, Avro
  • Cloud storage and external locations
  • Auto Loader: inference, evolution, rescued data; COPY INTO
  • JDBC, REST APIs, rate limits, retries; incremental ingestion and checkpoints
  • Idempotency, duplicates, audit columns; schema drift and corrupt records

Lab: Ingest files and API data, handle schema drift, and make the whole process restartable.

08Data modelling and transformation5 hrs
  • Implementing Bronze, Silver and Gold; cleansing, standardising, dedup and validation
  • Star schema, facts and dimensions; surrogate keys, grain
  • SCD Type 1 and 2; incremental aggregates; late-arriving facts
  • CDC and out-of-order events; MERGE, window logic, CDC vs CDF
  • Schema contracts, reusable patterns, quarantine and reconciliation

Lab: Build a customer and orders model with SCD Type 2 and incremental Gold aggregates.

09Lakeflow Spark Declarative Pipelines5 hrs
  • Declarative vs imperative pipelines; pipelines in SQL and Python
  • Streaming tables, materialized views, flows, sinks
  • Expectations, quality metrics, quarantine
  • AUTO CDC, SCD Type 1/2, event ordering
  • Configuration, publishing modes, permissions; event logs, troubleshooting, cost

Lab: Create a Bronze-to-Gold pipeline with data quality rules and incremental CDC.

10Lakeflow Jobs and orchestration4 hrs
  • Multi-task jobs, dependencies, task types
  • Parameters, task values, dynamic inputs
  • Schedules, triggers, concurrency, retries, notifications, timeouts, repair runs
  • Pipeline, notebook, Python and SQL tasks; idempotent workflows and backfills
  • Run history and job cost

Lab: Orchestrate ingestion, transforms, quality checks and reporting with retries, alerts and backfills.

Production & professional level

What separates a pipeline that ran once from a platform a business can rely on.

11Unity Catalog and enterprise governanceProduction
  • Grants, ownership, inheritance, service principals
  • Storage credentials, external locations, secrets
  • Row filters, column masks, PII and retention
  • Lineage, audit logs, discovery, least privilege

Lab: Set up role-based access, column masking, a governed volume and an auditable pipeline.

12Performance and cost engineeringProduction
  • Spark UI, query profiles, execution plans
  • Reducing shuffles, handling skew, broadcast joins
  • Partition sizing, small files, compaction
  • Data skipping, stats, Z-ORDER, liquid clustering, Photon
  • Caching, predicate pushdown, column pruning; system tables and cost allocation

Lab: Speed up a slow pipeline and an expensive SQL workload; document runtime, cost and correctness before and after.

13Structured Streaming and real-time dataProduction
  • Micro-batch execution; sources, sinks, triggers, checkpoints, offsets
  • Watermarks, event time, late data, state
  • Stream-stream and stream-static joins; exactly-once semantics
  • Kafka / Event Hubs and Delta streaming; dedup and replay
  • Failures, checkpoint recovery, monitoring

Lab: Build a near-real-time events pipeline that handles late events and survives a checkpoint recovery.

14Testing, CI/CD and deploymentProduction
  • Git workflows, branches, PRs, reviews
  • Databricks Asset Bundles: YAML config, targets, variables, environments
  • Dev, test and prod deployment; GitHub Actions and service principals
  • Unit, integration and data-quality tests; CLI, REST APIs, rollback
  • Secrets and release management

Lab: Deploy a full pipeline to test, then promote it to production through a Git-based workflow.

15Monitoring, observability and troubleshootingProduction
  • Job, pipeline and streaming monitoring; system tables, event logs, audit records, alerts
  • Freshness, volume anomalies, schema drift
  • Root-cause analysis for slow or failed jobs
  • SLAs, incident response, runbooks, reconciliation and dashboards

Lab: Diagnose five injected production failures from logs, metrics and execution details.

16Data sharing and interoperabilityProduction
  • Delta Sharing and secure sharing patterns; Lakehouse Federation
  • Unity Catalog with external engines; Apache Iceberg
  • Open table formats and trade-offs; data contracts and ownership

Lab: Design a governed data-sharing solution and justify the right integration pattern.

Modern Databricks & AI

AI-powered ETL, RAG, agents, apps and Lakebase. These are an extra career track and are not counted as exam coverage unless the live official guide includes them.

17Lakeflow Connect and advanced ingestionAI track
  • Managed connectors and supported SaaS, database and file sources
  • Incremental ingestion and connector state; standard vs managed connectors
  • Limits, permissions, schema changes, cost; Auto Loader vs JDBC vs APIs vs connectors

Lab: Design ingestion for several sources and implement one supported connector end to end.

18AI Functions and AI-powered ETLAI track
  • Where LLMs help in data transformation; AI Functions in SQL, notebooks and pipelines
  • Extraction, classification, enrichment, summaries
  • Prompt design, structured outputs, batch inference, cost control
  • Deterministic logic vs LLM calls; PII, quality checks, human review

Lab: Turn resumes or job descriptions into validated, structured data and compare against a rule-based baseline.

19Vector Search, embeddings and RAGAI track
  • Embeddings, chunking, semantic similarity; vector indexes and metadata filters
  • Hybrid retrieval and reranking; RAG architecture
  • Evaluation sets, hallucination checks, governance, latency, quality and cost

Lab: Build a document Q&A assistant, measure retrieval quality, and add access controls.

20Agentic AI on DatabricksAI track
  • LLM apps vs agents; tool calling and structured outputs
  • Multi-step workflows, state, human approval
  • Evaluation, tracing, safety boundaries; governed retrieval and SQL tools
  • Model serving, deployment and production monitoring

Lab: Build a data-quality investigation agent that queries approved data and writes an evidence-based report.

21Databricks Apps, Lakebase and operational appsAI track
  • Interactive apps on Databricks Apps; connecting apps to warehouses and governed data
  • Lakebase Postgres: projects, branches, transactions, CRUD and transactional state
  • Service principals, permissions, secrets; syncing Lakebase and lakehouse tables

Lab: Build a data-quality management app with transactional status updates and lakehouse history.

22Capstone: architecture and production readinessCapstone
  • Turning business requirements into architecture
  • Choosing ingestion, storage, transform and serving patterns
  • Contracts, SLAs, security, recovery, cost estimation and tuning
  • CI/CD, monitoring, incident response; presenting trade-offs like a senior engineer

Lab: Deliver a complete solution: architecture, code, tests, deployment, quality metrics and a walkthrough.

Three projects you can talk about

Interviewers ask about what you've built, so these aren't copy-the-notebook exercises. Each comes with requirements, a starter repo, tasks, expected output, failure scenarios and a review checklist.

01 · Beginner

Retail sales lakehouse

Ingest CSV and JSON data, build Bronze, Silver and Gold tables, clean the data with PySpark and SQL, and finish with a sales dashboard.

Delta Lake Medallion Joins Schema handling

Flow: CSV / JSON → Bronze → Silver → Gold

02 · Beginner → Intermediate

Incremental ingestion framework

Ingest files and paginated APIs with restartability, schema evolution, duplicate prevention, error logging and incremental processing.

Auto Loader COPY INTO APIs Idempotency

Flow: Files / API → Auto Loader → Bronze (checkpointed)

03 · Intermediate

Customer 360 and CDC

Combine customer, order and transaction data, resolve duplicates, implement CDC with SCD Type 2, and build incremental Gold metrics.

MERGE INTO AUTO CDC SCD Type 2 Reconciliation

Flow: Customers + orders + txns → MERGE → SCD2 → 360 view

Every project includes

Built to be explained in an interview, not just run once.

  • A clear business problem and architecture diagram
  • A working implementation, not a concept notebook
  • Data-quality checks, validation results and performance notes
  • Source data, expected schemas and business rules
  • At least three failure scenarios and how to recover
  • Security and access-control decisions
  • README, setup guide, interview explanation and Git repo

Two exam tracks, built in

Exam prep doesn't wait for the last week. Short assessments run through the core modules, then each exam gets its own revision track.

Track A

Data Engineer Associate

For freshers and engineers new to Databricks. Aligned to the official exam guide effective May 4, 2026, and re-checked before every exam cohort.

Platform fundamentals Modules 1, 5, 6
Ingestion and loading Modules 7, 9, 17
Transformation and modelling Modules 2, 4, 6, 8, 9
Lakeflow Jobs Module 10
CI/CD Module 14
Troubleshooting and optimisation Modules 4, 12, 15
Governance and security Module 11
  • Topic-wise question bank with an explanation for every option
  • Two timed mock exams matching the verified format
  • Final revision sheet: common traps and feature comparisons
Track B

Data Engineer Professional

For working engineers focused on production. Aligned to the official Professional guide (November 30, 2025 version), re-checked against the live guide before revision modules go out.

Developing processing code Modules 4, 6, 8, 9, 13
Ingestion and acquisition Modules 7, 17
Transformation and quality Modules 8, 9, 15
Sharing and federation Module 16
Monitoring and alerting Modules 10, 12, 15
Cost and performance Modules 4, 6, 12
Security, governance and compliance Modules 11, 16
Debugging and deployment Modules 14, 15
Data modelling Modules 6, 8
  • Scenario questions that need real trade-off thinking
  • Performance case studies with execution plans
  • CI/CD and production incident case studies
  • Two timed mock exams aligned to the live guide

The AI and app modules are an extra career track. They are not counted as exam coverage unless the live official guide includes them.

Practice for the actual interview

Interview practice runs through the whole course, then comes together in its own track.

50 SQL challenges
30 PySpark challenges
10 Debugging tasks
3 Mock interviews

SQL & PySpark coding

  • SQL problems from joins and windows to messy business cases
  • PySpark problems: dedup, CDC, nested data, joins, aggregations
  • Tasks where you fix someone else's broken code

Architecture & scenario rounds

  • Design an incremental pipeline for billions of events
  • Handle duplicate, out-of-order and late records
  • Auto Loader vs COPY INTO vs JDBC vs Lakeflow Connect

Spark internals & troubleshooting

  • Why did this job suddenly get slower?
  • When does a broadcast join help, and when does it hurt?
  • Recovering a failed streaming job safely

Mock interviews & project defence

  • Three mock interviews, each harder than the last
  • Defending optimisation decisions with evidence
  • Talking about projects in clear, business terms

Two levels. Your pace.

You don't need every advanced topic before you're useful. Level 1 gets you job-ready; Level 2 takes you deeper.

Level 1: job-ready data engineer

Beginner → Intermediate · 8–10 weeks at 12–15 hrs/week

  • Weeks 1–2 SQL, Python, Spark and lakehouse fundamentals
  • Weeks 3–4 Delta Lake, ingestion, medallion architecture, data modelling
  • Weeks 5–6 Lakeflow pipelines, jobs, governance and compute
  • Weeks 7–8 Testing, CI/CD, troubleshooting and Associate prep
  • Weeks 9–10 Polishing projects, mock interviews, targeted revision

Outcome: a solid foundation, finished projects, and structured preparation for the Associate exam and entry-level interviews.

Level 2: professional engineer + AI

Intermediate → Advanced · another 6–8 weeks

  • Stage 1 Advanced Spark, streaming, CDC and pipeline optimisation
  • Stage 2 Production monitoring, governance, security, data sharing
  • Stage 3 CI/CD, deployment, incident response, cost optimisation
  • Stage 4 AI Functions, RAG, Vector Search, agents, Apps and Lakebase
  • Stage 5 Professional exam prep, architecture interviews, capstone defence

Outcome: broader production skills, a stronger portfolio and focused prep for the Professional exam. Durations are estimates; if you already know SQL, Python and Spark, you'll move faster.

What we promise, and what we don't

  • Taught by a working data engineer, not slide-readers
  • Pre-recorded lessons with support, and about 70% of the course is hands-on
  • Content maintained against Databricks' own exam guides and product docs
  • Three projects, a capstone, and a refund policy stated clearly up front

We don't promise

Exam results or job placement. Certification exams are run by Databricks; this course prepares you thoroughly but cannot guarantee exam outcomes or job offers.

Enroll in Databricks Zero to Hero

22 modules, 120–150 hours, three projects and a capstone. Full course complete by end of November 2026.

Enroll now

What learners say

Enroll now