Data Engineer

☐ Linux & CLI · Required · 2–3 hrs

File system, grep, tail, awk, ssh. You will debug live cloud pipelines in a terminal daily. No GUI can save you at 2am during an incident.

• Linux and Shell Scripting for Data Engineers (Data Engineering): https://chaduvuko.com/learn/data-engineering/linux-shell

☐ Git & GitHub · Required · 2–3 hrs

Commits, branches, pull requests, merge conflicts. Every professional data job uses Git from day one. Interviewers check your GitHub.

• Git and Version Control for Data Projects (Data Engineering): https://chaduvuko.com/learn/data-engineering/git-for-data

☐ Networking Basics · Optional · 3 hrs

TCP/IP, DNS, HTTP, ports. Enough to understand why cloud resources need VNets, NSGs, and private endpoints.

• What is a Network? (Networking): https://chaduvuko.com/learn/networking/what-is-a-network
• The TCP/IP Model — How the Internet Works (Networking): https://chaduvuko.com/learn/networking/tcp-ip-model
• DNS — How Domain Names Become IP Addresses (Networking): https://chaduvuko.com/learn/networking/dns
• HTTP and HTTPS — The Protocol Powering the Web (Networking): https://chaduvuko.com/learn/networking/http-and-https

☐ Python · Required · 5–6 hrs

Functions, file I/O, REST API calls, error handling. Aim to write working scripts without Googling syntax. Pandas comes right after.

• What is Python? Setup & Your First Program (Python): https://chaduvuko.com/learn/python/what-is-python-setup
• Functions — Defining, Parameters, Return Values (Python): https://chaduvuko.com/learn/python/functions
• Reading & Writing Files (Python): https://chaduvuko.com/learn/python/reading-writing-files
• Exception Handling (Python): https://chaduvuko.com/learn/python/exception-handling

☐ SQL · Required · 4–5 hrs

SELECT, JOINs, GROUP BY, subqueries. The language used in every data role, every single day. You cannot skip this.

• Your First Query — SELECT & FROM (SQL): https://chaduvuko.com/learn/sql/select-from
• Introduction to Joins (SQL): https://chaduvuko.com/learn/sql/joins-intro
• Grouping Data — GROUP BY (SQL): https://chaduvuko.com/learn/sql/group-by
• Subqueries — Queries Inside Queries (SQL): https://chaduvuko.com/learn/sql/subqueries

☐ Pandas & NumPy · Required · 6–8 hrs

DataFrames, groupby, merge, reshape, apply. The standard Python manipulation layer before data reaches a warehouse or Spark cluster.

• Intro to NumPy and pandas (Python): https://chaduvuko.com/learn/python/numpy-pandas-intro
• Pandas for Data Analysis (AI & Machine Learning): https://chaduvuko.com/learn/ai-ml/programming/pandas-dataframes
• NumPy Arrays and Broadcasting (AI & Machine Learning): https://chaduvuko.com/learn/ai-ml/programming/numpy-arrays

☐ Advanced SQL · Required · 5–6 hrs

Window functions, CTEs, EXPLAIN plans, query optimisation, indexing. This is what separates junior from senior in every interview.

• Window Functions — OVER & PARTITION BY (SQL): https://chaduvuko.com/learn/sql/window-functions-intro
• CTEs — WITH Clause (SQL): https://chaduvuko.com/learn/sql/cte-with-clause
• Recursive CTEs — Hierarchies & Trees (SQL): https://chaduvuko.com/learn/sql/recursive-cte
• EXPLAIN & EXPLAIN ANALYZE (SQL): https://chaduvuko.com/learn/sql/explain-analyze

☐ Cloud Fundamentals · Required · 3–4 hrs

IaaS vs PaaS vs SaaS, regions, resource groups, IAM roles, storage accounts. Required before touching any Azure, AWS, or GCP service.

• Azure Introduction (Microsoft Azure): https://chaduvuko.com/learn/azure/introduction
• AWS Introduction (AWS): https://chaduvuko.com/learn/aws/introduction
• GCP Introduction (Google Cloud): https://chaduvuko.com/learn/gcp/introduction

☐ Azure Data Factory · Required · 8–10 hrs

Linked Services, Datasets, Pipelines, ForEach, Parameters, Triggers, Monitoring. The backbone of all six Chaduvuko real-world projects.

• Azure Data Factory (Microsoft Azure): https://chaduvuko.com/learn/azure/adf

☐ Orchestration · Optional · 6–8 hrs

Apache Airflow, ADF triggers, scheduling patterns. How production pipelines run automatically at scale without anyone pressing a button.

• Pipeline Orchestration — What a Scheduler Does (Data Engineering): https://chaduvuko.com/learn/data-engineering/pipeline-orchestration
• Cloud Composer (Google Cloud): https://chaduvuko.com/learn/gcp/composer

☐ Project 01 — Copy CSV · Chaduvuko · 3–4 hrs

First real ADF pipeline. FreshCart store data. Blob → ADLS Gen2. Every error you will hit is documented in the Error Library.

• Project 01 — Copy CSV to ADLS (Projects): https://chaduvuko.com/learn/projects/azure-batch-pipeline

☐ Project 02 — ForEach · Chaduvuko · 4–5 hrs

Copy all 10 FreshCart stores in one pipeline run using ForEach. Critical rule: ForEach must be Sequential, not Parallel, when writing to variables.

• Project 02 — ForEach Loop (Projects): https://chaduvuko.com/learn/projects/azure-projects-02

☐ Project 03 — Parameters · Chaduvuko · 4–5 hrs

Parameterized pipeline with run date. Dynamic paths. The pattern used in every production ADF pipeline at real companies.

• Project 03 — Run Date Pipeline (Projects): https://chaduvuko.com/learn/projects/azure-project-03

☐ dbt · Required · 6–8 hrs

Models, tests, docs, lineage. Now the standard transformation layer at modern data companies. Replaces stored procedures.

• What is dbt? (dbt): https://chaduvuko.com/learn/dbt/what-is-dbt
• Models: SELECT Statements as the Building Block (dbt): https://chaduvuko.com/learn/dbt/models-basics
• Incremental Models in Depth (dbt): https://chaduvuko.com/learn/dbt/incremental-models
• Testing: Generic and Singular Tests (dbt): https://chaduvuko.com/learn/dbt/testing-basics

☐ Data Modeling · Required · 5–6 hrs

Star schema, medallion architecture (bronze/silver/gold), slowly changing dimensions. The foundation behind every warehouse you will build.

• Data Modelling — Dimensional, Star and Snowflake Schema (Data Engineering): https://chaduvuko.com/learn/data-engineering/data-modelling
• Slowly Changing Dimensions — SCD Types 1, 2 and 3 (Data Engineering): https://chaduvuko.com/learn/data-engineering/slowly-changing-dimensions
• Data Vault 2.0 — Hubs, Links and Satellites (Data Engineering): https://chaduvuko.com/learn/data-engineering/data-vault

☐ Monitoring & Alerts · Optional · 3–4 hrs

ADF pipeline monitoring, alert rules, Azure Monitor. Day one at work: your manager will ask why last night's pipeline failed.

• Data Observability — Metrics, Logging and Anomaly Detection (Data Engineering): https://chaduvuko.com/learn/data-engineering/monitoring-observability

☐ System Design for DE · Required · 8–10 hrs

Design a data lake, a streaming pipeline, a warehouse schema under load. Asked at every senior interview. Amazon, Stripe, Shopify all ask this.

• Data Engineering System Design (Data Engineering): https://chaduvuko.com/learn/data-engineering/system-design-de

☐ Interview Prep · Chaduvuko · 10–12 hrs

Role-specific Q&A bank curated for US job market. Real questions from DoorDash, Instacart, and GCC companies — with model answers.

• Interview Prep — 60 Complete Answers (Data Engineering): https://chaduvuko.com/learn/data-engineering/de-interview-questions
• Top 50 SQL Interview Questions (SQL): https://chaduvuko.com/learn/sql/interview-questions