Skip to content
← Back to Microsoft Azure track
Reference

Microsoft Azure Cheat Sheet

Identity, ADLS Gen2, Data Factory, Databricks, Synapse, Event Hubs, Key Vault and Fabric — the Azure data track on one page. Bookmark it.

Subscription & resource groups
Sign in and pick a subscription
az login
az account set --subscription "Lake Dev"
az account show --query '{name:name, id:id}' -o json
One group per environmentdev and prod stay apart
az group create --name rg-lake-dev --location eastus
az group list --query "[].name" -o tsv
# delete the group and the lake, factory, and vault go with it
Where you are
az account list --query "[].{name:name, isDefault:isDefault}" -o table
# pipelines, storage, and the warehouse belong in the same region
RBAC & managed identity
Data-plane roleEntra ID holds the identity
az role assignment create --assignee analyst@contoso.com --role "Storage Blob Data Contributor" --scope "/subscriptions/<sub-id>/resourceGroups/rg-lake-dev/providers/Microsoft.Storage/storageAccounts/stlakedev"
User-assigned identity
az identity create --resource-group rg-lake-dev --name id-pipeline-dev
az identity show --resource-group rg-lake-dev --name id-pipeline-dev --query principalId -o tsv
Grant that identityno storage account keys
az role assignment create --assignee-object-id <principal-id> --assignee-principal-type ServicePrincipal --role "Storage Blob Data Contributor" --scope "/subscriptions/<sub-id>/resourceGroups/rg-lake-dev/providers/Microsoft.Storage/storageAccounts/stlakedev"
# ADF, Databricks, and Synapse use this identity
ADLS Gen2
Account with a hierarchy
az storage account create --resource-group rg-lake-dev --name stlakedev --location eastus --sku Standard_LRS --kind StorageV2 --hierarchical-namespace true
File system and folder
az storage fs create --name lake --account-name stlakedev --auth-mode login
az storage fs directory create --file-system lake --name bronze/orders/dt=2026-03-15 --account-name stlakedev --auth-mode login
Upload a day of orders
az storage fs file upload --account-name stlakedev --auth-mode login --file-system lake --path bronze/orders/dt=2026-03-15/orders.csv --source ./orders.csv
Medallion paths
abfss://lake@stlakedev.dfs.core.windows.net/bronze/orders/dt=2026-03-15/
abfss://lake@stlakedev.dfs.core.windows.net/silver/orders/
abfss://lake@stlakedev.dfs.core.windows.net/gold/daily_revenue/
# keep raw files forever — reprocessing starts from bronze
Azure Data Factory
Pipeline shape
# parameter: dt (string)
# 1 Copy — SQL source to Parquet
#    sink path: bronze/orders/dt=@{pipeline().parameters.dt}
# 2 DatabricksNotebook — silver_orders
#    dependsOn LandBronze, status Succeeded
#    baseParameters dt = @pipeline().parameters.dt
Create and run
az datafactory pipeline create --resource-group rg-lake-dev --factory-name adf-lake-dev --name nightly-orders --pipeline pipeline.json
az datafactory pipeline create-run --resource-group rg-lake-dev --factory-name adf-lake-dev --name nightly-orders --parameters '{"dt":"2026-03-15"}'
Check the run
az datafactory pipeline-run show --resource-group rg-lake-dev --factory-name adf-lake-dev --run-id 00000000-0000-0000-0000-000000000000
Daily tumbling window
{
  "name": "daily-orders",
  "properties": {
    "type": "TumblingWindowTrigger",
    "runtimeState": "Started",
    "typeProperties": {
      "frequency": "Hour",
      "interval": 24,
      "startTime": "2026-03-01T00:00:00Z",
      "maxConcurrency": 1
    },
    "pipeline": {
      "pipelineReference": {"referenceName": "nightly-orders", "type": "PipelineReference"},
      "parameters": {"dt": "@formatDateTime(trigger().outputs.windowStartTime, 'yyyy-MM-dd')"}
    }
  }
}
Azure Databricks
Read bronze
df = spark.read.parquet(
    "abfss://lake@stlakedev.dfs.core.windows.net/bronze/orders/")
df = df.filter("dt = '2026-03-15'")
Overwrite one partition
(df.write.format("delta")
  .mode("overwrite")
  .option("replaceWhere", "dt = '2026-03-15'")
  .partitionBy("dt")
  .save("abfss://lake@stlakedev.dfs.core.windows.net/silver/orders"))
MERGE into silver
spark.sql("""
MERGE INTO silver.orders t
USING updates s
ON t.order_id = s.order_id
WHEN MATCHED THEN UPDATE SET *
WHEN NOT MATCHED THEN INSERT *
""")
Secret from Key Vaultnever print the value
password = dbutils.secrets.get(scope="kv-lake-dev", key="sql-password")
# the scope is backed by Key Vault; the notebook has no connection string
Azure Synapse
Serverless OPENROWSETquery the lake in place
SELECT dt, SUM(amount) AS revenue
FROM OPENROWSET(
  BULK 'https://stlakedev.dfs.core.windows.net/lake/silver/orders/dt=2026-03-15/**',
  FORMAT = 'PARQUET'
) AS rows
GROUP BY dt;
Dedicated pool table
CREATE TABLE gold.daily_revenue (
  dt date NOT NULL,
  revenue decimal(12, 2)
)
WITH (
  DISTRIBUTION = HASH(dt),
  CLUSTERED COLUMNSTORE INDEX
);
COPY with managed identity
COPY INTO gold.daily_revenue
FROM 'https://stlakedev.dfs.core.windows.net/lake/gold/daily_revenue/*.parquet'
WITH (
  FILE_TYPE = 'PARQUET',
  CREDENTIAL = (IDENTITY = 'Managed Identity')
);
Serverless view
CREATE VIEW gold.v_daily_revenue AS
SELECT dt, SUM(amount) AS revenue
FROM OPENROWSET(
  BULK 'https://stlakedev.dfs.core.windows.net/lake/silver/orders/**',
  FORMAT = 'PARQUET'
) WITH (amount decimal(12, 2), dt date) AS rows
GROUP BY dt;
Event Hubs
Namespace and hubStandard or higher for Capture
az eventhubs namespace create --resource-group rg-lake-dev --name ehlakedev --location eastus --sku Standard
az eventhubs eventhub create --resource-group rg-lake-dev --namespace-name ehlakedev --name orders --partition-count 4 --retention-time 1
Consumer group
az eventhubs eventhub consumer-group create --resource-group rg-lake-dev --namespace-name ehlakedev --eventhub-name orders --name spark
# each reader (Spark, a second app) gets its own group
Capture to the lake
az eventhubs eventhub update --resource-group rg-lake-dev --namespace-name ehlakedev --name orders --enable-capture true --capture-interval 300 --destination-name EventHubArchive.AzureBlockBlob --storage-account stlakedev --blob-container lake --archive-name-format "{Namespace}/{EventHub}/{PartitionId}/{Year}/{Month}/{Day}/{Hour}/{Minute}/{Second}"
Kafka endpoint
# bootstrap: ehlakedev.servicebus.windows.net:9093
# topic = hub name "orders"
# group.id = spark
# authenticate with the pipeline identity, not a connection string in git
Key Vault
Vault with RBACnot access policies
az keyvault create --resource-group rg-lake-dev --name kv-lake-dev --location eastus --enable-rbac-authorization true
az keyvault secret set --vault-name kv-lake-dev --name sql-password --value "replace-me"
Secrets User on the identity
az role assignment create --assignee-object-id <principal-id> --assignee-principal-type ServicePrincipal --role "Key Vault Secrets User" --scope $(az keyvault show --name kv-lake-dev --query id -o tsv)
ADF secret reference
"password": {
  "type": "AzureKeyVaultSecret",
  "store": {
    "referenceName": "kv-lake-dev",
    "type": "LinkedServiceReference"
  },
  "secretName": "sql-password"
}
One vault per environment
# kv-lake-dev and kv-lake-prod are different vaults
# a dev identity has no role on the prod vault
az keyvault secret list --vault-name kv-lake-dev --query "[].name" -o tsv
Microsoft Fabric
OneLake path
abfss://LakeDev@onelake.dfs.fabric.microsoft.com/Orders.Lakehouse/Files/bronze/orders/
abfss://LakeDev@onelake.dfs.fabric.microsoft.com/Orders.Lakehouse/Tables/silver_orders/
Read it from a notebook
df = spark.read.parquet(
    "abfss://LakeDev@onelake.dfs.fabric.microsoft.com/Orders.Lakehouse/Files/bronze/orders")
SQL endpointsame tables, no copy
SELECT dt, SUM(amount) AS revenue
FROM orders
WHERE dt = '2026-03-15'
GROUP BY dt;
Shortcut back to ADLS
# Lakehouse shortcut — files stay in ADLS Gen2
# source: https://stlakedev.dfs.core.windows.net/lake/bronze
# target: Files/bronze
# dev and prod workspaces shortcut to different storage accounts

Projects to build

The fastest way to make this stick. Build one, then put it in your portfolio.

Beginner

Bronze day on ADLS

Create a hierarchical storage account, upload one day of orders under bronze/orders/dt=…, and count the rows with a Synapse serverless OPENROWSET query. Grant yourself Storage Blob Data Contributor. Do not use an account key.

ADLS Gen2RBACSynapse serverless SQL
Beginner

Secret-free notebook

Store a SQL password in a Key Vault that uses RBAC. A Databricks notebook reads it through a Key Vault-backed secret scope and never displays the value.

Key VaultRBACDatabricks secrets
Intermediate

One ADF slice

A Data Factory pipeline, run for a single date, copies a source table into ADLS bronze as Parquet, then runs a Databricks notebook that MERGEs that date into a Delta silver table.

Data FactoryADLS Gen2DatabricksDelta Lake
Intermediate

Captured orders

Stand up a Standard Event Hubs namespace and an orders hub with Capture into the lake. Read one captured hour from Databricks and write a cleaned Delta table partitioned by date.

Event HubsCaptureADLS Gen2Databricks
Advanced

Fabric on the same files

Shortcut the existing ADLS bronze and silver into a Fabric lakehouse and query them from the SQL endpoint. Dev and prod are separate workspaces with different RBAC, and neither workspace stores a second copy of the files.

Microsoft FabricOneLake shortcutsADLS Gen2RBAC