Data Science Cheat Sheet

Turn a decision into a question,
then into evidence someone can act on.
The lifecycle in the live workflow lesson:
  frame, collect, clean and explore,
  analyse or model, validate, communicate.

data analyst      what already happened
data scientist     clean, check, sometimes predict
ML engineer        ship and run the model
data engineer      the pipes that make rows exist
The live lesson is the long version of this split.

Live now
  what data science is
  the workflow
  how the role differs from the other three
Ahead
  a browser Python playground and StreamPulse
  cleaning, exploration, charts, then a first model
This page stays at that scope.

the decision it will change
the unit: a user, a day, a title
the metric
the time window
"Analyze the data" is not a plan.

"Which plan has the higher cancel rate
 this quarter, in this extract?"
Names the metric, the group, and the window.
It does not pretend to know why.

What result would change the decision?
What will you refuse to claim?
Which table could even hold the answer?
If no table can, the question is still a wish.

users            who signed up
titles           the catalog
subscriptions    plan, price, status, cancel reason
watch_history    minutes and whether they finished
ratings          a score and a review
One fictional service, reused as the track grows.

A row is one observation.
A column is one attribute.
user_id and title_id are how tables meet.
If you cannot name the grain, do not average yet.

user = {
    "user_id": 1,
    "country": "IN",
    "plan": "basic",
    "signup_date": "2024-01-03",
}
# later lessons move this into a table tool
# sketch — that library is not required here

def missing(value):
    return value is None or value == ""

def complete(rows, keys):
    kept = [
        row for row in rows
        if all(not missing(row.get(key)) for key in keys)
    ]
    return kept, len(rows) - len(kept)

def as_number(value):
    try:
        return float(value)
    except (TypeError, ValueError):
        return None
# a blank that became 0 will move every mean

Duplicates: same key twice. Keep one, and say which.
Outliers: far from the rest. A bug, or the story.
Strings: strip space and agree on case.
Write down every drop. A silent filter is a new dataset.

def mean(xs):
    return sum(xs) / len(xs)

def median(xs):
    ordered = sorted(xs)
    mid = len(ordered) // 2
    if len(ordered) % 2:
        return ordered[mid]
    return (ordered[mid - 1] + ordered[mid]) / 2
# skew pulls the mean and leaves the median

def counts(rows, key):
    tally = {}
    for row in rows:
        value = row.get(key)
        tally[value] = tally.get(value, 0) + 1
    return tally

Two columns can move together because
  one influences the other,
  both share a cause,
  or the collection process ties them.
Correlation is a clue. It is not the decision.

how many rows, and how many missing
the grain
one distribution: the shape, not only the average
one comparison across a group
Stop when the next chart would not change the question.

def majority(labels):
    return max(set(labels), key=labels.count)

# a model maps features to a guess
# a baseline — majority class, or the mean — comes first
# if you cannot beat it, do not add complexity

def holdout(rows, frac=0.8):
    cut = int(len(rows) * frac)
    return rows[:cut], rows[cut:]
# shuffle, or cut by time if the decision is about later
# test rows do not teach the rule

linear regression    a weighted sum for a number
logistic regression  a weighted sum, then a probability
train / test split   so the score is on unseen rows
overfitting          memorising this extract
# sketch — no modelling library is imported here

the question
the number, with how many rows
the caveat
A chart is optional. The sentence is not.

# shape of a result, not a finding from the dataset
# "4 of 10 basic rows in this extract are canceled.
#  The extract is small, so this is a hint."
"Basic looks worse" is not a result.

bar       compare categories
line      something over time
scatter   two numbers, one point each
The title states the claim, not the chart type.
Chart libraries show up in later lessons, not this page.

pipelines, warehouses, freshness   data engineering
dashboards, SQL, weekly KPIs       analytics
questions, checks, a careful model data science
deploy, latency, monitoring        ML engineering
The job title on the post is the weak signal.

"Why did cancels rise?"
  analyst        slice what already happened
  scientist      test a reason, and attach a caveat
  ML engineer    score who might cancel next, in production
  data engineer  make sure yesterday's events arrived

The live "what is data science" lesson
walks a working day, not a package list.
Excel still fits a small tidy table.
Python is how a cleaning step gets repeated
and shown. The playground lesson is still ahead.

Is "cancel" the same column in every table?
Did a join multiply rows?
Is the window the one the decision uses?
Would someone act — and on which number?

def rate(flags):
    if not flags:
        return None
    return sum(1 for flag in flags if flag) / len(flags)
# print the numerator and the denominator beside it

A feature that only exists after the outcome
makes a brilliant score that fails next month.
When the decision is about the future,
hold out a later period, not a random shuffle
of the same week.