ML System Design — End to End

An architect designing a building does not start by choosing the colour of the walls. They start with: who lives here, how many people, what activities happen inside, what is the budget, what are the structural constraints of the land. The colour comes last. ML system design is the same — the model choice (colour of the walls) comes after you understand the data availability, latency requirements, and scale constraints. Most candidates start with model selection and never get to the questions that actually determine system feasibility.

In ML system design interviews, an interviewer would rather see you ask the right clarifying questions than immediately jump to "I would use a Transformer." The right questions demonstrate systems thinking. The immediate model answer demonstrates pattern matching.

# ── Fraud detection system design — capacity and tradeoff analysis ───

# Question 1: Problem framing
print("PROBLEM FRAMING")
print("=" * 55)
print("""
ML task:     Binary classification — fraud vs legitimate
             Output: fraud probability score (0-1), not binary
             Thresholds set by risk team based on business cost

Business metric:
  Primary:   Transaction fraud rate (TFR) — fraud_amount / total_amount
  Secondary: False positive rate (FPR) — declined_legitimate / all_legitimate

Why score not binary:
  Different thresholds for different transaction types:
  - Low value UPI (< Rs 500): high threshold (0.95) — FP very costly
  - High value bank transfer (> Rs 1L): low threshold (0.30) — FN very costly
  - International: medium threshold (0.60)
""")

# Question 6: Scale (do this early for fraud — it's the key constraint)
print("SCALE ANALYSIS")
print("=" * 55)

transactions_per_day = 10_000_000        # 10M transactions/day
peak_tps             = transactions_per_day / 86400 * 10   # 10× peak = ~1,157 TPS
fraud_rate           = 0.001             # 0.1% = 10,000 fraud transactions/day

print(f"Transaction volume:  {transactions_per_day:,}/day = {transactions_per_day/86400:.0f} avg TPS")
print(f"Peak TPS:            {peak_tps:.0f} TPS  (10× peak factor)")
print(f"Expected fraud:      {int(transactions_per_day * fraud_rate):,} transactions/day")
print(f"Latency budget:      < 50ms total  (before payment clears)")
print(f"Model budget:        < 10ms  (50ms - feature fetch - network)")

# Capacity for 1,157 TPS at 10ms each:
import math
model_time_ms    = 10
replicas_needed  = math.ceil(peak_tps * model_time_ms / 1000)
print(f"
Replicas needed:     {replicas_needed} pods × (1000ms/10ms) = handles {1000//model_time_ms} RPS each")

# Question 3: Features — three tiers by computation cost
print("
FEATURE TIERS")
print("=" * 55)
feature_tiers = {
    'Tier 1 — Request (0ms)': [
        'transaction_amount', 'merchant_category', 'payment_method',
        'device_fingerprint', 'ip_address', 'billing_zip',
        'hour_of_day', 'is_international',
    ],
    'Tier 2 — Feature Store (1-3ms Redis)': [
        'user_7d_transaction_count', 'user_7d_total_spend',
        'user_30d_distinct_merchants', 'user_30d_avg_amount',
        'merchant_fraud_rate_30d', 'ip_country_mismatch_flag',
        'velocity_1h_amount', 'device_new_flag',
    ],
    'Tier 3 — Computed (5ms, parallel)': [
        'transaction_vs_user_avg_ratio',   # txn / user 30d avg
        'amount_round_number_flag',         # Rs 10000.00 exact
        'time_since_last_transaction_s',
        'distance_from_last_merchant_km',
    ],
}
for tier, features in feature_tiers.items():
    print(f"
  {tier}:")
    for f in features:
        print(f"    → {f}")

# Question 4: Model — why an ensemble
print("
MODEL SELECTION")
print("=" * 55)
print("""
Primary:  LightGBM score (tabular features, fast, interpretable)
          Trained on 6-month rolling window
          Feature importance for regulatory compliance (RBI audit)

Secondary: Rule engine running in parallel (100% recall for known patterns)
           Hard rules: velocity limits, blacklisted IPs, blocked BINs
           Soft rules: unusual merchant for user, odd hours

Output: max(lgbm_score, rule_score) → final risk score

Why not deep learning:
  - Regulatory: RBI requires explainability for declined transactions
  - Latency: transformer inference > 50ms for tabular data
  - Data: 0.1% fraud rate with 10M daily → only 10k fraud labels/day
    LightGBM is far more data-efficient than deep learning for this

Class imbalance handling:
  - scale_pos_weight=999 in LightGBM (ratio of negatives to positives)
  - Focal loss as alternative: penalises easy negatives more
  - Evaluation: precision-recall AUC not ROC-AUC (imbalanced classes)
""")

# Question 7: Monitoring — adversarial drift is the hardest problem
print("MONITORING — ADVERSARIAL DRIFT")
print("=" * 55)
print("""
Normal drift: feature distributions shift gradually (monthly retraining)
Adversarial drift: fraudsters adapt to the model within DAYS

Detection:
  - Monitor fraud rate per merchant category hourly
  - Monitor false positive rate daily (declined legit transactions)
  - Monitor feature distribution of FRAUD transactions (not all)
    → New fraud pattern appears as a cluster in a previously clean region

Response:
  - New fraud pattern detected → add hard rule IMMEDIATELY (same day)
  - Retrain model weekly with new fraud samples
  - Deploy rule before model update — rules are instant, models take days

Feedback loop:
  - Chargebacks confirm fraud (7-30 day delay)
  - Merchant disputes confirm false positives (same day)
  - Manual review team labels 2% of flagged transactions same-day
    → Creates fast label feedback for concept drift detection
""")

# ── Recommendation system capacity analysis ───────────────────────────
print("MEESHO RECOMMENDATION SCALE ANALYSIS")
print("=" * 55)

# User and product scale
dau           = 15_000_000   # 15M daily active users
products      = 50_000_000   # 50M products in catalogue
sessions_per_user = 3        # avg 3 sessions per day per DAU
recs_per_session = 4         # recommendation requests per session
total_rps = dau * sessions_per_user * recs_per_session / 86400

print(f"DAU:                    {dau/1e6:.0f}M")
print(f"Product catalogue:      {products/1e6:.0f}M")
print(f"Recommendation RPS:     {total_rps:.0f} avg  (~{total_rps*5:.0f} peak)")

# Why two-stage is necessary
print("
WHY TWO-STAGE IS NECESSARY:")
ranking_model_ms  = 5    # ms per (user, product) pair
full_catalogue_ms = ranking_model_ms * products / 1000
print(f"  Ranking all {products/1e6:.0f}M products: {full_catalogue_ms/1000:.0f} seconds — impossible")
print(f"  Ranking 500 candidates:              {ranking_model_ms * 500:.0f}ms — feasible")

# Offline training cadence
print("
TRAINING CADENCE:")
print("""
  Retrieval models (ALS, CLIP embeddings):  Weekly
    → Catalogue changes slowly, embeddings expensive to compute

  Ranking model:  Daily
    → Click patterns change quickly, model relatively cheap to retrain

  Popularity/trending features:  Hourly
    → Trending products change fast, must be fresh

  Cold start (new product):  Real-time
    → New product uploaded → compute CLIP embedding → add to index immediately
    → Use content-based retrieval until enough interactions for collaborative
""")

# Evaluation metrics — what actually matters
print("EVALUATION METRICS:")
metrics = [
    ('CTR (Click-Through Rate)',   'Online',  '% users who click a recommendation'),
    ('Conversion rate',            'Online',  '% recommendations that lead to purchase'),
    ('Diversity (ILD)',            'Offline', 'Intra-list diversity — are recommendations varied?'),
    ('Coverage',                   'Offline', '% of catalogue that appears in recommendations'),
    ('Novelty',                    'Offline', '% of recommendations user has never seen'),
    ('NDCG@10',                    'Offline', 'Normalised Discounted Cumulative Gain at position 10'),
]
for metric, where, desc in metrics:
    print(f"  {metric:<28}: {where:<8}  {desc}")

# ── What interviewers score — the actual rubric ───────────────────────

SCORING_RUBRIC = {
    'Problem framing (10%)': [
        'Asks clarifying questions before designing',
        'Correctly identifies ML task type (regression/classification/ranking)',
        'Defines business metric separate from ML metric',
        'States success criteria quantitatively',
    ],
    'Data and features (25%)': [
        'Identifies relevant data sources including non-obvious ones',
        'Addresses data quality and bias',
        'Addresses cold start problem for new entities',
        'Explains feature computation (batch vs real-time)',
        'Mentions point-in-time correctness for training',
    ],
    'Model selection (20%)': [
        'Proposes a simple baseline before complex model',
        'Justifies model choice with explicit tradeoffs',
        'Addresses class imbalance if present',
        'Mentions explainability requirements if regulatory context',
    ],
    'System design (25%)': [
        'Correctly determines online vs batch serving',
        'Mentions feature store for pre-computed features',
        'Provides latency breakdown (feature fetch + model + network)',
        'Capacity estimates with numbers',
        'Fallback when model unavailable',
    ],
    'Monitoring (15%)': [
        'Identifies what drifts in this specific system',
        'Proposes concrete monitoring metrics',
        'States retraining trigger condition',
        'Addresses feedback loop and label delay',
    ],
    'Communication (5%)': [
        'Draws architecture diagram',
        'States assumptions explicitly',
        'Handles interviewer's probing questions without getting flustered',
    ],
}

for category, criteria in SCORING_RUBRIC.items():
    print(f"
{category}:")
    for c in criteria:
        print(f"  ✓ {c}")

# ── Common mistakes that kill interview scores ────────────────────────
print("

COMMON MISTAKES:")
mistakes = [
    ('Jumping to model selection',    'First 5 minutes spent on LightGBM vs XGBoost. Interviewer wants system design.'),
    ('Forgetting cold start',         '"New restaurant has no history" — always asked as a follow-up. Address proactively.'),
    ('No fallback plan',              '"What if the model is down?" — must have a rule-based or static fallback.'),
    ('Missing label strategy',        '"How do you get training labels?" — must be answered for every system.'),
    ('Batch when online needed',      '"Pre-compute all (user, merchant) pairs" — combinatorial explosion.'),
    ('Ignoring data leakage',         '"Use future features in training" — always mention point-in-time correctness.'),
    ('No numbers',                    '"Large scale, many requests" — always give orders of magnitude.'),
]
print(f"  {'Mistake':<30} {'Why it hurts'}")
print("  " + "─" * 70)
for mistake, why in mistakes:
    print(f"  {mistake:<30} {why}")