Dot Product and Similarity

In ML, a vector is a fixed-length list where every position has a consistent meaning. Position 0 might mean "bass intensity" for every song in the database. Position 1 might mean "tempo" for every song. If you swap the positions, comparison breaks completely.

This is called a feature vector. In ML, you will represent customers, songs, words, images, and loan applications all as feature vectors. The dot product measures similarity between any two of them.

Think of it like a compatibility score between two people on a dating app. Both people rate 5 things on a scale of 1–10: outdoors, movies, cooking, travel, fitness. Person A: [8, 3, 7, 9, 6]. Person B: [7, 2, 8, 10, 5].

To find compatibility: multiply each matching pair (8×7, 3×2, 7×8, 9×10, 6×5) and add them up (56 + 6 + 56 + 90 + 30 = 238). A high number means they care about the same things. This IS the dot product.

The dot product rewards agreement. When both vectors are high at the same position, that position contributes a large product. When one is high and the other is low, the contribution is small.

import numpy as np

# ── Manual dot product — every step visible ───────────────────────────
hiphop   = [0.9, 0.7, 0.8, 0.1, 0.5]   # bass, tempo, energy, acoustic, vocal
similar  = [0.8, 0.6, 0.9, 0.2, 0.4]
classical= [0.1, 0.2, 0.1, 0.9, 0.8]

def dot_product_manual(a, b):
    """
    Step 1: multiply each matching position
    Step 2: sum all the products
    That is the entire dot product operation.
    """
    assert len(a) == len(b), "Vectors must be same length"
    products = [a[i] * b[i] for i in range(len(a))]
    print(f"  Products at each position: {[round(p, 3) for p in products]}")
    total = sum(products)
    return total

print("Dot product — hip-hop vs similar:")
score_similar   = dot_product_manual(hiphop, similar)
print(f"  Result: {score_similar:.3f}\n")

print("Dot product — hip-hop vs classical:")
score_classical = dot_product_manual(hiphop, classical)
print(f"  Result: {score_classical:.3f}\n")

print(f"Conclusion: similar score ({score_similar:.3f}) > classical score ({score_classical:.3f})")
print("→ Spotify recommends 'similar track' over 'classical piano'")

# ── NumPy way — what you use in production ─────────────────────────────
a = np.array(hiphop)
b = np.array(similar)
c = np.array(classical)

print(f"\nNumPy dot product:")
print(f"  np.dot(hiphop, similar):   {np.dot(a, b):.3f}")
print(f"  a @ b (same thing):        {a @ b:.3f}")  # @ is the dot product operator
print(f"  np.dot(hiphop, classical): {np.dot(a, c):.3f}")

Think of two arrows. Arrow A is 10cm long pointing northeast. Arrow B is 2cm long also pointing northeast. They point in exactly the same direction — they represent the same preferences. But if you measure their dot product, Arrow A wins simply because it is longer.

Cosine similarity fixes this by dividing the dot product by the length of both vectors — it strips away magnitude and measures only direction. Two arrows pointing northeast give cosine similarity = 1.0 regardless of how long they are.

||a|| is the "length" (norm) of vector a: the square root of the sum of all values squared. It is always a positive number.

import numpy as np

# ── The magnitude (norm) problem ──────────────────────────────────────
# Two users with same taste but different activity levels
user_active  = np.array([0.9, 0.8, 0.7, 0.1, 0.2]) * 10   # active user — bigger numbers
user_casual  = np.array([0.9, 0.8, 0.7, 0.1, 0.2])          # casual user — same direction!
restaurant   = np.array([0.8, 0.9, 0.6, 0.2, 0.3])

print("Raw dot product (confused by magnitude):")
print(f"  active user  · restaurant: {np.dot(user_active, restaurant):.2f}")
print(f"  casual user  · restaurant: {np.dot(user_casual, restaurant):.2f}")
print("  Active user scores 10× higher — but they have IDENTICAL taste!")

# ── Vector norm — what is the 'length' of a vector? ───────────────────
def vector_norm(v):
    """
    The length (magnitude) of a vector.
    = sqrt(v[0]² + v[1]² + v[2]² + ...)
    Called the L2 norm or Euclidean norm.
    """
    return np.sqrt(np.sum(v ** 2))

print(f"\nVector norms:")
print(f"  ||user_active||: {vector_norm(user_active):.4f}")
print(f"  ||user_casual||: {vector_norm(user_casual):.4f}")
print(f"  Ratio: {vector_norm(user_active) / vector_norm(user_casual):.1f}× — same direction, 10× length")

# ── Cosine similarity — fixes the magnitude problem ────────────────────
def cosine_similarity(a, b):
    """
    cosine_similarity = dot_product(a, b) / (norm(a) * norm(b))

    Step 1: compute the raw dot product
    Step 2: divide by the product of both norms
    Result: always between -1 and +1, magnitude removed
    """
    dot    = np.dot(a, b)
    norm_a = np.linalg.norm(a)   # same as vector_norm(a)
    norm_b = np.linalg.norm(b)
    return dot / (norm_a * norm_b)

print("\nCosine similarity (magnitude-independent):")
print(f"  active user  · restaurant: {cosine_similarity(user_active, restaurant):.4f}")
print(f"  casual user  · restaurant: {cosine_similarity(user_casual, restaurant):.4f}")
print("  Identical! Cosine similarity sees they have the same taste.")

# ── Real example: song similarity ─────────────────────────────────────
hiphop    = np.array([0.9, 0.7, 0.8, 0.1, 0.5])
similar   = np.array([0.8, 0.6, 0.9, 0.2, 0.4])
classical = np.array([0.1, 0.2, 0.1, 0.9, 0.8])

print("\nCosine similarity — song comparison:")
print(f"  hip-hop vs similar:   {cosine_similarity(hiphop, similar):.4f}  ← high, similar genre")
print(f"  hip-hop vs classical: {cosine_similarity(hiphop, classical):.4f}  ← low, different genre")

# ── sklearn version — what you use in production ──────────────────────
from sklearn.metrics.pairwise import cosine_similarity as sk_cosine

# sklearn expects 2D arrays (matrices), not 1D vectors
X = np.array([hiphop, similar, classical])
similarity_matrix = sk_cosine(X)
labels = ['hip-hop', 'similar', 'classical']

print("\nFull similarity matrix (sklearn):")
print(f"{'':12}", end='')
for l in labels: print(f"{l:12}", end='')
print()
for i, row_label in enumerate(labels):
    print(f"{row_label:12}", end='')
    for j in range(len(labels)):
        print(f"{similarity_matrix[i,j]:.4f}      ", end='')
    print()

import numpy as np

# ── Dot product in a neural network layer ────────────────────────────
# One neuron: takes an input vector, has a weight vector, computes dot product
# Think of weights as "how much does this neuron care about each input feature"

def neuron_forward(inputs, weights, bias):
    """
    One neuron's computation:
    1. dot product of inputs and weights (how much each input matters)
    2. add bias (shift the output)
    3. activation function applied later (not shown here)
    """
    return np.dot(inputs, weights) + bias

# A delivery time prediction neuron
# It has learned: distance and prep time matter most, traffic matters some
inputs  = np.array([5.2, 8.0, 22.0])   # distance_km, traffic_score, prep_min
weights = np.array([0.4, 0.15, 0.3])   # learned weights
bias    = 8.6

raw_output = neuron_forward(inputs, weights, bias)
print(f"Neuron output (predicted delivery time): {raw_output:.1f} min")

# ── Attention in transformers (simplified preview) ─────────────────────
# Each word has a Query vector (what it's looking for)
# and a Key vector (what it offers to others)
# Attention score = Query · Key

# Sentence: "The quick brown fox"
# Simplified 3D embeddings for illustration
words = ['The', 'quick', 'brown', 'fox']
# Query vectors (what each word is looking for)
queries = np.array([
    [0.1, 0.9, 0.2],   # 'The' — looks for: noun
    [0.8, 0.3, 0.1],   # 'quick' — looks for: what it modifies
    [0.7, 0.4, 0.2],   # 'brown' — looks for: what it modifies
    [0.2, 0.8, 0.9],   # 'fox' — looks for: its adjectives
])
# Key vectors (what each word offers)
keys = np.array([
    [0.1, 0.2, 0.3],   # 'The' — offers: article
    [0.6, 0.5, 0.1],   # 'quick' — offers: adjective
    [0.7, 0.4, 0.1],   # 'brown' — offers: adjective
    [0.3, 0.9, 0.8],   # 'fox' — offers: noun
])

# How much does 'fox' attend to each word?
fox_query = queries[3]
attention_scores = [np.dot(fox_query, keys[i]) for i in range(4)]
print(f"\nAttention scores for 'fox':")
for word, score in zip(words, attention_scores):
    bar = '█' * int(score * 20)
    print(f"  fox → {word:<8}: {bar} {score:.3f}")
# Fox should attend most to 'quick' and 'brown' (its adjectives)

# ── Embedding similarity search ────────────────────────────────────────
# Find most similar products to a query in an embedding space
np.random.seed(42)
product_names = [
    'Nike Running Shoes', 'Adidas Sneakers', 'Formal Oxford Shoes',
    'Flip Flops', 'Sports Socks', 'Running Shorts',
]
# Simulate product embeddings (in real life these come from a trained model)
product_embeddings = np.random.randn(6, 8)

# Query: user just looked at "Nike Running Shoes"
query_embedding = product_embeddings[0]

# Compute cosine similarity to all products
from sklearn.metrics.pairwise import cosine_similarity
sims = cosine_similarity(
    query_embedding.reshape(1, -1),
    product_embeddings
)[0]

print("\nProducts most similar to 'Nike Running Shoes':")
ranked = sorted(zip(product_names, sims), key=lambda x: x[1], reverse=True)
for name, score in ranked:
    marker = ' ← query' if name == 'Nike Running Shoes' else ''
    print(f"  {name:<25}: {score:.4f}{marker}")

Two cities on a map: Mumbai (18.97°N, 72.83°E) and Pune (18.52°N, 73.85°E). The Euclidean distance between them as points on the map is roughly √((18.97−18.52)² + (72.83−73.85)²) — just like the Pythagorean theorem applied to their coordinate difference. This is Euclidean distance.

In ML, "distance" works the same way but in higher dimensions — instead of 2 coordinates you might have 512 coordinates (features). The formula is identical, just with more terms under the square root.

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity, euclidean_distances

# ── Euclidean distance from scratch ───────────────────────────────────
def euclidean_distance(a, b):
    """
    Straight-line distance between two points in n-dimensional space.
    = sqrt( sum of (aᵢ - bᵢ)² for all i )
    Pythagorean theorem generalised to n dimensions.
    """
    differences = a - b
    squared     = differences ** 2
    summed      = np.sum(squared)
    return np.sqrt(summed)

# Customer feature vectors (after scaling to 0-1 range)
# [avg_order_value, orders_per_month, avg_rating, distance_preference]
customer_a = np.array([0.8, 0.9, 0.7, 0.3])   # high-value frequent customer
customer_b = np.array([0.7, 0.8, 0.8, 0.4])   # similar to A
customer_c = np.array([0.2, 0.1, 0.6, 0.9])   # very different — low value, rare

print("Euclidean distances:")
print(f"  A to B: {euclidean_distance(customer_a, customer_b):.4f}  ← small, similar customers")
print(f"  A to C: {euclidean_distance(customer_a, customer_c):.4f}  ← large, different customers")

print("\nCosine similarities (same customers):")
print(f"  A to B: {cosine_similarity(customer_a.reshape(1,-1), customer_b.reshape(1,-1))[0,0]:.4f}")
print(f"  A to C: {cosine_similarity(customer_a.reshape(1,-1), customer_c.reshape(1,-1))[0,0]:.4f}")

# ── When they disagree — an important case ─────────────────────────────
# Two customers with exactly the same PATTERN but different AMOUNTS
casual_customer = np.array([0.2, 0.2, 0.6, 0.3])       # same preferences, less active
active_customer = np.array([0.8, 0.8, 0.6, 0.3]) * 4   # 4× more active, same pattern

print("\nCasual vs active customer (same pattern, different magnitude):")
print(f"  Euclidean distance:  {euclidean_distance(casual_customer, active_customer):.4f}")
print(f"  Cosine similarity:   {cosine_similarity(casual_customer.reshape(1,-1), active_customer.reshape(1,-1))[0,0]:.4f}")
print("  Euclidean: they look far apart (because active has larger values)")
print("  Cosine:    they look identical (because they point the same direction)")
print("  → For segmentation by BEHAVIOUR: use cosine")
print("  → For segmentation by ACTIVITY LEVEL: use euclidean")

import numpy as np
from sklearn.preprocessing import normalize
from sklearn.metrics.pairwise import cosine_similarity

# Flipkart electronics — simplified feature vectors
# Features: [price_normalised, brand_premium, battery_life, screen_size,
#            camera_quality, performance_score, weight_light]
products = {
    'iPhone 15':         [0.95, 1.0,  0.7,  0.6, 1.0,  0.95, 0.8],
    'Samsung Galaxy S24':[0.85, 0.9,  0.75, 0.65,0.95, 0.9,  0.75],
    'OnePlus 12':        [0.65, 0.7,  0.8,  0.7, 0.85, 0.88, 0.7],
    'Redmi Note 13':     [0.25, 0.4,  0.85, 0.72,0.65, 0.6,  0.65],
    'iPhone SE':         [0.55, 1.0,  0.5,  0.35,0.75, 0.8,  0.9],
    'iPad Air':          [0.75, 1.0,  0.9,  1.0, 0.85, 0.9,  0.5],
    'Samsung Tab S9':    [0.7,  0.85, 0.85, 0.95,0.8,  0.85, 0.45],
    'Boat Earbuds':      [0.1,  0.3,  0.9,  0.0, 0.0,  0.4,  1.0],
}

names       = list(products.keys())
embeddings  = np.array(list(products.values()))

# Normalise so cosine similarity is just a dot product (unit vectors)
# This is the standard approach for any similarity search
embeddings_norm = normalize(embeddings, norm='l2')

# Build the full similarity matrix
sim_matrix = cosine_similarity(embeddings_norm)

def get_similar_products(product_name, top_k=3):
    """Given a product name, return the top_k most similar products."""
    idx   = names.index(product_name)
    sims  = sim_matrix[idx]
    # Sort by similarity, exclude the product itself
    ranked = sorted(
        [(names[i], sims[i]) for i in range(len(names)) if i != idx],
        key=lambda x: x[1], reverse=True,
    )
    return ranked[:top_k]

print("Product similarity recommendations:\n")
for product in ['iPhone 15', 'Redmi Note 13', 'iPad Air', 'Boat Earbuds']:
    print(f"  Customer viewing: {product}")
    for rec, score in get_similar_products(product):
        bar = '█' * int(score * 20)
        print(f"    → {rec:<22}: {bar} {score:.3f}")
    print()

# ── Production note: in real Flipkart, embeddings come from a trained
# neural network (not hand-crafted features). The similarity computation
# is identical — just replace the hand-crafted vectors with model embeddings.
print("In production:")
print("  Step 1: train a model to generate product embeddings")
print("  Step 2: store all embeddings in a vector database (FAISS, Pinecone)")
print("  Step 3: at query time: embed the viewed product, find nearest neighbours")
print("  Step 4: return top-k most similar products")