What Are Embeddings, and What Can You Actually Do With Them?

Rae is building a customer-support chatbot for her company’s product. Her keyword search keeps missing obvious matches — a customer searches “puppy” and the bot skips the help article titled “young dogs.” How does Netflix know that liking space movies means you’ll probably enjoy a moon documentary? Or that Googling “puppy care” means you also want results for “young dogs”?

It looks like magic. It’s math — specifically, a concept called embeddings. So this guide starts with the frustration of rigid keyword logic and moves toward the flexible, “human-like” understanding that modern AI relies on.

1. The Library Problem: Why Computers Can’t Read

Think of a computer as a very fast, very obedient, but very literal librarian. Ask for a book about “puppies” and they’ll check every cover in the building. But a book titled The Care of Young Dogs? They walk right past it.

Why? To a computer, “puppy” and “dog” are just different strings of characters. The letters don’t line up in the same order, so they aren’t the same thing.

Here’s what happens when standard Python logic tries to compare meanings:

word_1 = "puppy"
word_2 = "young dog"

# Does the computer think these are the same?
print(f"Is '{word_1}' exactly the same as '{word_2}'?")
print(word_1 == word_2)

# What if we search for one inside the other?
print(f"Is '{word_1}' found inside '{word_2}'?")
print(word_1 in word_2)

This block attempts standard string comparison. `word_1 == word_2` checks for an exact match, which fails because the characters are different. `word_1 in word_2` checks if "puppy" is a substring of "young dog," which also fails. This demonstrates the fundamental limitation of keyword-based logic: it operates on characters, not concepts.

What this actually means: Both checks return False. You and I know these terms mean almost the same thing. The computer sees zero overlap. This is the “Search Problem” — keyword matching misses context and synonyms entirely.

2. Think of it as a Map: The Intuition of Vector Space

The fix: stop treating words as strings of letters, and start treating them as points on a map.

Picture a big map. We set the North-South axis to “Animal vs. Object” and the East-West axis to “Size.”

• A Golden Retriever sits at (10, 10) — very animal, very large.
• A Chihuahua sits at (10, 1) — very animal, very small.
• A Toaster sits at (-10, 2) — very object, very small.

On this map, the Golden Retriever and the Chihuahua are close neighbors. The toaster is far away.

An embedding is just that list of coordinates. In practice, we don’t stop at two dimensions (North and Size) — we use hundreds. Each number captures a sliver of meaning, like “is it fluffy?” or “is it electronic?”

Here’s a conceptual look at what those coordinates might look like for different items:

# Conceptual coordinates: [Animal-ness, Sweetness, Tech-focus]
embeddings = {
    "apple_fruit": [0.1, 0.9, 0.0],
    "banana":      [0.1, 0.8, 0.0],
    "apple_iphone":[0.0, 0.1, 0.9],
}

def simple_distance(a, b):
    # A simple way to see how 'far apart' two lists are
    return sum(abs(x - y) for x, y in zip(a, b))

print(f"Distance Fruit-Apple to Banana: {simple_distance(embeddings['apple_fruit'], embeddings['banana']):.2f}")
print(f"Distance Fruit-Apple to iPhone: {simple_distance(embeddings['apple_fruit'], embeddings['apple_iphone']):.2f}")

Here we define a dictionary mapping items to 3-dimensional coordinate lists. The `simple_distance` function calculates the Manhattan distance (L1 norm) between two points by zipping the lists together, taking the absolute difference of each pair, and summing them. This shows how numerical proximity represents conceptual similarity.

What this actually means: The distance between the two fruits is only 0.1, while the distance between the fruit and the phone is 1.8. The word “Apple” appears in both, but the coordinates put them in different worlds.

3. Let’s See What Happens: Creating Your First Embedding

You might be wondering: who decides what these coordinates are? Nobody’s asking you to manually score every word on “fluffiness.”

No. Pre-trained models have already read billions of sentences and learned these patterns for us. The sentence-transformers library is one of the most popular tools for this.

Let’s generate some real embeddings with a model called all-MiniLM-L6-v2. It’s small, fast, and surprisingly capable.

from sentence_transformers import SentenceTransformer

# Load the model
model = SentenceTransformer('all-MiniLM-L6-v2')

# Our sentences
sentences = [
    "The kitten is sleeping on the rug.",
    "A young cat is napping on a carpet.",
    "The stock market saw a sharp decline today."
]

# Turn sentences into numbers (embeddings)
embeddings = model.encode(sentences)

print(f"Shape of the embedding: {embeddings[0].shape}")
print(f"First 5 numbers of the first sentence: {embeddings[0][:5]}")

We import `SentenceTransformer` and load a specific pre-trained model. We pass a list of strings to `model.encode()`, which handles tokenization and neural network inference, returning a NumPy array (or list of arrays) where each row is the embedding for a sentence. We print the shape (length) of the first embedding and a slice of its first 5 values to show it's just a list of floats.

This is the hardest part: When you run this, each sentence becomes a list of 384 numbers. Don’t try to read them. You won’t find a specific “cat” number. Instead, the relationship between all 384 numbers creates the meaning. The computer has compressed the “vibe” of the sentence into a mathematical point.

4. The Magic Trick: Measuring Similarity

With these points in space, we can measure how closely they point in the same direction using Cosine Similarity.

Think of it as a ruler that gives you a score between -1 and 1.

• 1.0 means the sentences mean exactly the same thing.
• 0.0 (or a small number close to it, positive or negative) means they share no meaningful semantic overlap.
• -1.0 would mean exactly opposite meaning — in practice, sentence embeddings rarely get anywhere near that extreme.

Cosine similarity measures the cosine of the angle between two non-zero vectors. It is calculated as:

$\text{Cosine Similarity}(A, B) = \frac{A \cdot B}{\|A\| \|B\|}$

Plain English	Statistical symbol	Python equivalent
Dot product of vectors A and B	$A \cdot B$	np.dot(A, B)
Magnitude (length) of vector A	$\|A\|$	np.linalg.norm(A)
Cosine of the angle between A and B	$\cos(\theta)$	cosine_similarity([A], [B])[0][0]

Now, let’s check our sentences from the previous step:

from sklearn.metrics.pairwise import cosine_similarity

# Compare the kitten sentence to the others
sim_kitten_cat = cosine_similarity([embeddings[0]], [embeddings[1]])[0][0]
sim_kitten_stock = cosine_similarity([embeddings[0]], [embeddings[2]])[0][0]

print(f"Similarity (Kitten vs Cat): {sim_kitten_cat:.4f}")
print(f"Similarity (Kitten vs Stock Market): {sim_kitten_stock:.4f}")

We import `cosine_similarity` from scikit-learn. We pass our embedding arrays (wrapped in lists to make them 2D) for the kitten and cat sentences. The function returns a matrix, so we use `[0][0]` to extract the single scalar similarity score. We do the same for the kitten and stock market sentences, then print both formatted to 4 decimal places.

What this actually means: The similarity score for the kitten and the cat comes out to about 0.64. That’s high relative to the unrelated pair below, but don’t anchor on it as “64% similar” in some absolute sense — cosine scores from sentence-embedding models aren’t calibrated to that kind of intuition, and 0.64 is already a strong match for this particular model. That’s exactly why you compare scores against each other rather than against a fixed threshold like “0.9 means a match.” The stock market sentence scores around 0.03 against the kitten sentence — close to zero, which is what you’d expect from two sentences that share no topic.

5. What Can You Actually Do? (The ‘So What?’ Factor)

So, we can turn sentences into numbers and compare them. Why does this matter for your business or project? Here are three ways this shows up every day:

1. Semantic Search: Instead of matching keywords, you match concepts. Someone searches “How do I fix my laptop?” and you surface the help article titled “Troubleshooting Hardware Issues.”
2. Clustering: 10,000 customer reviews is too many to read by hand. Turn them into embeddings and group them. You might find 500 people complaining about “battery life” — all using different words.
3. Recommendation Engines: Like a specific article? The system finds other articles whose embeddings sit “nearby” in vector space.

Let’s build a tiny search engine to see this in action:

database = [
    "Python is a great programming language.",
    "I love eating spicy tacos.",
    "The weather is sunny in California.",
    "Coding in Java can be complex."
]
db_embeddings = model.encode(database)

query = "I enjoy hot Mexican food"
query_embedding = model.encode([query])

# Find the most similar sentence
scores = cosine_similarity(query_embedding, db_embeddings)[0]
print(f"All scores: {scores}")

best_idx = scores.argmax()
print(f"Query: {query}")
print(f"Best Match: {database[best_idx]} (Score: {scores[best_idx]:.4f})")

We define a `database` of strings and encode them all into embeddings. We then define a `query` string and encode it. `cosine_similarity` compares the single query embedding against all database embeddings, returning an array of scores — one per database sentence, in order. We print that full array before picking a winner, because the array itself is informative: it reads roughly `[0.05, 0.70, 0.23, -0.04]`. `scores.argmax()` finds the index of the highest score, which we use to retrieve the best matching sentence from the original `database` list.

What this actually means: The system picked the taco sentence with a high score (~0.70), even though the words “enjoy,” “hot,” “Mexican,” and “food” never appeared in the original list. It understood the intent. Notice something else in the printed score array, too: the “Coding in Java” sentence scores negative (around -0.04) against “I enjoy hot Mexican food.” Cosine similarity’s true range is -1 to 1, not 0 to 1 — most unrelated sentence pairs land just above zero, but some land just below it. A near-zero-but-not-exactly-zero score (in either direction) isn’t a bug; it’s just how these models represent “no real relationship,” since nothing forces unrelated meanings to sit at a mathematically perfect origin.

**Embeddings / Semantic Search vs. Traditional Keyword Search (TF-IDF/BM25)**

Traditional keyword search methods like TF-IDF or BM25 rely on exact term frequency. They are incredibly fast, require no GPU, and excel when exact terminology is critical (e.g., searching for error codes like “ERR_4021”, specific product names, or legal document IDs).

Embeddings, on the other hand, capture semantic meaning. They win when users use synonyms, paraphrase, or ask conceptual questions (“How do I reset my password?” vs “Password recovery steps”). However, embeddings require pre-processing everything with a model and can sometimes miss exact keyword matches that BM25 would catch instantly.

When to use which:

• BM25 / Keyword Search: Use for exact-match lookups, part numbers, legal clause searches, or when compute resources are extremely constrained.
• Embeddings: Use for chatbot intent matching, FAQ discovery, and any time a user might phrase a question differently than the document’s original wording.
• When hybrid earns its cost: Running both means maintaining two retrieval systems and a fusion step that combines their rankings — real engineering and tuning overhead, not a free win. It’s worth that cost when your query traffic is genuinely mixed (some users type exact part numbers or error codes, others type natural-language questions) and you have the team bandwidth to run and tune two retrievers. It’s usually not worth it when your query shape is uniform, your corpus is small enough that either method alone performs well, or you don’t have the capacity to maintain two systems — in those cases, pick the one that matches your dominant query type instead.

Summary

What we covered:

• Computers are literal; they need numbers to understand meaning.
• Embeddings are coordinates on a map where similar meanings live close together.
• Python libraries like sentence-transformers generate these numbers for us.
• Cosine similarity is the “ruler” we use to measure how close two ideas are — and its true range is -1 to 1, not 0 to 1.

Rae now sees that embeddings turn text into coordinates that capture meaning. But she still has thousands of help-doc chunks to search through. How does she actually store and query all these vectors fast? That’s the next problem she has to solve.

Check Your Understanding

Questions below move from simple recall up to open-ended design, roughly following Bloom’s Taxonomy.

Remember What is an embedding, using the article’s “coordinates on a map” definition?

Understand In your own words, explain why “puppy” and “young dog” have completely different string representations but end up close together as embeddings.

Apply Using the article’s simple_distance function (sum of absolute differences), calculate the distance between two conceptual embeddings [0.2, 0.7, 0.1] and [0.3, 0.6, 0.1].

Analyze The article’s semantic search example matches “I enjoy hot Mexican food” to “I love eating spicy tacos” even though none of the query’s words appear in the matched sentence. Walk through why a keyword-matching search (like the article’s Section 1 example) would have completely failed this query, while the embedding-based search succeeded.

Evaluate The article’s semantic-search demo prints a full score array that includes a negative value (the “Coding in Java” sentence scores below zero against “I enjoy hot Mexican food”). Critique the simplified claim that “0.0 means completely unrelated”: given that cosine similarity’s true range is -1 to 1, what does a near-zero-or-negative score actually tell you about how these embedding models represent unrelated concepts, and why shouldn’t you treat 0.0 as a hard floor?

Create Design a clustering-based analysis (following the article’s Section 5 “Clustering” use case) for a new scenario: a product team has 5,000 open-ended survey responses to “What’s missing from our app?” and wants to find recurring themes without reading every response. Describe the steps you’d take using embeddings, and what output (not just a similarity score) would actually be useful for a product manager.

---

Related articles

• Vector Databases Compared: When You Actually Need One)
• Building Your First RAG Pipeline: Chunking, Embedding, and Retrieval)

References & Further reading

• Mikolov, T., et al. (2013). Distributed Representations of Words and Phrases and their Compositionality. arXiv:1310.4546
• OpenAI Embeddings Documentation
• Sentence-Transformers Documentation

---