Embeddings and RAG from scratch

Models only know what was in their training data or what is in the prompt. You can't retrain the model, so the job becomes: find the right text and put it in the prompt. Doing that well is called RAG (retrieval-augmented generation), and the "find" half runs on embeddings.

What an embedding is

An embedding turns text into a list of numbers (a vector), usually hundreds of numbers long. Text with similar meaning gets similar numbers.

"How do I reset my password?" and "I'm locked out of my account" share almost no words, but their vectors are close together. That's the whole trick, and it's why searching by meaning beats searching by keyword.

const res = await gemini.embedContent({ contents: "How do I reset my password?" });
res.embedding.values;   // [0.021, -0.118, 0.093, … ] — a few hundred numbers

Measuring closeness

Cosine similarity scores how closely two vectors point in the same direction: 1.0 means the same direction (similar meaning), 0 means unrelated. It's a short function, and it's all the "vector search" you need until you have a lot of documents.

function cosineSimilarity(a, b) {
  let dot = 0, magA = 0, magB = 0;
  for (let i = 0; i < a.length; i++) {
    dot  += a[i] * b[i];
    magA += a[i] * a[i];
    magB += b[i] * b[i];
  }
  return dot / (Math.sqrt(magA) * Math.sqrt(magB));
}

The RAG pipeline

1. Chunk: split your documents into passages of a few hundred words. Let neighbors overlap a little so an idea isn't cut in half.
2. Embed every chunk once, and store the vectors.
3. Embed the question when it arrives.
4. Retrieve the top few chunks by similarity.
5. Generate: put those chunks in the prompt and tell the model to answer only from them.

const prompt = `Answer using ONLY the context below.
If the answer isn't there, say "I don't know".

Context:
${topChunks.join("\n---\n")}

Question: ${question}`;

That instruction matters. Without it, the model mixes the retrieved facts with half-remembered training data, and you lose the main benefit of RAG: answers you can trace back to a source (grounding).

Where RAG goes wrong

• Usually, the search. When a RAG system answers badly, the passage it needed was almost always missing from the prompt. Log what you retrieved before you blame the model.
• Chunks too big or too small. Too big, and the answer is buried in unrelated text. Too small, and the surrounding context that gave it meaning is cut off.
• Mixed embedding models. The question and the documents must be embedded with the same model. Vectors from different models can't be compared.

Your exercise is the search half: the part that decides whether the whole system works.