Ship a Document Chatbot

Table of Contents

1. From Library to Product
2. The Ingest Command
3. A Command-Line Chatbot with Citations
4. The Web Endpoint
5. Testing the Endpoint
6. Checklist: Who May See What
7. Checklist: Secrets, Privacy and Data Residency
8. Checklist: Keeping the Index Fresh
9. Checklist: Cost and Latency
10. Checklist: Citations, Feedback and Monitoring
11. When RAG Is the Wrong Tool
12. Recipe: The ingest, cli and server Modules
13. Try It Yourself
14. Quick Review: Common Questions

1. From Library to Product

You have a pipeline that works and a test that tells you when it stops working. But a pipeline is not a product. Nobody can use a folder of modules. This chapter adds the three pieces that turn raglab into something people can use, and then gives you the checklist that separates a demo from a system you can leave running:

• src/ingest.js: a command that builds or updates index.json.
• src/cli.js: an ask loop in the terminal that prints citations.
• src/server.js: a POST /api/ask web endpoint with validation, a rate limit and safe errors.

Everything runs on the mock server, as before (node mock-llm.js). To use a real provider, set LLM_BASE_URL, LLM_API_KEY, LLM_MODEL and LLM_EMBED_MODEL, and re-run ingest, because vectors from one embedding model cannot be searched with another (chapter 3).

2. The Ingest Command

Until now buildIndex ran in your scripts. A product needs a clear separate step: ingest the documents once, save the index, and let the chat program load it quickly. src/ingest.js (full file in section 12) wraps buildIndex and saves index.json. Run it twice:

$ node src/ingest.js
chunks: 33, re-embedded: 33, reused: 0
saved 33 chunks to index.json in 30 ms
$ node src/ingest.js
chunks: 33, re-embedded: 0, reused: 33
saved 33 chunks to index.json in 2 ms

The first run embeds all 33 chunks. The second finds the content hash of every chunk unchanged (chapter 4), so it reuses all 33 vectors and calls the embeddings API zero times. The millisecond numbers vary. Now the case that matters, a document that changes. I worked on a copy of the docs, with its own INDEX_PATH, so the real index stayed untouched:

$ node src/ingest.js docs-copy   # after editing the fee in membership.md
chunks: 33, re-embedded: 1, reused: 32
saved 33 chunks to index-demo.json in 21 ms
$ node src/ingest.js docs-copy   # after deleting faq.md
chunks: 28, re-embedded: 0, reused: 28
saved 28 chunks to index-demo.json in 3 ms
$ node src/ingest.js docs-copy --force
chunks: 28, re-embedded: 28, reused: 0
saved 28 chunks to index-demo.json in 22 ms

Editing one number in membership.md re-embedded one chunk and reused 32. Deleting faq.md shrank the index to 28 chunks, and the deleted file's vectors are gone, because the index is always rebuilt from the files that exist now. --force throws the old vectors away and embeds everything. The hash covers the chunk text and the model name, so changing the chunk size or the model name already re-embeds what changed. Use --force for a clean slate: when you suspect the index is wrong, or when a provider updates a model without changing its name.

Should index.json be in Git? It is derived data: you can always regenerate it. Many teams ignore it and run ingest during deployment. Others commit it, to avoid paying for embeddings on every deploy. Either choice is fine, but pick one on purpose, and never let the index and the documents drift apart.

3. A Command-Line Chatbot with Citations

src/cli.js loads index.json (running ingest first if it is missing) and loops: read a question, call ask(), print the answer and the passages it cites. Two commands: /sources shows everything that was retrieved for the last question, with scores, and /quit exits. The program also works with piped input, which gives stable output for testing and for this tutorial:

printf 'How much does it cost to join each term?\n/sources\nWhen is the Saturday workshop?\nWho is the vice president?\n/sources\n/quit\n' | node src/cli.js
Loaded 33 chunks. Ask about the club. /sources shows evidence, /quit exits.
bot> Membership costs 10 US dollars per term. [1]
     [1] membership.md > Membership fee
  [1] membership.md > Membership fee  (score 0.29)
  [2] projects.md > Term projects  (score 0.212)
bot> Laptops must be returned on a Saturday before the workshop begins, so the mentors can check them. [1]
     [1] laptops.md > How long
bot> I don't know based on the provided documents.
  [1] laptops.md > Who can borrow  (score 0.239)

Everything here is stable, because the mock is deterministic. Three things to read from it:

1. The first answer cites [1] and the line below it names the file and section. A user (or you) can open membership.md and check.
2. The second answer is wrong. The workshop time is 9:00 to 12:00 in schedule.md, but the mock grabbed a sentence from the laptop rules. The citation line says laptops.md > How long, and that is how you notice. A citation does not make an answer correct, it makes an answer checkable. This is the mock model's weakness, and it is exactly why you evaluate (chapter 7) and show sources.
3. For the vice president question, the answer is the refusal, and /sources still lists one passage. Retrieval found something above the threshold, but the model correctly said the passage does not answer the question. The cli prints sources only for passages the answer actually cites, so a refusal shows none.

Interactively you get a you> prompt on each turn. The code checks process.stdin.isTTY to print it only when a person is typing. This cli is one-shot: each question stands alone. For follow-ups ("and how long can I keep it?") you would add the question-rewriting step from chapter 5.

Notice the try/catch around ask() in the cli. If the provider is down, the loop prints one friendly line and keeps going instead of crashing. Never print the raw error object in a tool other people use.

4. The Web Endpoint

For a web page or a mobile app you need an HTTP endpoint. The server is built on the same rules as the proxy from API Keys and a Backend Proxy: the key never leaves the server, every input is validated on the server, a rate limit protects your bill, and visitors only ever see generic error messages. That chapter explains each rule in depth, and I will not repeat it. Read it first if you have not. Here is what is new, the handler:

async function handleAsk(req, res, store) {
  checkRateLimit(req.socket.remoteAddress);
  const question = validate(await readJson(req));
  const started = Date.now();
  let result;
  try {
    result = await ask(question, store); // we choose k, minScore and the model, never the visitor
  } catch (err) {
    console.error("upstream error:", err.status ?? err.message); // details for us only
    throw new HttpError(502, "The AI service had a problem. Please try again later.");
  }
  // One structured log line per question. Note: the question text is NOT logged.
  console.log(JSON.stringify({ ms: Date.now() - started, qchars: question.length, refused: result.refused,
    sources: result.sources.length, top: result.sources[0]?.score ?? null }));
  // Return only what the page needs. Scores and raw text stay on the server,
  // and a refusal cites nothing.
  sendJson(res, 200, {
    answer: result.answer,
    sources: result.refused ? [] : result.sources.map(({ n, source, heading }) => ({ n, source, heading })),
  });
}

What each choice protects:

• The visitor sends only question. Nothing else is read. The server decides k, the threshold and the model, so a stranger cannot pick an expensive model or ask for 500 passages.
• Length limits. At most 500 characters per question, 4 KB per body. You pay for every token, including the question's.
• A generic 502 if the provider fails. The real status goes to your log, never to the visitor.
• Only citations leave the server. The response has the answer and [{ n, source, heading }]. Raw passage text and scores stay inside, which keeps you in control of what the page can display. (If you want to show a snippet, return a short one on purpose.)
• A refusal returns no sources, so a page will not show "sources" next to "I don't know".
• One log line per question, without the question text. More on that in section 10.

Start the mock and the server (I used ports 8871 and 3070; with defaults you will have 8787 and 3000):

PORT=8871 node mock-llm.js
PORT=3070 LLM_BASE_URL=http://localhost:8871 node src/server.js
# raglab server on http://localhost:3070

Now try it with curl. First a good question, then an unanswerable one, then bad inputs and a hostile request that tries to choose the model and k:

curl -s localhost:3070/health
curl -s localhost:3070/api/ask -X POST -H 'content-type: application/json' \
  -d '{"question":"How much does it cost to join each term?"}'
# ... the same call for the other questions ...
{"ok":true,"chunks":33}
{"answer":"Membership costs 10 US dollars per term. [1]","sources":[{"n":1,"source":"membership.md","heading":"Membership fee"},{"n":2,"source":"projects.md","heading":"Term projects"}]} [200]
{"answer":"I don't know based on the provided documents.","sources":[]} [200]
{"error":{"message":"question must be a non-empty string."}} [400]
{"answer":"The club meets in Room 204 of the Riverside Learning Center, near the\nTonle Sap river. [1]","sources":[{"n":1,"source":"about.md","heading":"Where we meet"},{"n":2,"source":"about.md","heading":"About Mekong Code Club"},{"n":3,"source":"mentors.md","heading":"What mentors do not do"},{"n":4,"source":"membership.md","heading":"How to pay"}]} [200]
{"error":{"message":"Body must be valid JSON."}} [400]
{"error":{"message":"A question can have at most 500 characters."}} [400]
{"error":{"message":"Request body is too large."}} [413]
{"error":{"message":"Use POST."}} [405]

In order: the health check, a cited answer, a refusal with "sources":[], an empty question (400), a hostile request that was served normally because model and k were ignored (note it returned four sources, not 99), bad JSON (400), a 600-character question (400), a 5,000-character body stopped at 4 KB (413) and a GET on the endpoint (405). The /health route lets a host check that your server is alive, and it does not call the model.

The server logs, one JSON line per question:

raglab server on http://localhost:3070
{"ms":15,"qchars":40,"refused":false,"sources":2,"top":0.29}
{"ms":3,"qchars":26,"refused":true,"sources":1,"top":0.239}
{"ms":2,"qchars":33,"refused":false,"sources":4,"top":0.535}

The ms values vary. The line records how long it took, how long the question was, whether the system refused, how many passages were retrieved (before a refusal hides them) and the top score. It deliberately does not contain the question or any answer.

Now the failure path. I started a second server with LLM_MODEL=mock-error (the model that always fails) and asked one question:

{"error":{"message":"The AI service had a problem. Please try again later."}} [502]
server log:
upstream error: 500

The visitor gets one generic sentence and a 502. Your log says upstream error: 500. And the rate limit, with RATE_LIMIT_PER_MIN=3, five calls and then the full sixth reply:

200
200
200
429
429
HTTP/1.1 429 Too Many Requests
retry-after: 60
{"error":{"message":"Too many requests. Try again in 60 seconds."}}

As in the proxy chapter, this limiter is per IP, in memory, and resets on restart. It is a seat belt, not an armoured car. Authentication, so you know who is asking, is the next layer (and it matters for section 6).

5. Testing the Endpoint

Safety rules that are not tested drift. test/server.test.js imports the real server, uses the mock upstream, and checks the contract: the response has exactly answer and sources, a refusal cites nothing, bad input gets 400, extra fields are ignored, and the sixth request in a minute gets 429:

node --test --test-reporter=spec test/server.test.js
✔ a question gets an answer and its sources, and nothing else (6.60475ms)
✔ a refusal cites no sources (3.730292ms)
✔ bad input gets 400, and extra fields are ignored (4.455916ms)
✔ the sixth request in a minute gets 429 (0.616625ms)
ℹ pass 4
ℹ fail 0

The names and counts are stable and the milliseconds vary. The test file follows the same pattern as the proxy chapter (set the environment before importing, share one rate-limit budget, keep the order in mind), so it is short:

import test from "node:test";
import assert from "node:assert/strict";
import { startMock } from "../mock-llm.js";

process.env.LLM_BASE_URL = "http://localhost:8872";
process.env.RATE_LIMIT_PER_MIN = "5";
const { buildIndex } = await import("../src/indexer.js");
const { startServer } = await import("../src/server.js");

const mock = await startMock(8872);
const store = await buildIndex("docs", {});
const api = await startServer(3072, store);
test.after(() => { mock.close(); api.close(); });

const post = (body) =>
  fetch("http://localhost:3072/api/ask", {
    method: "POST",
    headers: { "content-type": "application/json" },
    body: JSON.stringify(body),
  });

// The tests share one budget of 5 requests per minute, so order matters.
test("a question gets an answer and its sources, and nothing else", async () => {
  const res = await post({ question: "How much does it cost to join each term?" }); // 1
  assert.equal(res.status, 200);
  const body = await res.json();
  assert.deepEqual(Object.keys(body), ["answer", "sources"]);
  assert.match(body.answer, /10 US dollars/);
  assert.deepEqual(body.sources[0], { n: 1, source: "membership.md", heading: "Membership fee" });
});

test("a refusal cites no sources", async () => {
  const body = await (await post({ question: "Who is the vice president?" })).json(); // 2
  assert.match(body.answer, /don't know/);
  assert.deepEqual(body.sources, []);
});

test("bad input gets 400, and extra fields are ignored", async () => {
  assert.equal((await post({ question: "" })).status, 400); // 3
  assert.equal((await post({ question: "x".repeat(501) })).status, 400); // 4
  const res = await post({ question: "Which room does the club meet in?", model: "mock-error", k: 99 }); // 5
  assert.equal(res.status, 200); // the visitor cannot choose the model
});

test("the sixth request in a minute gets 429", async () => {
  const res = await post({ question: "hi" }); // 6
  assert.equal(res.status, 429);
  assert.ok(res.headers.get("retry-after"));
});

6. Checklist: Who May See What

From here the chapter is a checklist. These are the things that turn up after the demo works, and they are the reason many RAG projects are unsafe. The first, and the most important:

Access control must be enforced during retrieval, before the model sees any text.

Imagine the club's mentors.md is meant for mentors only. If your chatbot searches all documents and relies on the prompt ("do not reveal mentor information to students"), you have built a leak. The model has already read the text, and prompts are not locks. A clever question, or one injected passage (chapter 5), can pull the text out. The rule is: what the user may not read must never enter the prompt.

The vector store from chapter 4 has a filter option for exactly this. The filter runs inside the search, so restricted chunks never become hits. Save this as acl-demo.js:

import { buildIndex } from "./src/indexer.js";
import { embed } from "./src/embed.js";
import { buildPrompt } from "./src/rag.js";
import { chat } from "./src/client.js";

// Who may read which file. Anything not listed is public.
const ACCESS = { "mentors.md": ["mentor"], "conduct.md": ["mentor", "member"] };
const canRead = (role, chunk) => (ACCESS[chunk.source] ?? [role]).includes(role);

const store = await buildIndex("docs", {});
async function askAs(role, question) {
  const [qv] = await embed([question]);
  // The filter runs INSIDE the search: restricted chunks never become hits.
  const hits = store.search(qv, { k: 4, minScore: 0.2, filter: (c) => canRead(role, c) });
  const messages = buildPrompt(question, hits);
  const leaked = messages[1].content.includes("mentors.md");
  const r = await chat({ messages });
  console.log(`${role.padEnd(7)} sources=[${hits.map((h) => h.chunk.source).join(", ")}]  mentors.md in prompt: ${leaked}`);
  console.log(`        ${r.text.replace(/\s+/g, " ")}`);
}
const q = "When do the mentors meet each month?";
await askAs("member", q);
await askAs("mentor", q);

node acl-demo.js
member  sources=[membership.md, projects.md, laptops.md]  mentors.md in prompt: false
        I don't know based on the provided documents.
mentor  sources=[mentors.md, mentors.md, membership.md, mentors.md]  mentors.md in prompt: true
        Mentors meet on the first Wednesday of every month at 19:00 to plan the next topics. [2]

The student's search never returned a chunk from mentors.md, and the prompt that would be sent to the model does not contain the file, so the model cannot leak it. The mentor asked the same question and got the answer. (ACCESS[chunk.source] ?? [role] means "if the file is not listed, it is public": a missing entry falls back to the caller's own role.)

Real systems need more than this demo:

• The role comes from authentication, never from the request body. A role field the browser sends is a field a stranger can edit. Use the signed-in user's identity on the server (see Keycloak Crash Course for sign-in).
• Store the permission with each chunk (an allowed list, a team or a document owner) when you ingest, so the filter does not depend on file names.
• Filter before ranking, not after. If you retrieve the top four and then remove restricted ones, you may return nothing, or leak through the rank order and scores.
• Citations leak too. A source title like "Layoff plan 2027" is information. Do not show a source the user may not open.
• Cached answers need the same rule. An answer generated for an admin must never be served from a cache to a student.
• Test it. Add eval cases that ask as a low-privilege user for restricted facts, and require a refusal.

7. Checklist: Secrets, Privacy and Data Residency

Never index secrets. Anything in the index can, in principle, be retrieved and quoted. Do not ingest passwords, API keys, private keys, .env files, national ID numbers or full database exports. Scan the documents before ingest, and keep secrets out of the folder in the first place. If a secret was indexed once, rotate it (it is burned) and rebuild the index.

Personal data (PII). Student lists, phone numbers, health notes and grades are personal data. Ask first: do we need it in the chatbot? If not, leave it out. If yes, apply the access rules above, keep a record of what is indexed, and be able to delete a person's data from the documents and the index when they ask.

Where do the documents go? With a hosted provider, ingest sends your text to a third party (for embeddings), and every question sends the retrieved passages (for the answer). That is a decision, not a detail:

• Read the provider's terms for how long they keep requests and whether they use them to train models. Terms differ and change, so read the current ones for your plan.
• Some organisations and laws restrict where data may be processed or stored (data residency) or require a written agreement first. Ask the person responsible for data in your school or company before you send anything real.
• If the documents cannot leave your control, the alternative is to run both models yourself on your own machines. That costs hardware and effort, and the quality depends on the models you can run.

If you are unsure, start with documents that are already public, and add private ones only after someone has approved the plan.

The index is a copy of your documents. index.json contains the chunk text, so it deserves the same protection as the originals: access rules, backups, and no public hosting. Embedding vectors alone are harder to read, but they are not a safe way to hide text. Treat them as sensitive as well.

Prompt injection (chapter 5) is also a security issue: a poisoned document can try to give the model orders. The mitigations are the same: mark passages as untrusted data, give the model no tools and no secrets to misuse, and if you later let it act, isolate those actions as in Docker Sandbox Crash Course.

8. Checklist: Keeping the Index Fresh

A chatbot that confidently quotes last term's fee is worse than no chatbot. Freshness is a process:

• Re-ingest on change. Run node src/ingest.js whenever the documents change: by hand for a small project, in a scheduled job, or in the deploy pipeline. Content hashing keeps it cheap.
• Handle deletes. As you saw, our ingest rebuilds from the files that exist, so a deleted document disappears. Systems that update a database one document at a time need an explicit "remove these chunks" step, and often forget it. Test it.
• Show dates. Put "last updated" in your documents or metadata and show it in the citation, so a reader can judge whether it is stale.
• Re-embed when the model changes (--force), and never mix vectors from two models in one index.
• One owner per document. Conflicting or outdated documents are the failures that code cannot fix (chapter 7). Someone must be responsible for the source of truth.
• Reload the running server. Our server reads index.json at start-up. After ingesting, restart it, or add a reload step. Forgetting this is a classic "I updated the document but the bot still says the old thing".

9. Checklist: Cost and Latency

Cost model. RAG has two very different kinds of cost:

• Embedding the documents: paid once per chunk, and again only when a chunk or the model changes. For a document set like ours it is negligible.
• Each question: embed the question (tiny), search (your own CPU, free), then generate. The generation call carries the system prompt, the retrieved passages and the question as input, plus the answer as output. This is where the money goes, and it grows with k and chunk size.

Here is that for the club corpus, with example prices that I invented for this lesson (the script labels them too). Your provider's real prices will be different, so read their current price list:

import { buildIndex } from "./src/indexer.js";
import { ask } from "./src/rag.js";

// EXAMPLE prices in US dollars per million tokens. Invented for this lesson:
// look up your provider's real price list before you plan anything.
const PRICE = { embed: 0.1, input: 1.0, output: 3.0 };
const tokens = (text) => Math.ceil(text.length / 4); // the same rough rule the mock uses

const store = await buildIndex("docs", {});
const corpusTokens = store.items.reduce((s, it) => s + tokens(it.chunk.text), 0);
const once = (corpusTokens / 1e6) * PRICE.embed;
console.log(`corpus: ${store.size} chunks, about ${corpusTokens} tokens`);
console.log(`embed once:  $${once.toFixed(6)}   (the money is rounding error)`);

const r = await ask("How much does it cost to join each term?", store);
const { prompt_tokens, completion_tokens } = r.usage;
const queryEmbed = tokens("How much does it cost to join each term?");
const perQuestion = (queryEmbed / 1e6) * PRICE.embed + (prompt_tokens / 1e6) * PRICE.input + (completion_tokens / 1e6) * PRICE.output;
console.log(`one question: ${queryEmbed} tokens to embed, ${prompt_tokens} in, ${completion_tokens} out`);
console.log(`  embedding the question: $${((queryEmbed / 1e6) * PRICE.embed).toFixed(8)}`);
console.log(`  prompt (input):         $${((prompt_tokens / 1e6) * PRICE.input).toFixed(8)}`);
console.log(`  answer (output):        $${((completion_tokens / 1e6) * PRICE.output).toFixed(8)}`);
console.log(`1,000 questions a month: $${(perQuestion * 1000).toFixed(4)}`);

node cost.js
corpus: 33 chunks, about 1547 tokens
embed once:  $0.000155   (the money is rounding error)
one question: 10 tokens to embed, 237 in, 11 out
  embedding the question: $0.00000100
  prompt (input):         $0.00023700
  answer (output):        $0.00003300
1,000 questions a month: $0.2710

The token counts use the mock's rough "4 characters per token" rule, so they are estimates, not real tokenizer counts. The shape is what matters: indexing the whole corpus costs a fraction of a cent, embedding a question costs almost nothing, and the generation call is the bill (the prompt is the larger part here, because it carries the instructions and the passages for an 11-token answer). That tells you where to save: fewer, better chunks (chapter 6), a refusal gate that skips the model for hopeless questions (chapter 5), a smaller model for easy questions, and a short answer limit.

Latency. A RAG answer waits for several steps one after another: embed the question, search, then generate. The generation step is usually the slowest and grows with the answer length, so stream the answer to the page (the same server-sent events as in Streaming Responses) so the user sees words quickly. Our log shows ms of a few milliseconds, but that is the mock, and says nothing about a real provider. Measure your own. Brute-force search over a few thousand chunks is fast. If you reach many thousands or more, look at the vector-database options from chapter 4.

10. Checklist: Citations, Feedback and Monitoring

Show citations in the interface. The endpoint returns n, source and heading. In the page, turn [1] in the answer into a link or button that opens the source section, and list the sources under the answer. When you render the answer, use textContent (or your framework's escaping), never innerHTML: the answer includes text derived from documents and from the model, and treating it as HTML is an XSS hole, as in A Web Chat UI. Remember the lesson from section 3: citations let a human check the answer. Make that one click.

Collect feedback. Add a thumbs up and thumbs down to each answer, sent to a second small endpoint together with an id of the answer. A thumbs-down is the best possible signal: it marks a real question that failed. Feed those into the eval set (chapter 7, section 12), with the privacy care described there.

Monitor what matters. Our one log line per question already records the numbers you need. Watch these over time:

• the refusal rate. A sudden rise can mean the index is broken or stale, or users ask something new. A rate of zero is suspicious too;
• the top score distribution. If it drifts down, the questions and the documents are moving apart;
• the latency and the cost per question;
• errors from the provider and 429s (yours and theirs);
• the thumbs-down questions, read by a human each week.

And keep a spending limit with your provider, for the same reason as in the proxy chapter.

11. When RAG Is the Wrong Tool

RAG is a good tool, not the answer to every problem. Do not use it when:

• The corpus is tiny. Our club handbook is about 1,500 tokens by the rough count above. You could paste all of it into every prompt, with no chunking, embeddings or index, and get a simpler and often more accurate system. We built the pipeline because you needed to learn it, not because the club needed it. If your whole text fits comfortably in the model's context and the cost is acceptable, try that first.
• The question needs the whole document. "Summarise this contract", "what changed between version 1 and 2?" and "list every deadline in the handbook" need many parts at once, but RAG hands the model only a few chunks. Use the full text (or a map-and-combine approach over the whole thing), not top-k retrieval.
• The data is structured. "How many members joined in March?" or "which laptops are on loan?" are questions for a database. Put the data in tables and answer with SQL (a model can help write the query, and you must validate it). Searching text chunks for numbers is slow, unreliable and costly.
• You need exact, guaranteed answers. A tax rate, a drug dose or a legal deadline should not come from a probabilistic summary. Use a lookup, a form, or send people to the source.
• The documents are bad. If they are outdated, contradictory or missing, RAG makes the mess more fluent. Fix the documents first.
• You can use plain search. Sometimes your users only need a good search box that returns the right page. Keyword search plus a link is cheaper, instant and cannot invent anything.

A useful test: would a person with a search box and these documents get the answer in one minute? If yes, RAG can save them that minute. If the answer needs reading, comparing and reasoning across the whole set, plan for something stronger than top-k retrieval.

12. Recipe: The ingest, cli and server Modules

The final files. package.json first, with scripts for the commands:

{
  "name": "raglab",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "ingest": "node src/ingest.js",
    "chat": "node src/cli.js",
    "serve": "node src/server.js",
    "test": "node --test"
  }
}

src/ingest.js exports ingest() so the cli can call it, and runs itself only when started directly:

// Rebuild or update index.json from the documents folder.
// Usage: node src/ingest.js [docsDir] [--force]
import fs from "node:fs/promises";
import { buildIndex } from "./indexer.js";

export const INDEX_PATH = process.env.INDEX_PATH ?? "index.json";

export async function ingest(dir = "docs", { force = false } = {}) {
  if (force) await fs.rm(INDEX_PATH, { force: true }); // forget old vectors: re-embed everything
  const started = Date.now();
  const store = await buildIndex(dir, { indexPath: INDEX_PATH, log: (m) => console.log(m) });
  console.log(`saved ${store.size} chunks to ${INDEX_PATH} in ${Date.now() - started} ms`);
  return store;
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const args = process.argv.slice(2);
  await ingest(args.find((a) => !a.startsWith("--")) ?? "docs", { force: args.includes("--force") });
}

src/cli.js, the ask loop:

// An ask loop with citations. Works interactively and with piped input.
// Commands: /sources (show the last answer's sources with scores), /quit
import readline from "node:readline";
import fs from "node:fs";
import { VectorStore } from "./store.js";
import { ask } from "./rag.js";
import { ingest, INDEX_PATH } from "./ingest.js";

const store = fs.existsSync(INDEX_PATH) ? await VectorStore.load(INDEX_PATH) : await ingest("docs");
console.log(`Loaded ${store.size} chunks. Ask about the club. /sources shows evidence, /quit exits.`);

const rl = readline.createInterface({ input: process.stdin, prompt: "you> " });
const interactive = process.stdin.isTTY;
let last = null;
if (interactive) rl.prompt();

for await (const line of rl) {
  const text = line.trim();
  if (text === "/quit") break;
  if (text === "/sources") {
    if (!last) console.log("Nothing to show yet.");
    else if (last.sources.length === 0) console.log("No passage was similar enough to the question.");
    else for (const s of last.sources) console.log(`  [${s.n}] ${s.source} > ${s.heading}  (score ${s.score})`);
  } else if (text) {
    try {
      last = await ask(text, store);
      console.log(`bot> ${last.answer.replace(/\s+/g, " ")}`);
      // Show only the passages the answer actually cites.
      const cited = new Set([...last.answer.matchAll(/\[(\d+)\]/g)].map((m) => Number(m[1])));
      for (const s of last.sources.filter((s) => cited.has(s.n))) console.log(`     [${s.n}] ${s.source} > ${s.heading}`);
    } catch (err) {
      console.log(`bot> Sorry, something went wrong (${err.status ?? err.message}). Try again.`);
    }
  }
  if (interactive) rl.prompt();
}
rl.close();

And src/server.js. The limits are constants at the top, so every decision that protects your money is visible in one place. It exports startServer(port, store) so the tests can pass in a store and a port:

// POST /api/ask { question } -> { answer, sources }. Same safety rules as the
// chatbot series' proxy: validate, limit, hide upstream details.
import http from "node:http";
import { VectorStore } from "./store.js";
import { ask } from "./rag.js";
import { INDEX_PATH } from "./ingest.js";

const MAX_BODY_BYTES = 4 * 1024;
const MAX_QUESTION_CHARS = 500;
const RATE_LIMIT = Number(process.env.RATE_LIMIT_PER_MIN ?? 10);

class HttpError extends Error {
  constructor(status, message) { super(message); this.status = status; }
}

const hits = new Map(); // ip -> { count, resetAt }  (fixed window, in memory)
function checkRateLimit(ip) {
  const now = Date.now();
  let e = hits.get(ip);
  if (!e || e.resetAt <= now) { e = { count: 0, resetAt: now + 60_000 }; hits.set(ip, e); }
  if (++e.count > RATE_LIMIT) throw new HttpError(429, `Too many requests. Try again in ${Math.ceil((e.resetAt - now) / 1000)} seconds.`);
}

async function readJson(req) {
  let size = 0;
  const chunks = [];
  for await (const c of req) {
    size += c.length;
    if (size > MAX_BODY_BYTES) throw new HttpError(413, "Request body is too large.");
    chunks.push(c);
  }
  try { return JSON.parse(Buffer.concat(chunks).toString("utf8")); }
  catch { throw new HttpError(400, "Body must be valid JSON."); }
}

function validate(body) {
  const q = body?.question;
  if (typeof q !== "string" || q.trim() === "") throw new HttpError(400, "question must be a non-empty string.");
  if (q.length > MAX_QUESTION_CHARS) throw new HttpError(400, `A question can have at most ${MAX_QUESTION_CHARS} characters.`);
  return q.trim();
}

const sendJson = (res, status, obj, headers = {}) => {
  res.writeHead(status, { "content-type": "application/json", "cache-control": "no-store", ...headers });
  res.end(JSON.stringify(obj));
};

async function handleAsk(req, res, store) {
  checkRateLimit(req.socket.remoteAddress);
  const question = validate(await readJson(req));
  const started = Date.now();
  let result;
  try {
    result = await ask(question, store); // we choose k, minScore and the model, never the visitor
  } catch (err) {
    console.error("upstream error:", err.status ?? err.message); // details for us only
    throw new HttpError(502, "The AI service had a problem. Please try again later.");
  }
  // One structured log line per question. Note: the question text is NOT logged.
  console.log(JSON.stringify({ ms: Date.now() - started, qchars: question.length, refused: result.refused,
    sources: result.sources.length, top: result.sources[0]?.score ?? null }));
  // Return only what the page needs. Scores and raw text stay on the server,
  // and a refusal cites nothing.
  sendJson(res, 200, {
    answer: result.answer,
    sources: result.refused ? [] : result.sources.map(({ n, source, heading }) => ({ n, source, heading })),
  });
}

export async function startServer(port = 3000, store) {
  store ??= await VectorStore.load(INDEX_PATH);
  const server = http.createServer(async (req, res) => {
    try {
      const { pathname } = new URL(req.url, "http://x");
      if (pathname === "/health") return sendJson(res, 200, { ok: true, chunks: store.size });
      if (pathname !== "/api/ask") throw new HttpError(404, "Not found.");
      if (req.method !== "POST") throw new HttpError(405, "Use POST.");
      await handleAsk(req, res, store);
    } catch (err) {
      if (res.headersSent) return res.end();
      if (!(err instanceof HttpError)) console.error("unexpected:", err);
      const status = err instanceof HttpError ? err.status : 500;
      sendJson(res, status, { error: { message: err instanceof HttpError ? err.message : "Something went wrong." } },
        status === 429 ? { "retry-after": "60" } : {});
    }
  });
  return new Promise((resolve) => server.listen(port, () => resolve(server)));
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const port = Number(process.env.PORT ?? 3000);
  await startServer(port);
  console.log(`raglab server on http://localhost:${port}`);
}

This is a teaching server. Before real users, add HTTPS (your host usually provides it), authentication and per-user limits (section 6), request IDs in logs, a shared rate-limit store if you run more than one copy, timeouts and retries (see Errors, Retries and Cost), streaming, and monitoring.

13. Try It Yourself

1. Reload without a restart. Make the server re-read index.json when the file changes (look at fs.watch, or reload on SIGHUP). Test it by editing a document, running ingest, and asking again.
2. Add /api/ask streaming. Reuse chatStream from the chatbot series to stream the answer, and send the sources in a final event. Which parts of ask() do you have to split?
3. Permissions in the index. Add an allowed list to each chunk at ingest (from a small permissions.json), and make the server take the user's role from a header you set in a test. Write an eval case that proves a student cannot get a mentor-only fact.
4. Add a feedback endpoint. POST /api/feedback with { id, good }. Store it in a file, validate it as strictly as question, and decide what you will and will not store.
5. A scheduled ingest. Write a script that runs ingest and exits with a non-zero code if more than half the chunks changed (a likely sign of a broken document export). Why is a loud failure better than a silent re-index?
6. Decide honestly. Take three real document collections (your school's rules, a textbook, a spreadsheet of results). For each, say whether you would use RAG, paste-everything, SQL or plain search, and write two sentences why.

14. Quick Review: Common Questions

Questions beginners often ask about this chapter. Try answering each one yourself first, then click the question to open the answer.

Where to go next. You can now ingest documents, answer from them with citations, measure quality, and ship it with sensible safety. So far the model only reads. The next series, AI Agents with Tools & MCP (planned), lets a model act: calling functions, searching your documents as one tool among many, and connecting to outside systems through the Model Context Protocol, with the safety questions that come with it. Running anything a model chooses belongs in isolation, so Docker Sandbox Crash Course is good preparation. If you want to revisit the foundations, go back to Build a Chatbot with an LLM API for the API, streaming and error handling that this series builds on, and to How LLMs Work (for Developers) for tokens, embeddings, attention and why models hallucinate.