A complete chatbot project — building a document-grounded conversational assistant using RAG, connecting a pretrained LLM to a custom knowledge base rather than training a chatbot from scratch, exactly as most real production chatbots are actually built today.
Problem Statement
Build a chatbot that answers questions about a specific document collection (e.g. a company's FAQ, a product's documentation) accurately and with cited sources, rather than relying purely on an LLM's general trained-in knowledge.
Dataset
Any collection of text documents relevant to the chosen domain — FAQ pages, documentation, or similar — the more focused and accurate the source documents, the better the resulting chatbot's answers will be.
Architecture & Approach
This is a direct, practical application of RAG (see RAG): documents are chunked, embedded, and indexed in a vector database; at query time, relevant chunks are retrieved and provided as context to an LLM, which generates an answer grounded in that retrieved context.
Step-by-Step Build
from sentence_transformers import SentenceTransformer
import faiss
import numpy as np
# 1. Index the document collection
embed_model = SentenceTransformer('all-MiniLM-L6-v2')
def chunk_document(text, chunk_size=300, overlap=50):
words = text.split()
chunks = []
for i in range(0, len(words), chunk_size - overlap):
chunks.append(' '.join(words[i:i + chunk_size]))
return chunks
all_chunks = []
for doc in documents:
all_chunks.extend(chunk_document(doc))
chunk_embeddings = embed_model.encode(all_chunks)
index = faiss.IndexFlatL2(chunk_embeddings.shape[1])
index.add(np.array(chunk_embeddings).astype('float32'))
# 2. The retrieval + generation function
def answer_question(query, k=3, conversation_history=None):
query_embedding = embed_model.encode([query]).astype('float32')
_, indices = index.search(query_embedding, k)
retrieved_chunks = [all_chunks[i] for i in indices[0]]
context = "\n\n".join(f"[Source {i+1}]: {chunk}" for i, chunk in enumerate(retrieved_chunks))
prompt = f"""Answer the question using ONLY the context below. Cite the source number(s) you used.
If the context doesn't contain the answer, say so clearly rather than guessing.
Context:
{context}
Question: {query}
Answer:"""
response = llm_client.generate(prompt=prompt, conversation_history=conversation_history)
return response, retrieved_chunks
# 3. A simple multi-turn conversation loop
conversation_history = []
while True:
user_input = input("You: ")
if user_input.lower() == "quit": break
answer, sources = answer_question(user_input, conversation_history=conversation_history)
print(f"Bot: {answer}")
conversation_history.append({"role": "user", "content": user_input})
conversation_history.append({"role": "assistant", "content": answer})Expected Results
For questions well-covered by the source documents, expect accurate, grounded answers with correct source citations; for questions outside the document collection's coverage, the model should (if the prompt is well-designed) explicitly say it doesn't have the information, rather than hallucinating a plausible-sounding but ungrounded answer.
Key Learnings & Extensions
- Chunk size and overlap meaningfully affect retrieval quality — experiment with different values and observe how answer quality changes, directly reinforcing the chunking guidance from RAG.
- Extension: Add explicit tool calling (e.g. a calculator, or a live lookup tool) so the chatbot can go beyond pure document retrieval when needed — connecting directly to Tool Calling.
- Extension: Build a simple evaluation set of question/expected-answer pairs and measure how often the chatbot's answers are actually correct and properly grounded — moving from "it seems to work" to a measured evaluation.
