Capstone: Company Docs Q&A

Build a complete RAG system that answers questions from your own documentation. It handles chunking, embedding, hybrid search, re-ranking, conversational context, and returns answers with citations. Every piece draws from the five days of the course and runs against a real PostgreSQL + pgvector database.

What you'll learn

• A full RAG pipeline: chunk documents, embed chunks, store in pgvector, search with hybrid retrieval, re-rank, answer with citations
• Conversational memory threads context across turns so follow-up questions work
• Monitoring and caching keep it fast and debuggable in production

Architecture

The system coordinates document ingestion, storage, and retrieval through a multi-step pipeline. Below is the high-level architecture mapping out how data flows from raw markdown to the final LLM response.

Build it

Phase 1: Document ingestion pipeline

First, build a pipeline to read markdown files, split them into overlapping chunks, and generate embeddings. This snippet demonstrates how to process the files and insert them into pgvector.

def ingest_docs(directory):
    """Ingest all markdown files from a directory."""
    for filepath in Path(directory).glob("**/*.md"):
        content = filepath.read_text()
        chunks = recursive_chunk(content, chunk_size=500, overlap=50)

        for i, chunk in enumerate(chunks):
            embedding = generate_embedding(chunk)
            cursor.execute("""
                INSERT INTO documents (id, content, embedding, metadata)
                VALUES (%s, %s, %s, %s)
                ON CONFLICT (id) DO NOTHING
            """, (f"{filepath.stem}:{i}", chunk, embedding, json.dumps({
                "source": str(filepath),
                "chunk": i,
                "total_chunks": len(chunks),
            })))

Phase 2: Hybrid search with re-ranking

Next, implement a hybrid search function that combines vector similarity with keyword ranking. The initial broad search retrieves candidate documents, which are then passed to a cross-encoder for precise re-ranking.

def search(query, top_k=10):
    embedding = generate_embedding(query)
    results = cursor.execute("""
        SELECT content, metadata,
               1 - (embedding <=> %s) AS vector_score,
               ts_rank(search_vector, plainto_tsquery('english', %s)) AS text_score
        FROM documents
        WHERE status = 'active'
        ORDER BY (0.7 * (1 - (embedding <=> %s)) + 0.3 * ts_rank(search_vector, plainto_tsquery('english', %s))) DESC
        LIMIT %s
    """, (embedding, query, embedding, query, top_k * 3)).fetchall()

    return rerank(query, results, top_k)

Phase 3: Conversational Q&A

Finally, wire the retrieval pipeline into a conversational endpoint. This code reformulates the user's query using conversation history, retrieves the context, and instructs the model to answer with citations.

def answer_question(question, conversation_history=None):
    query = reformulate_query(question, conversation_history)
    results = search(query)
    context = format_context(results)

    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        system="Answer based on the provided context. Cite sources using [Doc N] notation. If you don't know, say so.",
        messages=[{"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"}],
    )

    return response.content[0].text

Confirm it worked

1. Ingest a directory of markdown files: ingest_docs("./docs/")
2. Ask a question: answer_question("What is the return policy?")
3. Verify the answer cites specific documents
4. Ask a follow-up: answer_question("What about for electronics?", history)
5. Verify the follow-up maintains context

Module map

Capstone component	Course lesson
Recursive chunking	Recursive Chunking
Embedding and pgvector storage	pgvector Setup
Hybrid search	Hybrid Search
Re-ranking	Re-ranking
Query reformulation	Conversational RAG
Caching	Caching
Monitoring	Monitoring & Observability