RAG Knowledge Assistant

Business Challenge

An enterprise company’s employees were wasting hours every week searching for information. The company had thousands of documents — policies, procedures, technical specs, project plans, meeting notes — scattered across Google Drive, Confluence, Notion, and SharePoint. Finding the right document was hard; finding the right answer inside that document was harder.

The problem was quantified in a survey: employees spent a significant portion of their week just looking for information. New employees were particularly affected, taking weeks to get up to speed because they couldn’t find the documentation they needed. The company had tried enterprise search tools, but keyword search didn’t understand questions — it just matched words.

Solution

We built a RAG knowledge assistant that lets employees ask questions in natural language and get cited, accurate answers. The system ingests documents from all the company’s knowledge sources, chunks them into searchable pieces, and stores them in a vector database. When an employee asks a question, the system retrieves the most relevant chunks and generates an answer with citations.

The assistant is available through a web app and through Slack, so employees can ask questions where they already work. The web app provides a rich interface with source previews and follow-up questions, while the Slack integration lets teams ask questions in channels and get answers that everyone can see. The system understands context, so “What’s our remote work policy?” and “Can I work from another country?” both find the right document.

Technologies

The system uses LangChain for the RAG pipeline, Pinecone for vector storage, OpenAI for embeddings and answer generation, and FastAPI for the backend service. Document ingestion connectors pull from Google Drive, Confluence, Notion, and SharePoint on a schedule, with incremental updates to keep the knowledge base current.

The RAG pipeline uses hybrid search — combining semantic vector search with keyword matching — to maximize retrieval accuracy. A re-ranking model orders the retrieved chunks by relevance before they’re passed to the generation model. The system includes a feedback mechanism: employees can rate answers and flag incorrect information, which triggers a re-ingestion of the source document and a review of the retrieval pipeline.

Results

Information retrieval became significantly faster — employees get answers in seconds instead of searching for lengthy periods. Answer accuracy was rated highly by employees, and the citations meant employees could verify the source and trust the answer.

Company-wide, the system saved meaningful hours per department — time that was previously spent searching, asking colleagues, and reading irrelevant documents. New employee onboarding time dropped noticeably because they could ask the assistant instead of hunting for documentation. The Slack integration was particularly popular, with a large share of questions asked in Slack channels where the answers benefited the whole team.

Screenshots

RAG assistant web interface with cited answers

Asking questions and getting answers in Slack

Architecture

The system has two main components: ingestion and query. The ingestion pipeline pulls documents from multiple sources via connectors, cleans and chunks the text, generates embeddings, and stores them in Pinecone with metadata (source, date, author, department). Ingestion runs on a schedule with change detection to avoid reprocessing unchanged documents.

The query pipeline receives a question via the web app or Slack, generates an embedding, retrieves relevant chunks using hybrid search, re-ranks them, and generates a cited answer. The FastAPI backend handles both interfaces, with a WebSocket connection for the web app’s streaming responses and a Slack bot handler for the Slack integration. A feedback service collects ratings and corrections, feeding into a continuous improvement loop.