Skip to content
All projects

Product Knowledge RAG Bot

Production RAG chatbot that answers user questions from help articles and 1 GB of tabular data.

Role
Backend Developer (Intern)
Timeframe
May 2024 - June 2024
Stack
  • Python
  • LangChain
  • OpenAI GPT-4 Turbo
  • OpenAI Embeddings
  • Pinecone

Problem

The client runs a B2B data analytics platform and wanted a chatbot that answers user questions in plain English. Answers had to come from two very different sources: product help articles in Word format, and large tabular exports from the platform’s database. The exports came to about 1 GB across three CSV files, far more than the embedding API or the vector database accepts in one pass.

Architecture

Two offline ingestion scripts built on LangChain load the source data and write it into Pinecone. The article pipeline loads the .docx files, splits them into 4,000-character chunks, and embeds them with OpenAI’s text-embedding-ada-002. The CSV pipeline groups rows into multi-row documents and upserts them in batches, so the full exports fit within API and index limits. At question time, the bot embeds the user’s question, pulls the top 3 matching chunks from Pinecone, and sends them to GPT-4 Turbo with a grounded prompt that keeps the answer to the retrieved content.

  1. Ingestion embeds articles and CSV rows into Pinecone
  2. User sends a question to the RAG bot
  3. Question is embedded, top 3 chunks retrieved from Pinecone
  4. GPT-4 Turbo answers from the retrieved chunks only

What I built

  • Built a retrieval-augmented Q&A bot in Python with LangChain, Pinecone, and GPT-4 Turbo that answers only from retrieved content.
  • Built a CSV-to-Pinecone ingestion pipeline that packs 25 rows into each document and upserts 50 documents per batch. I designed it after the first full-size run exceeded the OpenAI tokens-per-minute limit and Pinecone’s per-vector metadata limit.
  • Migrated CSV retrieval from an early local Chroma test setup to Pinecone for production.
  • Built the article pipeline: .docx loading, 4,000-character chunking, and ada-002 embeddings in a cosine-similarity Pinecone index.
  • Wrote the grounding prompt: a fixed support persona, answers limited to the retrieved data, a clarifying question when the request is unclear, and an explicit “not sure” instead of a guess.
  • Tested a direct-SQL alternative with psycopg2 scripts against the PostgreSQL source database, including fuzzy ILIKE name matching versus exact names.

Results

  • Took the bot to production with the full ~1 GB of CSV exports and 16 help articles indexed in Pinecone.
  • Row grouping and batched upserts made the full dataset load within OpenAI’s rate limit and Pinecone’s metadata limit, where the first single-pass attempt had failed.