Skip to content
All projects

MongoDB RAG Bot

Flask API that answers plain-English questions by having GPT-4o write MongoDB aggregation queries.

Role
Backend Developer (Intern)
Timeframe
June 2024 - July 2024
Stack
  • Python
  • Flask
  • LangChain
  • OpenAI GPT-4o
  • MongoDB
  • Pinecone

Problem

The client wanted people to ask questions about data in a MongoDB collection in plain English and get a short, readable answer. Most questions are counts, groupings, and filters, so the answer has to come from a real database query, not from retrieved text. The database schema grew large, and passing the whole schema to the LLM made the generated queries worse. The model’s output also had to be parsed safely before running it.

Architecture

A client sends a JSON question to a Flask REST API, which hands it to a query agent built on LangChain. The agent searches a vector database of schema field descriptions and pulls back only the fields relevant to the question. It puts those fields into the prompt and asks GPT-4o for a MongoDB aggregation pipeline, then parses the pipeline and runs it with PyMongo. A second GPT-4o call turns the raw documents into a short answer or a markdown table. Every request logs the question, answer, token counts, and cost.

  1. Client POSTs a question to the Flask /api/query endpoint
  2. Agent pulls only the relevant schema fields from the vector DB
  3. GPT-4o writes a MongoDB aggregation pipeline from those fields
  4. Parsed aggregation pipeline runs on MongoDB via PyMongo
  5. GPT-4o summarizes results, Flask logs tokens and cost

What I built

  • Built the full service end to end in Python: a two-stage LangChain and GPT-4o pipeline that turns a question into an aggregation pipeline, runs it on MongoDB, and writes the answer in natural language.
  • Designed schema retrieval with a vector database, so each prompt carries only the relevant fields instead of the full schema, which had been degrading query quality.
  • Wrote the query-generation prompt: read-only instructions, handling of case mismatches and “N/A” values, and a fixed refusal for off-topic questions so they never reach the database.
  • Replaced eval() on model output with ast.literal_eval() after stripping code fences, so the generated pipeline is parsed as data and never executed as Python.
  • Exposed three Flask endpoints with CORS for querying and for reading and updating the schema, with input validation and JSON error responses.
  • Added per-request token and cost tracking with LangChain’s OpenAI callback, plus structured logging to separate info and error files.
  • Evaluated embedding and model options along the way, including sentence-transformers, Ollama embeddings, and Llama 3 running locally through Ollama.

Results

  • A question such as “Provide a table with the count of movies per movie genre” returns a markdown table computed by MongoDB, not guessed by the model.
  • Refactored a single prototype script into an API layer, an agent class, and a prompt module, and handed it off as a self-contained package with a README and a Postman collection covering all three endpoints.
  • The client’s in-house team used the delivered service as the starting point of their own codebase and kept extending it.
  • LLM spend is visible per request through logged token counts and cost.