How to get Whisper transcripts into a vector database

Transcribing audio with Whisper is the easy part. The valuable part — being able to ask questions of your recordings — needs that text in a vector database so an AI agent can retrieve from it. Here’s the pipeline.

Step 1 — Transcribe (with diarization)

Run Whisper (or WhisperX for word-level timestamps and diarization). Keep speaker labels and timestamps — they make retrieved chunks far more useful.

Step 2 — Chunk the transcript

Long transcripts must be split into retrieval-sized chunks. Chunk on natural boundaries (speaker turns, topics) rather than fixed character counts, and carry metadata — speaker, timestamp, source file — on every chunk.

Step 3 — Embed

Turn each chunk into a vector with an embedding model. You can do this locally (e.g. via Ollama with nomic-embed-text) to keep everything on-prem, or call a hosted embedding API.

Step 4 — Store in a vector database

Upsert the vectors plus metadata into ChromaDB, Qdrant, Weaviate, Pinecone or Postgres (pgvector). Use deterministic IDs so re-running is idempotent and doesn’t create duplicates.

Step 5 — Retrieve

At query time, embed the question, search the vector store for the nearest chunks, and pass them to your LLM — this is RAG. For agents, exposing the store through MCP lets any compatible client query it.

Build vs buy

You can wire this together yourself, or use a tool that ships the whole pipeline. Products like NoParrot do transcription, diarization, chunking, embedding and vector-DB push out of the box — see the best transcription for RAG / agents ranking.