Getting RAG Retrieval Right

Good answers depend entirely on good retrieval. Notes on metadata indexing, filtered vector queries, and a schema-drift bug that silently returned nothing.

In a RAG system the model is only as good as the chunks you hand it. Most "the AI is dumb" complaints are really retrieval problems in disguise.

Filtering needs a metadata index

I store each chunk with metadata — source page, section, type — so I can scope retrieval. The catch in Cloudflare Vectorize: filtered queries require a metadata index to exist up front. Without it, your filter quietly matches nothing.

Schema drift bites silently

A mismatch between the metadata I wrote and the shape I queried meant perfectly good vectors came back empty, with no error at all. Vector stores rarely throw on this — they just return zero results — so I learned to assert the schema on both write and read.

Re-embedding is part of the job

Change the chunking, the model, or the metadata shape, and old vectors no longer match. A knowledge re-embed step isn't optional; it's maintenance you plan for.

Once retrieval is trustworthy, the LLM almost looks smarter than it is — because every answer is standing on the right context.