Keeping a RAG Index in Sync as Documents Change

Take a shared drive of 20,000 files that average 30 chunks each. That is 20,000 x 30 = 600,000 chunks. If 1 percent of the files change in a typical week, a full rebuild re-embeds all 600,000 chunks to capture the work of 200 x 30 = 6,000. An incremental RAG index update processes only those 6,000, a hundredth of the work, and can run every hour instead of every weekend. The catch is that incremental indexing has to get three things right that a full rebuild gets for free: noticing changes, replacing old chunks, and removing files that no longer exist.

Pulling files out of SharePoint or Google Drive through their change feeds is covered in connecting a cloud drive to a private RAG; here the question is what the index does once it knows something changed.

The five events an index has to handle

Every sync run sorts source files into buckets, each with its own action.

Event	How it is detected	Index action
New file	Source ID not in the manifest	Extract, chunk, embed, insert
Changed content	Same ID, different content hash	Delete old chunks, insert new ones
Deleted file	In the manifest, missing from the source	Delete all its chunks
Moved or renamed	Same ID, new path	Update path metadata only
Permissions changed	Same content, different access list	Update access metadata only

The last two rows matter because they are cheap if handled correctly and expensive if not. A sync that keys files by path sees a rename as one deletion plus one new file and re-embeds content that did not change. Permission-only changes need their own comparison, because the content hash stays the same; permission-aware RAG covers that case.

A manifest table and a two-stage change check

The core of incremental indexing is a manifest: one row per source document, separate from the chunk table. A workable set of columns:

• source_id, a stable identifier (a drive item ID, or a path if nothing better exists)
• path, size_bytes, modified_at as last seen
• content_hash, a hash of the extracted text
• chunker_version and embed_model, which settings produced the current chunks
• indexed_at and last_seen_at

Checking for changes then runs in two stages. The cheap stage compares size and modified time against the manifest. Files that match are skipped without being opened. Files that differ go to the second stage: extract the text, hash it, and compare with content_hash. Only a different hash triggers chunking and embedding.

Hash the extracted text, not the file bytes

A DOCX or XLSX file is a zip archive with internal metadata. Opening a document and saving it without edits can change its bytes, and the modified time changes too. Hashing the raw file would treat that as a new version and re-embed it. Hashing the extracted, whitespace-normalized text skips it, because the words did not change. Text extraction costs CPU time, but it is far cheaper than embedding.

Pick a modern hash function. LangChain's indexing API, which tracks documents by a hash of their content and metadata, defaults to SHA-1 but its reference documentation warns that SHA-1 is not collision-resistant and lists sha256, sha512 and blake2b as alternatives. It also says that changing the key encoder means changing the index as well, to avoid duplicated documents, so choose once.

Replacing a changed document's chunks

When a hash differs, the safe pattern is delete then insert, keyed on source_id, inside one database transaction:

BEGIN;
DELETE FROM chunks WHERE source_id = $1;
INSERT INTO chunks (source_id, chunk_no, text, embedding, ...) VALUES ...;
UPDATE manifest SET content_hash = $2, indexed_at = now() WHERE source_id = $1;
COMMIT;

Done this way, a search running at the same moment sees either the old version or the new one, never both and never neither. Skipping the delete is the classic bug: the new chunks land next to the old ones, and the retriever happily returns a 2024 clause and its 2026 replacement side by side.

Chunk-level hashes save re-embedding, sometimes

A 40-page policy where one paragraph changed still produces a new document hash. Hashing each chunk as well lets the indexer reuse embeddings for chunks whose text is identical to before. Whether that saves anything depends on the chunker. A fixed-size chunker that cuts every 500 tokens shifts every boundary after the edit, so most chunks downstream of it hash differently. A chunker that splits on headings keeps the untouched sections identical. In a 40-page document split into 40 sections, editing one paragraph changes one chunk instead of, say, the 25 that follow it. That is one more reason to prefer the structure-based splitting described in chunking documents for RAG.

LlamaIndex's ingestion pipeline implements the document-level version of this: with a docstore attached, it keeps a map of document ID to document hash, re-processes and upserts a document whose hash changed, and skips one whose hash did not.

Deletions are the step that gets skipped

New and changed files announce themselves. Deleted files do not: a scan of the current folder never encounters them. Finding them requires comparing the full set of source IDs in the manifest against the full set at the source. Whatever is in the manifest but not at the source gets its chunks deleted.

Change feeds help here. Microsoft Graph's delta query reports removed items with a deleted facet, so a connector built on it gets deletions as events. A plain file share has no such feed, so the comparison has to be a full listing, typically once a night even if changed files are picked up hourly.

Framework defaults deserve a careful read. LangChain's indexing API has cleanup modes named None, incremental, full and scoped_full. Only full deletes every document the loader did not return in the current run (scoped_full does the same, but only for source IDs seen in that run), and the documentation warns that the loader must then return the entire dataset, or documents that should stay will be removed. The incremental mode cleans up superseded versions of documents it saw, which is not the same as removing documents that are gone.

Guard against mass deletion

The full-comparison approach has a sharp failure mode. If the file share is unmounted, a credential expires or an API returns an empty page, the listing comes back with zero files and the sync dutifully deletes the whole index. Add a threshold: if a run would delete more than a set share of the manifest, for example 5 percent, stop and alert instead. For 20,000 files that is a cap of 20,000 x 0.05 = 1,000 deletions per run. A real reorganization that exceeds it can be approved by hand.

On pgvector, deleted rows also leave work for the database. The pgvector README notes that vacuuming can take a while for HNSW indexes and suggests running REINDEX INDEX CONCURRENTLY before VACUUM. A nightly sync with heavy churn should schedule that, not leave it to chance.

Re-embedding when the model or chunker changes

Everything above assumes the embedding model and chunking settings stay fixed. When either changes, every chunk is stale at once. Vectors from two different embedding models are not comparable, as choosing a local embedding model explains, so a mixed index returns garbage rankings rather than an error.

The embed_model and chunker_version columns in the manifest make this manageable:

1. Build the new index beside the old one, in a new table or a new vector column, while the old one keeps serving.
2. Backfill it from stored chunk text, not by re-downloading every file.
3. Keep the incremental sync writing to both during the backfill, so changes made in the meantime are not lost.
4. Run the evaluation question set against the new index, then switch queries over.
5. Drop the old index once the new one has served for an agreed period.

Estimate the backfill before starting. Measure embedding throughput on the actual server with a sample of real chunks. If a test run embeds 10,000 chunks in 50 seconds, the rate is 200 chunks per second, and 600,000 chunks take 600,000 / 200 = 3,000 seconds, about 50 minutes. A slower card, longer chunks or a larger model can move that by an order of magnitude, so use the measured figure, not a published benchmark.

How a stale index shows up in answers

An out-of-sync index rarely fails loudly. It produces answers that look fine and are wrong. The common symptoms:

• Outdated citations. The answer quotes last year's travel policy because the new version was saved under a new file name and the old one was never removed.
• Contradictions within one answer. Old and new chunks of the same document are both retrieved, and the model blends them.
• Dead links. A citation points to a file that was deleted or moved, and clicking it fails.
• Missing knowledge. Questions about a document added last week return "no relevant information" because the sync job has silently stopped.

A few checks catch these before staff do. Record the time of the last successful sync and alert if it is older than twice the schedule. Compare the manifest row count with the source file count each night. Keep one canary document whose content is edited on a schedule with a known phrase, and run a question that should retrieve the new phrase. If the answer still quotes the old phrase, the pipeline is broken somewhere between the source and the index.

A practical schedule for most offices: pick up new and changed files every 15 to 60 minutes using modified times or a change feed, run a full reconciliation for deletions nightly with the mass-deletion guard on, and treat any embedding model or chunker change as a planned rebuild with its own evaluation.

Frequently asked questions

Should old versions of a document stay searchable?

Usually not in the default index, because mixing versions is what causes contradictory answers. If people genuinely need history, such as contract amendments, index each version as its own document with the version date in the metadata and filter to the current one unless the question asks for history.

Can a sync rely on modified dates alone?

Modified dates are a good first filter but not a reliable final answer. Copying files can preserve or reset them, restored backups carry old dates, and some tools change content without touching them. Use the date to decide what to inspect and the content hash to decide what to re-embed.

Do scanned PDFs break content hashing?

They can. OCR output may differ slightly between runs or engine versions, so re-running OCR on an unchanged scan can produce a new text hash and a pointless re-embed. Store the OCR text, re-run OCR only when the file bytes change, and record the OCR engine version in the manifest the same way as chunker_version.