Update README.md for RAG
* Update README.md
Add more instructions for processing small files
* Update README.md
@@ -155,6 +155,8 @@ python index_dense_embeddings.py --retriever-embedding-size 768 \
--embeddings-dir /tmp/wiki_passage_embeddings/ --embeddings-name wiki_passages
```
+Note the default index factory setting is `IVF4096_HNSW128,PQ128`, if you are processing small files, you may encounter errors such as `Error: 'nx >= k' failed`, then you need to set `--compressed-indexer-factory` to other indexes in the [index factory in FAISS](https://github.com/facebookresearch/faiss/wiki/The-index-factory) such as `HNSW32`.
+
## Directory Structure / Custom Components
I will outline here the structure of the RAG directory, and where you might want to add custom components if you so desire.
