RAG

Retrieval Augmented Generation - generation based on the context from documents.

RAG includes several steps:

1. Splitting documents
2. Embedding splits into the vector database
3. Retrieving most similar to the user's query chunks from the vector db
4. Providing these chunks (+ the user's query) to the LLM as a context

We need to implement the following:

1. Function, to split and embed .md documents
2. Function, to split and embed .pdf documents
3. Function (or class) to have a single instance of vector db per folder
4. Retriever (+reranker - advanced RAG with better outcome) function
5. LLM call function

Moreover, user wants to have a reference to the document, from which LLM got information