Page Comparison

...

• Larger token inputs provide better context, resulting in more accurate RAG outcomes.

• Ideal for customers who require detailed and extensive document analysis.

...

Improving RAG with Chunk Relevancy Sort

Chunk Relevancy Sort improves RAG results by resorting the retrieved document chunks based on their relevancy compared to the user query. This approach uses a language model to evaluate the relevancy and can also be configured as a filter to consider only highly relevant chunks. It is particular useful when no additional infrastructure can be acquired and rate limits are of no concerns. However, this additional step increases latency and comes with additional token costs per generated response.

...

• Enhances accuracy by focusing on relevant data.

• Suitable for scenarios where slightly increased latency is acceptable for better results.

• Does not require additional infrastructure.

...

Improving RAG with Reranker

...

The Reranker provides improves the accuracy of the generated response by reranking the retrieved chunks based on a predicted relevancy score. This method uses a dedicated, pre-trained model to predict the relevancy score for each document chunk. It is particularly useful when rate limits are a concern, or when only a model with smaller throughput is available. However, this additional step increases latency and comes with additional costs for infrastructure.

...

1. More Input Tokens:

   • Generate answer with GPT-4 Omni and at least 30k tokens context window

2. Chunk Relevancy Sort:

   • Evaluate relevancy with GPT-4 Omni-Mini (potentially filter only for highly relevant chunks)

   • Generate answer with GPT-4 Omni and at least 7k tokens context window

3. Reranker:

   1. Generate answer with GPT-4 Omni and at least 7k tokens context window

PTU Purchased

...

Available Models:

• GPT-4 Omni

• GPT-4 Omni-Mini

...

...

...