Research

Overview of Key Sections:

1. Evaluation Metrics:

   • Recall: Measures the ability of the system to retrieve all relevant documents from a given dataset, emphasizing the system's completeness.

   • Normalized Discounted Cumulative Gain (NDCG): Assesses the quality of ranked search results, factoring in the relevance and ranking position of documents.

2. Constructing Evaluation Dataset:

   • Question Generation: Involves creating queries from document chunks using an LLM, designed to simulate real-world user inquiries.

   • Grouping Chunks: Clustering related chunks based on similarity to address queries that span multiple documents.

   • Ranking Chunks: Prioritizing chunks based on their relevance to the queries using automated scoring.

   • Critics: Addressing potential limitations and biases introduced by automated question generation and chunk ranking.

3. Evaluation Techniques:

   • Semantic vs. Combined Search: Comparing traditional semantic search capabilities against a combined method that incorporates full-text search.

   • Re-ranking: Implementing advanced techniques to re-assess the initial search results, enhancing the precision of document retrieval.

4. Results and Discussion:

   • Detailed presentation of findings, focusing on the effectiveness of various search and re-ranking strategies.

   • Discussion on the potential improvements and adjustments based on the evaluation outcomes.

This section presents research conducted by Unique on the following subjects