...

1. Formulates a search query optimized for search engines (based on the entire discussion history)

2. Executes a web search using the configured search engine.

3. Scraps the HTML content of top web pages returned by the search engine

4. Divide the scrapped content into small chunks of text

   1. [Optional] Embed the chunks locally and perform a similarity search based on the generated query.

5. Passes the relevant information to the context and answers the user’s question/query

...

Configuration settings (technical)

Reference in Code

WebSearch

Web Search Config

Provided below is the general structure of the configuration for the Web Search module.

...

Configuraion Example

...

Code Block
language json
{ "WebSearchConfig": { "debugging": false, "model_name": "AZURE_GPT_4o_2024_0513", "bing_config": { "search_market": "de-CH", "search_country_code": "CH", "custom_search_config": { "id": null, "boosted_search_domains": [] } }, "search_engine_name": "bing", "data_sources_config": { "chunk_size": 600, "fetch_size": 10, "max_workers": 10, "chunk_overlap": 50, "embedding_batch_size": 128 }, "query_generation_config": { "query_instruction": "The user's search query, optimized for search engines, incorporates relevant details from the conversation, especially if it is a follow-up question. Always use the user's language message." }, "answer_generation_config": { "embedding_reranking": false, "limit_token_sources": 10000, "max_chunks_to_consider": 500, "number_history_interactions_included": 4 } } }

Definition

{
  "type": "function",
  "function": {
    "name": "WebSearch",
    "parameters": {
      "type": "object",
      "required": [
        "query"
      ],
      "properties": {
        "query": {
          "type": "string",
          "description": "User's query that needs to be used to retrieve information from the web using a search engine."
        }
      }
    },
    "description": "Useful to answer user's query based on information from the web."
  }
}

...