IPCC Knowledge Base Integration

[aleenaharoldpeter]

Description

The chatbot currently relies primarily on CABook as its knowledge source. While CABook provides valuable climate-related information, it does not cover every topic, scenario, or level of detail that users may ask about.

To improve knowledge coverage, we want to integrate IPCC reports as a secondary knowledge source and fallback mechanism. The primary retrieval source should remain CABook. However, when CABook retrieval returns weak, low-confidence, incomplete, or irrelevant results, the chatbot should be able to retrieve supporting information from IPCC reports.

The goal is not to replace CABook, but rather to use IPCC content to supplement the chatbot's knowledge when CABook alone is insufficient to answer a user's question effectively.

This issue tracks the overall effort required to ingest, process, retrieve, and utilize IPCC content within the chatbot pipeline.

Proposed Retrieval Flow

1. User submits a question.
2. Retrieve relevant content from CABook.
3. Evaluate retrieval quality/confidence.
4. If CABook provides sufficient information:
5. Generate the response using CABook content.
6. If CABook retrieval is weak, incomplete, or lacks relevant information:
7. Retrieve supporting information from the IPCC knowledge base.
8. Use the IPCC content as fallback context for response generation.
9. Return the final response to the user.

Motivation

• Improve climate knowledge coverage.
• Reduce unanswered or partially answered climate questions.
• Provide a reliable fallback source when CABook lacks relevant information.
• Leverage authoritative climate science from IPCC reports.
• Maintain CABook as the primary knowledge source while extending its coverage.

Success Criteria

• IPCC documents can be processed and indexed.
• Relevant IPCC sections can be retrieved efficiently.
• CABook remains the primary retrieval source.
• IPCC retrieval is triggered when CABook results are weak, incomplete, or insufficient.
• The chatbot can successfully answer climate-related questions that CABook alone cannot answer.
• Responses generated using fallback information remain coherent and relevant.
• Source attribution for IPCC-derived information can be provided where appropriate.

Note: IPCC integration is intended as a fallback knowledge source. CABook remains the primary retrieval source, and IPCC content should only be consulted when CABook retrieval does not provide sufficient information to answer a user's query.