Knowledge

​

Introduction

Knowledge in Dify is a collection of your own data that can be integrated into your AI apps. It allows you to provide LLMs with domain-specific information as context, ensuring their responses are more accurate, relevant, and less prone to hallucinations.

This is made possible through Retrieval-Augmented Generation (RAG). It means that instead of relying solely on its pre-trained public data, the LLM uses your custom knowledge as an additional source of truth:

1. (Retrieval) When a user asks a question, the system first retrieves the most relevant information from the incorporated knowledge.

2. (Augmented) This retrieved information is then combined with the user’s original query and sent to the LLM as augmented context.

3. (Generation) The LLM uses this context to generate a more precise answer.

Knowledge is stored and managed in knowledge bases. You can create multiple knowledge bases, each tailored to different domains, use cases, or data sources, and selectively integrate them into your application as needed.

​

Build with Knowledge

With Dify knowledge, you can build AI apps that are grounded in your own data and domain-specific expertise. Here are some common use cases:

• Customer support chatbots: Build smarter support bots that provide accurate answers from your up-to-date product documentation, FAQs, and troubleshooting guides.

• Internal knowledge portals: Build AI-powered search and Q&A systems for employees to quickly access company policies and procedures.

• Content generation tools: Build intelligent writing tools that generate reports, articles, or emails based on specific background materials.

• Research & analysis applications: Build applications that assist in research by retrieving and summarizing information from specific knowledge repositories like academic papers, market reports, or legal documents.

​

Set Up

• Create a ready-to-use knowledge base: Import data, then set how it’s chunked, indexed, and searched.

• Build a custom knowledge base: Design your own processing as a Knowledge Pipeline when the standard settings don’t fit your content, choosing the data sources and each step that extracts, chunks, and indexes it.

• Connect to an external knowledge base: Search a knowledge base you already run through its API, without moving the content into Dify.

​

Test and Improve

• Test and validate retrieval: Try what your users will ask and see which chunks come back.

• Adjust knowledge base settings: Change the embedding model and search settings, or switch the index method from Economical to High Quality.

• Enhance retrieval with metadata: Label documents so searches can be narrowed to the ones that match.

​

Use and Maintain

• Integrate into applications: Connect a knowledge base to your apps and set how they retrieve from it.

• Manage content: Add, edit, disable, or delete documents and chunks as your content changes.

---

Read More:

• Dify v1.1.0: Filtering Knowledge Retrieval with Customized Metadata

• Dify v0.15.0: Introducing Parent-child Retrieval for Enhanced Knowledge

• Introducing Hybrid Search and Rerank to Improve the Retrieval Accuracy of the RAG System

• Dify.AI’s New Dataset Feature Enhancements: Citations and Attributions

• Text Embedding: Basic Concepts and Implementation Principles

• Enhance Dify RAG with InfraNodus: Expand Your LLM’s Context

• Dify.AI x Jina AI: Dify now Integrates Jina Embedding Model