Reverse invoking a Model refers to the ability of a plugin to call Dify’s internal LLM capabilities, including all model types and functions within the platform, such as TTS, Rerank, etc. If you are not familiar with the basic concepts of reverse invocation, please read Reverse Invocation of Dify Services first.
However, please note that invoking a model requires passing a ModelConfig
type parameter. Its structure can be referenced in the General Specifications Definition, and this structure will have slight differences for different types of models.
For example, for LLM
type models, it also needs to include completion_params
and mode
parameters. You can manually construct this structure or use model-selector
type parameters or configurations.
Invoke LLM
Entry Point
Endpoint
def invoke(
self,
model_config: LLMModelConfig,
prompt_messages: list[PromptMessage],
tools: list[PromptMessageTool] | None = None,
stop: list[str] | None = None,
stream: bool = True,
) -> Generator[LLMResultChunk, None, None] | LLMResult:
pass
Please note that if the model you are invoking does not have tool_call
capability, the tools
passed here will not take effect.
Use Case
If you want to invoke OpenAI’s gpt-4o-mini
model within a Tool
, please refer to the following example code:
from collections.abc import Generator
from typing import Any
from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage
class LLMTool(Tool):
def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
response = self.session.model.llm.invoke(
model_config=LLMModelConfig(
provider='openai',
model='gpt-4o-mini',
mode='chat',
completion_params={}
),
prompt_messages=[
SystemPromptMessage(
content='you are a helpful assistant'
),
UserPromptMessage(
content=tool_parameters.get('query')
)
],
stream=True
)
for chunk in response:
if chunk.delta.message:
assert isinstance(chunk.delta.message.content, str)
yield self.create_text_message(text=chunk.delta.message.content)
Note that the query
parameter from tool_parameters
is passed in the code.
Best Practice
It is not recommended to manually construct LLMModelConfig
. Instead, allow users to select the model they want to use on the UI. In this case, you can modify the tool’s parameter list by adding a model
parameter as follows:
identity:
name: llm
author: Dify
label:
en_US: LLM
zh_Hans: LLM
pt_BR: LLM
description:
human:
en_US: A tool for invoking a large language model
zh_Hans: 用于调用大型语言模型的工具
pt_BR: A tool for invoking a large language model
llm: A tool for invoking a large language model
parameters:
- name: prompt
type: string
required: true
label:
en_US: Prompt string
zh_Hans: 提示字符串
pt_BR: Prompt string
human_description:
en_US: used for searching
zh_Hans: 用于搜索网页内容
pt_BR: used for searching
llm_description: key words for searching
form: llm
- name: model
type: model-selector
scope: llm
required: true
label:
en_US: Model
zh_Hans: 使用的模型
pt_BR: Model
human_description:
en_US: Model
zh_Hans: 使用的模型
pt_BR: Model
llm_description: which Model to invoke
form: form
extra:
python:
source: tools/llm.py
Please note that in this example, the scope
of model
is specified as llm
. This means the user can only select llm
type parameters. Thus, the code from the previous use case can be modified as follows:
from collections.abc import Generator
from typing import Any
from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage
class LLMTool(Tool):
def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
response = self.session.model.llm.invoke(
model_config=tool_parameters.get('model'),
prompt_messages=[
SystemPromptMessage(
content='you are a helpful assistant'
),
UserPromptMessage(
content=tool_parameters.get('query') # Assuming 'query' is still needed, otherwise use 'prompt' from parameters
)
],
stream=True
)
for chunk in response:
if chunk.delta.message:
assert isinstance(chunk.delta.message.content, str)
yield self.create_text_message(text=chunk.delta.message.content)
Invoke Summary
You can request this endpoint to summarize a piece of text. It will use the system model within your current workspace to summarize the text.
Entry Point
self.session.model.summary
Endpoint
text
is the text to be summarized.
instruction
is the additional instruction you want to add, allowing you to summarize the text stylistically.
def invoke(
self, text: str, instruction: str,
) -> str:
Invoke TextEmbedding
Entry Point
self.session.model.text_embedding
Endpoint
def invoke(
self, model_config: TextEmbeddingResult, texts: list[str]
) -> TextEmbeddingResult:
pass
Invoke Rerank
Entry Point
self.session.model.rerank
Endpoint
def invoke(
self, model_config: RerankModelConfig, docs: list[str], query: str
) -> RerankResult:
pass
Invoke TTS
Entry Point
Endpoint
def invoke(
self, model_config: TTSModelConfig, content_text: str
) -> Generator[bytes, None, None]:
pass
Please note that the bytes
stream returned by the tts
endpoint is an mp3
audio byte stream. Each iteration returns a complete audio segment. If you want to perform more in-depth processing tasks, please choose an appropriate library.
Invoke Speech2Text
Entry Point
self.session.model.speech2text
Endpoint
def invoke(
self, model_config: Speech2TextModelConfig, file: IO[bytes]
) -> str:
pass
Where file
is an audio file encoded in mp3
format.
Invoke Moderation
Entry Point
self.session.model.moderation
Endpoint
def invoke(self, model_config: ModerationModelConfig, text: str) -> bool:
pass
If this endpoint returns true
, it indicates that the text
contains sensitive content.
Edit this page | Report an issue
Responses are generated using AI and may contain mistakes.