Reverse Invocation Model

Invoke LLM, embedding, rerank, TTS, speech-to-text, and moderation models from your plugin

A plugin can reverse invoke Dify’s internal LLM capabilities, including all model types and functions within the platform, such as TTS and Rerank. If you are unfamiliar with the basics of reverse invocation, first read Reverse Invocation of Dify Services.

Every model invocation takes a ModelConfig type parameter. Its structure is defined in the General Specifications Definition and varies slightly by model type.

For example, LLM type models also require completion_params and mode parameters. You can construct this structure manually or use model-selector type parameters or configurations.

Invoke LLM

Entry Point

    self.session.model.llm

Interface

    def invoke(
        self,
        model_config: LLMModelConfig,
        prompt_messages: list[PromptMessage],
        tools: list[PromptMessageTool] | None = None,
        stop: list[str] | None = None,
        stream: bool = True,
    ) -> Generator[LLMResultChunk, None, None] | LLMResult:
        pass

If the model you are invoking does not have tool_call capability, the tools passed here will not take effect.

Use Case

This example invokes OpenAI’s gpt-4o-mini model within a Tool:

from collections.abc import Generator
from typing import Any

from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage

class LLMTool(Tool):
    def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
        response = self.session.model.llm.invoke(
            model_config=LLMModelConfig(
                provider='openai',
                model='gpt-4o-mini',
                mode='chat',
                completion_params={}
            ),
            prompt_messages=[
                SystemPromptMessage(
                    content='you are a helpful assistant'
                ),
                UserPromptMessage(
                    content=tool_parameters.get('query')
                )
            ],
            stream=True
        )

        for chunk in response:
            if chunk.delta.message:
                assert isinstance(chunk.delta.message.content, str)
                yield self.create_text_message(text=chunk.delta.message.content)

Note that the code passes the query parameter from tool_parameters.

Best Practice

Avoid constructing LLMModelConfig manually. Instead, let users select the model they want in the UI by adding a model parameter to the tool’s parameter list:

identity:
  name: llm
  author: Dify
  label:
    en_US: LLM
    zh_Hans: LLM
    pt_BR: LLM
description:
  human:
    en_US: A tool for invoking a large language model
    zh_Hans: 用于调用大型语言模型的工具
    pt_BR: A tool for invoking a large language model
  llm: A tool for invoking a large language model
parameters:
  - name: prompt
    type: string
    required: true
    label:
      en_US: Prompt string
      zh_Hans: 提示字符串
      pt_BR: Prompt string
    human_description:
      en_US: used for searching
      zh_Hans: 用于搜索网页内容
      pt_BR: used for searching
    llm_description: key words for searching
    form: llm
  - name: model
    type: model-selector
    scope: llm
    required: true
    label:
      en_US: Model
      zh_Hans: 使用的模型
      pt_BR: Model
    human_description:
      en_US: Model
      zh_Hans: 使用的模型
      pt_BR: Model
    llm_description: which Model to invoke
    form: form
extra:
  python:
    source: tools/llm.py

Because the scope of the model parameter is llm, users can only select llm type models. The previous use case then becomes:

from collections.abc import Generator
from typing import Any

from dify_plugin import Tool
from dify_plugin.entities.model.llm import LLMModelConfig
from dify_plugin.entities.tool import ToolInvokeMessage
from dify_plugin.entities.model.message import SystemPromptMessage, UserPromptMessage

class LLMTool(Tool):
    def _invoke(self, tool_parameters: dict[str, Any]) -> Generator[ToolInvokeMessage]:
        response = self.session.model.llm.invoke(
            model_config=tool_parameters.get('model'),
            prompt_messages=[
                SystemPromptMessage(
                    content='you are a helpful assistant'
                ),
                UserPromptMessage(
                    content=tool_parameters.get('prompt')
                )
            ],
            stream=True
        )

        for chunk in response:
            if chunk.delta.message:
                assert isinstance(chunk.delta.message.content, str)
                yield self.create_text_message(text=chunk.delta.message.content)

Invoke Summary

This interface summarizes a piece of text using the system model within your current workspace.

Entry Point

    self.session.model.summary

Interface

    def invoke(
        self, text: str, instruction: str,
    ) -> str:

• text: The text to summarize.
• instruction: Additional instructions, letting you control the style of the summary.

Invoke TextEmbedding

Entry Point

    self.session.model.text_embedding

Interface

    def invoke(
        self,
        model_config: TextEmbeddingModelConfig,
        texts: list[str],
        input_type: EmbeddingInputType = EmbeddingInputType.QUERY,
    ) -> TextEmbeddingResult:
        pass

Invoke Rerank

Entry Point

    self.session.model.rerank

Interface

    def invoke(
        self, model_config: RerankModelConfig, docs: list[str], query: str
    ) -> RerankResult:
        pass

Invoke TTS

Entry Point

    self.session.model.tts

Interface

    def invoke(
        self, model_config: TTSModelConfig, content_text: str
    ) -> Generator[bytes, None, None]:
        pass

The bytes stream returned by the tts interface is an mp3 audio byte stream, and each iteration returns a complete audio segment. For more in-depth processing, choose an appropriate audio library.

Invoke Speech2Text

Entry Point

    self.session.model.speech2text

Interface

    def invoke(
        self, model_config: Speech2TextModelConfig, file: IO[bytes]
    ) -> str:
        pass

Here, file is an audio file encoded in mp3 format.

Invoke Moderation

Entry Point

    self.session.model.moderation

Interface

    def invoke(self, model_config: ModerationModelConfig, text: str) -> bool:
        pass

A return value of true indicates that the text contains sensitive content.

Related Resources

• Reverse Invocation of Dify Services - Understand the fundamental concepts of reverse invocation
• Reverse Invocation of App - Learn how to invoke Apps within the platform
• Reverse Invocation of Tool - Learn how to invoke other plugins
• Model Plugin Development Guide - Learn how to develop custom model plugins
• Model Designing Rules - Understand the design principles of model plugins

---

Edit this page | Report an issue