Preface¶

In the chat window of the DSH web GUI, users can paste or upload image attachments in the input area; however, text models typically cannot directly process image blocks. The dsh-llm-vision-bridge solves this problem: it routes requests containing images to a vision model, generates text descriptions via the vision model, and then passes these descriptions back to the DeepSeek text model to continue the conversation.

Below, we introduce its positioning, capabilities, installation methods, and typical usage.

What This Is¶

The repository Einskyle/dsh-llm-vision-bridge provides a DSH plugin licensed under MIT. It targets the DSH web GUI, delegating image attachments to a configured vision model (e.g., pi-ai / llama.cpp Qwen3-VL), and then uses the generated text descriptions to continue the conversation with the DeepSeek text model.

It registers a deepseek-vision provider at DSH’s LlmAdapter seam, allowing conversations to route image requests to this bridging provider.

Core Capabilities¶

Below are the key behaviors provided by this plugin:

• Registers the deepseek-vision provider, integrating with the DSH LlmAdapter seam.
• When a request includes images, routes the image attachments to the configured vision model and passes the text descriptions generated by the vision model to DeepSeek to continue the conversation.
• When a request does not include images, it directly passes the request to a fallback provider, maintaining zero additional overhead.
• Uses an LRU description cache to avoid repeated descriptions of the same “image + prompt” combination.
• Automatically retries retryable 503, 429, and timeout errors.
• Supports configurable failure policies: placeholder or error.

Installation and Activation¶

First, install the plugin. The GitHub installation command is as follows:

dsh plugin --profile web add github:Einskyle/dsh-llm-vision-bridge

After installation, restart the web service:

pnpm dsh web

Before sending images, you need to set the conversation or main model to the bridging provider, for example:

agent-default-model.provider: deepseek-vision

If the current conversation or main model is not deepseek-vision, the host’s image admission check may fail, resulting in MODEL_DOES_NOT_SUPPORT_IMAGES.

Typical Usage¶

Here’s a step-by-step walkthrough:

1. Configure the bridging options under Settings → Models → llm-vision-bridge, including fields such as visionProvider, visionModel, and onVisionFailure.

2. Set the main model to deepseek-vision:

agent-default-model.provider: deepseek-vision

3. In the chat composer, paste or upload an image (supported formats are PNG, JPEG, WebP, GIF), optionally attach a question, and send. The plugin will first have the vision model generate a description, then let DeepSeek respond based on that description.

4. If you only need plain text mode, switch the main model back to deepseek-official:

agent-default-model.provider: deepseek-official

At this point, image uploads will be rejected by admission, which is expected behavior.

Changes made in Settings take effect without requiring a restart.

Vision Model Side Requirements¶

The vision calls use the pi-ai adapter, so the vision provider needs to be configured on the pi-ai adapter. The vision provider cannot be the bridging provider itself—that is, it cannot point to deepseek-vision—otherwise, recursion occurs.

The local llama.cpp gateway is the default vision route, but it is not mandatory; any endpoint that accepts image input and is OpenAI-compatible can be used.

If the vision provider declares apiKeyEnv but credentials are missing, pi-ai will report MISSING_CREDENTIAL. When using cloud-based OpenAI-compatible routes, existing stored credentials are required.

Applicable Scenarios and Notes¶

This plugin is suitable for scenarios where DSH web GUI is already in use and there is a desire to let text models leverage external vision models to handle image attachments. It does not feed images directly into the DeepSeek text model; instead, it uses a “describe first, then answer” approach to keep the text model engaged in the conversation.

Notes for usage:

• The conversation or main model must support image admission, typically by setting it to deepseek-vision.
• The vision provider must be independent of the bridging provider and cannot be configured as deepseek-vision.
• Cloud-based vision endpoints require correctly configured credentials; otherwise, MISSING_CREDENTIAL may occur.
• The plugin runs with the current dsh process permissions; inspect the source code and the MIT license before installation.
• Retryable 503, 429, and timeout errors are automatically retried; on final failure, the onVisionFailure policy is applied.

Repository¶

GitHub repository link:

https://github.com/Einskyle/dsh-llm-vision-bridge