Preface¶
A common issue when building agents in DeepSeek Harness (DSH) is that the main model only processes text and cannot directly understand local image paths or network image URLs. The existing approach usually involves writing additional scripts to call a multimodal model API and then feeding the textual result back into the conversation. DSH’s “everything is a plugin” philosophy allows packaging such capabilities into bundle plugins, which can be registered as tools for the agent to call.
The following introduces dsh-vision. It sends images to Alibaba Cloud Bailian or Zhipu multimodal models and returns Chinese text descriptions, providing image recognition capabilities for DSH models that lack native image understanding.
What Is This¶
dsh-vision is a DSH bundle plugin maintained by Terry12138qy, licensed under MIT. It adapts the main image recognition workflow of asuojun/claude-vision-skill into a DSH plugin and registers a vision tool.
This plugin solves the problem in DSH conversations where the model “sees an image path but cannot directly understand the image.” It encapsulates the steps of reading the image, base64 encoding, calling the multimodal API, and returning text descriptions, making them available for agents or Code Mode.
Core Features¶
The verified capabilities of dsh-vision currently include:
- Accepts local image paths or network URLs.
- Sends images to multimodal large models and returns Chinese text descriptions.
- Primary model is Alibaba Cloud Bailian (DashScope)
qwen3.5-omni-plus. - Automatically switches to the Zhipu
glm-4.6v-flashfallback model and retries on failure. - Failure reasons include exhausted quota, overdue payment, and rate limiting.
- Packaged as a DSH bundle plugin, registering the
visiontool. - Can be invoked in Code Mode via
await tools.vision(...). - Supports cancellation via
exec.signal. - Configuration priority: environment variables >
cordis.patch.ymlconfig > default values.
Installation and Enabling¶
First, execute the installation command:
dsh plugin --profile web add github:Terry12138qy/dsh-vision
If installing via a local path:
dsh plugin --profile web add file:/absolute/path/dsh-vision
After installation, you need to restart dsh web for the plugin to take effect.
Configuration¶
It is recommended to configure API keys via environment variables to avoid committing keys to the repository; you can also edit the config in cordis.patch.yml. Environment variables take precedence over the config in cordis.patch.yml.
| Purpose | Environment Variable | config Field |
|---|---|---|
| Primary model API Key | DASHSCOPE_API_KEY |
apiKey |
| Primary model Base URL | DASHSCOPE_BASE_URL |
baseUrl |
| Primary model name | VISION_MODEL |
model |
| Fallback model API Key | FALLBACK_API_KEY |
fallbackApiKey |
| Fallback model Base URL | FALLBACK_BASE_URL |
fallbackBaseUrl |
| Fallback model name | FALLBACK_MODEL |
fallbackModel |
The API Key, Base URL, and model name for both the primary and fallback models can be configured using the fields above. Once configured, the plugin will prioritize values from environment variables.
Typical Usage¶
After configuration, simply send an image path or URL in the conversation, and the agent will invoke the vision tool. Example:
Take a look at this image C:\path\to\image.png
You can also call it directly in Code Mode:
await tools.vision(...)
When the primary model call fails, the plugin will switch to the fallback model and retry, ultimately returning a text description.
Use Cases and Notes¶
dsh-vision is suitable for scenarios where the main model in DSH does not directly process images but needs to convert images into text descriptions for subsequent Q&A, summarization, code analysis, or document organization.
Before use, note the following:
- The plugin runs with the current
dshprocess permissions; you should review the source code, dependencies, and license before installation. - This adaptation code is licensed under the MIT License.
- The main image recognition workflow in
lib/vision.jsis ported fromasuojun/claude-vision-skill; you must retain its attribution and comply with the original project’s requirements. - It is recommended to place API keys in environment variables and avoid committing API keys to the repository.
- If you publish to GitHub later, remember to add the
dsh-plugintopic to the repository to make it discoverable within the DSH ecosystem.
Conclusion¶
The value of dsh-vision lies in encapsulating the image recognition workflow into an installable and callable vision tool for DSH, allowing models without native image understanding to accomplish vision tasks with the help of multimodal models. Plugin directory page: https://www.skillhub.cn/plugins/Terry12138qy/dsh-vision, GitHub repository: https://github.com/Terry12138qy/dsh-vision. The community directory is an independent site, not an official app store, and does not constitute an official affiliation with DeepSeek / High-Flyer.
