Preface¶

In daily usage of DSH, many conversations are handled by text models. If you want the model to process images, the common approach is to manually send the image to an external vision service and paste the result back into the conversation. This becomes cumbersome in continuous tasks.

Favio8/dsh-plugin-deepeye turns vision capabilities into a native Cordis plugin for DSH: it first provides a set of callable tools, and then handles the scenario of “pasting images into pure-text conversations.”

What This Is¶

dsh-plugin-deepeye is maintained by Favio8 and is licensed under MIT.

It is a native Cordis plugin that provides vision capabilities for DeepSeek Harness (DSH), covering image description, OCR text extraction, visual question answering, UI layout analysis, clipboard screenshot analysis, and more.

Core Features¶

Here are some of the main capabilities.

• Tool-based vision calls: Provides vision_describe, vision_ocr, vision_ask, vision_layout, and vision_clipboard tools.
• Multiple backends: Supports OpenAI (GPT-4o), Google Gemini, and custom OpenAI-compatible endpoints.
• Paste image compatibility: Supports automatic translation of pasted images (pasteCompat).
• Result caching: Supports LRU result caching (with TTL).
• Image preprocessing: Automatically scales down oversized images and converts them to JPEG to save tokens.
• Clipboard reading: Non-blocking asynchronous reading of the system clipboard.

Installation and Activation¶

First, make sure your environment meets the requirement of Node >=22.19.0. Then run:

dsh plugin --profile web add dsh-plugin-deepeye

After the above steps, check whether the plugin layer appears in the profile configuration:

dsh --profile web --dump-config

Confirm that the dsh-plugin-deepeye layer appears in the output. To start:

dsh web

Here, web is an alias for --profile web.

Configuring OpenAI¶

Set the OPENAI_API_KEY environment variable, then configure provider: openai in cordis.yml.

export OPENAI_API_KEY=sk-xxx

- id: deepeye-vision
  name: 'dsh-plugin-deepeye'
  config:
    provider: openai

Configuring Custom Endpoints¶

Custom OpenAI-compatible endpoints are typically overlaid using patch files. Example command:

dsh web --patch ./examples/zhipu-glm4v.cordis.patch.yml

When using a custom provider, baseUrl and model in the example are required.

Paste Image Compatibility¶

The plugin supports pasteCompat, which can be set to off, auto, or force, with auto as the default.

- id: deepeye-vision
  config:
    pasteCompat: auto

Notes¶

• The plugin runs with the permissions of the current dsh process. Check the source code and license before installation.
• If sharp is not installed, the plugin will still run, but images will not be scaled or converted to JPEG, and a one-time warning will appear in the startup logs.
• Licensed under MIT.

Links¶

GitHub repository: https://github.com/Favio8/dsh-plugin-deepeye