Introduction to the DSH Multimodal Plugin
The `dsh-multimodal` plugin is designed to address the limitation in DeepSeek Harness (DSH) where native text models cannot process images or other non-text files. It intercepts inputs before the session model and, through a separately configured processing model chain, converts images, videos, and audio into text descriptions before further processing. It supports preset management, intelligent fallback, custom prompts, and file size limits. Users can install it via the CLI and enable it in the settings to submit multimodal content by pasting, dragging and dropping, or specifying attachment paths. The plugin also provides flexible failure-handling strategies and a text-native passthrough capability.
Read Moredsh-multimodal: Integrating Visual Transcription, OCR, and Image Generation into DeepSeek Harness
DSH-Multimodal is a plugin for DeepSeek Harness (DSH), designed to address the pain point of needing to convert screenshots, error images, and design mockups into text in text-based conversations, while also integrating generated images directly into the session. The plugin offers three core features: visual transcription, OCR, and text-to-image generation, invoking user-configured vision providers and image services through an OpenAI-compatible backend. For visual processing, the plugin supports word-by-word transcription of images before DeepSeek continues processing, featuring LRU caching, multi-image parallelism, failover, and cost-based routing capabilities. OCR supports multiple output formats and scenario-based presets. The text-to-image feature supports displaying cards within the session, with zoom, download, and retry functionality, as well as support for custom backend integration. The plugin supports integration with mainstream platforms such as Zhipu, Bailian, and local...
Read More