dsh-plugin-mm-vision: Adding Structured Vision Capabilities to Text-Only LLMs
dsh-plugin-mm-vision is a DeepSeek Harness (DSH) plugin designed to provide image understanding capabilities for pure text LLMs. The plugin registers the `mm_vision` tool, invoking an external vision model to convert images into structured spatial textual descriptions containing precise percentage coordinates, enabling text models to analyze spatial relationships without directly processing the images. Core features include "synesthesia encoding," applicable to candlestick charts, UI screenshots, and natural photos; it supports pixel-level coordinate localization and an optional 40x30 pixel color block grid reconstruction, and features an automatic mode that adapts output details based on the characteristics of charts or natural images. The plugin supports configuring any OpenAI-compatible vision model and provides a TTL cache mechanism to improve response speed. Installation requires the DSH environment and Node.js.
Read More