Preface¶
The core philosophy of DeepSeek Harness (DSH) is pluginization. When developers need to build agents that can operate desktop applications, existing solutions often require writing additional automation scripts or performing complex build configurations. The dsh-codex-style-computer-use plugin aims to solve this pain point by providing native Windows desktop control capabilities for DeepSeek Harness.
Plugin Overview¶
This plugin uses prebuilt native C# Workers to allow agents to “see” and “operate” the Windows desktop like Codex. The plugin is maintained by ltmroberthk915 and released under the MIT license.
Core Features¶
1. Observation
The plugin provides window lists, screenshots (with optional region specification, JPEG/PNG formats, and downsampling), Set-of-Marks mappings for clickable elements, and a UI Automation tree with precise AutomationIds.
2. Action
It supports clicking, moving, dragging, scrolling, selecting text ranges, typing input, and pressing keys, by driving controls through UI Automation rather than relying on coordinates.
3. Batch Execution
The computer_batch invocation can execute a series of actions sequentially and stop at the first failure. This allows known sequences to require only one model round trip instead of multiple.
4. Interactive Questions
The computer_ask invocation asks questions in the same chat card, without a countdown.
5. Manual Brake
When an action is manually interrupted, a stop signal is pushed into the session, allowing the agent to recognize that the operation has been paused instead of discovering it only in the next turn.
Installation and Enablement¶
This plugin can be installed via dsh-market without requiring PowerShell 7 or build steps.
dsh plugin --profile web add dsh-codex-style-computer-use
Environment requirements:
* Operating system: Windows (os: win32)
* Node.js: ≥ 22
* .NET Framework: 4.x (bundled with Windows)
Typical Use Cases¶
- Desktop Application Collaboration: Instruct the agent to view the screen, locate controls, and perform clicks and input.
- Manual Takeover: When you need to use your own mouse or keyboard, the agent will wait and will not continue executing.
- Sequence Execution: Use the
computer_batchinvocation to run a known set of operations in one pass. - Card-Based Questions: Use
computer_askto question the user directly in the current chat card.
Notes¶
- System Limitations: Supported only on Windows platforms.
- Privacy Protection: The plugin does not read user files. It only writes Worker binaries to
%LOCALAPPDATA%\dsh-computer-use\worker\, and writes audit logs and screenshots to$DSH_HOME/data/computer-use/. - Default Behavior: Approval prompts remain enabled by default.
Conclusion¶
By providing observation and action capabilities, this plugin lowers the barrier for DeepSeek Harness agents to interact with the Windows desktop. For more details, refer to the GitHub repository or the community directory.
