dsh-desktop-view:在 DSH Web UI 中观看并接管 Agent 所在桌面

Introduction

When running desktop Agents in DeepSeek Harness (DSH), a common situation is: the model clicks windows on the physical device, inputs CAPTCHAs, scans QR codes, and you can only see text descriptions in the chat. It is difficult to confirm the real-time interface state, and it is hard to take over a step or two at critical moments. Traditional approaches often involve opening a separate port for remote desktop or exposing credentials to the browser, resulting in high security and operational costs.

The following introduces the community plugin dsh-desktop-view (maintainer: soffits, category: Client). It takes the desktop stream generated by cua-spacesd and relays it to the right side of the interface via DSH’s existing login and web services; it defaults to viewing only, and requires manual “Take Control / Give Back” when needed, without opening additional ports or subdomains.

What is this

dsh-desktop-view provides a real-time view and audio of the machine running the Agent in the DSH Web UI: you can see and hear the operations on the desktop; in scenarios like CAPTCHAs, QR code logins, or accidental pop-ups, you can temporarily take over and return control so the Agent can continue. The stream and tickets remain on the server side, and the browser only holds a short-lived relay ticket, bound to your DSH login.

Core Features

Interface Entry

  • Right Sidebar Desktop Tab: Can be accessed from the sidebar guide or from the monitor button in the session header.
  • Fullscreen Page /desktop-view/, provided by your DSH same-origin; adapted for mobile, supports touch, and can be opened via a direct link.
  • URL Parameter ?control=1: Direct access to control mode. When the Agent needs assistance, it can send
    /desktop-view/?control=1&help=1&session=<its session id>, and after returning control, you can wake it up to continue the task.

Viewing and Controlling

  • Default is viewing mode: Visible and audible, clicks, key presses, clipboard, and file operations are rejected.
  • Select Take control to enter control mode; Give back returns to viewing mode. Control ends automatically after a long period of no input or when the view is closed.
  • Even if the Desktop tab is hidden (switching to other sidebars, collapsing the sidebar, or changing sessions), the control state is retained; the stream pauses, and viewing resumes when opened again.

Capabilities in Control Mode

While holding control, you can use the full capabilities of the desktop viewer: mouse, keyboard, and touch, on-screen keyboard, clipboard sync, drag-and-drop upload, file browsing and download, monitor selection, and scaling; if supported by the desktop, the microphone can also be used.

Collaboration with the Agent

The plugin writes a small JSON to stateFile, which the Agent should read before operating on the desktop. Every change in control rights notifies the session that opened this view. Relevant prompts will be displayed in the chat as brief Desktop control lines (depending on the notifyAgent configuration).

stateFile example:

{
  "controller": "user",
  "since": "2026-11-08T12:00:00.000Z",
  "sessionId": "01H…",
  "updatedAt": "2026-11-08T12:00:04.120Z"
}

When the user holds control, controller is "user"; otherwise it is null, and it may record lastReason (released, idle, disconnected, replaced, stopped). The script or Skill driving the desktop should yield operations when controller is not null.

Security Considerations (README Summary)

  • Requests on the plugin path go through DSH access control: Host/Origin fences, signed Cookies, accessible only by the owner.
  • cua-spacesd’s env token and the viewer ticket issued by the plugin do not leave the Host process; the browser only holds a short-lived relay ticket.
  • Viewing mode uses a read-only viewer ticket; the daemon side rejects input, clipboard, and files; only one view can hold control at a time.
  • The plugin page uses no-store, no referrer, and strict same-origin CSP.

The displayed desktop viewer is provided by the runtime cua-spacesd (Cua FSL-1.1-MIT); this plugin is independent of Cua and does not include or redistribute that viewer.

Environment and Dependencies

Please confirm the following before installation:

Item Requirement
DSH 0.2.2-rc.1 or higher (engines specifies dsh >=0.2.2-rc.1 <0.3.0-0)
cua-spacesd 0.12 or higher, running on the same machine as DSH; DSH process can access env token (usually 127.0.0.1:3211 on Cua desktop mirror, token at /run/cua/env-token)
Node >= 22 (package.json engines)
Browser Chrome, Edge, Firefox, or Safari 17+; mobile support for iOS Safari and Android Chrome

Installation and Enablement

The official recommendation is to add directly from GitHub (web profile):

dsh plugin --profile web add github:soffits/dsh-desktop-view

You can also add the corresponding bundle in the plugin manager in the DSH Web UI. You must restart DSH after adding.

The DSH ecosystem emphasizes “Everything is a plugin”; community directories like SkillHub are independent sites with no official affiliation to DeepSeek/Hyperbolic, and the installation steps in this article are based on the repository README.

Configuration Options

Each option appears in the fields of the desktop-view plugin row in the Web UI:

Config Default Description
endpoint http://127.0.0.1:3211 cua-spacesd listening address; it is advisable to keep it loopback, as this address receives the env token
tokenFile /run/cua/env-token env token file path, always stays on the server side
maxFps 30 Viewer requested frame rate; lowering can reduce CPU and bandwidth usage
clipboard true Whether to sync clipboard when taking control
filesRoot ~ Desktop directory that can be browsed, uploaded, and downloaded when taking control; leave empty to disable file transfer
microphone true Whether to allow sending microphone input to the desktop when taking control and if desktop support is available
idleSeconds 120 Seconds of no input before automatically returning from control to viewing
stateFile /tmp/dsh-desktop-view/control.json Path where the plugin writes control state for the Agent to read
notifyAgent wake How to notify the session when control rights change: wake / quiet / off (see README for details on wake behavior)

Typical Usage

  1. Install the plugin and restart DSH, confirming that local cua-spacesd is running and the token path is accessible.
  2. In the Web UI, open the session currently using the desktop. Enter from the right Desktop or the session header monitor to first verify Agent operations in viewing mode.
  3. When encountering an interface requiring human intervention, click Take control to complete CAPTCHAs, logins, or dialog handling; when clipboard or files are needed, use sync and upload/download in control mode (subject to clipboard and filesRoot constraints).
  4. After completion, click Give back, or rely on idle timeout to return automatically; if the Agent previously sent a link with help=1, it may be woken to continue after you return control based on the notifyAgent strategy.
  5. On mobile devices, you can directly open /desktop-view/ or a link with ?control=1 to complete the same flow in a touch environment.

Applicable Scenarios and Notes

Suitable for: Development and operations scenarios where you need to watch, listen to, and take control of a desktop from a browser without exposing additional ports, for Agents running on DSH that depend on a real desktop environment (e.g., Cua).

Pre-installation Advice:

  • The plugin runs with current DSH process permissions; in control mode, file and clipboard capabilities are limited by configuration, but you should still evaluate based on the source code and MIT license.
  • Use only in environments consistent with the README: cua-spacesd, DSH version, and loopback token assumptions; mismatches can cause stream pull failures or ticket signing failures.
  • If a custom Skill drives the same desktop, be sure to read stateFile periodically and stop automation clicks/inputs while the user is in control to avoid competing for control.

Conclusion

dsh-desktop-view brings the audio-visual stream of the machine where the Agent resides into the DSH same-origin Web UI: watch by default, take control when needed, return control to continue, and leverage DSH login and relay to control exposure. If you are searching for client-type plugins on SkillHub or GitHub, you can jump to the directory page soffits/dsh-desktop-view; for complete documentation and issue feedback, see the GitHub repository.

羽毛球分组比赛记分
小程序二维码

欢迎使用《羽毛球分组比赛记分》微信小程序

Xiaoye