Preface¶

The philosophy of DSH is “everything is a plugin.” When having agents handle web tasks, common bottlenecks aren’t just “can it click” or “can it fill out forms,” but also include login, credential boundaries, session persistence, and whether the page’s visual state can be understood. Existing approaches that simply call a headless browser often can’t show a window, can’t allow manual takeover for login, and sessions are easily lost.

dsh-browser-companion addresses these issues by providing a browser plugin for DSH: it gives the agent a persistent profile, a visible window, a manual login channel, and a set of relatively safe browser operation tools.

What This Is¶

dsh-browser-companion is a personal DSH browser plugin. It provides the agent with a dedicated, persistent, and visible browser environment. The core goals are:

• The agent can open pages, click, type, scroll, manage tabs, etc.;
• Users can complete login, enter passwords, or complete OAuth in the real window;
• Passwords and OAuth secrets don’t pass through the agent;
• Cookies, localStorage, and login state are saved in the same profile and can be retained after restarts.

The license is MIT. The GitHub repository address is:

https://github.com/Tianyu209/dsh-browser-companion

Core Features¶

Persistent Browser Profile¶

The plugin preserves login state within the browser profile. Verified capabilities include:

• Persistent cookies;
• Persistent localStorage;
• Persistent login state.

This means that once a site is logged into, subsequent tasks can reuse the same profile instead of re-authenticating every time.

Visible Window¶

The plugin provides a visible window mode, with the default configuration:

headless: false

This means that when the agent operates the browser, users can see the actual browser window. This is important for observing agent actions and manually taking over the login process.

Manual Login¶

The plugin provides browser_login and browser_wait_login:

• browser_login opens a visible login window where the user enters passwords or completes OAuth themselves;
• browser_wait_login waits for the login to complete, checking criteria such as cookies / URL.

Passwords and OAuth secrets are handled by the user in the browser window and don’t pass through the agent.

Browser Operation Tools¶

The verified agent tools include:

open, snapshot, click, type, press, screenshot, see, select, check, upload, hover, scroll, tabs, status

A few commonly used tools:

• open: Opens a page;
• snapshot: Gets accessibility snapshot / refs;
• click / type / press: Basic interactions;
• select / check: Form control operations;
• upload / hover / scroll / tabs / status: More granular page operations.

Security Boundaries¶

The plugin restricts sensitive inputs:

• browser_type refuses to fill in password fields;
• The agent is instructed not to fill in password fields;
• Passwords and OAuth secrets are entered by the user in the visible browser window.

Visual Understanding¶

browser_see uses a local vision model (OpenAI-compatible) to help the agent understand images, visual layouts, or complex interfaces. By default, browser_snapshot should be preferred because it’s faster, lower cost, and comes with refs; only use browser_see when visual judgment is needed.

If browser_see is not configured, it returns a friendly message rather than interrupting the agent.

Installation and Enablement¶

First, install the dependencies:

npm install -g agent-browser
agent-browser install

Then add the plugin to the web profile:

dsh plugin --profile web add dsh-browser-companion

Finally, start or restart DSH:

dsh web

If using pnpm and installing from GitHub encounters the agent-browser build script being blocked, add the following to the profile’s pnpm-workspace.yaml:

allowBuilds:
  agent-browser: true

Then reinstall.

Typical Usage¶

Here’s a login flow:

1. The agent needs to access a site, but logging in is required first.

2. The agent calls browser_login to open a visible login window.

3. The user completes the login in the window, entering passwords or completing OAuth authorization.

4. The user informs the agent that the login is complete.

5. The agent calls browser_wait_login to wait for the login state to be confirmed.

6. The agent continues subsequent operations using the same profile’s cookies.

Page reading also follows a simple principle:

1. Use browser_snapshot by default for a fast, free accessibility tree with refs.

2. Only use browser_see when dealing with images, visual layouts, or complex UIs.

3. If the local vision model is not configured, browser_see returns a friendly prompt.

Optional Configuration¶

You can modify the following configuration items in the DSH WebUI plugin settings:

• executablePath: Specifies the browser executable;
• visionBaseUrl / visionModel: Configures the local vision model for browser_see;
• profilePath: Specifies the browser profile path.

Modify these only when you need to customize the browser, vision model, or profile location.

Use Cases and Considerations¶

Suitable for the following scenarios:

• DSH agents need to complete web operations, but login must be done manually;
• Browser login state needs to be maintained to avoid re-authentication for each task;
• A visible browser window is needed for manual observation and takeover;
• The agent needs to understand images, visual layouts, or complex UIs.

Notes before use:

• The plugin runs locally and executes with the current dsh process permissions; review the source code and license before installing.
• Don’t commit real paths, tokens, or personal profile data; use sample configurations and replace values locally.
• Passwords and OAuth secrets should always be handled by the user in the visible browser window, not passed to the agent as plain text.
• browser_see depends on the local vision model configuration; if not configured, it won’t interrupt the agent but will also not provide visual understanding results.

Conclusion¶

The value of dsh-browser-companion is quite specific: it separates “agent operating the browser” from “manual login / credential boundaries,” while maintaining a visible window and persistent profile. For scenarios where you want DSH to handle real web tasks without handing passwords to the agent, it’s a fairly straightforward plugin.

GitHub repository:

https://github.com/Tianyu209/dsh-browser-companion

DSH Community Directory: You can search for the plugin by name dsh-browser-companion in the DSH community directory; this article does not provide unverified directory page URLs.