Introduction¶

In DeepSeek Harness (DSH), integrating local inference is commonly done by manually maintaining llama-server launch parameters, remembering ports and model paths, and writing OpenAI-compatible endpoints manually in the profile. Changing parameters requires aligning multiple sets of commands, and it is not easy to view logs, PIDs, and template differences simultaneously in the settings panel. dsh-plugin-llama-manager targets DSH users on Windows: it provides an independent full-page “Local Models” workspace in the left sidebar, uses multiple sets of launch templates to start/stop llama-server, and registers the local services of selected templates into the top model selector of DSH. The functional design references Local Llama Agent Manager, and the interface adopts DSH’s full-page layout.

What is this¶

dsh-plugin-llama-manager is a DSH client plugin maintained by ckilwork (current version 1.4.6, MIT license). It solves the problem of: centrally managing llama.cpp’s llama-server.exe, GGUF model paths, and launch parameters within the DSH process, forwarding requests via a local proxy to OpenAI-compatible interfaces, while optionally binding Agent prompt injection schemes that take effect per template.

Plugin data defaults to $DSH_HOME/llama-manager; if DSH_HOME is not set, it falls back to .dsh/llama-manager in the user directory. When registering models to the selector, it backs up the current profile’s cordis.patch.yml before writing the llama-manager provider.

Core Features¶

Independent Full Page and Multiple Templates¶

• A “Local Models” entry is added to the left sidebar, displayed as a full page, not occupying the settings panel space.
• Supports creating, copying, renaming, deleting, and switching multiple sets of launch templates; prompts if parameters are unsaved.

Service Start/Stop and Status¶

• Start, stop, and restart llama-server; display PID, uptime, readiness status, and reasons for abnormal exits.
• Configurable idle auto-unload: 0 (no unload) / 5 / 10 / 30 minutes or custom; stops automatically without requests to release VRAM, and restarting with the same template next time works fine.

Launch Parameters and Model Library¶

Launch parameter overrides include: main model, mmproj, draft models (MTP / EAGLE3), context length, GPU layers, port, parallel slots, Flash Attention, Jinja, inference format, KV cache type, etc. The plugin scans for .gguf, mmproj, draft models in the model directory, and llama-server.exe in the llama.cpp directory; paths can be selected from dropdowns, and recent paths are remembered.

Logs, Command Preview, and Connectivity Test¶

• Real-time logs merge stdout/stderr, color-coded by level, support filtering, auto-scroll, and clearing; can parse prompt/eval token speeds from logs.
• “Launch Command Preview” matches the real launch parameters (API Key is masked) and can be copied to PowerShell for execution (not applicable to cmd.exe).
• Sends minimal OpenAI-compatible requests to /v1/chat/completions, displays response, latency, and model name.

Integration with DSH Model Selector¶

When the plugin starts, it automatically registers the llama-manager provider (default local proxy http://127.0.0.1:18087/v1) to the current profile; the selector displays models based on the launch template of the selected model file. Empty templates are not registered. Usually, a restart of DSH is required after the first write to the profile configuration for changes to take effect. The proxy does not truncate streaming output when switching models during response generation; if the service is not ready, it switches first before forwarding.

Agent Prompt Injection Scheme¶

Supports binding and enabling proxy injection based on launch templates, disabled by default; retains the original DSH system prompt and does not rewrite the DSH Agent configuration. The process is to save the scheme first, then select and bind it in the template, and save the template.

Directory and Uninstall¶

Provides directory settings, file browsing, and opening model/data directories. One-click uninstall removes the plugin and cleans up the llama-manager provider in cordis.patch.yml, while preserving templates and logs under $DSH_HOME/llama-manager for reinstall use.

Environment Requirements¶

• DeepSeek Harness Desktop, or DSH Web running on Windows via the web client plugin mechanism.
• llama-server.exe installed from llama.cpp, and at least one .gguf model.
• Windows (currently only verified on Windows; paths and process management are implemented based on Windows).
• Node.js >= 20 is only required for build/development or executing the uninstall script; the installed plugin is run by DSH.

Installation and Activation¶

The distribution package contains a pre-built client. After downloading dsh-plugin-llama-manager-1.4.6.tgz, execute in the package directory (replace desktop with the actual profile name):

dsh plugin --profile desktop add (Resolve-Path ./dsh-plugin-llama-manager-1.4.6.tgz).Path

If dsh is not in PATH, use the full path to resources/runtime/cli/bin/dsh.cmd in the installation directory. Exit and restart DSH completely after installation. The “Local Models” icon should appear in the left sidebar.

If it doesn’t appear, you can check: if there is an error for llama-manager in DSH logs; if client/client.js exists and contains window.__ModuleLoader__; if the profile’s package.json lists dsh-plugin-llama-manager.

Seeing “Local Models” in the Model Selector¶

1. Restart DSH after the first installation or upgrade to load the new client code.
2. The plugin writes the provider to cordis.patch.yml.
3. If the configuration is not hot-loaded, restart once more; “Local Models (llama.cpp Manager)” and valid templates should appear in “Settings → Models” and the top selector. After that, you can use “Resync” in the top right of the model library.

When building from source yourself, execute in the repository root directory:

cd ./dsh-plugin-llama-manager
npm ci --ignore-scripts
npm run build

Seeing ✓ 已生成 client\client.js indicates a successful build; you can also use npm pack --ignore-scripts to generate an installation package of the same format.

Typical Usage¶

The following steps are consistent with the usage instructions in the README and can be followed in order.

1. Configure Template: Create or copy a template on the left; click on .gguf in the model library to fill in the path, select llama-server.exe, mmproj, and draft models from dropdowns or “Browse…”.
2. Check Parameters: View “Launch Command Preview” and hints about missing paths, port conflicts, etc.
3. Save Template: Click “Save Template” after changes; you can also start directly without saving (using the current draft), but switching templates will prompt for lost unsaved changes.
4. Start Service: Click “Start Service” at the top; loading large models for the first time may take tens of seconds to minutes; once ready, the status is “Running”.
5. Idle Auto-Unload: Select a duration below “Parallel Slots”; it will stop automatically when no new requests arrive, and the log will note the automatic stop.
6. Connectivity Test: Perform the connectivity test at the bottom of the log card to verify the port, model name, and API Key.
7. Agent Prompt Injection (Optional): Save the scheme on the right, bind and enable it in “Prompt Injection for Current Model Template”, and save the template; it takes effect for requests proxied by the new template.

By default, llama-server listens on 127.0.0.1:8080, and the OpenAI-compatible endpoint is http://127.0.0.1:8080/v1. If changed to 0.0.0.0 for LAN access, it is recommended to set an API Key simultaneously.

Suitable Scenarios and Notes¶

Who is it for: Users who are already doing agent or conversational development on Windows using DSH and want to use local GGUF inference while needing to integrate with DSH’s model selector and OpenAI-compatible call chains.

Permissions and Security: The plugin runs with the permissions of the current DSH process; the HTTP interface is mounted at /api/llama-manager and only accepts loopback addresses. You can view the source code and MIT license on GitHub before installing. When stopping the service, it uses TerminateProcess first, then falls back to taskkill; when DSH exits, the llama-server started by the plugin will be terminated synchronously.

Ecosystem Note: DSH adopts an “everything is a plugin” extension approach; the community plugin directory is maintained by an independent site and has no official affiliation with DeepSeek / Hypothesis. The facts in this article are derived from the plugin README and package metadata; the installation command follows the dsh plugin --profile … add …tgz in the README, please do not concatenate undocumented installation methods based solely on the repository name.

Conclusion¶

dsh-plugin-llama-manager brings llama.cpp start/stop, parameter templates, logs, and the DSH model selector into a single Windows client interface, suitable for developers who need local GGUF but don’t want to leave the DSH workflow. Page: SkillHub Plugin Page; Source code and full details: GitHub - ckilwork/dsh-plugin-llama-manager.