Introduction¶
DeepSeek Harness (DSH) is a plugin-based agent development framework. When developing LLM calls with the Web Profile, developers often encounter issues such as request failures caused by upstream Provider rate limiting, difficult-to-manage retry logic, and inconsistent model catalogs. leaves615/dsh-llm-ctl aims to solve these pain points by taking over the llm/stream and agent/request-error seams, enforcing admission control before requests are sent, and providing independent request recovery and model-management capabilities.
Core Capabilities¶
This plugin mainly includes the following four capabilities:
1. Rate-Limit Queuing¶
All llm/stream calls, including Agent main-loop calls and title generation, must first acquire a Provider concurrency slot before being sent.
* Strict FIFO: Requests are released in first-in, first-out order.
* Cooldown Mechanism: When an upstream 429 error is encountered, the plugin puts the entire Provider into a cooldown period based on the Retry-After response header or a local exponential backoff policy. Queued requests wait until the cooldown ends instead of failing immediately.
* Wait Budget: A maxWaitMs wait budget is configured. If the budget is exceeded, the plugin reports QUEUE_TIMEOUT, avoiding prolonged ineffective waits.
2. Independent Self-Healing¶
On the agent/request-error seam, the plugin first records the cooldown state and then consults downstream.
* Fallback Retry: If the dsh-llm-retry plugin is not installed, this plugin enables bounded-budget retry, with a default of 3 attempts. This means that even without a retry plugin, the first 429 error will not cause the task to fail immediately.
* Resource Isolation: This plugin only intervenes in retry handling when downstream has not already taken over retry logic.
3. Model Hiding¶
Provides two-level switches for Provider-level and single-model hiding, persisted in the llm-ctl configuration.
* Persistence and Restore: Configuration is stored in DSH settings and supports one-click restoration of all hidden states.
* Filtering and Fallback: If the default model is hidden, the system automatically falls back to the first visible model.
* Menu Integration: Switches are displayed on the model settings page, and the model menu supports filtering by the p: prefix.
4. Upstream Discovery¶
Provides the ability to refresh Provider model catalogs.
* Rediscovery: Click the refresh button on a Provider card to re-fetch the upstream model list.
* Security Mechanism: Discovery requests are executed server-side using the saved Credential and do not pass through the browser channel, reducing the risk of key leakage.
Installation and Enablement¶
Prerequisites: DSH web profile, Node.js >= 22. This plugin has been tested against @deepseek-ai/dsh-llm 0.1.2-rc.1.
- Run the following command to install the plugin:
dsh plugin --profile web add -w @leaves615/dsh-llm-ctl
- Restart the DSH Web service to load the plugin:
dsh web
Typical Usage¶
After installation, the plugin works directly with default configuration.
* View Switches: Go to the model settings page. Each Provider shows an additional control card.
* Queue Control: Go to the LLM Queue Control section in the settings navigation, then adjust the concurrency limit, queue depth, and wait budget.
* Menu Filtering: Enter p:provider_name in the model menu to filter models by Provider.
Notes¶
- Permissions and Limitations: This plugin runs with the current DSH process permissions. It does not perform billing statistics, token-level rate limiting, or cross-process distributed queuing, and it does not modify Adapter protocol logic.
- Known Issues: The server-side
session.modelCatalogcurrently cannot intercept hiding rules, so filtering is only effective on the browser side. It also does not write custom session log events. - Ecosystem Relationship: This plugin is positioned as a gatekeeper. It does not conflict with the
dsh-llm-retryplugin, which is responsible for executing retries, and it coexists with thedsh-model-search-plugin, which only searches and does not hide models.
Conclusion¶
leaves615/dsh-llm-ctl adds logic at key DSH seams to address Provider rate-limit management and model visibility control. It is suitable for developers who need fine-grained control over LLM call flows and resource utilization.
