Introduction¶

DeepSeek Harness (DSH) uses a plugin-based architecture, allowing developers to extend its functionality. In agent or high-concurrency scenarios, frequent LLM API calls can easily trigger API rate limits or cause excessive resource consumption. dsh-llm-rate-limiter is a plugin designed to provide independent call-frequency control for each model to address this issue.

Plugin Overview¶

This is an LLM call rate limiting plugin for DeepSeek Harness. It is maintained by Leafyezi233 and licensed under the MIT License. The plugin supports per-model configuration of concurrency, RPM (requests per minute), and burst limits, and provides two handling strategies: queue or reject. It also includes a built-in interactive GUI configuration panel and real-time status monitoring.

Core Features¶

1. Per-model independent rate limiting: Independent concurrency limits, RPM, and burst size can be configured for different provider/model combinations.
2. Dual algorithm support: Provides a token-bucket algorithm (allows bursts) and a sliding-window algorithm (smooth and strictly limits RPM).
3. Queue and reject modes: When rate limiting is triggered, requests can either wait in a queue or be rejected directly (can be combined with a retry plugin for backoff).
4. Interactive GUI configuration: In the DSH settings interface, you can directly expand the LLM Rate Limiting card to configure it, without editing files.
5. Real-time status panel: Displays real-time counters, progress bars, and event logs to monitor rate-limiting behavior.
6. Hot reload: Configuration changes take effect immediately without restarting the DSH process.
7. Full-chain interception: Intercepts all llm/stream calls in agent turns.

Installation and Activation¶

Installing the plugin requires the DeepSeek Harness command-line tool.

1. Run the following command in the terminal to add the plugin:

    dsh plugin add <your-profile> @leaf233/dsh-llm-rate-limiter

Replace `<your-profile>` with the directory name of your configuration profile.

2. Ensure the plugin is added to the package.json dependencies of the profile. The plugin will automatically detect the cordis.patch.yml entry.

Typical Usage¶

Configuration can be done via the GUI or by editing configuration files.

Configure via GUI¶

1. Start the web interface: dsh web.
2. Go to Settings → Plugins.
3. In the Official group, find the LLM Rate Limiting card.
4. Expand it and configure the default strategy, model-specific overrides, and throttling behavior in the form.

Configure via File¶

Edit cordis.patch.yml or settings.yaml in the profile directory, and define the top-level llm-rate-limiter strategy. Example:

llm-rate-limiter:
  enabled: true
  strategy: token-bucket       # Algorithm choice: token-bucket or sliding-window
  defaults:
    maxConcurrent: 5           # Default maximum concurrency
    maxRpm: 60                 # Default requests per minute
    burstSize: 10              # Token bucket capacity (token-bucket only)
    refillRate: 1              # Tokens refilled per second (token-bucket only)
  models:
    "deepseek/deepseek-chat":
      maxConcurrent: 8
      maxRpm: 120
    "openai/gpt-4o":
      maxConcurrent: 2
      maxRpm: 10
      burstSize: 3
  onThrottled: queue           # Behavior when rate limiting is triggered: queue or reject
  maxQueueWaitMs: 60000        # Maximum queue wait time (milliseconds)

Notes and Use Cases¶

1. Version dependency: The current version (v0.3.0) requires DSH version 0.1.7-rc.1 or higher. If using an older version, pin @leaf233/dsh-llm-rate-limiter@0.2.x.
2. Dependency check: If the framework’s connection.requestRejection is unavailable, the plugin will refuse to mount instead of publishing unauthenticated routes.
3. Git installation limitation: Installing via a GitHub URL may fail due to pnpm’s allowBuilds restrictions. In this case, add the plugin name to pnpm-workspace.yaml in the profile root directory, and then run the installation command again.
4. Ecosystem positioning: This is a community plugin with no official affiliation to DeepSeek / High-Flyer. Please review the source code and license yourself.

Summary¶

dsh-llm-rate-limiter provides a complete solution from algorithm selection to real-time monitoring, making it suitable for developers who need fine-grained control of LLM call frequency in DSH. Through its GUI panel, you can intuitively adjust strategies and observe runtime status, reducing development and debugging costs.

Plugin homepage: https://github.com/Leafyezi233/dsh-llm-rate-limiter
Community directory: https://www.skillhub.cn/plugins/Leafyezi233/dsh-llm-rate-limiter