Foreword¶
When using DSH to connect to a proxy or official models, you often encounter a problem: with the same request, some interfaces have a fast first byte but slow token generation, while others are the opposite. It is hard to tell just by looking at the conversation response body, and manually scripting the requests is also troublesome. The dsh-speed-monitor plugin puts this into DSH’s own interface: open the sidebar, and the right panel sends streaming requests to the model at fixed intervals, plotting the first byte, total time, and throughput as curves.
It is maintained by doraasn, categorized as “Workflow,” with the philosophy of “speed testing without leaving DSH”: the model list is read directly from DSH settings, requests hit the baseURL configured by the provider, and keys use DSH’s own credentials file. Below is an introduction to its features, installation, and usage.
Positioning and Maintenance¶
The plugin name is dsh-speed-monitor, current version 0.9.8, MIT licensed. The package.json declares a requirement for the dsh engine of >=0.2.1-rc.2. The repository is at https://github.com/doraasn/dsh-speed-monitor, and the community directory page is at https://www.skillhub.cn/plugins/doraasn/dsh-speed-monitor. The directory page is an independent site with no official affiliation to DeepSeek / Huafan.
Core Features¶
Model list comes from DSH. The Client reads the llm-* namespace from /api/settings/describe, merging llm-pi-ai.providers with llm-deepseek.models. The plugin does not hardcode models. New providers added after leaving the panel will appear in the dropdown after restarting. The model display name is also consistent with DSH, using the combination of provider displayName + model name, for example DeepSeek · DeepSeek-V4.1-Flash.
Multiple models side-by-side. Each model on a line, can be added/removed, and the model and thinking level for each row are saved independently. The default target is written as astra-hub / deepseek-v4.1-flash, falling back to the first model in the list on errors.
Sampling interval. Defaults to one round per minute, selectable as 1 / 5 / 15 / 30 / 60 minutes. For each model per round, it warms up once then samples 3 times, taking the median for storage.
Single metric curves. Switch between First Byte / Total Time / Throughput at the top. Each model has one line, and the legend can be clicked to hide. Hovering displays the three values for that sample. Latency unit is milliseconds (higher is slower), throughput unit is tok/s (higher is faster).
No lines connecting gaps. When the interval between adjacent sampling points exceeds 3 times the median interval or 2 minutes, the curve will break. This is a deliberate design, not data loss.
Sampling Method¶
Values are not obtained by a single point; the method refers to term-llm benchmark and llm-speed-test:
| Step | Action | Reason |
|---|---|---|
| Warmup | Discard the first request for each model per round | Exclude DNS, TCP connection establishment, and prefix cache cold starts |
| Sampling | Default 3 times, adjustable 1~5 times | Large jitter in single attempts, median is more stable |
| Aggregation | Take median for storage, record min / max simultaneously | Median resists outliers, min/max shows fluctuation |
| Serial | Execute serially with 300ms interval between models | Concurrency steals bandwidth, inflating values |
| Failure | Record ok/runs and error messages |
Distinguish between “slow” and “down/hung” |
The panel overview card displays the median: First byte has ±(max-min)/2 fluctuation, total time has a range; a point on each curve = one round of sampling (median of 3 samples).
Installation and Enablement¶
The installation method provided in the README is via the UI, no command line commands:
- Sidebar → Plugins → Add Plugin, paste repository address,
.tgzor local directory - After enabling, restart DSH (Host changes are not hot-loaded)
- The sidebar appears with a “Speed Monitor” entry, click to open the panel
Do not try to construct installation commands from the plugin name; the README does not contain dsh plugin add ... or similar syntax. You also do not need to manually git clone.
Metric Definition¶
Before reading the curves, confirm the meaning of each field:
| Field | Meaning |
|---|---|
ttfb_ms |
Request sent → Response header received |
decode_ms |
First content event → Last content event |
total_ms |
Request sent → Stream end |
tokens |
completion_tokens |
tps_decode |
Decoding throughput = tokens ÷ decode |
tps_e2e |
End-to-end throughput = tokens ÷ total |
First byte is low, but total and tps_decode are both poor, indicating queuing or slow generation; first byte is high but decode is normal, usually indicating connection establishment or prefix cache issues.
Data Source and Interface¶
The plugin does not create a new configuration system, reusing DSH’s settings and storage:
| Item | Description |
|---|---|
| Model List | DSH settings (namespaces like llm-pi-ai / llm-deepseek) |
| Request | POST {baseURL}/chat/completions, stream: true, stream_options.include_usage: true |
| Storage | ~/.dsh/storages/speed-monitor.json (config), speed-monitor.jsonl (samples), speed-monitor.providers.json (model list cache) |
| Keys | apiKeyEnv in provider config, pointing to a field in ~/.dsh/.credentials.yaml |
Keys do not leave the host: they are read from ~/.dsh/.credentials.yaml according to apiKeyEnv, and the Client cannot access them.
The panel frontend calls these HTTP interfaces:
| Method | Path | Description |
|---|---|---|
GET |
/dsh-speed/api/state |
{ config, providers, items, error } |
POST |
/dsh-speed/api/providers |
Write model list { providers } |
POST |
/dsh-speed/api/config |
Save { intervalMinutes, runs, targets } |
POST |
/dsh-speed/api/sample |
Sample immediately, can carry { id } to sample only one target |
Typical Usage¶
After installation and restart:
- Configure at least one
llm-*provider in DSH settings,apiKeyEnvpoints to a field with value in~/.dsh/.credentials.yaml, andbaseURLis filled completely - Sidebar click “Speed Monitor”, the panel will bring in existing models from the list
- First click Sample Now to run a round, confirm there are values in the row instead of
-- - After confirmation, adjust the sampling interval back to the default 1 minute to record long-term
- The first few curves are for early reference; the median only makes sense after accumulating samples from each round
Common Questions¶
| Phenomenon | Handling |
|---|---|
| Panel prompts “No available model list yet” | No llm-* providers configured in DSH settings |
| A row always shows ‘–‘ | The provider’s apiKeyEnv has no value in ~/.dsh/.credentials.yaml, or baseURL is empty |
| Curves don’t move | Sampling interval is increased; or all requests failed (red error will be shown in the row) |
| Want to reset | Delete ~/.dsh/storages/speed-monitor.jsonl |
Applicable Scenarios and Notes¶
Suitable for people who already have multiple providers connected in DSH and want to leave a long-term latency/throughput record, or for comparing before and after switching proxy addresses. It measures the streaming performance from network to the provider, excluding the actual reasoning quality generated by the model.
A few points to note: The plugin runs with the current DSH process permissions and will read/write config files under ~/.dsh/storages/ in the process’s working directory. Before installing, open the repository to verify source code and MIT license, confirm that the interface paths and storage paths meet expectations, then decide whether to enable. Also, speed test requests will actually consume provider quota, confirm account balance before running.
Conclusion¶
The value of dsh-speed-monitor lies in making speed testing a permanent feature within the DSH interface, reusing DSH’s model list and credentials for configuration, without needing a separate script. The directory page is https://www.skillhub.cn/plugins/doraasn/dsh-speed-monitor, source code is at https://github.com/doraasn/dsh-speed-monitor.
