dsh-composer-tokens: Real-Time Token Counting Plugin for the DeepSeek Harness Input Box

Introduction¶

In DeepSeek Harness (DSH) agent development scenarios, input token estimation directly affects context window usage and cost control. The official Web GUI does not natively provide a real-time, precise input token counter, especially when system prompts, tool definitions, and skill references are involved, making it difficult for developers to intuitively judge context consumption.

dsh-composer-tokens is a pure client-side plugin maintained by wbycloud. It attaches to the Web GUI’s input box component to provide real-time token counting and detailed statistics. The plugin is implemented with @huggingface/tokenizers and supports both precise counting and offline estimation modes.

Features¶

Real-time Counting and Display¶

The plugin injects an inline badge at the conversation.input.right position, displaying the expected total token count (baseline + draft + frame(s)) and the context usage percentage.
- Display format: ≈12,345 · 38%
- Debounce mechanism: Typing triggers a 250 ms debounced refresh for the draft portion to avoid frequent recalculations.
- Color-coded levels: The total token count for the current request is colored: green for <100k, amber for 100k-300k, and red for >300k.

Detailed Info Tooltip¶

Hover with the mouse or focus with the keyboard to expand the Tooltip and view detailed data, including:
- Baseline: Based on the actual anchor from the most recent request (projectedTokens).
- Draft delta: The current changes inside the input field.
- Usage: The four buckets of actual tokenUsage accumulated in the session.
- Cache hit rate: Displayed only after provider usage arrives.
- Engine and model: The precise engine (HF Tokenizers) or the heuristic engine (estimation), along with the current model’s scope marker.

Calculation Basis¶

• Baseline first: Prefer projectedTokens (which already includes system prompts + tool definitions).
• Draft inclusion: Skill text and referenced text (@skill) are included in the draft based on the send path.
• Estimation fallback: For unknown models, offline usage without cache, or failed loading, use the ceil(len/4) heuristic (marked with ~).

Installation and Activation¶

Prerequisites¶

• Node.js ≥ 22
• pnpm
• Build artifacts: If you modify the source code, run npm run bundle first to generate lib/client.js.

Installation Command¶

It is recommended to install the npm published version:

dsh plugin --profile web add dsh-composer-tokens

Local Development Installation¶

For development and debugging, install the local directory directly:

dsh plugin --profile web add D:\path\to\dsh-composer-tokens

Legacy Version Migration¶

Starting with v0.1, the plugin ships with dsh.bundle.patch, so after installation you do not need to manually add a loader line to cordis.patch.yml. If you added one manually before, delete the old line to avoid conflicts.

Usage¶

Basic Display¶

Start the Web GUI (dsh web) and refresh the page. A token count badge appears on the right side of the input box and to the left of the send button. Before the first request in a new session, it displays a placeholder based on the previous session’s size; after sending, it automatically calibrates.

Model Override Configuration¶

The plugin supports overriding the tokenizer family for a model or a custom URL via localStorage.

localStorage.setItem("dsh-composer-tokens.overrides", JSON.stringify({
  "qwen-72b": "deepseek",          // specify a built-in family
  "my-model": "https://…/tokenizer.json"  // or a custom tokenizer.json URL
}));

Development Build¶

The plugin source includes React components, the Tokenizer engine, and data-fetching logic.

npm run prepare-tokenizer   # Download/validate tokenizer data and generate tokenizer-data.ts
npm run bundle              # Generate lib/client.js
npm run watch               # Incremental build (with HMR)

Important Notes¶

1. Output tokens excluded: Because output tokens are unpredictable, the badge only shows estimated input-side tokens.
2. Model-switching cooldown: Model switching does not emit a wire event; session cache and projected frames use a 3-second cooldown re-query mechanism.
3. Very long drafts: If a draft exceeds 100,000 characters, the trailing portion automatically switches to estimation mode.
4. Inline display limits: The input field has limited space, so the badge may be truncated; the full value is available in the Tooltip.
5. Windows pnpm link: During local development, pnpm link across drive letters may create file symbolic links that Node cannot resolve; you must manually rebuild them as directory junctions.
6. Legacy installation cleanup: Users who manually added loader lines before v0.1 must manually remove the corresponding lines from cordis.patch.yml when uninstalling or updating.

References¶

• Plugin directory: https://www.skillhub.cn/plugins/wbycloud/dsh-composer-tokens
• Source repository: https://github.com/wbycloud/dsh-composer-tokens