Preface¶

When working with DSH plugins or comparing candidate profiles, a common set of questions arises: under the same task and repository snapshot, which candidate profile is more stable? Which runs should be blocked early by hard gates? How do we weigh quality, duration, tokens, and cost against each other?

dsh-arena targets these local experiments: it compares multiple candidate profiles under the same task and repository snapshot, first applying deterministic hard gates, then Pareto ranking and winner summarization, with results viewable in a Web matrix. Below, we cover its positioning, capabilities, installation methods, and applicable boundaries.

What This Is¶

dsh-arena is a local experiment and evaluation workbench plugin for DeepSeek Harness (DSH). It is maintained by Apageoflove and is licensed under MIT.

Its package.json declares version 0.2.0 with private: true; verified sources indicate it has not been published to npm.

It brings the following into a single local workbench:

• Candidate profile comparison under the same task and repository snapshot
• Deterministic hard gates
• Pareto ranking and winner summarization
• Local persistence, run queues, abort, and timeout support
• Web matrix interface
• Automatic credential redaction in exported reports
• Import of local Arena JSON reports for offline rendering

Core Capabilities¶

Candidate Profile Comparison¶

It first performs comparisons under the same task and repository snapshot, then compares results across multiple candidate profile runs.

Hard Gates and Pareto¶

• Hard gates are deterministic: failing one results in immediate failure and exclusion from ranking.
• Ranking uses Pareto ordering and summarizes winners.
• Report views include run and hard gate status, quality, duration, token and cost metrics, Pareto candidates with suggested winners, and audit alerts.

Local Execution and Persistence¶

• Local persistence is atomic.
• Run queues have a concurrency limit.
• Abort and timeout are supported.
• Paths are restricted to dataRoot; out-of-bounds paths are rejected outright.
• Persistence can be disabled via persistencePath: false.

Adapter-Driven Execution¶

Execution is adapter-driven, with no guessing or reliance on undocumented DSH headless commands.

Only bindExecutor can create run orchestrators, and the adapter must be provided by the caller. It does not invent its own DSH headless commands or silently trigger model calls.

Web Matrix and Offline Reports¶

• The Web matrix interface refreshes in real time.
• Supports importing local Arena JSON reports and offline rendering of candidate/model run matrices, run and hard gate status, quality/duration/token/cost metrics, Pareto candidates with suggested winners, and audit alerts.
• Before exporting JSON/JSONL/Markdown, inline authorization / token / key / secret / password / cookie fields are redacted based on field names and values.
• Files remain in the browser; no uploads or telemetry.

Installation and Enabling¶

Build first, then execute the installation and verification commands:

npm run build
npm test
pnpm dsh plugin --profile arena add <this directory path>
pnpm dsh --profile arena --dump-config
pnpm dsh --profile arena web

If installing from a local tgz file, use:

pnpm dsh plugin --profile arena add dsh-arena-0.2.0.tgz

After the steps above, pnpm dsh --profile arena --dump-config is used to view configuration, and pnpm dsh --profile arena web starts the Web interface.

Typical Usage¶

Build and Test¶

npm run build
npm test

According to verified sources, package.json engines requires node >=22.18.0, and it mentions that building and testing on Node 24 requires no additional dependencies.

Local Development Patch¶

For local development, you can use cordis.local.patch.yml:

pnpm dsh web --patch <this directory>/cordis.local.patch.yml

This command uses the cordis.local.patch.yml in the current directory as a local patch.

Execution Boundaries¶

• Wait for await arena.ready before the first modification.
• Only bindExecutor can create run orchestrators; adapters must be provided by the caller.
• Paths are restricted to dataRoot; out-of-bounds paths are rejected outright.
• On Windows, pnpm/dsh are .cmd launchers and must be invoked through a command interpreter.
• Task text is quote-escaped before being assembled into commands.

Applicable Scenarios and Notes¶

Suitable for developers who need local DSH candidate profile comparison, hard gate determination, Pareto ranking, report export, and Web-based viewing.

Before installation, it is recommended to review the source code and license. This project is licensed under MIT. The plugin runs with the permissions of the current dsh process, so confirm you trust the code in the directory before installing.

Boundaries to note:

• Only follows officially documented Cordis contracts; does not touch undocumented internal interfaces of the DSH Client/Host.
• Capability detection uses feature probing (ctx.arena), not private slots.
• Web / Headless E2E on real DSH has not yet been validated, so this repository does not claim 100% compatibility.
• dsh.compatibility.dshReleases lists 0.1.0-rc.8, 0.1.1-rc.1, and 0.1.1-rc.2 as unknown.
• peerDependencies require @deepseek-ai/cordis ^4.0.1 and @deepseek-ai/schemastery ^3.18.1.
• Verified sources only provide installation commands for local paths or local tgz files; no npm package installation or remote installation commands are provided.

Conclusion¶

The value of dsh-arena lies in consolidating the local evaluation workflow for DSH candidate profiles: hard gates, Pareto, metrics, redacted reports, and local Web viewing all happen within a single workbench.

GitHub: https://github.com/Apageoflove/DSH-arena

Community directory page: No URL provided in verified sources.