DeepSeek Harness Plugin: dsh-jev

Most coding agents spend expensive reasoning tokens on “mechanical decisions,” such as determining which module a bug belongs to, whether it blocks core flows, or the risk level of a change. This can prevent the primary model (such as DeepSeek) from focusing on deep code generation and debugging.

dsh-jev is a plugin for DeepSeek Harness (DSH). It provides semantic tool routing and typed decisions through TypeSafe Jev. It offloads judgment work to lightweight models and returns probability distributions and scores, freeing the primary model from simple classification tasks.

Core Features¶

This plugin provides the following capabilities:

Typed Evaluation: Provides the typesafe_evaluate and jev_evaluate tools, supporting three types: choice (category selection), noul (0-1 truth probability), and score (rating).
Skill and Tool Discovery: Provides the jev_find_skill and jev_find_tools tools, recommending relevant skills (SKILL.md) or tools based on semantic matching.
Management Commands: Integrates the /typesafe and /jev slash commands for status inspection, login authentication, testing, calibration, and management.
Cost and Quota Tracking: Supports tracking daily request counts, token usage, and USD cost, and supports setting daily limits.
Calibration Toolkit: Provides threshold calibration tools that optimize thresholds from historical samples using metrics such as AUC, precision, and recall.
CI/CD Validation: Provides the dsh-jev-gate (or jev-gate) CLI tool for acceptance checks in CI/CD workflows.
Graceful Degradation: When Jev is not configured or offline, it automatically falls back to local keyword heuristic search.

Installation and Enablement¶

Run the following command in the DSH configuration directory (for example, ~/.dsh/profiles/web) to install it:

pnpm add file:C:/Users/lldois/workspace/dsh-jev

After installation, add the plugin to the dsh.profile.bundles configuration in package.json:

{
  "dsh": {
    "profile": {
      "bundles": [
        "@deepseek-ai/dsh-base",
        "@deepseek-ai/dsh-web-app",
        "dsh-jev"
      ]
    }
  }
}

Configuration and Usage¶

Configure the API Key:
You can configure it via a slash command or by setting environment variables.
- Slash Command:

        /typesafe login your_api_key_here

    The key is saved to `~/.pi/agent/pi-typesafe/auth.json`.

*   **Environment Variables**:

        export TYPESAFE_API_KEY=your_key_here
        export PI_TYPESAFE_ENABLED=1

Run Calibration:
If you need to adjust decision thresholds, you can use the calibration tool:

    /typesafe calibrate

Tool Call Example:
A model or code can invoke typed evaluation by passing a JSON state and questions. For example, to determine a bug’s area and severity:

    {
      "state": {
        "title": "升级后无法登录",
        "body": "输入密码后一直回到登录页"
      },
      "questions": {
        "area": {
          "type": "choice",
          "instructions": "这个问题属于哪个模块？",
          "criteria": {
            "auth": "登录与身份验证",
            "ui": "界面与布局",
            "other": "都不符合"
          }
        },
        "blocking": {
          "type": "noul",
          "instructions": "这个问题是否阻止用户继续使用产品？"
        },
        "severity": {
          "type": "score",
          "instructions": "评估这个问题的严重程度：",
          "criteria": [
            "仅影响外观",
            "存在可用绕过方案",
            "阻止核心流程"
          ]
        }
      }
    }

Notes¶

Version Requirement: The DSH version must be 0.1.5-rc.1 or higher.
Cost: API calls incur charges; input token prices are approximately $0.042 per million.
Dependencies: Peer dependencies required by the plugin (such as cordis, schemastery, dsh-tools, etc.) are marked optional and are usually provided by the DSH environment.
License: MIT License.

Summary¶

dsh-jev provides structured decision-making capabilities for DeepSeek Harness. By offloading mechanical classification, scoring, and routing tasks from the primary model, you can use Jev to obtain probability distributions quickly and at low cost, enabling deterministic code logic (such as if probability > 0.85).

Plugin directory: SkillHub
Source repository: GitHub