Preface¶

After running a successful task in DSH, experience often stays only within the session. When encountering a similar task next time, the model still has to explore from scratch, and the pitfalls encountered previously might be stepped on again. Existing approaches rely on manually writing READMEs or relying on human memory to remember lessons; neither allows these to automatically surface before the next execution.

yangyiqun747/dsh-plugin-playbook is a “Experience Reuse + Delivery Gate” plugin for DSH, maintained by yangyiqun747, categorized under workflows. The idea is: after a task succeeds once, the “reusable SOP + pitfalls encountered” are precipitated into a playbook; when encountering a similar task next time, it appears as a native Skill in the skill directory, and the model loads it on demand and follows it. Once a task family accumulates enough samples, the SOP can be upgraded into a delivery gate: before execution, the user must review it, and before delivery, objective evidence must be obtained item by item; otherwise, the write action will be hard-intercepted.

Below is an introduction to the practical implementation of this plugin.

Core Functions¶

The plugin provides three sets of capabilities in total.

Experience Library Tools: 5¶

1. playbook_save: Save an experience.
2. playbook_search: Retrieve by relevance.
3. playbook_get: Get the full text.
4. playbook_promote: Promote an entry to active.
5. playbook_feedback: Write back the result of this usage (ok or fail).

Gate Tools: 6¶

1. gate_status: View which gates exist in the current session and workspace, as well as historical pass rates.
2. gate_propose: Propose a gate (writes as “unconfirmed” and does not intercept any actions).
3. gate_confirm: Ask the user via the native question channel; only after the user agrees is it persisted.
4. gate_start: First submit the current plan to the user for review via the native question channel; the gate is only enabled after the user clicks “Start Execution”.
5. gate_check ×N: Record objective evidence for each item (command + exit code + output summary).
6. gate_finish: Release for delivery after all required items are ready, and write back statistics.

Plus a hard-intercept guard (intercepts write / edit / present on tools/pre-execute until a confirmed L2+ gate passes item by item), a Skill Provider (active status playbooks appear in the skill directory as playbook-<id>, loaded using the built-in skill tool), a slash command /playbook, and two system prompts (playbook:usage for experience library usage, playbook:gates for current gate status reporting).

Storage: Pure Markdown, git-friendly and editable¶

Data is stored within the workspace:

<workspace>/.dsh/playbooks/<id>.md     # Experience
<workspace>/.dsh/gates/<id>.md         # Gates

The frontmatter includes id, title, status (draft / active / deprecated), scope, tags, whenToUse, and stats (uses, ok, fail, lastUsed, lastOutcome). The body is segmented into “When to Use / Prerequisites / SOP / Pitfalls / Verification / Evidence”.

The filename <id> will be normalized to ASCII kebab-case. The reason is practical: active entries need to become native Skills, and the skill name (playbook-<id>) must be ASCII. The rule is to strip ASCII alphanumeric characters from the title to form kebab-case (e.g., “Developing a Host Plugin for DSH” → dsh-host); if there is no ASCII in the title, it falls back to pb-<8-character-content-hash>. The Chinese title itself is unaffected and is fully preserved in the title field in frontmatter.

Recall Methods¶

1. Skill Directory: active entries appear as playbook-<id>, using whenToUse as the description. The model judges relevance itself and loads it.
2. Active Retrieval: playbook_search scores based on tags (weight 6/4) and keywords (title 5, tags 4, whenToUse 3, body 1), and adjusts slightly by stats.ok/fail, sorting verified valid entries first.
3. Full Text: playbook_get.

Lifecycle and Auto-degradation¶

playbook_save       -> draft       Only converts to active after user confirmation
playbook_promote    -> active      active entries are exposed as Skills and recalled
                  ↓ Reuse
playbook_feedback(ok|fail)        Writes back stats
                  ↓ Continuous failure
             Auto-degradation -> draft / deprecated

Auto-degradation is triggered only on active entries, with the following rules:

• fail >= 2 and ok === 0 → deprecated (never succeeded, eliminated immediately).
• fail >= 3 and fail > ok * 2 → deprecated (succeeded but generally ineffective).
• fail >= 2 and fail > ok → revert to draft (needs revision).

The source text specifically notes that the order cannot be reversed: the draft branch must be listed last, otherwise the entry would leave active after the second failure, and subsequent active-exclusive deprecated judgments would become dead code.

Delivery Gates: Review first, then release¶

The experience library solves “how to do it,” while the gate solves “whether you dare to deliver after finishing.” Gate definitions can only be changed after user confirmation; the plugin can only propose.

The risk score quantifies a task family into R, which then determines how hard the gate is. Simplified, D is calculated from factors like scope of change, chain length, cognitive density, etc. (discounted if there is an objective verification method, plus points for irreversible actions), and P is calculated from historical pass rates and the number of user reworks. Both are synthesized via R = 0.5 × D + 0.5 × (1 − P). The weight of user rework (reworks) in P is twice that of a normal success/failure.

A complete workflow is:

1. gate_status            View which gates exist in the current session and workspace, and historical pass rates
2. gate_propose           Propose a gate (unconfirmed, no intercept)
3. gate_confirm           Ask the user directly via the native question channel; only persisted after agreement
4. gate_start             First submit the plan to the user for review via the native question channel;
                          the gate is enabled only after the user clicks "Start Execution"
5. gate_check ×N          Record objective evidence for each item (command + exit code + output summary)
6. gate_finish(pass)      All required items ready -> Release for delivery, write back statistics
                          (When L3, additionally requires at least one item in step 5 to be recorded by another agent)

You can view it directly without going through the model: /playbook gates lists the gates in the workspace (level, whether confirmed by user, pass rate, recommended level based on current data), and /playbook gates <id> views the check items and history of a single gate.

Four Invariants¶

1. No Self-Confirmation for Release: Both gate_confirm and gate_start ask the user directly via ctx.userQuestions.ask, and the answer is provided by a human on the interface. The model can only use user_confirmed as a fallback when the native channel is unavailable, and this fallback is recorded in history / tool output.
2. Objective evidence is required for L2+: gate_check requires command + exit code + output summary; self-evaluation does not count as evidence; L3 additionally requires that at least one piece of evidence is recorded by an agent other than the one that opened the gate.
3. Gate definitions can only change after user confirmation: gate_propose always writes confirmed_by_user: false, and even if the level is changed, it must be reconfirmed.
4. Circuit breaker and escape hatch must exist: After the same gate hard-intercepts 2 times in a row (MAX_BOUNCES), it automatically opens the circuit breaker, releases, and returns decision-making power to the user; when the user says “don’t use a gate this time,” use gate_finish(exempt) to exit, and the exemption is not counted in statistics.

Gate state is stored by workspace, not by agent¶

gate_start creates the session on the gate directory in the workspace, so all agents in the same workspace share a gate. Sub-agents dispatched via native subagent land in the same session; evidence they record by calling gate_check themselves is marked verifier: <sub-agent id> and counted as independent; the sub-agent’s write / edit / present are also hard-intercepted; gate_finish can only be called by the agent that opened the gate.

This is how the L3 “Independent Verifier” is implemented: it does not ask the model to declare “I found someone to verify,” but rather lets another agent call the tool, and the host fills in its identity. The model cannot forge.

Relationship between failure degradation and existing mechanisms¶

The guard does not silently release when it makes a mistake: it writes the error into the gate status section, warns the user, and then processes it as “passed but released + warning”.

The plugin does not reinvent the wheel: execution review uses the native ctx.userQuestions.ask, action-level blocking uses the native tools/pre-execute, and gate status reporting uses the native ctx.systemPrompt.section. The plugin does not import any @deepseek-ai/*. plan mode is best-effort—during the first real run, ctx.get('planMode') returns null in the plugin context, so gate_start no longer depends on it; user review goes through the question channel, and if plan mode can get it, it switches accordingly; if not, it writes the reason into the plan_mode field. When tasks are already running in the AgentTeams team, gate restrictions are not stacked; it uses their requirements/review verdict, and the gate is only responsible for writing the verdict back into gate_check evidence.

The README includes a set of real-world test records in a host: tools/pre-execute hard-intercept works (when a gate is active, write is rejected, the error message contains the missing evidence list, and the probe file is not created); ctx.userQuestions.ask confirms the gate works; ctx.get('planMode') cannot be obtained in the plugin context; there was a bug in the gate_propose factors persistence (fixed); there was a bug in the prompt section reporting failed items (fixed); repeated install_bundle returns application: failed. Each corresponds to a regression test or the specific original output.

Installation and Enabling¶

dsh plugin --profile desktop add <absolute path of this directory>

To uninstall:

dsh plugin --profile desktop remove dsh-plugin-playbook

Install from a local directory path, not from GitHub. The package is ESM, with main being lib/index.js, and declares a cordis.patch.yml patch file; Node version requirement is >=22.

After enabling, the configuration is written in config in cordis.patch.yml:

- insert:
    - id: playbook
      name: dsh-plugin-playbook
      config:
        dir: .dsh/playbooks      # Experience library, relative to workspace
        gateDir: .dsh/gates      # Gate library, relative to workspace
        promptSectionOrder: 140  # System prompt section ordering
        slashCommand: true

If install_bundle reports an error during installation that is unrelated to this plugin (e.g., ENOENT ... dsh-plugin-whale-maid-1.0.0.tgz), it is because a file: dependency in the profile’s pnpm-lock.yaml points to a deleted local package, and pnpm reads it every time it installs. The fix is to run pnpm pack --pack-destination <expected directory of lockfile> in the corresponding source directory, then recalculate using SHA512 and update the integrity of that entry in the lockfile.

To test:

npm test
# Equivalent to node --test "tests/*.test.mjs"

Note not to write it as node --test tests/, as Node will treat it as a module path and report MODULE_NOT_FOUND.

Typical Usage¶

The complete workflow is as follows.

1. Precipitate experience after a task succeeds: Call playbook_save to write the SOP and “pitfalls” sections.

---
id: "windows-sandbox-build"
title: "Running builds in Windows sandbox"
status: "active"
scope: "project"
tags: ["windows", "sandbox", "build"]
whenToUse: "When needing to execute pnpm/node builds inside a sandbox"
stats:
  uses: 3
  ok: 2
  fail: 1
---

# Running builds in Windows sandbox

## When to use
## Prerequisites
## SOP
## Pitfalls
## Verification
## Evidence

2. Confirm promotion: Use playbook_promote to upgrade to active; thereafter, it appears in the skill directory as playbook-windows-sandbox-build.

3. Retrieval for next similar task: playbook_search recalls by scoring based on tags and keywords, or directly playbook_get to get the full text.

4. Propose a gate when the task family has enough samples: gate_propose provides check items and level suggestions; at this point, it does not intercept. gate_confirm goes through the native question channel to ask the user for confirmation.

5. Review before execution and open the gate: gate_start submits the plan to the user for review; the gate becomes effective only after the user clicks “Start Execution”.

6. Collect evidence item by item: After completing a check, call gate_check to record the command, exit code, and output summary.

7. Delivery determination: After all required items are ready, call gate_finish(pass) to release and write back statistics; otherwise, only gate_finish(fail) is possible. failedEvidence only counts required items; optional item failures do not block pass.

Applicable Scenarios and Notes¶

Suitable for: situations where the same type of task appears repeatedly and the last successful approach is worth solidifying, especially task families with high frequency, large scope of change, or where one “dare not deliver directly after finishing.” For personal practice or one-off scripts, L0 does not set a gate, and the plugin’s experience library only has benefits if one is willing to write SOPs.

Note a few points:

• The plugin runs with the permissions of the current dsh process; installed code will read and write files in your workspace, so you should check the source code and license (MIT) before installing.
• Stars and download counts are not high on the directory page; it is a personal maintenance project in the community, and long-term maintenance requires your own judgment.
• Gate behavior depends on the availability of native tools/pre-execute and ctx.userQuestions.ask; the fallback when the sheltered channel is unavailable is recorded, and defaults to “passed but released + warning” rather than silent release.
• Storage is pure Markdown within the workspace, so it can be managed directly with git, but this also means there is no additional permission isolation or encryption.

Conclusion¶

In one sentence, the value of this plugin is: successful experience → structured playbook → automatic recall for similar tasks → user confirmation as a gate → mandatory compliance before delivery. It does not solve the model’s capability problem, but rather the two issues of experience not being retained and the lack of objective evidence during delivery.

Directory Page: https://www.skillhub.cn/plugins/yangyiqun747/dsh-plugin-playbook

GitHub: https://github.com/yangyiqun747/dsh-plugin-playbook