dsh-llm-verifier: Verification, Ranking, and Controlled Application of Multi-candidate Patches
dsh-llm-verifier is a DeepSeek Harness plugin maintained by Web0926 that solidifies the workflow of "generating multiple candidate patches → test filtering → LLM ranking → manual approval and application". It runs 3 or 5 candidates for the same coding task, each working in an independent detached Git worktree. It first executes deterministic verification to eliminate failed candidates, then uses llm-verifier to rank them. If only one passes, it is selected directly; with 2 candidates, it uses a pivot; and with 3 to 5 candidates, it uses two pivots. The plugin imposes two approval gates: one before candidate execution and one before applying the patch. Before application, it verifies the repository path, base HEAD, clean status, and the patch's SHA-256. Verification commands can be automatically detected or explicitly specified and support npm/pnpm.
Read Moredsh-llm-verifier: Configurable LLM Verification Plugin for DSH
dsh-llm-verifier is a native DSH plugin designed to introduce an independent review mechanism to address challenges in selecting among multiple candidate solutions, tracking progress in long-running tasks, and validating session outcomes. Based on a configurable judge model, it performs evaluations using explicit tools—verifier_compare, verifier_select, verifier_track, and verifier_current_session—employing an A-T fine-grained scoring scale that is normalized to a 0-1 score. The plugin supports probabilistic expectation scoring, mitigation of position bias, and persistent caching, automatically scheduling the validation process at agent/turn-stopping boundaries. After installation, DSH automatically aligns the configuration, allowing users to set up the provider, model, and concurrency strategies via a web interface, or integrate it as an npm
Read More