aispin-dev/llm-as-a-verifier-dsh: LLM-as-a-Verifier DSH Plugin and Best-of-N Mode

The DSH plugin `llm-as-a-verifier-dsh` leverages an “everything-is-a-plugin” architecture and a Best-of-N dialogue mode to provide low-cost test-time scaling for DeepSeek V4 Flash. Built on the paper’s algorithm, it uses expected-value scoring and the PPT (Probability Pivot Tournament) to select the best answer from candidates; for example, Bo-5 requires only 11 scoring calls. The plugin offers tool, service, and auto modes, supports zero-configuration startup, adapts to endpoint capabilities, and includes a fail-open fallback mechanism. On Terminal-Bench 2.1, it achieves 88% accuracy, reproducing frontier-model performance at roughly 1/11th the cost.

Read More