Introduction¶
When agents execute review tasks, a common problem is that conclusions are influenced by a single perspective, or judges are swayed by seeing each other’s scores. Many teams write “Please review by role” in a single prompt, but this is still a self-conversation within the same context, and bias cannot be isolated. The review-workflow plugin introduced below splits the review into “N invisible judges + an aggregator + an independent Critic”. The process is fixed at 6 steps, with checkpoints at each step.
The plugin repository is YaoleWu/review-workflow, categorized as Workflow, currently 3 stars, 1 fork, MIT license, version 0.1.0. The installation command in the README points to github:LeslieWylie/review-workflow, and the repository author field also states LeslieWylie; the body text follows the original source, and the README command takes precedence during installation. The SkillsHub directory page is an independent community site and has no official affiliation with DeepSeek or InfiniFlow.
What is it¶
A generic multi-judge review workflow skill. It is generalized from anonymized internal collaboration review processes, providing only the skeleton of the review process; specific review dimensions, scoring criteria, and templates are provided by the user. Applicable scenarios include project reviews, paper reviews, code reviews, design reviews, etc.
There are three core mechanisms:
- Multiple judges run in parallel using independent subagents and cannot see each other’s scores.
- Disagreements are handled in two stages: first, anchoring matching, then Delta-level adjudication.
- The Critic uses an independent subagent to verify process compliance and challenge conclusions with insufficient reasoning.
Core Features¶
| Module | Content |
|---|---|
| 6-step workflow + 6 Checkpoints | Material completeness check → Prepare bundle → N judges parallel scoring → Aggregate disagreements → Critic review → Final decision |
| N parallel independent scoring | Judges run in independent subagents, mutually invisible |
| Anchoring consistency check | Anchoring matching → Delta-level adjudication, two-stage handling of disagreements |
| Independent Critic review | Independent subagent verifies process compliance |
| Re-review support | Incremental re-review, only re-score dimensions related to modifications |
| Configurable | Review dimensions, scoring criteria, templates, number of judges can be configured |
Two notable design principles are: Checkpoint enforcement (do not proceed to the next step if not passed); all intermediate artifacts are saved for traceability.
Installation and Enablement¶
The installation command comes from the README:
dsh plugin --profile web add "github:LeslieWylie/review-workflow"
After restarting, load the skill in any session:
load review-workflow
Required Configuration¶
This skill requires project-level configuration, placed in the project’s own skill or configuration file; it cannot run directly relying on the plugin’s default values (the plugin only provides the process skeleton).
| Config Item | Description | Example |
|--------|------|
| rubric | Review dimensions and scoring criteria | 5-dimension 5-point scale, with anchoring descriptions for each dimension |
| bundle template | Review material package template | bundle.md |
| panelist prompt | Judge prompt template | panelist-prompt.md |
| aggregation template | Aggregation report template | aggregation.md |
| critic template | Review report template | critic-report.md |
| checklist | Material completeness checklist | proposal-checklist.md |
| panelist count | Number of judges | 2 |
| scoring threshold | Pass/Fail threshold for each dimension | ≥3/5 |
Typical Usage¶
Following the process in the README, a complete review consists of 6 steps:
- Material completeness check: Verify submitted materials against the checklist; return if failed.
- Prepare bundle: Generate the review material package based on the bundle template.
- N parallel scoring: Each judge scores in an independent subagent according to the rubric, invisible to others.
- Aggregate disagreements: First perform anchoring matching, then Delta-level adjudication for those still inconsistent.
- Critic review: Independent subagent checks process compliance and challenges conclusions with insufficient justification.
- Final decision: Output the aggregation report after passing the checkpoint.
When re-reviewing is needed, only have judges re-score dimensions related to the modification items; there is no need to review the entire package again.
Applicable Scenarios and Notes¶
Suitable for scenarios requiring structured, multi-person, traceable review results, especially when conclusions affect resource allocation or release decisions. If it is just a quick glance at code, letting the agent read it directly is faster and not worth setting up this workflow.
Two points to note. First, the plugin runs with the permissions of the current dsh process; the source code should be checked before installation to confirm its behavior matches the project. Second, project-level configuration must be provided by the user; the plugin does not contain generic review dimensions and standards; expecting to review directly after installation is a misconception. The repository license is MIT, which is relatively loose at the authorization level, but the source code still needs to be reviewed by yourself.
Conclusion¶
The value of review-workflow lies not in the review content itself, but in fixing the review process: role isolation, checkpoint enforcement, and process traceability. Suitable for teams that already have structured review needs and are willing to define rubrics and templates themselves.
- Directory page: https://www.skillhub.cn/plugins/YaoleWu/review-workflow
- GitHub: https://github.com/YaoleWu/review-workflow
