dsh-llm-gate: DeepSeek Harness Single-Model Provider Concurrency Gating Plugin
2026-10-01
DeepSeek Harness plugin
DeepSeek
DSH plugin
dsh-llm-gate
Concurrent Gate Control
model inference
`dsh-llm-gate` is a plugin for DeepSeek Harness (DSH), designed to address the issue that underlying single-slot model services (such as `llama.cpp`) cannot handle concurrent requests, causing requests to time out and be dropped. The plugin stores excess requests in an internal FIFO queue to ensure orderly processing. It supports provider-level concurrency control, queue management, and end-to-end coverage, and can handle errors such as a full queue or timeout. Users need to configure concurrency limits, queue length, and wait timeout via a configuration file, while being mindful of serialization constraints and dependencies.
Read More