dsh-context-budget: Configuring the DSH Model Context Window and Compression Threshold
The `dsh-context-budget` plugin is designed to resolve default context window issues caused by model ID and catalog mismatches when manually configuring third-party gateways in DSH. It adds an intuitive configuration group to the pi-ai provider card for setting the context window, compaction threshold (with reserved-budget support), and thinking effort control. After installation, the configuration is automatically written to `cordis.patch.yml` and validated, supporting thinking-level adjustments for multiple providers. The plugin applies only to pi-ai routing, and persisted configuration must be cleaned up manually after uninstallation.
Read Moredsh-context-budget: A Plugin for Keeping Local Model Context Within a Controllable GPU Range
The `dsh-context-budget` plugin is designed to address the sharp performance degradation of local models under long contexts. It measures prefill speed and enforces hard limits and checks for token count, first-token latency, and cold prefill time. When a limit is exceeded, it can print a warning or perform compaction, keeping the context within the range the GPU can process. The plugin coexists with `compaction-basic` and provides a `/context-budget` command to monitor status, helping users optimize the experience of using local large language models.
Read More