dsh-model-deploy
`dsh-model-deploy` is a plugin for DeepSeek Harness (DSH) designed to ensure that models meet performance targets on target hardware through benchmarking. It provides an end-to-end workflow from static model checks and environment probing to local and SSH remote benchmarking, and supports gate thresholds for result validation. Users need to install the required Python dependencies and ONNX Runtime (with GPU support) and configure SSH. The plugin offers model checking, environment probing, and performance testing via the CLI and DSH toolkit, supports dynamic input models, and detects execution provider fallbacks. The current version (V0.1) does not yet support backends such as TensorRT.
Read MoreDeepSeek Harness Plugin: dsh-model-deploy
The DeepSeek Harness plugin `dsh-model-deploy` is designed to address model selection and resource analysis issues before LLM deployment. It supports cross-platform analysis across 38 models and 20 hardware options, covering VRAM breakdown, latency, throughput, power consumption estimation, and parallel strategy selection. Users can use it by installing it with a specified profile and restarting, and its publicly available estimation formulas have been validated against real-world deployment anchors, making it suitable for quick verification and capacity planning before procurement.
Read More