huggingface-local-models: Run Hugging Face GGUF Models Locally with Skill
huggingface-local-models is an Agent Skill in the Hugging Face skills repository that guides local model selection and inference on CPU, Metal, CUDA, or ROCm using llama.cpp and GGUF. The default workflow includes searching the Hub with apps=llama.cpp, reviewing quantization recommendations on the local-app page, confirming filenames via the tree API, and launching llama-cli or llama-server with the -hf option, using --hf-repo/--hf-file when needed or converting from Transformers. The article covers installation and enabling, key considerations for quantization selection, a smoke test for an OpenAI-compatible local service, and usage notes.
Read MoreDeepSeek Harness Plugin: dsh-local-llm
`dsh-local-llm` is a community plugin for DeepSeek Harness designed to run GGUF-format models locally. It provides model downloading, management, and an OpenAI-compatible inference interface launched via `llama-server`. The plugin supports downloading models from Hugging Face, SSE streaming, automatic context-size detection, and secure path handling. Users need Node.js 18+ and can start the service with the specified configuration to call local models directly within Harness, achieving low-latency, privacy-preserving inference.
Read More