Preface¶

The philosophy of DSH is “everything is a plugin”: tool capabilities can be installed separately and then combined into an agent workflow. A specific pain point is: when a user sends a Bilibili link, the agent cannot directly hand the video to the model for processing—it needs to first extract the text, comments, danmaku, and necessary frames.

dsh-bilibili is a plugin provided for DSH, maintained by CZX2244, licensed under MIT. It adds a bilibili_extract tool to the agent for analyzing Bilibili videos: first extracting text, then capturing key frames on demand, and finally generating a summary.

What This Is¶

Below is its position within DSH.

dsh-bilibili is a DeepSeek Harness tool plugin. Its input is a Bilibili video link, and its output is a structured analysis and summary. Users can send bilibili.com/video/BV..., b23.tv short links, or bare BV IDs, and the model invokes bilibili_extract during the session for analysis.

Core Features¶

Text Extraction¶

The plugin provides full text extraction capabilities, including:

• Metadata
• Timestamped transcription
• Top comments and replies
• Danmaku

For videos without a subtitle track, the plugin can automatically transcribe. By default, it uses Bijian ASR, but can also switch to sherpa-onnx or whisper.cpp local engines.

Key Frame Capture¶

The plugin supports on-demand key frame capture, with the following verified capabilities:

• Automatic frame selection: scene-change detection, even-interval backfill, sharp-frame preference
• Optional frame visual description: local Ollama / llama.cpp, or OpenAI-compatible vision API
• Download-first capture: qualifying videos (≤30 min / ≤800MB) are downloaded locally first, falling back to remote frame-by-frame extraction on failure

Two-Pass Workflow¶

The recommended approach is to do text first, then visuals:

1. Call bilibili_extract to read the transcription;
2. The agent scans the transcription to determine when frames are needed;
3. Request key frames again with timestamps.

This keeps the first pass lightweight and adds visual information in the second pass.

Output Templates and Helper Tools¶

The plugin provides replaceable output templates. By default, it uses templates/summary.md, but templates/timeline.md can also be used.

In addition to bilibili_extract, the plugin also registers:

• bilibili_login: QR-code login
• bilibili_doctor: Environment checks

Runtime includes the following robustness handling:

• Bilibili 412 exponential backoff
• SESSDATA hints for logged-in subtitle scenarios
• ffmpeg pipe-free

Installation and Enablement¶

Prerequisites are:

• Node 18+
• ffmpeg on PATH
• pnpm

The installation command is as follows:

dsh plugin --profile web add git+https://github.com/CZX2244/dsh-bilibili

After the steps above, bilibili_extract becomes a callable tool for the agent.

Typical Usage¶

Two-Pass Invocation¶

First, extract only text:

bilibili_extract(url, extract_frames: false)

After the agent scans the transcription, it lists the timestamps where frames are needed in a fixed format:

[suggest_frame] mm:ss reason

Then make a second pass request based on these time points:

bilibili_extract(url, timestamps: <time points from the previous round's [suggest_frame] list>)

The key point here is: the first pass gets text, and the second pass supplements key frames.

Switching Templates¶

Set the template path in the config:

summaryTemplate: 'C:/path/my-template.md'

If the custom template path is invalid, the plugin falls back to the bundled template.

Rechecking the Environment¶

When you need to check the environment on demand, use bilibili_doctor. If dependencies or configuration have just changed, you can use refresh=true to force a new probe:

bilibili_doctor(refresh: true)

Applicable Scenarios and Notes¶

Suitable for handling Bilibili links in a DSH agent: breaking down videos into transcriptions, comments, danmaku, and supplementing key frames or frame descriptions when necessary.

Please note before use:

• The plugin runs with the current dsh process permissions; the source code and license should be reviewed before installation.
• The license is MIT.
• The plugin does not bundle third-party binaries or models; the open-source projects and services it invokes are listed in THIRD_PARTY_NOTICES.md.
• When ffmpeg is missing, video download and frame capture are skipped.
• When the local ASR engine is not ready, it skips before downloading the audio stream.
• Inaccessibility of the main Bilibili API marks the plugin as unavailable; the local ASR engine also relies on playurl to obtain audio streams.
• Reports do not contain raw credentials; only SESSDATA validity is shown.
• Captured frame times may deviate from the requested time by approximately ±0.5s or more due to key frame spacing, scene alignment, and sharpness optimization.

Conclusion¶

dsh-bilibili provides a practical processing path: text first, then key frames, then template-based summary output.

GitHub repository: https://github.com/CZX2244/dsh-bilibili

The DSH community directory is an independent site with no official affiliation to DeepSeek / High-Flyer; verified sources did not provide a directory page URL, so this article does not list a specific directory page link.