Introduction¶

Developers who have created DSH plugins often encounter a specific requirement: the Agent can already understand the intent, but cannot produce an audio finished product on its own—especially for covers, a task that involves a long chain of dependencies on local models and GPUs. The common approach is to stitch together scripts using yt-dlp, UVR separation, RVC voice swapping, alignment, and mixing. Scripts are scattered around, parameters and intermediate products are unmanaged, and if it crashes midway, you have to start over.

The dsh-cover introduced below addresses this: a DSH Host plugin that strings together source retrieval, stem separation, key detection, pitch shifting, voice swapping, alignment, mixing, QC, and export into a nine-stage pipeline. It is triggered by natural language, and outputs and QC results are saved to a fixed directory. The maintainer is lol9706262-source, the license is MIT, and the version is 1.0.0.

What is it¶

One-sentence positioning: You provide a song title (or link, local file) and a voice ID via a sentence, and the plugin runs the full cover pipeline, handing the WAV and 320k MP3 back to the Agent.

Key facts:

• Pure Host plugin, zero npm dependencies at runtime. All DSP is in runtime/*.py, executed by Applio’s Python interpreter.
• It needs to be loaded when the DSH main process starts, so you must restart the DSH main process after installation, do not open a new session.
• Three tools: cover_doctor (environment self-check), cover_make (full process), cover_status (read run.json to see progress and gate results).

Installation and Enabling¶

The installation command is per the original repository:

dsh plugin --profile desktop add github:lol9706262-source/dsh-cover

Or install from a local directory (for modified source code or offline running):

dsh plugin --profile desktop add file:C:/path/to/dsh-cover

After installation, run an environment check once (this does not modify any files):

cover_doctor

It checks Python, core.py, separator, ffmpeg, yt-dlp, voice table, GPU, and disk. Fix this before running the official process directly.

To uninstall:

dsh plugin --profile desktop remove dsh-cover

Typical Usage¶

Three entry formats, directly stated:

Cover a song "山水之间" with my voice
Cover https://www.bilibili.com/video/BV1Vb421B7XQ with myvoice
Cover D:\Music\山水之间.mp3 with myvoice

Upon receiving this, the Agent calls cover_make. It sequentially runs the 9 steps: source -> stems -> key -> pitch -> voice swap -> align -> mix -> QC -> export, finally returning the final WAV and 320k MP3.

How Source Retrieval Works¶

The source has three forms, with priority from high to low:

1. Local file exists → Adopt directly, no copy, no transcoding.
2. http(s) link → yt-dlp download; domains containing bilibili.com are marked as bilibili.
3. Song title keywords → Try sequentially by fetchOrder (default inmp3,bilibili,youtube).

Among the three keyword sources, inmp3 has the highest hit rate and requires no cookies, but the bitrate is only 128k or 96~98k AAC; bilibili has the best quality (commonly 184~263k on 30280 tracks); youtube’s ytsearch1: is a fallback, currently blocked by bot checks. The bitrate preference rule is: if the inmp3 bitrate is lower than fetchMinKbps (default 160), automatically retry Bilibili; only replace if Bilibili’s is 8 kbps higher; otherwise keep inmp3. The entire process is recorded in fallback_log.

A common pitfall: when using keyword source, use the keyword itself as the directory name (e.g., 晴天 周杰伦), not the Bilibili title—Bilibili titles often contain decorations like “【Hi-Res无损】” which messes up the directory name. The true matched title is in run.json’s fetch.picked.title.

How the Three Tools Divide Tasks¶

When troubleshooting, check cover_status first; it directly extracts the current stage and time from run.json without needing to sift through logs yourself.

Artifacts and Resume¶

Structure under <coversRoot>\<Song Title>\:

run.json              Status and time per stage, can resume if interrupted
src/                  Source (URL -> WAV)
sep/wav/              Vocals / Instrumental separated by UVR-MDX
sep/instrumental_shiftN.wav   Pitch-shifted accompaniment (generated only if |pitch|≥1.5 and != 0)
conv/vocal_<voiceId>.wav     Voice swap result (44.1 kHz mono)
mix/report.json       Parameters and QC measured values for three variants
mix/gates.json        Gate-by-gate judgments
out/*_standard.wav    Final WAV (three variants + dry)
out/*.mp3             320k MP3
out/provenance.json   Model hash / key basis / alignment results / gate conclusions

Two design convergences: audio is only written to out/, mix/ only keeps report.json and gates.json, and the same WAV is not stored twice. run.json includes inputs fingerprints (file size + mtime, plus pitch/voice/chunk params for the voice swap step) for each stage; if fingerprints don’t match, it recalculates and won’t use old conclusions. run.json without the inputs field is considered expired.

How Pitching is Done¶

Instead of guessing by running inference first, it directly compares two f0 medians:

Semitones = 12 × log2(Voice Median / Original Median)

The voice median comes from f0_median_hz in _voices/<voiceId>.json (training set f0 median, used as a baseline for “how high you can comfortably sing”). The original median is extracted from the original track using pyin. If the high position exceeds the voice limit by more than 3.5 semitones → lower pitch; if the low position is insufficient → raise pitch; otherwise keep 0. --pitch can hard override.

Then it categorizes by absolute semitone number:

| |pitch| | Mode | Action |
|—|—|—|
| < 1.5 | none | Take 0, do nothing |
| ≥ 1.5 | both | Move vocal --pitch + accompaniment asetrate/atempo together |

There is no “vocal-only” tier. If vocals are shifted and accompaniment stays in the original key, the whole song will definitely be out of tune. Early versions had a vocal-only option, but testing showed it was wrong and has been deleted. This is a crucial trade-off in this pipeline: better not to shift than to shift only half.

Long Song Processing under VRAM Constraints¶

convert.py defaults to --chunk-seconds 120, snapping cut points to the lowest energy frames in ±8 sec windows (breaths or interludes), with 0.3 sec overlap between blocks for equal-power cross-fading; if a single block fails, it automatically halves the chunk, with a lower limit of 20 seconds.

The reason for chunking is specific: feeding a 4m30s vocal track into RVC at once causes cuDNN to throw RuntimeError: GET was unable to find an engine to execute this computation—that is the CUDA-side ALLOC_FAILED translated by torch. On a 1050 Ti 4GB in the repo, 4m12s passes, 4m30s crashes, with the memory peak hitting the limit.

The second pitfall is sample rate. Applio outputs the model’s own sample rate (commonly 40000 or 48000 Hz), not the input file’s. Early versions treated 40000 Hz samples as 44100 Hz, effectively compressing every block by 0.907x and padding with zeros, resulting in a fragmented song, manifesting as “alignment deviation 1800ms, correlation 0.36”. Now it reads back and uses librosa.resample to the target sample rate before stitching. This has been fixed and is a real pitfall that was encountered.

Mixing and Gating¶

Mixing produces three variants. Default parameters are as follows (repo says reproducible: peak -0.58 / RMS -13.1 / crest 12.5):

Mastering order is fixed: first boost gain to -13 dBFS RMS, then tanh soft limiter (ceiling 0.94 / knee 0.65). Reversing this flattens dynamics.

Final gating:

4.5–9.5 kHz narrowband whistle metrics are not used for judgment, only output as reference values. If gating fails, export is not done by default; force: true can force it. The dry track is not subject to the final gate.

Parameter Quick Reference¶

Common parameters for cover_make:

How to Configure the Voice Table¶

<voicesDir>\<voiceId>.json, one voice per file, key is the voice id:

{
  "myvoice": {
    "pth": "<applioRoot>\\logs\\myvoice\\myvoice_300e_27900s.pth",
    "index": "<applioRoot>\\logs\\myvoice\\myvoice.index",
    "sample_rate": 40000,
    "f0_median_hz": 172.89,
    "f0_p95_hz": 258.75,
    "f0_min_hz": 53.63,
    "f0_max_hz": 319.34,
    "defaults": { "index_rate": 0.75, "protect": 0.33, "volume_envelope": 1.0, "f0_method": "rmvpe" }
  }
}

Two easy mistakes: sample_rate must be the RVC model’s own sample rate (check training config), not the input audio; f0_median_hz is the key detection baseline. Fill it with your training set f0 median so the plugin can judge if you are comfortable singing this song. After adding new jsons to this directory, cover_doctor will recognize them.

Benchmarks¶

The following are recorded benchmarks in the repo for reference on evaluating time (1050 Ti 4 GB):

Corresponding stage times for “Shan Shui Zhi Jian 276.06s”: source 0.2 s / stems 62 s / key 36 s / voice swap 223 s / align 5.6 s / mix 36 s / QC 0.1 s / export 12 s, total 374 s. The final QC of the product is peak -0.609, RMS -13.032, crest 12.42, clipped 0, and all three variants passed the gate.

Dependency Boundaries and Known Limitations¶

Dependencies: Only need numpy / soundfile / librosa / scipy already in Applio env; does not introduce pyloudnorm / pedalboard / pyrubberband; ffmpeg uses <applioRoot>\ffmpeg.exe (this version’s amix has no normalize, so mixing goes through Python per-sample); pitch shifting uses asetrate + aresample + atempo, duration unchanged.

Known boundaries are stated plainly:

• Only 2-stem UVR separation, cannot do lead vocal/harmony separation.
• No chunking mechanism in the separation stage; long song stem separation VRAM peak depends on audio-separator itself.
• 5m03s (“Ordinary Road”) stems 62s, voice swap 4m56s, total 483s.
• yt-dlp goes to YouTube; domestic networks need self-solving; direct links work, search often blocked by bot checks; for real YouTube search, add cookiesFromBrowser: "chrome" or paste link directly.

Suitable Scenarios and Notes¶

Suitable for scenarios where you already have Applio + RVC voice models and want the Agent to output a cover directly in conversation; especially suitable for 4GB VRAM environments that need chunking to run, and for those who want a fixed directory convention to manage intermediate products without scattered scripts.

Two points to note. First, this is a Host plugin running with the permissions of the current DSH main process. You should read the source code and license (MIT) before installing to ensure compatibility with your existing Applio/RVC environment. Second, it does not solve source retrieval or copyright issues: using yt-dlp to grab Bilibili or YouTube accompaniment, and using it to cover and distribute the product, is the user’s own judgment and responsibility—the plugin itself only strings together retrieval, separation, voice swapping, and QC.

Conclusion¶

The value of dsh-cover lies in compressing the cover pipeline into a single natural language sentence and a voice ID. The intermediate source selection, VRAM chunking control, pitch shifting constrained by hardware, and mixing gating are done by the plugin in nine steps. The output is unified in out/. It doesn’t solve separation granularity or copyright, but it ensures consistency in the three areas of “runs smoothly, crashes are clear, and is reproducible.”

• Directory page: https://www.skillhub.cn/plugins/lol9706262-source/dsh-cover
• GitHub: https://github.com/lol9706262-source/dsh-cover