dsh-incident-studio: Aligning screen frames, log lines, and key/mouse inputs to a single millisecond timeline

Introduction

When developing agents (Agents), a recurring problem arises: you reproduce a fault on a real interface, but the Agent only gets a description you typed and a log file. The obvious bug on screen is lost in hundreds of lines of logs; and “what I pressed at that moment” is only remembered by you. Manually pasting screenshots, GIFs, and scrolling logs to piece together evidence often leaves a piece missing.

dsh-incident-studio (Plugin name: dsh-incident-studio, Maintainer: Eclipse-NotFound, Category: Workflow) is a DSH plugin designed for this purpose. It places screen footage, log lines, and mouse/keyboard inputs on the same millisecond timeline. After recording, you mark, select intervals, drag boxes to annotate, write a sentence, and then one-click fill the keyframe strips, GIFs, aligned log slices, and description into the composer for the Agent.

It is not bound to any specific game or project: log sources are configured, and any program capable of writing files can connect; the program can also push events directly via an HTTP interface.

Core Features

During recording, the plugin synchronously collects four types of evidence, sharing a common wall-clock time:

Evidence Source Accuracy
Visuals Host frame capture (three paths, see below) Default 6 fps, with wall-clock time per frame
Events Log file follower + pinning + HTTP push Write time of line, ~dozens of ms; push can carry precise time
Mouse/Keyboard Polling helper (only when target program is in foreground) ~30 Hz, error ±1 sampling interval
Your Description Annotation box on screen + one sentence Attached to selected interval

Every line of log delivered to the Agent carries a relative timestamp in the format [00:12.340], sourced from the same image frames and key press events. The Agent doesn’t need to guess which line it is, nor does it need to guess what you pressed at that time.

After recording ends, the way you hand content to the Agent is “Artifacts + Compact Summary”: phenomenon, interval, artifact path, annotation description, mouse/keyboard statistics, and up to 80 lines of logs (error lines and pins are prioritized, pin lines carry “previous input”); the full record is in bundle.md.

Installation and Activation

Requires DeepSeek Harness 0.2.1+ (package.json has engines.dsh set to >=0.2.1-rc.1 <0.3.0). The client.js inside the plugin package is committed to the repository, no build required.

# Install directly from repository (client.js is committed, no build needed)
dsh plugin --profile <profile> add https://github.com/Eclipse-NotFound/dsh-incident-studio

The desktop profile is managed by the app itself: install via Settings → Plugins using the repository address. The README specifically warns not to install the CLI into profiles/web, or it won’t appear on the desktop. After installation, you need to restart once for the host side to load; the browser side loads with the host release and pages hot-update.

There are two ways to disable:

# Remove dsh-incident-studio from dsh.profile.bundles, or add to profile's patch:
- id: incident-studio
  disabled: true

Usage Flow

The panel entry is the + in the right sidebar (Guide page “Incident Studio”), or the 🎥 Capture button below the composer. The first mount opens it once and saves it to localStorage.

  1. Configure Log Source (Only need to do this once): Panel → Settings → Fill in the absolute path or relative workspace path of the log file. The file can appear later than the start of recording; starting recording first then launching the program also works. Fill in the Target Process Name on the same page (e.g., Remains), so the mouse/keyboard recording knows which window to record when in the foreground.

  2. Recording: Select source → “● Start recording”. When a problem occurs, press “⚑ Mark”; you can do this multiple times; the mark automatically records the last key you pressed before the problem appeared.

  3. Review and Annotate: The default interval is the last mark ±3 seconds. Moving the playback head locates the log list synchronously; clicking a log line seeks back to the corresponding visual. The Input block can expand every key press/click within the interval, clicking a single item also seeks. Drag a red box on the visual, and each box can have a sentence of description.

  4. Delivery: Build artifacts → Fill composer. Attachments default to keyframe strips, GIF is optional; press Enter to send after confirmation, or click Send now.

Artifact Disk Location

Artifacts are written to .dsh-incidents/<recording-id>/ in the current session workspace:

.dsh-incidents/20261104-193045-a1b2c3/
  frames-20261104193045.png   6-frame horizontal strip, with timestamp and red box
  clip-20261104193045.gif     Interval GIF (460 px wide)
  input.jsonl                 Raw mouse/keyboard events, one JSON per line
  bundle.md                   Complete summary: description, annotation coordinates, all event lines, input records
  bundle.json                 Machine-readable version of the above

Program Side: HTTP Push Interface

Besides following log files, the local process (mod, test script, dev server) can push events directly:

curl -X POST http://127.0.0.1:<dsh-port>/dsh-incident-studio/api/ingest \
  -H "content-type: application/json" \
  -H "x-incident-token: <token shown in panel settings>" \
  -d '{"text":"进入 raiders 房间","src":"RConnect","level":"info"}'

x-incident-token is unique per process and visible in panel settings; pushing from the same-origin panel doesn’t require a token. If the request body carries wall (epoch milliseconds), it is pushed to the timeline at the time you provided; otherwise, the time of reception is used.

Three Capture Paths

The plugin selects the path based on the host process’s capabilities. The products of all three paths are “frames with wall-clock time,” and subsequent annotation, export, and delivery are identical:

Host Process Capability Path Description
electron.desktopCapturer Timed JPEG capture Default 6 fps / 960 px / q62, can select source by window
Only Node (Electron utility process / pure dsh web) Windows GDI screen capture helper tools/grab-frames.ps1 Entire virtual desktop; persistent PowerShell writing JPEGs, host deletes temp files after receiving frames
Real browser opening dsh web getDisplayMedia + MediaRecorder Requires screen sharing authorization, local frame extraction after stopping recording

README notes a test conclusion: DSH desktop host cannot get desktopCapturer (nativeReason: "desktopCapturer unavailable" in /state), so the desktop actually uses GDI. GDI uses System.Drawing + Graphics.CopyFromScreen, approx 5KB per frame at 320 px / 4 fps. The trade-off is the inability to select a single window. The frame buffer default limit is 900 frames / 96 MB / 5 minutes, oldest frames are dropped if exceeded.

Trade-offs for Mouse/Keyboard Recording

tools/record-input.ps1 is a persistent helper, reading GetAsyncKeyState every 20 ms, only writing changes. Designed to deliberately not install keyboard hooks, do key translation, or touch ToUnicode: hooks look like keyloggers and drag system input delay into your process. Recording scope defaults to when the target program holds the foreground; switching away writes a log entry.

First startup does a real probe, result in $DSH_HOME/storages/dsh-incident-studio/input-capability.json, visible in panel settings as available key counts or specific reasons. Depends on Windows + built-in Windows PowerShell 5.1.

Config is written to <DSH_HOME>/storages/dsh-incident-studio/profiles.json (DSH_HOME defaults to ~/.dsh), changing this file changes panel settings:

{
  "version": 1,
  "activeId": "my-project",
  "profiles": [
    {
      "id": "my-project",
      "name": "我的项目",
      "sources": [
        { "id": "main", "label": "主日志", "color": "#6aa9ff", "kind": "file",
          "path": "C:/Users/you/AppData/Roaming/MyApp/logs/main.log", "enabled": true, "preRoll": true }
      ],
      "input": { "enabled": true, "targets": "Remains.exe", "scope": "target", "pollMs": 20, "trailHz": 0 }
    }
  ]
}

path does not expand %VAR% and $HOME; preRoll (default on) captures the file tail up to 96 KB / 800 lines and marks it as “pre-recording” when recording starts; input.targets is the target process name (max 8), when scope is "all", it records all foreground windows except the plugin itself.

Applicable Scenarios and Notes

Suitable for people who need to reproduce problems on a real interface—scenarios like game mods, GUI clients, frontend interactions where “one look makes it clear but finding it in logs is hard.” The plugin is not bound to a project; log sources and target processes are configured by you, allowing you to follow multiple log files simultaneously.

Points to note:

  • The plugin runs with the permissions of the current dsh host process; frame capture, mouse/keyboard collection, and HTTP push may touch local sensitive data. Before installing or enabling, you should read the source code and LICENSE (MIT) to confirm the boundaries of data collection meet your environment requirements.
  • Frame capture and mouse/keyboard recording only occur after pressing “Start Recording”, stopping stops it. The desktop GDI captures the entire virtual desktop; please move sensitive windows away first.
  • Mouse/keyboard recording does not record any text content, only when the target program holds the foreground; the plugin host process itself is always excluded, and input in the panel is not recorded.

Conclusion

The value of this plugin lies in verifiability: visuals, logs, and keys on the same timeline. What the Agent gets is no longer “the moment you guessed,” but evidence with coordinates, timestamps, and input sequences. The project’s category in the community directory is “Workflow”, 0 stars, still early stage; functional boundaries and pitfall records are in the README, it is recommended to read through before deciding to introduce it.

Directory page: https://www.skillhub.cn/plugins/Eclipse-NotFound/dsh-incident-studio
Repository: https://github.com/Eclipse-NotFound/dsh-incident-studio

羽毛球分组比赛记分
小程序二维码

欢迎使用《羽毛球分组比赛记分》微信小程序

Xiaoye