Introduction¶

DeepSeek Harness (DSH) adopts a plugin-based architecture, and its ecosystem already includes several memory plugins. dsh-memory-search is a long-term memory retrieval plugin developed by JackZo400. It does not rely on cloud APIs; instead, it uses local embeddings and SQLite FTS5 to implement hybrid retrieval, addressing the problem that Agents forget context over time.

Core Features¶

This plugin focuses on converting local Markdown notes into memories that are searchable by an Agent.

1. Hybrid Retrieval: Combines local embedding vector search with SQLite FTS5 keyword search.
2. RRF Fusion: Uses Reciprocal Rank Fusion to merge the results from the two retrieval paths, balancing semantics and keywords.
3. Zero API Cost: The vector model runs locally, so no external services are required.
4. Fallback Capability: It can still be used without installing a model (keyword search only), suitable for lightweight scenarios.
5. Dual Output: Supports the Agent actively calling the memory_search tool, as well as automatically retrieving relevant memories each turn and injecting them into the context.
6. Confidentiality Filtering: Supports filtering memory visibility based on file name or session mapping (for example, public, personal, secret).
7. Incremental Indexing: Supports incrementally rebuilding the index based on file modification time (mtime) and file size.

Installation and Configuration¶

Installation¶

The installation command is as follows:

dsh plugin --profile web add github:JackZo400/dsh-memory-search

Environment Requirements¶

• Node.js version must be ≥ 22.5 (uses built-in node:sqlite).
• To enable vector retrieval, install node-llama-cpp and the corresponding GGUF embedding model (this dependency is listed in optionalDependencies and does not affect keyword retrieval).

Configuration Example¶

After installation, configuration must be inserted into the patch of the corresponding profile. The following is a basic configuration example:

- insert:
    - id: memory-search
      name: dsh-memory-search
      config:
        dbPath: .dsh-memory/index.db    # 索引库路径，相对路径基于 dsh 工作目录
        roots:                          # 语料根目录，递归收集 .md 文件
          - ./memory
        modelPath: ''                   # 嵌入模型路径；留空则只跑关键词
        recall:
          enabled: true                 # 开启每轮自动召回
          limit: 4                      # 每轮最多召回条数
          budgetChars: 1600             # 注入上下文的字符预算
          order: 130                    # 在系统提示中的排序位置
        embedding:
          enabled: true
          threads: 6                    # CPU 线程数
          contextSize: 1024             # 上下文窗口
          batchSize: 256
          queryInstruction: ''          # 可选：嵌入指令前缀
        search:
          candidatePool: 40             # 每路召回的候选数量
          rrfK: 10                      # RRF 平滑常数
        exclude: ['**/.index/**', '**/node_modules/**']
        syncIntervalMs: 300000          # 扫描语料变更的时间间隔（毫秒）
        acceptSpecChange: false         # 换切块规则时需设为 true

Configuration Key Points¶

• Corpus Root Directories (roots): If not configured, the index remains empty. It is recommended to place Markdown notes under this directory.
• Model Path (modelPath): When left empty, the plugin only builds a keyword index. If embedding.enabled is enabled but the model path is not provided, semantic retrieval will be unavailable.
• Chunking Rules (acceptSpecChange): The plugin uses heuristic chunking (based on blank lines, lists, and headings). If you need to modify the chunking logic and recompute vectors, you must set acceptSpecChange to true.
• Confidentiality Filtering: When sessionMap is not configured, all sessions are treated as the owner, and no filtering is applied.

How It Works and Corpus Preparation¶

Workflow¶

1. Scanning and Chunking: The plugin scans .md files according to roots, and chunks them using heuristic rules such as blank lines, list items, and headings.
2. Index Building:
   • Vector index: Converts chunk content into vectors and stores them in SQLite BLOBs.
   • Keyword index: Builds an FTS5 index using Chinese bigram tokenization.
3. Query Retrieval:
   • The user query is processed by vector search and keyword search separately.
   • The two result lists are fused using RRF according to their ranks.
   • Filtering is performed according to confidentiality rules and source weights.
   • The Top-K results are returned.

Corpus Preparation Recommendations¶

• Chunking Guidelines: It is recommended to write one item per chunk, keep blank lines between paragraphs, and avoid putting two related items in the same chunk, which can affect retrieval accuracy.
• Confidentiality Files: File names can be used to distinguish confidentiality levels, such as public.md, personal.md, and secret.md.

Testing¶

The project provides testing scripts to verify chunking, tokenization, indexing, and retrieval.

node test/selftest.mjs         # 自检切块、分词、索引与检索
node test/plugin-selftest.mjs  # 插件契约测试（工具注册、提示注入等）
node test/spec-guard.mjs       # 切块规范闸测试
npm test                       # 执行上述所有测试

If you need to evaluate retrieval quality, you can run the evaluation script (modify the questions based on your own corpus):

node test/recall-eval.mjs --db .dsh-memory/index.db --corpus ./memory

Known Limitations¶

1. Chunking Rules: It relies on heuristic rules (blank lines/lists/headings), and the handling of long-paragraph notes is not as precise as dedicated segmentation models.
2. Chinese Tokenization: It only supports bigram tokenization, lacks a vocabulary list, and has limited grammatical understanding.
3. Model Compatibility: It has only been validated on Qwen3-Embedding-0.6B. When switching models, note the dimensions and the queryInstruction setting.
4. Performance: Vector retrieval uses a full scan; performance may degrade when the corpus is too large (tens of thousands of chunks).