mganter/knowledge-base-framework
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Support external vault roots across tooling

Browse Source
This commit is contained in:
TotenKnopf
2026-09-20 17:05:21 +02:00
parent 1cb6369282
commit 293b582e97
16 changed files with 573 additions and 204 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/knowledge-base-ingest-resource/SKILL.md
+56 -77
View File
@@ -9,8 +9,23 @@ Capture an external resource into the vault as a reference note, store its
original bytes under `raw/`, decompose it into topics, and link those topics into
the knowledge graph. Follow every step.
Obey `AGENTS.md` and `02_areas/ways-of-working.md` (read them if not already
loaded this session; if `ways-of-working.md` does not exist yet, skip it).
Before ingesting, call `read_instructions` on `knowledge-database` and read both
returned documents: the framework's `AGENTS.md` and `AGENT_QUERY_GUIDE.md`.
The server resolves their location; they are not data-vault nodes and must not
be fetched with `read_note`. If this call fails, report the missing instructions
before changing the vault.
Keep the caller's task instructions in scope. For additional vault policies,
query `graph_search_nodes(type="policy")` and `graph_search_nodes(type="guideline")`,
then `read_note` relevant results and follow their policy links. Do not assume
`02_areas/ways-of-working.md` exists; an empty search means no policies were found,
not that framework or caller instructions cease to apply.
The `knowledge-database` MCP server is the only interface for vault access: use `graph_summary`, `graph_search_nodes`, `graph_neighbors` and
`read_note` to find context; `capture_raw` for source bytes; `write_note` and
`edit_note` for notes; `refresh_graph(validate=true)` to regenerate and validate.
Use `git_commit` / `git_push` only when the caller explicitly authorizes them.
Skills stay in the framework. Never infer the vault from this skill's location.
The output is a `type: note` reference note at
`03_resources/<resource-name>/index.md`, one `type: note` **topic note** per
@@ -47,10 +62,9 @@ Pick a short kebab-case `<resource-name>` from the title / filename / video
title. Keep it descriptive and unique (e.g. `cyclonedx-sbom-spec`,
`oauth2-rfc6749`).
```bash
ls 03_resources/ # check the slug is not already taken
mkdir -p 03_resources/<resource-name>/raw
```
Check for an existing resource with `graph_search_nodes(query="<resource-name>")`
and `read_note(path="03_resources/<resource-name>/index.md")`. The capture and
write tools create parent directories; no separate folder-creation step is needed.
If the slug already exists, either you are re-ingesting (update in place, tell the
user) or pick a more specific slug.
@@ -59,47 +73,31 @@ user) or pick a more specific slug.
Always preserve the source bytes untouched. Pick the branch that matches step 1.
**Local file** — copy verbatim, keep the original name:
```bash
cp "<source-path>" "03_resources/<resource-name>/raw/"
```
**Local file** — use `capture_raw` with the supplied `source_path` and a
vault-relative `path` such as `03_resources/<resource-name>/raw/<filename>`.
This preserves the original bytes and filename.
**PDF / binary at a URL** — download verbatim:
```bash
curl -fsSL "<url>" -o "03_resources/<resource-name>/raw/<filename>"
```
**PDF / binary at a URL** — use `capture_raw(url="<url>", path=".../raw/<filename>")`.
Check the returned byte count and SHA-256.
**Webpage / link** — record the URL and save the readable content. Use `WebFetch`
to retrieve it (request markdown), then write the result and a `source.url`:
```bash
printf '%s\n' "<url>" > "03_resources/<resource-name>/raw/source.url"
```
Save the fetched body to `raw/page.md` (write the `WebFetch` output to that file).
**Webpage / link** — save the URL using
`write_note(path=".../raw/source.url", content="<url>\n")`. Capture the original
page with `capture_raw(url="<url>", path=".../raw/page.html")`. If the invoking
agent allows `WebFetch`, it may fetch a readable version of that external source;
save the returned body with `write_note(path=".../raw/page.md", content=...)`.
**YouTube video** — pull the transcript into `raw/`. Record the URL first, then
try, in order:
```bash
printf '%s\n' "<url>" > "03_resources/<resource-name>/raw/source.url"
# Preferred: yt-dlp grabs subs + metadata without downloading the video
yt-dlp --skip-download --write-subs --write-auto-subs --sub-lang en \
--convert-subs srt --write-info-json \
-o "03_resources/<resource-name>/raw/%(title)s.%(ext)s" "<url>"
# Fallback if yt-dlp is absent: python transcript API
python3 -c "import youtube_transcript_api" 2>/dev/null && \
python3 -c "from youtube_transcript_api import YouTubeTranscriptApi as A; \
import sys; vid='<VIDEO_ID>'; \
open('03_resources/<resource-name>/raw/transcript.txt','w').write(\
'\n'.join(s['text'] for s in A.get_transcript(vid)))"
```
If neither tool is installed, tell the user which to install (`pip install yt-dlp`
or `pip install youtube-transcript-api`) and stop before fabricating a transcript.
Never invent transcript text.
**YouTube video** — record the URL with `write_note`. If a transcript is supplied,
capture its file with `capture_raw(source_path=..., path=".../raw/transcript.txt")`
or preserve supplied text with `write_note`. These MCP tools do not extract
YouTube transcripts. If none is supplied, report the missing transcript and stop
before writing content notes. Never invent transcript text or run shell extractors.
## 4. Read the captured content
Read what you just stored in `raw/` (the file, `page.md`, the transcript). This —
Use `read_note(path="03_resources/<resource-name>/raw/<filename>")` for captured
text. Path reads access the file directly and work before graph regeneration.
For a captured binary, use the invoking agent’s explicitly permitted inspection
tool on the absolute path returned by `capture_raw`; if unavailable, report it. This —
not the live URL — is the source of truth for the rest of the steps, so the note
matches the bytes we preserved. For long content, read enough to identify every
distinct topic, not just the intro.
@@ -110,24 +108,10 @@ Reason about the **individual topics** inside the resource — the concrete
concepts, standards, formats, tools, or entities it discusses. For each topic,
decide whether the vault already knows about it, and whether it points outward.
For every candidate topic, search the vault for a matching note:
```bash
# free-text over the graph (id / title / path / tags)
jq -r --arg q "<topic>" '.nodes[]
| select((.id|ascii_downcase|contains($q)) or (.title|ascii_downcase|contains($q)))
| "\(.id)\t\(.path)"' index/graph.json
# by tag
jq -r --arg t "<topic-tag>" '.nodes[] | select(.tags[]? == $t) | "\(.id)\t\(.path)"' index/graph.json
# full-text fallback across the PARA buckets
grep -rIl "<topic>" 01_projects 02_areas 03_resources 04_archives
```
Prefer the `knowledge-database` MCP server when available (`graph_search_nodes`,
`graph_neighbors`, `read_note`) — it is the same index. Open promising matches and
confirm they are really the same concept before linking.
For every candidate topic, use `graph_search_nodes(query="<topic>")` and tag
filters from `graph_summary`, then traverse `graph_neighbors` and `read_note`
promising matches. Confirm they cover the same concept before linking.
The query searches metadata, not note bodies; read candidate notes when needed.
Classify each topic:
- **Internal** — the vault has a note for it → link with
@@ -145,7 +129,7 @@ Classify each topic:
## 6. Write a note per topic
Each distinct topic from step 5 becomes its own note at
Use `write_note` to create each distinct topic from step 5 at
`03_resources/<resource-name>/<topic-name>.md` (kebab-case slug). This is what
makes topics first-class, individually linkable graph nodes.
@@ -188,11 +172,11 @@ the *source* needs from the *destination* — "what to expect there".
- Link each topic note back to its resource with
`[<resource title>](resource/<resource-name>){"type":"part-of"}`, and outward to
internal notes (same link form) and external sources (URLs).
- Legacy `[[wiki links]]` are rejected by `make validate-graph` — never write them.
- Legacy `[[wiki links]]` are rejected by `refresh_graph(validate=true)` — never write them.
## 7. Write the resource note (index)
Filename: `03_resources/<resource-name>/index.md`. This is the hub that ties the
Use `write_note` for `03_resources/<resource-name>/index.md`. This hub ties the
raw source to the topic notes.
```md
@@ -230,7 +214,7 @@ last_reviewed: <today, YYYY-MM-DD>
- `ingested` and `last_reviewed` = today (the `currentDate` from session context).
- `type` must stay within the vault taxonomy enum
(`hub|project|area|guideline|policy|feature|roadmap|decision|note|meta|reference`),
which `generate-graph.py --validate` enforces — use `note`;
which `refresh_graph(validate=true)` enforces — use `note`;
the `resource` tag is what marks it.
- Markdown link targets resolve by `id`, rel-path, title, or unique stem — always
use the `id` as the target, e.g. `[Human title](resource/foo/bar)`.
@@ -244,16 +228,14 @@ So the resource is reachable in the graph, not just reaching out:
the most relevant existing notes (the strong internal topics from step 5) — e.g.
under a `## Resources` or `## Related` section, creating it if absent. Add a
`{"type":"…","link-info":"…"}` brace where the relationship is more specific than
`related`. Use individual edits.
`related`. Use `edit_note` for each existing note.
- If several resources accumulate, maintain `03_resources/index.md` as a hub that
lists them, and link the new note from there (create the hub if it does not
exist, with `type: hub`, `tags: [resource, hub]`).
## 9. Regenerate and validate the graph
```bash
make graph
```
Use `refresh_graph(validate=true)`.
This regenerates `index/graph.json` and validates links and edge relationship
types. If validation reports unresolved links, fix the targets (typos, wrong `id`,
@@ -265,17 +247,14 @@ invalid graph, so this must pass.
## 10. Stage (do not commit unless asked)
Per `ways-of-working.md`, NEVER use `git add -A` or `git add .`. Stage each
touched path individually:
Without explicit publishing authorization, leave the changes uncommitted and
report the touched paths. The MCP interface has no stage-only tool.
```bash
git add 03_resources/<resource-name>/ # note + raw/
git add index/graph.json
git add <any-existing-note-you-backlinked>.md
```
Only commit or open a PR if the user explicitly asks. Changes go via small, clear
pull requests.
When a commit is authorized, use `git_commit(message=..., paths=[...])` with an
explicit list covering the resource folder, backlinked notes and `index/graph.json`.
This tool stages those paths and commits them; do not use wildcard paths.
Use `git_push` only when pushing is explicitly authorized. Follow the caller's
branch and review policy.
## 11. Report