mganter/knowledge-base-framework
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Support external vault roots across tooling

Browse Source
This commit is contained in:
TotenKnopf
2026-09-20 17:05:21 +02:00
parent 1cb6369282
commit 293b582e97
16 changed files with 573 additions and 204 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/knowledge-base-ingest-resource/SKILL.md
+56 -77
View File
@@ -9,8 +9,23 @@ Capture an external resource into the vault as a reference note, store its
original bytes under `raw/`, decompose it into topics, and link those topics into
the knowledge graph. Follow every step.
Obey `AGENTS.md` and `02_areas/ways-of-working.md` (read them if not already
loaded this session; if `ways-of-working.md` does not exist yet, skip it).
Before ingesting, call `read_instructions` on `knowledge-database` and read both
returned documents: the framework's `AGENTS.md` and `AGENT_QUERY_GUIDE.md`.
The server resolves their location; they are not data-vault nodes and must not
be fetched with `read_note`. If this call fails, report the missing instructions
before changing the vault.
Keep the caller's task instructions in scope. For additional vault policies,
query `graph_search_nodes(type="policy")` and `graph_search_nodes(type="guideline")`,
then `read_note` relevant results and follow their policy links. Do not assume
`02_areas/ways-of-working.md` exists; an empty search means no policies were found,
not that framework or caller instructions cease to apply.
The `knowledge-database` MCP server is the only interface for vault access: use `graph_summary`, `graph_search_nodes`, `graph_neighbors` and
`read_note` to find context; `capture_raw` for source bytes; `write_note` and
`edit_note` for notes; `refresh_graph(validate=true)` to regenerate and validate.
Use `git_commit` / `git_push` only when the caller explicitly authorizes them.
Skills stay in the framework. Never infer the vault from this skill's location.
The output is a `type: note` reference note at
`03_resources/<resource-name>/index.md`, one `type: note` **topic note** per
@@ -47,10 +62,9 @@ Pick a short kebab-case `<resource-name>` from the title / filename / video
title. Keep it descriptive and unique (e.g. `cyclonedx-sbom-spec`,
`oauth2-rfc6749`).
```bash
ls 03_resources/ # check the slug is not already taken
mkdir -p 03_resources/<resource-name>/raw
```
Check for an existing resource with `graph_search_nodes(query="<resource-name>")`
and `read_note(path="03_resources/<resource-name>/index.md")`. The capture and
write tools create parent directories; no separate folder-creation step is needed.
If the slug already exists, either you are re-ingesting (update in place, tell the
user) or pick a more specific slug.
@@ -59,47 +73,31 @@ user) or pick a more specific slug.
Always preserve the source bytes untouched. Pick the branch that matches step 1.
**Local file** — copy verbatim, keep the original name:
```bash
cp "<source-path>" "03_resources/<resource-name>/raw/"
```
**Local file** — use `capture_raw` with the supplied `source_path` and a
vault-relative `path` such as `03_resources/<resource-name>/raw/<filename>`.
This preserves the original bytes and filename.
**PDF / binary at a URL** — download verbatim:
```bash
curl -fsSL "<url>" -o "03_resources/<resource-name>/raw/<filename>"
```
**PDF / binary at a URL** — use `capture_raw(url="<url>", path=".../raw/<filename>")`.
Check the returned byte count and SHA-256.
**Webpage / link** — record the URL and save the readable content. Use `WebFetch`
to retrieve it (request markdown), then write the result and a `source.url`:
```bash
printf '%s\n' "<url>" > "03_resources/<resource-name>/raw/source.url"
```
Save the fetched body to `raw/page.md` (write the `WebFetch` output to that file).
**Webpage / link** — save the URL using
`write_note(path=".../raw/source.url", content="<url>\n")`. Capture the original
page with `capture_raw(url="<url>", path=".../raw/page.html")`. If the invoking
agent allows `WebFetch`, it may fetch a readable version of that external source;
save the returned body with `write_note(path=".../raw/page.md", content=...)`.
**YouTube video** — pull the transcript into `raw/`. Record the URL first, then
try, in order:
```bash
printf '%s\n' "<url>" > "03_resources/<resource-name>/raw/source.url"
# Preferred: yt-dlp grabs subs + metadata without downloading the video
yt-dlp --skip-download --write-subs --write-auto-subs --sub-lang en \
--convert-subs srt --write-info-json \
-o "03_resources/<resource-name>/raw/%(title)s.%(ext)s" "<url>"
# Fallback if yt-dlp is absent: python transcript API
python3 -c "import youtube_transcript_api" 2>/dev/null && \
python3 -c "from youtube_transcript_api import YouTubeTranscriptApi as A; \
import sys; vid='<VIDEO_ID>'; \
open('03_resources/<resource-name>/raw/transcript.txt','w').write(\
'\n'.join(s['text'] for s in A.get_transcript(vid)))"
```
If neither tool is installed, tell the user which to install (`pip install yt-dlp`
or `pip install youtube-transcript-api`) and stop before fabricating a transcript.
Never invent transcript text.
**YouTube video** — record the URL with `write_note`. If a transcript is supplied,
capture its file with `capture_raw(source_path=..., path=".../raw/transcript.txt")`
or preserve supplied text with `write_note`. These MCP tools do not extract
YouTube transcripts. If none is supplied, report the missing transcript and stop
before writing content notes. Never invent transcript text or run shell extractors.
## 4. Read the captured content
Read what you just stored in `raw/` (the file, `page.md`, the transcript). This —
Use `read_note(path="03_resources/<resource-name>/raw/<filename>")` for captured
text. Path reads access the file directly and work before graph regeneration.
For a captured binary, use the invoking agent’s explicitly permitted inspection
tool on the absolute path returned by `capture_raw`; if unavailable, report it. This —
not the live URL — is the source of truth for the rest of the steps, so the note
matches the bytes we preserved. For long content, read enough to identify every
distinct topic, not just the intro.
@@ -110,24 +108,10 @@ Reason about the **individual topics** inside the resource — the concrete
concepts, standards, formats, tools, or entities it discusses. For each topic,
decide whether the vault already knows about it, and whether it points outward.
For every candidate topic, search the vault for a matching note:
```bash
# free-text over the graph (id / title / path / tags)
jq -r --arg q "<topic>" '.nodes[]
| select((.id|ascii_downcase|contains($q)) or (.title|ascii_downcase|contains($q)))
| "\(.id)\t\(.path)"' index/graph.json
# by tag
jq -r --arg t "<topic-tag>" '.nodes[] | select(.tags[]? == $t) | "\(.id)\t\(.path)"' index/graph.json
# full-text fallback across the PARA buckets
grep -rIl "<topic>" 01_projects 02_areas 03_resources 04_archives
```
Prefer the `knowledge-database` MCP server when available (`graph_search_nodes`,
`graph_neighbors`, `read_note`) — it is the same index. Open promising matches and
confirm they are really the same concept before linking.
For every candidate topic, use `graph_search_nodes(query="<topic>")` and tag
filters from `graph_summary`, then traverse `graph_neighbors` and `read_note`
promising matches. Confirm they cover the same concept before linking.
The query searches metadata, not note bodies; read candidate notes when needed.
Classify each topic:
- **Internal** — the vault has a note for it → link with
@@ -145,7 +129,7 @@ Classify each topic:
## 6. Write a note per topic
Each distinct topic from step 5 becomes its own note at
Use `write_note` to create each distinct topic from step 5 at
`03_resources/<resource-name>/<topic-name>.md` (kebab-case slug). This is what
makes topics first-class, individually linkable graph nodes.
@@ -188,11 +172,11 @@ the *source* needs from the *destination* — "what to expect there".
- Link each topic note back to its resource with
`[<resource title>](resource/<resource-name>){"type":"part-of"}`, and outward to
internal notes (same link form) and external sources (URLs).
- Legacy `[[wiki links]]` are rejected by `make validate-graph` — never write them.
- Legacy `[[wiki links]]` are rejected by `refresh_graph(validate=true)` — never write them.
## 7. Write the resource note (index)
Filename: `03_resources/<resource-name>/index.md`. This is the hub that ties the
Use `write_note` for `03_resources/<resource-name>/index.md`. This hub ties the
raw source to the topic notes.
```md
@@ -230,7 +214,7 @@ last_reviewed: <today, YYYY-MM-DD>
- `ingested` and `last_reviewed` = today (the `currentDate` from session context).
- `type` must stay within the vault taxonomy enum
(`hub|project|area|guideline|policy|feature|roadmap|decision|note|meta|reference`),
which `generate-graph.py --validate` enforces — use `note`;
which `refresh_graph(validate=true)` enforces — use `note`;
the `resource` tag is what marks it.
- Markdown link targets resolve by `id`, rel-path, title, or unique stem — always
use the `id` as the target, e.g. `[Human title](resource/foo/bar)`.
@@ -244,16 +228,14 @@ So the resource is reachable in the graph, not just reaching out:
the most relevant existing notes (the strong internal topics from step 5) — e.g.
under a `## Resources` or `## Related` section, creating it if absent. Add a
`{"type":"…","link-info":"…"}` brace where the relationship is more specific than
`related`. Use individual edits.
`related`. Use `edit_note` for each existing note.
- If several resources accumulate, maintain `03_resources/index.md` as a hub that
lists them, and link the new note from there (create the hub if it does not
exist, with `type: hub`, `tags: [resource, hub]`).
## 9. Regenerate and validate the graph
```bash
make graph
```
Use `refresh_graph(validate=true)`.
This regenerates `index/graph.json` and validates links and edge relationship
types. If validation reports unresolved links, fix the targets (typos, wrong `id`,
@@ -265,17 +247,14 @@ invalid graph, so this must pass.
## 10. Stage (do not commit unless asked)
Per `ways-of-working.md`, NEVER use `git add -A` or `git add .`. Stage each
touched path individually:
Without explicit publishing authorization, leave the changes uncommitted and
report the touched paths. The MCP interface has no stage-only tool.
```bash
git add 03_resources/<resource-name>/ # note + raw/
git add index/graph.json
git add <any-existing-note-you-backlinked>.md
```
Only commit or open a PR if the user explicitly asks. Changes go via small, clear
pull requests.
When a commit is authorized, use `git_commit(message=..., paths=[...])` with an
explicit list covering the resource folder, backlinked notes and `index/graph.json`.
This tool stages those paths and commits them; do not use wildcard paths.
Use `git_push` only when pushing is explicitly authorized. Follow the caller's
branch and review policy.
## 11. Report
skills/knowledge-base-read/SKILL.md
+37 -53
View File
@@ -11,8 +11,18 @@ Markdown is the source of truth; `index/graph.json` is a generated navigation
index. Use this skill to answer questions FROM the vault and to surface the
exact notes that back the answer.
Obey `AGENTS.md` (read them if not loaded this
session). This skill is read-only — do not modify notes here.
Call `read_instructions` on `knowledge-database` to read the framework's
`AGENTS.md` and `AGENT_QUERY_GUIDE.md` before querying the vault. These documents
are not data-vault nodes. If unavailable, report the missing instructions.
This skill is read-only — do not modify notes here.
## Vault access
The `knowledge-database` MCP server is the only interface for vault access. Use `graph_summary` to orient, `graph_search_nodes` to find notes,
`graph_neighbors` to traverse links, and `read_note` to read the source.
Use `refresh_graph` when the generated index is stale. Skills and operational
documentation belong to the framework; never infer the data vault from this
skill's file location.
## Layout (PARA)
@@ -25,84 +35,58 @@ session). This skill is read-only — do not modify notes here.
03_resources/repos/repo-landscape.md index of documented repos
04_archives/ inactive items
index/graph.json generated graph (nodes + edges)
README.md vault hub; AGENT_QUERY_GUIDE.md graph query reference
```
Every note carries taxonomy frontmatter: `id`, `title`, `type`
(`hub|project|area|guideline|policy|feature|roadmap|decision|note|meta|reference`, enforced by
`generate-graph.py --validate`), `status`
`refresh_graph(validate=true)`), `status`
(`active|draft|deprecated`), `tags`, `last_reviewed`, optional `repo`. Notes are
linked with typed markdown links — `[text](note-id){"type":"…"}` — that resolve
by `id`. Legacy `[[wiki links]]` are rejected by `make validate-graph`.
by `id`. Legacy `[[wiki links]]` are rejected by `refresh_graph(validate=true)`.
## 1. Orient
Start broad, then narrow:
- `README.md` — vault hub and entry points.
- `03_resources/repos/repo-landscape.md` — index of every documented repo, grouped.
- `02_areas/ways-of-working.md` — global engineering policy.
- `03_resources/meta/taxonomy.md` — what the frontmatter fields and PARA buckets mean.
For a repo question, read `03_resources/repos/<repo>/info.md` directly
(`ls 03_resources/repos/` to confirm the repo is documented; if absent, say so —
it is not yet in the vault).
Start broad, then narrow. Begin with `graph_summary`, then find hubs using
`graph_search_nodes(type="hub")` and read relevant ones with `read_note`.
Discover repo scopes and tags from the graph rather than assuming a particular
folder layout. An external data vault need not contain a root README or
framework operational documents.
## 2. Query the graph
`index/graph.json` is for navigation by tag / type / repo / links. Use `jq`.
Full schema and more examples are in `AGENT_QUERY_GUIDE.md`.
Use the MCP tools directly:
```bash
# all documented repos
jq -r '.nodes[].repo' index/graph.json | sort -u
| Need | Tool |
|------|------|
| Repo scopes, tag spellings, most-linked notes | `graph_summary` |
| Notes in a repo | `graph_search_nodes(repo="<repo>")` |
| Notes of a type | `graph_search_nodes(type="policy")` |
| Notes with a tag | `graph_search_nodes(tag="<tag>")` |
| Search ids, titles, paths and tags | `graph_search_nodes(query="<term>")` |
| Incoming/outgoing links, relationship types and annotations | `graph_neighbors(id="<note-id>")` |
# every note in one repo
jq -r '.nodes[] | select(.repo=="tenant-controller") | "\(.id)\t\(.path)"' index/graph.json
# all notes of a type (e.g. policies, decisions, projects)
jq -r '.nodes[] | select(.type=="policy") | "\(.id)\t\(.path)"' index/graph.json
# notes carrying a tag
jq -r '.nodes[] | select(.tags[]? == "kcp") | "\(.id)\t\(.path)"' index/graph.json
# most-linked (central) notes
jq -r '.nodes | sort_by(-(.inbound_count // 0))[] | "\(.inbound_count // 0)\t\(.id)"' index/graph.json | head
# outgoing links from a note, with relationship type + annotation
NOTE_ID="repo/controller-util/info"
jq -r --arg id "$NOTE_ID" '.edges[] | select(.from==$id) | "\(.type)\t\(.to)\t\(.annotation)"' index/graph.json
# incoming links to a note (who references it, and how)
jq -r --arg id "$NOTE_ID" '.edges[] | select(.to==$id) | "\(.type)\t\(.from)\t\(.annotation)"' index/graph.json
# edges of a specific relationship type (e.g. supersedes, depends-on)
jq -r '.edges[] | select(.type=="supersedes") | "\(.from) -> \(.to)"' index/graph.json
```
Full-text fallback when you do not know the tag/type/id:
```bash
grep -rIl "<term>" 01_projects 02_areas 03_resources 04_archives
```
`query` does not search note bodies. Read candidate notes with `read_note` and
follow their neighbors when metadata search is insufficient. Report any search
coverage limits; do not use filesystem or shell searches.
## 3. Read the source notes
The graph gives ids and paths; the ANSWER lives in the Markdown. Open the
matched `.path` files and read them. Follow the typed markdown links
The graph gives ids and paths; the ANSWER lives in the Markdown. Read matched
notes with `read_note(id="<note-id>")` or `read_note(path="<vault-relative-path>")`. Follow the typed markdown links
`[text](id){"type":"…"}`, resolving each to its node `path` to pull in related
policies, decisions, and repo info. Use each edge's `type` and `annotation` to
judge which links are worth following; links are the intended navigation, so
traverse them rather than guessing.
If the graph looks stale or a path is missing, you may regenerate read-only:
`python3 scripts/generate-graph.py` (does not touch Markdown). Do not commit.
If the graph looks stale or a path is missing, use `refresh_graph`. This updates
only the generated index; it does not touch Markdown. Do not commit.
## 4. Answer with citations
- Answer the question directly from what the notes say.
- Cite the backing notes by vault-relative `path` (e.g.
`03_resources/repos/gittea/info.md`) and/or `id` so the user can verify.
`03_resources/<resource>/index.md`) and/or `id` so the user can verify.
- Distinguish vault facts from inference. If the vault does not cover it, say so
plainly — do not invent. Note `status: draft`/`deprecated` when relevant.
- For "what links to / relates to X" questions, report the inbound/outbound