knowledge-base-framework/skills/knowledge-base-ingest-resource/SKILL.md

name, description

name

description

knowledge-base-ingest-resource

Ingest an external resource (local file, URL/webpage, PDF, YouTube video, etc.) into this knowledge-base vault (PARA-organized). Copies the original into 03_resources/<resource-name>/raw/, reads the content, breaks it into individual topics, and wires each topic to related notes already in the vault (internal typed markdown links) and to outside sources (URLs). For YouTube, pulls the transcript into raw/. Use when the user says "ingest", "add this resource", "save this link/video/paper", or hands you a file/URL to capture and link.

Ingest a Resource

Capture an external resource into the vault as a reference note, store its original bytes under raw/, decompose it into topics, and link those topics into the knowledge graph. Follow every step.

Before ingesting, call read_instructions on knowledge-database and read both returned documents: the framework's AGENTS.md and AGENT_QUERY_GUIDE.md. The server resolves their location; they are not data-vault nodes and must not be fetched with read_note. If this call fails, report the missing instructions before changing the vault.

Keep the caller's task instructions in scope. For additional vault policies, query graph_search_nodes(type="policy") and graph_search_nodes(type="guideline"), then read_note relevant results and follow their policy links. Do not assume 02_areas/ways-of-working.md exists; an empty search means no policies were found, not that framework or caller instructions cease to apply.

The knowledge-database MCP server is the only interface for vault access: use graph_summary, graph_search_nodes, graph_neighbors and read_note to find context; capture_raw for source bytes; write_note and edit_note for notes; refresh_graph(validate=true) to regenerate and validate. Use git_commit / git_push only when the caller explicitly authorizes them. Skills stay in the framework. Never infer the vault from this skill's location.

The output is a type: note reference note at 03_resources/<resource-name>/index.md, one type: note topic note per distinct topic at 03_resources/<resource-name>/<topic-name>.md, and the untouched original preserved in 03_resources/<resource-name>/raw/ — all linked into the graph with typed markdown links ([title](id){"type":"…","link-info":"…"}).

03_resources/<resource-name>/
  index.md            # the resource note (summary, source, topic index)
  <topic-name>.md     # one note per topic distilled from the resource
  <topic-name>.md
  raw/                # original bytes / transcript / fetched page

1. Identify the resource and its type

The resource comes from the conversation — a file path, a URL, a pasted link, or an attachment. Classify it; this drives how you capture it in step 3:

Type	Signal	Capture into raw/
Local file	a path that exists on disk	copy the file verbatim
YouTube video	host is youtube.com / youtu.be	pull the transcript (+ metadata)
Webpage / link	any other http(s) URL	save the fetched page (markdown/html) + URL
PDF / doc	path or URL ending in .pdf/.docx/...	copy/download the binary verbatim

If the resource type is genuinely ambiguous (e.g. a bare string that could be a slug or a search term), ask the user. Otherwise proceed.

2. Derive the resource name and create the folder

Pick a short kebab-case <resource-name> from the title / filename / video title. Keep it descriptive and unique (e.g. cyclonedx-sbom-spec, oauth2-rfc6749).

Check for an existing resource with graph_search_nodes(query="<resource-name>") and read_note(path="03_resources/<resource-name>/index.md"). The capture and write tools create parent directories; no separate folder-creation step is needed.

If the slug already exists, either you are re-ingesting (update in place, tell the user) or pick a more specific slug.

3. Copy the original into raw/

Always preserve the source bytes untouched. Pick the branch that matches step 1.

Local file — use capture_raw with the supplied source_path and a vault-relative path such as 03_resources/<resource-name>/raw/<filename>. This preserves the original bytes and filename.

PDF / binary at a URL — use capture_raw(url="<url>", path=".../raw/<filename>"). Check the returned byte count and SHA-256.

Webpage / link — save the URL using write_note(path=".../raw/source.url", content="<url>\n"). Capture the original page with capture_raw(url="<url>", path=".../raw/page.html"). If the invoking agent allows WebFetch, it may fetch a readable version of that external source; save the returned body with write_note(path=".../raw/page.md", content=...).

YouTube video — record the URL with write_note. If a transcript is supplied, capture its file with capture_raw(source_path=..., path=".../raw/transcript.txt") or preserve supplied text with write_note. These MCP tools do not extract YouTube transcripts. If none is supplied, report the missing transcript and stop before writing content notes. Never invent transcript text or run shell extractors.

4. Read the captured content

Use read_note(path="03_resources/<resource-name>/raw/<filename>") for captured text. Path reads access the file directly and work before graph regeneration. For a captured binary, use the invoking agent’s explicitly permitted inspection tool on the absolute path returned by capture_raw; if unavailable, report it. This — not the live URL — is the source of truth for the rest of the steps, so the note matches the bytes we preserved. For long content, read enough to identify every distinct topic, not just the intro.

5. Decompose into topics and find links

Reason about the individual topics inside the resource — the concrete concepts, standards, formats, tools, or entities it discusses. For each topic, decide whether the vault already knows about it, and whether it points outward.

For every candidate topic, use graph_search_nodes(query="<topic>") and tag filters from graph_summary, then traverse graph_neighbors and read_note promising matches. Confirm they cover the same concept before linking. The query searches metadata, not note bodies; read candidate notes when needed.

Classify each topic:

• Internal — the vault has a note for it → link with [<title>](<note-id>){"type":"…"}.
• External — no vault note, but it has an authoritative source (a spec, RFC, vendor page) → record the URL so it can become a note later.
• Unlinked — neither yet → list it as a candidate so a future note can fill it.

Example: a CycloneDX SBOM resource that mentions CPE and purls → search the vault for cpe, purl, sbom; link [text](id){"type":"…"} to whichever notes exist, and capture the CPE/purl spec URLs as external for the rest. A resource describing CycloneDX vendor references → reason about which existing supplier/vendor or component-identity notes it relates to and link those.

6. Write a note per topic

Use write_note to create each distinct topic from step 5 at 03_resources/<resource-name>/<topic-name>.md (kebab-case slug). This is what makes topics first-class, individually linkable graph nodes.

---
id: resource/<resource-name>/<topic-name>
title: <Topic Title>
type: note
status: active
tags: [resource, <topic-name>, <extra-tags...>]
source: <original url or path>
ingested: <today, YYYY-MM-DD>
last_reviewed: <today, YYYY-MM-DD>
---

# <Topic Title>

<What the resource says about this topic — a few sentences, in your words.>

## In this resource
From [<resource title>](resource/<resource-name>){"type":"part-of","link-info":"<how the topic appears / why it matters here>"}

## Related
- [<title>](<existing-vault-note-id>){"type":"related","link-info":"<what to expect there>"}   # internal
- <https://authoritative-spec-url>   # external — spec / RFC / vendor page

Links are markdown links whose target is the note id, with an optional {"type":"…","link-info":"…"} brace. type must be one of the controlled vocabulary (the pre-commit hook rejects others): related, supersedes, superseded-by, depends-on, part-of, parent, derived-from, see-also, source. Omit the brace for a plain related link. link-info is what the source needs from the destination — "what to expect there".

• One topic = one file. Skip trivia; write a note only for topics worth recalling.
• If the vault already has a note for the topic (an internal match from step 5), do NOT duplicate it — link it from the resource note instead and, if useful, add a backlink (step 7). Create a topic note here only for topics the vault does not yet cover.
• Link each topic note back to its resource with [<resource title>](resource/<resource-name>){"type":"part-of"}, and outward to internal notes (same link form) and external sources (URLs).
• Legacy [[wiki links]] are rejected by refresh_graph(validate=true) — never write them.

7. Write the resource note (index)

Use write_note for 03_resources/<resource-name>/index.md. This hub ties the raw source to the topic notes.

---
id: resource/<resource-name>
title: <Human Title>
type: note
status: active
tags: [resource, <topic-tags...>]
source: <original url or path>
ingested: <today, YYYY-MM-DD>
last_reviewed: <today, YYYY-MM-DD>
---

# <Human Title>

## Summary
<2–5 sentences: what this resource is and why it is worth keeping.>

## Source
- Type: <local file | webpage | youtube | pdf | ...>
- Origin: <url or original path>
- Raw: `03_resources/<resource-name>/raw/`

## Topics
- [<Topic>](resource/<resource-name>/<topic-name>){"type":"part-of","link-info":"<one line>"}   # new topic note
- [<Topic>](<existing-note-id>){"type":"related","link-info":"<one line>"}                       # vault already had it
- **<Topic>** — <one line> → <https://external-spec-url>                                          # external only
- **<Topic>** — <one line> (no note yet)                                                          # candidate

## Related
- [<title>](<related-note-id>){"type":"related","link-info":"<what to expect there>"}

• ingested and last_reviewed = today (the currentDate from session context).
• type must stay within the vault taxonomy enum (hub|project|area|guideline|policy|feature|roadmap|decision|note|meta|reference), which refresh_graph(validate=true) enforces — use note; the resource tag is what marks it.
• Markdown link targets resolve by id, rel-path, title, or unique stem — always use the id as the target, e.g. [Human title](resource/foo/bar).
• Do not leave a section empty; write None. if a section truly has nothing.

8. Wire backlinks

So the resource is reachable in the graph, not just reaching out:

• Add a [<title>](resource/<resource-name>) (or a relevant topic note) link from the most relevant existing notes (the strong internal topics from step 5) — e.g. under a ## Resources or ## Related section, creating it if absent. Add a {"type":"…","link-info":"…"} brace where the relationship is more specific than related. Use edit_note for each existing note.
• If several resources accumulate, maintain 03_resources/index.md as a hub that lists them, and link the new note from there (create the hub if it does not exist, with type: hub, tags: [resource, hub]).

9. Regenerate and validate the graph

Use refresh_graph(validate=true).

This regenerates index/graph.json and validates links and edge relationship types. If validation reports unresolved links, fix the targets (typos, wrong id, or a link to a note that does not exist yet — either create it or downgrade it to plain text / external URL); if it reports an unknown edge type, correct it to a value in the controlled vocabulary. Re-run until clean. The pre-commit hook rejects a stale or invalid graph, so this must pass.

10. Stage (do not commit unless asked)

Without explicit publishing authorization, leave the changes uncommitted and report the touched paths. The MCP interface has no stage-only tool.

When a commit is authorized, use git_commit(message=..., paths=[...]) with an explicit list covering the resource folder, backlinked notes and index/graph.json. This tool stages those paths and commits them; do not use wildcard paths. Use git_push only when pushing is explicitly authorized. Follow the caller's branch and review policy.

11. Report

Summarize: the resource id/path, type, what landed in raw/, the topic notes you created (<topic-name>.md), which topics were linked to existing vault notes (typed markdown links) vs. captured as external vs. left as candidates, files touched, and the graph-validation result.