VibeVM
Contents
On this page
en
Publisher
org.vibevm.core
Version
1.0.0latest
Audiences
user
Reading time
84 min
Rendered
Read aloud
never

PROP-005: Optional package index — per-org metadata + standalone index server

01Milestone: retrofits into M2.10 (vibe search) and M1.10 (vibe outdated) from ROADMAP.md. Slices land independently; index is opt-in everywhere.

02Status: implemented; folded into the workspace 2026-05-22. Slices 1–8 (the vibe-index server + CLI) and slices 9–10 (publisher hook + consumer fast path) are shipped, plus M2.10 vibe search. vibe-index lives at crates/vibe-index/ as a workspace member (§6) and parses through vibe-core::Manifest. See §9 item 11 for the de-rot and fold that got it there.

03Related: PROP-001 (git backend), PROP-002 ([[registry]] / [[mirror]] / [[override]] / content-hashed identity), PROP-003 (features / subskills / describes / conditional deps), PROP-004 §5.x (gap analysis), spec://org.vibevm.core/vibevm/common/PROP-000 (especially §15 dep weight, §16 JTD, §17 production architecture, §18 complexity ≥ RPM, §20 token secrecy).

04Out-of-band research summary (2026-05-06, prior session). Comparative inventory of indexing strategies in production package managers — Maven Central (Lucene + directory layout), npm (CouchDB replicated DB), PyPI (PEP 503/691 simple API), RPM/DNF (repodata/primary.xml.gz + libsolv), Deb/APT (Packages.gz RFC822), Cargo (git index → sparse HTTP), Go modules (proxy + Merkle sumdb), Nix flakes (per-flake flake.lock, no global index), Homebrew (mono-repo formula), OCI registries (/v2/_catalog). Three candidate paths surfaced for vibevm: (a) Cargo-sparse-style per-package JSON files in an org-level index; (b) DNF-style single repodata directory with full SAT-ready dep graph; (c) Nix-flake-style indexless live-resolve (current state) with optional flake-registry-shape short-name mapping. PROP-005 picks (a) augmented by (b)'s integrity-manifest pattern (repomd.json).

1. Motivation

05vibevm today resolves packages live against the host's git API:

  • 06vibe install flow:wal translates to git ls-remote <org-url>/flow-wal.git to enumerate tags, then git archive (or git fetch && git checkout) to read the manifest at the candidate ref. One pkgref = at least one round-trip per registry walked, two if the package is not in the first registry.
  • vibe outdated (M1.10) calls MultiRegistryResolver::resolve(<pkgref>@Latest) per locked package — one round-trip each.
  • vibe search (M2.10, not yet shipped) cannot work at all without enumerating an org. GitHub's GET /orgs/<org>/repos is rate-limited to 60 req/h unauth or 5000 req/h with a token; GitVerse exposes no org-scoped repo listing in its public API.

07This works at the M0 / M1 demonstration scale (3 packages in vibespecs). It does not work at v1 shipping scale (target: hundreds of packages per org, multiple orgs configured per project).

08Failure modes the live-resolve path produces in practice:

  1. 09Cold-cache install latency grows linearly with the dep graph. A project with 20 transitive deps spread across two registries spends 30–60 s in git ls-remote alone before any actual content fetch.
  2. Rate-limit visibility for vibe outdated. Polling N packages at refresh time burns N requests; against an unauthenticated GitHub registry, this exhausts the quota at 60 packages.
  3. vibe search is impossible. Even with an authenticated org-listing endpoint, parsing every repo's vibe.toml at every search would be intractable.
  4. Discovery story is silent. A consumer with a fresh checkout of an unknown vibevm org has no way to enumerate "what packages live here?" without scraping the host UI.
  5. Mirror-driven offline workflows degrade silently. PROP-002 §2.3 makes mirror dispatch invisible to the lockfile, but ls-remote against a mirror still leaks live host calls when the resolver wants to know "what versions exist?" — there's no offline catalog.

10What every other production package manager does — and what vibevm now needs — is an index: a small set of files, regenerable from authoritative package state, that lets consumers perform list, search, outdated, and resolve-version-shortlist operations against cached / mirror-able metadata instead of live git. RPM (primary.xml.gz), Cargo (sparse index), Deb (Packages.gz), npm (CouchDB document per package) all converge on the same shape: derived metadata files alongside or near the artefact storage, regenerated by a tool, served as plain HTTP.

2. Decisions

2.1 Index is OPTIONAL — zero impact when absent

11req r1

12Decision. The index layer is strictly additive. Every existing vibevm code path keeps working exactly as today when no index is present. No registry is required to have an index. No project is required to consume one; a consumer that finds none falls back to the live git ls-remote path that exists today.

13Discovery is a ladder, and the eternal handshake is its first rung. A consumer probes two candidate bases in order — <index-url>/v1/index, then <index-url> — and at each one asks hello.json before any repomd.json. A 200 whose body parses as a handshake and carries a world of the epoch this build reads settles the probe: that world's path refines the base every later file is fetched from. Only «no handshake here» (404, 5xx, connect failure) moves on to the next candidate and, when neither answers, to the repomd.json probe at the same two bases — the compatibility surface for indexes published before the handshake existed. Nothing at all → the live path, silently, exactly as before.

14Asked first, not asked beside — because successor is the in-band forwarding pointer for an index that MOVED, and it is readable exactly when the old address no longer serves a catalog. A handshake sought only next to a repomd.json that answered would be read in every case except the one it exists for. The price is up to two extra GETs, and it is paid only by indexes that have no handshake.

15A probe answers found, absent, or refused, and the third is what keeps this section honest. «Absent» is the only outcome that falls through quietly, because it is the one that means what the fall-through assumes: nothing is published here. An index that IS there and cannot serve this consumer — it refused us (401/403), its body does not parse as a handshake, its handshake format is one this build does not read, or it publishes no world of this build's epoch — answers refused, carrying the offered epochs, this build's epoch, a recipe, and whatever the document said in min_client / notice / successor. Collapsing that into «absent» would make a private, broken or newer-than-us index indistinguishable from a missing one, which is the silence PROP-044 §2 forbids: a break that announces itself is normal life, a riddle is what strands users.

16A successor is named to the operator, never followed by the client. Automatic following needs a cycle watchman and a trust rule for the address it lands on, and neither is decided; naming the address costs the human one command and invents no policy.

17Rationale.

  • 18Backward compat for existing vibespecs (GitHub) + vibespecs-gitverse (GitVerse) registries — they keep working unchanged. Index is something the org owner opts into.
  • Decouples the index design from Phase A of vibevm — operators with three packages do not need an index; operators with three hundred do.
  • Removes the "central server is now load-bearing" failure mode: if the index disappears, the live path is still there.
  • 19Consequence. All optimisations the index unlocks (cold-cache install speed, vibe search, faster vibe outdated) are opportunistic.
  • The integrity story (content_hash per PROP-002 §2.1) does not change — content_hash is verified at fetch time regardless of whether the resolve path went through the index.

2.2 Form factor: per-org index living in a separate git repository

20req r2

21Decision. Each vibevm registry org that opts in maintains a dedicated git repository named index (configurable; default name index) under the same org root:

  • 22https://github.com/vibespecs/index
  • git@gitverse.ru:vibespecs/index.git

23Inside this repository sits a fixed file layout (§2.4) holding the org's catalog. The repository is:

  • 24Cloneable like any other — same auth model as the package repos (HTTPS public read; SSH or token push for the maintainer).
  • HTTP-fetchable at raw URLs without cloning — https://raw.githubusercontent.com/vibespecs/index/main/repomd.json (GitHub) / https://gitverse.ru/api/v1/repos/vibespecs/index/raw/main/repomd.json (GitVerse). Consumers default to raw HTTP (one GET, no clone); falling back to git clone only when the host's raw-HTTP shape is unknown.
  • Mirror-able trivially — the [[mirror]] machinery from PROP-002 §2.3 applies unchanged: a mirror at https://mirror.internal/vibespecs/index is a drop-in.

25Why a dedicated repo, not files-in-package-repos.

  • 26Discovery. A single <org>/index repo answers "what's in this org?" with one HTTP GET. Per-package metadata files would still require enumerating the org first — chicken-and-egg.
  • Atomicity of catalog state. A single index repo can be replaced as a whole, signed as a whole, mirrored as a whole. Per-package files leave catalog consistency to the consumer to reconstruct.
  • Decoupling. Index regeneration does not touch package repos. Authors do not need to run any utility. The org owner runs vibe-index on a cadence; package repos stay pristine.
  • Mirror parity with PROP-002 §2.3. The same mirror chain that covers package repos covers the index. Operators who already understand [[mirror]] get the index covered for free.

27Why not a hosted central HTTP service (npm-style registry.npmjs.org):

  • 28Requires running infra. vibevm's deliberate posture per PROP-000 §17 is "every org self-hosts on hosting they already use" — git platforms are that hosting.
  • Single-vendor. We rejected this shape in PROP-002 §1 (the "Nix's failure pattern"). Centralising the index is the same anti-pattern at one layer up.
  • HTTP service is available (§2.5) — the vibe-index serve mode lets an operator run one — but it is not the default consumption path. Most consumers go through static raw-HTTP files in the index git repo.

29Configurable but defaulted. A [[registry]] block pins a custom index location — the key exists in RegistrySection (one type serving both the project vibe.toml and the machine-global ~/.vibe/registry.toml, so the columns share one vocabulary), and this exact block parses (pinned by test prop005_index_url_example_parses, which carries it verbatim):

30[[registry]]
name = "vibespecs"
url = "https://github.com/vibespecs"
naming = "fqdn"
index_url = "https://raw.githubusercontent.com/vibespecs/index/main"  # explicit override
# or, to point at a hosted server:
# index_url = "https://index.vibespecs.dev"
# or, to disable index lookup entirely:
# index_url = "none"

31The bottom rung is host-aware. Canonical public https://github.com/<org> maps to https://raw.githubusercontent.com/<org>/index/<registry-ref>; other hosts retain <registry-url>/index. This makes both fresh and already- seeded GitHub configurations whose index_url field is absent consume the static repository rather than its HTML page. The full ladder remains env override → manifest key → host-aware default; exact none on either explicit rung disables lookup. Lookalike hosts, nested paths, userinfo, unsafe owners/refs, queries and fragments are never rewritten.

32For three months this block was a parse refusal, and saying so was the point — until 2026-08-20, when the key landed (B-083, owner-ruled «построить ключ»). RegistrySection carried deny_unknown_fields without index_url, so a reader copying the example got an error; the fact above honestly said plan while the tree disagreed with the document. The strictness itself was never the defect and stays: the section still refuses unknown fields, now with index_url among the known ones — pinned red-then-green by the landing (the pre-landing parse error is quoted in the worker report; registry_section_still_refuses_unknown_fields_alongside_index_url keeps the refusal honest).

33The environment variable VIBEVM_INDEX_URL_<REGISTRY> is the ladder's top rung — the operator's per-run re-point, no longer the only source. Until 2026-08-20 it was the sole locator, deliberately weaker than the manifest field it stood in for (per-shell, per-run, travelling with neither project nor lockfile) — which is why it never closed the requirement above. Now it overrides the key: env beats index_url beats the default, and none at either explicit rung disables the index. The name normalization (ASCII alphanumerics upper-cased, the rest to _``) is unchanged.

34404 / connect-failure on the index → silent fallback to live ls-remote: no error message, because the operator never promised an index. This half is built, and it is the absent outcome of ##A-PROBE-HAS-THREE-OUTCOMES-NOT-TWO — the other two outcomes are never silent.

2.3 Source of truth: package repos remain authoritative; index is a hot cache

35req r1

36Decision. Package repositories are the source of truth for content (manifests, files, tags). The index is a derived hot cache, regeneratable from the authoritative package state.

  • 37This matters because it disambiguates the failure mode: if the index disagrees with reality, reality wins.
  • A consumer that resolves a package through the index still verifies content_hash against the actually-fetched bytes per PROP-002 §2.1. A mismatch between an index-recorded content_hash and the actually-fetched one is a hard IntegrityError, surfaced with both values and a hint to refresh the index. No silent acceptance.

38Server-mode wrinkle. The vibe-index serve mode (§2.5) is described in the prompt as "the only writer; source of truth". This is true for the index data. The server holds the canonical RAM copy, persists it to disk on every mutation, and refuses writes from other processes (file lock + mtime checks at startup). But the server's writes are derived from package-repo state — the ground truth for "is this version published?" is still the actual git tag in the actual package repo. A divergence (index says v0.1.0 exists; the package repo's tag was force-deleted) surfaces at consumer install time as an integrity failure on cross-source content_hash verification, and the operator has the diagnostic clear: "your index lies; reindex".

2.4 File layout inside the index repo

39req r2

40Decision. The index repo's working tree (and equivalently its raw-HTTP-served file tree) carries:

41<index-root>/
├── hello.json                                 # the eternal handshake — read FIRST, dispatches to a world
├── repomd.json                                # manifest — hashes & metadata for all other files
├── primary.jsonl                              # one line per (group, name, version)
├── primary.jsonl.gz                           # gzipped variant
├── by-name/
│   └── <name>.json                            # candidate set — every (group, name) sharing one bare name
├── by-cap/
│   └── <capability-slug>.jsonl                # provides-index — pkgrefs that advertise this capability
├── by-purl/
│   └── <purl-slug>.jsonl                      # describes-index — pkgrefs that describe this PURL
└── README.md                                  # human-readable "what is this directory" pointer

42hello.json — the eternal handshake (PROP-044 §3): {vibe, worlds[], min_client?, notice?, successor?}, the one document whose keys never change meaning, through which a client of any age learns which worlds this index currently serves, where each lives, and where the handshake itself moved. Its shape is schemas/hello/e1/hello.jtd.json and its type is generated like every other wire type (§2.12); the index writes it, and §2.1 is where a consumer reads it.

43The handshake is deliberately absent from repomd.json::files, and the asymmetry is the design. repomd.json is the manifest of one world; the handshake stands above worlds and dispatches to them, so a world's manifest can no more vouch for it than a chapter can vouch for the table of contents. The consequence to hold: the handshake is the one served file the manifest's sha256 map does not cover, and what verifies it instead is that it parses — an HTTP 200 whose body is not a handshake is a loud refusal naming the broken index, never a quiet fall-through to repomd.json (§2.1).

44What the writer owns is a stated whitelist, not whatever the directory happens to hold — four root files (hello.json, repomd.json, primary.jsonl, primary.jsonl.gz) and three trees (by-name/, by-cap/, by-purl/), named once in the code so the two readers that ask "is this catalog still the projection of its journal?" — vibe-index rebuild <data-dir> --check (with cargo xtask rebuild --check <data-dir> as a thin compatibility wrapper) and the golden-corpus test — cannot compare different sets. The rest of the directory is not the writer's and is not compared: README.md and .gitignore are written once by init, and the whole of state/ (§2.13) is the server's own bookkeeping. A blacklist would have to enumerate the world and would rot the day the directory grew a file nobody listed.

45repomd.json — the manifest, modelled after RPM's repomd.xml:

46{
  "schema_version": 1,
  "registry": "vibespecs",
  "registry_url": "https://github.com/vibespecs",
  "naming": "fqdn",
  "generated_at": "2026-05-06T12:00:00Z",
  "generator": "vibe-index 0.1.0",
  "package_count": 42,
  "version_count": 117,
  "files": {
    "primary.jsonl": {
      "kind": "file",
      "size": 184522,
      "sha256": "<hex>"
    },
    "primary.jsonl.gz": {
      "kind": "file",
      "size": 38421,
      "sha256": "<hex>"
    },
    "by-name": {
      "kind": "directory",
      "entries": 42
    },
    "by-cap": {
      "kind": "directory",
      "entries": 18
    },
    "by-purl": {
      "kind": "directory",
      "entries": 5
    }
  }
}
  • 47Every entry of files carries a kind tag — "file" or "directory" — and a reader dispatches on it rather than on which shape happens to fit. The tag was half-present until 2026-08-14: directories carried it, files did not, and the union was matched by shape, so an entry that lost a field was silently re-read as the other kind instead of being refused. A wrong answer that looks like a right one is the one failure re-fetching cannot cure (PROP-044 §2), which is why the asymmetry was broken deliberately rather than tolerated; the break note is formats/breaks/001.md. A file entry missing its tag is now a parse refusal.
  • repomd.json is the format's point of trust: it records a sha256 for every file beneath it, which is what makes «verify the catalog» a single well-posed question. Today the party that asks it is the operatorvibe-index verify re-hashes a data directory against the manifest — and that is the whole shipped claim (formulation narrowed to what exists, owner ruling 2026-08-20, BACKLOG.md B-084). The consumer-side half — fetch it once with an ETag round-trip, then verify each fetched sub-file against the recorded hash — is deliberately deferred to the post-1.0 campaign (campaigns/packages-2026-09/deferrals.md#release-1-0), together with the reader-predicate question it shares with B-080.
  • No shipped consumer does that yet, and saying so is the point of writing it down. The index client asks repomd.json as an existence probe and then fetches the candidate-set file directly: it reads no sha256 and sends no ETag — measured as zero occurrences of either in the client, against 43 elsewhere in the same crate, so the zero is the client's silence and not the instrument's. The comparison the fact above describes exists only as the operator verb vibe-index verify, run against a data directory, which is a different party at a different time.
  • What that leaves exposed is metadata in transit, not content. content_hash is verified against the actually-fetched bytes at fetch time no matter how the version was chosen (§2.3), so a tampered index can misdirect a consumer toward the wrong version — it cannot make it install bytes nobody checked. A substituted by-name file, by contrast, is read as it arrives. Both halves belong in one sentence: an integrity story stated only in its strong half is the kind of claim a reader plans around and a defect hides behind.
  • This pattern (manifest-with-checksums) is what RPM, Deb, and OCI all share, and is what gives us a path to GPG signing without re-architecting.

48primary.jsonl — newline-delimited JSON, one record per (group, name, version). Lines are sorted by (group, name, version) — the PROP-008 §2.2 identity ordering. Each line is one §2.6 entry. JSONL is chosen over JSON-array because:

  • 49Append-friendly (a publish hook can append and re-sort + rewrite, or for incremental update merge a sorted insert).
  • Streamable by consumers (line-at-a-time parse; no need to buffer the whole file).
  • grep-able (operators can inspect the index manually).
  • Diffable in git (per-line diffs survive re-sorts cleanly).

50primary.jsonl.gz — gzip-compressed equivalent for HTTP-bandwidth-conscious consumers. ~5× compression typical for JSON. Byte-identical content; gzip is deterministic at level 6 with the standard zlib dictionary so the file is reproducible across machines (we pin level and disable mtime in the gzip header to keep the SHA-256 stable). Both primary.jsonl and primary.jsonl.gz ship; consumer chooses based on Accept-Encoding.

51by-name/<name>.json — the candidate-set file for one bare package name (PROP-008 §2.8). A single HTTP GET fetches every (group, name) package that shares the short name <name>, each with all its versions — ~1–10 KB. This is the path short-name resolution (PROP-008 §2.6) walks: one GET per registry yields the whole candidate set, so a collision (PROP-008 §2.7) is detected at once. The directory level keyed on kind before PROP-008; kind left package identity, so <name> alone is the key.

52{
  "name": "wal",
  "indexed_at": "2026-05-06T12:00:00Z",
  "packages": [
    {
      "group": "org.vibevm",
      "name": "wal",
      "indexed_at": "2026-05-06T12:00:00Z",
      "latest_stable": "0.1.0",
      "versions": [
        { /* one §2.6 entry */ },
        { /* … */ }
      ]
    }
  ]
}

53tombstone{reason, superseded_by?}, carried by the candidate-set file only when the bare name is buried, and omitted from the wire otherwise (which is why the live example above does not show it). A name that ever existed answers with the current thing, a forwarding pointer, or a tombstone carrying a reason — never with silence (PROP-044 §2). The consequence that is easy to miss: the file is written even when the name has no packages left, because an absent file is the silence the law forbids. A buried name therefore looks like this:

54{
  "name": "wal-old",
  "indexed_at": "2026-05-06T12:00:00Z",
  "tombstone": { "reason": "renamed to `wal`", "superseded_by": "org.vibevm/wal" },
  "packages": []
}

55by-cap/<capability-slug>.jsonl — for vibe install --capability ui:landing-page style queries (PROP-003 capability-driven resolution). Each line: {"kind":"feat","name":"welcome-page","version":"0.3.0","capability":"ui:landing-page@0.3.0"}. <capability-slug> is the capability string with : and / and @ replaced by -- (filesystem-safe; reversible). Optional in v0; populated when present.

56by-purl/<purl-slug>.jsonl — for "what vibevm packages document this upstream library?" queries (PROP-003 §2.5.6 describes). Same shape as by-cap; <purl-slug> is the PURL with / replaced by --.

57Sort invariants. Every file with multiple entries is sorted deterministically:

  • 58primary.jsonl — sort key (group, name, version) with versions in ascending semver order.
  • by-name/<name>.jsonpackages sorted by group; each package's versions array sorted ascending.
  • by-cap/<slug>.jsonl and by-purl/<slug>.jsonl — sort key (group, name, version).

59Determinism matters because the index repo lives in git: a non-deterministic order would produce churn diffs on every regenerate, defeating the value of git as the transport.

2.5 Two modes: CLI tool and HTTP server

60req r1

61Decision. A single binary, vibe-index, ships in two modes selected by subcommand:

  • 62CLI mode (default, every subcommand except serve) — operates directly on a data directory of index files. Reads on-disk state, mutates, writes back atomically. Suited for: scripted vibe-index reindex invocations, manual operator commands, CI pipelines, post-publish hooks.
  • Server mode (vibe-index serve) — boots an HTTP server. Holds the index in RAM; persists every mutation back to disk. Single-writer (the server) — no other process should mutate the data dir while the server runs (file lock at <data-dir>/state/server.lock; broken lock → server refuses to start with a clear error). Suited for: hosted index endpoints, real-time publish-time updates from vibe registry publish.

63Why one binary, not two. Same code paths (the in-memory Index struct, the persistence layer, the scanner) are shared. Two binaries would force consumers to install both. clap-style subcommand dispatch handles the mode selection.

64Distribution. The utility lives at crates/vibe-index/ as a member of the top-level vibevm workspace — built, tested and gated by the same cargo … --workspace invocations as every other crate. §6 records why the original standalone-workspace decision was reversed and what the reversal bought.

2.6 Index entry shape (the canonical record)

65req r1

66Decision. Every (group, name, version) entry carries the following fields. This is the shape lines of primary.jsonl follow, and the elements each by-name/<name>.json candidate's versions[] carry. The authority for that shape is schemas/index/e1/entry.jtd.json; this section is its reading aid. The JTD file is the source of truth, the Rust is generated from it into crates/vibe-wire/src/generated/ and re-exported through crates/vibe-index/src/types/entry/, and cargo xtask check-codegen refuses any drift between the two. The record is defined once, in the shared version_entry vocabulary, because the candidate-set file and the journal carry it transitively and the schema language has no cross-file reference — entry.jtd.json is the root that names it. So the index entry stands in exactly the same arrangement as the wire reports under the root schemas/, not a different one. Where this section and the schema disagree, the schema wins and this section is the defect. What this section carries that the schema cannot is the provenance below — where each field comes from — and that is a copy of nothing.

67{
  "schema_version": 1,
  "kind": "flow",
  "group": "org.vibevm",
  "name": "wal",
  "version": "0.1.0",
  "content_hash": "sha256:8136ecdbc25d4555cbab6e9574f153b252a05c62b55b5e0255def645458c9544",
  "source_url": "git@gitverse.ru:vibespecs/flow-wal.git",
  "source_ref": "v0.1.0",
  "resolved_commit": "1c3a1355abcdef0123456789abcdef0123456789",
  "registry": "vibespecs",
  "workspace_origin": null,
  "license": "EULA",
  "authors": ["Oleg Chirukhin"],
  "description": "Write-Ahead Log discipline for human-AI development sessions",
  "homepage": null,
  "keywords": ["wal", "memory", "discipline", "session-management"],
  "describes": null,
  "compatibility": {
    "min_vibe_version": "0.1.0",
    "requires_kinds": []
  },
  "provides": {
    "capabilities": []
  },
  "requires": {
    "packages": [],
    "capabilities": []
  },
  "requires_any": [],
  "obsoletes": { "packages": [] },
  "conflicts": { "packages": [] },
  "features": {
    "default": [],
    "exclusive": {}
  },
  "subskills": [],
  "i18n": {
    "available": ["en"],
    "default": "en"
  },
  "boot_snippet": {
    "source": "boot/10-flow-wal.md",
    "category": "flow"
  },
  "files_count": 5,
  "must_understand": [],
  "yanked": false,
  "frozen": false,
  "indexed_at": "2026-05-06T12:00:00Z",
  "indexed_by": "vibe-index 0.1.0"
}

68The three slots — must_understand, yanked, frozen — appear above in their significant form, but all three are omitted from the wire when empty (an empty list; false). A live record for an un-yanked snapshot package therefore carries none of them, and the same holds for tombstone on the candidate-set file (§2.4). Reading this example as "these keys are always present" is the one mistake it invites.

69Field provenance.

  • 70kind / group / name / version / license / authors / description / homepage / keywords / describes / compatibility / provides / requires / requires_any / obsoletes / conflicts / features / i18n / boot_snippet.source / boot_snippet.category — read directly from vibe.toml at the tagged ref. (M1.18's loading model, PROP-009, retired the author-chosen boot_snippet.filename; a snippet is now its source path plus an ordering category.)
  • group — the mandatory reverse-FQDN qualifier from [package].group (PROP-008 §2.1). With name it forms the package identity; kind is metadata and identifies nothing (PROP-008 §2.2 / §2.3).
  • workspace_origin — the [origin] provenance marker (PROP-007 §2.8, PROP-008 §2.8), present only on a copy vibe workspace publish generated from a workspace member; absent (null) for a standalone publish.
  • subskills — collected by walking <package-root>/subskills/<path>/vibe-subskill.toml at the tagged ref; each entry: {path, delivery, describes, description, channels}. Same fields the lockfile records.
  • content_hash — computed by the same algorithm vibe-registry::compute_content_hash uses (sha256 over deterministically-ordered file bytes). Index uses the same hash as the lockfile, so cross-checks are byte-equal.
  • source_url — the canonical org URL (§2.4 of PROP-002) composed with the package repo name. Mirror URLs do not appear here (same invariant as the lockfile).
  • source_refv<version> by default (the tag).
  • resolved_commit — the commit SHA the tag pointed to at index time. Pinning to commit lets us notice tag-rewrites later.
  • registry — local alias from [[registry]].name.
  • files_count — informational, useful for sanity-checking integrity diffs.
  • indexed_at / indexed_by — provenance for the index entry itself (when, by which tool version).
  • must_understand — the reader capabilities a consumer must have to act on this record (PROP-044 §4.5) — a different vocabulary from the package's own provides.capabilities. Written by the projector; never read from vibe.toml. A reader that does not understand every string in the list skips this record and says so — what «says so» is, exactly, is §2.19; unknown fields outside the list are ignored as before. This is the exact inversion of additive-only: the writer declares what is mandatory, per record, addressably and revocably, instead of the schema promising ignorability forever.
  • yanked — the version is withdrawn. Journal-borne, not authored: frozen content cannot withdraw itself, so the fact arrives from the registry's facts journal and is projected here (PROP-044 §2a).
  • frozen — projected from the package manifest's [package].frozen, never a registry's opinion. Absence = false = snapshot: content may flow under the same version string, and a hash mismatch is news. true is the author's one-way freeze: bytes immutable, and a hash mismatch is an alarm. The flag lives inside the hashed content, so a version self-describes even offline and every registry serving those bytes necessarily agrees — a registry only observes the freeze in its journal and projects it (PROP-044 §2a; terminology §2b — snapshot and frozen are the two states of one boolean axis, with no third).

71Forward compatibility. schema_version: 1 is recorded at file scope (in repomd.json) and at entry scope. Entries carrying fields a reader does not know coexist with it: unknown fields are tolerated, and known-but-absent ones default. This sentence was false for as long as it stood. Fifteen catalog aggregates carried deny_unknown_fields, so a reader meeting one unknown key refused the whole file — the opposite of what the fact promised — and the contradiction survived because nobody could remove the attribute safely. That was not caution: while every mutation read the catalog to rewrite it, tolerance would have SILENTLY DELETED the tolerated fields on the next write, which is worse than refusing. Phase 3 removed the condition rather than the symptom — a mutation is now an append to the journal plus a reprojection, so nothing read is ever written back, there is nothing to lose, and the strictness could finally go (PROP-044 §4.4). What tolerance still does NOT mean is acting on a record one does not understand; that refusal moved to the per-record capability set (##NEVER-SILENT-SCHEMA).

2.7 Identity and trust

72req r1

73Decision. The digest of content_hash is the join key between the index and the lockfile, and it joins only at a stated recipe (PROP-002 §2.1). A consumer that fetches flow:wal@0.1.0 via the index records the digest a no-index fetch would have produced — for every tree except the ones recipe 1 exists to disambiguate, where the two recipes deliberately disagree and that disagreement is the point. Index entries are advisory; the bytes are authoritative.

74The index and the lockfile currently stand at different recipes: the index emits sha256-tree/1:, the lockfile keeps the bare sha256: until its own format moves. So their strings are deliberately unequal, and nothing compares them as strings — the index client does not read the field at all. Written down because the asymmetry looks like a defect to a cold reader and is instead the thing that keeps a live lockfile from being rewritten by a change it did not ask for.

75The repomd.json::files[*].sha256 covers integrity of the index files themselves. The per-entry content_hash covers integrity of the package content, and carries its recipe so a check compares like with like. The two are independent: a tampered index file fails its file-hash check; a tampered package repo (force-pushed tag) fails its content-hash check at fetch time.

76Out of scope for v0: GPG-signed repomd.json.asc, Merkle-log audit trail (Go sumdb-style). §9 tracks both.

2.8 Reindexation: full and incremental

77req r1

78Decision. Two regeneration modes. Both are available via the CLI (vibe-index reindex); the HTTP trigger was withdrawn by owner ruling 2026-08-20 (##TRIGGER-HTTP):

79Full reindex. Walk every package repo in the org; for each repo, list tags; for each v<semver> tag, read vibe.toml and subskills/**/vibe-subskill.toml at that ref; compute content_hash; assemble §2.6 entry. Replace the in-memory index wholesale, then atomic-write the on-disk files.

80Sources for the walk:

  • 81--from-clones <org-dir> — local directory of bare/regular clones. Authoritative for the operator who already maintains a vendor mirror; offline-capable. Default path for owners who run vibe-index reindex on a cron against their own server's clone tree.
  • --from-github <org> — REST API walk against api.github.com. Requires a token (read-only repo-scope). Used by hosted index instances that don't keep a clone tree.
  • --from-gitverse <org> — equivalent against GitVerse's API once it exposes org-scoped repo enumeration; today returns "not implemented" (mirrors the publish-stub pattern from vibe-publish/src/gitverse.rs).

82Incremental reindex. Detect what changed since the last run and update only the affected entries.

  • 83For --from-clones: compare each repo's git rev-parse HEAD and git tag -l output to a checkpoint stored at <data-dir>/state/checkpoint.json. Repos with new tags or a new HEAD commit on main (in case a manifest changed without a tag) are re-walked; others skip.
  • For --from-github: use the If-Modified-Since / ETag headers on /orgs/<org>/repos and /repos/<org>/<name>/tags to skip unchanged repos.

84Incremental is the default cadence target (one run per minute on an active org); full is the bootstrap path and the "trust nothing" recovery option.

2.8.1 The organisation image, and what keeps caching it honest

85Enumerating the organisation on every operation is a cost, and the cost is the small half of the problem. For local clones it is a directory walk and cheap. For a git host it is a paged API walk on every single operation. But the reason to think about it is not speed — it is that the picture is stale the moment it is taken.

86The premise «between operations nobody can change the organisation» is already false, and not because of sibling workers (owner accepted this correction, 2026-08-06). A developer publishing a package creates a repository and pushes a tag straight to the git host, never passing through the index service. The image goes stale with one worker exactly as with ten. The real axis is whether every change goes through the index — and today none has to.

87--cache-org is on by default (owner ruling, 2026-08-06), with an explicit negative form to turn it off. The name describes the mechanism rather than the assumption, which is deliberate: the assumption the default must NOT make is the one the fact above rejects.

88The cheap freshness check is what makes that default honest, and it is therefore not an enhancement. Git hosts answer «has anything changed» with a conditional request that costs almost nothing and needs no walk; the cached image carries the validator its enumeration came with, and every run offers it back. Without this step, «on by default» would silently mean the very assumption the owner rejected. With it, the image is cached and never treated as truth without asking.

89A host that gives no validator makes the index enumerate, never trust. The absence of an answer is not an answer. This is the direction the whole design has to fail in, because the opposite default — no validator, assume fresh — is indistinguishable from a working cache right up until a package cannot be found.

90An image is keyed to the organisation and the API base it came from, and a cache taken for one is never used for another. Cheap to state, expensive to omit: the failure would be an index confidently serving another organisation's picture.

91Hit and miss are reported, in both renderings. An operator must be able to tell «this came from the cache» from «this was enumerated», because a cache that silently serves stale data is indistinguishable from one that works — which is the disease this whole section is written against.

92rescan-org is its own verb and it is unconditional (owner ruling, 2026-08-06). It enumerates regardless of the cache and regardless of any validator, and refreshes the image. It exists because a missed change is invisible from the inside: no freshness mechanism promises completeness, and a full walk does. Webhooks (§2.16) reduce how often it is needed; they never remove the need.

93Decision (owner, 2026-08-20), closing a fork that had been open since the journal phase: a full walk says nothing about packages that were in the previous catalog and are not in this one. No comparison, no warning, no tombstone.

  • 94What is being declined is a real, cheap capability, not an impossible one. The walk already holds the previous set and the new set in memory at the same moment and simply never compares them — computing «forty before, thirty-nine now, this one is gone» costs almost nothing. The owner's answer is that the report is not wanted, which settles the question the machinery could not: a disappearance has too many innocent causes — a repository made private, renamed, moved between organisations, or an enumeration that was simply narrower — for the index to have an opinion about it.
  • Why this does not contradict the no-silence law. The law that a name which ever existed must not answer with silence governs withdrawal — an act an operator performed and a record they chose to leave. A package absent from a walk performed nothing and chose nothing; the walk is a photograph, not a claim about intent. Burial is how a name is closed on purpose (§2.11), and it stays the only way one is.

95The cache and its freshness check govern the host-API path. For a local-clone walk the enumeration is a directory read, and wrapping a validator around it would buy nothing and add a way to be wrong.

96With no image on disk the behaviour is exactly what it was — enumerate, build, write the image — with no warning and no error, and turning the flag off leaves no image and no report field. A default that changes the first run's behaviour is a default that has to be explained; this one does not.

97Triggers.

  • 98CLI: vibe-index reindex <data-dir> --from-clones <org-dir> — direct invocation.
  • HTTP: withdrawn. The route this fact used to specify — POST /v1/admin/reindex with a pollable job id — was retired by owner ruling 2026-08-20 (BACKLOG.md B-085) without ever being built: reindexation is an operator verb, not a network operation. The reasoning is the one §2.16 already applied to webhooks: a network trigger for a heavy rebuild is a DoS lever, and handing it to the admin token deserves a deliberate decision, not an inherited TODO. The operator's trigger is the CLI verb — reached by cron, a host-native scheduler, or the host's own hook mechanism invoking the CLI (§11). A route may return post-1.0 at the first real operator's need (campaigns/packages-2026-09/deferrals.md#release-1-0).
  • git hook (server-side, on the index repo's host): owner installs a post-receive hook on the org's hosted git that posts to POST /v1/admin/reindex whenever a package repo gets a push to a v* tag. Documented in §11; not shipped as part of the binary.
  • cron: crontab line invokes vibe-index reindex --incremental every N minutes. Documented; not enforced.

2.9 Single-writer server mode

99req r1

100Decision. The HTTP server is the only writer when running. It locks the data directory via a PID file (<data-dir>/state/server.lock) at startup; refuses to start if the lock is held by another live process; refuses CLI mutations against the same data directory by detecting the lock from CLI side (CLI-mode add / remove / reindex errors with "server is running on this data dir; use the HTTP API").

101In-memory state model:

102Arc<RwLock<Index>>
   │
   ├─ readers (search, list, get)        — RwLock::read()
   └─ writers (add, remove, reindex)     — RwLock::write()

103On every successful write, the server:

  1. 104Updates the in-memory Index.
  2. Re-serialises the affected files (primary.jsonl, the touched by-name/<name>.json, optionally by-cap / by-purl).
  3. Writes each file atomically: tmp next to the destination, fsync, rename.
  4. Updates repomd.json last (the manifest is replaced as a whole; readers that hold the old repomd.json see a consistent old view; readers that pick up the new repomd.json see consistent new files).
  5. Optionally (if --auto-commit-push flag is on): git add -A && git commit -m "auto: index update" && git push origin <branch> against the data directory if it is a git working tree. v0 ships without this — operator runs commit/push manually or via separate cron. v1 adds --auto-commit-push.

105The writer never calls now(). Every timestamp a write stamps — the manifest's generated_at and each candidate-set file's indexed_at — arrives as an argument: the CLI passes the moment its command began, the server passes the moment of the mutation event, and the index and entry modules contain no clock call at all (a panel step refuses one). One state therefore produces one byte sequence, which is what makes "rebuild and compare" a real verification, an empty diff a real no-op, and a wire-diff a quantitative measure of a break rather than a wall of timestamp churn (PROP-044 §4.3). Determinism here is an instrument, not tidiness: without it every later recoverability check measures the clock instead of the content.

106The writer stamps its own schema version only into an artifact it creates from scratch. A catalog it read keeps the version that catalog carried: the value is state, not a constant of whichever binary happens to be running. Otherwise a catalog written by a later version and opened by an older binary for any mutation would silently shed its own marker and start claiming ours — a file that still looks consistent while asserting something untrue about itself, which is the failure re-fetching cannot cure (PROP-044 §2, law 1). What the writer does when the version it read is one it cannot serve is a separate question and not answered here; this fact only forbids the silent overwrite.

107The clause the journal adds: a projection has no version to keep. Once a mutation is an append to the journal followed by a reprojection, the writer no longer reads a catalog at all — it builds one from the facts. There is therefore nothing to preserve, and stamping this build's constant asserts something true about the artifact just written, rather than overwriting a claim some other writer made. The rule above is unchanged and still binds every path that does read a catalog before rewriting it; what changed is how many such paths exist. The protection it was reaching for did not disappear with them — it moved up a floor, to the journal's own epoch and to each record's must_understand set, where a build meeting facts from a newer world refuses them by name instead of quietly reading a subset and re-labelling the result (PROP-044 §4.5). Read the two together and the invariant is one thing said twice: a file never claims an authorship it does not have.

108Concurrency. axum + tokio. Reads do not block reads. Writes block reads (RwLock) for the duration of the in-memory mutation; disk I/O happens after lock release for any path it can (e.g. primary.jsonl rewrites are queued and serialised by a single dedicated writer task). For the request rates we target (max ~10 writes/min during a publish burst, ~1000 reads/min during a CI install storm), a coarse RwLock is sufficient.

109Process model. Single process. No replication. An operator who needs HA runs the server behind a load balancer with N replicas and a shared filesystem — but that's outside v0. v0 expects one process per data directory.

2.10 HTTP API surface

110req r1

111Decision. REST API, JSON over HTTP. CORS open on read endpoints (so a future web UI can hit it from a browser). Routes:

112GET    /healthz                                   # liveness
GET    /readyz                                    # readiness (index loaded, no in-flight reindex)

# Static index files (raw — same shape as the on-disk files; mirror-friendly).
# The handshake leads the block because it leads the client (§2.1).
GET    /v1/index/hello.json
GET    /v1/index/repomd.json
GET    /v1/index/primary.jsonl
GET    /v1/index/primary.jsonl.gz
GET    /v1/index/by-name/{name}.json
GET    /v1/index/by-cap/{slug}.jsonl
GET    /v1/index/by-purl/{slug}.jsonl

# Structured query (richer than the raw files)
GET    /v1/packages                               # ?kind=&q=&limit=&offset=
GET    /v1/packages/{group}/{name}                # all versions of one package
GET    /v1/packages/{group}/{name}/{version}      # one specific version (entry)
GET    /v1/capabilities/{cap}                     # who provides this capability
GET    /v1/purls/{purl}                           # who describes this upstream

# Mutations (auth required)
POST   /v1/packages                               # body: full §2.6 entry — insert/upsert
DELETE /v1/packages/{group}/{name}/{version}      # remove one version
DELETE /v1/packages/{group}/{name}                # remove all versions of a package

# Admin (auth required)
POST   /v1/admin/reindex                          # body: { mode, source, args } — SPECIFIED, NOT BUILT
GET    /v1/admin/status                           # uptime, last reindex, pkg count, server version

# Observability
GET    /metrics                                   # Prometheus text format

113The admin surface the server builds is one route — GET /v1/admin/status — and since 2026-08-20 that is the specified surface, not a gap. The reindex trigger spent months as «specified and unbuilt»: the router registers 16 paths and none of them was it, the handler module holds status alone, and the code's own note («reindex POST lands in slice 6») described a slice that closed without it. The fork BACKLOG.md B-085 recorded — build it, or retire it in favour of the CLI verb — was resolved by owner ruling 2026-08-20: retired. ##TRIGGER-HTTP carries the reasoning, §11's hook recipe now invokes the CLI verb, and the route returns, if ever, at the first real operator's need.

114Authentication. Bearer tokens via Authorization: Bearer <token>. Tokens are read from <data-dir>/state/admin.tokens (one token per line; comment lines start with #). Read endpoints accept missing/invalid tokens silently. Write endpoints require a valid token; mismatch → 401 with a generic message ("authentication required"; do not echo the supplied token nor say which valid prefix it matched). Tokens never appear in logs (logging redacts the Authorization header).

115Per-host lockdown. By default the server binds to 127.0.0.1:8412 — local-only. Operators expose externally by setting --bind 0.0.0.0:8412 and putting it behind a reverse proxy with TLS. v0 does not ship TLS termination; this is the reverse proxy's job. (Same posture as cargo's sparse index protocol: the upstream is HTTP — TLS is for the CDN / proxy in front.)

116Errors. Application/json error shape, taken from RFC 7807 Problem Details (lightweight subset) — four members always, and one extension member when the error is a quarantine refusal (§2.19):

117{ "type": "vibe-index/error/integrity-mismatch", "title": "content_hash mismatch", "status": 409, "detail": "…" }

118instance is not emitted, and the subset is the four members above plus unavailable. RFC 7807 makes every member optional, so omitting instance is conformance rather than a gap — but naming a member the body does not carry teaches a client to look for it. What the body does carry beyond the four is the refusal row, as an extension member, which is the mechanism that RFC provides for precisely this and the reason the status can stay 404 while the answer stops being «not found».

2.11 CLI surface

119req r2

120Decision. vibe-index [--log-level LEVEL] <subcommand> <data-dir> [args]. The data directory is a required positional on every verb — the one argument no invocation can omit, so it is not dressed as an option. One global flag stands above the verbs: --log-level off|error|warn|info|debug|trace.

121# Lifecycle
vibe-index init <data-dir> --registry NAME --registry-url URL [--naming fqdn|kind-name|name|kind/name] [--force]
vibe-index dump <data-dir> [--format jsonl|json]
vibe-index verify <data-dir> [--json]                # recompute file hashes, check repomd
vibe-index rebuild <data-dir> --check                # journal-only reprojection + byte comparison

# Reindex
vibe-index reindex <data-dir> --from-clones <org-dir>                  [--full | --incremental] [--json]
vibe-index reindex <data-dir> --from-github <org> [--token-file FILE] [--api-base URL] [--clone-cache DIR]
                                                    [--cache-org | --no-cache-org] [--full | --incremental] [--json]
vibe-index reindex <data-dir> --from-gitverse <org>                    # emits stub-not-implemented today
vibe-index rescan-org <data-dir> --from-github <org> [--token-file FILE] [--api-base URL] [--clone-cache DIR] [--json]

# Read
vibe-index get <data-dir> <group> <name> [--version V] [--json]
vibe-index list <data-dir> [--kind K] [--limit N] [--offset M] [--json]
vibe-index search <data-dir> <query> [--kind K] [--limit N] [--json]
vibe-index capabilities <data-dir> <capability> [--json]
vibe-index purls <data-dir> <purl> [--json]
vibe-index outdated <data-dir> [--lockfile PATH] [--json]        # given a vibe.lock, print upgrade candidates

# Write (CLI-mode; refused if server is holding the lock)
vibe-index add <data-dir> --manifest <package.toml-path> --repo-url URL [--ref REF --commit SHA]
vibe-index remove <data-dir> <group> <name> [--version V]
vibe-index yank <data-dir> <group> <name> --version V --reason TEXT
vibe-index bury <data-dir> <name> --reason TEXT [--superseded-by GROUP/NAME]   # no group: the name is closed for all of them

# Server
vibe-index serve <data-dir> [--bind ADDR] [--auth-tokens-file FILE] [--read-only] [--auto-commit-push]
                            [--rate-limit-per-token N] [--rate-limit-per-ip N]
vibe-index stop <data-dir>                                       # graceful shutdown via lock-file PID

122Decision (owner, 2026-08-19). Taking a package out of circulation is three distinct operations over two different entities, and the surface must offer all three rather than making one stand in for the others:

123
operation entity what remains state today
full deletion — remove a version, or every version of a name nothing. The package is indistinguishable from one that never existed built — the verb emits its journal fact and the projector applies it
withdraw one version — the yank verb one version the record, carrying yanked on the wire: a build that already pinned this version keeps working, a fresh resolution passes over it built. The verb emits the fact, the projector sets the flag from it, and the wire omits the flag when false
retire a name — the bury verb a bare name a tombstone: the reason, and a successor to redirect to (§2.4) built. The verb emits the fact; the projector is the only arm that PRODUCES a carrier from one, dropping the name's packages across every group and leaving the tombstone in their place
  • 124Why three verbs and not one with flags. They differ in what a reader is owed afterwards, which is the only thing a caller actually chooses between: deletion owes silence, yanking owes "still here, do not pick it", retiring owes "gone, and here is where to look instead". A single verb with a mode would let the wrong one be selected by a typo — and two of the three outcomes are not reversible by re-running the command.
  • All three operations are built, and this line is the record of how the distance closed rather than a measurement of what remains. It has said three different things, each true when written, and the sequence is worth keeping because it is what a status marker is FOR: first «retirement is genuinely unbuilt» and the nearest existing fact, a rename, refusing outright as an unbuilt carrier; then «both remaining operations are one verb away», once the buried fact gave retirement its carrier; now neither is away from anything — yank emits its fact and the projector sets yanked = true, bury emits its fact and the projector drops the name's packages across every group and leaves the tombstone. The anchor is kept and its text corrected, never replaced, so a reader arriving from any of the three states lands on the same coordinate and learns which one they were reading.
  • A latent mine, measured while recording this, and it decides the shape of the work. The tombstone carrier is populated only by reading a catalog off disk; the projection built from the journal never sets it. Since the journal phase there is no read-then-write path — a mutation builds its state from the facts and writes that out — so a tombstone placed on disk by anything other than a journal fact would be erased by the next mutation, silently and with no failure anywhere. Nothing could reach this while nothing produced a tombstone at all, which is exactly what made it worth writing down before a producer existed: the retirement verb is not «write the field», it is «add the fact», and an implementation that takes the shorter route passes its own tests and loses the tombstone on the first unrelated publish. The producer built to this rule is the buried fact's projector arm, and the rule is now guarded rather than remembered — a test that buries a name, publishes something unrelated after it, and asserts the stone still stands, proved red by neutering the producer before it was believed green.
  • Decision (owner, 2026-08-19): renaming gets no carrier of its own — the tombstone already is one, and the journal gets ONE retirement fact carrying reason plus an optional successor. The renamed arm of the event vocabulary is retired in the same act.
  • Why not a second thing. This document's own worked example of a tombstone is a rename — {reason: "renamed to …", superseded_by: "org.vibevm/wal"} (§2.4) — and superseded_by is precisely the "go here instead" pointer a rename needs. More binding than the example: the standing naming law says a rename is a NEW IDENTITY, versions never transfer. A first-class rename relation would assert continuity between the old coordinate and the new one, which is the thing that law forbids; "the old name is closed, and here is where to look" is the only model consistent with it. And one question — where did this package go? — must have one place to look, or the two places eventually disagree.
  • Why the journal collapses them too, against this project's usual habit of keeping distinctions the projection folds. The deciding fact is measured, not aesthetic: the existing renamed arm carries from and to and no reason, while a tombstone requires one. So keeping it forces a choice between synthesising prose into a required field and adding a reason to it — after which the two facts differ only in whether the successor is optional, which is one thing spelled twice. The usual argument for keeping them apart (a rename is a stronger claim than a retirement-with-pointer) does not survive the naming law: under it, «renamed A→B» asserts nothing beyond «A is closed; B is where to look». The distinction the vocabulary would preserve is one this project has already decided not to make.
  • What it costs, and why now. This edits the truth layer's vocabulary, which is heavier than a catalog change and is done as a declared break with a note, never as a side effect. It was nearly free at the moment it was made and will not stay so for the next such change: nothing emitted renamed — only tests constructed it, and the projector refused it by design — no rename had ever been recorded anywhere in the tree, and that was measured beside a control rather than assumed (the same search that found no "kind":"renamed" did find "kind":"yanked" in the golden corpus), and there is no external consumer of the journal. The reverse direction is not lost, only relocated: a reader holding the NEW name learns its old one from the journal, which keeps the retirement fact and its successor forever. That is the right home — the catalog answers where do I go, the journal answers what happened.
  • A consequence carried out with the change, named here before it was made so it would not be left lying — and discharged. §2.18 listed Renamed among the arms the projector refuses because their carriers are unbuilt; renamed left the vocabulary and retirement gained a projected carrier in the same commit, so the sentence stopped being true in both halves at once and was corrected there. What this record could not name, and the landing had to find: the same commit falsifies three more statements of present state in this section — the state today column above, ##YANK-IS-A-VERB-AWAY's «retirement is genuinely unbuilt», and ##A-TOMBSTONE-THAT-IS-NOT-A-JOURNAL-FACT-ERASES-ITSELF's «nothing produces a tombstone at all». A contract that predicts one stale sentence and carries four is the argument for measuring the perimeter by file rather than by naming what one remembers (harvest/renamed-perimeter.md).
  • <status stage="spec" state="void">Retired 2026-08-19, hours after it was written, when the owner named the verb. It recorded that yank had a precedent to borrow and retirement did not, and left the name to the owner. Heir: ##THE-RETIREMENT-VERB-IS-BURY. This line stays so its name is never reused and inbound links do not break.</status>
  • Decision, taken while building the fact and recorded here because a future reader will re-open it. A Published fact for a name that carries a tombstone clears the tombstone; the name lives again and the projection carries no stone beside its packages. Why: §2.4 says the candidate-set file carries a tombstone «only when the bare name is buried», and its worked example shows an empty package list beside it — so a file holding packages AND a stone is a shape this contract never describes, and a reader would have to consult something else to tell «gone» from «here». The fold has no veto: it answers with the state as of the last fact, and refusing a publish would make the projection a policy engine rather than a projection. Considered and rejected: keeping the stone as history (it would make the two states of a candidate file overlap, and history is the journal's job — the burial is recorded there forever either way); suppressing the stone at write time when packages exist (that is a rendering trick over untruthful state, and the state is what rebuild --check compares). Revisit when: an operator needs «this name is closed» to outlive a re-publication — that is a different claim from a tombstone and would need its own carrier, not a change to this one.
  • The verbs are yank and bury (owner, 2026-08-19). yank borrows the ecosystem precedent it already has. bury is not invented for the occasion: this contract already describes the state in that word — the tombstone is «carried by the candidate-set file only when the bare name is buried», and the worked example is introduced as «a buried name therefore looks like this» (§2.4). The command and the state it produces therefore speak one word instead of two, which is the property the neighbours' candidates (deprecate / retract / relocate) each failed in a different way.

125What every withdrawal verb refuses, and why the two refusals must not share a message. Each of the three verbs answers its question against the projection of the journal, never against the catalog on disk, and refuses before appending anything — so a refused command leaves the journal exactly as it found it. Two refusals apply to all three, and they come from different laws:

  1. 126The target is already in the state the verb produces — the version already yanked, the name already buried. Refused because it is already law here rather than new: ##A-MUTATION-THAT-CHANGES-NOTHING-COMMITS-NOTHING. A second identical record would be a trace left by a change that did not happen.
  2. Otherwise, the target does not stand in the projection at all — no such version, or a bare name carrying neither packages nor a tombstone. Refused because the journal carries no false facts: a record of withdrawing what was never there asserts a state that never held. This is the rule remove has enforced since it was built.

127Why that condition is a conjunction and not a convenience, stated because the tempting relaxation defeats another verb. An operator who removes every version of a name leaves it carrying nothing — and full deletion's whole guarantee is that the result is indistinguishable from a package that never existed (PROP-010 §2.6). Letting bury then plant a tombstone on that empty name would put the deleted package's name back on the wire, in a file written precisely so it can be read — quietly undoing the operation the operator chose. So «nothing to bury» is the correct answer for an emptied name, and a future reader who finds the refusal unhelpful is looking at the mechanism that makes deletion mean what it says.

128The two conditions overlap unless the verb is written to keep them apart, and bury is where that bites. A buried name has no packages in any group, so «nothing stands here» is also true of it — a verb that tested the plainer condition first would answer «no such name» about a name whose tombstone it is holding. The requirement is that the two partition; how is the implementation's business. Two ways are equivalent and both are correct: test «already in that state» first, or spell the plainer condition as the conjunction it really is («no packages anywhere AND no tombstone»). What is NOT correct is the naive reading, in which the more specific case never reaches its own message.

  • 129The messages must differ, and that is not politeness. «Nothing to yank» told about an already-yanked version sends the operator looking for a problem that does not exist — they will re-publish, re-check the identity, or file a bug against the index. The two conditions are distinguishable at zero cost, so a verb that collapses them is discarding information it already holds.
  • bury takes a bare name and no group qualifier, unlike every other writing verb here. Not an oversight and not a convenience: the tombstone rides on by-name/<name>.json (§2.4), the candidate-set file that spans every group, so there is no per-group tombstone to address. Burying closes the name for all of its groups at once, and a --group flag would promise a narrowing the format cannot express.
  • --superseded-by is recorded as given and not resolved. The field is a redirect pointer «never an automatic rewrite» (§2.4): nothing follows it mechanically, so validating that it names a package this index knows would enforce an obligation the format never made — and would forbid the ordinary case of redirecting to a package published elsewhere, or not yet published at all. A wrong successor is a wrong sentence in a tombstone, which is the same class of error as a wrong reason.

130--json is a property of nine verbs, not of the binary. Every verb that ANSWERS a question carries it — get, list, search, capabilities, purls, outdated, verify, reindex, rescan-org; the six that perform an action and report only success (init, add, remove, dump, serve, stop) do not. dump is the instructive exception: it is machine output already, so a --json switch on it would be a second spelling of --format json. Saying «all subcommands» would send a script author looking for a flag six verbs do not have.

131--log-level is global (it may be written before or after the subcommand) and it folds into the one lever VIBE_LOG, which the subscriber reads exactly once at start-up: passing the flag SETS that variable, so the process environment always explains the output an operator is looking at. The flag speaks a closed set of six values while the variable keeps the full directive language — one thing with a coarse dial and a fine one, never two spellings of the same power.

132The tracing subscriber is installed on every invocation, at warn by default, and that is a decision rather than a default nobody chose. It is the binary's job and not the library's; there is no RUST_LOG fallback and no second lever. The reason is the refusal path: a version this build cannot act on is reported at WARN when a catalog loads (§2.19), and a publication failure is reported at WARN when the server pushes (§2.17). Both are things an operator must be able to see on any subcommand — so a subscriber installed only under some flag would make observability an accidental property of which verb happened to be running, which is how a message that exists is never read. The ordering that makes the flag honest is part of the same decision: the fold writes VIBE_LOG at the very top of main, after the parse and before the subscriber, so --help and parse errors still answer before any log and the variable is never left describing output it no longer governs.

133Help-text smoke lives under crates/vibe-index/tests/help_smoke.rs, mirroring every_subcommand_renders_help in vibe-cli.

2.12 Data structures

134req r1

135Decision. The catalog's wire types are generated from the schemas of §2.6 and re-exported, never written by hand: the definitions live in vibe_wire::generated beside the JTD they come from, crates/vibe-index/src/types/ re-exports them so every vibe_index::types::* path keeps its meaning, and cargo xtask check-codegen is the gate that refuses a drift between schema and type. VersionEntry comes from the shared version_entry vocabulary; NameEntry / PackageEntry / Tombstone from schemas/index/e1/by_name.jtd.json; BindingSite from by_purl.

136Two shapes stay hand-written, and each says why. Repomd / RepomdFileEntry — its size is a u64 where the schema language reaches only u32 (an open owner fork, BACKLOG.md B-091 — filed as B-056 and renumbered 2026-08-19, because that coordinate carried a closed row too and a reader following it landed there), and its files union is tagged by this document's own law (§2.4). And Index below, which is not a wire type at all: it is the server's in-RAM state, no single document ever carries it, and it holds two members the catalog deliberately never serialises — the reader's quarantine record and the per-name tombstones the writer projects back onto the candidate-set files:

137pub struct Index {
    pub schema_version: u32,
    pub registry: String,
    pub registry_url: String,
    pub naming: NamingConvention,
    pub generator: String,
    pub generated_at: DateTime<Utc>,

    pub by_pkgref: BTreeMap<PkgKey, PackageEntry>,
    /// The reader's record of the versions it refused to act on —
    /// in memory only, never written into any catalog file.
    pub quarantined: Vec<Quarantined>,
    /// Per-name tombstones; `write_to` projects them back onto the
    /// `by-name/<name>.json` it builds.
    pub tombstones: BTreeMap<String, Tombstone>,
}

// generated — schemas/index/e1/by_name.jtd.json
pub struct PackageEntry {
    pub group: Group,
    pub name: String,
    pub indexed_at: Timestamp,
    pub versions: Vec<VersionEntry>,        // ascending by version
    pub latest_stable: Option<Version>,
}

// generated — the shared `version_entry` vocabulary (§2.6)
pub struct VersionEntry { /* … */ }

138The generated types carry a fixed trait floor — Debug, Clone, PartialEq, Eq beside the serde pair — and Default is deliberately not in it. The dividing question is whether the trait says anything about the FORMAT: the four are properties of the Rust representation and the wire knows nothing of them, so emitting them unconditionally is the same class of decision as canonical ordering. Default is different in kind — «does this type have a meaningful empty value» is a judgement about the type, not a fact about its form. An empty ProvidesEntry means «provides nothing»; an empty VersionEntry means nothing at all, since twenty-odd of its fields are required. So Default lives in hand-written impls beside the generated tree, on the sub-structures where it is meaningful, and VersionEntry has none.

139The consequence, which nothing in the tree guards: because the record derives no Default, every literal that builds one names all its fields, and there is no ..Default::default() tail anywhere to shorten them. That is not an inconvenience to be optimised away — it is what makes adding a field a decision at every construction site instead of a silent zero. A future session that "simplifies" by deriving Default on the record would change how records are built from fixtures without any test going red, which is why the boundary is written here rather than left to be re-derived.

140What moving to generated types cost, measured rather than estimated, so the price is not re-paid in reverse. Three traits left with the hand-written shapes and the tree was checked for each: Ord / Hash / PartialOrd — wanted by nobody, and a comment claiming one of them justified duplicating a vocabulary had been false since it was written; Copy on the kind vocabulary — really lost, and the call sites that had it became explicit clones; Default — restored by hand where it means something. The classification is what matters, not the counts, which is why the counts live in the dated measurement (campaigns/packages-2026-09/harvest/f42c-reexport-radius.md) and not in this sentence. Anyone proposing to restore a duplicate type for the sake of Copy is proposing to unpay this, and should read what it bought first.

141A vocabulary fragment that changes in a new epoch is a DIFFERENT fragment, and it is separated by its NAME, not by a directory. The shared home holds fragments once, by name, and every schema module that pulls one re-exports it; when a second epoch needs a changed shape, the changed shape gets its own name and sits beside the old one in the same home. Worlds stay separated because two names never denote one type — the same rule that makes the shared home possible at all. Putting epochs into the home's directory structure instead would fork the home and give one fragment two addresses, which is the collision the single-home phase exists to prevent. Revisit when: the first second-epoch schema pulls a first-epoch fragment — that is the day this rule is exercised for real, and until then it has never been tested.

142PkgKey = (Group, String) — the (group, name) identity of PROP-008 §2.2, and the order by_pkgref walks in. kind is metadata and identifies nothing, so it is not part of the key.

143Search keeps no stored index. The postings are built per query against the loaded Index (index/search.rs) rather than held as a field, so no mutation has anything to invalidate. Token = lowercased ASCII alphanumeric run; ~30-stopword filter (the same list vibe-check::activation_conflict uses, deliberately reused for consistency). Ranking is term-overlap — one point per query token a hit carries — with the (group, name) identity breaking ties. Good enough for ≤10k packages; tantivy is a v1 upgrade if it isn't. Search answers only over what this build can act on: it asks the quarantine::usable_* accessors, never pkg.versions raw (§2.6).

2.13 Persistence layer

144req r1

145Decision. <data-dir>/ layout:

146<data-dir>/
├── hello.json                        # the eternal handshake (§2.4)
├── repomd.json                       # the manifest (§2.4)
├── primary.jsonl
├── primary.jsonl.gz
├── by-name/
│   └── <name>.json                   # no <kind>/ level — kind left package identity (PROP-008)
├── by-cap/
│   └── <slug>.jsonl
├── by-purl/
│   └── <slug>.jsonl
├── README.md                         # auto-generated; explains "this is a vibevm index"
├── .gitignore                        # written by `init`; covers state/
└── state/                            # NOT mirrored (gitignored when data-dir is a git working tree)
    ├── journal/<YYYY>-<MM>.ndjson    # the registry facts journal — the AUTHORITATIVE layer
    ├── server.lock                   # PID file, present only when serve is running
    ├── admin.tokens                  # bearer tokens (gitignored)
    ├── checkpoint.json               # incremental-reindex bookkeeping (last commit/tag per repo)
    └── org-cache.json                # the organisation image and its validator (§2.8.1)

147The one thing to notice in that tree: state/ is not mirrored, and the journal lives there. Everything above state/ is a projection and may be deleted and rebuilt; state/journal/ is the layer it is rebuilt FROM (§2.3), and the server refuses to start without it. So the directory's gitignore boundary and its truth boundary run in opposite directions — the served half is disposable, the unserved half is not — and an operator who backs up «the index» by copying what the mirror carries has backed up the derivative and left the original.

148/metrics counts from memory, not from a file. The counters are atomics in the server's own state and reset with the process, which is what an operational counter means; no state/stats.json exists, and a reader looking for one would find a durable-looking name for a volatile fact.

149The data-dir doubles as a git working tree of the org's index repo. state/ is .gitignored (the init subcommand writes a default .gitignore). Operators commit + push the rest manually, or via --auto-commit-push — built 2026-08-06, contract in §2.17.

150Atomic write protocol. For each file F to be replaced:

  1. 151Write F.tmp next to F.
  2. fsync(F.tmp).
  3. rename(F.tmp, F).
  4. fsync(parent_dir(F)) on POSIX. (No-op on Windows where the directory has no fsync semantics; rename itself is atomic.)

152Step 4 went unperformed until 2026-08-20; it is now atomic_write's closing act. The anchor keeps the finding's name: for one campaign every sync_all in the crate was on a FILE and no code path opened a directory to flush it — the gap was filed as BACKLOG.md B-087 rather than quietly dropped from the protocol, because a durability step deleted for being unimplemented is how a guarantee becomes folklore. What step 4 buys is the durability of the rename across a power loss on POSIX: without it the new bytes are safe and the directory entry pointing at them may not be. The B-087 landing closed it the conservative way — the code caught up to the protocol as ratified: fsync_parent_dir in index/persistence.rs runs after the rename (POSIX real, errors surfaced, never swallowed; Windows a no-op with the reason recorded in code, exactly as this step's own parenthesis allows), and every production projection writer reaches it through the one atomic_write. The journal shard stays outside by measured verdict, not omission: it is an append-in-place NDJSON write with no tmp→rename, so this protocol does not describe it; the lockfile and auth sync_all sites are test fixtures on read-only production paths.

153repomd.json is replaced last among the files it vouches for, so a reader that fetches repomd.json first then chases hashes always sees consistent files. The handshake is written after it and is the one root file that does not weaken the rule, because the manifest never claimed it (§2.4) — the precedent being README.md and .gitignore, which init writes and the map does not carry either.

2.14 Integration with the rest of vibevm

154req r1

155Consumer side (vibe-cli, vibe-registry).

  • 156crates/vibe-registry/src/multi_registry_resolver/ carries an optional index-aware fast path. Before falling back to per-repo git ls-remote, it opens a session by the discovery ladder of §2.1 — the handshake first, the manifest as the compatibility tail — and on success reads by-name/<name>.json for the pkgref, selects the candidate whose group matches, and picks the matching version locally: zero ls-remote calls. An absent probe falls through to today's path; a refused one surfaces its reason instead of pretending nothing was there.
  • Index-derived content_hash does NOT replace fetch-time verification. The actual git fetch still happens; the post-fetch compute_content_hash still runs; mismatch still errors out per PROP-002 §2.1.

157Publisher side (vibe-publish).

  • 158crates/vibe-publish/src/post_hook.rs carries an optional post-publish hook: when both VIBEVM_INDEX_URL_<REGISTRY> and VIBEVM_INDEX_TOKEN_<REGISTRY> are set for the registry being published to, the publisher POSTs the new entry to <index-url>/v1/packages with a bearer token after a successful push_release. Failure of the index POST does NOT fail the publish — it logs a warning and the operator's next vibe-index reindex covers the gap.
  • Both settings are per-REGISTRY environment variables, and the distinction matters. The suffix is the registry's local alias from [[registry]].name, not the host — one host can serve several registries and one registry can move hosts, so keying on the host would name the wrong thing in both directions. The manifest fields this document once promised (index_url, index_token) do not exist (§2.2), so the environment is not one source among several here: it is the only one.
  • Direct-push (--repo-url) bypasses index updates entirely (no registry context).

159vibe outdated (M1.10 follow-up).

  • 160Adds a fast path: when a registry has an index, query by-name/<name>.json for the latest version instead of git ls-remote. Same envelope shape; ~100× faster for large lockfiles.

161vibe search (M2.10 — this is what unblocks it).

  • 162Walks every configured registry's index through the same client the resolver uses — probe, then query — rather than downloading primary.jsonl.gz and scanning it locally. The whole-file scan was the shape this document first imagined and is not what shipped: asking the index a question keeps the bandwidth proportional to the answer instead of to the catalog, and it puts one discovery ladder (§2.1) under every consumer instead of two. Index is the enabling layer for M2.10; vibe search is the headline consumer of this PROP.

163Each integration point is a separate slice. v0 of vibe-index ships without any of them — the index can be populated and consumed via raw HTTP / git clone before vibevm consumers know about it. Integration slices land in M2.10 / M1.10 follow-ups.

2.15 What index must NEVER do

164req r1

  • 165Never replace vibe.toml as the source of truth. A package with a missing index entry still installs from git per the live path. A package with a divergent index entry triggers IntegrityError, never silent acceptance.
  • Never modify package repos. The index utility reads package repos (for the --from-clones walk) but never writes to them.
  • Never echo tokens. Same discipline as PROP-000 §20. Auth tokens for the server, GitHub API tokens for --from-github, publish tokens propagated through hooks — none ever appear in stdout / stderr / logs / JSON envelopes.
  • Never assume mirror infrastructure. The index is opt-in everywhere; no consumer or publisher fails because the index disappeared.
  • Never make breaking schema changes silently. The refusal is what matters and it survives; where it LIVES moved with the journal. A build must never read the subset it understands and carry on as though it understood the whole — but the carrier of that refusal is no longer a version number on the catalog, because the catalog is a projection any build rewrites from facts. It is the per-record capability set: a record naming a capability the reader lacks is refused BY NAME, with a recipe, and the rest of the catalog still loads (PROP-044 §4.5; the answer's shape and the surfaces that owe it are §2.19). That is strictly louder than a version compare, which could only say "somewhere in here is something newer than you". Unknown FIELDS are a different question and are answered by ##FORWARD-COMPAT: they are tolerated, because tolerating them can no longer lose them.

2.16 Webhooks — feeding the index instead of polling it

166The problem, and it is not performance. To learn what changed, the index enumerates the organisation. That is a cost, but the cost is the small half. The large half is that the picture goes stale without anyone doing anything wrong: a developer publishing a package creates a repository and pushes a tag straight to the git host, never passing through the index service. So the index's image is behind from the moment it is taken, and a stale index is worse than a slow one — the package exists and cannot be found.

167This is what makes the organisation cache honest. §2.8's cache and its cheap freshness check reduce how often the index asks; they do not change who knows first. A webhook does: the image becomes authoritative because it is fed, rather than because we assumed nobody else writes. Both mechanisms ship — the freshness check is what keeps the cache truthful when no webhook is configured, and every deployment starts that way.

168The endpoint is per host flavour, deliberately. POST /v1/hooks/{source} where {source} names the git host's flavour (github today; other flavours are added when one is measured, not reserved in advance). One route per flavour rather than one generic route, because the payload shape and the signature scheme are host-specific and a single endpoint would have to sniff which it received — which is guessing, dressed as convenience.

169A webhook authenticates with its own shared secret, never with an admin token. The sender signs the request body under a per-hook secret and the index verifies that signature; an unverifiable request is refused and not processed. Two reasons, and the second is the important one. (i) The secret is held by a third party — the git host — and must be rotatable without touching the tokens that authorise real writes. (ii) Least authority: an admin token authorises arbitrary writes, while a webhook may only cause the index to re-read one repository. Handing a notification channel the authority to write anything is how a notification becomes an attack surface. Secret storage follows §7 — file, never a flag, never a log.

170The load-bearing rule: a payload says that something changed, never what it now is. The index reads the manifest from the git host itself and applies the per-package add / remove of §2.8 for the named repository only. Nothing from the request body is ever written into an index record. This follows from §2.3 — package repos remain authoritative — and it is the rule that keeps an attacker-influenced input from becoming index content. A webhook may never trigger a full reindex, both because that is the expensive thing it exists to avoid and because a cheap request that causes an expensive walk is a denial-of-service lever.

171Deliveries arrive twice, out of order, or not at all, and the design assumes all three. The handler is idempotent by construction — re-reading a repository and upserting its versions is the same operation performed twice — and it never infers from one event that it saw the previous one. Consequence, stated so it is not quietly dropped later: webhooks reduce staleness, they do not abolish the full walk. The explicit rescan-org verb of §2.8 stays unconditional exactly because a missed delivery is invisible from inside.

172What each failure answers. An unverifiable signature is 401 and is not processed. A payload naming a repository outside this server's configured organisation is 400 and is not processed — the scope check is not optional, since the repository name is the one field of the payload we act on. A verified, in-scope delivery whose re-read fails is 202: the request was accepted and the work is ours to retry. Answering 5xx to a sender that retries on a schedule we do not control would turn our own outage into a retry storm — the failure is on our side, and the status code should say so rather than invite the sender to hammer.

173The GitHub-Actions alternative, and why it is the fallback rather than the default. The same effect is reachable without any endpoint: an Action in the package repository POSTs to the write API of §2.10 with an admin token. It is genuinely simpler — no new route, no signature verification, nothing to specify. What it costs is exactly what #WEBHOOK-SECRET-IS-NOT-THE-ADMIN-TOKEN protects: every participating repository then holds a credential that can write anything, and the number of places a broad secret lives grows with the organisation. The webhook keeps one narrow secret in one place. So: webhook by default; the Action is the honest answer for an organisation that already runs its publishing through Actions and prefers one mechanism to two, and it has one capability the webhook does not — it can also push the built index files, being a runner with a checkout.

174Named, not invented — two things this section does NOT settle. (i) Whether the server should require the pushed ref to look like a release tag before acting, or re-read on any push to the default branch as well: the second catches a manifest edited without a tag, which §2.8's incremental walk already treats as a real case, and the first is cheaper. Decide it against a measured event volume, not here. (ii) The GitVerse flavour is unmeasured: their public API could not enumerate an organisation, and nothing here establishes what their webhooks can do. Writing a gitverse route from that ignorance would be inventing a contract for a system nobody in this repository has watched.

2.16.1 Setting one up — the operator's walkthrough

175Why this walkthrough sits inside the specification and not in docs/ (owner ruling, 2026-08-06). It describes how to configure a mechanism whose properties this document defines. Kept beside the contract it changes when the contract changes, because the two are the same file and the same commit; kept in docs/ it drifts. That is not a hypothetical: this repository measured two independent instances of exactly that drift in a single week — the index's own format documentation against its code, and an owner guide promising a gate step that did not exist.

176Read the steps below as a specification, not as instructions that work today. The endpoint is designed here and not built; the block is therefore an example and carries no @fact/code: marker, which is precisely the distinction that keeps a fenced block from asserting something nobody can falsify. When the route ships, this block becomes the fact's body and comes due with it.

177# 1. On the index host: put the shared secret where the server reads it.
#    One secret per configured hook; file, not a flag (§7).
$ printf '%s' "$SECRET" > ./vibespecs-index/state/webhook.secret

# 2. Start the server with the hook route enabled.
$ vibe-index serve ./vibespecs-index --bind 0.0.0.0:8412 \
    --auth-tokens-file ./vibespecs-index/state/admin.tokens \
    --webhook-secret-file ./vibespecs-index/state/webhook.secret

# 3. On the git host, at the ORGANISATION level (not per repository —
#    a per-repo hook is one more thing to remember on every new package):
#      payload URL   https://<index-host>/v1/hooks/github
#      content type  application/json
#      secret        the same $SECRET
#      events        pushes and tag/release creation only — not "everything"
#
# 4. Verify the wiring before trusting it: push a tag to any package repo,
#    then ask the index what it now knows about that package.
$ vibe-index get ./vibespecs-index <group> <name>

178Step 4 is not politeness. A hook that is configured and silently not arriving looks exactly like a hook that is arriving and finding nothing to do, and the difference is invisible from the index side — which is the same shape as #WEBHOOK-DELIVERY-IS-UNRELIABLE, seen at setup time instead of at run time. Verify once against a known change; after that, trust the mechanism and keep rescan-org for the deliveries you will never know you missed.

2.17 Auto-publication — the server carries its own result to the host

179What it fixes. The server already accepts an authenticated write, writes the files atomically, recomputes the manifest and verifies integrity. The one thing it could not do was carry the result to where it is served from. --auto-commit-push closes that: after each successful mutation the server commits the data directory and pushes it. Built 2026-08-06 on the owner's ruling; the flag had been declared and discarded by one line since the server shipped.

180Where it publishes is the operator's setting, and vibe-index does not mint a second place to say so. The data directory is already a git working tree (§2.4), and a working tree's remote and branch are configured with plain git. So the push carries no refspec and names no remote — it goes where the tree is already pointed. A private repository is a legitimate target by construction: it is simply what the operator cloned. Rejected: a --push-remote / --push-url pair, and a target block in the on-disk config — both would be a second home for a value git already owns, which is the defect class this repository keeps paying for.

181Startup refuses rather than warns, and this is the flag's most important behaviour. state/ holds the bearer tokens (§7). If the data directory's .gitignore does not cover them — a directory created before init wrote one, or one edited since — git add -A would stage those tokens and push them to a host that may be public. So with the flag set, the server does not start unless it confirms state/admin.tokens is ignored, and the refusal says so in those words. The check runs once at startup rather than per mutation, because the operator must learn the configuration is unsafe before the first token leaves the machine, not after.

182The second refusal: the flag set over a data directory that is not a git working copy also stops the server, naming what to do. Publishing by committing a directory that is not tracked is not a degraded mode, it is a no-op that would look like success forever.

183A push failure never turns a successful write into an error. By the time publication runs the mutation is on disk and in memory; the HTTP write has happened. So a failure is logged at warn with git's own message and counted as vibe_index_publish_failures_total, and the request still answers as it would have. It is not rolled back either: a network outage must not be able to corrupt index state. Transient failures self-heal — git accumulates, and the next successful push carries the queued commits.

184Nothing to commit is success, not an error — the opposite of the publish flow's rule for the same operation, and deliberately so. The index lock is released before the push, so a second mutation can land while the first is publishing and the first commit carries both. The second then finds nothing staged, and that is the normal course of events rather than a caller's mistake.

185One publication at a time, and the response waits for it. Two concurrent mutations must not interleave two commits in one working copy, so publication takes its own lock — not the index lock, which is released earlier and correctly so. The handler awaits the result on a blocking thread, which means a 200 says «persisted and published». For an index that is what the operator asked for, and mutations are publish events rather than a hot path.

186The commit message names what moved — the upsert or the removal, with the package coordinate. Each of the three mutating routes knows its own change, so the index's history reads as a log of publications rather than a wall of identical messages.

187<status stage="spec" state="void">Retired 2026-08-14 by the determinism phase. It recorded a real consequence of the writer stamping a fresh generation time on every write: a repeated identical upsert produced a diff and therefore a commit, so the empty-diff path fired only on the overlap case. Both halves of that consequence are gone — the writer takes its clock as an input (##THE-WRITER-TAKES-ITS-CLOCK-AS-AN-INPUT), and a mutation that changes nothing no longer writes at all (##A-MUTATION-THAT-CHANGES-NOTHING-COMMITS-NOTHING). Its closing sentence asked whether the index's own write should be deterministic and called that a question about the format rather than about this flag; the format answered yes. This tombstone stays so the old sentence's name is never reused and inbound links do not break.</status>

188A mutation that changes nothing writes nothing, and therefore commits nothing. An upsert whose entry equals the one already stored under that version number leaves the in-memory state untouched, never reaches the writer, and never reaches the publisher; the two removal routes have always behaved this way, and the upsert route now matches them. The response is still success — the resource is already in the requested state, which is what idempotency means over HTTP — and the distinction between created and changed is kept, because a differing entry under an existing version number is an update and must still land. Determinism alone would not have bought this: with the clock arriving per mutation event, a repeat would still have moved generated_at and produced a diff. The point is not to produce an empty diff but to not create the work, so that the catalog's history records events that actually happened.

189No identity is invented. The commit uses whatever git identity the host is configured with; if there is none, git commit fails and that failure takes the path above — logged and counted, never fatal to the write. Inventing a fallback author would put a name in an organisation's published history that nobody chose.

2.18 Channels — author-named version pointers

190req r2

191Decision (owner rulings, 2026-08-13; not built — this section is the contract the build will follow). A channel is an author-controlled named pointer (group, name, channel) → version — npm's dist-tags and Docker's tags are the prior art. The pointer map is flat: several channels may point at one version (a release that is both latest and stable is the everyday case), and no promotion semantics (betastable as a registry operation) are baked into the format — promotion is the author's workflow, not registry law. A channel may point at a snapshot or at a frozen version (PROP-044 §2b) — the axes are orthogonal.

  • 192What already exists, so nobody mints it twice: the journal's event vocabulary carries ChannelSet {group, name, channel, version} and ChannelUnset {group, name, channel} in exactly the shape below, and the generated entry types carry a channels list. What is NOT built is the projection and the surfaces. And the projector's treatment of that gap is the load-bearing part: meeting a channel act it refuses the whole projection by name — «the journal holds a ChannelSet record, but its carrier (channels) is not built in this vibe-index; skipping the record would project a catalog the journal does not describe» — rather than skipping the record and continuing. The journal is truth (§2.3); a projector that quietly dropped an event it did not understand would publish a catalog asserting a state nobody recorded, which is the one failure re-fetching cannot cure. Notice and ForceReplaced stand in the same place for the same reason. Renamed stood there too until the retirement collapse: it left the vocabulary, and its successor Buried is the one arm that went the other way — it gained a carrier and now PRODUCES a tombstone instead of refusing, which is what makes it the first producing arm this projector has ever had (§2.11).
  • Channel-name grammar: [a-z][a-z0-9-]* — it must not start with a digit (versions do) or a version-requirement operator (^ ~ = < > *), which is what makes @beta unambiguous in a pkgref's version position.
  • Authority is the journal; the catalog only projects. Channel state changes are registry facts: Published carries the manifest-declared channels (below), and the explicit acts ChannelSet {group, name, channel, version} / ChannelUnset retarget or clear a pointer. No hand-edited pointer file exists anywhere — a hand-written vibeversions.toml inside the derived catalog would be a secretly-authoritative fact (PROP-044 law 2). The projection lands the map in the NameEntry (channels: {stable → 1.1.0, beta → 1.2.0-rc.1}) — the same by-name/<name>.json candidate file the resolver already fetches, so channels cost zero additional round-trips.
  • The manifest declares membership as a publish-time fact, never as the pointer. [package] channels = ["stable", "lts-2026"] (a list — multiplicity is first-class; the singular channel = "…" is rejected with a did-you-mean) records which channels this version was published into — immutable with the content, honest forever. Publication moves each named pointer to this version (npm's publish --tag semantics), so routine channel management is just publishing — no separate command. The pointer itself cannot live in the manifest: retargeting stable back to a frozen 1.1.0 (the rollback — the main use case) would require editing frozen bytes, which is forbidden; that act is the journal's ChannelSet, via vibe registry channel set <group>:<name> stable 1.1.0.
  • LATEST and STABLE are the two built-in channels (Maven's <latest>/<release> pair). A channel is authored from the first explicit act (a manifest declaration or a channel set) and stays authored until channel unset; while unauthored it is computed at projection time: STABLE = the greatest non-prerelease version, LATEST = the greatest version outright — both by the ordering below. A new publication without declarations does not move an authored pointer: if the author said stable = 1.2.0, releasing 1.3.0 does not silently make it stable.
  • The ordering (owner ruling, 2026-08-13): SemVer precedence first, natural-sort tie-break on build metadata second. SemVer 2.0.0 is not modified: +build is legal in published versions, the coordinate is the full version string including +… (uniqueness is hard on the string), and precedence ignores metadata exactly as the standard demands — every foreign semver library agrees with us. Where SemVer declares two versions equal, our resolver breaks the tie deterministically: versions with metadata outrank the bare version (a +stamp is a rebuild atop it), and among metadata the greater under natural sort (digit runs compare numerically, text lexicographically) is the fresher — so +20260813… beats yesterday's stamp. Reproduction is never at stake — the lockfile pins content_hash — the tie-break only answers "which is latest", and only those who chose to publish + twins pay the axis any attention.
  • The resolver default (owner ruling, 2026-08-13): vibe install pkg with no requirement and no channel takes STABLE when it exists, else LATEST — and the frozen/snapshot state does not influence selection at all. Selection and integrity are separate axes: the chosen version is pinned by hash in the lockfile, and after selection the freeze contract governs mismatches (frozen — alarm; snapshot — news). Requesting a channel is the explicit act: { channel = "beta" } on a dependency in vibe.toml (mutually exclusive with a version requirement) or @beta in a pkgref's version position.
  • Resolution through a channel pins {channel, resolved version, content_hash, locator} in the lockfile: vibe install reproduces the pin; vibe update re-follows the pointer; --locked turns any drift into a loud CI error.
  • An authored pointer at a dead target is a loud state. When a channel's target version is yanked or removed, computed channels simply recompute past it, but an authored pointer refuses at resolve time with a recipe («stable указывает на отозванную 1.2.0 — автор должен переставить или снять; потребитель может явно взять версию»). Silently hopping to "the next best" would be a choice the author never made.
  • The degraded ladder (catalog unreachable). (1) A lockfile answers without the catalog at all — resolve-by-lock never needs it. (2) No lock but a local catalog cache → resolve against the cache, loudly stamped «по снимку каталога от <даты>». (3) Cold resolve with the provider alive → enumerate versions at the provider (ls-remote --tags-class), read manifests at tags, and let the local resolver reconstruct channels from the publish-time channels declarations — approximate exactly where the author manually retargeted, and the output says so («по перечислению провайдера, каталог недоступен»). (4) Neither reachable → refusal with a recipe. Degradation is always announced, never silent (PROP-044 law 1).

2.19 The unavailable answer — what a surface says about a version it will not serve

193req r1

194Decision. A surface that cannot act on a record does not drop it — it names it. PROP-044 §4.5 gives the law («the refusal surfaces at the point of use with a generated recipe»); this section gives its shape in this catalog. A version whose must_understand (§2.6) names a capability this build lacks is unavailable to this build, and every surface that computes an answer says so out loud. Quietly narrowing the answer instead would be the silence PROP-044 §2 forbids: the package exists and cannot be found, which is a riddle rather than a break.

195The answer row is one shape, used by every surface: {group, name, version, missing, recipe}. It carries the full coordinate even where the envelope around it already names the package — a row that identifies itself survives being copied out of its envelope by a script, and a context-dependent one does not. missing is exactly the subset of the record's must_understand this build does not understand — not the whole declaration, because a reader that understands three of four capabilities must be told about the fourth, not about all four. recipe is the generated text that says what a person or a script does about it.

196The recipe is built in one place and never written as a literal at a call site. One home, N surfaces: a literal per surface is N texts that drift, and the one that drifts is the one nobody reads until it matters. It is degenerate today by measurement, not by omission — no reader capability has been built yet, so every missing capability is one this build simply does not know and there is no second class of recipe to write. The per-capability table this grows into gets its first row from the first capability that lands; inventing rows for capabilities that do not exist would be machinery for a consumer that does not exist.

197Quarantine is the READER's judgement about a (record × build) pair, never a property of the record — so it is derived at the point of use from the record's own must_understand and is never stored on the wire. The consequence worth the ink: the command line and the server agree by construction, not by two implementations being kept in step. The predicate reads the record, so it does not matter which carrier the record arrived in — and the carriers genuinely differ, since a catalog LOADED from disk arrives with a quarantine record while one PROJECTED from the journal arrives with an empty one. Two surfaces that agreed only because someone remembered to update both would disagree the first time one of them was forgotten.

198The safe default is a property of the construction. The answering path asks the named accessors (quarantine::usable_*) and never reads the stored version list or latest_stable raw; the writer's path, the mutations, and the operational counters ask the raw state deliberately — the catalog is the projection of the journal (§2.3), and a reader's capabilities have no business shrinking what is WRITTEN or miscounting what the index HOLDS. The asymmetry is stated in the doc-comments of both sides, and that statement is the only defence against the next author reaching for the wrong accessor: the two calls look identical at the call site and differ only in what they mean.

199Which surfaces owe it, as a rule rather than a list — because a list rots and a rule does not: every surface that COMPUTES an answer owes the refusal; a surface that serves a stored file verbatim does not. Computing covers each read verb that selects, narrows, ranks or aggregates, and each HTTP route that answers from the in-RAM index. Serving verbatim covers the raw file routes of §2.10.

200The raw candidate-set file is not silent, and making it «speak» could only mean removing information from it. by-name/<name>.json hands back the record word for word, must_understand included — and that declaration IS the explanation of the refusal, delivered to a client that can then apply its own capability set rather than ours. Silence lived exactly in the surfaces that computed an answer and dropped a record without a word; the file that says everything was never the problem.

201Over HTTP the status stays 404 and the body carries the reason. «You did not get the thing» is preserved for every client that only reads status codes, while the problem document's type and title name the refusal in its own words — not «resource not found» — and an extension member carries the whole answer row (§2.10 fixes the RFC 7807 shape; extension members are what that RFC provides for exactly this). The judgement rides the envelope and never enters the record: a VersionEntry is generated from the schema and says nothing about any reader.

3. Architecture

3.1 Crate layout

202design r1

203crates/vibe-index/                          # a member of the vibevm workspace
├── Cargo.toml                              # depends on vibe-core + vibe-wire; no [workspace] table
├── README.md                               # operator-facing — how to run, common recipes
├── src/
│   ├── main.rs                             # bin entrypoint — clap dispatch
│   ├── lib.rs                              # exports, top-level Error/Result
│   ├── error.rs
│   ├── cli/                                # one file per verb (§2.11) + kinds.rs
│   ├── journal/                            # THE AUTHORITATIVE LAYER (§2.3)
│   │   ├── record.rs                       # the event vocabulary
│   │   ├── store.rs                        # append-only shards under state/journal/
│   │   ├── project.rs                      # journal → catalog; refuses unbuilt carriers
│   │   └── mod.rs
│   ├── index/
│   │   ├── mod.rs                          # the writer's owned surface (§2.4)
│   │   ├── memory.rs                       # Index struct + ops
│   │   ├── quarantine.rs                   # the reader's judgement + the refusal (§2.19)
│   │   ├── persistence.rs                  # atomic write/read of files
│   │   ├── primary.rs                      # JSONL serialise/parse
│   │   ├── by_name.rs                      # candidate-set JSON
│   │   ├── inverted.rs                     # by-cap / by-purl
│   │   ├── repomd.rs                       # repomd.json
│   │   ├── checkpoint.rs                   # incremental-reindex state
│   │   └── search.rs                       # per-query postings, not a stored index
│   ├── scanner/
│   │   ├── mod.rs                          # source-of-truth walkers
│   │   ├── from_clones.rs                  # walk org-dir clones via shell git
│   │   ├── from_github.rs                  # GitHub REST API walk
│   │   ├── org_walk.rs                     # the organisation enumeration
│   │   ├── org_cache.rs                    # the org image + its validator (§2.8.1)
│   │   ├── manifest.rs                     # parses through vibe-core
│   │   └── git_cli.rs                      # the shelled-out git
│   ├── server/
│   │   ├── mod.rs                          # axum app builder — the 16 routes of §2.10
│   │   ├── routes/                         # health · index_files · packages · capabilities
│   │   │                                   #   · purls · admin · metrics
│   │   ├── auth.rs
│   │   ├── error.rs                        # RFC-7807 mapper + the refusal extension
│   │   ├── rate_limit.rs                   # per-token / per-IP buckets (§9 Q10)
│   │   ├── metrics.rs                      # hand-rolled text serialiser
│   │   └── state.rs                        # AppState
│   ├── types/                              # re-export seam over the generated wire types
│   │   ├── mod.rs
│   │   ├── entry/                          # aggregate · content · relations
│   │   ├── repomd.rs                       # the one hand-written shape (§2.12)
│   │   └── kinds.rs                        # PackageKind, NamingConvention dupes
│   ├── publish.rs                          # auto-commit-and-push (§2.17)
│   ├── lock.rs                             # the single-writer PID lock
│   ├── lockfile.rs                         # reading a vibe.lock for `outdated`
│   ├── hash_recipe.rs                      # the recipe a content_hash rides with
│   └── content_hash.rs                     # mirrors vibe-registry::compute_content_hash exactly
├── fixtures/
│   ├── golden-flow-wal-1.0.0/              # the parity fixture
│   └── golden-order-trap-0.1.0/            # the tree where recipes 0 and 1 disagree
├── tests/                                  # help_smoke · cli_{lifecycle,read,write} · server_e2e
│                                           #   · server_writes · auto_publish · rate_limit_e2e
│                                           #   · org_cache_e2e · scanner_e2e · from_github_e2e
│                                           #   · golden_corpus · round_trip_published
│                                           #   · content_hash_parity · six wire_parity_*
└── docs/
    ├── operator-handbook.md
    ├── consumer-protocol.md                # HTTP API reference
    └── format.md                           # repomd / primary / by-name / by-cap / by-purl

204There is no LICENSE inside the crate, and that is the correct state. Cargo.toml carries license-file.workspace = true, so the crate inherits the repository's licence rather than keeping a second copy that can disagree with it — the same single-home rule this document applies to normative values, applied to the one value a licence is. The repository's licence is UPL-1.0.

3.2 Dependencies

205design r1

206Minimal Rust crates to keep redistribution clean:

  • 207clap (derive) — CLI dispatch.
  • tokio — async runtime for the server, with the four features named below.
  • axum — HTTP framework. Mature, minimal, integrates with tower middleware.
  • tower / tower-http — auth, CORS, tracing layers.
  • serde / serde_json — JSON.
  • toml — read package manifests.
  • semver — version handling. Same dep vibe-core uses; pin same version.
  • sha2 — content_hash. Same as vibe-registry.
  • flate2 — gzip primary.jsonl.gz.
  • walkdir — directory traversal (matches vibe-registry).
  • tracing / tracing-subscriber — logging.
  • chrono — timestamps.
  • thiserror — error enums.
  • gix (or shell-out via std::process::Command) — read git tags / show files at refs. Decision §3.3.
  • reqwest--from-github HTTP client.
  • tempfile — atomic write helpers.
  • <status stage="spec" state="void">Retired 2026-08-18 by measurement: the prometheus crate is not a dependency and never became one — /metrics renders the exposition format from a hand-written serialiser. The heir is ##THE-METRICS-DEPENDENCY-WAS-NOT-TAKEN below, which records the choice and its reason. This tombstone stays so the anchor's name is never reused and inbound links do not break.</status>
  • specmark — the in-code spec markers (scope!, #[spec]) the traceability map is built from.
  • vibe-wire — the generated wire types this crate's types module re-exports (§2.12). A runtime dependency, not a test one: the library's types ARE the wire's types.

208No prometheus crate is pulled, and the omission is the design. /metrics renders the Prometheus text exposition format from a hand-written serialiser, because the surface is a handful of counters and the exposition format is stable text — so the dependency would buy formatting we can write once and cost a tree we then carry forever. This is what «minimal crates to keep redistribution clean» means when it is applied rather than stated.

209tokio is taken with four features — signal, sync, time, fs — not full. full is the shape a project reaches for before it knows what it uses; naming the four is the same discipline as the paragraph above, one level down.

210vibe-core dependency. vibe-index parses vibe.toml and vibe-subskill.toml through vibe-core's own Manifest / SubskillManifest types, so the index can never drift from the manifest schema. This reverses the proposal's original standalone-no-vibe-core stance — §6 records the reversal, §9 item 11 the de-rot finding that forced it. What stays duplicated is small and stable: the four-variant PackageKind / NamingConvention (src/types/kinds.rs, frozen by VIBEVM-SPEC.md §4, needing the Ord + clap::ValueEnum the vibe-core originals lack) and the compute_content_hash algorithm (src/content_hash.rs, gated by tests/content_hash_parity.rs). compute_content_hash folds into vibe-core once it is lowered out of vibe-registry.

211The parity gate is wider than one fixture and one algorithm. It runs BOTH implementations over BOTH fixtures in BOTH recipes: fixtures/golden-flow-wal-1.0.0/ is the ordinary package, and fixtures/golden-order-trap-0.1.0/ is the tree built to make recipes 0 and 1 disagree — a directory whose name is continued by a sibling file at a byte below /, the only shape on which component-wise and byte-wise ordering part company (PROP-002 §2.1). A single fixture would let the two implementations agree by accident on every tree that never exercises the difference, which is exactly how a hash regression once reached a consumer before any golden noticed.

212Deliberately NOT pulling:

  • 213A database (SQLite / PostgreSQL). All state in RAM + flat files.

3.3 Git access in the scanner

214design r1

215Decision. Use shell-out to git via std::process::Command for the scanner's read paths (git tag, git show <ref>:<path>, git rev-parse <tag>). Same path vibe-registry::shell.rs already follows. Rationale matches PROP-001 §2.1: shell-out works on every platform git works on, no per-host bindings to maintain.

216Not gix for v0: smaller dep tree wins. v1 may switch if perf demands and gix's read API matures further.

3.4 Threading model

217design r1

  • 218CLI mode: synchronous. tokio runtime is created only in serve subcommand.
  • Server mode: tokio multi-thread runtime. Routes are async; Arc<RwLock<Index>> is tokio::sync::RwLock (async lock).
  • <status stage="spec" state="void">Retired 2026-08-18 by measurement. It described a dedicated index_writer tokio task fed by an mpsc channel, so that fsync stalls would not block the request handlers. Neither the task nor the channel was ever built — measured as zero occurrences of both names in the crate, against a live control — and the problem they were designed for dissolved when a mutation became an append to the journal plus a reprojection. The heir is ##THE-MUTATION-IS-WRITTEN-BY-ITS-OWN-HANDLER below. This tombstone stays so the old sentence's name is never reused and inbound links do not break.</status>

219Each mutating handler does the whole write itself, under the index's async write-lock: replay the journal, project a probe, append the event, reproject, write the catalog, swap the in-memory index. There is no writer task and no channel to post to. The queue that the task was reaching for turned out to be unnecessary once mutations became journal appends: a handler holds the lock for one append plus one projection, and the operations it serialises are publish events rather than a hot path (§2.17 makes the same argument about awaiting the push). What IS serialised separately is publication — its own lock, and a blocking thread, because a git command must not run on the async executor.

3.5 Configuration precedence

220design r1

221For every flag with a default, precedence is: explicit CLI flag > env var (VIBE_INDEX_*) > on-disk config (<data-dir>/state/config.toml, optional) > built-in default. Same shape vibe show config already uses on the consumer side. (Built 2026-08-20, B-086: the machine is crates/vibe-index/src/config.rs — every resolution carries the value AND its source, surfaced by the vibe-index config <data-dir> verb; the file rung parses strictly, an unknown key is a loud refusal. Four members ride it (log-level — with VIBE_LOG as the recorded legacy synonym below VIBE_INDEX_LOG, git, api-base, dump-format); the remaining ~17 flags-with-defaults join by the worked pattern — the mechanical continuation and its two owner forks (boolean value dialect; the --limit naming collision) are listed in the R3-LADDER worker report and the release-TZ log. data-dir is not a member: it is the required positional (##CLI-SURFACE) and the file rung lives inside it.)

222None of that ladder exists. There is no config.toml anywhere in the crate and no VIBE_INDEX_* family: a flag with a default gets it from its own declaration, full stop. Two environment variables do exist and neither is part of a precedence chain — VIBE_INDEX_GIT overrides the git binary the scanner shells out to, and VIBE_LOG is the logging lever --log-level folds into (§2.11). The requirement stands and the fork is BACKLOG.md B-086: build the ladder, or say plainly that this binary is configured by flags and two named variables. What must not survive is the middle state, where a document describes a resolution order an operator can neither use nor observe.

4. Phase plan (slices)

223Each slice = one or more conventional commits. The utility becomes useful at slice 5 (read endpoints + reindex from clones); the rest are progressive enhancements.

4.1 Slice 1 — skeleton

224crates/vibe-index/ standalone crate with Cargo.toml + src/main.rs + src/lib.rs. clap dispatch with stub subcommands that all print "not yet implemented". vibe-index --version works. tests/help_smoke.rs passes.

225Commit: feat(services/vibe-index): skeleton crate + clap subcommand dispatch.

4.2 Slice 2 — types + persistence

226src/types/ (entry / repomd), src/index/ (memory, persistence, primary, by_name, repomd). JTD schemas in schemas/. Atomic write protocol. vibe-index init works (writes empty repomd.json + empty primary.jsonl). vibe-index dump works. vibe-index verify works (checks file hashes). Round-trip tests.

227Commits:

  • 228feat(services/vibe-index): index entry + repomd schemas + JTD
  • feat(services/vibe-index): in-memory index + atomic persistence
  • feat(services/vibe-index): vibe-index init/dump/verify

4.3 Slice 3 — scanner + reindex from clones

229src/scanner/from_clones.rs walks <org-dir>/<repo>/.git directories; src/content_hash.rs mirrors vibe-registry::compute_content_hash; vibe-index reindex --from-clones works against fixtures/sample-org/. Parity test against vibe-registry. Incremental mode = full for now (deferred to slice 7).

230Commits:

  • 231feat(services/vibe-index): content_hash parity with vibe-registry
  • feat(services/vibe-index): scanner — walk org-dir clones
  • feat(services/vibe-index): vibe-index reindex --from-clones

4.4 Slice 4 — read CLI subcommands

232get, list, search, capabilities, purls, outdated. Inverted text index for search. JSON output for every subcommand. cli_e2e.rs covers each.

233Commits:

  • 234feat(services/vibe-index): inverted text index for search
  • feat(services/vibe-index): vibe-index get/list/search/capabilities/purls
  • feat(services/vibe-index): vibe-index outdated against a vibe.lock

4.5 Slice 5 — HTTP server (read-only)

235vibe-index serve --read-only. axum app exposes /healthz, /readyz, /v1/index/*, /v1/packages*, /v1/capabilities/*, /v1/purls/*, /metrics. PID lock file. CORS open. server_e2e.rs covers each route.

236Commits:

  • 237feat(services/vibe-index): axum server skeleton + healthz/readyz
  • feat(services/vibe-index): GET /v1/index/* file routes
  • feat(services/vibe-index): GET /v1/packages query routes
  • feat(services/vibe-index): /metrics prometheus endpoint

238After slice 5: vibe-index is independently usable as a read-only server fed by reindex --from-clones. This is the "MVP" mark.

4.6 Slice 6 — write CLI + write HTTP + auth

239vibe-index add / vibe-index remove. HTTP POST /v1/packages, DELETE /v1/packages/.... Bearer-token auth via <data-dir>/state/admin.tokens. Write-side server-vs-CLI lock arbitration.

240Commits:

  • 241feat(services/vibe-index): vibe-index add/remove (CLI)
  • feat(services/vibe-index): bearer-token auth + admin.tokens loader
  • feat(services/vibe-index): POST/DELETE /v1/packages routes

4.7 Slice 7 — incremental reindex

242<data-dir>/state/checkpoint.json. vibe-index reindex --incremental --from-clones walks the diff between checkpoint and current state. Test: full vs incremental produce identical output.

243Commit: feat(services/vibe-index): incremental reindex via checkpoint.

4.8 Slice 8 — --from-github mode

244reqwest-based GitHub API walk. --token-file FILE. Rate-limit-aware backoff. Same shape as --from-clones from caller's POV.

245Commit: feat(services/vibe-index): reindex --from-github (REST API walk).

4.9 Slice 9 — vibe-publish post-publish hook

246crates/vibe-publish/src/lib.rs::Publisher::publish gains optional index POST after successful push. New env var VIBEVM_INDEX_TOKEN_<HOST>. New [[registry]].index_url / [[registry]].index_token fields in the project manifest.

247Commit: feat(vibe-publish): POST to registry index after publish (opt-in).

4.10 Slice 10 — consumer-side integration

248crates/vibe-registry/src/multi_registry_resolver.rs gains the index-aware fast path. Falls back transparently on 404 / connect-failure. Live e2e test against an index-equipped registry.

249Commit: feat(vibe-registry): consume registry index for resolve fast path (opt-in).

4.11 Slice 11 — docs + manual-test smoke

250crates/vibe-index/docs/ filled in (operator-handbook, consumer-protocol, format). manual-tests/M2.10-index-smoke.md walks the live e2e: bootstrap an index from a fresh org dir, serve it, install a package through it, search it.

251Commits:

  • 252docs(vibe-index): operator handbook + consumer protocol + format reference
  • test: manual-test smoke for index bootstrap + consume

5. Test plan

253Per slice (specifics in §4); cumulative state at GA:

  • 254Unit: every type round-trips through serde JSON / TOML; every CLI subcommand has at least one happy-path test; every server route has at least one happy-path + one auth-fail test.
  • Integration: the byte-identity check moved out of the crate's own fixtures and into the campaign's golden corpus — formats/corpora/index/e1/ holds a journal and the catalog it projects to, tests/golden_corpus.rs compares them, and vibe-index rebuild <data-dir> --check tears the catalog down and rebuilds it from the journal. The maintainer-side cargo xtask rebuild --check <data-dir> surface calls the same library implementation as a compatibility wrapper. Incremental reindex applied to the same starting state is still byte-identical to a full one. The move is the point: the golden now lives beside the format registry that governs it rather than beside one consumer of it.
  • Parity: tests/content_hash_parity.rs runs both implementations over both fixtures in both recipes (§3.2), and asserts equality within each recipe. CI gates the merge if they diverge.
  • End-to-end: tests/server_e2e.rs spawns the server in-process (axum's oneshot style), drives every documented route over HTTP, asserts response shapes.
  • Crash recovery: tests/persistence_atomic.rs simulates mid-write crash by failing the rename step; asserts the previous version remains readable.
  • Hermetic, with no live tier at all: every test runs without network, and the GitHub API walk is proved against a mock REST server on a random port whose canned responses point at local bare repositories, so git clone resolves entirely against the filesystem (tests/from_github_e2e.rs). The opt-in live run against the real registry this section once promised does not exist and is not owed: a test that needs the internet and a real organisation is one nobody runs, so it proves nothing on the day it would have mattered, while the mock proves the walk's shape on every commit. What it deliberately does not prove is that the real host still answers the way the mock does — that question belongs to the manual tier, not to an ignored test.

6. Distribution — essential member of the VibeVM bundle

255design r2

256Decision (revised 2026-05-22). vibe-index lives at crates/vibe-index/ as a member of the top-level vibevm workspace. It is built, tested, clippy-gated, and fmt-checked by the same cargo … --workspace invocations as every other crate, and it depends on vibe-core directly.

257Why this reverses the original standalone-workspace decision. The proposal first placed vibe-index in its own Cargo workspace under services/, outside crates/, so an org owner could vendor just that subdirectory. The cost was a hand-duplicated vibe.toml parser with nothing tying it to vibe-core — and, sitting outside cargo test --workspace, nothing routinely exercising it. It rotted silently against the M1.17 / M1.18 manifest-schema churn (§9 item 11). Folding the crate back in kills both failure modes at once: the scanner now parses through vibe-core::Manifest (one source of truth — the schema cannot drift), and the routine workspace gate covers it (drift is caught the moment it appears).

258Redistribution. vibe-index is an Essential component of every native VibeVM distribution, installed and switched atomically beside vibe; this is the default on-premises operator path. Its matching source is available through vibe self source. A developer may still build it from the workspace with cargo install --path crates/vibe-index, but consumers no longer need to clone the repository merely to operate an index. Keeping it in the workspace still prevents manifest-schema drift through its direct vibe-core dependency.

259Gate. tools/self-check.sh no longer special-cases a second workspace — the workspace-wide steps (fmt, then cargo test --workspace, then cargo clippy --workspace --all-targets -- -D warnings) cover vibe-index like any member.

260What has changed since is the opposite of a special case: the panel grew steps that look specifically at this crate, and they are the interesting half. A clock gate greps crates/vibe-index/src/{index,types,journal} for any call that reads the wall clock, because determinism here is an instrument rather than a preference (§2.9); check-codegen refuses a drift between the schemas and the generated types (§2.12); specmap --check refuses a stale traceability map; and the wire-derive ratchet refuses a hand-written wire. None of them is a workspace step, and none of them would fire from cargo test alone — which is why the number of steps in the panel is not a proxy for what it checks.

7. Auth, secrets, scope

261req r1

262PROP-000 §20 covers the token-secrecy invariant; PROP-005 inherits it verbatim. Specifically:

  • 263Server admin tokens (<data-dir>/state/admin.tokens) — file mode 0600, never echoed in logs / responses / error messages, gitignored.
  • GitHub API tokens (--from-github --token-file FILE) — same discipline. Read once into memory, scrubbed from the env, never persisted outside the source file.
  • Index POST tokens (VIBEVM_INDEX_TOKEN_<HOST> for the publish-side hook) — per-host shape mirrors VIBEVM_PUBLISH_TOKEN_<HOST>.

264Scope discipline. The server's mutation endpoints accept entries only for the registry the server was started with (<data-dir>/repomd.json::registry). A POST attempting to land an entry whose registry field disagrees with the server's configured registry → 400 with a clear message. Same shape vibe-publish::validate_scope enforces on the publish side.

8. Operations

265A typical setup for an org owner who wants to host an index:

266# One-time bootstrap (on a host with the org's clones available).
$ vibe-index init  ./vibespecs-index   --registry vibespecs   --registry-url https://github.com/vibespecs   --naming fqdn
$ vibe-index reindex ./vibespecs-index --from-clones  /var/lib/vibespecs-mirror

# Push the static files to <org>/index repo (operators wire this once):
$ cd ./vibespecs-index
$ git init && git remote add origin https://github.com/vibespecs/index
$ git add . && git commit -m "initial index"
$ git push -u origin main

# Run the live server (optional — only if hosting the HTTP-API path):
$ vibe-index serve ./vibespecs-index --bind 0.0.0.0:8412 \
    --auth-tokens-file ./vibespecs-index/state/admin.tokens

# Periodic incremental refresh (cron):
$ */5 * * * *  vibe-index reindex /home/owner/vibespecs-index --incremental --from-clones /var/lib/vibespecs-mirror

267Most consumers see only the static raw-HTTP files; the server is for orgs that need real-time publish updates.

9. Open questions

  1. 268Index file location: <org>/index repo vs <org>/<package-repo>/index/... per-package? PROP-005 picks <org>/index. The alternative — per-package files inside each package repo — was rejected because it leaves catalog discovery a chicken-and-egg problem (you need to enumerate the org first). If new evidence emerges that orgs object to a top-level index repo (naming conflicts, permission boundaries), we revisit.
  2. primary.jsonl.gz compression: gzip vs zstd? v0 picks gzip — universally supported by every HTTP client; deterministic with mtime=0. v1 may add a primary.jsonl.zst alongside.
  3. GPG signing of repomd.json? Tracked here, not shipped in v0. Shape: repomd.json.asc next to repomd.json; consumers verify against a per-registry public key recorded in [[registry]].pgp_key. v1.
  4. Merkle log (Go sumdb-style transparent log)? Tracked here. v2+. Useful for adversarial environments; v0/v1 trust the host.
  5. Auto-commit-and-push from server — ANSWERED and built 2026-08-06; the contract is §2.17. The risk this question named — the server holding push credentials is a step up in trust — is unchanged and is why the flag stays opt-in with manual commit/push as the default. What the build added to the answer is a second risk the question had not seen: the credentials are not the only secret in reach, because the data directory also holds the server's own bearer tokens, and the publishing step is a git add -A away from them. Hence the startup refusal in §2.17 rather than a warning.
  6. Multi-registry server — should one server instance host multiple data dirs (one per registry)? v0 says no (one process per registry). Trivial scale-out via process supervision; we revisit if multi-tenancy demand emerges.
  7. WebSockets / Server-Sent Events for live publish notifications — out of scope. Polling /v1/admin/status::last_reindex is sufficient at our scale.
  8. OCI registry shape — could we host the index inside an OCI registry instead of git? Out of scope; revisit if the OCI tooling becomes universal among vibevm operators.
  9. Capability- vs PURL-driven search — v0 ships by-cap and by-purl as separate files. If usage shows one dominates, the loser may be folded into the inverted text index. Empirical question.
  10. Rate-limiting on the server — shipped after the v0 plan: server/rate_limit.rs is a per-token and per-IP token-bucket limiter, disabled by default and opt-in by flag. Production deployments may still front it with a reverse proxy's limiter; the two compose.
  1. 269Standalone-workspace duplication — RESOLVED 2026-05-22 by folding the crate in. The 2026-05-22 de-rot found crates/vibe-index/ had silently rotted: its duplicated vibe.toml parser still expected the pre-M1.17 shape ([writes], [dependencies], [boot_snippet].filename) and could not parse a current manifest, and its content_hash parity test had drifted off a fixture renamed by the M1.17 manifest unification. §3.2 had weighed the duplication cost for compute_content_hash alone ("the algorithm doesn't change"); the manifest schema, by contrast, churned hard through M1.17 / M1.18, and the duplicate parser had no cross-check to catch the drift — the standalone workspace also sat outside the routine cargo test --workspace gate. Resolution: fold vibe-index into the crates/ workspace and parse through vibe-core::Manifest (see §6). The duplicated parser is deleted; only the tiny, schema-frozen PackageKind / NamingConvention and the compute_content_hash algorithm remain duplicated, both justified in §3.2.

10. Acceptance criteria

270A given slice is considered accepted when:

  • 271All tests in its slice pass.
  • cargo clippy --workspace --all-targets -- -D warnings is clean.
  • cargo fmt --check is clean.
  • tools/self-check.sh is green.
  • Help-text smoke covers any new subcommand.
  • A manual walk through crates/vibe-index/docs/operator-handbook.md succeeds against fixtures/sample-org/.

272PROP-005 is considered closed once slices 1–8 land. Slices 9–11 are integration with the rest of vibevm and ship under their respective milestone PRs.

11. Wire-up scripts (informational, not shipped)

273For operators wiring the index into their hosting:

274git post-receive hook on the org's hosted git (Forgejo/Gitea/GitVerse-style) — push to a package repo triggers an incremental CLI reindex on the index host (the HTTP trigger was withdrawn — ##TRIGGER-HTTP; the hook therefore invokes the operator verb, over ssh when the index lives on another machine, directly when it is local):

275#!/bin/sh
# /var/git/<org>/<repo>.git/hooks/post-receive
# Runs on the git host. Same machine as the index: drop the ssh wrapper
# and call the verb directly.
while read oldrev newrev refname; do
    case "$refname" in
        refs/tags/v*)
            ssh index-host 'vibe-index reindex /var/lib/vibespecs-index \
                --incremental --from-clones /var/lib/vibespecs-mirror' \
                >>/var/log/vibe-index-hook.log 2>&1 \
                || echo "vibe-index reindex failed (non-fatal)"
            ;;
    esac
done

276cron line:

277*/5 * * * *  vibe-index reindex /home/owner/vibespecs-index --incremental --from-clones /var/lib/vibespecs-mirror >>/var/log/vibe-index.log 2>&1

278Neither is shipped as a binary — operators integrate at their own host, and the hook shape varies enough across hosting platforms that one-size-fits-all isn't worth shipping.

279Of the two, only the cron line is in crates/vibe-index/docs/operator-handbook.md; the post-receive hook is documented here and nowhere else. Worth keeping stated: the hook's earlier form posted to POST /v1/admin/reindex — a route that never shipped and was withdrawn 2026-08-20 (##TRIGGER-HTTP) — so the artefact that did not reach the handbook was exactly the one that would not have worked from it. The recipe above now invokes the CLI verb; copying it into the handbook becomes legitimate the day an operator asks for it.

12. Version history

  • 2802026-05-06 — draft 1. Initial proposal. Open for review.
  • 2026-05-22 — reconciled with the implementation, then folded into the workspace. A state review found PROP-005 already implemented (slices 1–10 + M2.10 vibe search) but rotted; the de-rot realigned the scanner with the current vibe.toml schema and corrected this document (§2.6 boot_snippet, the vibe.toml filename, §2.10 rate-limiter status). The fold then moved vibe-index from its own services/ workspace into crates/vibe-index/ and switched it to parse through vibe-core::Manifest — §3.2, §6, and §9 item 11 are revised for the reversed standalone-workspace decision.
  • 2026-05-22 — group-native (PROP-008 Phase 7). The index entry gained the mandatory group field and the optional workspace_origin (§2.6); the by-name/ layer was re-keyed from by-name/<kind>/<name>.json to the candidate-set file by-name/<name>.json (§2.4) — one GET per registry now yields every group sharing a bare name; primary.jsonl / by-cap / by-purl sort on the (group, name, version) identity; the HTTP /v1/packages/{group}/{name} routes, the vibe-index get/remove CLI, and the naming = "fqdn" default followed. The vibe-registry index client and the vibe-publish post-publish hook were realigned to the new shape. PROP-008 §2.8's index extension is shipped.

For an agent

This page has a machine mirror. The citation carries the version rather than latest, so what an agent quotes does not move under it.

spec://org.vibevm.core/vibevm@1.0.0/modules/vibe-index/PROP-005-package-index

.md.xmlllms.txt