diff --git a/docs/superpowers/plans/2026-10-24-house-encyclopedia.md b/docs/superpowers/plans/2026-10-24-house-encyclopedia.md new file mode 100644 index 0000000..3fa286a --- /dev/null +++ b/docs/superpowers/plans/2026-10-24-house-encyclopedia.md @@ -0,0 +1,792 @@ +# House Encyclopedia Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** The Wiki service generates bilingual (de/en) Wikipedia-style articles from its notes. It commits them to the notes repo and serves them at `wiki.renner.uno/house`: public parts are open to anonymous readers, and private parts are visible only to signed-in wiki-group members. + +**Architecture:** +- A new `Article` document kind lives under `articles//.md` in the notes repo and is indexed by the existing sync. +- Once a night (06:00 Europe/Berlin, after the 04:30 distill), an encyclopedia reconciler syncs, then regenerates only stale articles and commits all changes in one multi-file Gitea commit. An article is stale when a source hash changed, it is missing, or its subject was removed. Sync and the webhook do not trigger it. + - The reconciler runs only when the catalog is at the store's head. It first plans the article set, stored in `articles/_plan.json`. + - For each stale article it then runs: write German, apply the deterministic leak filter, run the LLM leak check, translate to English block by block, and filter and leak-check the English again. + - Attempt memory in `articles/_state.json` bounds cost: a rejected subject is retried only when its inputs change, and a failed plan is not retried for the same note set. + - The reconciler never stops the host: every failure except cancellation is recorded for `/status` and the run returns. +- Article pages are static-SSR Razor components with their own Wikipedia-like layout and `[AllowAnonymous]`. Private blocks are removed server-side for anonymous readers. +- Group gating happens in Authentik: a policy binding on the `wiki` OIDC application. + +**Tech Stack:** .NET 10, ASP.NET Core minimal APIs, Blazor (static SSR for article pages), Markdig 1.4.0, MudBlazor 9.10.0 (existing pages only), raw `HttpClient` against the Anthropic Messages API, the Gitea contents API, xunit 2.9.3 with hand-written fakes, Mermaid (vendored JS). + +**Spec:** `docs/superpowers/specs/2026-10-24-house-encyclopedia-design.md`. Read it before starting any task. + +## Global Constraints + +- Repo `konairius/Wiki`: `TreatWarningsAsErrors=true`, `Nullable` enabled, primary-constructor sealed classes, `CancellationToken ct` last, no mocking library, hand-written fakes only. +- Verification for every task: `dotnet build Wiki.slnx -c Release` then `dotnet test Wiki.slnx -c Release --logger "console;verbosity=minimal"`. Both must be green. +- Notes repo is `konairius/notes`, branch `main`. Articles live at `articles//.md`, where `` is `de` or `en`. The plan file is `articles/_plan.json` and the attempt memory is `articles/_state.json`; `Classify` ignores both because of the `_` prefix. +- Article slugs are English kebab-case and are the same in both languages. Titles are localized. +- Private content is never present in HTML served to an anonymous reader. It is removed server-side, never hidden by CSS. +- `wiki.renner.uno` stays off the internet: its Ingress carries neither `dns.renner.uno/public` nor `netbird.renner.uno/public`. +- Generic examples that name nothing concrete in this house stay public. +- When in doubt, content is private. This covers an unterminated private block, a failed leak check, unparseable model output, and a failed filter. +- Public article pages send `` and the response headers `X-Robots-Tag: noindex` and `Cache-Control: no-store`. +- An anonymous reader never sees a private article's title, a link to it, or a listing entry for it. An article is private when either language's file says so; the writer always gives both languages the same visibility. +- No CDN at runtime: Mermaid is vendored under `wwwroot/lib/mermaid/`. +- Article generation reuses the Anthropic key at `Wiki:AnswerModels:Models:claude-sonnet:ApiKey`. No new secret is added. +- Spend limits are enforced on the API side. On a quota or overload error, stop the run, keep everything generated so far, and retry on the next nightly run. Any other Anthropic HTTP 400 is a rejection of that subject, remembered like any other rejection, not a transient. +- Commits go to branch `feat/house-encyclopedia` with author `Claude Bot `, via a PR. Never push to `main` directly. + +## Review Focus + +1. **Unterminated `::: private` block.** An article body opens a private block and never closes it. Everything after the opener must be treated as private (Task 2: `RedactPrivate_UnterminatedBlock_DropsRest`). +2. **The translator moves a private boundary.** A translated unit that contains a `:::` or fence line would close a private block early in English. The model never sees or controls block boundaries; such a reply is rejected (Task 7: `Translate_UnitWithStructuralLine_Throws`, `Translate_PrivateBlockStaysPrivate`). +3. **A private subject leaks by name.** For an anonymous reader, links, infobox entries and listings that point at a private article are dropped with their text (Task 10: `Render_Anonymous_LinkToInvisibleSlug_DroppedWithText`; Task 11: `PrivateArticle_Anonymous_ShowsNoTitle`, `PublicArticle_Anonymous_NoPrivateArticleTitle`). +4. **Model returns non-JSON or truncated output** from planning, the leak check or the translator. The article is not published and the previous version stays (Task 7: `Write_LeakCheckUnparseable_Throws`, `Translate_WrongBlockCount_Throws`). Planning: the old plan stays (Task 6: `Plan_ModelGarbage_KeepsExistingPlan`). +5. **A reconciler exception stops the host.** An unhandled exception in a `BackgroundService` stops the whole wiki. A missing Anthropic key, a JSON error or a Gitea error is recorded on `/status` and the service keeps running (Task 9: `Service_ReconcilerThrows_KeepsRunning`). +6. **A missing or shrunken plan deletes articles.** Without a usable plan nothing is written; deletes happen only for subjects the previous plan had and the new plan dropped; the pilot never deletes (Task 8: `Reconcile_NoPlan_WritesNothing`, `Reconcile_PilotCapShrinks_DeletesNothing`, `Reconcile_SubjectDropped_DeletesBothLanguages`). +7. **Paying again for the same failure.** A rejected subject or a failed plan would otherwise be retried every night on unchanged input, and a non-idempotent run would rewrite everything. A rerun on unchanged input makes no model call and no commit (Task 8: `Reconcile_RejectedSameInputs_NotRetried`, `Reconcile_PlanFailedSameNotes_DoesNotReplan`, `Reconcile_AfterOwnCommit_IsNoop`; Task 9: `Service_RunsOncePerDayAtRunAt`). + +--- + +### Task 1: `Article` document kind + +**Files:** +- Modify: `src/Wiki.Core/Model/Document.cs` (enum `DocumentKind`) +- Modify: `src/Wiki.Core/Parsing/DocumentParser.cs` (`Classify`, `PathFor`) +- Modify: `src/Wiki.Application/Sync/SyncUseCase.cs` (`AllKinds`) +- Modify: `src/Wiki.Adapters.Gitea/GiteaRepository.cs` (`ListAsync` kind filter, if it filters by path) +- Test: `test/Wiki.Core.Tests/DocumentParserTests.cs` (extend; create it if absent) + +**Interfaces:** +- Produces: `DocumentKind.Article`. + - `DocumentParser.Classify("articles/de/wall-switches.md") == (DocumentKind.Article, "de/wall-switches")`. + - `DocumentParser.PathFor(DocumentKind.Article, "en/heating") == "articles/en/heating.md"`. + - Article slugs in the catalog are `/`. + +- [ ] **Step 1: Write failing tests** + +```csharp +[Theory] +[InlineData("articles/de/wall-switches.md", "de/wall-switches")] +[InlineData("articles/en/heating.md", "en/heating")] +public void Classify_Article(string path, string slug) + => Assert.Equal((DocumentKind.Article, slug), DocumentParser.Classify(path)); + +[Theory] +[InlineData("articles/_plan.json")] +[InlineData("articles/_state.json")] +[InlineData("articles/fr/x.md")] // only de|en +[InlineData("articles/de/_draft.md")] +[InlineData("articles/de/sub/x.md")] // no nesting +[InlineData("articles/x.md")] +public void Classify_ArticleRejects(string path) => Assert.Null(DocumentParser.Classify(path)); + +[Fact] +public void PathFor_Article() => Assert.Equal("articles/en/heating.md", DocumentParser.PathFor(DocumentKind.Article, "en/heating")); +``` + +Add a `SyncUseCaseTests` case named `Sync_IndexesArticles_ButDoesNotEmbedThem`. The `FakeStore` holds one `wiki/` note and one `articles/de/x.md`. After `RunAsync`, the catalog contains both, and `FakeEmbeddings` was called only for the wiki note. + +- [ ] **Step 2: Run the tests.** `dotnet test Wiki.slnx --filter "FullyQualifiedName~DocumentParserTests|FullyQualifiedName~SyncUseCaseTests"`. Expected: FAIL (`Article` is not defined). +- [ ] **Step 3: Implement.** Add the enum member, the `Classify` branch (exactly two path segments after `articles/`, language `de|en`, the existing `_`-prefix rule), the `PathFor` case, and `Article` in `AllKinds`. Keep embedding restricted to `DocumentKind.Wiki`. If `GiteaRepository.ListAsync` maps paths through `Classify`, no change is needed there; verify it. +- [ ] **Step 4: Run the full suite.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(core): Article document kind under articles//`. + +--- + +### Task 2: Article format: frontmatter, private blocks, redaction + +**Files:** +- Create: `src/Wiki.Core/Model/Article.cs` +- Create: `src/Wiki.Core/Parsing/ArticleBody.cs` +- Create: `src/Wiki.Core/Parsing/ArticleFile.cs` +- Test: `test/Wiki.Core.Tests/ArticleBodyTests.cs`, `test/Wiki.Core.Tests/ArticleFileTests.cs` + +**Interfaces:** +- Produces: + ```csharp + public enum ArticleVisibility { Public, Private } + public enum ArticleCategory { Room, System, Device, Topic } + public sealed record ArticleSource(string Slug, string ContentHash); + public sealed record Article(string Lang, string Slug, string Title, ArticleCategory Category, + ArticleVisibility Visibility, IReadOnlyList Sources, DateOnly Generated, + string Model, string Body); + public sealed record BodyBlock(int Index, string Text); + + public static class ArticleBody + { + public const string PrivateOpen = "::: private"; + public const string Close = ":::"; + public static string RedactPrivate(string body); // anonymous view + public static IReadOnlyList PublicBlocks(string body); // outside private blocks & code fences + public static string Privatize(string body, IReadOnlySet blockIndexes); // wrap those public blocks in ::: private + // translation (Task 7) + public static IReadOnlyList TranslationUnits(string body); + public static string Reassemble(string body, IReadOnlyList translated); // throws FormatException + public static IReadOnlyList Skeleton(string body); // structural lines in order + } + public sealed record TranslationUnit(int Index, bool Private, string Text); + public static class ArticleFile + { + public static string Build(Article article); // frontmatter + body + public static Article Parse(Document document); // from catalog Document of kind Article + } + ``` +- Body conventions, which the Task 7 prompts must teach the writer: + - `::: infobox` … `:::` holds a two-column pipe table. + - `::: private` … `:::` marks a private block. + - Mermaid diagrams go in ```` ```mermaid ```` fences. + - Links to other articles use `[[slug]]`. + - Nesting is not allowed. +- Translation units, which keep private boundaries out of the model's hands: + - Structural lines are `::: private`, `::: infobox`, `:::` and code-fence lines. They are never sent to the translator. + - A unit is a maximal run of non-blank, non-structural lines. The whole body of a ```` ```mermaid ```` fence is one unit, because its labels need translating. The body of any other code fence is not a unit and is copied verbatim. + - `Private` is true for units inside a `::: private` block, and for everything after an unterminated opener, the same rule as `RedactPrivate`. + - `Reassemble` replaces unit `i` with `translated[i]` and copies every structural line and blank line from `body`. It throws `FormatException` when the count differs from the unit count, when a translated text contains a structural line, when a translated non-mermaid unit contains a blank line (it would split a block and shift the numbering), or when a translated text's multiset of `[[…]]` links differs from its unit's. + - `Skeleton` returns the structural lines in order. Two bodies with equal skeletons have the same private blocks and the same `PublicBlocks` numbering. +- Frontmatter keys: + - `title`, `lang`, `subject` (the slug without language), `category` (lowercase), `visibility` (lowercase) + - `sources`: a YAML list of `slug@contenthash` + - `generated` (`yyyy-MM-dd`), `model` + +- [ ] **Step 1: Write failing tests.** + - `RedactPrivate_RemovesBlock_KeepsLockedMarker`: given `"A\n\n::: private\nsecret\n:::\n\nB"`, the result contains `A`, `B` and `::: private-locked`, and does not contain `secret`. + - `RedactPrivate_UnterminatedBlock_DropsRest`: given `"A\n\n::: private\nsecret\n\nmore"`, the result does not contain `secret` or `more`. + - `RedactPrivate_IgnoresFenceInsideCode`: a `::: private` line inside a ```` ``` ```` code fence is not treated as an opener. + - `PublicBlocks_SkipsPrivateAndNumbersSequentially`: on `"A\n\n::: private\nx\n:::\n\nB"` the indexes are `[0:"A", 1:"B"]`. + - `PublicBlocks_InfoboxIsOneBlock`: an infobox container counts as one block. + - `Privatize_WrapsChosenBlocks`: `Privatize(body, {1})` wraps only `B`. Applying `PublicBlocks` again returns only `A`. + - `TranslationUnits_MarksPrivateUnits`: on `"A\n\n::: private\nx\n:::\n\nB"` the units are `[0:A public, 1:x private, 2:B public]`. + - `TranslationUnits_MermaidFenceIsOneUnit_CodeFenceIsNot`. + - `TranslationUnits_UnterminatedPrivate_RestIsPrivate`. + - `Reassemble_KeepsSkeleton`: `Skeleton(Reassemble(body, texts)) == Skeleton(body)`. + - `Reassemble_WrongCount_Throws`. + - `Reassemble_UnitWithStructuralLine_Throws`: a translated private unit `"x\n:::\nleak"` throws `FormatException`. + - `Reassemble_LinkSetChanged_Throws`: a unit with `[[heating]]` translated without it throws. + - `ArticleFile_RoundTrip`: `Build`, then `DocumentParser.Parse`, then `ArticleFile.Parse`, gives an equal `Article`, including two sources `schalter-konvention-blueprint-system@abc123` and `heizung@def456`. +- [ ] **Step 2: Run them.** Expected: FAIL (the types are not defined). +- [ ] **Step 3: Implement.** `RedactPrivate` replaces each private block with the two lines `::: private-locked` and `:::`. A line-based scanner tracks code-fence state. Build the frontmatter by hand with a `StringBuilder`, following the `DepositFactUseCase.BuildNote` style, and reuse its YAML scalar quoting rules. +- [ ] **Step 4: Run the suite.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(core): article body format and private-block redaction`. + +--- + +### Task 3: Deterministic leak filter + +**Files:** +- Create: `src/Wiki.Core/Parsing/LeakScanner.cs` +- Test: `test/Wiki.Core.Tests/LeakScannerTests.cs` + +**Interfaces:** +- Consumes: `ArticleBody.PublicBlocks` and `ArticleBody.Privatize` (Task 2). +- Produces: + ```csharp + public sealed record LeakMatch(int BlockIndex, string Rule, string Value); + public static class LeakScanner + { + public static IReadOnlyList Scan(string body, IReadOnlyList sources); // public blocks only + public static string PrivatizeLeaks(string body, IReadOnlyList sources); // Privatize(body, blocks with any match) + } + ``` +- Only this house's identifiers count, so generic examples stay public (spec: "Generic examples stay public"): + - a candidate counts as a match when its value occurs, case-insensitively, in any of `sources` (the full texts of the article's source notes); + - a name ending in `renner.uno` always counts, sources or not; + - reserved example values never count, even when they occur in sources: `192.0.2.0/24`, `198.51.100.0/24`, `203.0.113.0/24`, `2001:db8::/32`, and `example.com`, `example.org`, `example.net` or any name under `.example`. +- Rules (rule names are used in the tests): + - `ipv4` + - `ipv6`, matching at least two `:` groups of hex digits, e.g. `fd00::1` + - `mac` + - `fqdn`: any `label.label.tld` with at least 2 dots, or any name ending in `.renner.uno`, `.local`, `.lan`, `.internal`, or `.svc.cluster.local` + - `entity-id`: `(light|switch|sensor|binary_sensor|cover|climate|script|automation|scene|input_\w+|button|number|select|fan|media_player|person|device_tracker|lock|alarm_control_panel|camera|vacuum|water_heater|update)\.[a-z0-9_]+` + - `path`: tokens starting with `/`, `~/` or `./`, containing a further `/` + - `secret-path`: `apps//`, `secret/…`, `kv/…`, `vault_[a-z_]+` + +- [ ] **Step 1: Write failing tests.** Use a `[Theory]` with one `InlineData` per rule: + - `ipv4`: `"Der Server 10.2.40.1 antwortet"` + - `ipv6`: `"fd00::1"` + - `mac`: `"aa:bb:cc:dd:ee:ff"` + - `fqdn`: `"wiki.renner.uno"`, `"ak-outpost.authentik.svc.cluster.local"` + - `entity-id`: `"light.eg_kuche_ambiente"` + - `path`: `"/var/run/secrets/x"` + - `secret-path`: `"apps/ai/wiki-anthropic"` + + Each case passes its value in `sources` and yields that rule name. Then: + - `Scan_IgnoresPrivateBlocks`: the same string inside `::: private` yields no match. + - `Scan_GenericExampleNotInSources_NoMatch`: `"Eine Entität heißt zum Beispiel light.wohnzimmer"` with sources not containing `light.wohnzimmer` yields no match. + - `Scan_MatchIsCaseInsensitive`: body `LIGHT.EG_KUCHE_AMBIENTE`, source `light.eg_kuche_ambiente`, yields `entity-id`. + - `Scan_ReservedExample_NoMatch`: `"192.0.2.10"`, `"2001:db8::1"` and `"nas.example.com"` yield no match, even when present in sources. + - `Scan_RennerUno_AlwaysMatches`: `"gitea.renner.uno"` with empty sources yields `fqdn`. + - `Scan_NoFalsePositive_OnPlainProse`: `"Die Wärmepumpe heizt den Estrich. Version 2.5 ist aktuell."` yields no match, so a version number is not an IPv4 address. + - `PrivatizeLeaks_MovesOnlyLeakingBlock`. +- [ ] **Step 2: Run them.** Expected: FAIL. +- [ ] **Step 3: Implement** the rules as `[GeneratedRegex]` partial methods. Inline code spans are scanned too, because a hostname in backticks is still a leak. +- [ ] **Step 4: Run them.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(core): deterministic leak scanner for public article text`. + +--- + +### Task 4: Note store: multi-file commit, text and binary read by path + +**Files:** +- Modify: `src/Wiki.Core/Ports/Ports.cs` (`INoteStore`) +- Modify: `src/Wiki.Core/Model/Retrieval.cs` (new records) +- Modify: `src/Wiki.Adapters.Gitea/GiteaRepository.cs` +- Modify: `test/Wiki.Application.Tests/Fakes.cs`, `test/Wiki.Host.Tests/Fakes.cs` (`FakeStore`) +- Modify: `test/Wiki.Contracts/Gitea/FakeGiteaHandler.cs`, `test/Wiki.Contracts/NoteStoreContract.cs` + +**Interfaces:** +- Produces: + ```csharp + public enum FileChangeOp { Create, Update, Delete } + public sealed record FileChange(FileChangeOp Op, string Path, string? Text, string? BlobSha); + public sealed record StoredText(string Text, string BlobSha); + public sealed class StoreConflictException(string message) : Exception(message); + // INoteStore additions: + Task CommitAsync(IReadOnlyList changes, string message, CancellationToken ct); // the new commit sha + Task ReadTextAsync(string path, string commit, CancellationToken ct); // null on 404 + Task ReadBytesAsync(string path, string commit, CancellationToken ct); + ``` + - `BlobSha` is required for `Update` and `Delete`. For articles it is the `ContentHash` from `ListAsync`, which is the Gitea blob SHA. For files outside the catalog (`articles/_plan.json`, `articles/_state.json`) it is `StoredText.BlobSha`. + - `ReadTextAsync` exists because `ReadAsync` takes a `DocumentRef`, which `_`-prefixed files never have. It replaces a separate blob-SHA lookup: one call returns both. + - `CommitAsync` must produce exactly one commit. +- Gitea mapping: + - `POST api/v1/repos/{repo}/contents` with body `{ branch, message, files: [{ operation: "create"|"update"|"delete", path, content (base64), sha }] }`. The returned sha is the response's `commit.sha`. + - HTTP 409 or 422 throws `StoreConflictException`. + - `ReadTextAsync` uses `GET contents/{path}?ref=commit` and returns the base64-decoded `.content` with `.sha`. A 404 returns null. + - `ReadBytesAsync` uses `raw/{path}?ref=commit`. +- `FakeStore.CommitAsync` applies the changes, advances `Head` to a new value (`c2`, `c3`, …) and returns it, so a sync after a fake commit sees a new head. The fake's blob SHA is the same SHA-1 of the text that `ListAsync` reports. + +- [ ] **Step 1: Write failing contract tests** in `NoteStoreContract`: + - `Commit_CreatesUpdatesDeletesInOneCommit`: after the call, `GetHeadAsync` advanced once and equals the returned sha, and `ListAsync` shows the create, the update with new content, and the delete gone. + - `Commit_StaleSha_ThrowsConflict`. + - `ReadText_ReturnsTextAndUsableSha`: read `articles/_plan.json`, then an `Update` with that `BlobSha` succeeds. + - `ReadText_MissingFile_IsNull`. + - `ReadBytes_ReturnsExactBytes`, using a 3-byte PNG header. +- [ ] **Step 2: Run them.** Expected: FAIL (compile errors in the fakes). +- [ ] **Step 3: Implement** the port members, the Gitea adapter, `FakeGiteaHandler` support for the batch contents endpoint and the contents GET, and both `FakeStore`s. The `FakeStore`s record a `Commits` list of `(message, changes)`. +- [ ] **Step 4: Run the suite.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(store): multi-file commit, text and binary read by path`. + +--- + +### Task 5: Text model port and Anthropic adapter + +**Files:** +- Modify: `src/Wiki.Core/Ports/Ports.cs` +- Create: `src/Wiki.Core/Model/TextModel.cs` +- Create: `src/Wiki.Adapters.Anthropic/AnthropicTextModel.cs` +- Modify: `src/Wiki.Adapters.Anthropic/MessagesTransport.cs`, plus the registration extensions file in the same project +- Test: `test/Wiki.Contracts/Anthropic/AnthropicTextModelTests.cs`, reusing the existing fake handler + +**Interfaces:** +- Produces: + ```csharp + public sealed record TextReply(string Text, string ModelId, long InputTokens, long OutputTokens, bool Truncated); + public sealed class ModelUnavailableException(string message, Exception? inner = null) : Exception(message, inner); + public interface ITextModel : IAdapterHealth + { + string ModelId { get; } + Task CompleteAsync(string system, string user, int maxTokens, CancellationToken ct); + } + // registration + public static IServiceCollection AddAnthropicTextModel(this IServiceCollection services, Action configure); + ``` + - `Truncated` is true when the stream reports `stop_reason == "max_tokens"`. + - The transport throws `ModelUnavailableException` in three cases: + - 429 or 529 after `MaxRetries`; + - a stream `error` event of type `rate_limit_error` or `overloaded_error`; + - HTTP 400 whose error message mentions credit, billing or a usage limit. This covers API-side spend limits. + + All other failures keep throwing `HttpRequestException`, now constructed with its `StatusCode` set, so Task 7 can tell a 400 (a rejection) from a 5xx or network error (a transient). + - A missing API key keeps throwing `InvalidOperationException` (`MessagesTransport.PostWithRetryAsync`). It is not caught by the use cases; Task 9's coordinator records it. + - Existing answer and fact adapters inherit the new exception type. `ModelUnavailableException` must map to 502 in the host, the same as `HttpRequestException`. + +- [ ] **Step 1: Write failing tests.** + - `Complete_AccumulatesStreamedText`. + - `Complete_MaxTokensStop_SetsTruncated`. + - `Complete_429AfterRetries_ThrowsModelUnavailable`, using an injected no-op `delay`. + - `Complete_CreditError400_ThrowsModelUnavailable`. + - `Complete_Other400_ThrowsHttpRequestWithStatus`: `StatusCode == HttpStatusCode.BadRequest`. + - `Complete_PassesMaxTokensAndSystem`: assert the request JSON has `max_tokens` 16000 and the `system` string. +- [ ] **Step 2: Run them.** Expected: FAIL. +- [ ] **Step 3: Implement.** Add an overload `MessagesTransport.StreamAsync(..., int maxTokens, ...)` and keep the existing callers unchanged. `AnthropicTextModel.Name` is `anthropic-text:{ModelId}`. +- [ ] **Step 4: Run the suite.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(anthropic): long-form text model port with typed unavailability`. + +--- + +### Task 6: Article-set planning + +**Files:** +- Create: `src/Wiki.Application/Encyclopedia/ArticlePlan.cs` +- Create: `src/Wiki.Application/Encyclopedia/PlanArticlesUseCase.cs` +- Create: `src/Wiki.Application/Encyclopedia/EncyclopediaState.cs` +- Create: `src/Wiki.Application/Encyclopedia/Prompts.cs` (all prompt texts, as `const string`, plus `const int Version`) +- Test: `test/Wiki.Application.Tests/PlanArticlesUseCaseTests.cs`, `test/Wiki.Application.Tests/EncyclopediaStateTests.cs`, with a `FakeTextModel` added to `Fakes.cs` + +**Interfaces:** +- Consumes: `ITextModel` (Task 5), `IDocumentCatalog.ListDocumentsAsync(DocumentKind.Wiki, ct)`. +- Produces: + ```csharp + public sealed record PlannedArticle(string Slug, string TitleDe, string TitleEn, ArticleCategory Category, IReadOnlyList Sources, IReadOnlyList Related); + public sealed record ArticlePlan(IReadOnlyList Articles, IReadOnlyList CoveredNotes); // CoveredNotes = all wiki slugs seen when planned + public static class ArticlePlanJson { public static string Write(ArticlePlan p); public static ArticlePlan? Read(string json); } // Read: null on malformed JSON, never throws + public sealed class PlanArticlesUseCase(ITextModel model, EncyclopediaSettings settings, ILogger log) + { + public static bool NeedsReplan(ArticlePlan? current, PlanAttempt? failedPlan, IReadOnlyList notes); + public async Task PlanAsync(ArticlePlan? current, IReadOnlyList notes, CancellationToken ct); // null = keep current + } + public sealed record EncyclopediaSettings(int PlanMaxTokens, int ArticleMaxTokens, int MaxArticlesPerRun, + int MaxSourceCharsPerArticle, IReadOnlyList PilotSlugs, int? MaxArticles); + + // attempt memory, persisted as articles/_state.json + public enum AttemptOutcome { Written, Rejected } + public sealed record SubjectAttempt(string Fingerprint, AttemptOutcome Outcome, DateTimeOffset At); + public sealed record PlanAttempt(string NoteSetFingerprint, DateTimeOffset At); + public sealed record EncyclopediaState(IReadOnlyDictionary Subjects, PlanAttempt? FailedPlan) + { + public static EncyclopediaState Empty { get; } + public static string NoteSetFingerprint(IReadOnlyList notes); // SHA-256 over Prompts.Version + sorted wiki slugs + public static string SubjectFingerprint(PlannedArticle planned, IReadOnlyList notes); // SHA-256 over Prompts.Version, titles, category, sorted slug@ContentHash of its sources + } + public static class EncyclopediaStateJson { public static string Write(EncyclopediaState s); public static EncyclopediaState Read(string json); } // Read: Empty on malformed JSON + ``` + - `EncyclopediaSettings` lives in Application so the use cases don't depend on Host options. Task 9 maps it. + - `NeedsReplan` is true when `current` is null or its `CoveredNotes` differ from the note slugs, **and** `failedPlan` is not for the same `NoteSetFingerprint`. A failed plan is therefore retried only when notes are added or removed, or `Prompts.Version` changes. + - `Prompts.Version` is bumped by hand whenever a prompt changes, so a prompt fix retries earlier rejections. +- Prompt contract (`Prompts.Plan`): + - Input: the current plan's JSON, if any. Then one line per note: `slug | title | tags | body length in chars`. + - Instruction: the Wikipedia notability rule from the spec. A subject gets its own article when there is enough material, e.g. many automations or a complicated history. Otherwise its notes go into the parent system's article as a section. + - Instruction: every note is used by at least one article or deliberately skipped (meta and wiki-tooling notes). + - Instruction: keep the existing slugs for existing subjects. + - Instruction: list each article's sources most important first. When an article's sources add up to more than `MaxSourceCharsPerArticle` characters of body, split the subject into narrower articles. + - Reply: JSON only, in the shape `{"articles":[{"slug","titleDe","titleEn","category":"room|system|device|topic","sources":[...],"related":[...]}]}`. +- Validation after parsing: + - drop sources that are not known note slugs; + - drop articles with zero sources; + - de-duplicate slugs; + - slugs must match `^[a-z0-9]+(-[a-z0-9]+)*$`; + - source cap: keep sources in the listed order while their summed body length stays within `MaxSourceCharsPerArticle` (default 300 000, about 100k tokens). The first source is always kept. Drop the rest, the lowest-ranked, and log a warning with the slug and the dropped count. + - The plan is never truncated by the pilot settings. `PilotSlugs` and `MaxArticles` limit what Task 8 writes, not what is planned. +- Failure results: garbage, truncated, zero valid articles, or an `HttpRequestException` with status 400 return null (logged). `ModelUnavailableException` and any other exception propagate to the reconciler. + +- [ ] **Step 1: Write failing tests.** + - `NeedsReplan_NullPlan_True`. + - `NeedsReplan_SameSlugs_False`. + - `NeedsReplan_NoteAdded_True`. + - `NeedsReplan_FailedForSameNoteSet_False`: `current` null, `failedPlan` has the notes' fingerprint. + - `NeedsReplan_FailedForOtherNoteSet_True`. + - `Plan_ParsesAndValidates`: the fake returns JSON containing one unknown source and one bad slug. Both are dropped. + - `Plan_ModelGarbage_KeepsExistingPlan`: the fake returns `"Sure! Here is"`, and `PlanAsync` returns null. + - `Plan_Truncated_KeepsExistingPlan`: the reply has `Truncated=true`, and `PlanAsync` returns null. + - `Plan_Http400_ReturnsNull`. + - `Plan_ModelUnavailable_Propagates`. + - `Plan_SourcesOverCharCap_DropsLowestRanked`: three sources of 200k chars each with the cap at 300k keep only the first. + - `Plan_PromptContainsExistingSlugs`: assert the user prompt sent to the fake contains the current plan's slugs. + - `ArticlePlanJson_Malformed_ReadsNull`. + - `EncyclopediaStateJson_RoundTrip` and `EncyclopediaStateJson_Malformed_ReadsEmpty`. + - `SubjectFingerprint_ChangesWithSourceHashOrTitle`: equal for identical inputs, different when one source's `ContentHash` or `TitleDe` changes. +- [ ] **Step 2: Run them.** Expected: FAIL. +- [ ] **Step 3: Implement.** Use `System.Text.Json`. Strip a ```` ```json ```` fence from the reply if one is present. +- [ ] **Step 4: Run them.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(encyclopedia): article-set planning and attempt memory`. + +--- + +### Task 7: Article writing pipeline + +**Files:** +- Create: `src/Wiki.Application/Encyclopedia/WriteArticleUseCase.cs` +- Modify: `src/Wiki.Application/Encyclopedia/Prompts.cs` (`Write`, `LeakCheck`, `Translate`) +- Test: `test/Wiki.Application.Tests/WriteArticleUseCaseTests.cs` + +**Interfaces:** +- Consumes: + - `ITextModel`; + - `LeakScanner.Scan`, `ArticleBody.PublicBlocks`, `ArticleBody.Privatize`; + - `ArticleBody.TranslationUnits`, `Reassemble`, `Skeleton`; + - `ArticleFile.Build`; + - `PlannedArticle`; + - source `Document`s; + - `TimeProvider`. +- Produces: + ```csharp + public sealed record WrittenArticle(Article De, Article En); + public sealed class WriteArticleUseCase(ITextModel model, EncyclopediaSettings settings, TimeProvider clock) + { + public async Task WriteAsync(PlannedArticle planned, IReadOnlyList sources, + IReadOnlyList allArticles, CancellationToken ct); + } + public sealed class ArticleRejectedException(string slug, string reason) : Exception($"{slug}: {reason}"); + ``` +- Pipeline: + 1. **Write.** `Prompts.Write` produces German markdown. The first line of the reply is `VISIBILITY: public|private`, followed by the body. + 2. **Filter.** `LeakScanner.Scan` the German body, passing the source notes' full texts as `sources`, and `Privatize` the matching blocks. + 3. **Leak check.** `Prompts.LeakCheck` receives only the numbered `PublicBlocks` and replies `{"privatize":[ints]}`. Apply `Privatize`. The prompt says that a generic example explaining a principle, naming nothing concrete in this house, is not a leak. + 4. **Translate block by block.** Take `TranslationUnits` of the German body and send them as a numbered JSON array `[{"i":0,"text":"…"},…]`. `Prompts.Translate` requires a JSON array of exactly as many strings back, in order, keeping `[[slug]]` links and mermaid syntax unchanged. Build the English body with `Reassemble`. Private and public units go in the same request; the model sees no container lines and so cannot move a private boundary. + 5. **Filter and leak-check the English.** `Scan` the English body with the same `sources`, then run `Prompts.LeakCheck` on the English `PublicBlocks`. The English check is not skipped: the translation is new model text and may reintroduce a name the German check removed. Every block index privatized in this step is privatized in the German body too. Both bodies share one skeleton, so the indexes align and the two languages keep the same private blocks. + 6. **Check structure.** If `Skeleton(de) != Skeleton(en)`, throw `ArticleRejectedException`. This is a defensive invariant; steps 4 and 5 should make it unreachable. +- Visibility: one value from the `VISIBILITY` line, applied to both languages. `En.Visibility == De.Visibility` always. +- Links: the writer may link any slug in `allArticles` anywhere in the body. It does not know which planned articles will turn out private, so hiding is the renderer's job (Task 10 drops links to invisible slugs with their text). `Prompts.Write` tells it to refer to other articles only through `[[slug]]` and never to repeat their title next to the link; the link renders the title. +- `Prompts.Write` must state: + - the audience (non-technical first, then deeper); + - the order: lead → infobox → principle → examples → a short History at the end; + - the body conventions from Task 2; + - the link targets: only slugs in `allArticles`; + - that every claim comes from the sources; + - that the newest confirmed fact is used and disputed claims are omitted; + - the private rule: hostnames, IPs, device IDs, network and security layout, restart and server procedures, and attacker-useful detail are private, and private is the choice when unsure; + - that generic examples explaining a principle stay public, and that formats are illustrated with reserved example values (`192.0.2.x`, `2001:db8::`, `example.com`), never with this house's real ones; + - that no PR numbers, incident logs or entity IDs appear in public text; + - that interior photos of equipment are private unless clearly harmless; + - which images are available: the `attachments` frontmatter of the sources. +- Fail-closed rules: + - **Any model reply `Truncated`:** throw `ArticleRejectedException`. + - **Unparseable leak-check JSON:** throw. + - **Translator reply not a JSON array of strings, or of the wrong length, or `Reassemble` throws `FormatException`:** throw `ArticleRejectedException`. + - **`HttpRequestException` with status 400:** throw `ArticleRejectedException` (a credit or limit 400 is already `ModelUnavailableException`, Task 5). Other `HttpRequestException`s propagate as transients. + - **Missing `VISIBILITY` line:** treat as private. + - **Leak-check indexes out of range:** ignore those indexes. + +- [ ] **Step 1: Write failing tests**, using a `FakeTextModel` that returns scripted replies in order. + - `Write_RunsWriteLeakCheckTranslateLeakCheckInOrder`: assert four calls, with system prompts starting with `[write]`, `[leak-check]`, `[translate]`, `[leak-check]`. + - `Write_LeakCheckPrivatizesChosenBlocks`. + - `Write_DeterministicFilterRunsBeforeLeakCheck`: a source note contains `10.2.40.1`, and so does block 1 of the German body. The leak-check prompt does not contain `10.2.40.1`. + - `Write_GenericExampleStaysPublic`: the German body explains the address format with `192.0.2.10` in a public block, and the leak check returns `{"privatize":[]}`. The block is still public in both languages. + - `Write_PromptsAllowGenericExamples`: the `[write]` and `[leak-check]` system prompts both contain the phrase about generic examples. + - `Write_EnglishLeakFilteredToo`: the translation introduces `wiki.renner.uno` in unit 2, which ends up in a private block in English, and the matching German block is private too. + - `Write_EnglishLeakCheckPrivatizesBothLanguages`: the second leak check returns `{"privatize":[0]}`; block 0 is private in both bodies and `Skeleton(De.Body) == Skeleton(En.Body)`. + - `Write_LeakCheckUnparseable_Throws`. + - `Write_Truncated_Throws`. + - `Write_MissingVisibility_IsPrivate`. + - `Write_VisibilitySameInBothLanguages`. + - `Write_SourcesCarryHashes`: `De.Sources` equals the source docs' `(Slug, ContentHash)`. + - `Write_Http400_IsRejected`. + - `Translate_SendsNoStructuralLines`: the `[translate]` user prompt contains no line starting with `:::`. + - `Translate_WrongBlockCount_Throws`: the translator returns one string fewer than units sent. + - `Translate_NotJsonArray_Throws`. + - `Translate_UnitWithStructuralLine_Throws`: the translation of a private unit contains a `:::` line. + - `Translate_PrivateBlockStaysPrivate`: a German private unit `GEHEIM` translated as `SECRET` is inside `::: private` in English, and `ArticleBody.RedactPrivate(En.Body)` does not contain `SECRET`. +- [ ] **Step 2: Run them.** Expected: FAIL. +- [ ] **Step 3: Implement.** Each prompt's system string begins with a role marker (`[write]`, `[leak-check]`, `[translate]`) so the tests can tell the calls apart. Source notes are passed in full, each wrapped as `…`. +- [ ] **Step 4: Run them.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(encyclopedia): write, leak-check and translate pipeline`. + +--- + +### Task 8: Encyclopedia reconciler + +**Files:** +- Create: `src/Wiki.Application/Encyclopedia/EncyclopediaReconciler.cs` +- Test: `test/Wiki.Application.Tests/EncyclopediaReconcilerTests.cs` + +**Interfaces:** +- Consumes: + - `INoteStore.GetHeadAsync`, `CommitAsync`, `ReadTextAsync` (Task 4); + - `IDocumentCatalog.GetStateAsync`, `ListDocumentsAsync(DocumentKind.Wiki, ct)` and `ListDocumentsAsync(DocumentKind.Article, ct)`, called separately; + - `PlanArticlesUseCase` (Task 6) and `WriteArticleUseCase` (Task 7); + - `ArticleFile.Parse`, `ArticleFile.Build`; + - `ArticlePlanJson`, `EncyclopediaState`, `EncyclopediaStateJson` (Task 6); + - `TimeProvider`. +- Produces: + ```csharp + public sealed record ReconcileSummary(int Planned, int Written, int Deleted, int Rejected, int Remembered, + bool StoppedByModel, string? Skipped, string? Commit); // Remembered = rejected earlier, inputs unchanged, not retried + public sealed class EncyclopediaReconciler(INoteStore store, IDocumentCatalog catalog, PlanArticlesUseCase planner, + WriteArticleUseCase writer, EncyclopediaSettings settings, TimeProvider clock, ILogger log) + : IEncyclopediaRun + { + public async Task RunAsync(CancellationToken ct); + public static IReadOnlyList Stale(ArticlePlan plan, IReadOnlyList notes, IReadOnlyList
existing); + public static IReadOnlyList Due(IReadOnlyList stale, ArticlePlan plan, + IReadOnlyList notes, EncyclopediaState state, EncyclopediaSettings settings); + } + public interface IEncyclopediaRun { Task RunAsync(CancellationToken ct); } + ``` +- Algorithm: + 1. **Freshness gate.** If `catalog.GetStateAsync().Commit != store.GetHeadAsync().Commit`, return `Skipped = "catalog-behind"` with no model call and no write. Task 9 syncs right before each run, so this is a safety check; the next nightly run retries. + 2. Read `articles/_plan.json` and `articles/_state.json` with `ReadTextAsync` at the head. Null means none yet. A malformed plan reads as null and a malformed state as `Empty`, both logged. + 3. If `NeedsReplan(current, state.FailedPlan, notes)`, call `PlanAsync`. + - Non-null result: it becomes the plan, staged as `_plan.json`, and `FailedPlan` is cleared. + - Null result: record `FailedPlan = (NoteSetFingerprint(notes), now)` and keep `current`. + - `ModelUnavailableException`: stop the run. Keep the current plan, stage nothing, return `StoppedByModel = true`. No memory is recorded; it is a transient. + 4. **No usable plan** (`current` null and planning failed): return `Skipped = "no-plan"`. The only file committed is `_state.json` with the failed-plan memory; no article file is created, updated or deleted. + 5. Compute `Stale`. An article is stale when any of these is true: + - it is missing in either language; + - the set of its sources differs from the plan; + - any source hash differs from the catalog; + - the plan's title changed. + 6. Compute `Due` from the stale list, in plan order: + - If `PilotSlugs` is non-empty, keep only those slugs. Otherwise, if `MaxArticles` is set, keep only articles among the first `MaxArticles` of the plan. + - Drop a subject whose `state.Subjects[slug]` is `Rejected` with the same `SubjectFingerprint`; count it as `Remembered`. + - Take at most `MaxArticlesPerRun`. + 7. Write each due article: + - success: stage both files and record `(fingerprint, Written, now)`; + - `ArticleRejectedException` (includes a non-credit HTTP 400, Task 7): log, count as rejected, record `(fingerprint, Rejected, now)`, continue. The old version stays; + - other `HttpRequestException`: log, count as rejected, record nothing, continue. The next night retries; + - `ModelUnavailableException`: stop the loop and set `StoppedByModel=true`. Keep what is staged. + 8. **Deletes.** Only when step 3 produced a new plan: for each slug in the previous plan and not in the new one, stage a `Delete` in both languages for the files that exist, and drop its state entry. Articles outside both plans, and articles excluded by `PilotSlugs` or `MaxArticles`, are never deleted. + 9. Stage `_state.json` only when its serialized content changed. If anything is staged, call `CommitAsync` once with the message `articles: N written, M deleted, R rejected`. + - On `StoreConflictException`, log and return. The next run retries. +- Idempotence: after its own commit and a sync, nothing is stale or due and the state is unchanged, so a second run makes no model call and no commit. Add no extra guard. + +- [ ] **Step 1: Write failing tests**, using `FakeStore`, `FakeCatalog`, `FakeTextModel` and `FakeTimeProvider`. Seed the catalog state with the store's `Head` unless the test says otherwise. + - `Reconcile_FirstRun_PlansWritesCommitsOnce`: exactly one commit, containing `_plan.json`, `_state.json`, plus `de/` and `en/` files. + - `Reconcile_CatalogBehindHead_MakesNoModelCalls`: catalog state `c1`, store head `c2`; zero model calls, zero commits, `Skipped == "catalog-behind"`. + - `Reconcile_AfterOwnCommit_IsNoop`: run (the fake commit advances `Head`, Task 4), sync the catalog with `SyncUseCase.RunAsync(force: false)`, which now sees a new head, then run again. The second run passes the freshness gate and makes zero commits and zero model calls. + - `Reconcile_SourceChanged_RewritesOnlyThat`. + - `Reconcile_RespectsMaxArticlesPerRun`. + - `Reconcile_ModelUnavailable_CommitsWhatIsDone`: the second article throws. The commit contains the first article, and `StoppedByModel` is true. + - `Reconcile_PlanModelUnavailable_StopsKeepsPlan`: `PlanAsync` throws; zero commits, `_plan.json` unchanged, `StoppedByModel` true. + - `Reconcile_Rejected_KeepsOldVersion`: no change for that article's paths; `_state.json` records `Rejected`. + - `Reconcile_RejectedSameInputs_NotRetried`: a second run with unchanged sources makes no model call for that subject and counts it `Remembered`. + - `Reconcile_RejectedInputsChanged_Retried`: after one source's hash changes, the subject is written again. + - `Reconcile_Http400_RememberedAsRejected`. + - `Reconcile_PlanFailedSameNotes_DoesNotReplan`: planning returns garbage; a second run on the same notes makes zero model calls. + - `Reconcile_NoPlan_WritesNothing`: no plan exists and planning fails; no `articles/de/` or `articles/en/` path is touched, the only committed path is `articles/_state.json`, and existing article files remain. + - `Reconcile_SubjectDropped_DeletesBothLanguages`: the new plan lacks a slug the previous plan had. + - `Reconcile_ArticleNotInAnyPlan_NotDeleted`. + - `Reconcile_PilotCapShrinks_DeletesNothing`: three articles exist; `PilotSlugs` is changed to one of them; no `Delete` is staged and no commit is made. + - `Reconcile_PilotSlugs_WritesOnlyListed`. + - `Reconcile_StoreConflict_ReturnsWithoutThrow`. +- [ ] **Step 2: Run them.** Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run them.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(encyclopedia): incremental reconciler with single-commit writes`. + +--- + +### Task 9: Composition and post-sync trigger + +**Files:** +- Modify: `src/Wiki.Host/Composition/WikiOptions.cs` (new `EncyclopediaOptions Encyclopedia`) +- Create: `src/Wiki.Application/Encyclopedia/EncyclopediaCoordinator.cs` +- Create: `src/Wiki.Host/Composition/EncyclopediaBackgroundService.cs` +- Modify: `src/Wiki.Application/Status/StatusQuery.cs`, `src/Wiki.Host/Components/Pages/Status.razor` +- Modify: `src/Wiki.Host/Program.cs`, `src/Wiki.Host/appsettings.json` +- Test: `test/Wiki.Application.Tests/EncyclopediaCoordinatorTests.cs`, `test/Wiki.Host.Tests/EncyclopediaBackgroundServiceTests.cs` + +**Interfaces:** +- Consumes: `IEncyclopediaRun` and `ReconcileSummary` (Task 8), `SyncCoordinator.RunAsync(bool force, CancellationToken ct)`, `TimeProvider`. +- Produces: + ```csharp + public sealed class EncyclopediaOptions + { + public bool Enabled { get; set; } = false; + public string AnswerModel { get; set; } = "claude-sonnet"; // key read from Wiki:AnswerModels:Models:{AnswerModel}:ApiKey + public string Model { get; set; } = "claude-sonnet-5"; + public TimeOnly RunAt { get; set; } = new(6, 0); // local time in TimeZone; after the 04:30 distill + public string TimeZone { get; set; } = "Europe/Berlin"; + public int ArticleMaxTokens { get; set; } = 16000; + public int PlanMaxTokens { get; set; } = 32000; + public int MaxArticlesPerRun { get; set; } = 10; + public int MaxSourceCharsPerArticle { get; set; } = 300_000; + public List PilotSlugs { get; set; } = []; // when non-empty, only these subjects are written + public int? MaxArticles { get; set; } = null; // fallback write cap: first N planned subjects + } + // Application: one run at a time, failures recorded, never thrown (the SyncCoordinator pattern) + public sealed class EncyclopediaCoordinator(IEncyclopediaRun run, TimeProvider clock, ILogger log) + { + public bool Running { get; } + public ReconcileSummary? LastRun { get; } + public string? LastError { get; } + public DateTimeOffset? LastAttemptAt { get; } + public Task RunAsync(CancellationToken ct); + } + // Host + public sealed class EncyclopediaBackgroundService(SyncCoordinator sync, EncyclopediaCoordinator encyclopedia, + EncyclopediaOptions options, TimeProvider clock, ILogger log) : BackgroundService; + ``` + - `appsettings.json` gets `"Encyclopedia": { "Enabled": false }`. + - When `Enabled` is false, register nothing encyclopedia-related besides the options. + - **Schedule.** The service runs once a night at `RunAt` in `TimeZone`. It does not run on startup. It computes the next occurrence from `clock.GetUtcNow()` via `TimeZoneInfo` (so DST is handled) and waits with `Task.Delay(delay, clock, ct)`, so tests drive it with `FakeTimeProvider`. A run takes as long as it takes; the next occurrence is computed after it ends, so runs never overlap. + - **Before each run** it awaits `sync.RunAsync(force: false, ct)` so the catalog is current. The Task 8 freshness gate stays as a safety check. Sync and the push webhook no longer trigger the encyclopedia; the reconciler's own commit causes an ordinary sync that does not reconcile. + - **Never stops the host.** `EncyclopediaCoordinator.RunAsync` catches every exception except `OperationCanceledException` when `ct` is cancelled, sets `LastError` (message truncated to 300 chars), logs and returns null, exactly like `SyncCoordinator.RunAsync`. This covers a missing Anthropic key (`InvalidOperationException` from `MessagesTransport`), a Gitea error, and anything else. The service loop wraps each iteration in the same catch, so even a sync or scheduling bug only logs. + - `EncyclopediaSettings` is mapped from the options in `Program.cs`. + - Register the `ITextModel` via `AddAnthropicTextModel`, using `Model` and the API key described above, and add it to the `IReadOnlyList` list in `Program.cs` (`.. sp.GetServices()`, empty when disabled) so `/status` shows it. + - `StatusQuery` takes an optional `EncyclopediaCoordinator? encyclopedia = null` and adds `ReconcileSummary? LastEncyclopediaRun`, `string? LastEncyclopediaError` and `DateTimeOffset? LastEncyclopediaAttemptAt` to its output. `/status` shows them. + +- [ ] **Step 1: Write failing tests.** + - `Coordinator_RunThrows_RecordsTruncatedErrorReturnsNull`: the fake run throws `InvalidOperationException` with a 400-char message; `RunAsync` returns null and `LastError` has 300 chars. + - `Coordinator_Cancelled_Propagates`. + - `Service_RunsOncePerDayAtRunAt`: start the `FakeTimeProvider` at 05:00 Berlin. No run at start; one run after advancing to 06:00; still one at 05:59 the next day; two at 06:00. + - `Service_DoesNotRunOnStartup`: start at 07:00; no run until 06:00 the next day. + - `Service_SyncsBeforeRun`: the store head moves to `c2` before 06:00; when the fake run is invoked, the catalog state commit is `c2`. + - `Service_ReconcilerThrows_KeepsRunning`: the fake run throws `JsonException` on the first night; the service is still running, `LastError` is set, and the second night runs again. + - `Host_EncyclopediaDisabled_NoTextModelRegistered`: build the `WikiHostFactory` and assert `ITextModel` is not resolvable. + - `Host_EncyclopediaEnabled_TextModelInAdapterHealth`: the resolved `IReadOnlyList` contains an adapter named `anthropic-text:…`. +- [ ] **Step 2: Run them.** Expected: FAIL. +- [ ] **Step 3: Implement.** +- [ ] **Step 4: Run the suite.** Expected: PASS. +- [ ] **Step 5: Commit.** Message: `feat(host): run the encyclopedia nightly after a sync`. + +--- + +### Task 10: Article rendering + +**Files:** +- Create: `src/Wiki.Host/Rendering/ArticleMarkdown.cs` +- Create: `src/Wiki.Host/wwwroot/lib/mermaid/mermaid.min.js` (vendored `mermaid@11.4.1` `dist/mermaid.min.js`, with its LICENSE beside it) +- Create: `src/Wiki.Host/wwwroot/house.css` +- Modify: `src/Wiki.Core/Model/Attachment.cs` (expose the extension list) +- Test: `test/Wiki.Host.Tests/ArticleMarkdownTests.cs` + +**Interfaces:** +- Consumes: `ArticleBody.RedactPrivate` (Task 2), `Attachment` (`src/Wiki.Core/Model/Attachment.cs`). +- Produces: + ```csharp + public sealed record TocEntry(int Level, string Id, string Text); + public sealed record RenderedArticle(string Html, string? InfoboxHtml, IReadOnlyList Toc); + public static class ArticleMarkdown + { + // signedIn=false ⇒ body passed through RedactPrivate first. + // visibleTitles: slug → title in `lang` of every article this reader may see. + public static RenderedArticle Render(string body, string lang, string slug, bool signedIn, IReadOnlyDictionary visibleTitles); + // the valid raw/ image paths of a body, in order; Render and Task 11 both index into this list + public static IReadOnlyList ImagePaths(string visibleBody); + } + // Attachment.cs addition + public static IReadOnlyCollection FileExtensions { get; } // the values of the existing Extensions map: jpg, png, webp, gif + ``` +- Rendering rules: + - **Pipeline:** `UseAdvancedExtensions().DisableHtml()`, which gives custom containers, auto-identifiers and diagrams. Raw HTML stays disabled, so the model can never inject markup. + - **Infobox:** the `::: infobox` container is extracted into `InfoboxHtml` and removed from `Html`. + - **Private blocks:** `::: private` renders as `
` with a lock label. `::: private-locked` renders as `
`, containing the localized "Privater Abschnitt — anmelden zum Lesen" or "Private section — log in to read" and a link to `/house/login?returnUrl=…`. + - **Links:** `[[slug]]` with `slug` in `visibleTitles` renders as a link to `/house/{lang}/{slug}` whose text is the title. A slug not in `visibleTitles` is dropped entirely, text included, for every reader: anonymous readers must not learn a private subject's name, and signed-in readers see no dead links. Dropping a link also drops an adjacent list separator (`, ` or ` · `), and a list item or infobox row left with no text is removed. This covers "related" lists and infobox entries, which are written as `[[slug]]` too. + - **Mermaid:** a ```` ```mermaid ```` fence becomes `
` or Markdig's `
`. Pin whichever Markdig 1.4.0 emits in the test. + - **Images:** `ImagePaths` returns the image references whose URL matches `^raw/[A-Za-z0-9._-]+\.(jpg|png|webp|gif)$`, the extension group built from `Attachment.FileExtensions`. Those are rewritten to `/house/media/{lang}/{slug}/{n}`, where `n` is the index into `ImagePaths` of the redacted body. Any other image reference (other prefix, `..`, a subdirectory, a non-image extension, an external URL) is dropped. Task 11 serves them. + - **TOC:** built from the h2 and h3 headings, with the ids Markdig assigns. Each TOC entry links to `#id`. + - **Overlap fix:** in `house.css`, the infobox floats right. `.mermaid`, `figure`, `table` and `h2` get `clear: right`, so a diagram never sits under the infobox. This is the mockup bug the owner flagged. + +- [ ] **Step 1: Write failing tests.** + - `Render_Anonymous_ContainsNoPrivateText`: the private block contains `GEHEIM`, and `Html` does not. + - `Render_SignedIn_ShowsPrivateBlock`. + - `Render_InfoboxExtracted`. + - `Render_TocMatchesHeadingIds`: each `TocEntry.Id` appears as `id="…"` in `Html`. + - `Render_LinkUsesTitle`: `[[heating]]` with `visibleTitles["heating"] = "Heizung"` renders `Heizung`. + - `Render_Anonymous_LinkToInvisibleSlug_DroppedWithText`: neither `network-internals` nor its title appears in `Html`. + - `Render_InfoboxRowWithOnlyInvisibleLink_Removed`: an infobox row `| Verwandt | [[network-internals]] |` is gone; a row `[[heating]], [[network-internals]]` keeps only the heating link and no trailing comma. + - `Render_RawHtmlEscaped`: the input `