diff --git a/docs/superpowers/specs/2026-10-26-encyclopedia-bounded-planning-design.md b/docs/superpowers/specs/2026-10-26-encyclopedia-bounded-planning-design.md
index 4d01f5b..a20e006 100644
--- a/docs/superpowers/specs/2026-10-26-encyclopedia-bounded-planning-design.md
+++ b/docs/superpowers/specs/2026-10-26-encyclopedia-bounded-planning-design.md
@@ -42,7 +42,7 @@ Same entry point (`POST /api/v1/encyclopedia/run` from Wiki#18, called by the `w
1. **Diff.** Notes in the catalog but not in `assignments` are *new*. Notes in `assignments` but no longer in the catalog are *removed*: dropped from every article's sources. An article left without sources is deleted (its files removed; no redirect, the subject is gone). A note whose content changed keeps its assignment; the existing staleness check rewrites its articles.
2. **Assign** new notes, in batches (below). Cap: `MaxAssignBatchesPerRun` (default 120 — enough to bootstrap the current wiki in one night; ~100 batches).
3. **Split** articles that are too big. Cap: `MaxSplitsPerRun` (default 5).
-4. **Fold** articles that are too thin. Cap: `MaxFoldsPerRun` (default 5).
+4. **Fold** articles that are too thin. Cap: `MaxFoldsPerRun` (default 100, see the consolidation spec, *Overviews and caps*).
5. **Write** stale articles, as today (`MaxArticlesPerRun`, PilotSlugs, MaxArticles unchanged).
A step that hits its cap leaves the rest for the next night; nothing done so far is lost.
diff --git a/docs/superpowers/specs/2026-10-26-encyclopedia-consolidation-design.md b/docs/superpowers/specs/2026-10-26-encyclopedia-consolidation-design.md
index 842fe98..780bc0a 100644
--- a/docs/superpowers/specs/2026-10-26-encyclopedia-consolidation-design.md
+++ b/docs/superpowers/specs/2026-10-26-encyclopedia-consolidation-design.md
@@ -34,11 +34,11 @@ Consolidation leaves many redirects with natural names, which the model may prop
**Passes.** The step runs passes until one changes nothing. A reply that changes the plan (a merge, or a parent link that was not there) ends the pass, and the next pass starts again from the first seed. A merge shifts every later group in slug order, since the merged article's place in the groups goes to the next-nearest article; restarting means the groups of the last pass are exactly the groups the next run forms from an unchanged plan. Re-forming the groups is local vector arithmetic only. Every pass counts its calls towards the same cap, so the loop ends: each extra pass needs a call that changed something.
-**Call.** One per group, at most `MaxConsolidationsPerRun` (default 60) per run. 60 groups of 8 cover ~300+ articles in one bootstrap night even with overlapping groups (299 articles need at least 38). Input, per member: `slug | titleDe | category | scope | source count | current parent` — bounded by the group size. `ConsolidateMaxTokens` 8000 (the reply is short; thinking tokens count, as for fold).
+**Call.** One per group, at most `MaxConsolidationsPerRun` (default 150, see *Overviews and caps*) per run. Input, per member: `slug | titleDe | category | scope | source count | current parent` — bounded by the group size. `ConsolidateMaxTokens` 8000 (the reply is short; thinking tokens count, as for fold).
-**Reply** (JSON): `{"merges":[{"from","into"}],"parents":[{"child","parent"}]}`. A merge says both articles are about the same subject; a parent link says which article is the overview of which.
+**Reply** (JSON): `{"merges":[{"from","into"}],"parents":[{"child","parent"}],"overviews":[…]}`. A merge says both articles are about the same subject; a parent link says which article is the overview of which; an overview proposes a new article above group members (see *Overviews and caps*).
-**Validation.** Both slugs of an entry must be members of the group and differ. A `from` merges once (first wins); an article merged away is never a merge target. Parent entries are resolved through the merges of the same reply (a merged child or parent stands for its target); a link that would close a cycle anywhere in the plan is dropped. Invalid entries are dropped one by one; a reply that is truncated, not JSON, rejected with 400 or has neither array is unusable.
+**Validation.** Both slugs of an entry must be members of the group and differ. A `from` merges once (first wins); an article merged away is never a merge target. Parent entries are resolved through the merges of the same reply (a merged child or parent stands for its target); a link that would close a cycle anywhere in the plan is dropped. Invalid entries are dropped one by one; a reply that is truncated, not JSON, rejected with 400 or has none of the arrays (`merges`, `parents`, `overviews`) is unusable.
**Merge** uses fold's mechanics (`ArticlePlan.Fold`): sources move into the target, `redirects[from] = into`, public when both language files of `from` were public at the run's start (`publicRedirects`), redirects to `from` move along. Additionally the children of `from` move to `into`; if `into` was a child of `from`, it takes `from`'s parent. A move that would close a cycle leaves the child without parent.
@@ -67,6 +67,36 @@ Category `room` exists; the prompts now use it:
Nothing has been written or published yet; only `_plan.json` and `_state.json` exist. The plan file gains `"version": 3`; a plan without it (the current v2) or with another version reads as "no plan", so the next run bootstraps with the new rules. When the stored plan does not read, the stored state is dropped as well (its subjects, failed batches and kept articles belong to the old plan). On the same reset, every article file whose slug the new plan does not contain is deleted, like the files of an article the plan dropped, so that no file outlives the plan that made it. That covers this switch and any later format change or corruption. `Prompts.Version` goes to 8.
+## Overviews and caps
+
+The first plan built with these rules (375 articles) still had no hierarchy where the natural parent had no article: heating was `vitocal-waermepumpe`, `open3e`, `viessmann-can-bus-integration`, `vitocharge-vx3`, `viessmann-vitocal-energy-management-ems`, … side by side with no "Heizung" above them; lighting likewise; the 13 room articles had no floor. Consolidation can only link existing articles. 113 articles had a single source while fold made 2 of its 5 calls, and duplicates such as `open3e` / `open3e-integration` or `kg-werkstatt` / `ug-werkstatt` lay in groups the 60-call cap did not reach.
+
+**Overview proposals.** The consolidate reply may hold a third array, `"overviews":[{"slug","titleDe","titleEn","category","scope","children":[…]}]`: a new article over two or more group members that share a broader subject with no article. Validation, entry by entry, invalid ones dropped:
+
+- children: slugs of the group, resolved through the reply's merges, each once; a child that has a `parents` entry of the reply, or belongs to an earlier overview of the reply, is left out. Fewer than two children left: the entry is dropped.
+- slug: an overview slug that is a live article is a reference: its children get that article as parent (the article may lie outside the group; a link that would close a cycle is dropped when applied). A slug that is a redirect or retired is dropped, children unchanged: the model never sees redirects, and a redirect's target (say a device article a thin "Heizung" was folded into) is not the overview it means. Otherwise the slug must match the slug pattern and be free (not proposed twice in the reply), and the titles, a known category and the scope must be present.
+- parent: the overview takes the current parent most of its children have (children without parent do not count); on a tie, or when none has one, none. It does not take a reply field, so the model cannot hang it anywhere outside the group.
+
+A reply with only `overviews` is usable; a reply with none of the three arrays is not.
+
+**Applying.** After the merges and parent links: the overview is added with no sources and `"overview": true`, each child gets it as parent, then the overview gets the shared parent (a parent that would close a cycle is dropped, so the children are never refused). Each link counts towards `Parented`. An overview is a change of the plan like a merge: it ends the pass, and the next pass forms its group with the newcomer.
+
+**The overview mark.** `PlannedArticle.Overview` (plan field `overview`; missing reads as false, so the plan format stays 3) marks an article consolidation made as an overview. Its `sources` are its own notes only: none at first, later any note assign gives it, or the sources of an article merged into it. `ReplaceArticle` keeps the mark like the parent.
+
+**Borrowed sources.** What an overview is written from is derived every run from the plan as it stands, never stored: its own sources, then those of its current sub-articles taken in turn (the first of each, then the second, …), each once, at most 12 borrowed (`MaxOverviewSources`); a sub-article that is itself an overview lends what it is written from. Staleness, the subject fingerprint (kept, rejected) and the write all use this list, and consolidation shows its count as the source count. So a sub-article that leaves (re-parented, merged, folded, deleted) takes its notes out of the overview, and the overview is rewritten; a sub-article's new note reaches the overview when it falls within the cap. Borrowed notes are not assigned to the overview, so merging or folding an overview moves only its own sources into the target, never its former children's. The write prompt's existing rule ("an article with sub-articles is their overview") and the sub-article links make it an overview page. Alternative considered: the children's scopes instead of their notes, which needs a new writer input and gives the writer no facts to cite.
+
+**Lifetime.** Note removal never deletes an overview, even when it loses its last own source. After the planning steps (whether they finished or not), `PruneOverviews` ends every overview without sub-articles, repeatedly, so an overview of overviews follows: one without own sources is deleted (its files with it; redirects to it are retired) and its slug is not retired, so the subject can get its overview back under the natural slug; one with own sources becomes an ordinary article.
+
+**Never folded for being thin.** An overview has few or no own sources and would be thin. Fold skips any article that has sub-articles, whether consolidation, split or a parent link made it an overview: folding it would only move its children to the target. Such an article is not held back from writing as awaiting fold either. An article that loses all its children may be folded again.
+
+**Overviews of overviews.** The new article gets an article vector like any other, so later groups can find it beside other overviews and give it a parent or propose an overview above it ("Haustechnik" over "Heizung" and "Beleuchtung").
+
+**Rooms and floors.** The consolidate prompt makes a floor overview (category room, "Erdgeschoss", "Obergeschoss", "Dachgeschoss", "Kellergeschoss") the parent of a room, and says that UG, KG and Keller are the same floor of this house, so room articles that differ only in that are merged. The assign prompt names the floors EG, OG, DG and KG, the basement always KG.
+
+**Caps.** `MaxFoldsPerRun` 5 → 100 and `MaxConsolidationsPerRun` 60 → 150. A bootstrap plan of ~400 articles has ~100+ thin articles and needs several consolidation passes (every overview or merge restarts the pass, and groups overlap); at 5 folds a night the thin articles would take weeks. Each call is small (a fold ~0.5k input tokens and an 8k reply cap, a consolidation ~1k input tokens), and both steps only ask what is not remembered, so a normal night stays far below either cap. The caps bound the worst case, not the normal run.
+
+**Prompt version.** `Prompts.Version` goes to 9. The group fingerprint includes it, so every remembered consolidation group (and failed group) is asked again with the overview prompt; kept fold decisions and rejected subjects are retried too, since their fingerprints include it as well. The plan format stays 3: the overview mark is an optional field, and the existing plan is kept. The consolidate prompt also carries assign's rule that titles and slugs name nothing concrete in the house.
+
## Cost
-Bootstrap: up to 60 consolidation calls of ~1k input tokens each on top of the assign calls. A normal night: the groups around changed articles.
+Bootstrap: up to 150 consolidation calls of ~1k input tokens each and up to 100 fold calls on top of the assign calls. A normal night: the groups around changed articles.
diff --git a/src/Wiki.Application/Encyclopedia/ArticlePlan.cs b/src/Wiki.Application/Encyclopedia/ArticlePlan.cs
index 015e75e..ad3661d 100644
--- a/src/Wiki.Application/Encyclopedia/ArticlePlan.cs
+++ b/src/Wiki.Application/Encyclopedia/ArticlePlan.cs
@@ -7,6 +7,8 @@ namespace Wiki.Application.Encyclopedia;
///
/// is one plain sentence on what the article covers; are note slugs, most relevant first.
/// is the slug of the overview article this one is a sub-article of.
+/// marks an article made as the overview of its sub-articles: it is written from its own sources
+/// and some of theirs, and lives as long as it has sub-articles, with or without own sources.
///
public sealed record PlannedArticle(
string Slug,
@@ -15,7 +17,8 @@ public sealed record PlannedArticle(
ArticleCategory Category,
string Scope,
IReadOnlyList Sources,
- string? Parent = null);
+ string? Parent = null,
+ bool Overview = false);
///
/// The stored plan: the articles, which notes feed them, and where retired slugs point.
@@ -100,8 +103,8 @@ public sealed class ArticlePlan
}
///
- /// Drops everywhere. Returns the slugs of articles deleted for lack of sources; redirects to them go
- /// too, and their children lose their parent.
+ /// Drops everywhere. Returns the slugs of articles deleted for lack of sources (never an overview,
+ /// see ); redirects to them go too, and their children lose their parent.
///
public IReadOnlyList RemoveNote(string note)
{
@@ -111,7 +114,7 @@ public sealed class ArticlePlan
{
if (!a.Sources.Contains(note)) continue;
var sources = a.Sources.Where(s => s != note).ToList();
- if (sources.Count == 0)
+ if (sources.Count == 0 && !a.Overview)
{
_articles.Remove(a);
deleted.Add(a.Slug);
@@ -121,33 +124,60 @@ public sealed class ArticlePlan
foreach (var slug in deleted)
{
_retired.Add(slug);
- foreach (var child in ChildrenOf(slug)) Replace(child with { Parent = null });
- foreach (var key in _redirects.Where(r => r.Value == slug).Select(r => r.Key).ToList())
- {
- _redirects.Remove(key);
- _publicRedirects.Remove(key);
- _retired.Add(key);
- }
- foreach (var n in _assignments.Keys.ToList())
- {
- if (!_assignments[n].Contains(slug)) continue;
- var rest = _assignments[n].Where(s => s != slug).ToList();
- // an empty list would read as a deliberate skip; the note is assigned afresh instead
- if (rest.Count == 0) _assignments.Remove(n);
- else _assignments[n] = rest;
- }
+ Forget(slug);
}
return deleted;
}
///
- /// Same slug, new content; the parent stays ( changes it). Assignments follow the new
- /// : a note that left loses the article, one that joined gains it.
+ /// Ends every overview without sub-articles, until none is left: one without own sources is deleted, redirects to it go and
+ /// its slug stays free for a later overview; one with own sources becomes an ordinary article. Returns the deleted slugs.
+ ///
+ public IReadOnlyList PruneOverviews()
+ {
+ var deleted = new List();
+ while (_articles.FirstOrDefault(a => a.Overview && ChildrenOf(a.Slug).Count == 0) is { } a)
+ {
+ if (a.Sources.Count > 0)
+ {
+ Replace(a with { Overview = false });
+ continue;
+ }
+ _articles.Remove(a);
+ deleted.Add(a.Slug);
+ Forget(a.Slug);
+ }
+ return deleted;
+ }
+
+ /// Drops the links to the removed article : children lose their parent, redirects to it are retired, assignments lose it.
+ private void Forget(string slug)
+ {
+ foreach (var child in ChildrenOf(slug)) Replace(child with { Parent = null });
+ foreach (var key in _redirects.Where(r => r.Value == slug).Select(r => r.Key).ToList())
+ {
+ _redirects.Remove(key);
+ _publicRedirects.Remove(key);
+ _retired.Add(key);
+ }
+ foreach (var n in _assignments.Keys.ToList())
+ {
+ if (!_assignments[n].Contains(slug)) continue;
+ var rest = _assignments[n].Where(s => s != slug).ToList();
+ // an empty list would read as a deliberate skip; the note is assigned afresh instead
+ if (rest.Count == 0) _assignments.Remove(n);
+ else _assignments[n] = rest;
+ }
+ }
+
+ ///
+ /// Same slug, new content; the parent and the overview mark stay ( changes the parent). Assignments
+ /// follow the new : a note that left loses the article, one that joined gains it.
///
public void ReplaceArticle(PlannedArticle a)
{
var old = Find(a.Slug) ?? throw new InvalidOperationException($"Unknown article '{a.Slug}'");
- a = a with { Parent = old.Parent };
+ a = a with { Parent = old.Parent, Overview = old.Overview };
Replace(a);
foreach (var note in old.Sources.Except(a.Sources))
{
@@ -245,6 +275,7 @@ public static class ArticlePlanJson
a.Scope,
a.Sources,
a.Parent,
+ a.Overview,
}),
Assignments = new SortedDictionary>(p.Assignments.ToDictionary(), StringComparer.Ordinal),
Redirects = new SortedDictionary(p.Redirects.ToDictionary(), StringComparer.Ordinal),
@@ -307,7 +338,8 @@ public static class ArticlePlanJson
if (slug is null || !PlanParsing.SlugPattern().IsMatch(slug)
|| !PlanParsing.TryCategory(PlanParsing.Text(o["category"]), out var category)) continue;
articles.Add(new PlannedArticle(slug, PlanParsing.Text(o["titleDe"]) ?? slug, PlanParsing.Text(o["titleEn"]) ?? slug, category,
- PlanParsing.Text(o["scope"]) ?? "", PlanParsing.Strings(o["sources"]), PlanParsing.Text(o["parent"])));
+ PlanParsing.Text(o["scope"]) ?? "", PlanParsing.Strings(o["sources"]), PlanParsing.Text(o["parent"]),
+ o["overview"] is JsonValue overview && overview.TryGetValue(out var isOverview) && isOverview));
}
return articles;
}
diff --git a/src/Wiki.Application/Encyclopedia/ConsolidateArticlesUseCase.cs b/src/Wiki.Application/Encyclopedia/ConsolidateArticlesUseCase.cs
index a6e6e48..69111e6 100644
--- a/src/Wiki.Application/Encyclopedia/ConsolidateArticlesUseCase.cs
+++ b/src/Wiki.Application/Encyclopedia/ConsolidateArticlesUseCase.cs
@@ -15,22 +15,35 @@ public sealed record ArticleMerge(string From, string Into);
public sealed record ArticleParent(string Child, string Parent);
///
-/// What one consolidation call found in its group, in reply order. No article is merged twice or merged away after being a
-/// target; parents name the merge targets, at most one per child. Parent cycles are not checked here: the plan refuses them.
+/// A new article that is the overview of , at least two group members. has
+/// no sources; its parent is the current parent most children have, none on a tie.
///
-public sealed record ConsolidateOutcome(IReadOnlyList Merges, IReadOnlyList Parents);
+public sealed record ArticleOverview(PlannedArticle Article, IReadOnlyList Children);
-/// Compares a group of nearby articles: which share a subject, which is the overview of which, in one bounded model call.
+///
+/// What one consolidation call found in its group, in reply order. No article is merged twice or merged away after being a
+/// target; parents and overview children name the merge targets, each child gets at most one parent or overview. A parent may
+/// lie outside the group when the reply proposed it as an overview under the slug of a live article.
+/// Parent cycles are not checked here: the plan refuses them.
+///
+public sealed record ConsolidateOutcome(IReadOnlyList Merges, IReadOnlyList Parents,
+ IReadOnlyList Overviews);
+
+///
+/// Compares a group of nearby articles: which share a subject, which is the overview of which, and which share an overview
+/// subject that has no article yet, in one bounded model call.
+///
public sealed class ConsolidateArticlesUseCase(ITextModel model, EncyclopediaSettings settings, ILogger log)
{
///
/// The first articles of are shown; entries
- /// naming any other slug are dropped. Null when the reply is unusable. Model-unavailable and other errors propagate.
+ /// naming any other slug are dropped. decides which proposed overview slugs are free.
+ /// Null when the reply is unusable. Model-unavailable and other errors propagate.
///
- public async Task ConsolidateAsync(IReadOnlyList group, CancellationToken ct)
+ public async Task ConsolidateAsync(IReadOnlyList group, ArticlePlan plan, CancellationToken ct)
{
var shown = group.Take(settings.ConsolidateGroupSize).ToList();
- if (shown.Count < 2) return new ConsolidateOutcome([], []);
+ if (shown.Count < 2) return new ConsolidateOutcome([], [], []);
TextReply reply;
try
{
@@ -49,16 +62,17 @@ public sealed class ConsolidateArticlesUseCase(ITextModel model, EncyclopediaSet
}
var root = PlanParsing.Object(reply.Text);
- if (root is null || (root["merges"] is not JsonArray && root["parents"] is not JsonArray))
+ if (root is null || (root["merges"] is not JsonArray && root["parents"] is not JsonArray && root["overviews"] is not JsonArray))
{
- log.LogWarning("Consolidate reply for the group of {Seed} is not {{\"merges\":[…],\"parents\":[…]}}", shown[0].Slug);
+ log.LogWarning("Consolidate reply for the group of {Seed} is not {{\"merges\":[…],\"parents\":[…],\"overviews\":[…]}}", shown[0].Slug);
return null;
}
- return Validate(root, shown.Select(a => a.Slug).ToHashSet(StringComparer.Ordinal));
+ return Validate(root, shown, plan);
}
- private ConsolidateOutcome Validate(JsonObject root, HashSet members)
+ private ConsolidateOutcome Validate(JsonObject root, List group, ArticlePlan plan)
{
+ var members = group.Select(a => a.Slug).ToHashSet(StringComparer.Ordinal);
var merges = new List();
var into = new Dictionary(StringComparer.Ordinal);
foreach (var (from, to) in Pairs(root["merges"], "from", "into", members))
@@ -84,7 +98,59 @@ public sealed class ConsolidateArticlesUseCase(ITextModel model, EncyclopediaSet
}
parents.Add(new ArticleParent(child, parent));
}
- return new ConsolidateOutcome(merges, parents);
+
+ var overviews = new List();
+ if (root["overviews"] is JsonArray items)
+ foreach (var item in items)
+ {
+ if (item is not JsonObject o || Text(o["slug"]) is not { } slug) continue;
+ var children = Strings(o["children"]).Where(members.Contains).Select(c => into.GetValueOrDefault(c, c))
+ .Distinct(StringComparer.Ordinal)
+ .Where(c => !parents.Any(p => p.Child == c) && !overviews.Any(v => v.Children.Contains(c)))
+ .ToList();
+ if (children.Count < 2)
+ {
+ log.LogWarning("Dropping overview {Slug}: fewer than two group articles left as its children", slug);
+ continue;
+ }
+ // an overview slug that is a live article makes that article the parent; a redirected or retired one is taken
+ if (plan.Find(slug) is not null)
+ {
+ foreach (var child in children.Where(c => c != slug)) parents.Add(new ArticleParent(child, slug));
+ continue;
+ }
+ if (ReadOverview(o, slug, plan, overviews) is not { } article) continue;
+ overviews.Add(new ArticleOverview(article with { Parent = SharedParent(children, group) }, children));
+ }
+ return new ConsolidateOutcome(merges, parents, overviews);
+ }
+
+ /// The current parent most of have, other than one of them; null when none or tied.
+ private static string? SharedParent(List children, List group)
+ {
+ var counts = children.Select(c => group.First(a => a.Slug == c).Parent).OfType()
+ .Where(p => !children.Contains(p))
+ .GroupBy(p => p, StringComparer.Ordinal)
+ .Select(g => (Parent: g.Key, Count: g.Count()))
+ .OrderByDescending(x => x.Count)
+ .ToList();
+ return counts.Count > 0 && (counts.Count == 1 || counts[1].Count < counts[0].Count) ? counts[0].Parent : null;
+ }
+
+ private PlannedArticle? ReadOverview(JsonObject o, string slug, ArticlePlan plan, List seen)
+ {
+ if (!SlugPattern().IsMatch(slug) || plan.SlugTaken(slug) || seen.Any(v => v.Article.Slug == slug))
+ {
+ log.LogWarning("Dropping overview {Slug}: invalid, taken or duplicate slug", slug);
+ return null;
+ }
+ if (!TryCategory(Text(o["category"]), out var category)
+ || Text(o["scope"]) is not { } scope || Text(o["titleDe"]) is not { } titleDe || Text(o["titleEn"]) is not { } titleEn)
+ {
+ log.LogWarning("Dropping overview {Slug}: unknown category, or missing scope or title", slug);
+ return null;
+ }
+ return new PlannedArticle(slug, titleDe, titleEn, category, scope, []);
}
/// The entries of whose two fields are different members of the group.
diff --git a/src/Wiki.Application/Encyclopedia/EncyclopediaReconciler.cs b/src/Wiki.Application/Encyclopedia/EncyclopediaReconciler.cs
index 62b042e..1e049f4 100644
--- a/src/Wiki.Application/Encyclopedia/EncyclopediaReconciler.cs
+++ b/src/Wiki.Application/Encyclopedia/EncyclopediaReconciler.cs
@@ -44,6 +44,8 @@ public sealed class EncyclopediaReconciler(
private const string StatePath = "articles/_state.json";
/// Sub-articles an article's write prompt lists as link targets, at most.
private const int MaxChildLinks = 20;
+ /// Sources an overview borrows from its sub-articles, at most.
+ private const int MaxOverviewSources = 12;
private static readonly string[] Languages = ["de", "en"];
/// The inputs and mutable outputs of one run.
@@ -126,6 +128,8 @@ public sealed class EncyclopediaReconciler(
log.LogError(e, "Planning failed unexpectedly; committing the plan so far, then failing the run");
failure = ExceptionDispatchInfo.Capture(e);
}
+ foreach (var slug in plan.PruneOverviews())
+ log.LogInformation("Overview {Slug} has no sub-article and no source left; deleting it", slug);
var hashByPath = articleDocs.ToDictionary(d => d.Path, d => d.ContentHash, StringComparer.Ordinal);
var stale = Stale(plan, notes, existing, settings);
@@ -298,7 +302,7 @@ public sealed class EncyclopediaReconciler(
ConsolidateOutcome? outcome;
try
{
- outcome = await consolidator.ConsolidateAsync(group, ct);
+ outcome = await consolidator.ConsolidateAsync(group.Select(a => Written(plan, a)).ToList(), plan, ct);
}
catch (HttpRequestException e)
{
@@ -335,7 +339,7 @@ public sealed class EncyclopediaReconciler(
///
/// Merges as folds (public redirect when the merged article was public at the run's start), then the parent links that
- /// close no cycle. True when the plan changed.
+ /// close no cycle, then the overviews with their children. True when the plan changed.
///
private bool Apply(Run run, ConsolidateOutcome outcome)
{
@@ -349,20 +353,57 @@ public sealed class EncyclopediaReconciler(
run.Merged++;
changed = true;
}
- foreach (var p in outcome.Parents)
+ foreach (var p in outcome.Parents) changed |= Parent(run, p.Child, p.Parent);
+ foreach (var o in outcome.Overviews)
{
- if (plan.Resolve(p.Child) is not { } child || plan.Resolve(p.Parent) is not { } parent || child == parent
- || plan.Find(child)!.Parent == parent) continue;
- if (plan.SetParent(child, parent))
- {
- run.Parented++;
- changed = true;
- }
- else log.LogWarning("Dropping parent {Parent} of {Child}: it would close a cycle", parent, child);
+ var children = o.Children.Select(plan.Resolve).OfType().Distinct(StringComparer.Ordinal).ToList();
+ if (children.Count < 2 || plan.SlugTaken(o.Article.Slug)) continue;
+ log.LogInformation("Adding overview {Slug} of {Children}", o.Article.Slug, children);
+ plan.AddArticle(o.Article with { Sources = [], Parent = null, Overview = true });
+ foreach (var child in children) Parent(run, child, o.Article.Slug);
+ if (o.Article.Parent is { } parent) Parent(run, o.Article.Slug, parent);
+ changed = true;
}
return changed;
}
+ /// Sets the parent of unless it is set already or would close a cycle; true when set.
+ private bool Parent(Run run, string rawChild, string rawParent)
+ {
+ var plan = run.Plan;
+ if (plan.Resolve(rawChild) is not { } child || plan.Resolve(rawParent) is not { } parent || child == parent
+ || plan.Find(child)!.Parent == parent) return false;
+ if (!plan.SetParent(child, parent))
+ {
+ log.LogWarning("Dropping parent {Parent} of {Child}: it would close a cycle", parent, child);
+ return false;
+ }
+ run.Parented++;
+ return true;
+ }
+
+ ///
+ /// with the sources it is written from. An overview's are its own, then those of its current
+ /// sub-articles taken in turn (the first of each, then the second, …), each once, at most
+ /// borrowed; a sub-article that is an overview lends what it is written from. Any other article is returned as it is.
+ /// Derived from the plan as it stands, never stored, so a sub-article that leaves takes its notes along.
+ ///
+ public static PlannedArticle Written(ArticlePlan plan, PlannedArticle a)
+ {
+ if (!a.Overview) return a;
+ var lent = plan.ChildrenOf(a.Slug).Select(c => Written(plan, c).Sources).ToList();
+ var sources = a.Sources.ToList();
+ var borrowed = 0;
+ for (var i = 0; borrowed < MaxOverviewSources && lent.Any(s => i < s.Count); i++)
+ foreach (var s in lent.Where(s => i < s.Count))
+ {
+ if (borrowed == MaxOverviewSources || sources.Contains(s[i])) continue;
+ sources.Add(s[i]);
+ borrowed++;
+ }
+ return a with { Sources = sources };
+ }
+
/// Oversized articles, biggest first, at most calls. The children get the overview as parent.
private async Task SplitAsync(Run run, CancellationToken ct)
{
@@ -400,7 +441,7 @@ public sealed class EncyclopediaReconciler(
}
///
- /// Thin articles with a nearby article, not kept before at the same inputs and not split off this run, at most
+ /// Thin articles without sub-articles with a nearby article, not kept before at the same inputs and not split off this run, at most
/// calls; past the cap only the nearby lookup runs, to find those with no
/// fold pending (). An answer other than a usable fold is remembered like a keep, so the article
/// is not asked again until its inputs change; a failed call is retried next run.
@@ -409,12 +450,12 @@ public sealed class EncyclopediaReconciler(
{
var plan = run.Plan;
var children = run.SplitOff;
- var thin = plan.Articles.Where(a => FoldArticleUseCase.IsThin(a, run.NoteBySlug, settings)).Select(a => a.Slug).ToList();
+ var thin = plan.Articles.Where(a => IsThin(run, a)).Select(a => a.Slug).ToList();
var calls = 0;
foreach (var slug in thin)
{
if (children.Contains(slug)) continue;
- if (plan.Find(slug) is not { } article || !FoldArticleUseCase.IsThin(article, run.NoteBySlug, settings)) continue;
+ if (plan.Find(slug) is not { } article || !IsThin(run, article)) continue;
var fingerprint = EncyclopediaState.SubjectFingerprint(article, run.Notes);
if (run.Kept.GetValueOrDefault(slug) == fingerprint) continue;
var nearby = Articles(plan, await neighbours.NearbyArticlesAsync(article, run.NoteBySlug, plan, ct));
@@ -452,12 +493,16 @@ public sealed class EncyclopediaReconciler(
///
private IReadOnlySet AwaitingFold(Run run) =>
run.Plan.Articles
- .Where(a => FoldArticleUseCase.IsThin(a, run.NoteBySlug, settings)
+ .Where(a => IsThin(run, a)
&& !run.Unfoldable.Contains(a.Slug)
&& run.Kept.GetValueOrDefault(a.Slug) != EncyclopediaState.SubjectFingerprint(a, run.Notes))
.Select(a => a.Slug)
.ToHashSet(StringComparer.Ordinal);
+ /// Thin by and no overview: an article with sub-articles is never folded for being thin.
+ private bool IsThin(Run run, PlannedArticle a) =>
+ FoldArticleUseCase.IsThin(a, run.NoteBySlug, settings) && run.Plan.ChildrenOf(a.Slug).Count == 0;
+
private async Task<(int Written, int Rejected, bool Stopped, ExceptionDispatchInfo? Failure)> WriteAsync(
Run run, IReadOnlyList due, Dictionary hashByPath, List changes, CancellationToken ct)
{
@@ -525,9 +570,9 @@ public sealed class EncyclopediaReconciler(
}
///
- /// Planned articles, in plan order, whose files no longer match the plan and the notes: missing in either language,
- /// a different set of sources than the plan's sources the writer gets (), a source hash other
- /// than the catalog's, or another title.
+ /// Planned articles, in plan order and as , whose files no longer match the plan and the notes: missing
+ /// in either language, a different set of sources than the plan's sources the writer gets (), a
+ /// source hash other than the catalog's, or another title.
///
public static IReadOnlyList Stale(ArticlePlan plan, IReadOnlyList notes, IReadOnlyList existing,
EncyclopediaSettings settings)
@@ -535,6 +580,7 @@ public sealed class EncyclopediaReconciler(
var bySlug = notes.GroupBy(n => n.Slug, StringComparer.Ordinal).ToDictionary(g => g.Key, g => g.First(), StringComparer.Ordinal);
var articles = existing.GroupBy(a => (a.Lang, a.Slug)).ToDictionary(g => g.Key, g => g.First());
return plan.Articles
+ .Select(a => Written(plan, a))
.Where(p =>
{
var sources = WriteSources(p, bySlug, settings);
diff --git a/src/Wiki.Application/Encyclopedia/EncyclopediaSettings.cs b/src/Wiki.Application/Encyclopedia/EncyclopediaSettings.cs
index 378a8c5..6a3d208 100644
--- a/src/Wiki.Application/Encyclopedia/EncyclopediaSettings.cs
+++ b/src/Wiki.Application/Encyclopedia/EncyclopediaSettings.cs
@@ -16,13 +16,13 @@ public sealed record EncyclopediaSettings(
int NeighbourSources = 10,
int MaxAssignBatchesPerRun = 120,
int MaxSplitsPerRun = 5,
- int MaxFoldsPerRun = 5,
+ int MaxFoldsPerRun = 100,
int MaxSourcesPerArticle = 120,
int FoldMinSources = 3,
int FoldMinChars = 4000,
// articles per consolidation group, the seed included
int ConsolidateGroupSize = 8,
- int MaxConsolidationsPerRun = 60,
+ int MaxConsolidationsPerRun = 150,
// thinking tokens count against max_tokens
int AssignMaxTokens = 32000,
int SplitMaxTokens = 32000,
diff --git a/src/Wiki.Application/Encyclopedia/Prompts.cs b/src/Wiki.Application/Encyclopedia/Prompts.cs
index 1e71279..856bcfb 100644
--- a/src/Wiki.Application/Encyclopedia/Prompts.cs
+++ b/src/Wiki.Application/Encyclopedia/Prompts.cs
@@ -6,7 +6,7 @@ namespace Wiki.Application.Encyclopedia;
///
public static class Prompts
{
- public const int Version = 8;
+ public const int Version = 9;
public const string Assign = """
[assign]
@@ -16,7 +16,7 @@ public static class Prompts
For each note decide which articles it feeds: existing articles from the list, or new articles you propose.
- Prefer an existing article from the list when the note fits its scope.
- Granularity follows the material, like notability on Wikipedia. A subject gets its own new article only when the notes hold enough on it. A subject that was set once and forgotten is not an article of its own; its note feeds the article of its parent system as a section.
- - A note about something in a specific room additionally feeds that room's article (category room), titled by floor and room, for example "EG Schlafzimmer". Create the room article when none is in the list.
+ - A note about something in a specific room additionally feeds that room's article (category room), titled by floor and room, for example "EG Schlafzimmer". The floors are EG, OG, DG and KG; the basement is always KG, never UG or Keller. Create the room article when none is in the list.
- Skip a note only when it is a meta note or about the wiki's own tooling.
- A note may feed several articles.
- Titles and slugs name nothing concrete in this house: no hostnames, device IDs, addresses or IP addresses. Name the subject by what it is or does, for example "Netzwerkspeicher" rather than the server's hostname.
@@ -60,14 +60,18 @@ public static class Prompts
You tidy a group of nearby articles of an encyclopedia about one household: the house and its homelab IT. Its readers are everyone who needs to know things about the house, from the owner to guests.
You receive the articles, one per line as `slug | title | category | scope | source count | current parent` (`-` for none).
- - Merge two articles when they are about the same subject, for example the same device under two names. Merge the one with fewer sources into the other. An article you merge away is never the target of another merge.
+ - Merge two articles when they are about the same subject, for example the same device under two names, or an integration and the software it is built on. Merge the one with fewer sources into the other. An article you merge away is never the target of another merge.
- Give an article a parent when the parent is the overview of a broader subject it belongs to, for example a heat pump under the heating system, or a device under the system it is part of. A room is the parent of nothing but its own parts; a device in a room keeps its system as parent.
- - Change a current parent only when another article of the group is clearly the better overview.
+ - When two or more articles of the group belong to a broader subject that has no article, propose that overview as a new article with them as its children, for example "Heizung" over the heat pump, its bus integration and its energy management, or "Beleuchtung" over the switch convention and the light controllers. Propose an overview only for a subject a reader would look up; an overview of overviews follows later from another group.
+ - A room's parent is the overview of its floor, category room, titled by the floor: "Erdgeschoss" (EG), "Obergeschoss" (OG), "Dachgeschoss" (DG) or "Kellergeschoss" (KG). UG, KG and Keller name the same floor, the basement: room articles that differ only in that are the same room and are merged.
+ - Change a current parent only when another article of the group, or an overview you propose, is clearly the better overview.
- Articles about different subjects, and siblings, stay as they are. When nothing needs to change, reply with empty lists.
- - Use only slugs from the list.
+ - Use only slugs from the list, except for the slug of an overview you propose. Children of an overview are slugs from the list.
+ - Titles and slugs name nothing concrete in this house: no hostnames, device IDs, addresses or IP addresses. Name the subject by what it is or does.
+ - Slugs are lowercase ASCII words joined by single hyphens. Categories are room, system, device or topic. scope is one plain sentence on what the article covers.
Reply with JSON only, no prose and no code fence. Shape:
- {"merges":[{"from":"article-slug","into":"article-slug"}],"parents":[{"child":"article-slug","parent":"article-slug"}]}
+ {"merges":[{"from":"article-slug","into":"article-slug"}],"parents":[{"child":"article-slug","parent":"article-slug"}],"overviews":[{"slug":"...","titleDe":"...","titleEn":"...","category":"room|system|device|topic","scope":"...","children":["article-slug"]}]}
""";
/// The generic-examples rule, shared by and .
diff --git a/src/Wiki.Host/Composition/WikiOptions.cs b/src/Wiki.Host/Composition/WikiOptions.cs
index 3bb8366..ec9a9f3 100644
--- a/src/Wiki.Host/Composition/WikiOptions.cs
+++ b/src/Wiki.Host/Composition/WikiOptions.cs
@@ -97,13 +97,13 @@ public sealed class EncyclopediaOptions
public int NeighbourSources { get; set; } = 10;
public int MaxAssignBatchesPerRun { get; set; } = 120;
public int MaxSplitsPerRun { get; set; } = 5;
- public int MaxFoldsPerRun { get; set; } = 5;
+ public int MaxFoldsPerRun { get; set; } = 100;
public int MaxSourcesPerArticle { get; set; } = 120;
public int FoldMinSources { get; set; } = 3;
public int FoldMinChars { get; set; } = 4000;
/// Articles per consolidation group, the seed included.
public int ConsolidateGroupSize { get; set; } = 8;
- public int MaxConsolidationsPerRun { get; set; } = 60;
+ public int MaxConsolidationsPerRun { get; set; } = 150;
// thinking tokens count against max_tokens
public int AssignMaxTokens { get; set; } = 32000;
public int SplitMaxTokens { get; set; } = 32000;
diff --git a/test/Wiki.Application.Tests/ArticlePlanTests.cs b/test/Wiki.Application.Tests/ArticlePlanTests.cs
index 8240c27..70b3fd2 100644
--- a/test/Wiki.Application.Tests/ArticlePlanTests.cs
+++ b/test/Wiki.Application.Tests/ArticlePlanTests.cs
@@ -426,6 +426,91 @@ public class ArticlePlanTests
Assert.True(plan.IsAssigned("n3"));
}
+ private static ArticlePlan WithOverview(params string[] ownSources)
+ {
+ var plan = ArticlePlan.Empty();
+ plan.AddArticle(Article("o", ownSources) with { Overview = true });
+ plan.AddArticle(Article("c", "n9") with { Parent = "o" });
+ foreach (var n in ownSources) plan.Assign(n, ["o"]);
+ plan.Assign("n9", ["c"]);
+ return plan;
+ }
+
+ [Fact]
+ public void RemoveNote_OverviewLosingItsLastOwnSource_Stays()
+ {
+ var plan = WithOverview("n1");
+
+ Assert.Empty(plan.RemoveNote("n1"));
+
+ Assert.Empty(plan.Find("o")!.Sources);
+ Assert.Equal("o", plan.Find("c")!.Parent);
+ }
+
+ [Fact]
+ public void PruneOverviews_WithoutSubArticlesOrOwnSources_Deleted_SlugStaysFree()
+ {
+ var plan = WithOverview();
+ plan.AddArticle(Article("top") with { Overview = true });
+ plan.SetParent("o", "top");
+ plan.AddArticle(Article("old", "n5"));
+ plan.Fold("old", "o", isPublic: false);
+ plan.RemoveNote("n5");
+ plan.RemoveNote("n9");
+
+ // c went with its last note; o and then top have no sub-article left
+ Assert.Equal(["o", "top"], plan.PruneOverviews());
+
+ Assert.Null(plan.Find("o"));
+ Assert.Null(plan.Find("top"));
+ Assert.False(plan.SlugTaken("o"));
+ Assert.DoesNotContain("old", plan.Redirects.Keys);
+ Assert.Contains("old", plan.Retired);
+ }
+
+ [Fact]
+ public void PruneOverviews_WithOwnSources_BecomesAnOrdinaryArticle()
+ {
+ var plan = WithOverview("n1");
+ plan.RemoveNote("n9");
+
+ Assert.Empty(plan.PruneOverviews());
+
+ Assert.False(plan.Find("o")!.Overview);
+ Assert.Equal(["n1"], plan.Find("o")!.Sources);
+ }
+
+ [Fact]
+ public void PruneOverviews_WithSubArticles_Kept()
+ {
+ var plan = WithOverview();
+
+ Assert.Empty(plan.PruneOverviews());
+ Assert.True(plan.Find("o")!.Overview);
+ }
+
+ [Fact]
+ public void ReplaceArticle_KeepsTheOverviewMark()
+ {
+ var plan = WithOverview("n1");
+
+ plan.ReplaceArticle(Article("o", "n1", "n2"));
+
+ Assert.True(plan.Find("o")!.Overview);
+ }
+
+ [Fact]
+ public void Json_OverviewMark_RoundTrips_MissingReadsAsNone()
+ {
+ var plan = ArticlePlanJson.Read(ArticlePlanJson.Write(WithOverview()))!;
+
+ Assert.True(plan.Find("o")!.Overview);
+ Assert.False(plan.Find("c")!.Overview);
+ Assert.False(ArticlePlanJson.Read("""
+ {"version":3,"articles":[{"slug":"a","category":"topic","sources":["n1"]}],"assignments":{"n1":["a"]}}
+ """)!.Find("a")!.Overview);
+ }
+
private sealed class PlannedArticleComparer : IEqualityComparer
{
public bool Equals(PlannedArticle? x, PlannedArticle? y) =>
diff --git a/test/Wiki.Application.Tests/ConsolidateArticlesUseCaseTests.cs b/test/Wiki.Application.Tests/ConsolidateArticlesUseCaseTests.cs
index d718c55..cbade16 100644
--- a/test/Wiki.Application.Tests/ConsolidateArticlesUseCaseTests.cs
+++ b/test/Wiki.Application.Tests/ConsolidateArticlesUseCaseTests.cs
@@ -26,7 +26,7 @@ public class ConsolidateArticlesUseCaseTests
{"merges":[{"from":"vitocal-heat-pump","into":"heat-pump"}],"parents":[{"child":"heat-pump","parent":"heating"},{"child":"radiators","parent":"heating"}]}
""");
- var outcome = await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None);
+ var outcome = await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None);
Assert.NotNull(outcome);
Assert.Equal([new ArticleMerge("vitocal-heat-pump", "heat-pump")], outcome.Merges);
@@ -43,7 +43,7 @@ public class ConsolidateArticlesUseCaseTests
{
var model = new FakeTextModel().Reply("""{"merges":[],"parents":[]}""");
- await UseCase(model).ConsolidateAsync([.. Heating, Article("garden")], CancellationToken.None);
+ await UseCase(model).ConsolidateAsync([.. Heating, Article("garden")], ArticlePlan.Empty(), CancellationToken.None);
Assert.DoesNotContain("garden", model.Calls[0].User);
}
@@ -56,7 +56,7 @@ public class ConsolidateArticlesUseCaseTests
"parents":[{"child":"radiators","parent":"radiators"},{"child":"garden","parent":"heating"},{"child":"radiators","parent":"garden"}]}
""");
- var outcome = await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None);
+ var outcome = await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None);
Assert.Empty(outcome!.Merges);
Assert.Empty(outcome.Parents);
@@ -70,7 +70,7 @@ public class ConsolidateArticlesUseCaseTests
{"from":"radiators","into":"vitocal-heat-pump"},{"from":"vitocal-heat-pump","into":"heating"}],"parents":[]}
""");
- var outcome = await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None);
+ var outcome = await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None);
Assert.Equal([new ArticleMerge("vitocal-heat-pump", "heat-pump")], outcome!.Merges);
}
@@ -84,17 +84,168 @@ public class ConsolidateArticlesUseCaseTests
{"child":"heat-pump","parent":"radiators"},{"child":"heat-pump","parent":"vitocal-heat-pump"}]}
""");
- var outcome = await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None);
+ var outcome = await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None);
Assert.Equal([new ArticleParent("heat-pump", "heating"), new ArticleParent("radiators", "heat-pump")], outcome!.Parents);
}
+ private static string OverviewReply(string slug, params string[] children) => $$"""
+ {"merges":[],"parents":[],"overviews":[{"slug":"{{slug}}","titleDe":"Heizung","titleEn":"Heating","category":"system","scope":"The heating system.","children":[{{string.Join(',', children.Select(c => $"\"{c}\""))}}]}]}
+ """;
+
+ private static readonly PlannedArticle[] Peers =
+ [Article("vitocal", parent: "house"), Article("can-bus", parent: "house"), Article("ems", parent: "house"), Article("garden")];
+
+ private static ArticlePlan PlanOf(params PlannedArticle[] articles)
+ {
+ var plan = ArticlePlan.Empty();
+ foreach (var a in articles) plan.AddArticle(a with { Parent = null });
+ foreach (var a in articles.Where(a => a.Parent is not null)) plan.SetParent(a.Slug, a.Parent);
+ return plan;
+ }
+
+ private static readonly ArticlePlan PeersPlan = PlanOf([Article("house"), .. Peers]);
+
+ [Fact]
+ public async Task Consolidate_Overview_OverGroupMembers()
+ {
+ var model = new FakeTextModel().Reply(OverviewReply("heizung", "vitocal", "can-bus", "ems"));
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, PeersPlan, CancellationToken.None);
+
+ var overview = Assert.Single(outcome!.Overviews);
+ var expected = new PlannedArticle("heizung", "Heizung", "Heating", ArticleCategory.System, "The heating system.", [], "house");
+ Assert.Equal(expected, overview.Article with { Sources = expected.Sources });
+ Assert.Empty(overview.Article.Sources);
+ Assert.Equal(["vitocal", "can-bus", "ems"], overview.Children);
+ Assert.Empty(outcome.Parents);
+ }
+
+ [Theory]
+ [InlineData("house", "vitocal", "garden")]
+ [InlineData("house", "vitocal", "can-bus", "pond")]
+ [InlineData(null, "vitocal", "pond")]
+ [InlineData(null, "vitocal", "pond", "garden")]
+ public async Task Consolidate_Overview_TakesTheParentMostChildrenShare_TieNone(string? parent, params string[] children)
+ {
+ var model = new FakeTextModel().Reply(OverviewReply("heizung", children));
+
+ var outcome = await UseCase(model).ConsolidateAsync([Article("pond", parent: "yard"), Peers[0], Peers[1], Peers[3]], PeersPlan, CancellationToken.None);
+
+ Assert.Equal(parent, Assert.Single(outcome!.Overviews).Article.Parent);
+ }
+
+ [Theory]
+ [InlineData("Heizung", "vitocal", "can-bus")]
+ [InlineData("heizung", "vitocal")]
+ [InlineData("heizung", "vitocal", "outsider")]
+ [InlineData("heizung", "vitocal", "vitocal")]
+ public async Task Consolidate_Overview_BadSlugOrFewerThanTwoGroupChildren_Dropped(string slug, params string[] children)
+ {
+ var model = new FakeTextModel().Reply(OverviewReply(slug, children));
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, PeersPlan, CancellationToken.None);
+
+ Assert.Empty(outcome!.Overviews);
+ }
+
+ [Fact]
+ public async Task Consolidate_Overview_MissingTitleOrCategory_Dropped()
+ {
+ var model = new FakeTextModel().Reply("""
+ {"overviews":[{"slug":"heizung","titleDe":"Heizung","category":"system","scope":"s","children":["vitocal","ems"]},
+ {"slug":"heizung-2","titleDe":"Heizung","titleEn":"Heating","category":"floor","scope":"s","children":["vitocal","ems"]}]}
+ """);
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, PeersPlan, CancellationToken.None);
+
+ Assert.Empty(outcome!.Overviews);
+ }
+
+ [Fact]
+ public async Task Consolidate_Overview_NamingALiveArticle_ParentsTheChildrenThere()
+ {
+ var plan = PlanOf([Article("house"), Article("heating"), .. Peers]);
+ var model = new FakeTextModel().Reply("""
+ {"overviews":[{"slug":"heating","titleDe":"H","titleEn":"H","category":"system","scope":"s","children":["vitocal","can-bus"]}]}
+ """);
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, plan, CancellationToken.None);
+
+ Assert.Empty(outcome!.Overviews);
+ Assert.Equal([new ArticleParent("vitocal", "heating"), new ArticleParent("can-bus", "heating")], outcome.Parents);
+ }
+
+ [Fact]
+ public async Task Consolidate_Overview_NamingARedirect_Dropped()
+ {
+ // the redirect target is an article the model never saw in this group
+ var plan = PlanOf([Article("house"), Article("heating"), Article("old-heating"), .. Peers]);
+ plan.Fold("old-heating", "heating", false);
+ var model = new FakeTextModel().Reply(OverviewReply("old-heating", "ems", "garden"));
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, plan, CancellationToken.None);
+
+ Assert.Empty(outcome!.Overviews);
+ Assert.Empty(outcome.Parents);
+ }
+
+ [Fact]
+ public async Task Consolidate_Overview_SlugProposedTwice_SecondDropped()
+ {
+ var model = new FakeTextModel().Reply("""
+ {"overviews":[{"slug":"heizung","titleDe":"Heizung","titleEn":"Heating","category":"system","scope":"s","children":["vitocal","can-bus"]},
+ {"slug":"heizung","titleDe":"Heizung","titleEn":"Heating","category":"system","scope":"s","children":["ems","garden"]}]}
+ """);
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, PeersPlan, CancellationToken.None);
+
+ Assert.Equal(["vitocal", "can-bus"], Assert.Single(outcome!.Overviews).Children);
+ }
+
+ [Fact]
+ public async Task Consolidate_Overview_RetiredSlug_Dropped()
+ {
+ var plan = PlanOf([Article("house"), Article("gone", sources: 1), .. Peers]);
+ plan.RemoveNote("gone-n0");
+ var model = new FakeTextModel().Reply(OverviewReply("gone", "vitocal", "can-bus"));
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, plan, CancellationToken.None);
+
+ Assert.Empty(outcome!.Overviews);
+ Assert.Empty(outcome.Parents);
+ }
+
+ [Fact]
+ public async Task Consolidate_Overview_ChildClaimedOnce_FollowsMerges()
+ {
+ var model = new FakeTextModel().Reply("""
+ {"merges":[{"from":"ems","into":"vitocal"}],"parents":[{"child":"garden","parent":"can-bus"}],
+ "overviews":[{"slug":"heizung","titleDe":"Heizung","titleEn":"Heating","category":"system","scope":"s","children":["vitocal","ems","can-bus","garden"]},
+ {"slug":"heizung","titleDe":"Heizung","titleEn":"Heating","category":"system","scope":"s","children":["vitocal","can-bus"]},
+ {"slug":"technik","titleDe":"Technik","titleEn":"Technology","category":"topic","scope":"s","children":["vitocal","can-bus"]}]}
+ """);
+
+ var outcome = await UseCase(model).ConsolidateAsync(Peers, PeersPlan, CancellationToken.None);
+
+ // ems stands for vitocal; garden has a parent entry already
+ Assert.Equal(["vitocal", "can-bus"], Assert.Single(outcome!.Overviews).Children);
+ }
+
+ [Fact]
+ public async Task Consolidate_OnlyOverviews_IsUsable()
+ {
+ var model = new FakeTextModel().Reply("""{"overviews":[]}""");
+
+ Assert.NotNull(await UseCase(model).ConsolidateAsync(Peers, PeersPlan, CancellationToken.None));
+ }
+
[Fact]
public async Task Consolidate_NothingToDo_IsUsable()
{
var model = new FakeTextModel().Reply("""{"merges":[]}""");
- var outcome = await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None);
+ var outcome = await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None);
Assert.NotNull(outcome);
Assert.Empty(outcome.Merges);
@@ -109,7 +260,7 @@ public class ConsolidateArticlesUseCaseTests
{
var model = new FakeTextModel().Reply(reply, truncated);
- Assert.Null(await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None));
+ Assert.Null(await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None));
}
[Fact]
@@ -117,7 +268,7 @@ public class ConsolidateArticlesUseCaseTests
{
var model = new FakeTextModel().Throw(new HttpRequestException("invalid", null, HttpStatusCode.BadRequest));
- Assert.Null(await UseCase(model).ConsolidateAsync(Heating, CancellationToken.None));
+ Assert.Null(await UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None));
}
[Fact]
@@ -125,6 +276,6 @@ public class ConsolidateArticlesUseCaseTests
{
var model = new FakeTextModel().Throw(new HttpRequestException("bad gateway", null, HttpStatusCode.BadGateway));
- await Assert.ThrowsAsync(() => UseCase(model).ConsolidateAsync(Heating, CancellationToken.None));
+ await Assert.ThrowsAsync(() => UseCase(model).ConsolidateAsync(Heating, ArticlePlan.Empty(), CancellationToken.None));
}
}
diff --git a/test/Wiki.Application.Tests/EncyclopediaReconcilerTests.cs b/test/Wiki.Application.Tests/EncyclopediaReconcilerTests.cs
index 5a60e53..9dff202 100644
--- a/test/Wiki.Application.Tests/EncyclopediaReconcilerTests.cs
+++ b/test/Wiki.Application.Tests/EncyclopediaReconcilerTests.cs
@@ -1181,6 +1181,143 @@ public class EncyclopediaReconcilerTests
Assert.Equal("kitchen", Plan.Find("heating")!.Parent);
}
+ private static string OverviewConsolidation(string slug, params string[] children) =>
+ JsonSerializer.Serialize(new
+ {
+ merges = Array.Empty