Compare commits

13 Commits
Author SHA1 Message Date
k.renner fbd5a2529a Merge pull request 'fix(auth): the wiki-index BasicAuth credentials read the API again' (#15) from feat/basic-auth-machines into main
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #15
2026-10-19 14:13:52 +00:00
hermes-bot 50fb31b950 fix(auth): the wiki-index BasicAuth credentials read the API again
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
#14 put the read API behind a session or an API key. External agents
reach the wiki through wiki-index.renner.uno with BasicAuth only, and
since #14 they got 401: a regression.

The service now checks `Authorization: Basic` itself against the same
htpasswd lines Traefik's wiki-api-basic-auth middleware uses
(Auth:Basic:UsersPath: a file, or a mounted Secret directory where every
visible key is read, as Traefik reads every key of an Opaque Secret).
This is a credential check in the service, not trust in a proxy: the
header proves nothing unless the password matches the hash.

- Hash formats Traefik accepts: bcrypt ($2a$/$2b$/$2y$, BCrypt.Net-Next
  4.2.1), Apache MD5 ($apr1$, implemented here and checked against the
  Traefik docs example and `openssl passwd -apr1`), SHA1 ({SHA}). Any
  other format authenticates nobody.
- The identity is a machine named after the BasicAuth user.
- Policy "read" accepts it; "write" stays API-key only, and the UI stays
  OIDC only.
- A verified (user, password, hash) triple is cached in memory, so bcrypt
  costs once per process.
- With no UsersPath configured the scheme authenticates nobody, so this
  image is safe before the deployment mounts the Secret.

Tests: all three hash formats read the API; a wrong password, an unknown
user, an empty password and a user found only in a hidden Secret entry
are 401; BasicAuth cannot write and does not open the UI.
2026-10-19 14:06:55 +00:00
k.renner 311f59d82a Merge pull request 'feat(auth): humans sign in with OIDC; no header names a user' (#14) from feat/oidc-login into main
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #14
2026-10-19 10:13:28 +00:00
hermes-bot 4820bc9f6d fix(auth): redirect URI from the request host, no PublicOrigin setting
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
Auth:Oidc:PublicOrigin was a hand copy of the Ingress host. Traefik routes on
the Host header and Authentik matches redirect URIs strictly, so the URI is
https://<request host>/signin-oidc; https because TLS ends at Traefik and
browsers only reach the wiki over https.
2026-10-19 09:45:56 +00:00
hermes-bot 0ecf1aaac5 feat(auth): humans sign in with OIDC; no header names a user
ci/woodpecker/push/test Pipeline was successful
The ProxyHeader scheme took the username from X-Authentik-Username for any
peer inside Auth:ReverseProxy:TrustedProxies. The deployment set that to
the whole pod CIDR (a hand copy of the Talos podSubnets), so every pod the
network policies let reach :8080 (Hermes, Gitea, Traefik, all of ai and
monitoring) could claim any human's name on the Facts page and on saved
answers. Authentik's proxy provider sends no signed identity header, so the
header cannot be verified at all.

Humans now sign in with OpenID Connect against Authentik: a public client
with PKCE, the username is preferred_username from the ID token, whose
signature is checked against the provider's JWKS. Nothing about the
network path or the request headers names a user any more.

- Auth:Oidc:Authority, ClientId and PublicOrigin are required; the host
  refuses to start without them. PublicOrigin builds the redirect URI,
  because TLS ends at Traefik and the request scheme is http.
- UI pages, static assets and the Blazor hub need a session (fallback
  policy); an anonymous browser is redirected to the provider.
- The read API (/api/v1 reads, /ask, /questions, /rank, /status) needs a
  session or an API key (policy "read") and answers 401, never a redirect.
  Before, it was anonymous and relied on the gate, BasicAuth or network
  policy in front.
- Writes still need an API key (policy "write").
- /health, /ready and the HMAC-signed /webhook stay anonymous.
- ProxyHeaderAuthenticationHandler and Auth:ReverseProxy are removed.

Tests: an in-process OpenID provider issues RS256 ID tokens. They cover
the PKCE challenge and redirect URI, the full code flow ending in a
session named after the token's user, rejection of a token signed by a
foreign key, the Authentik header signing in nobody, and 401s for
credential-less API calls.
2026-10-19 09:44:58 +00:00
k.renner 53c03a8e53 Merge pull request 'feat(facts): pictures as part of a statement, kept next to the raw source' (#13) from feat/fact-images into main
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
ci/woodpecker/manual/test Pipeline was successful
Reviewed-on: #13
2026-09-25 07:55:43 +00:00
claude-botandClaude Fable 5.1 c94afb004c feat(facts): pictures as part of a statement, kept next to the raw source
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
A statement may carry images (a type label, for instance). The drafter
sees them as image blocks and transcribes what is printed; the deposit
stores each picture beside the note as raw/<stem>-<n>.<ext>, lists it in
the frontmatter and embeds it in the body. INoteStore gains a binary
create and a browse URL, so the raw page can link to the pictures. The
page takes pictures from a file picker (camera on phones) or from paste,
resized in the browser to 1600px.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-25 09:52:16 +02:00
k.renner 125911a4e6 Merge pull request 'feat(facts): one text box; the drafter turns a statement into a raw source' (#12) from feat/fact-from-statement into main
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #12
2026-09-25 06:57:04 +00:00
claude-botandClaude Fable 5.1 8b8e56d1b2 feat(facts): one text box; the drafter turns a statement into a raw source
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
IFactDrafter port: a statement in, title/content/tags out. Anthropic
implements it over the transport the answer model already uses (extracted
into MessagesTransport). RecordStatementUseCase drafts, sets the person as
source, keeps the statement verbatim in the body and deposits through the
existing use case. The Facts page is the text box and the receipt; the API
shape for machines is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-25 08:55:45 +02:00
k.renner e0a0cf3085 Merge pull request 'feat(ui): Facts page deposits a raw source under the signed-in user' (#11) from feat/facts-page into main
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #11
2026-09-25 06:26:41 +00:00
claude-botandClaude Fable 5.1 09ec2cf1d9 feat(ui): Facts page deposits a raw source under the signed-in user
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
Same DepositFactUseCase as POST /api/v1/facts, attributed as a human
identity from the Authentik proxy header. Cascading authentication state
carries the principal into the Blazor circuit; without a signed-in user
the form is replaced by a notice. Lists the raw sources awaiting the
nightly distill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-25 08:24:00 +02:00
k.renner 4cd8ef1dd4 Merge pull request 'fix(ask): judge every question in one request' (#10) from fix/judge-one-request into main
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #10
2026-09-25 06:01:55 +00:00
claude-botandClaude Fable 5.1 347ac9d7a6 fix(ask): judge every question in one request
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
The state dominates a System One request and each question is scored in
isolation against it, so sending context choices separately only re-sent
the state per request and added round trips. A live ask carried 22,957
input tokens over two requests for one question; one request carries it
once. Drops the now unused AskOptions.MaxParallel.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-25 08:00:44 +02:00
43 changed files with 1654 additions and 375 deletions
+8 -1
View File
@@ -15,7 +15,14 @@ test/Wiki.Contracts one abstract suite per port; every adapter passes its sui
Adapters are selected in `appsettings.json` under `Wiki:`; secrets come from environment
variables (`Wiki__NoteStore__Gitea__Token`, `Wiki__Webhook__Secret`, `Auth__ApiKeys__<name>`,
`Wiki__Judges__Models__<name>__ApiKey`, `Wiki__AnswerModels__Models__<name>__ApiKey`).
`Wiki__Judges__Models__<name>__ApiKey`, `Wiki__AnswerModels__Models__<name>__ApiKey`). Each
answer-model entry also provides the fact drafter of the same name (`/facts`).
Humans sign in with OpenID Connect (public client, PKCE): `Auth__Oidc__Authority` and
`Auth__Oidc__ClientId` are required; the redirect URI is `https://<request host>/signin-oidc`. Machines send `X-Api-Key`, or BasicAuth
checked against the htpasswd lines in `Auth__Basic__UsersPath`, a file or a mounted Secret directory
(bcrypt, apr1 or SHA1; read API only). Only `/health`, `/ready` and the HMAC-signed `/webhook`
answer without either.
## Run
@@ -1,8 +1,3 @@
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using Wiki.Core.Model;
using Wiki.Core.Ports;
@@ -10,216 +5,28 @@ namespace Wiki.Adapters.Anthropic;
/// <summary>
/// Answers questions via Anthropic's Messages API, streamed over server-sent events. One HTTP
/// request per <see cref="AnswerAsync"/> call, retried on 429/529 with the server's Retry-After
/// (or exponential backoff), never retried for anything else.
/// request per <see cref="AnswerAsync"/> call; see <see cref="MessagesTransport"/> for retries.
/// </summary>
public sealed class AnthropicAnswerModel : IAnswerModel
{
private const string AnthropicVersion = "2023-06-01";
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
private readonly HttpClient _http;
private readonly AnthropicOptions _options;
private readonly Func<TimeSpan, CancellationToken, Task> _delay;
private readonly MessagesTransport _transport;
public AnthropicAnswerModel(HttpClient http, AnthropicOptions options, Func<TimeSpan, CancellationToken, Task>? delay = null)
{
_http = http;
_options = options;
_delay = delay ?? Task.Delay;
_transport = new MessagesTransport(http, options, delay);
ModelId = options.ModelId;
}
public string ModelId => _options.ModelId;
public string ModelId { get; }
public string Name => $"anthropic:{ModelId}";
public async IAsyncEnumerable<AnswerChunk> AnswerAsync(AnswerRequest request, [EnumeratorCancellation] CancellationToken ct)
public IAsyncEnumerable<AnswerChunk> AnswerAsync(AnswerRequest request, CancellationToken ct)
{
ArgumentNullException.ThrowIfNull(request);
ct.ThrowIfCancellationRequested();
var (system, user) = AnswerPrompt.Build(request);
var body = BuildRequestBody(system, user);
var stopwatch = Stopwatch.StartNew();
using var response = await PostWithRetryAsync(body, ct).ConfigureAwait(false);
var stream = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false);
await using (stream.ConfigureAwait(false))
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
long inputTokens = 0;
long outputTokens = 0;
string? eventName = null;
string? line;
while ((line = await reader.ReadLineAsync(ct).ConfigureAwait(false)) is not null)
{
if (line.Length == 0)
{
eventName = null;
continue;
}
if (line.StartsWith("event: ", StringComparison.Ordinal))
{
eventName = line["event: ".Length..];
continue;
}
if (!line.StartsWith("data: ", StringComparison.Ordinal))
{
continue;
}
var data = line["data: ".Length..];
using var payload = JsonDocument.Parse(data);
var root = payload.RootElement;
switch (eventName)
{
case "message_start":
inputTokens = root.GetProperty("message").GetProperty("usage").GetProperty("input_tokens").GetInt64();
break;
case "content_block_delta":
var delta = root.GetProperty("delta");
if (delta.GetProperty("type").GetString() == "text_delta")
{
yield return new AnswerChunk.Text(delta.GetProperty("text").GetString() ?? "");
}
break;
case "message_delta":
outputTokens = root.GetProperty("usage").GetProperty("output_tokens").GetInt64();
break;
case "error":
throw new HttpRequestException($"anthropic: {root.GetProperty("error").GetProperty("message").GetString()}");
// message_stop, ping, content_block_start, content_block_stop: nothing to record.
default:
break;
}
}
stopwatch.Stop();
yield return new AnswerChunk.Done(ModelId, inputTokens, outputTokens, stopwatch.Elapsed);
}
return _transport.StreamAsync(system, user, ModelId, ct);
}
public async Task<AdapterHealth> CheckAsync(CancellationToken ct)
{
var stopwatch = Stopwatch.StartNew();
try
{
if (string.IsNullOrWhiteSpace(_options.ApiKey))
{
throw new InvalidOperationException("Anthropic API key is not configured.");
}
using var requestMessage = new HttpRequestMessage(HttpMethod.Get, $"{_options.BaseUrl}/v1/models");
requestMessage.Headers.Add("x-api-key", _options.ApiKey);
requestMessage.Headers.Add("anthropic-version", AnthropicVersion);
using var response = await _http.SendAsync(requestMessage, ct).ConfigureAwait(false);
var responseBody = await response.Content.ReadAsStringAsync(ct).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
var truncated = responseBody.Length > 500 ? responseBody[..500] : responseBody;
return new AdapterHealth(Name, false, $"HTTP {(int)response.StatusCode}: {truncated}", stopwatch.Elapsed, DateTimeOffset.UtcNow);
}
using var document = JsonDocument.Parse(responseBody);
var count = document.RootElement.GetProperty("data").GetArrayLength();
return new AdapterHealth(Name, true, $"{count} models", stopwatch.Elapsed, DateTimeOffset.UtcNow);
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException or InvalidOperationException)
{
return new AdapterHealth(Name, false, ex.Message, stopwatch.Elapsed, DateTimeOffset.UtcNow);
}
}
private JsonObject BuildRequestBody(string system, string user) => new()
{
["model"] = _options.Model,
["max_tokens"] = _options.MaxTokens,
["system"] = system,
["messages"] = new JsonArray
{
new JsonObject
{
["role"] = "user",
["content"] = user,
},
},
["stream"] = true,
};
private async Task<HttpResponseMessage> PostWithRetryAsync(JsonObject body, CancellationToken ct)
{
// A missing key is a runtime condition the status page shows, not a reason the host cannot start.
if (string.IsNullOrWhiteSpace(_options.ApiKey))
{
throw new InvalidOperationException("Anthropic API key is not configured.");
}
var json = body.ToJsonString(JsonOptions);
var delay = TimeSpan.FromSeconds(1);
for (var attempt = 0; ; attempt++)
{
using var requestMessage = new HttpRequestMessage(HttpMethod.Post, $"{_options.BaseUrl}/v1/messages")
{
Content = new StringContent(json, Encoding.UTF8, "application/json"),
};
requestMessage.Headers.Add("x-api-key", _options.ApiKey);
requestMessage.Headers.Add("anthropic-version", AnthropicVersion);
var response = await _http.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, ct).ConfigureAwait(false);
if (response.IsSuccessStatusCode)
{
return response;
}
var statusCode = (int)response.StatusCode;
if ((statusCode == 429 || statusCode == 529) && attempt < _options.MaxRetries)
{
var wait = RetryAfter(response) ?? delay;
response.Dispose();
await _delay(wait, ct).ConfigureAwait(false);
delay = TimeSpan.FromSeconds(Math.Min(delay.TotalSeconds * 2, 30));
continue;
}
var errorBody = await response.Content.ReadAsStringAsync(ct).ConfigureAwait(false);
response.Dispose();
var truncated = errorBody.Length > 500 ? errorBody[..500] : errorBody;
throw new HttpRequestException($"anthropic: HTTP {statusCode}: {truncated}");
}
}
private static TimeSpan? RetryAfter(HttpResponseMessage response)
{
var header = response.Headers.RetryAfter;
if (header is null)
{
return null;
}
if (header.Delta is { } delta)
{
return delta;
}
if (header.Date is { } date)
{
var diff = date - DateTimeOffset.UtcNow;
return diff > TimeSpan.Zero ? diff : TimeSpan.Zero;
}
return null;
}
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => _transport.CheckAsync(Name, ct);
}
@@ -0,0 +1,55 @@
using System.Text;
using System.Text.Json.Nodes;
using Wiki.Core.Model;
using Wiki.Core.Ports;
namespace Wiki.Adapters.Anthropic;
/// <summary>Turns a statement into raw-source fields with one Messages API call. Same transport and options as the answer model.</summary>
public sealed class AnthropicFactDrafter : IFactDrafter
{
private readonly MessagesTransport _transport;
public AnthropicFactDrafter(HttpClient http, AnthropicOptions options, Func<TimeSpan, CancellationToken, Task>? delay = null)
{
_transport = new MessagesTransport(http, options, delay);
ModelId = options.ModelId;
}
public string ModelId { get; }
public string Name => $"anthropic-facts:{ModelId}";
public async Task<FactDraft> DraftAsync(FactStatement statement, CancellationToken ct)
{
ArgumentNullException.ThrowIfNull(statement);
if (string.IsNullOrWhiteSpace(statement.Text))
throw new ArgumentException("the statement is empty", nameof(statement));
var (system, user) = FactDraftPrompt.Build(statement);
var content = new JsonArray();
foreach (var a in statement.Attachments)
{
content.Add(new JsonObject
{
["type"] = "image",
["source"] = new JsonObject
{
["type"] = "base64",
["media_type"] = a.MediaType,
["data"] = Convert.ToBase64String(a.Bytes),
},
});
}
content.Add(new JsonObject { ["type"] = "text", ["text"] = user });
var text = new StringBuilder();
await foreach (var chunk in _transport.StreamAsync(system, content, ModelId, ct).ConfigureAwait(false))
{
if (chunk is AnswerChunk.Text t) text.Append(t.Delta);
}
return FactDraftPrompt.Parse(text.ToString());
}
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => _transport.CheckAsync(Name, ct);
}
@@ -0,0 +1,71 @@
using System.Globalization;
using System.Text;
using System.Text.Json;
using Wiki.Core.Model;
namespace Wiki.Adapters.Anthropic;
/// <summary>Builds the prompt that turns a statement into raw-source fields, and parses the model's JSON. Pure.</summary>
public static class FactDraftPrompt
{
private const string SystemPrompt =
"You turn one statement a person makes about their private homelab into the fields of a raw fact record for a wiki. " +
"Reply with a single JSON object and nothing else: {\"title\": string, \"content\": string, \"tags\": [string]}. " +
"title: a short noun phrase in the language of the statement, no trailing period. " +
"content: the statement restated faithfully as complete sentences in the language of the statement; keep every detail and every exception, add nothing that was not said, at least 40 characters. " +
"tags: one to five lowercase ASCII slugs (a-z, 0-9, hyphen) naming the topics, English. " +
"Images attached to the statement are part of it: transcribe every identifier printed on them exactly as printed (manufacturer, model, type, serial number, ratings, dates), mark characters you cannot read as [unreadable], never guess. " +
"No markdown, no code fences, no commentary.";
public static (string System, string User) Build(FactStatement statement)
{
ArgumentNullException.ThrowIfNull(statement);
var user = new StringBuilder();
user.Append("Stated by ").Append(statement.By.Name)
.Append(" on ").Append(statement.StatedAt.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)).Append(":\n\n");
user.Append(statement.Text);
if (statement.Attachments.Count > 0)
user.Append("\n\n(").Append(statement.Attachments.Count.ToString(CultureInfo.InvariantCulture)).Append(" image(s) attached.)");
return (SystemPrompt, user.ToString());
}
/// <summary>The draft in the model's reply. Tolerates text around the JSON object; rejects anything without title and content.</summary>
public static FactDraft Parse(string text)
{
ArgumentNullException.ThrowIfNull(text);
var start = text.IndexOf('{');
var end = text.LastIndexOf('}');
if (start < 0 || end <= start)
throw new InvalidOperationException("the drafter returned no JSON object");
try
{
using var document = JsonDocument.Parse(text[start..(end + 1)]);
var root = document.RootElement;
var title = root.TryGetProperty("title", out var t) && t.ValueKind == JsonValueKind.String ? t.GetString()?.Trim() : null;
var content = root.TryGetProperty("content", out var c) && c.ValueKind == JsonValueKind.String ? c.GetString()?.Trim() : null;
var tags = root.TryGetProperty("tags", out var g) && g.ValueKind == JsonValueKind.Array
? g.EnumerateArray().Where(x => x.ValueKind == JsonValueKind.String).Select(x => Slug(x.GetString() ?? "")).Where(s => s.Length > 0).Distinct().Take(5).ToList()
: [];
if (string.IsNullOrWhiteSpace(title) || string.IsNullOrWhiteSpace(content))
throw new InvalidOperationException("the drafter returned no title or content");
return new FactDraft(title, content, tags);
}
catch (JsonException e)
{
throw new InvalidOperationException($"the drafter returned malformed JSON: {e.Message}", e);
}
}
private static string Slug(string tag)
{
var sb = new StringBuilder(tag.Length);
var dash = false;
foreach (var ch in tag.ToLowerInvariant())
{
if (ch is (>= 'a' and <= 'z') or (>= '0' and <= '9')) { sb.Append(ch); dash = false; }
else if (!dash) { sb.Append('-'); dash = true; }
}
return sb.ToString().Trim('-');
}
}
@@ -0,0 +1,221 @@
using System.Diagnostics;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using Wiki.Core.Model;
namespace Wiki.Adapters.Anthropic;
/// <summary>
/// One streamed Messages API call, shared by every port the adapter serves. Retried on 429/529
/// with the server's Retry-After (or exponential backoff), never retried for anything else.
/// </summary>
internal sealed class MessagesTransport
{
private const string AnthropicVersion = "2023-06-01";
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
private readonly HttpClient _http;
private readonly AnthropicOptions _options;
private readonly Func<TimeSpan, CancellationToken, Task> _delay;
public MessagesTransport(HttpClient http, AnthropicOptions options, Func<TimeSpan, CancellationToken, Task>? delay)
{
_http = http;
_options = options;
_delay = delay ?? Task.Delay;
}
/// <summary>One user turn of text.</summary>
public IAsyncEnumerable<AnswerChunk> StreamAsync(string system, string user, string modelId, CancellationToken ct) =>
StreamAsync(system, JsonValue.Create(user), modelId, ct);
/// <summary>One user turn of content blocks (text, images).</summary>
public async IAsyncEnumerable<AnswerChunk> StreamAsync(string system, JsonNode userContent, string modelId, [EnumeratorCancellation] CancellationToken ct)
{
ct.ThrowIfCancellationRequested();
var body = BuildRequestBody(system, userContent);
var stopwatch = Stopwatch.StartNew();
using var response = await PostWithRetryAsync(body, ct).ConfigureAwait(false);
var stream = await response.Content.ReadAsStreamAsync(ct).ConfigureAwait(false);
await using (stream.ConfigureAwait(false))
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
long inputTokens = 0;
long outputTokens = 0;
string? eventName = null;
string? line;
while ((line = await reader.ReadLineAsync(ct).ConfigureAwait(false)) is not null)
{
if (line.Length == 0)
{
eventName = null;
continue;
}
if (line.StartsWith("event: ", StringComparison.Ordinal))
{
eventName = line["event: ".Length..];
continue;
}
if (!line.StartsWith("data: ", StringComparison.Ordinal))
{
continue;
}
var data = line["data: ".Length..];
using var payload = JsonDocument.Parse(data);
var root = payload.RootElement;
switch (eventName)
{
case "message_start":
inputTokens = root.GetProperty("message").GetProperty("usage").GetProperty("input_tokens").GetInt64();
break;
case "content_block_delta":
var delta = root.GetProperty("delta");
if (delta.GetProperty("type").GetString() == "text_delta")
{
yield return new AnswerChunk.Text(delta.GetProperty("text").GetString() ?? "");
}
break;
case "message_delta":
outputTokens = root.GetProperty("usage").GetProperty("output_tokens").GetInt64();
break;
case "error":
throw new HttpRequestException($"anthropic: {root.GetProperty("error").GetProperty("message").GetString()}");
// message_stop, ping, content_block_start, content_block_stop: nothing to record.
default:
break;
}
}
stopwatch.Stop();
yield return new AnswerChunk.Done(modelId, inputTokens, outputTokens, stopwatch.Elapsed);
}
}
public async Task<AdapterHealth> CheckAsync(string name, CancellationToken ct)
{
var stopwatch = Stopwatch.StartNew();
try
{
if (string.IsNullOrWhiteSpace(_options.ApiKey))
{
throw new InvalidOperationException("Anthropic API key is not configured.");
}
using var requestMessage = new HttpRequestMessage(HttpMethod.Get, $"{_options.BaseUrl}/v1/models");
requestMessage.Headers.Add("x-api-key", _options.ApiKey);
requestMessage.Headers.Add("anthropic-version", AnthropicVersion);
using var response = await _http.SendAsync(requestMessage, ct).ConfigureAwait(false);
var responseBody = await response.Content.ReadAsStringAsync(ct).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
var truncated = responseBody.Length > 500 ? responseBody[..500] : responseBody;
return new AdapterHealth(name, false, $"HTTP {(int)response.StatusCode}: {truncated}", stopwatch.Elapsed, DateTimeOffset.UtcNow);
}
using var document = JsonDocument.Parse(responseBody);
var count = document.RootElement.GetProperty("data").GetArrayLength();
return new AdapterHealth(name, true, $"{count} models", stopwatch.Elapsed, DateTimeOffset.UtcNow);
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException or InvalidOperationException)
{
return new AdapterHealth(name, false, ex.Message, stopwatch.Elapsed, DateTimeOffset.UtcNow);
}
}
private JsonObject BuildRequestBody(string system, JsonNode userContent) => new()
{
["model"] = _options.Model,
["max_tokens"] = _options.MaxTokens,
["system"] = system,
["messages"] = new JsonArray
{
new JsonObject
{
["role"] = "user",
["content"] = userContent,
},
},
["stream"] = true,
};
private async Task<HttpResponseMessage> PostWithRetryAsync(JsonObject body, CancellationToken ct)
{
// A missing key is a runtime condition the status page shows, not a reason the host cannot start.
if (string.IsNullOrWhiteSpace(_options.ApiKey))
{
throw new InvalidOperationException("Anthropic API key is not configured.");
}
var json = body.ToJsonString(JsonOptions);
var delay = TimeSpan.FromSeconds(1);
for (var attempt = 0; ; attempt++)
{
using var requestMessage = new HttpRequestMessage(HttpMethod.Post, $"{_options.BaseUrl}/v1/messages")
{
Content = new StringContent(json, Encoding.UTF8, "application/json"),
};
requestMessage.Headers.Add("x-api-key", _options.ApiKey);
requestMessage.Headers.Add("anthropic-version", AnthropicVersion);
var response = await _http.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, ct).ConfigureAwait(false);
if (response.IsSuccessStatusCode)
{
return response;
}
var statusCode = (int)response.StatusCode;
if ((statusCode == 429 || statusCode == 529) && attempt < _options.MaxRetries)
{
var wait = RetryAfter(response) ?? delay;
response.Dispose();
await _delay(wait, ct).ConfigureAwait(false);
delay = TimeSpan.FromSeconds(Math.Min(delay.TotalSeconds * 2, 30));
continue;
}
var errorBody = await response.Content.ReadAsStringAsync(ct).ConfigureAwait(false);
response.Dispose();
var truncated = errorBody.Length > 500 ? errorBody[..500] : errorBody;
throw new HttpRequestException($"anthropic: HTTP {statusCode}: {truncated}");
}
}
private static TimeSpan? RetryAfter(HttpResponseMessage response)
{
var header = response.Headers.RetryAfter;
if (header is null)
{
return null;
}
if (header.Delta is { } delta)
{
return delta;
}
if (header.Date is { } date)
{
var diff = date - DateTimeOffset.UtcNow;
return diff > TimeSpan.Zero ? diff : TimeSpan.Zero;
}
return null;
}
}
@@ -39,4 +39,38 @@ public static class ServiceCollectionExtensions
return services;
}
/// <summary>
/// Registers an <see cref="IFactDrafter"/> backed by Anthropic's Messages API, keyed by
/// <paramref name="name"/>. Usually registered alongside the answer model of the same name.
/// </summary>
public static IServiceCollection AddAnthropicFactDrafter(
this IServiceCollection services,
string name,
Action<AnthropicOptions> configure)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentException.ThrowIfNullOrWhiteSpace(name);
ArgumentNullException.ThrowIfNull(configure);
var options = new AnthropicOptions();
configure(options);
options.Validate();
var httpClientName = $"anthropic-facts:{name}";
services.AddHttpClient(httpClientName, client =>
{
client.Timeout = options.Timeout;
});
services.AddKeyedSingleton<IFactDrafter>(name, (sp, _) =>
{
var httpClientFactory = sp.GetRequiredService<IHttpClientFactory>();
var client = httpClientFactory.CreateClient(httpClientName);
return new AnthropicFactDrafter(client, options);
});
return services;
}
}
+8 -2
View File
@@ -79,13 +79,19 @@ public sealed class GiteaRepository : INoteStore
}
/// <summary>POST contents: Gitea answers 422 when the path already exists on the branch.</summary>
public async Task<StoreWrite> CreateAsync(string path, string text, string message, CancellationToken ct)
public Task<StoreWrite> CreateAsync(string path, string text, string message, CancellationToken ct) =>
CreateAsync(path, Encoding.UTF8.GetBytes(text), message, ct);
public Uri? BrowseUrl(string path) =>
new($"{_options.BaseUrl.TrimEnd('/')}/{_options.Repository}/src/branch/{_options.Branch}/{path}");
public async Task<StoreWrite> CreateAsync(string path, byte[] content, string message, CancellationToken ct)
{
var repo = GiteaUrls.EncodeSegments(_options.Repository);
var encodedPath = GiteaUrls.EncodeSegments(path);
var request = new
{
content = Convert.ToBase64String(Encoding.UTF8.GetBytes(text)),
content = Convert.ToBase64String(content),
message,
branch = _options.Branch,
};
+11 -29
View File
@@ -20,7 +20,7 @@ public enum ContextScope
Documents,
}
public sealed record AskOptions(int K = 6, int DocChars = 12_000, int StateChars = 60_000, int MaxParallel = 8);
public sealed record AskOptions(int K = 6, int DocChars = 12_000, int StateChars = 60_000);
public sealed record AskRequest(IReadOnlyList<string> Queries, IReadOnlyDictionary<string, AskQuestion>? Questions, AskOptions Options);
@@ -52,7 +52,7 @@ public sealed record AskResult(
/// <summary>
/// wiki_ask.py as a use case: rank every query, unite the top notes into one context (per-document
/// and total budget), expand context choices over the paragraph or document ids plus NONE, add the
/// covered gate, judge (every context choice in its own request, the rest batched), and attach the
/// covered gate, judge every question in one request, and attach the
/// answering paragraphs' text to the results. Judgements and quotations, never prose.
/// </summary>
public sealed class AskUseCase(RankUseCase rank, IDocumentCatalog catalog, IChunker chunker)
@@ -69,7 +69,7 @@ public sealed class AskUseCase(RankUseCase rank, IDocumentCatalog catalog, IChun
var questions = request.Questions ?? DefaultQuestions(request.Queries);
var (expanded, scopes) = Expand(questions, context, request.Queries, judge.MaxChoiceOptions);
var (answers, usage) = await JudgeInParallelAsync(judge, context, expanded, scopes, request.Options.MaxParallel, ct);
var (answers, usage) = await JudgeAsync(judge, context, expanded, ct);
var resolved = Resolve(answers, scopes, context);
double? covered = answers.TryGetValue(CoveredId, out var c) && c is Judgement.Noul n ? Math.Round(n.Probability, 3) : null;
return new AskResult(request.Queries, context, meta, resolved, covered, usage, judge.ModelId, Stopwatch.GetElapsedTime(started));
@@ -183,9 +183,12 @@ public sealed class AskUseCase(RankUseCase rank, IDocumentCatalog catalog, IChun
return (expanded, scopes);
}
private static async Task<(IReadOnlyDictionary<string, Judgement> Answers, JudgeUsage Usage)> JudgeInParallelAsync(
IJudge judge, IReadOnlyList<ContextDocument> context, IReadOnlyDictionary<string, TypedQuestion> questions,
IReadOnlyDictionary<string, ContextScope> scopes, int maxParallel, CancellationToken ct)
/// <summary>
/// One request for every question: the state dominates the request, and System One scores each
/// question in isolation against it, so splitting only re-sends the state and adds round trips.
/// </summary>
private static async Task<(IReadOnlyDictionary<string, Judgement> Answers, JudgeUsage Usage)> JudgeAsync(
IJudge judge, IReadOnlyList<ContextDocument> context, IReadOnlyDictionary<string, TypedQuestion> questions, CancellationToken ct)
{
var state = new
{
@@ -196,29 +199,8 @@ public sealed class AskUseCase(RankUseCase rank, IDocumentCatalog catalog, IChun
paragraphs = d.Paragraphs.Select(p => new { id = p.Id, heading = p.Heading, text = p.Text }),
}),
};
var groups = new List<Dictionary<string, TypedQuestion>>();
foreach (var (id, q) in questions.Where(kv => scopes.ContainsKey(kv.Key)))
groups.Add(new Dictionary<string, TypedQuestion> { [id] = q });
var rest = questions.Where(kv => !scopes.ContainsKey(kv.Key)).ToDictionary(kv => kv.Key, kv => kv.Value);
if (rest.Count > 0) groups.Add(rest);
using var gate = new SemaphoreSlim(Math.Max(1, maxParallel));
var tasks = groups.Select(async g =>
{
await gate.WaitAsync(ct);
try { return await judge.JudgeAsync(new JudgeRequest(state, g), ct); }
finally { gate.Release(); }
}).ToList();
var results = await Task.WhenAll(tasks);
var answers = new Dictionary<string, Judgement>();
var usage = JudgeUsage.None;
foreach (var r in results)
{
foreach (var (id, a) in r.Answers) answers[id] = a;
usage = usage.Add(r.Usage);
}
return (answers, usage);
var result = await judge.JudgeAsync(new JudgeRequest(state, questions), ct);
return (result.Answers, result.Usage);
}
private static IReadOnlyDictionary<string, AskAnswer> Resolve(
@@ -24,28 +24,42 @@ public sealed class DepositFactUseCase(INoteStore store, TimeProvider clock)
var now = clock.GetUtcNow();
var datePrefix = now.UtcDateTime.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture);
var slug = Slugify(fact.Title);
var text = BuildNote(fact, by, datePrefix);
var message = $"raw: {fact.Title}";
var path = $"raw/{datePrefix}-{slug}.md";
for (var attempt = 2; ; attempt++)
{
// Attachments take the note's stem, so they sort next to it and survive a suffix.
var stem = path[(path.LastIndexOf('/') + 1)..^3];
var names = fact.Attachments.Select((a, i) => $"{stem}-{i + 1}.{a.Extension}").ToList();
var text = BuildNote(fact, by, datePrefix, names);
StoreWrite written;
try
{
var written = await store.CreateAsync(path, text, message, ct);
return new FactReceipt(written.Path, written.Commit, now);
written = await store.CreateAsync(path, text, message, ct);
}
catch (DocumentExistsException) when (attempt <= MaxSuffixAttempts)
{
path = $"raw/{datePrefix}-{slug}-{attempt}.md";
continue;
}
for (var i = 0; i < names.Count; i++)
written = await store.CreateAsync($"raw/{names[i]}", fact.Attachments[i].Bytes, $"{message} (attachment {i + 1})", ct);
return new FactReceipt(path, written.Commit, now);
}
}
public static string BuildNote(RawSource fact, Identity by, string capturedDate)
public static string BuildNote(RawSource fact, Identity by, string capturedDate, IReadOnlyList<string>? attachments = null)
{
attachments ??= [];
var body = fact.Content;
if (!body.EndsWith('\n')) body += "\n";
if (attachments.Count > 0)
{
body += "\n";
for (var i = 0; i < attachments.Count; i++)
body += $"![{fact.Attachments[i].Caption ?? $"attachment {i + 1}"}]({attachments[i]})\n";
}
var sb = new StringBuilder();
sb.Append("---\n");
@@ -56,6 +70,7 @@ public sealed class DepositFactUseCase(INoteStore store, TimeProvider clock)
sb.Append("captured_by: ").Append(by.Name).Append('\n');
sb.Append("observed: ").Append(fact.ObservedAt.ToString("yyyy-MM-ddTHH:mm:ssK", CultureInfo.InvariantCulture)).Append('\n');
sb.Append("tags: [").Append(string.Join(", ", fact.Tags)).Append("]\n");
if (attachments.Count > 0) sb.Append("attachments: [").Append(string.Join(", ", attachments)).Append("]\n");
sb.Append("---\n");
sb.Append('\n');
sb.Append(body);
@@ -0,0 +1,42 @@
using System.Globalization;
using Wiki.Core.Model;
using Wiki.Core.Ports;
namespace Wiki.Application.Facts;
public sealed record RecordedStatement(FactDraft Draft, RawSource Fact, FactReceipt Receipt);
/// <summary>
/// A person states something; a drafter normalises it into title, content and tags; the raw
/// source that lands carries the drafter's restatement and the statement verbatim, with the
/// person as source. Provenance is the person's word, and the note says so.
/// </summary>
public sealed class RecordStatementUseCase(DepositFactUseCase deposit, TimeProvider clock)
{
public async Task<RecordedStatement> RecordAsync(string statement, IReadOnlyList<Attachment> attachments, Identity by, IFactDrafter drafter, CancellationToken ct)
{
ArgumentException.ThrowIfNullOrWhiteSpace(statement);
ArgumentNullException.ThrowIfNull(attachments);
ArgumentNullException.ThrowIfNull(by);
ArgumentNullException.ThrowIfNull(drafter);
var now = clock.GetUtcNow();
var text = statement.Trim();
var draft = await drafter.DraftAsync(new FactStatement(text, by, now) { Attachments = attachments }, ct);
var fact = new RawSource(
draft.Title,
$"stated by {by.Name} on {now.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)} via the wiki",
Body(draft.Content, text),
now,
draft.Tags) { Attachments = attachments };
var errors = fact.Validate();
if (errors.Count > 0)
throw new InvalidOperationException($"the draft is not a valid raw source: {string.Join("; ", errors)}");
var receipt = await deposit.DepositAsync(fact, by, ct);
return new RecordedStatement(draft, fact, receipt);
}
public static string Body(string content, string statement) =>
content.TrimEnd() + "\n\nStatement, verbatim:\n\n> " + statement.Trim().Replace("\n", "\n> ", StringComparison.Ordinal) + "\n";
}
+20
View File
@@ -0,0 +1,20 @@
namespace Wiki.Core.Model;
/// <summary>A binary that belongs to a raw source and is kept next to it as evidence, a photo of a type label for instance.</summary>
public sealed record Attachment(string MediaType, byte[] Bytes, string? Caption = null)
{
public const int MaxBytes = 8 * 1024 * 1024;
private static readonly Dictionary<string, string> Extensions = new(StringComparer.OrdinalIgnoreCase)
{
["image/jpeg"] = "jpg",
["image/png"] = "png",
["image/webp"] = "webp",
["image/gif"] = "gif",
};
/// <summary>The file extension for a supported media type, or null.</summary>
public static string? ExtensionFor(string mediaType) => Extensions.TryGetValue(mediaType, out var e) ? e : null;
public string? Extension => ExtensionFor(MediaType);
}
+11
View File
@@ -0,0 +1,11 @@
namespace Wiki.Core.Model;
/// <summary>What a person said, by whom and when. The drafter sees nothing else.</summary>
public sealed record FactStatement(string Text, Identity By, DateTimeOffset StatedAt)
{
/// <summary>Pictures that are part of the statement, a type label for instance.</summary>
public IReadOnlyList<Attachment> Attachments { get; init; } = [];
}
/// <summary>A statement normalised into the fields of a raw source. Source and provenance are the application's to add.</summary>
public sealed record FactDraft(string Title, string Content, IReadOnlyList<string> Tags);
+10
View File
@@ -13,6 +13,9 @@ public sealed record RawSource(
{
public const int MinimumContentLength = 40;
/// <summary>Binaries stored next to the note as evidence. Empty for most facts.</summary>
public IReadOnlyList<Attachment> Attachments { get; init; } = [];
/// <summary>Validation errors, empty when the fact may be stored.</summary>
public IReadOnlyList<string> Validate()
{
@@ -21,6 +24,13 @@ public sealed record RawSource(
if (string.IsNullOrWhiteSpace(Source)) errors.Add("source is required: where the fact was observed or quoted from");
if (Content is null || Content.Trim().Length < MinimumContentLength)
errors.Add($"content must be at least {MinimumContentLength} characters of observation");
for (var i = 0; i < Attachments.Count; i++)
{
var a = Attachments[i];
if (a.Extension is null) errors.Add($"attachment {i + 1}: unsupported media type {a.MediaType}");
if (a.Bytes.Length == 0) errors.Add($"attachment {i + 1}: empty");
if (a.Bytes.Length > Attachment.MaxBytes) errors.Add($"attachment {i + 1}: larger than {Attachment.MaxBytes / (1024 * 1024)} MiB");
}
return errors;
}
}
+13
View File
@@ -26,6 +26,12 @@ public interface INoteStore : IAdapterHealth
/// <summary>Creates a new document at the path on the current head. Throws <see cref="DocumentExistsException"/> when the path is taken.</summary>
Task<StoreWrite> CreateAsync(string path, string text, string message, CancellationToken ct);
/// <summary>Creates a new binary (an attachment) at the path. Same rules as the text overload.</summary>
Task<StoreWrite> CreateAsync(string path, byte[] content, string message, CancellationToken ct);
/// <summary>Where a person can look at the path in the store's own UI, or null when the store has none.</summary>
Uri? BrowseUrl(string path);
}
/// <summary>Text to vector. One instance per configured model.</summary>
@@ -88,3 +94,10 @@ public interface IAnswerModel : IAdapterHealth
string ModelId { get; }
IAsyncEnumerable<AnswerChunk> AnswerAsync(AnswerRequest request, CancellationToken ct);
}
/// <summary>A person's statement into the fields of a raw source. Sees only the statement.</summary>
public interface IFactDrafter : IAdapterHealth
{
string ModelId { get; }
Task<FactDraft> DraftAsync(FactStatement statement, CancellationToken ct);
}
+1 -1
View File
@@ -16,7 +16,7 @@ public static class ApiV1
v1.MapQuestionEndpoints();
v1.MapGet("/status", async (StatusQuery status, CancellationToken ct) =>
Results.Ok(await status.GetAsync(includeAdapterChecks: true, ct))).AllowAnonymous();
Results.Ok(await status.GetAsync(includeAdapterChecks: true, ct))).RequireAuthorization(Auth.Policies.Read);
v1.MapPost("/sync", async (SyncCoordinator sync, bool? force, CancellationToken ct) =>
{
+1 -1
View File
@@ -79,7 +79,7 @@ public static class AskEndpoints
{
return Results.Json(new { error = e.Message }, statusCode: StatusCodes.Status502BadGateway);
}
}).AllowAnonymous();
}).RequireAuthorization(Auth.Policies.Read);
}
public static AskQuestion Parse(string id, AskQuestionDto dto)
+2 -2
View File
@@ -20,8 +20,8 @@ public static class IndexEndpoints
{
public static void MapIndexEndpoints(this WebApplication app)
{
app.MapPost("/rank", RankAsync).AllowAnonymous();
app.MapPost("/api/v1/rank", RankAsync).AllowAnonymous();
app.MapPost("/rank", RankAsync).RequireAuthorization(Auth.Policies.Read);
app.MapPost("/api/v1/rank", RankAsync).RequireAuthorization(Auth.Policies.Read);
app.MapPost("/webhook", async (HttpRequest request, IConfiguration configuration, WikiOptions options, SyncBackgroundService sync, CancellationToken ct) =>
{
+2 -15
View File
@@ -1,9 +1,7 @@
using System.Text;
using System.Text.Json.Serialization;
using Microsoft.AspNetCore.Authentication;
using Wiki.Application.Questions;
using Wiki.Core.Ports;
using Wiki.Host.Auth;
namespace Wiki.Host.Api;
@@ -25,7 +23,7 @@ public static class QuestionEndpoints
{
v1.MapPost("/questions", async (
QuestionRequestDto body, QuestionUseCase questions, SaveAnswerUseCase save, IServiceProvider services,
IEmbeddingModel activeModel, IJudge activeJudge, IAnswerModel activeAnswer, HttpContext http, CancellationToken ct) =>
IEmbeddingModel activeModel, IJudge activeJudge, IAnswerModel activeAnswer, CancellationToken ct) =>
{
if (string.IsNullOrWhiteSpace(body.Question))
return Results.BadRequest(new { error = "question required" });
@@ -35,8 +33,6 @@ public static class QuestionEndpoints
if (err is not null) return Results.BadRequest(new { error = err });
var answerModel = Pick(services, activeAnswer, body.AnswerModel, out err) ?? activeAnswer;
if (err is not null) return Results.BadRequest(new { error = err });
if (body.Save == true && !await IsAuthenticatedAsync(http))
return Results.Json(new { error = "saving needs a signed-in user or an API key" }, statusCode: StatusCodes.Status401Unauthorized);
var text = new StringBuilder();
QuestionEvent.Evidence? evidence = null;
@@ -90,18 +86,9 @@ public static class QuestionEndpoints
askUsage = ask?.Usage,
saved,
});
}).AllowAnonymous();
}).RequireAuthorization(Auth.Policies.Read);
}
/// <summary>Humans arrive authenticated by the default (proxy header) scheme; machines present an API key.</summary>
private static async Task<bool> IsAuthenticatedAsync(HttpContext http)
{
if (http.User.Identity?.IsAuthenticated == true) return true;
var result = await http.AuthenticateAsync(ApiKeyAuthenticationHandler.SchemeName);
if (!result.Succeeded) return false;
http.User = result.Principal;
return true;
}
private static T? Pick<T>(IServiceProvider services, T active, string? name, out string? error) where T : class
{
+7 -7
View File
@@ -23,7 +23,7 @@ public static class ReadEndpoints
return Results.Ok(await queries.SearchAsync(m, q, take, ct));
}
return Results.Ok(await queries.ListAsync(tag, take, ct));
}).AllowAnonymous();
}).RequireAuthorization(Auth.Policies.Read);
v1.MapGet("/notes/{slug}", async (NoteQueries queries, string slug, CancellationToken ct) =>
{
@@ -40,28 +40,28 @@ public static class ReadEndpoints
sources = view.Sources.Select(s => new { s.Text, kind = s.Kind?.ToString(), s.Slug }),
unresolved = view.UnresolvedLinks,
});
}).AllowAnonymous();
}).RequireAuthorization(Auth.Policies.Read);
v1.MapGet("/tags", async (NoteQueries queries, CancellationToken ct) =>
Results.Ok((await queries.TagsAsync(ct)).OrderByDescending(kv => kv.Value).ThenBy(kv => kv.Key).Select(kv => new { tag = kv.Key, count = kv.Value })))
.AllowAnonymous();
.RequireAuthorization(Auth.Policies.Read);
v1.MapGet("/raw", async (NoteQueries queries, CancellationToken ct) =>
Results.Ok(await queries.RawQueueAsync(ct))).AllowAnonymous();
Results.Ok(await queries.RawQueueAsync(ct))).RequireAuthorization(Auth.Policies.Read);
v1.MapGet("/raw/{slug}", async (NoteQueries queries, string slug, CancellationToken ct) =>
{
var doc = await queries.GetDocumentAsync(DocumentKind.Raw, slug, ct) ?? await queries.GetDocumentAsync(DocumentKind.RawProcessed, slug, ct);
return doc is null ? Results.NotFound(new { error = "no such raw source" }) : Results.Ok(new { doc.Slug, doc.Title, doc.Kind, doc.Path, doc.Body });
}).AllowAnonymous();
}).RequireAuthorization(Auth.Policies.Read);
v1.MapGet("/reports", async (NoteQueries queries, string? type, int? limit, CancellationToken ct) =>
Results.Ok(await queries.ReportsAsync(type, limit ?? 50, ct))).AllowAnonymous();
Results.Ok(await queries.ReportsAsync(type, limit ?? 50, ct))).RequireAuthorization(Auth.Policies.Read);
v1.MapGet("/reports/{slug}", async (NoteQueries queries, string slug, CancellationToken ct) =>
{
var doc = await queries.GetDocumentAsync(DocumentKind.Output, slug, ct);
return doc is null ? Results.NotFound(new { error = "no such report" }) : Results.Ok(new { doc.Slug, doc.Title, doc.Type, doc.Path, doc.Sources, doc.Body });
}).AllowAnonymous();
}).RequireAuthorization(Auth.Policies.Read);
}
}
@@ -50,5 +50,13 @@ public sealed class ApiKeyAuthenticationHandler(
public static class Policies
{
/// <summary>A signed-in human (OIDC cookie), any API key or valid BasicAuth credentials.</summary>
public const string Read = "read";
/// <summary>An API key only, so a write carries a named machine identity.</summary>
public const string Write = "write";
}
public static class Claims
{
public const string Kind = "wiki:kind";
}
@@ -0,0 +1,144 @@
using System.Collections.Concurrent;
using System.Security.Claims;
using System.Security.Cryptography;
using System.Text;
using System.Text.Encodings.Web;
using Microsoft.AspNetCore.Authentication;
using Microsoft.Extensions.Options;
namespace Wiki.Host.Auth;
/// <summary>
/// Machine callers with the wiki-index BasicAuth credentials: the service checks
/// `Authorization: Basic` itself against the htpasswd lines in Auth:Basic:UsersPath, a file or a
/// mounted Secret directory (every key is read, as Traefik's middleware reads the same Secret), so
/// nothing depends on the request having passed a proxy.
/// Hashes: bcrypt ($2a$, $2b$, $2y$), Apache MD5 ($apr1$), SHA1 ({SHA}), the formats Traefik
/// accepts. Fail closed: no users, no matching user or an unknown hash format authenticates nobody.
/// </summary>
public sealed class BasicAuthenticationHandler(
IOptionsMonitor<AuthenticationSchemeOptions> options,
ILoggerFactory logger,
UrlEncoder encoder,
IConfiguration configuration)
: AuthenticationHandler<AuthenticationSchemeOptions>(options, logger, encoder)
{
public const string SchemeName = "Basic";
// bcrypt costs ~100 ms per check; a verified (user, password, hash) triple is remembered.
private static readonly ConcurrentDictionary<string, bool> Verified = new();
protected override async Task<AuthenticateResult> HandleAuthenticateAsync()
{
var authorization = Request.Headers.Authorization.FirstOrDefault();
if (authorization?.StartsWith("Basic ", StringComparison.OrdinalIgnoreCase) != true)
return AuthenticateResult.NoResult();
string user, password;
try
{
var decoded = Encoding.UTF8.GetString(Convert.FromBase64String(authorization["Basic ".Length..].Trim()));
var colon = decoded.IndexOf(':');
if (colon <= 0) return AuthenticateResult.Fail("malformed Basic credentials");
(user, password) = (decoded[..colon], decoded[(colon + 1)..]);
}
catch (FormatException)
{
return AuthenticateResult.Fail("malformed Basic credentials");
}
var lines = await ReadUsersAsync(configuration["Auth:Basic:UsersPath"], Context.RequestAborted);
if (lines.Count == 0)
return AuthenticateResult.Fail("no BasicAuth users configured");
foreach (var line in lines)
{
var colon = line.IndexOf(':');
if (colon <= 0 || line[..colon] != user) continue;
var hash = line[(colon + 1)..].Trim();
var key = Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes($"{user}\n{password}\n{hash}")));
if (Verified.GetOrAdd(key, _ => Htpasswd.Verify(password, hash)))
{
var identity = new ClaimsIdentity([new Claim(ClaimTypes.Name, user), new Claim(Claims.Kind, "machine")], Scheme.Name);
return AuthenticateResult.Success(new AuthenticationTicket(new ClaimsPrincipal(identity), Scheme.Name));
}
break;
}
return AuthenticateResult.Fail("invalid Basic credentials");
}
/// <summary>All lines of the file, or of every visible file in the directory (a Secret mount's
/// ..data and ..timestamp entries are skipped).</summary>
private static async Task<List<string>> ReadUsersAsync(string? path, CancellationToken ct)
{
if (string.IsNullOrWhiteSpace(path)) return [];
IEnumerable<string> files = File.Exists(path) ? [path]
: Directory.Exists(path) ? Directory.EnumerateFiles(path).Where(f => !Path.GetFileName(f).StartsWith('.')).Order()
: [];
var lines = new List<string>();
foreach (var file in files) lines.AddRange(await File.ReadAllLinesAsync(file, ct));
return lines;
}
}
/// <summary>Verifies a password against one htpasswd hash.</summary>
public static class Htpasswd
{
public static bool Verify(string password, string hash)
{
if (hash.StartsWith("$2a$") || hash.StartsWith("$2b$") || hash.StartsWith("$2y$"))
{
try { return BCrypt.Net.BCrypt.Verify(password, hash); }
catch (BCrypt.Net.SaltParseException) { return false; }
}
if (hash.StartsWith("$apr1$"))
{
var salt = hash["$apr1$".Length..].Split('$')[0];
return FixedEquals(Apr1(password, salt), hash);
}
if (hash.StartsWith("{SHA}"))
return FixedEquals("{SHA}" + Convert.ToBase64String(SHA1.HashData(Encoding.UTF8.GetBytes(password))), hash);
return false;
}
private static bool FixedEquals(string a, string b) =>
CryptographicOperations.FixedTimeEquals(Encoding.ASCII.GetBytes(a), Encoding.ASCII.GetBytes(b));
/// <summary>Apache's MD5-crypt variant, as written by `htpasswd -m`.</summary>
public static string Apr1(string password, string salt)
{
const string magic = "$apr1$";
if (salt.Length > 8) salt = salt[..8];
var pw = Encoding.UTF8.GetBytes(password);
var sl = Encoding.UTF8.GetBytes(salt);
var mg = Encoding.ASCII.GetBytes(magic);
var alt = MD5.HashData([.. pw, .. sl, .. pw]);
var ctx = new List<byte>([.. pw, .. mg, .. sl]);
for (var left = pw.Length; left > 0; left -= 16) ctx.AddRange(alt.Take(Math.Min(16, left)));
for (var i = pw.Length; i > 0; i >>= 1) ctx.Add((i & 1) != 0 ? (byte)0 : pw.Length > 0 ? pw[0] : (byte)0);
var final = MD5.HashData(ctx.ToArray());
for (var i = 0; i < 1000; i++)
{
var round = new List<byte>();
round.AddRange((i & 1) != 0 ? pw : final);
if (i % 3 != 0) round.AddRange(sl);
if (i % 7 != 0) round.AddRange(pw);
round.AddRange((i & 1) != 0 ? final : pw);
final = MD5.HashData(round.ToArray());
}
var sb = new StringBuilder(magic).Append(salt).Append('$');
void To64(int v, int n) { for (; n > 0; n--, v >>= 6) sb.Append(Itoa64[v & 0x3f]); }
To64((final[0] << 16) | (final[6] << 8) | final[12], 4);
To64((final[1] << 16) | (final[7] << 8) | final[13], 4);
To64((final[2] << 16) | (final[8] << 8) | final[14], 4);
To64((final[3] << 16) | (final[9] << 8) | final[15], 4);
To64((final[4] << 16) | (final[10] << 8) | final[5], 4);
To64(final[11], 2);
return sb.ToString();
}
private const string Itoa64 = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
}
@@ -1,71 +0,0 @@
using System.Net;
using System.Security.Claims;
using System.Text.Encodings.Web;
using Microsoft.AspNetCore.Authentication;
using Microsoft.Extensions.Options;
namespace Wiki.Host.Auth;
/// <summary>
/// Humans arrive through the Authentik proxy outpost, which sets X-Authentik-Username after
/// validating the session. The header is trusted only from the peers listed in
/// Auth:ReverseProxy:TrustedProxies; with none configured the scheme authenticates nobody.
/// </summary>
public sealed class ProxyHeaderAuthenticationHandler(
IOptionsMonitor<AuthenticationSchemeOptions> options,
ILoggerFactory logger,
UrlEncoder encoder,
IConfiguration configuration)
: AuthenticationHandler<AuthenticationSchemeOptions>(options, logger, encoder)
{
public const string SchemeName = "ProxyHeader";
public const string DefaultHeaderName = "X-Authentik-Username";
protected override Task<AuthenticateResult> HandleAuthenticateAsync()
{
var headerName = configuration["Auth:ReverseProxy:HeaderName"];
if (string.IsNullOrWhiteSpace(headerName)) headerName = DefaultHeaderName;
var user = Request.Headers[headerName].FirstOrDefault();
if (string.IsNullOrWhiteSpace(user))
return Task.FromResult(AuthenticateResult.NoResult());
var peer = Context.Connection.RemoteIpAddress;
if (peer is null || !IsTrustedProxy(peer))
{
Logger.LogWarning("ignoring {Header} from untrusted peer {Peer}", headerName, peer);
return Task.FromResult(AuthenticateResult.NoResult());
}
var identity = new ClaimsIdentity([new Claim(ClaimTypes.Name, user), new Claim(Claims.Kind, "human")], Scheme.Name);
return Task.FromResult(AuthenticateResult.Success(new AuthenticationTicket(new ClaimsPrincipal(identity), Scheme.Name)));
}
protected override Task HandleChallengeAsync(AuthenticationProperties properties)
{
Response.StatusCode = StatusCodes.Status401Unauthorized;
return Task.CompletedTask;
}
private bool IsTrustedProxy(IPAddress peer)
{
if (peer.IsIPv4MappedToIPv6) peer = peer.MapToIPv4();
var trusted = configuration.GetSection("Auth:ReverseProxy:TrustedProxies").Get<string[]>() ?? [];
foreach (var entry in trusted)
{
if (IPAddress.TryParse(entry, out var single))
{
if (single.Equals(peer)) return true;
continue;
}
if (IPNetwork.TryParse(entry, out var network) && network.Contains(peer))
return true;
}
return false;
}
}
public static class Claims
{
public const string Kind = "wiki:kind";
}
+97
View File
@@ -0,0 +1,97 @@
using System.Security.Claims;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authentication.Cookies;
using Microsoft.AspNetCore.Authentication.OpenIdConnect;
using Microsoft.AspNetCore.Authorization;
using Microsoft.IdentityModel.Protocols.OpenIdConnect;
namespace Wiki.Host.Auth;
/// <summary>
/// Humans sign in with OpenID Connect against Authentik (public client, PKCE): the username comes
/// from the ID token, whose signature the handler checks against the provider's JWKS, so no
/// network path or request header can assert an identity. Machines present an API key or the
/// BasicAuth credentials of the wiki-index machine door, which the service checks itself.
/// Every endpoint needs one of these unless it opts out (/health, /ready, the HMAC webhook).
/// </summary>
public static class WikiAuthentication
{
public const string UsernameClaim = "preferred_username";
public static IServiceCollection AddWikiAuthentication(this IServiceCollection services, IConfiguration configuration)
{
var authority = configuration["Auth:Oidc:Authority"];
var clientId = configuration["Auth:Oidc:ClientId"];
if (string.IsNullOrWhiteSpace(authority) || string.IsNullOrWhiteSpace(clientId))
throw new InvalidOperationException("Auth:Oidc:Authority and Auth:Oidc:ClientId are required; humans have no other way to sign in.");
services.AddAuthentication(options =>
{
options.DefaultScheme = CookieAuthenticationDefaults.AuthenticationScheme;
options.DefaultChallengeScheme = OpenIdConnectDefaults.AuthenticationScheme;
})
.AddCookie(options =>
{
options.Cookie.Name = "wiki.auth";
options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
options.Cookie.SameSite = SameSiteMode.Lax;
options.SlidingExpiration = true;
options.ExpireTimeSpan = TimeSpan.FromHours(12);
// Only the OIDC scheme sends anyone to a login; the cookie scheme answers API callers with a status.
options.Events.OnRedirectToLogin = ctx => { ctx.Response.StatusCode = StatusCodes.Status401Unauthorized; return Task.CompletedTask; };
options.Events.OnRedirectToAccessDenied = ctx => { ctx.Response.StatusCode = StatusCodes.Status403Forbidden; return Task.CompletedTask; };
})
.AddOpenIdConnect(options =>
{
options.Authority = authority;
options.ClientId = clientId;
options.ResponseType = OpenIdConnectResponseType.Code;
options.UsePkce = true;
options.Scope.Clear();
options.Scope.Add("openid");
options.Scope.Add("profile");
options.MapInboundClaims = false;
options.GetClaimsFromUserInfoEndpoint = true;
options.ClaimActions.MapUniqueJsonKey(UsernameClaim, UsernameClaim);
options.TokenValidationParameters.NameClaimType = UsernameClaim;
options.SaveTokens = false;
options.Events.OnRedirectToIdentityProvider = ctx =>
{
// TLS ends at Traefik, so the request scheme is http; browsers only ever reach the wiki over https.
ctx.ProtocolMessage.RedirectUri = $"https://{ctx.Request.Host}{ctx.Options.CallbackPath}";
return Task.CompletedTask;
};
options.Events.OnTokenValidated = ctx =>
{
if (ctx.Principal?.Identity is ClaimsIdentity identity)
{
if (string.IsNullOrWhiteSpace(identity.Name))
{
ctx.Fail($"ID token carries no {UsernameClaim}");
return Task.CompletedTask;
}
identity.AddClaim(new Claim(Claims.Kind, "human"));
}
return Task.CompletedTask;
};
})
.AddScheme<AuthenticationSchemeOptions, ApiKeyAuthenticationHandler>(ApiKeyAuthenticationHandler.SchemeName, null)
.AddScheme<AuthenticationSchemeOptions, BasicAuthenticationHandler>(BasicAuthenticationHandler.SchemeName, null);
services.AddAuthorization(options =>
{
// UI pages, static assets and the Blazor hub: a signed-in human. No explicit scheme, so the
// challenge goes to the default (OIDC) and an anonymous browser is sent to Authentik.
options.FallbackPolicy = new AuthorizationPolicyBuilder()
.RequireAuthenticatedUser()
.Build();
options.AddPolicy(Policies.Read, p => p
.AddAuthenticationSchemes(CookieAuthenticationDefaults.AuthenticationScheme, ApiKeyAuthenticationHandler.SchemeName, BasicAuthenticationHandler.SchemeName)
.RequireAuthenticatedUser());
options.AddPolicy(Policies.Write, p => p
.AddAuthenticationSchemes(ApiKeyAuthenticationHandler.SchemeName)
.RequireAuthenticatedUser());
});
return services;
}
}
+1
View File
@@ -18,6 +18,7 @@
<ReconnectModal />
<script src="@Assets["_framework/blazor.web.js"]"></script>
<script src="@Assets["_content/MudBlazor/MudBlazor.min.js"]"></script>
<script src="@Assets["facts.js"]"></script>
</body>
</html>
@@ -14,6 +14,7 @@
<MudNavMenu Color="Color.Primary" Bordered="false">
<MudNavLink Href="" Match="NavLinkMatch.All" Icon="@Icons.Material.Outlined.Search">Search</MudNavLink>
<MudNavLink Href="answers" Icon="@Icons.Material.Outlined.QuestionAnswer">Answers</MudNavLink>
<MudNavLink Href="facts" Icon="@Icons.Material.Outlined.NoteAdd">Facts</MudNavLink>
<MudNavLink Href="browse" Icon="@Icons.Material.Outlined.Label">Browse</MudNavLink>
<MudNavLink Href="maintenance" Icon="@Icons.Material.Outlined.Build">Maintenance</MudNavLink>
<MudNavLink Href="lab" Icon="@Icons.Material.Outlined.Science">Lab</MudNavLink>
+192
View File
@@ -0,0 +1,192 @@
@page "/facts"
@rendermode InteractiveServer
@using Microsoft.AspNetCore.Components.Authorization
@using Wiki.Application.Facts
@inject RecordStatementUseCase Record
@inject IFactDrafter Drafter
@inject NoteQueries Notes
@inject ISnackbar Snackbar
@inject IJSRuntime JS
<PageTitle>Facts</PageTitle>
<MudText Typo="Typo.h4" GutterBottom="true">Facts</MudText>
<MudText Typo="Typo.body2" Class="mb-4" Color="Color.Secondary">
Say what you observed, in one sentence or a few, and add or paste a picture when the thing itself says it best, a type label for instance. @Drafter.ModelId turns it into a raw source under your name, with your words kept verbatim and the pictures next to the note, and the nightly distill turns that into wiki notes. Observations only, never hypotheses.
</MudText>
@if (_user is null)
{
<MudAlert Severity="Severity.Warning" Class="mb-4">Not signed in. Facts are attributed to the person stating them, so this page needs the Authentik sign-in.</MudAlert>
}
else
{
<form @onsubmit="RecordAsync" @onsubmit:preventDefault="true">
<MudTextField T="string" @bind-Value="_statement" Immediate="true" Lines="3" Label="What did you observe?" Variant="Variant.Outlined" Disabled="@_running" Class="mb-2" />
<InputFile id="fact-files" OnChange="OnFilesAsync" multiple accept="image/*" style="display:none" />
@if (_images.Count > 0)
{
<MudStack Row Wrap="Wrap.Wrap" Spacing="2" Class="mb-2">
@foreach (var img in _images)
{
<MudPaper Outlined="true" Class="pa-1" Style="position: relative">
<img src="@img.DataUrl" alt="attachment" style="max-height: 140px; display: block" />
<MudIconButton Icon="@Icons.Material.Filled.Close" Size="Size.Small" Style="position: absolute; top: 0; right: 0" OnClick="@(() => _images.Remove(img))" Disabled="@_running" />
</MudPaper>
}
</MudStack>
}
<MudStack Row Justify="Justify.SpaceBetween" AlignItems="AlignItems.Center" Class="mb-4">
<MudStack Row AlignItems="AlignItems.Center" Spacing="2">
<MudButton Variant="Variant.Outlined" StartIcon="@Icons.Material.Outlined.AddPhotoAlternate" OnClick="PickAsync" Disabled="@_running">Add picture</MudButton>
<MudText Typo="Typo.caption" Color="Color.Secondary">or paste one · captured_by: @_user</MudText>
</MudStack>
<MudButton ButtonType="ButtonType.Submit" Variant="Variant.Filled" Color="Color.Primary" Disabled="@(_running || string.IsNullOrWhiteSpace(_statement))">Record</MudButton>
</MudStack>
</form>
}
@if (_running)
{
<MudProgressLinear Indeterminate="true" Class="mb-4" />
}
@if (_error is not null)
{
<MudAlert Severity="Severity.Error" Class="mb-4">@_error</MudAlert>
}
@if (_recorded is not null)
{
<MudPaper Class="pa-4 mb-6" Outlined="true">
<MudStack Row Justify="Justify.SpaceBetween" AlignItems="AlignItems.Center" Class="mb-2">
<MudText Typo="Typo.h6">@_recorded.Draft.Title</MudText>
<MudLink Href="@($"raw/{Slug(_recorded.Receipt.Path)}")">@_recorded.Receipt.Path</MudLink>
</MudStack>
<MudText Typo="Typo.body1" Style="white-space: pre-wrap" Class="mb-2">@_recorded.Draft.Content</MudText>
<MudStack Row Spacing="1">
@foreach (var t in _recorded.Draft.Tags)
{
<MudChip T="string" Size="Size.Small" Variant="Variant.Outlined">@t</MudChip>
}
</MudStack>
</MudPaper>
}
<MudText Typo="Typo.subtitle1" Class="mb-2">Raw sources awaiting distillation</MudText>
@if (_raw is null)
{
<MudProgressLinear Indeterminate="true" />
}
else if (_raw.Count == 0)
{
<MudText Typo="Typo.body2" Color="Color.Secondary">Nothing queued.</MudText>
}
else
{
<MudSimpleTable Dense="true" Hover="true">
<thead>
<tr><th>Captured</th><th>Title</th><th>Source</th></tr>
</thead>
<tbody>
@foreach (var r in _raw.OrderByDescending(r => r.Captured).ThenByDescending(r => r.Slug).Take(30))
{
<tr>
<td>@(r.Captured?.ToString("yyyy-MM-dd") ?? "")</td>
<td><MudLink Href="@($"raw/{r.Slug}")">@r.Title</MudLink></td>
<td><MudText Typo="Typo.caption" Color="Color.Secondary">@r.Source</MudText></td>
</tr>
}
</tbody>
</MudSimpleTable>
}
@code {
[CascadingParameter] private Task<AuthenticationState>? AuthState { get; set; }
private bool _running;
private string? _user;
private string? _statement;
private string? _error;
private RecordedStatement? _recorded;
private IReadOnlyList<RawQueueEntry>? _raw;
private readonly List<PendingImage> _images = [];
private sealed record PendingImage(string MediaType, byte[] Bytes, string DataUrl);
protected override async Task OnInitializedAsync()
{
if (AuthState is not null)
{
var state = await AuthState;
_user = state.User.Identity?.IsAuthenticated == true ? state.User.Identity.Name : null;
}
await LoadRawAsync();
}
protected override async Task OnAfterRenderAsync(bool firstRender)
{
if (firstRender && _user is not null)
await JS.InvokeVoidAsync("wikiFacts.listenPaste", "fact-files");
}
private async Task LoadRawAsync() => _raw = await Notes.RawQueueAsync(CancellationToken.None);
private async Task PickAsync() => await JS.InvokeVoidAsync("wikiFacts.pick", "fact-files");
private async Task OnFilesAsync(InputFileChangeEventArgs e)
{
foreach (var file in e.GetMultipleFiles(10))
{
if (!file.ContentType.StartsWith("image/", StringComparison.OrdinalIgnoreCase))
{
Snackbar.Add($"{file.Name}: not an image", Severity.Warning);
continue;
}
try
{
// Resized in the browser: labels stay legible at 1600px and the upload stays small.
var resized = await file.RequestImageFileAsync("image/jpeg", 1600, 1600);
using var ms = new MemoryStream();
await using (var stream = resized.OpenReadStream(Attachment.MaxBytes))
await stream.CopyToAsync(ms);
var bytes = ms.ToArray();
_images.Add(new PendingImage("image/jpeg", bytes, "data:image/jpeg;base64," + Convert.ToBase64String(bytes)));
}
catch (IOException ex)
{
Snackbar.Add($"{file.Name}: {ex.Message}", Severity.Error);
}
}
StateHasChanged();
}
private async Task RecordAsync()
{
if (_running || _user is null || string.IsNullOrWhiteSpace(_statement)) return;
_running = true;
_error = null;
_recorded = null;
try
{
var attachments = _images.Select(i => new Attachment(i.MediaType, i.Bytes)).ToList();
_recorded = await Record.RecordAsync(_statement, attachments, new Identity(_user, IdentityKind.Human), Drafter, CancellationToken.None);
Snackbar.Add($"recorded {_recorded.Receipt.Path}", Severity.Success);
_statement = null;
_images.Clear();
await LoadRawAsync();
}
catch (Exception e) when (e is HttpRequestException or InvalidOperationException)
{
_error = e.Message;
}
finally
{
_running = false;
}
}
private static string Slug(string path)
{
var file = path[(path.LastIndexOf('/') + 1)..];
return file.EndsWith(".md", StringComparison.Ordinal) ? file[..^3] : file;
}
}
+21
View File
@@ -1,6 +1,7 @@
@page "/raw/{slug}"
@rendermode InteractiveServer
@inject NoteQueries Notes
@inject INoteStore Store
<PageTitle>@(_doc?.Title ?? Slug)</PageTitle>
@@ -33,6 +34,19 @@ else
<div class="wiki-body">
@((MarkupString)_html!)
</div>
@if (Attachments(_doc) is { Count: > 0 } attachments)
{
<MudText Typo="Typo.subtitle2" Class="mt-4 mb-1">Attachments</MudText>
<MudList T="string" Dense="true">
@foreach (var a in attachments)
{
var href = Store.BrowseUrl($"raw/{a}")?.ToString();
<MudListItem T="string" Icon="@Icons.Material.Outlined.Image">
@if (href is null) { @a } else { <MudLink Href="@href" Target="_blank">@a</MudLink> }
</MudListItem>
}
</MudList>
}
}
@code {
@@ -52,4 +66,11 @@ else
}
private static string? Scalar(Document d, string key) => d.Frontmatter.TryGetValue(key, out var v) ? v.Scalar : null;
private static IReadOnlyList<string> Attachments(Document d)
{
if (!d.Frontmatter.TryGetValue("attachments", out var v)) return [];
if (v.List is { } list) return list;
return (v.Scalar ?? "").Trim('[', ']').Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
}
}
+13 -11
View File
@@ -1,4 +1,3 @@
using Microsoft.AspNetCore.Authentication;
using MudBlazor.Services;
using Wiki.Adapters.FileIndex;
using Wiki.Adapters.Gitea;
@@ -74,14 +73,16 @@ foreach (var (name, answer) in wiki.AnswerModels.Models)
switch (answer.Adapter)
{
case "Anthropic":
builder.Services.AddAnthropicAnswerModel(name, o =>
Action<Wiki.Adapters.Anthropic.AnthropicOptions> anthropic = o =>
{
o.BaseUrl = answer.Url;
o.Model = answer.Model;
o.ModelId = name;
o.MaxTokens = answer.MaxTokens;
o.ApiKey = builder.Configuration[$"Wiki:AnswerModels:Models:{name}:ApiKey"] ?? string.Empty;
});
};
builder.Services.AddAnthropicAnswerModel(name, anthropic);
builder.Services.AddAnthropicFactDrafter(name, anthropic);
break;
default:
throw new InvalidOperationException($"unknown answer-model adapter '{answer.Adapter}' for '{name}'");
@@ -90,6 +91,10 @@ foreach (var (name, answer) in wiki.AnswerModels.Models)
builder.Services.AddSingleton<IReadOnlyList<IAnswerModel>>(sp =>
wiki.AnswerModels.Models.Keys.Select(sp.GetRequiredKeyedService<IAnswerModel>).ToList());
builder.Services.AddSingleton<IAnswerModel>(sp => sp.GetRequiredKeyedService<IAnswerModel>(wiki.AnswerModels.Active));
// Fact drafters ride with the answer models: same entries, same keys.
builder.Services.AddSingleton<IReadOnlyList<IFactDrafter>>(sp =>
wiki.AnswerModels.Models.Keys.Select(sp.GetRequiredKeyedService<IFactDrafter>).ToList());
builder.Services.AddSingleton<IFactDrafter>(sp => sp.GetRequiredKeyedService<IFactDrafter>(wiki.AnswerModels.Active));
// One adapter may serve several ports; list it once.
builder.Services.AddSingleton<IReadOnlyList<IAdapterHealth>>(sp =>
{
@@ -101,6 +106,7 @@ builder.Services.AddSingleton<IReadOnlyList<IAdapterHealth>>(sp =>
.. sp.GetRequiredService<IReadOnlyList<IEmbeddingModel>>(),
.. sp.GetRequiredService<IReadOnlyList<IJudge>>(),
.. sp.GetRequiredService<IReadOnlyList<IAnswerModel>>(),
.. sp.GetRequiredService<IReadOnlyList<IFactDrafter>>(),
];
return all.Distinct().ToList();
});
@@ -114,6 +120,7 @@ builder.Services.AddSingleton<Wiki.Application.Reading.NoteQueries>();
builder.Services.AddSingleton<IChunker, Wiki.Core.Parsing.ParagraphChunker>();
builder.Services.AddSingleton<Wiki.Application.Ask.AskUseCase>();
builder.Services.AddSingleton<Wiki.Application.Facts.DepositFactUseCase>();
builder.Services.AddSingleton<Wiki.Application.Facts.RecordStatementUseCase>();
builder.Services.AddSingleton<Wiki.Application.Evaluation.EvaluateUseCase>();
builder.Services.AddSingleton<Wiki.Application.Questions.QuestionUseCase>();
builder.Services.AddSingleton<Wiki.Application.Questions.SaveAnswerUseCase>();
@@ -121,17 +128,12 @@ builder.Services.AddSingleton<StatusQuery>();
builder.Services.AddSingleton<SyncBackgroundService>();
builder.Services.AddHostedService(sp => sp.GetRequiredService<SyncBackgroundService>());
// ---- auth: humans arrive through the Authentik outpost, machines with an API key ---------
builder.Services.AddAuthentication(ProxyHeaderAuthenticationHandler.SchemeName)
.AddScheme<AuthenticationSchemeOptions, ProxyHeaderAuthenticationHandler>(ProxyHeaderAuthenticationHandler.SchemeName, null)
.AddScheme<AuthenticationSchemeOptions, ApiKeyAuthenticationHandler>(ApiKeyAuthenticationHandler.SchemeName, null);
builder.Services.AddAuthorization(options =>
{
options.AddPolicy(Policies.Write, p => p.AddAuthenticationSchemes(ApiKeyAuthenticationHandler.SchemeName).RequireAuthenticatedUser());
});
// ---- auth: humans sign in with OIDC against Authentik, machines with an API key ----------
builder.Services.AddWikiAuthentication(builder.Configuration);
// ---- UI ----------------------------------------------------------------------------------
builder.Services.AddRazorComponents().AddInteractiveServerComponents();
builder.Services.AddCascadingAuthenticationState();
builder.Services.AddMudServices();
var app = builder.Build();
+2
View File
@@ -10,7 +10,9 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="BCrypt.Net-Next" Version="4.2.1" />
<PackageReference Include="Markdig" Version="1.4.0" />
<PackageReference Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="10.0.13" />
<PackageReference Include="MudBlazor" Version="9.10.0" />
</ItemGroup>
+2 -1
View File
@@ -34,7 +34,8 @@
"Sync": { "IntervalSeconds": 1800, "RetrySeconds": 60 }
},
"Auth": {
"ReverseProxy": { "HeaderName": "X-Authentik-Username", "TrustedProxies": [] },
"Oidc": { "Authority": "", "ClientId": "" },
"Basic": { "UsersPath": "" },
"ApiKeys": {}
}
}
+31
View File
@@ -0,0 +1,31 @@
// Pasted images become files on the page's hidden <input type="file">, so Blazor's InputFile
// streams them like a picked file. One document listener; it looks the input up per paste.
window.wikiFacts = (() => {
let armed = false;
return {
listenPaste(inputId) {
if (armed) return;
armed = true;
document.addEventListener('paste', (ev) => {
const input = document.getElementById(inputId);
if (!input || !ev.clipboardData) return;
const files = [];
for (const item of ev.clipboardData.items) {
if (item.kind === 'file' && item.type.startsWith('image/')) {
const f = item.getAsFile();
if (f) files.push(f);
}
}
if (files.length === 0) return;
ev.preventDefault();
const dt = new DataTransfer();
for (const f of files) dt.items.add(f);
input.files = dt.files;
input.dispatchEvent(new Event('change', { bubbles: true }));
});
},
pick(inputId) {
document.getElementById(inputId)?.click();
},
};
})();
@@ -81,10 +81,10 @@ public class AskUseCaseTests
Assert.Equal("udm-dns#p1", where.Top[0].Id);
Assert.Contains("answers every query", where.Top[0].Text);
Assert.Equal("UDM intercepts DNS", where.Top[0].Title);
// the context choice went alone, the noul questions together
Assert.Equal(2, _judge.Requests.Count);
Assert.Contains(_judge.Requests, r => r.Questions.Count == 1 && r.Questions.ContainsKey("where_1"));
Assert.Equal(2, result.Usage.Requests);
// every question rides in the one request that carries the state
var sentToJudge = Assert.Single(_judge.Requests);
Assert.Equal(3, sentToJudge.Questions.Count);
Assert.Equal(1, result.Usage.Requests);
Assert.Equal("fake-judge", result.JudgeModel);
}
+12
View File
@@ -44,6 +44,18 @@ public sealed class FakeStore : INoteStore
return Task.FromResult(new StoreWrite(path, Convert.ToHexStringLower(SHA1.HashData(Encoding.UTF8.GetBytes(text)))));
}
public Dictionary<string, byte[]> Binaries { get; } = new();
public Task<StoreWrite> CreateAsync(string path, byte[] content, string message, CancellationToken ct)
{
if (Files.ContainsKey(path) || Binaries.ContainsKey(path)) throw new DocumentExistsException(path);
Binaries[path] = content;
Created.Add(($"{message}", path));
return Task.FromResult(new StoreWrite(path, Convert.ToHexStringLower(SHA1.HashData(content))));
}
public Uri? BrowseUrl(string path) => null;
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => Task.FromResult(new AdapterHealth(Name, true, null, TimeSpan.Zero, DateTimeOffset.UnixEpoch));
}
@@ -0,0 +1,88 @@
using Microsoft.Extensions.Time.Testing;
using Wiki.Application.Facts;
using Wiki.Core.Model;
using Wiki.Core.Ports;
namespace Wiki.Application.Tests;
public sealed class FakeFactDrafter : IFactDrafter
{
public List<FactStatement> Statements { get; } = [];
public FactDraft Draft { get; set; } = new("Fußbodenheizung im Erdgeschoss", "Alle Räume im Erdgeschoss haben Fußbodenheizung; die Garderobe hat keine.", ["heating", "ground-floor"]);
public string ModelId => "fake-drafter";
public string Name => "fake-drafter";
public Task<FactDraft> DraftAsync(FactStatement statement, CancellationToken ct)
{
Statements.Add(statement);
return Task.FromResult(Draft);
}
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => Task.FromResult(new AdapterHealth(Name, true, null, TimeSpan.Zero, DateTimeOffset.UnixEpoch));
}
public class RecordStatementUseCaseTests
{
private static readonly DateTimeOffset Now = new(2026, 9, 25, 8, 0, 0, TimeSpan.Zero);
private static readonly Identity By = new("k.renner", IdentityKind.Human);
private const string Statement = "Alle Räume im Erdgeschoss haben Fußbodenheizung außer die Garderobe";
[Fact]
public async Task StatementBecomesARawSourceUnderThePersonWithTheVerbatimText()
{
var store = new FakeStore();
var drafter = new FakeFactDrafter();
var useCase = new RecordStatementUseCase(new DepositFactUseCase(store, new FakeTimeProvider(Now)), new FakeTimeProvider(Now));
var recorded = await useCase.RecordAsync(" " + Statement + "\n", [], By, drafter, CancellationToken.None);
var sent = Assert.Single(drafter.Statements);
Assert.Equal(Statement, sent.Text);
Assert.Equal(By, sent.By);
Assert.Equal("raw/2026-09-25-fu-bodenheizung-im-erdgeschoss.md", recorded.Receipt.Path);
var text = store.Files[recorded.Receipt.Path];
Assert.Contains("title: Fußbodenheizung im Erdgeschoss\n", text);
Assert.Contains("source: stated by k.renner on 2026-09-25 via the wiki\n", text);
Assert.Contains("captured_by: k.renner\n", text);
Assert.Contains("tags: [heating, ground-floor]\n", text);
Assert.Contains("die Garderobe hat keine.\n\nStatement, verbatim:\n\n> " + Statement + "\n", text);
}
[Fact]
public async Task ADraftThatIsNoRawSourceIsNotStored()
{
var store = new FakeStore();
var drafter = new FakeFactDrafter { Draft = new FactDraft(" ", "the verbatim statement always makes the body long enough; only a missing title fails", []) };
var useCase = new RecordStatementUseCase(new DepositFactUseCase(store, new FakeTimeProvider(Now)), new FakeTimeProvider(Now));
await Assert.ThrowsAsync<InvalidOperationException>(() => useCase.RecordAsync(Statement, [], By, drafter, CancellationToken.None));
Assert.Empty(store.Files);
}
[Fact]
public async Task EmptyStatementsNeverReachTheDrafter()
{
var drafter = new FakeFactDrafter();
var useCase = new RecordStatementUseCase(new DepositFactUseCase(new FakeStore(), new FakeTimeProvider(Now)), new FakeTimeProvider(Now));
await Assert.ThrowsAsync<ArgumentException>(() => useCase.RecordAsync(" \n", [], By, drafter, CancellationToken.None));
Assert.Empty(drafter.Statements);
}
[Fact]
public async Task PicturesReachTheDrafterAndLandNextToTheNote()
{
var store = new FakeStore();
var drafter = new FakeFactDrafter { Draft = new FactDraft("Geschirrspüler Typenschild", "Der Geschirrspüler ist ein Bosch SMV4HCX48E, Seriennummer [unreadable].", ["dishwasher"]) };
var useCase = new RecordStatementUseCase(new DepositFactUseCase(store, new FakeTimeProvider(Now)), new FakeTimeProvider(Now));
var photo = new Attachment("image/jpeg", [0xFF, 0xD8, 0xFF, 1, 2, 3], "type label");
var recorded = await useCase.RecordAsync("Das ist das Typenschild unseres Geschirrspülers.", [photo], By, drafter, CancellationToken.None);
Assert.Same(photo, Assert.Single(Assert.Single(drafter.Statements).Attachments));
Assert.Equal("raw/2026-09-25-geschirrsp-ler-typenschild.md", recorded.Receipt.Path);
Assert.Equal(photo.Bytes, store.Binaries["raw/2026-09-25-geschirrsp-ler-typenschild-1.jpg"]);
var text = store.Files[recorded.Receipt.Path];
Assert.Contains("attachments: [2026-09-25-geschirrsp-ler-typenschild-1.jpg]\n", text);
Assert.EndsWith("![type label](2026-09-25-geschirrsp-ler-typenschild-1.jpg)\n", text);
}
}
@@ -0,0 +1,63 @@
using Wiki.Adapters.Anthropic;
using Wiki.Core.Ports;
namespace Wiki.Contracts.Anthropic;
/// <summary>Runs the shared <see cref="FactDrafterContract"/> against the Anthropic adapter over a fake server that replies with a JSON draft.</summary>
public sealed class AnthropicFactDrafterContractTests : FactDrafterContract
{
internal const string DraftJson =
"{\"title\": \"Fußbodenheizung im Erdgeschoss\", " +
"\"content\": \"Alle Räume im Erdgeschoss haben Fußbodenheizung. Die Garderobe ist die einzige Ausnahme.\", " +
"\"tags\": [\"Heating\", \"floor heating\", \"ground-floor\", \"heating\"]}";
protected override Task<IFactDrafter> CreateDrafterAsync()
{
var handler = new FakeAnthropicHandler { ResponseText = DraftJson };
var client = new HttpClient(handler);
var options = new AnthropicOptions
{
BaseUrl = "https://anthropic.test",
Model = "claude-sonnet-5",
ApiKey = "test-key",
};
IFactDrafter drafter = new AnthropicFactDrafter(client, options, delay: (_, _) => Task.CompletedTask);
return Task.FromResult(drafter);
}
[Fact]
public async Task PicturesTravelAsBase64ImageBlocksBeforeTheText()
{
var handler = new FakeAnthropicHandler { ResponseText = DraftJson };
var drafter = new AnthropicFactDrafter(new HttpClient(handler), new AnthropicOptions { BaseUrl = "https://anthropic.test", ApiKey = "k" }, delay: (_, _) => Task.CompletedTask);
var statement = Statement() with { Attachments = [new Wiki.Core.Model.Attachment("image/jpeg", [1, 2, 3])] };
await drafter.DraftAsync(statement, CancellationToken.None);
using var request = System.Text.Json.JsonDocument.Parse(handler.LastRequestJson!);
var content = request.RootElement.GetProperty("messages")[0].GetProperty("content");
Assert.Equal(2, content.GetArrayLength());
Assert.Equal("image", content[0].GetProperty("type").GetString());
Assert.Equal("image/jpeg", content[0].GetProperty("source").GetProperty("media_type").GetString());
Assert.Equal("AQID", content[0].GetProperty("source").GetProperty("data").GetString());
Assert.Equal("text", content[1].GetProperty("type").GetString());
Assert.Contains("1 image(s) attached", content[1].GetProperty("text").GetString());
}
[Fact]
public void ParseTakesTheObjectOutOfSurroundingTextAndNormalisesTags()
{
var draft = FactDraftPrompt.Parse("Here you go:\n```json\n" + DraftJson + "\n```");
Assert.Equal("Fußbodenheizung im Erdgeschoss", draft.Title);
Assert.Equal(["heating", "floor-heating", "ground-floor"], draft.Tags);
}
[Fact]
public void ParseRejectsRepliesWithoutTitleOrContent()
{
Assert.Throws<InvalidOperationException>(() => FactDraftPrompt.Parse("no json here"));
Assert.Throws<InvalidOperationException>(() => FactDraftPrompt.Parse("{\"title\": \"x\"}"));
Assert.Throws<InvalidOperationException>(() => FactDraftPrompt.Parse("{\"title\": \"x\", \"content\": }"));
}
}
@@ -18,6 +18,9 @@ internal sealed class FakeAnthropicHandler : HttpMessageHandler
public int TooManyRequestsBeforeSuccess { get; set; }
/// <summary>The text the fake streams back, in three deltas.</summary>
public string ResponseText { get; set; } = EchoText;
public bool RespondWithErrorEvent { get; set; }
public string? LastRequestJson { get; private set; }
@@ -72,7 +75,7 @@ internal sealed class FakeAnthropicHandler : HttpMessageHandler
throw new InvalidOperationException($"Malformed Anthropic request: {body}");
}
var sse = RespondWithErrorEvent ? BuildErrorSse() : BuildSuccessSse(body.Length);
var sse = RespondWithErrorEvent ? BuildErrorSse() : BuildSuccessSse(body.Length, ResponseText);
return new HttpResponseMessage(HttpStatusCode.OK)
{
@@ -80,10 +83,10 @@ internal sealed class FakeAnthropicHandler : HttpMessageHandler
};
}
private static string BuildSuccessSse(int requestBodyLength)
private static string BuildSuccessSse(int requestBodyLength, string text)
{
var inputTokens = requestBodyLength / 4;
var words = EchoText.Split(' ');
var words = text.Split(' ');
var third = words.Length / 3;
var deltas = new[]
{
@@ -0,0 +1,47 @@
using Wiki.Core.Model;
using Wiki.Core.Ports;
namespace Wiki.Contracts;
/// <summary>
/// Every <see cref="IFactDrafter"/> adapter derives from this. The adapter may talk to a fake
/// server; the contract checks the draft's shape and that empty input is refused before any call.
/// </summary>
public abstract class FactDrafterContract
{
protected abstract Task<IFactDrafter> CreateDrafterAsync();
protected static FactStatement Statement() => new(
"Alle Räume im Erdgeschoss haben Fußbodenheizung außer die Garderobe.",
new Identity("k.renner", IdentityKind.Human),
new DateTimeOffset(2026, 9, 25, 8, 0, 0, TimeSpan.Zero));
[Fact]
public async Task DraftsTitleContentAndSlugTags()
{
var drafter = await CreateDrafterAsync();
var draft = await drafter.DraftAsync(Statement(), CancellationToken.None);
Assert.False(string.IsNullOrWhiteSpace(draft.Title));
Assert.True(draft.Content.Trim().Length >= RawSource.MinimumContentLength);
Assert.NotEmpty(draft.Tags);
Assert.All(draft.Tags, t => Assert.Matches("^[a-z0-9]+(-[a-z0-9]+)*$", t));
Assert.Equal(draft.Tags.Count, draft.Tags.Distinct().Count());
}
[Fact]
public async Task RefusesAnEmptyStatement()
{
var drafter = await CreateDrafterAsync();
var empty = Statement() with { Text = " " };
await Assert.ThrowsAsync<ArgumentException>(() => drafter.DraftAsync(empty, CancellationToken.None));
}
[Fact]
public async Task ReportsHealth()
{
var drafter = await CreateDrafterAsync();
var health = await drafter.CheckAsync(CancellationToken.None);
Assert.Equal(drafter.Name, health.Adapter);
}
}
+12
View File
@@ -76,6 +76,18 @@ public abstract class NoteStoreContract
store.CreateAsync("wiki/udm-dns.md", "other", "x", CancellationToken.None));
}
[Fact]
public async Task CreatesABinaryAndRefusesAnExistingPath()
{
var store = await CreateStoreAsync();
var bytes = new byte[] { 0xFF, 0xD8, 0xFF, 0xE0, 1, 2, 3 };
var written = await store.CreateAsync("raw/2026-09-24-new-1.jpg", bytes, "raw: New (attachment 1)", CancellationToken.None);
Assert.Equal("raw/2026-09-24-new-1.jpg", written.Path);
Assert.False(string.IsNullOrWhiteSpace(written.Commit));
await Assert.ThrowsAsync<DocumentExistsException>(() =>
store.CreateAsync("raw/2026-09-24-new-1.jpg", bytes, "x", CancellationToken.None));
}
[Fact]
public async Task HealthReportsOk()
{
+75
View File
@@ -0,0 +1,75 @@
using System.Net;
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text;
using Microsoft.AspNetCore.Mvc.Testing;
using Wiki.Host.Auth;
namespace Wiki.Host.Tests;
/// <summary>A Secret-like directory: one user per hash format Traefik accepts, split over two keys, plus a
/// hidden entry that must be skipped. Every password is the user name.</summary>
public static class BasicUsers
{
// From the Traefik BasicAuth docs (htpasswd -m, password "test"); openssl passwd -apr1 agrees.
public const string Apr1Line = "test:$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8wj/";
public static readonly string Directory = Write();
private static string Write()
{
var dir = Path.Combine(Path.GetTempPath(), "wiki-host-tests", $"htpasswd-{Guid.NewGuid():N}");
System.IO.Directory.CreateDirectory(dir);
var bcrypt = BCrypt.Net.BCrypt.HashPassword("agent", 4).Replace("$2a$", "$2y$");
var sha = "{SHA}" + Convert.ToBase64String(System.Security.Cryptography.SHA1.HashData(Encoding.UTF8.GetBytes("legacy")));
File.WriteAllLines(Path.Combine(dir, "users"), [Apr1Line, $"agent:{bcrypt}"]);
File.WriteAllLines(Path.Combine(dir, "more"), [$"legacy:{sha}"]);
File.WriteAllLines(Path.Combine(dir, "..hidden"), ["ghost:{SHA}" + Convert.ToBase64String(System.Security.Cryptography.SHA1.HashData(Encoding.UTF8.GetBytes("ghost")))]);
return dir;
}
}
public class BasicAuthTests(WikiHostFactory factory) : IClassFixture<WikiHostFactory>
{
private HttpClient As(string user, string password)
{
var client = factory.CreateClient(new WebApplicationFactoryClientOptions { AllowAutoRedirect = false });
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(Encoding.UTF8.GetBytes($"{user}:{password}")));
return client;
}
[Fact]
public void Apr1MatchesApacheHtpasswd() =>
Assert.Equal("$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8wj/", Htpasswd.Apr1("test", "H6uskkkW"));
[Theory]
[InlineData("test", "test")] // apr1
[InlineData("agent", "agent")] // bcrypt $2y$
[InlineData("legacy", "legacy")] // {SHA}
public async Task TheMachineDoorCredentialsReadTheApi(string user, string password)
{
var client = As(user, password);
Assert.Equal(HttpStatusCode.OK, (await client.GetAsync("/api/v1/tags")).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await client.PostAsJsonAsync("/rank", new { query = " " })).StatusCode);
}
[Theory]
[InlineData("test", "wrong")]
[InlineData("nobody", "test")]
[InlineData("agent", "")]
[InlineData("ghost", "ghost")] // only in a hidden entry
public async Task WrongCredentialsAre401(string user, string password) =>
Assert.Equal(HttpStatusCode.Unauthorized, (await As(user, password).GetAsync("/api/v1/tags")).StatusCode);
[Fact]
public async Task BasicCredentialsCannotWrite()
{
var r = await As("test", "test").PostAsJsonAsync("/api/v1/facts", new { title = "t", source = "s", kind = "fact", content = new string('x', 50) });
Assert.Equal(HttpStatusCode.Unauthorized, r.StatusCode);
}
[Fact]
public async Task BasicCredentialsDoNotOpenTheUi() =>
Assert.Equal(HttpStatusCode.Redirect, (await As("test", "test").GetAsync("/facts")).StatusCode);
}
+44 -22
View File
@@ -20,16 +20,20 @@ public sealed class WikiHostFactory : WebApplicationFactory<Program>
public const string WebhookSecret = "hook-secret";
public const string ApiKey = "machine-key";
public FakeStore Store { get; } = new();
public FakeIdp Idp { get; } = new();
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Testing");
builder.UseSetting("Wiki:Webhook:Secret", WebhookSecret);
builder.UseSetting("Auth:ApiKeys:tester", ApiKey);
builder.UseSetting("Auth:ReverseProxy:TrustedProxies:0", "127.0.0.1");
builder.UseSetting("Auth:Oidc:Authority", FakeIdp.Issuer);
builder.UseSetting("Auth:Oidc:ClientId", FakeIdp.ClientId);
builder.UseSetting("Auth:Basic:UsersPath", BasicUsers.Directory);
builder.UseSetting("Wiki:Index:File:Directory", Path.Combine(Path.GetTempPath(), "wiki-host-tests", Guid.NewGuid().ToString("N")));
builder.ConfigureServices(services =>
{
Idp.Register(services);
services.RemoveAll<INoteStore>();
services.AddSingleton<INoteStore>(Store);
// The embedding model registered under the active name is replaced by a deterministic fake.
@@ -48,6 +52,11 @@ public sealed class WikiHostFactory : WebApplicationFactory<Program>
services.RemoveAll<IAnswerModel>();
services.AddSingleton<IAnswerModel>(answer);
services.AddSingleton<IReadOnlyList<IAnswerModel>>([answer]);
var drafter = new FakeFactDrafter();
services.AddKeyedSingleton<IFactDrafter>("claude-sonnet", drafter);
services.RemoveAll<IFactDrafter>();
services.AddSingleton<IFactDrafter>(drafter);
services.AddSingleton<IReadOnlyList<IFactDrafter>>([drafter]);
});
}
}
@@ -56,6 +65,17 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
{
private HttpClient Client => factory.CreateClient();
/// <summary>A machine caller: the read API needs a credential since humans sign in with OIDC.</summary>
private HttpClient Keyed
{
get
{
var client = factory.CreateClient();
client.DefaultRequestHeaders.Add("X-Api-Key", WikiHostFactory.ApiKey);
return client;
}
}
[Fact]
public async Task HealthIsAlwaysOk()
{
@@ -91,9 +111,10 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
[Fact]
public async Task RankRejectsEmptyQueryAndUnknownModel()
{
var empty = await Client.PostAsJsonAsync("/rank", new { query = " " });
var client = Keyed;
var empty = await client.PostAsJsonAsync("/rank", new { query = " " });
Assert.Equal(HttpStatusCode.BadRequest, empty.StatusCode);
var unknown = await Client.PostAsJsonAsync("/rank", new { query = "x", model = "nope" });
var unknown = await client.PostAsJsonAsync("/rank", new { query = "x", model = "nope" });
Assert.Equal(HttpStatusCode.BadRequest, unknown.StatusCode);
}
@@ -103,9 +124,9 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
factory.Store.Files["wiki/udm-dns.md"] = "---\ntitle: UDM intercepts DNS\ntype: wiki\ntags: [dns]\n---\n\n# UDM intercepts DNS\n\nThe UDM answers.\n";
var req = new HttpRequestMessage(HttpMethod.Post, "/api/v1/sync?force=true");
req.Headers.Add("X-Api-Key", WikiHostFactory.ApiKey);
await Client.SendAsync(req);
await Keyed.SendAsync(req);
var r = await Client.PostAsJsonAsync("/rank", new { query = "udm dns", k = 3 });
var r = await Keyed.PostAsJsonAsync("/rank", new { query = "udm dns", k = 3 });
Assert.Equal(HttpStatusCode.OK, r.StatusCode);
var body = await r.Content.ReadFromJsonAsync<RankResponse>();
Assert.NotNull(body);
@@ -144,29 +165,29 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
factory.Store.Files["output/2026-09-21-wiki-audit.md"] = "---\ntitle: Wiki audit\ntype: audit\ngenerated: 2026-09-21\n---\n\nreport\n";
var req = new HttpRequestMessage(HttpMethod.Post, "/api/v1/sync?force=true");
req.Headers.Add("X-Api-Key", WikiHostFactory.ApiKey);
Assert.Equal(HttpStatusCode.OK, (await Client.SendAsync(req)).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await Keyed.SendAsync(req)).StatusCode);
var notes = await Client.GetFromJsonAsync<List<NoteRow>>("/api/v1/notes?tag=ceph");
var notes = await Keyed.GetFromJsonAsync<List<NoteRow>>("/api/v1/notes?tag=ceph");
Assert.Equal("ceph-osd", Assert.Single(notes!).Slug);
var note = await Client.GetFromJsonAsync<NoteBody>("/api/v1/notes/ceph-osd");
var note = await Keyed.GetFromJsonAsync<NoteBody>("/api/v1/notes/ceph-osd");
Assert.NotNull(note);
Assert.Contains("See [[udm-dns]]", note.Body);
Assert.Equal(2, note.Links.Count);
Assert.Equal(HttpStatusCode.NotFound, (await Client.GetAsync("/api/v1/notes/nope")).StatusCode);
Assert.Equal(HttpStatusCode.NotFound, (await Keyed.GetAsync("/api/v1/notes/nope")).StatusCode);
var back = await Client.GetFromJsonAsync<NoteBody>("/api/v1/notes/udm-dns");
var back = await Keyed.GetFromJsonAsync<NoteBody>("/api/v1/notes/udm-dns");
Assert.Equal(2, back!.Backlinks.Count);
var tags = await Client.GetFromJsonAsync<List<TagRow>>("/api/v1/tags");
var tags = await Keyed.GetFromJsonAsync<List<TagRow>>("/api/v1/tags");
Assert.Contains(tags!, t => t.Tag == "ceph" && t.Count == 1);
var raw = await Client.GetFromJsonAsync<List<RawRow>>("/api/v1/raw");
var raw = await Keyed.GetFromJsonAsync<List<RawRow>>("/api/v1/raw");
Assert.Contains(raw!, r => r.Slug == "2026-09-23-pending");
var reports = await Client.GetFromJsonAsync<List<ReportRow>>("/api/v1/reports?type=audit");
var reports = await Keyed.GetFromJsonAsync<List<ReportRow>>("/api/v1/reports?type=audit");
Assert.Equal("Wiki audit", Assert.Single(reports!).Title);
Assert.Equal(HttpStatusCode.OK, (await Client.GetAsync("/api/v1/reports/2026-09-21-wiki-audit")).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await Keyed.GetAsync("/api/v1/reports/2026-09-21-wiki-audit")).StatusCode);
}
[Fact]
@@ -175,12 +196,12 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
factory.Store.Files["wiki/udm-dns.md"] = "---\ntitle: UDM intercepts DNS\ntype: wiki\ntags: [dns]\n---\n\n# UDM intercepts DNS\n\nThe UDM answers every query.\n";
var req = new HttpRequestMessage(HttpMethod.Post, "/api/v1/sync?force=true");
req.Headers.Add("X-Api-Key", WikiHostFactory.ApiKey);
await Client.SendAsync(req);
await Keyed.SendAsync(req);
var bad = await Client.PostAsJsonAsync("/api/v1/ask", new { queries = Array.Empty<string>() });
var bad = await Keyed.PostAsJsonAsync("/api/v1/ask", new { queries = Array.Empty<string>() });
Assert.Equal(HttpStatusCode.BadRequest, bad.StatusCode);
var r = await Client.PostAsJsonAsync("/api/v1/ask", new
var r = await Keyed.PostAsJsonAsync("/api/v1/ask", new
{
queries = new[] { "udm dns" },
k = 3,
@@ -247,13 +268,13 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
factory.Store.Files["wiki/udm-dns.md"] = "---\ntitle: UDM intercepts DNS\ntype: wiki\ntags: [dns]\n---\n\n# UDM intercepts DNS\n\nThe UDM answers every query.\n";
var req = new HttpRequestMessage(HttpMethod.Post, "/api/v1/sync?force=true");
req.Headers.Add("X-Api-Key", WikiHostFactory.ApiKey);
await Client.SendAsync(req);
await Keyed.SendAsync(req);
Assert.Equal(HttpStatusCode.BadRequest, (await Client.PostAsJsonAsync("/api/v1/questions", new { question = " " })).StatusCode);
Assert.Equal(HttpStatusCode.BadRequest, (await Keyed.PostAsJsonAsync("/api/v1/questions", new { question = " " })).StatusCode);
var unauthorizedSave = await Client.PostAsJsonAsync("/api/v1/questions", new { question = "udm dns", save = true });
Assert.Equal(HttpStatusCode.Unauthorized, unauthorizedSave.StatusCode);
var r = await Client.PostAsJsonAsync("/api/v1/questions", new { question = "udm dns", k = 3 });
var r = await Keyed.PostAsJsonAsync("/api/v1/questions", new { question = "udm dns", k = 3 });
Assert.Equal(HttpStatusCode.OK, r.StatusCode);
var body = await r.Content.ReadFromJsonAsync<QuestionResponse>();
Assert.NotNull(body);
@@ -264,7 +285,7 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
var save = new HttpRequestMessage(HttpMethod.Post, "/api/v1/questions") { Content = JsonContent.Create(new { question = "udm dns", k = 3, save = true }) };
save.Headers.Add("X-Api-Key", WikiHostFactory.ApiKey);
var saved = await (await Client.SendAsync(save)).Content.ReadFromJsonAsync<QuestionResponse>();
var saved = await (await Keyed.SendAsync(save)).Content.ReadFromJsonAsync<QuestionResponse>();
Assert.NotNull(saved?.Saved);
Assert.StartsWith("output/", saved.Saved.Path);
Assert.Contains("type: output", factory.Store.Files[saved.Saved.Path]);
@@ -313,7 +334,8 @@ public class EndpointTests(WikiHostFactory factory) : IClassFixture<WikiHostFact
[Fact]
public async Task StatusListsAdaptersAndIndexes()
{
var status = await Client.GetFromJsonAsync<StatusResponse>("/api/v1/status");
var client = Keyed;
var status = await client.GetFromJsonAsync<StatusResponse>("/api/v1/status");
Assert.NotNull(status);
Assert.Contains(status.Indexes, i => i.ModelId == "bge-m3" && i.Active);
Assert.NotEmpty(status.Adapters);
+95
View File
@@ -0,0 +1,95 @@
using System.Net;
using System.Security.Claims;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Microsoft.AspNetCore.Authentication.OpenIdConnect;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.IdentityModel.JsonWebTokens;
using Microsoft.IdentityModel.Protocols;
using Microsoft.IdentityModel.Protocols.OpenIdConnect;
using Microsoft.IdentityModel.Tokens;
namespace Wiki.Host.Tests;
/// <summary>
/// An in-process OpenID provider: static discovery, a token endpoint that issues RS256 ID tokens and a
/// userinfo endpoint, reached through the OIDC handler's backchannel. Records what the wiki sent.
/// </summary>
public sealed class FakeIdp : HttpMessageHandler
{
public const string Issuer = "https://idp.test/application/o/wiki/";
public const string ClientId = "wiki";
private const string AccessToken = "access-token";
private readonly RsaSecurityKey _key = new(RSA.Create(2048)) { KeyId = "idp-key" };
/// <summary>The user the next token response is issued for.</summary>
public string Username { get; set; } = "alice";
/// <summary>Signs the ID token with a key the wiki does not trust when set.</summary>
public bool SignWithForeignKey { get; set; }
/// <summary>The nonce the wiki put into the authorize request; the ID token must echo it.</summary>
public string? Nonce { get; set; }
public List<Dictionary<string, string>> TokenRequests { get; } = [];
public OpenIdConnectConfiguration Configuration { get; }
public FakeIdp()
{
Configuration = new OpenIdConnectConfiguration
{
Issuer = Issuer,
AuthorizationEndpoint = "https://idp.test/application/o/authorize/",
TokenEndpoint = "https://idp.test/application/o/token/",
UserInfoEndpoint = "https://idp.test/application/o/userinfo/",
};
Configuration.SigningKeys.Add(_key);
Configuration.CodeChallengeMethodsSupported.Add("S256");
}
public void Register(IServiceCollection services) =>
services.PostConfigure<OpenIdConnectOptions>(OpenIdConnectDefaults.AuthenticationScheme, o =>
{
o.Configuration = Configuration;
o.ConfigurationManager = new StaticConfigurationManager<OpenIdConnectConfiguration>(Configuration);
o.Backchannel = new HttpClient(this);
});
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken ct)
{
var url = request.RequestUri!.ToString();
if (url == Configuration.TokenEndpoint)
{
var form = await request.Content!.ReadAsStringAsync(ct);
TokenRequests.Add(form.Split('&').Select(kv => kv.Split('=', 2))
.ToDictionary(kv => Uri.UnescapeDataString(kv[0]), kv => Uri.UnescapeDataString(kv[1].Replace('+', ' '))));
return Json(new { access_token = AccessToken, token_type = "Bearer", expires_in = 300, id_token = IdToken() });
}
if (url == Configuration.UserInfoEndpoint)
return Json(new { sub = Sub, preferred_username = Username });
return new HttpResponseMessage(HttpStatusCode.NotFound);
}
private string Sub => Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(Username)));
private string IdToken()
{
var key = SignWithForeignKey ? new RsaSecurityKey(RSA.Create(2048)) { KeyId = "idp-key" } : _key;
var atHash = Base64UrlEncoder.Encode(SHA256.HashData(Encoding.ASCII.GetBytes(AccessToken))[..16]);
var now = DateTime.UtcNow;
return new JsonWebTokenHandler().CreateToken(new SecurityTokenDescriptor
{
Issuer = Issuer,
Audience = ClientId,
IssuedAt = now,
NotBefore = now,
Expires = now.AddMinutes(5),
Subject = new ClaimsIdentity([new Claim("sub", Sub), new Claim("preferred_username", Username)]),
Claims = new Dictionary<string, object> { ["nonce"] = Nonce ?? string.Empty, ["at_hash"] = atHash },
SigningCredentials = new SigningCredentials(key, SecurityAlgorithms.RsaSha256),
});
}
private static HttpResponseMessage Json(object body) =>
new(HttpStatusCode.OK) { Content = new StringContent(JsonSerializer.Serialize(body), Encoding.UTF8, "application/json") };
}
+26
View File
@@ -41,6 +41,17 @@ public sealed class FakeStore : INoteStore
return Task.FromResult(new StoreWrite(path, "c"));
}
public Dictionary<string, byte[]> Binaries { get; } = new();
public Task<StoreWrite> CreateAsync(string path, byte[] content, string message, CancellationToken ct)
{
if (Files.ContainsKey(path) || Binaries.ContainsKey(path)) throw new DocumentExistsException(path);
Binaries[path] = content;
return Task.FromResult(new StoreWrite(path, Convert.ToHexStringLower(SHA1.HashData(content))));
}
public Uri? BrowseUrl(string path) => null;
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => Task.FromResult(new AdapterHealth(Name, true, null, TimeSpan.Zero, DateTimeOffset.UnixEpoch));
}
@@ -105,3 +116,18 @@ public sealed class FakeEmbeddings(string modelId) : IEmbeddingModel
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => Task.FromResult(new AdapterHealth(Name, true, null, TimeSpan.Zero, DateTimeOffset.UnixEpoch));
}
public sealed class FakeFactDrafter : IFactDrafter
{
public string ModelId => "fake-drafter";
public string Name => "fake-drafter";
public Task<FactDraft> DraftAsync(FactStatement statement, CancellationToken ct)
{
var words = statement.Text.Split(' ', StringSplitOptions.RemoveEmptyEntries);
var title = string.Join(' ', words.Take(5));
return Task.FromResult(new FactDraft(title, statement.Text, ["fake"]));
}
public Task<AdapterHealth> CheckAsync(CancellationToken ct) => Task.FromResult(new AdapterHealth(Name, true, null, TimeSpan.Zero, DateTimeOffset.UnixEpoch));
}
+126
View File
@@ -0,0 +1,126 @@
using System.Net;
using System.Net.Http.Json;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.WebUtilities;
namespace Wiki.Host.Tests;
/// <summary>
/// Human sign-in: OIDC against the provider, nothing else. A browser without a session is sent to the
/// provider, API callers get a status, and no request header names a user.
/// </summary>
public class OidcTests(WikiHostFactory factory) : IClassFixture<WikiHostFactory>
{
private HttpClient Browser() => factory.CreateClient(new WebApplicationFactoryClientOptions
{
AllowAutoRedirect = false,
HandleCookies = true,
BaseAddress = new Uri("https://localhost"),
});
[Fact]
public async Task AnonymousBrowserIsSentToTheProviderWithPkce()
{
var page = await Browser().GetAsync("/facts");
Assert.Equal(HttpStatusCode.Redirect, page.StatusCode);
var location = page.Headers.Location!;
Assert.StartsWith(factory.Idp.Configuration.AuthorizationEndpoint, location.ToString());
var query = QueryHelpers.ParseQuery(location.Query);
Assert.Equal(FakeIdp.ClientId, query["client_id"]);
Assert.Equal("code", query["response_type"]);
Assert.Equal("S256", query["code_challenge_method"]);
Assert.False(string.IsNullOrEmpty(query["code_challenge"]));
Assert.Equal("https://localhost/signin-oidc", query["redirect_uri"]);
}
[Fact]
public async Task TheRedirectUriIsHttpsOnTheRequestHostEvenBehindTlsTermination()
{
// Traefik ends TLS and forwards plain http with the original Host.
var behindProxy = factory.CreateClient(new WebApplicationFactoryClientOptions
{
AllowAutoRedirect = false,
BaseAddress = new Uri("http://wiki.test"),
});
var page = await behindProxy.GetAsync("/facts");
var query = QueryHelpers.ParseQuery(page.Headers.Location!.Query);
Assert.Equal("https://wiki.test/signin-oidc", query["redirect_uri"]);
}
[Fact]
public async Task TheAuthentikHeaderSignsNobodyIn()
{
var browser = Browser();
var page = new HttpRequestMessage(HttpMethod.Get, "/facts");
page.Headers.Add("X-Authentik-Username", "k.renner");
Assert.Equal(HttpStatusCode.Redirect, (await browser.SendAsync(page)).StatusCode);
var api = new HttpRequestMessage(HttpMethod.Get, "/api/v1/notes");
api.Headers.Add("X-Authentik-Username", "k.renner");
Assert.Equal(HttpStatusCode.Unauthorized, (await browser.SendAsync(api)).StatusCode);
}
[Fact]
public async Task ApiWithoutCredentialsAnswers401NotARedirect()
{
var browser = Browser();
Assert.Equal(HttpStatusCode.Unauthorized, (await browser.GetAsync("/api/v1/notes")).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await browser.GetAsync("/api/v1/status")).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await browser.PostAsJsonAsync("/rank", new { query = "x" })).StatusCode);
Assert.Equal(HttpStatusCode.Unauthorized, (await browser.PostAsJsonAsync("/api/v1/questions", new { question = "x" })).StatusCode);
Assert.Equal(HttpStatusCode.OK, (await browser.GetAsync("/health")).StatusCode);
}
[Fact]
public async Task SignInTakesTheUsernameFromTheVerifiedIdToken()
{
var browser = Browser();
var (code, redirect) = await SignInAsync(browser, "alice");
Assert.Equal(HttpStatusCode.Redirect, code);
Assert.Equal("/facts", redirect);
var token = factory.Idp.TokenRequests[^1];
Assert.Equal("https://localhost/signin-oidc", token["redirect_uri"]);
Assert.True(token.ContainsKey("code_verifier"));
Assert.False(token.ContainsKey("client_secret"));
var page = await browser.GetAsync("/facts");
Assert.Equal(HttpStatusCode.OK, page.StatusCode);
var html = await page.Content.ReadAsStringAsync();
Assert.Contains("captured_by: alice", html);
// A signed-in human may read the API with the session cookie.
Assert.Equal(HttpStatusCode.OK, (await browser.GetAsync("/api/v1/tags")).StatusCode);
}
[Fact]
public async Task AnIdTokenSignedByAnotherKeyIsRejected()
{
factory.Idp.SignWithForeignKey = true;
try
{
var browser = Browser();
await SignInAsync(browser, "mallory");
Assert.Equal(HttpStatusCode.Redirect, (await browser.GetAsync("/facts")).StatusCode);
}
finally
{
factory.Idp.SignWithForeignKey = false;
}
}
/// <summary>Follows the code flow by hand: challenge, then the provider's form_post back to the callback.</summary>
private async Task<(HttpStatusCode, string?)> SignInAsync(HttpClient browser, string username)
{
var challenge = await browser.GetAsync("/facts");
var query = QueryHelpers.ParseQuery(challenge.Headers.Location!.Query);
factory.Idp.Username = username;
factory.Idp.Nonce = query["nonce"];
var callback = await browser.PostAsync("/signin-oidc", new FormUrlEncodedContent(new Dictionary<string, string>
{
["code"] = "the-code",
["state"] = query["state"].ToString(),
}));
return (callback.StatusCode, callback.Headers.Location?.OriginalString);
}
}