feat(review-bot): Anthropic web fetch for upstream release notes (stacked on #1610) #1611

Merged
k.renner merged 1 commits from daedalus/review-bot-web-fetch into main 2026-11-09 07:13:35 +00:00
Member

Stacked on #1610: merge that first. This branch contains its commit. Review only the last commit (feat(review-bot): Anthropic web fetch ...).

Gives the Anthropic backend Anthropic's server-side web fetch tool, so the reviewer can read upstream release notes and compare pages that the Renovate PR body doesn't carry. Example: kaniko from registry.gitlab.com has no source label, so Renovate links no notes.

Fit with the guard and the network policy (checked against platform.claude.com docs)

  • Network policy: no change. The fetch runs on Anthropic's side, as a server tool. The pod still only reaches Gitea (Traefik) and the model APIs.
  • Tool guard: extended. Before this, Ask looked only at mcp_tool_use and would have ignored a server_tool_use. Now any server tool call other than web_fetch, or a web_fetch while it is off, counts as a violation, and the review is not posted.
  • URL source: needed a code change. Web fetch only opens URLs from user messages, client tool results, or earlier fetch/search results. Results from the MCP connector don't count (docs: web-fetch-tool, "URL validation"). So the PR body that the agent reads through MCP does not make its links fetchable. The task message therefore now lists the PR body's https links on the allowed domains: no userinfo, deduplicated, at most 20, and labelled as untrusted data. With web fetch off, the task text is unchanged.

Trust-boundary change

  • Before: the reviewer read only Gitea, through a read-only token. Nothing outside reached the model.
  • After:
    • Anthropic fetches pages on github.com and gitlab.com (allowed_domains, subdomains included), at most 10 per review (WEB_FETCH_MAX_USES), each truncated at about 20k tokens. Basic version web_fetch_20250910: the later versions turn on code execution.
    • Which URLs it may open is decided by the PR description (author: Renovate, i.e. upstream metadata, or a human) and by the links on pages it fetched. Fetched page text is a new channel for prompt injection. The prompt already treats all of it as data, and the read-only guard bounds what an injected instruction can do: there is still no write tool, and a violation blocks the post.
    • Exfiltration: the model cannot fetch URLs that it composes itself. A URL that carries a credential is refused unless it is in the user message. Repo content can only leave via a URL already present in the body/fetched pages on the two domains. This is the residual risk Anthropic's docs warn about.
  • Off switch: remove WEB_FETCH_DOMAINS from kubernetes/apps/review-bot/deployment.yaml.

Not verified

  • I could not send a request to the real API: I have no Anthropic key, and the review-bot Secret isn't readable to me. Combining web_fetch_20250910 with the MCP toolset and the server-side fallback beta is tested only against the fake in fetch_test.go. The first review after rollout is the real test. If Anthropic rejects the tool combination, every Anthropic review fails at once (visible in kubectl -n review-bot logs deploy/review-bot). Then remove the env var.

Verification

  • go vet, go test ./... green.
  • New tests: link filter and cap, task text with web fetch on and off, tool definition in the request, guard with web fetch on and off.

Hermes-Card: t_81d3ebcd (daedalus)

**Stacked on #1610: merge that first.** This branch contains its commit. Review only the last commit (`feat(review-bot): Anthropic web fetch ...`). Gives the Anthropic backend Anthropic's server-side **web fetch** tool, so the reviewer can read upstream release notes and compare pages that the Renovate PR body doesn't carry. Example: kaniko from registry.gitlab.com has no source label, so Renovate links no notes. ## Fit with the guard and the network policy (checked against platform.claude.com docs) - **Network policy: no change.** The fetch runs on Anthropic's side, as a server tool. The pod still only reaches Gitea (Traefik) and the model APIs. - **Tool guard: extended.** Before this, `Ask` looked only at `mcp_tool_use` and would have ignored a `server_tool_use`. Now any server tool call other than `web_fetch`, or a `web_fetch` while it is off, counts as a violation, and the review is not posted. - **URL source: needed a code change.** Web fetch only opens URLs from user messages, client tool results, or earlier fetch/search results. Results from the MCP connector don't count (docs: web-fetch-tool, "URL validation"). So the PR body that the agent reads through MCP does not make its links fetchable. The task message therefore now lists the PR body's `https` links on the allowed domains: no userinfo, deduplicated, at most 20, and labelled as untrusted data. With web fetch off, the task text is unchanged. ## Trust-boundary change - **Before:** the reviewer read only Gitea, through a read-only token. Nothing outside reached the model. - **After:** - Anthropic fetches pages on `github.com` and `gitlab.com` (`allowed_domains`, subdomains included), at most 10 per review (`WEB_FETCH_MAX_USES`), each truncated at about 20k tokens. Basic version `web_fetch_20250910`: the later versions turn on code execution. - Which URLs it may open is decided by the PR description (author: Renovate, i.e. upstream metadata, or a human) and by the links on pages it fetched. Fetched page text is a new channel for prompt injection. The prompt already treats all of it as data, and the read-only guard bounds what an injected instruction can do: there is still no write tool, and a violation blocks the post. - Exfiltration: the model cannot fetch URLs that it composes itself. A URL that carries a credential is refused unless it is in the user message. Repo content can only leave via a URL already present in the body/fetched pages on the two domains. This is the residual risk Anthropic's docs warn about. - **Off switch:** remove `WEB_FETCH_DOMAINS` from `kubernetes/apps/review-bot/deployment.yaml`. ## Not verified - I could not send a request to the real API: I have no Anthropic key, and the review-bot Secret isn't readable to me. Combining `web_fetch_20250910` with the MCP toolset and the server-side fallback beta is tested only against the fake in `fetch_test.go`. The first review after rollout is the real test. If Anthropic rejects the tool combination, every Anthropic review fails at once (visible in `kubectl -n review-bot logs deploy/review-bot`). Then remove the env var. ## Verification - `go vet`, `go test ./...` green. - New tests: link filter and cap, task text with web fetch on and off, tool definition in the request, guard with web fetch on and off. Hermes-Card: t_81d3ebcd (daedalus)
hermes-bot added 2 commits 2026-11-09 06:43:16 +00:00
feat(review-bot,renovate): useful reviews of dependency updates
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/review-bot-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
354d2b18d7
The review-bot review of konairius/ical-proxy#22 (golang/alpine digests,
kaniko v1.25.20 -> v1.25.21) restated the diff and listed "digests and CI
not checked" as unverified. Three causes:

- Renovate wrote no release notes: it has no github.com token ("No
  github.com token has been configured. Skipping release notes retrieval",
  job renovate-29903220). The ExternalSecret now renders
  RENOVATE_GITHUB_COM_TOKEN (the current name; GITHUB_COM_TOKEN is
  deprecated) from its own OpenBao key apps/renovate/github, field `token`.
  No other service reads that key.
- defaultMCPTools still named per-method tools that gitea-mcp 1.13 folded
  into commit_read, branch_read, tag_read and release_read. tools/list on
  gitea-mcp.renner.uno has none of get_commit, list_commits,
  compare_commits, list_branches, get_tag, list_tags, get_release,
  get_latest_release, list_releases, so 9 of the 20 enabled names matched
  nothing and the prompt's compare_commits/list_commits steps had no tool.
  The list now has the 1.13 names; a test keeps it read-only.
- prompt.md had no guidance for dependency updates. New "Dependency
  updates" section: release notes between old and new version, the repo's
  use of what they change (path:line), one line for digest-only rebuilds,
  no unverified boilerplate. New mandatory check: CI status of the head
  (pull_request_read get_status; Woodpecker reports commit statuses).
feat(review-bot): Anthropic web fetch for upstream release notes
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/review-bot-test Pipeline was successful
c19df81858
Renovate PR bodies do not cover every dependency (kaniko from
registry.gitlab.com has no source URL, so Renovate links no notes), and
the reviewer had no way to read upstream changes. The Anthropic backend
now offers the server-side web fetch tool when WEB_FETCH_DOMAINS is set;
the deployment sets github.com,gitlab.com.

- Version web_fetch_20250910 (basic): later versions turn on code
  execution for dynamic filtering. max_uses 10 (WEB_FETCH_MAX_USES),
  max_content_tokens 20000 per page.
- Anthropic only fetches URLs that appeared in a user message, a client
  tool result or an earlier fetch/search result; MCP connector results do
  not count. So the task message now lists the https links of the PR body
  on the allowed domains (no userinfo, deduplicated, at most 20), marked
  as untrusted data. Without web fetch the task is unchanged.
- Tool guard: a server_tool_use other than web_fetch, or web_fetch while
  it is off, is a violation, so the review is not posted.
- Network policy: unchanged. The fetch runs on Anthropic's side; the pod
  still only talks to Gitea (Traefik) and the model APIs.
hermes-bot marked the pull request as ready for review 2026-11-09 06:43:20 +00:00
hermes-bot requested review from review-bot 2026-11-09 07:01:32 +00:00
review-bot approved these changes 2026-11-09 07:05:19 +00:00
review-bot left a comment
Member

🤖 Automated review — model claude-opus-5-5, commit c19df818588d95a99b7af2d91dcd8c433e4739d5.

Automated review by the review-bot: it approves only when it found nothing red or yellow, and a human still merges. Request a review from review-bot again for a fresh one.


Summary

This PR gives the Anthropic backend Anthropic's server-side web_fetch tool (web_fetch_20250910), limited to github.com and gitlab.com. Fetches are capped by WEB_FETCH_MAX_USES (default 10) and pages are truncated at 20k tokens. The task message now lists the PR body's https links on those domains, at most 20, as untrusted data. The read-only guard now also treats any server tool call other than an enabled web_fetch as a violation. As the body says, the branch also contains #1610's commit (354d2b18), so merging this PR first would merge #1610 too; I reviewed only the last commit c19df818. CI on the head is success for all three contexts (ci-scripts-test, ci-manifests-test, review-bot-test), and mergeable is true. The change is careful and the trust-boundary change is documented in the PR body. I found no defects.

⚪ Unverified

  • Real API request: the PR body says it was never sent to the real API. It combines web_fetch_20250910 with the MCP toolset and the server-side-fallback beta (review-bot/anthropic.go:79-90), and is tested only against the fake. I can't check whether Anthropic accepts that combination. If it doesn't, every Anthropic review fails until WEB_FETCH_DOMAINS is removed, as the PR body says.
  • WEB_FETCH_MAX_USES ≤ 0: review-bot/config.go:161 reads it with number(...), and anthropic.go:83 passes it through as MaxUses without a check. A value of 0 or less would be sent as max_uses: 0 or negative, and I couldn't check how the API handles that. Not reachable with the current deployment.

🟡 Suggestions

  • review-bot/fetch.go:23: TrimRight(raw, ".,;:!?*_") also strips a trailing _ that belongs to a URL. It's harmless for release and compare links, but trimming only sentence punctuation (.,;:!?) plus Markdown emphasis that is clearly paired would be more exact.
  • review-bot/anthropic.go:79: the tool is offered even when the task lists no links (reviewTask returns the plain task). Web fetch can then open no URL at all, and the model may burn turns on failed fetches. One option is to send the tool only when reviewTask produced links, e.g. by passing the link count or a flag into Ask. Optional.

🟢 Nits

  • review-bot/review.go:132: the backend name "anthropic" is a string literal. If the backends map key has a constant, use it.

Tests

The tests are adequate:

  • fetch_test.go covers the domain/userinfo/http filtering, deduplication and the 20-link cap.
  • It covers the task text with web fetch on and off, and with no links.
  • It checks the tool definition in the request body and the guard with web fetch on and off.

A test with a server tool other than web_fetch while web fetch is on (e.g. web_search) would pin down the first half of the guard condition at anthropic.go:125.

Recommendation

Looks reasonable: no blocking issues, and the comments above are optional. Merge #1610 first, as the PR body asks.

🤖 **Automated review** — model `claude-opus-5-5`, commit `c19df818588d95a99b7af2d91dcd8c433e4739d5`. > Automated review by the `review-bot`: it approves only when it found nothing red or yellow, and a human still merges. Request a review from `review-bot` again for a fresh one. --- ## Summary This PR gives the Anthropic backend Anthropic's server-side `web_fetch` tool (`web_fetch_20250910`), limited to `github.com` and `gitlab.com`. Fetches are capped by `WEB_FETCH_MAX_USES` (default 10) and pages are truncated at 20k tokens. The task message now lists the PR body's https links on those domains, at most 20, as untrusted data. The read-only guard now also treats any server tool call other than an enabled `web_fetch` as a violation. As the body says, the branch also contains #1610's commit (`354d2b18`), so merging this PR first would merge #1610 too; I reviewed only the last commit `c19df818`. CI on the head is `success` for all three contexts (`ci-scripts-test`, `ci-manifests-test`, `review-bot-test`), and `mergeable` is true. The change is careful and the trust-boundary change is documented in the PR body. I found no defects. ## ⚪ Unverified - **Real API request:** the PR body says it was never sent to the real API. It combines `web_fetch_20250910` with the MCP toolset and the `server-side-fallback` beta (`review-bot/anthropic.go:79-90`), and is tested only against the fake. I can't check whether Anthropic accepts that combination. If it doesn't, every Anthropic review fails until `WEB_FETCH_DOMAINS` is removed, as the PR body says. - **`WEB_FETCH_MAX_USES` ≤ 0:** `review-bot/config.go:161` reads it with `number(...)`, and `anthropic.go:83` passes it through as `MaxUses` without a check. A value of 0 or less would be sent as `max_uses: 0` or negative, and I couldn't check how the API handles that. Not reachable with the current deployment. ## 🟡 Suggestions - `review-bot/fetch.go:23`: `TrimRight(raw, ".,;:!?*_")` also strips a trailing `_` that belongs to a URL. It's harmless for release and compare links, but trimming only sentence punctuation (`.,;:!?`) plus Markdown emphasis that is clearly paired would be more exact. - `review-bot/anthropic.go:79`: the tool is offered even when the task lists no links (`reviewTask` returns the plain task). Web fetch can then open no URL at all, and the model may burn turns on failed fetches. One option is to send the tool only when `reviewTask` produced links, e.g. by passing the link count or a flag into `Ask`. Optional. ## 🟢 Nits - `review-bot/review.go:132`: the backend name `"anthropic"` is a string literal. If the backends map key has a constant, use it. ## Tests The tests are adequate: - `fetch_test.go` covers the domain/userinfo/http filtering, deduplication and the 20-link cap. - It covers the task text with web fetch on and off, and with no links. - It checks the tool definition in the request body and the guard with web fetch on and off. A test with a server tool other than `web_fetch` while web fetch is on (e.g. `web_search`) would pin down the first half of the guard condition at `anthropic.go:125`. ## Recommendation Looks reasonable: no blocking issues, and the comments above are optional. Merge #1610 first, as the PR body asks. <!-- pr-review:sha=c19df818588d95a99b7af2d91dcd8c433e4739d5 -->
k.renner merged commit e1f626077c into main 2026-11-09 07:13:35 +00:00
Sign in to join this conversation.