Skip to content

Pattern quality: Flag missing package-registry ecosystem in dependency-monitor network config - #228

Draft
github-actions[bot] wants to merge 1 commit into
mainfrom
fix/dependency-monitor-network-ecosystem-a999e282a0c7574f
Draft

Pattern quality: Flag missing package-registry ecosystem in dependency-monitor network config#228
github-actions[bot] wants to merge 1 commit into
mainfrom
fix/dependency-monitor-network-ecosystem-a999e282a0c7574f

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

What was evaluated

I read all 6 sample generated pattern/prompt/workflow triples in /tmp/gh-aw/data/generated-patterns-and-prompts.json (status-report, issue-triage, code-improvement, documentation-updater, dependency-monitor, pr-review), plus patterns/manifest.json, patterns/workflow-generation.json, the relevant patterns/archetypes/*.json files, and the generator in src/js/workflow.js.

Recurring quality issue found

dependency-monitor is the only archetype with capabilities.network: true, and it is the archetype whose job is explicitly to "check upstream repos or package registries for new versions." generateWorkflowFile() always emits:

network:
  allowed:
    - defaults
    - github

Per gh-aw's own network.md reference: defaults only covers basic infra (CAs, apt/OS metadata) and github only covers GitHub domains — neither includes any package-registry ecosystem (noderegistry.npmjs.org, pythonpypi.org, goproxy.golang.org, etc.), and the doc explicitly warns "never use network: defaults alone for workflows that build, test, or install packages." Since the wizard is a static site with no knowledge of the target repo's package manager at generation time, it currently ships a network allowlist that looks complete but would silently block the archetype's own core pre-step (checking registries for new releases) in essentially every real repository. This is a defect specific to the generator's output for this archetype, not a downstream-agent mistake — it would recur identically across any repo that uses this archetype.

What changed

  • src/js/workflow.js: when inferred.network is set, append an inline TODO comment to the generated network.allowed block instructing the downstream agent to add the repository's package ecosystem identifier(s) (e.g. node, python, go, rust), with a link to gh-aw's network.md.
  • patterns/archetypes/dependency-monitor.json: added a new tip surfacing the same gap, so it also appears as an explicit boundary-constraint requirement in the wizard's generated agent prompt.
  • test/workflow.test.js: updated the existing dependency-monitor network test to assert the new TODO comment is present.

No other archetypes were touched — this is the only archetype with network: true.

Validation

  • npm test: 295 passed, 9 pre-existing failures (all live-network-fetch tests in patterns.test.js/workflow.test.js that fail identically on the unmodified main branch in this sandboxed environment — confirmed via git stash comparison before making changes). No new failures introduced by this change.
  • npm run build: succeeds, dist/patterns/archetypes/dependency-monitor.json includes the new tip.

Follow-ups deliberately left out

  • Attempting to auto-infer the ecosystem from repo files client-side (the wizard has no access to the target repo's file tree at generation time — that inference has to happen downstream, which this change now explicitly asks for).
  • Auditing other archetypes for similar "capability implies concrete config the generator can't determine" gaps; scoped this fix to the one concrete, verifiable case found.

Generated by Pattern Quality Eval · auto · 140.8 AIC · ⌖ 8.37 AIC · ⊞ 7.6K ·

… config

The dependency-monitor archetype is the only one that enables
capabilities.network, and generateWorkflowFile() always emits
network.allowed: [defaults, github] for it. Per gh-aw's network.md,
"defaults" and "github" never include package-registry domains
(registry.npmjs.org, pypi.org, proxy.golang.org, etc.), so the
generated pre-step that "checks upstream repos or package registries
for new versions" would be silently blocked by the firewall in every
repository that actually has a package manager to check.

- Add an inline TODO comment in the generated network.allowed block
  telling the downstream agent to add the repo's ecosystem
  identifier(s), with a link to gh-aw's network.md.
- Surface the same gap as an explicit tip on the dependency-monitor
  archetype so it appears in the wizard's generated agent prompt
  under "boundary constraints".
- Update the existing workflow.test.js expectation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants