Compare commits

...

1239 Commits

Author SHA1 Message Date
Patrick Erichsen
a45b4c970b feat: add clawhub staging deploy workflow
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-05-07 19:24:41 -07:00
Patrick Erichsen
4d3b7dedba
feat: add docs nav link (#2097) 2026-05-07 19:16:04 -07:00
Patrick Erichsen
1aab139775
feat: enforce scoped plugin ownership (#2072) 2026-05-07 19:14:24 -07:00
Patrick Erichsen
88756d5997
docs: mention package transfer in publishing FAQ (#2096) 2026-05-07 19:13:17 -07:00
Patrick Erichsen
8c86d6f570
Fix plugin publish ownership visibility (#2073)
* fix: clarify plugin publish ownership state

* test: tolerate publish route migration in prod smoke

* fix: reserve publish route collisions

* fix: preflight package scope owner mismatches in CLI

* fix: keep package scope validation server-side

* docs: explain ClawHub publishing flow

* fix: include publishing docs link in scope errors

* fix: centralize docs links

* fix: build docs links with URL

* fix: shorten package scope docs hint
2026-05-07 19:03:40 -07:00
Patrick Erichsen
86898837fb
docs: split ClawHub public docs from specs (#2095)
* docs: split clawhub docs source

* docs: make clawhub docs product-facing

* docs: refine public clawhub docs routes
2026-05-07 18:54:47 -07:00
Patrick Erichsen
d7c774996e
docs: add RFC community review process (#2092)
Some checks are pending
CI / playwright-smoke (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
CI / static (push) Waiting to run
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
CI / types-build (push) Waiting to run
CI / e2e-http (push) Waiting to run
* docs: add RFC community review process

* chore: keep slug validator options internal
2026-05-07 14:57:55 -07:00
Vincent Koc
b8e5486f63
chore(ci): harden security ownership and workflow permissions (#2045)
* chore(security): expand protected automation owners

* chore(ci): default workflows to no token permissions
2026-05-07 01:13:54 -07:00
Momo
571a85f539
fix(slug): enforce length, pattern, and reserved-word rules on skill & soul slugs (#1879)
Merged via squash.

Prepared head SHA: d93026fd4b013876a4009974b4c7052b416f3bbf
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-05-07 14:37:58 +08:00
Patrick Erichsen
0f938fbabd
feat: add entity-scoped moderator commands (#2066)
Some checks are pending
CI / static (push) Waiting to run
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
CI / types-build (push) Waiting to run
CI / e2e-http (push) Waiting to run
CI / playwright-smoke (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-05-06 19:54:07 -07:00
Peter Steinberger
5e7797df72
fix: explain blocked and unauthorized API states 2026-05-07 03:47:08 +01:00
Patrick Erichsen
0b058d10bf
fix: repair clawhub mod installer 2026-05-06 19:09:33 -07:00
Patrick Erichsen
0749f16499
feat: split moderator commands into private cli 2026-05-06 19:05:07 -07:00
Patrick Erichsen
52c3b649e0
Merge pull request #2062 from openclaw/pe/maintenance-cleanup
chore: remove stale maintenance helpers
2026-05-06 17:34:07 -07:00
Patrick Erichsen
03328f7523 chore: remove stale maintenance helpers 2026-05-06 17:32:24 -07:00
Peter Steinberger
5019f2a78a
fix: surface blocked account auth state 2026-05-07 01:30:58 +01:00
Patrick Erichsen
e4a68d2a76
Merge pull request #2061 from openclaw/pe/security-review-grey-badge
Make security review badges neutral
2026-05-06 17:27:37 -07:00
Patrick Erichsen
bde371360d fix: make security review badges neutral 2026-05-06 17:21:18 -07:00
Patrick Erichsen
e62935762b
Merge pull request #2055 from openclaw/pe/artifact-moderation-cases
Add skill and package artifact moderation cases
2026-05-06 17:10:19 -07:00
Patrick Erichsen
ac08267403 test: update moderator summary expectation 2026-05-06 17:04:48 -07:00
Patrick Erichsen
9fe7532e27 feat: add report status backfill 2026-05-06 17:01:48 -07:00
Patrick Erichsen
3618d296af feat: summarize moderation CLI actions 2026-05-06 17:01:14 -07:00
Patrick Erichsen
cab18339e6 docs: align moderation wording with moderator role 2026-05-06 16:42:15 -07:00
Patrick Erichsen
5b1cfb4574 fix: rename report resolution status to confirmed 2026-05-06 16:39:55 -07:00
Patrick Erichsen
2bd6ed9198 test: add package artifact moderation lifecycle e2e 2026-05-06 16:26:53 -07:00
Peter Steinberger
4f4d7dd563
docs: document slug routing contract 2026-05-07 00:24:20 +01:00
Patrick Erichsen
d5d58a9dbc
Merge pull request #2058 from openclaw/pe/pr-template-screenshots
docs: add pull request template
2026-05-06 16:19:05 -07:00
Patrick Erichsen
7a2733947e
Merge pull request #2059 from openclaw/pe/friendly-404-page
Add a friendlier 404 page
2026-05-06 16:15:05 -07:00
Patrick Erichsen
4592e66879 feat: enforce artifact moderation state transitions 2026-05-06 16:05:23 -07:00
Patrick Erichsen
06500ea4ca feat: add friendly 404 page 2026-05-06 16:01:24 -07:00
Patrick Erichsen
d89e2ce1a1 docs: add pull request template 2026-05-06 15:35:56 -07:00
Patrick Erichsen
c4d1fcdbc6 feat: add skill artifact moderation cases 2026-05-06 15:22:54 -07:00
Peter Steinberger
e8deec13a2
fix: allow docs host auth callback form posts 2026-05-06 22:16:00 +01:00
Peter Steinberger
678935d014
fix: preserve docs auth return host 2026-05-06 22:10:47 +01:00
Patrick Erichsen
bb592363a7
Merge pull request #2054 from openclaw/pe/worktree-setup-bootstrap
fix: bootstrap ClawHub worktree setup
2026-05-06 13:48:25 -07:00
Patrick Erichsen
e3b59fce38 fix: align route tests with scoped plugin routes 2026-05-06 13:43:31 -07:00
Patrick Erichsen
1da5c53ce9 fix: bootstrap clawhub worktree setup 2026-05-06 13:37:09 -07:00
Patrick Erichsen
bfb16ceddf
Merge pull request #2051 from vyctorbrzezowski/contrib/dev-worktree-seed-readiness
fix: wait for Convex functions before seeding worktrees
2026-05-06 13:25:51 -07:00
Patrick Erichsen
f7fbd6bde4
Merge pull request #2052 from vyctorbrzezowski/contrib/browse-sidebar-sticky-offset
fix: keep browse sidebar below sticky header
2026-05-06 13:15:59 -07:00
Peter Steinberger
bec9362361
feat: add ClawHub docs auth broker 2026-05-06 21:08:39 +01:00
vyctorbrzezowski
c4688b3526 fix: keep browse sidebar below sticky header 2026-05-06 16:49:26 -03:00
Peter Steinberger
8d5eb14919
fix: keep scoped plugin URLs readable 2026-05-06 20:12:07 +01:00
Peter Steinberger
5aa4d13560
fix: route official extension slug aliases 2026-05-06 20:01:49 +01:00
vyctorbrzezowski
a046cff693 fix: wait for Convex functions before seeding worktrees 2026-05-06 15:23:03 -03:00
Peter Steinberger
57308e6059
docs(changelog): add 0.12 release notes 2026-05-06 07:17:31 +01:00
Peter Steinberger
32011a1f9a
test(e2e): open mobile nav in smoke 2026-05-06 07:09:11 +01:00
Patrick Erichsen
b735a529c2
Merge pull request #2041 from openclaw/pe/worktree-env-fallback
fix: discover shared env for worktree dev
2026-05-05 23:03:45 -07:00
Peter Steinberger
6925ec761c
fix(cli): support org-owned skill publishes 2026-05-06 07:00:40 +01:00
Patrick Erichsen
521fd2796a fix: discover shared env for worktree dev 2026-05-05 22:57:11 -07:00
Peter Steinberger
2b00f0b37e
Merge pull request #2040 from openclaw/pe/package-delete-cli-ui
feat: allow package owners to delete plugins
2026-05-06 06:42:28 +01:00
Peter Steinberger
88b6a941ec
fix(packages): finish delete flow cleanup 2026-05-06 06:38:36 +01:00
Patrick Erichsen
0c7607bd64 feat: allow package owners to delete plugins 2026-05-05 22:37:19 -07:00
Patrick Erichsen
bb6ef2ba44
Merge pull request #2037 from openclaw/pe/fix-package-moderation-queue
fix: normalize package moderation queue timestamps
2026-05-05 21:01:11 -07:00
Patrick Erichsen
df61771b7e fix: normalize package moderation queue timestamps 2026-05-05 20:50:08 -07:00
Patrick Erichsen
9605bb3d8e
Merge pull request #2036 from openclaw/pe/mobile-friendly-clawhub
Fix mobile layout responsiveness
2026-05-05 20:08:07 -07:00
Patrick Erichsen
33176522da fix: improve mobile layout responsiveness 2026-05-05 19:46:40 -07:00
Patrick Erichsen
102f47174d
Merge pull request #2035 from openclaw/pe/codex-worktree-setup
[codex] Add Codex worktree setup
2026-05-05 18:09:08 -07:00
Patrick Erichsen
cd9995c676 fix: leave contributing guide unchanged 2026-05-05 18:06:42 -07:00
Patrick Erichsen
6679f36a2f fix: remove hardcoded worktree env path 2026-05-05 18:04:15 -07:00
Patrick Erichsen
19993f93ed fix: start Convex from worktree helper 2026-05-05 17:52:26 -07:00
Patrick Erichsen
9028a7402a fix: avoid Bun ambient type in worktree helper 2026-05-05 17:42:26 -07:00
Patrick Erichsen
0a32b9857d feat: add Codex worktree setup 2026-05-05 17:33:49 -07:00
Patrick Erichsen
0a5b648f78
Merge pull request #2033 from openclaw/pe/cli-privileged-help-visibility
fix: gate privileged cli help by role
2026-05-05 15:25:05 -07:00
Patrick Erichsen
ebe77f7f63 fix: gate privileged cli help by role 2026-05-05 14:40:25 -07:00
Vincent Koc
caac39ce29
fix(convex): scope rate limit buckets by kind
Some checks failed
CI / static (push) Has been cancelled
CI / unit (push) Has been cancelled
CI / packages (push) Has been cancelled
CI / types-build (push) Has been cancelled
CI / e2e-http (push) Has been cancelled
CI / playwright-smoke (push) Has been cancelled
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Scope HTTP rate limit buckets by request kind and raise healthy production limits so plugin downloads are not throttled by unrelated API reads.
2026-05-04 20:25:15 -07:00
Patrick Erichsen
d24422a005
Merge pull request #2016 from openclaw/pe/package-lookup-not-found
fix(packages): return not found for invalid lookups
2026-05-04 20:23:00 -07:00
Patrick Erichsen
1c62c5fff0 fix(packages): return not found for invalid lookups 2026-05-04 20:18:46 -07:00
Peter Steinberger
395862fadf
ci: use app token for Convex AI update PRs
Some checks are pending
CI / static (push) Waiting to run
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
CI / types-build (push) Waiting to run
CI / e2e-http (push) Waiting to run
CI / playwright-smoke (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-05-04 09:52:34 +01:00
Peter Steinberger
ba7a108af1
fix: keep oxlint underscore rule disabled 2026-05-04 08:08:59 +01:00
Peter Steinberger
6c3f911e8e
test: fix http api rate limit mock 2026-05-04 08:06:04 +01:00
Peter Steinberger
facf20ceb6
fix: raise admin api rate limits 2026-05-04 07:56:36 +01:00
Peter Steinberger
0690891781
fix: raise trusted publish rate limit 2026-05-04 06:35:30 +01:00
Peter Steinberger
0df30649ca
fix: validate clawpack runtime entries against extracted files 2026-05-04 06:06:30 +01:00
Peter Steinberger
bbdde7fd53
fix: keep package dry-run metadata-only 2026-05-03 23:23:36 +01:00
Peter Steinberger
3d6f3b49a5
fix: reject code plugins without runtime output 2026-05-03 23:19:00 +01:00
Peter Steinberger
0b842636dc
fix: allow admin plugin release publishes 2026-05-03 22:59:58 +01:00
Peter Steinberger
2d2d791e9f
fix: infer package owner from scoped names 2026-05-03 22:53:01 +01:00
Peter Steinberger
96e3d7ebd4
fix: raise authenticated write rate limit 2026-05-03 22:44:21 +01:00
Peter Steinberger
768a50149e
fix: support monorepo package publishes 2026-05-03 21:04:34 +01:00
Vincent Koc
199e6a0cdf
fix(api): expose legacy zip artifact aliases
Some checks are pending
CI / types-build (push) Waiting to run
CI / static (push) Waiting to run
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
CI / e2e-http (push) Waiting to run
CI / playwright-smoke (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
Expose legacy ZIP resolver compatibility aliases without confusing publish-time content hashes for downloaded archive integrity.
2026-05-03 10:34:37 -07:00
Vincent Koc
59fc54ff64
fix(web): canonicalize scoped plugin paths 2026-05-03 09:35:02 -07:00
Vincent Koc
343781a668
fix(api): decode scoped package paths 2026-05-03 09:26:13 -07:00
Vincent Koc
62b10f829d
fix(packages): use single-window search fallback 2026-05-03 02:20:53 -07:00
Vincent Koc
eb3113c1f3
fix(packages): rebuild search queries per page 2026-05-03 02:09:54 -07:00
Vincent Koc
887e81eb85
fix(api): return lean skill list payloads 2026-05-03 02:04:16 -07:00
Vincent Koc
d6cfc891f0
fix(search): reduce lexical fallback scan budget 2026-05-03 02:02:19 -07:00
Vincent Koc
cf5778d7d5
fix(api): route package search through digest index 2026-05-03 02:01:20 -07:00
Vincent Koc
e76b72cdb1
fix(search): cap vector hydration window 2026-05-03 01:13:31 -07:00
Vincent Koc
f53b49041a
fix(api): avoid redundant latest tag version reads 2026-05-03 01:11:43 -07:00
Vincent Koc
21abd07672
fix(search): bound lexical fallback scans 2026-05-03 01:09:44 -07:00
Vincent Koc
6085ee4852
fix(convex): raise download rate limit 2026-05-03 00:24:42 -07:00
Vincent Koc
05653453ea
fix(convex): reduce download and token write contention 2026-05-03 00:06:59 -07:00
Vincent Koc
86f8aa88af
test(convex): update leaderboard page size expectation 2026-05-02 23:45:27 -07:00
Vincent Koc
2d42c3d57a
fix(convex): reduce hot rate-limit and catalog reads 2026-05-02 23:42:15 -07:00
Vincent Koc
cf5a6f6e8b
fix(scanner): avoid generic pay purchase tags 2026-05-02 23:13:59 -07:00
Vincent Koc
46354c9967
fix(security): flag python file upload exfiltration 2026-05-02 22:50:57 -07:00
Vincent Koc
063ee210a7
fix(github): keep scanner appeal issues open 2026-05-02 22:50:09 -07:00
Vincent Koc
f84c894e4e
fix(ui): restore skill downloads and search paging 2026-05-02 22:46:30 -07:00
Vincent Koc
f8141bc517
fix(convex): gate large index deletion 2026-05-02 19:07:44 -07:00
Vincent Koc
ca4899078d
fix(convex): retain built rate limit index 2026-05-02 18:37:10 -07:00
Vincent Koc
51d4633df0
fix(convex): avoid rate limit index backfill 2026-05-02 18:19:08 -07:00
Vincent Koc
6139dcd052
fix(convex): drop unused package stat index 2026-05-02 17:20:27 -07:00
Vincent Koc
ab48c07b98
test(rate-limits): expect consumed shard quota 2026-05-02 16:19:00 -07:00
Vincent Koc
0a49b75e2f
fix(convex): reduce hot stat write contention 2026-05-02 16:16:06 -07:00
Vincent Koc
5e9c61a185
fix(convex): bound skill health reads 2026-05-02 16:16:02 -07:00
Vincent Koc
8234c92dcf
fix(packages): keep mirror artifact URLs on public host 2026-05-02 15:54:57 -07:00
Vincent Koc
9edff6fd38
build(schema): update package response dist 2026-05-02 15:45:05 -07:00
Vincent Koc
2ebcdd4ed0
test(packages): include required plugin manifests 2026-05-02 15:38:08 -07:00
Vincent Koc
f5183cae9b
fix(security): flag confirmation bypasses 2026-05-02 15:33:49 -07:00
Vincent Koc
4196789c6d
chore(cli): bump to 0.12.2 2026-05-02 14:43:47 -07:00
Vincent Koc
4c69f2af2e
fix(schema): allow nullable package sha 2026-05-02 14:42:57 -07:00
Vincent Koc
4c52dc23c1
fix(cli): allow legacy package downloads 2026-05-02 14:39:04 -07:00
Vincent Koc
8916167505
style(api): format scoped route changes 2026-05-02 14:29:08 -07:00
Vincent Koc
f4f2da7fe7
fix(api): resolve scoped package routes 2026-05-02 14:28:27 -07:00
Vincent Koc
01529aaaf1
fix(cli): publish code plugins as clawpacks 2026-05-02 14:27:10 -07:00
Vincent Koc
05efb81669
chore(cli): bump to 0.12.1 2026-05-02 13:47:48 -07:00
Vincent Koc
ca0d0bd1bd
docs(security): clarify clawpack scan scope 2026-05-02 13:47:05 -07:00
Vincent Koc
f82e07fd3a
ci: add clean production deploy tags 2026-05-02 13:29:59 -07:00
Vincent Koc
f2a61c9d94
fix(packages): scan clawpack artifacts with virustotal 2026-05-02 13:09:03 -07:00
Vincent Koc
3c09df3b77
ci: tag production frontend deploys 2026-05-02 13:06:55 -07:00
Vincent Koc
6d4cf0cfe7
test(packages): avoid unsafe optional chaining
Some checks are pending
CI / e2e-http (push) Waiting to run
CI / playwright-smoke (push) Waiting to run
CI / types-build (push) Waiting to run
CI / static (push) Waiting to run
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-05-02 12:44:28 -07:00
Vincent Koc
e4aa4c7459
style: format clawpack rollout changes 2026-05-02 12:43:44 -07:00
Vincent Koc
3aff30b955
fix(plugins): hide staged bundle publish ux 2026-05-02 12:42:59 -07:00
Vincent Koc
c9a225aef7
fix(plugins): show clawpack artifact downloads 2026-05-02 12:41:53 -07:00
Vincent Koc
0fe234e68d
feat(cli): add clawpack pack command 2026-05-02 12:40:38 -07:00
Vincent Koc
56743ce3d8
fix(packages): store clawpack metadata only 2026-05-02 12:38:57 -07:00
Vincent Koc
1fdfbcd51f
fix(packages): cap clawpack tarballs at 120mb 2026-05-02 12:38:11 -07:00
Vincent Koc
bf1e112d5a
style(packages): format package updates 2026-05-02 11:41:12 -07:00
Vincent Koc
7266f4f927
docs(packages): clarify plugin package metadata 2026-05-02 11:39:43 -07:00
Vincent Koc
77927830f3
fix(api): accept scoped npm packuments 2026-05-02 11:38:39 -07:00
Vincent Koc
e599d23f69
fix(packages): use real bundle markers 2026-05-02 11:37:28 -07:00
Vincent Koc
4c8738f1ef
fix(clawpack): require plugin manifests 2026-05-02 11:34:51 -07:00
Vincent Koc
e01c7a9f31
fix(packages): make host metadata optional 2026-05-02 11:33:38 -07:00
Vincent Koc
c9a5b8508d
test(packages): satisfy migration lint 2026-05-02 11:03:41 -07:00
Vincent Koc
cb320fe2ab
docs(packages): document official migrations 2026-05-02 11:02:35 -07:00
Vincent Koc
773df44f17
feat(cli): manage official migrations 2026-05-02 11:02:00 -07:00
Vincent Koc
402ddddbd7
feat(api): manage official migrations 2026-05-02 10:59:56 -07:00
Vincent Koc
238f3f6b14
feat(packages): persist official migrations 2026-05-02 10:58:23 -07:00
Vincent Koc
539bf60e97
chore(schema): build official migration types 2026-05-02 10:57:35 -07:00
Vincent Koc
6527ab6a9f
feat(packages): add official migration schema 2026-05-02 10:55:03 -07:00
Vincent Koc
63164eb762
docs(cli): document package migration status 2026-05-02 10:53:54 -07:00
Vincent Koc
669e14b92c
feat(cli): show package migration status 2026-05-02 10:53:36 -07:00
Vincent Koc
28da510571
feat(packages): resolve package appeals 2026-05-02 10:50:57 -07:00
Vincent Koc
6e5578ee6d
feat(packages): submit package appeals 2026-05-02 10:47:26 -07:00
Vincent Koc
68017740e7
feat(packages): show moderation status 2026-05-02 10:44:37 -07:00
Vincent Koc
ff68eeb5d1
feat(packages): triage package reports 2026-05-02 10:40:13 -07:00
Vincent Koc
276760d703
feat(packages): report packages for review 2026-05-02 10:35:28 -07:00
Vincent Koc
1b33c949f1
feat(packages): filter by artifact availability 2026-05-02 10:25:33 -07:00
Vincent Koc
417537a13f
feat(packages): list moderation queue 2026-05-02 10:17:09 -07:00
Vincent Koc
6e15ed65e0
feat(cli): filter packages by environment 2026-05-02 10:07:55 -07:00
Vincent Koc
58dcd55076
style(dashboard): format pagination changes 2026-05-02 10:06:37 -07:00
Vincent Koc
c9ad1305ff
feat(packages): require environment metadata 2026-05-02 10:05:49 -07:00
Vlad Ursul
964fc0fa87
feat(dashboard): add skill pagination
Adds indexed, paginated dashboard skill loading and Load More UI.\n\nMaintainer validation after rebasing onto current main:\n- bun run test -- convex/skills.dashboard.test.ts src/routes/-dashboard.test.tsx\n- bun run test -- convex/skills.dashboard.test.ts convex/skills.list.test.ts\n- bunx tsc -p tsconfig.json --noEmit\n- bunx tsc -p packages/schema/tsconfig.json --noEmit\n- bunx tsc -p packages/clawhub/tsconfig.json --noEmit\n- bun run lint\n- bun run build\n\nNote: full bun run test currently has unrelated package publish route failures on current main; PR-focused tests and build are clean. Vercel PR preview remains blocked by fork deployment authorization.
2026-05-02 12:01:07 -05:00
Vincent Koc
bc234c7d89
feat(packages): report openclaw readiness 2026-05-02 09:57:12 -07:00
Vincent Koc
87a286fe1f
feat(packages): backfill package artifact kinds 2026-05-02 09:52:37 -07:00
Vincent Koc
00970bbee9
feat(packages): require code plugin host targets 2026-05-02 09:40:00 -07:00
Val Alexander
7979ff4249
chore: update ClawHub UI code owner
Update frontend/UI CODEOWNERS entries to use @BunsDev while preserving secops review ownership.
2026-05-02 11:38:28 -05:00
Val Alexander
f3c4cbb99a
feat: clarify about page policy patterns
Summary:
- Refresh the About page Recent Patterns section to explicitly allow specific maintainer-approved patterns.
- Replace the top-nav git icon with the GitHub mark for GitHub sign-in.
- Clean up ClawPack internal type exports and make Convex integrity hashing compatible with CI WebCrypto.

Validation:
- bun run format:check
- bun run lint
- bun run ci:static
- bun run ci:unit
- bunx tsc --noEmit
- bunx tsc -p packages/schema/tsconfig.json --noEmit && bunx tsc -p packages/clawhub/tsconfig.json --noEmit
- bun run test -- convex/lib/clawpack.test.ts
- VITE_CONVEX_URL=https://example.invalid bun run build
- GitHub checks for PR #1980 all passed
2026-05-02 11:33:49 -05:00
Vincent Koc
bed2d4b1b0
feat(packages): moderate package releases 2026-05-02 09:28:40 -07:00
Vincent Koc
81759fd857
chore(convex): refresh generated api 2026-05-02 09:28:25 -07:00
Vincent Koc
f3cf886ce5
feat(cli): download and verify package artifacts 2026-05-02 09:10:17 -07:00
Vincent Koc
2176dbf4c2
fix(packages): satisfy clawpack lint gates 2026-05-02 08:52:06 -07:00
Vincent Koc
80e8b599f9
test(cli): cover clawpack publish upload 2026-05-02 08:49:56 -07:00
Vincent Koc
7d636b771b
test(api): cover clawpack package routes 2026-05-02 08:47:32 -07:00
Vincent Koc
e94cc91b8e
docs(packages): document clawpack artifact paths 2026-05-02 08:46:17 -07:00
Vincent Koc
0774d0fe92
feat(cli): publish uploaded clawpacks 2026-05-02 08:44:41 -07:00
Vincent Koc
1261062585
feat(api): serve clawpack mirror artifacts 2026-05-02 08:42:38 -07:00
Vincent Koc
88d0cc7888
feat(packages): accept clawpack uploads 2026-05-02 08:37:45 -07:00
Vincent Koc
87848016ff
feat(packages): widen artifact schema 2026-05-02 08:34:47 -07:00
Vincent Koc
86e58d6031
feat(packages): add clawpack parser 2026-05-02 08:33:41 -07:00
Peter Steinberger
48e66714ac
fix: add package identity repair admin
Some checks are pending
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
CI / types-build (push) Waiting to run
CI / e2e-http (push) Waiting to run
CI / playwright-smoke (push) Waiting to run
CI / static (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-05-02 06:47:24 +01:00
Peter Steinberger
0c705e159f
fix: allow JSON Schema manifests in package publish 2026-05-02 05:41:51 +01:00
Peter Steinberger
5409df4123
fix: keep beta plugin packages off latest 2026-05-02 05:15:50 +01:00
Peter Steinberger
ac15e5adea
fix: add package owner transfer repair 2026-05-02 04:56:46 +01:00
Peter Steinberger
880d9e0572
feat: reserve OpenClaw plugin package names 2026-05-01 22:51:03 +01:00
Patrick Erichsen
63dfbd8876
Merge pull request #1967 from openclaw/pe/clawscan
Some checks are pending
CI / static (push) Waiting to run
CI / unit (push) Waiting to run
CI / packages (push) Waiting to run
CI / types-build (push) Waiting to run
CI / e2e-http (push) Waiting to run
CI / playwright-smoke (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
Clarify ClawScan artifact prompt boundaries
2026-05-01 06:32:59 -07:00
Patrick Erichsen
4a7b7b7024 Update securityPrompt.ts 2026-05-01 06:32:07 -07:00
Patrick Erichsen
34e26093ab Update securityPrompt.ts 2026-05-01 06:31:25 -07:00
Patrick Erichsen
601d29b0e9 Update securityPrompt.ts 2026-05-01 06:30:53 -07:00
Patrick Erichsen
bff959c8f0 fix: rely on JSON artifact neutralization 2026-05-01 06:28:29 -07:00
Patrick Erichsen
34a2c657b6 Merge remote-tracking branch 'origin/main' into pe/clawscan
# Conflicts:
#	convex/lib/securityPrompt.ts
2026-05-01 06:20:57 -07:00
Patrick Erichsen
fc6555fa1c Update securityPrompt.ts 2026-05-01 06:11:53 -07:00
Patrick Erichsen
e7ad7c628d fix: wrap ClawScan skill artifacts in prompt boundary 2026-05-01 06:07:39 -07:00
Vincent Koc
7c61d55833
ci: expand pr validation coverage
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Split PR validation into explicit static, unit, package, type/build, HTTP e2e, and browser-smoke gates. Add local ci:* scripts and document the required status checks.
2026-05-01 02:20:40 -07:00
Vincent Koc
89becd866a
Revert "feat: add health probes"
This reverts commit bb945c740e.
2026-04-30 23:56:11 -07:00
Vincent Koc
eada4d5dcb
Revert "fix: keep probe helper types private"
This reverts commit 7f15dcc225.
2026-04-30 23:56:11 -07:00
Vincent Koc
7f15dcc225
fix: keep probe helper types private 2026-04-30 23:46:39 -07:00
Vincent Koc
bb945c740e
feat: add health probes 2026-04-30 23:44:31 -07:00
Vincent Koc
dfc0d540d8
chore(ci): enforce formatting 2026-04-30 23:39:28 -07:00
Vincent Koc
cd37acadbb
fix(security): add skill redaction hide mutation 2026-04-30 23:33:50 -07:00
Vincent Koc
c9fe6db34d
fix(search): index skill first-token recall 2026-04-30 23:27:37 -07:00
Vincent Koc
5fe321a43f
fix(ci): treat cli schema as deadcode entry 2026-04-30 23:22:17 -07:00
Vincent Koc
9e15c5a6fa
chore(ci): add deadcode gate 2026-04-30 23:17:07 -07:00
Vincent Koc
026b911d58
chore(search): allow manual digest backfill 2026-04-30 23:13:56 -07:00
Vincent Koc
08326f7718
chore(search): expose digest backfill cursor 2026-04-30 23:08:22 -07:00
Vincent Koc
881514f444
fix(search): add normalized skill prefix recall 2026-04-30 23:01:28 -07:00
Vincent Koc
3f17fd55e5
fix(security): fully strip hidden html comments 2026-04-30 22:39:35 -07:00
Vincent Koc
3f2153e678
fix(security): neutralize llm eval prompt injection 2026-04-30 22:00:05 -07:00
Vincent Koc
9ea3ed896f
fix(security): fail closed when vt is unavailable 2026-04-30 18:34:38 -07:00
Vincent Koc
7ea5fc085c
fix(ci): skip frontend smoke on backend deploys 2026-04-30 18:32:55 -07:00
Patrick Erichsen
1306ab6640
Merge pull request #1961 from openclaw/pe/clawscan
feat: label "suspicious" as "review" for scans
2026-04-30 16:23:35 -07:00
Patrick Erichsen
f7c5ae5a16 feat: label "suspicious" as "review" for scans 2026-04-30 15:54:45 -07:00
Patrick Erichsen
631b357a10
Merge pull request #1948 from openclaw/pe/clawscan
feat: move ClawScan eval runner into ClawHub
2026-04-30 15:01:06 -07:00
Patrick Erichsen
42bc312151 feat: export redacted skill content for security dataset 2026-04-30 14:51:26 -07:00
Peter Steinberger
12c72366f6
fix: raise public read rate limits
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-04-30 19:53:23 +01:00
Peter Steinberger
27d7d4afa4
ci: stabilize production deploy smoke 2026-04-30 19:50:24 +01:00
Peter Steinberger
cb3852ef16
fix: sync schema dist for cli delete reason 2026-04-30 19:39:49 +01:00
Peter Steinberger
50768641f9
fix: satisfy lint on latest main 2026-04-30 19:34:10 +01:00
Peter Steinberger
651e54ed7c
fix: record skill moderation reasons from CLI 2026-04-30 19:30:40 +01:00
Patrick Erichsen
3bbbd858d4 chore: rename ClawScan security signals eval
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-30 09:24:29 -07:00
Patrick Erichsen
9a8607038e fix: satisfy ClawScan eval lint 2026-04-30 08:59:35 -07:00
Patrick Erichsen
2bac472615 feat: parameterize ClawScan eval HF split 2026-04-30 08:58:07 -07:00
Patrick Erichsen
b27072312b chore: simplify ClawScan eval defaults 2026-04-30 08:57:01 -07:00
Patrick Erichsen
6bebc0f572 fix: satisfy maintenance lint rule 2026-04-30 08:39:46 -07:00
Patrick Erichsen
efa349c856 Merge remote-tracking branch 'origin/main' into pe/clawscan 2026-04-30 08:39:07 -07:00
Patrick Erichsen
21f2cfbd9c ci: remove format check from build job 2026-04-30 08:36:42 -07:00
Patrick Erichsen
b96af7391c feat: move ClawScan eval runner into ClawHub 2026-04-30 08:21:06 -07:00
Vincent Koc
cfc4ba9b6a
fix(maintenance): add skill version privacy removal
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-30 03:52:03 -07:00
Vincent Koc
9b27c1a1d3
fix(convex): page owner-publisher digest syncs
Fixes #1195.

Fixes #1182.
2026-04-30 03:32:43 -07:00
Vincent Koc
9e09581c05
fix(rate-limit): scope anonymous download fallback buckets 2026-04-30 03:20:29 -07:00
Vincent Koc
97c409d56b
fix(github): catch suspicious skill rescan requests 2026-04-30 03:05:01 -07:00
Vincent Koc
6c93d2096e
fix(security): flag disabled tls verification 2026-04-30 03:04:21 -07:00
Vincent Koc
65d02e57b0
fix(web): add frontend security headers 2026-04-30 03:04:20 -07:00
Vincent Koc
ae83b2188c
fix(api): require explicit license acceptance 2026-04-30 03:04:19 -07:00
Vincent Koc
d97942b996
fix(security): lock down virustotal result lookup 2026-04-30 03:04:18 -07:00
Vincent Koc
a3125daf78
fix(github): add third-party skill closeout label 2026-04-30 02:54:10 -07:00
Val Alexander
23eec67163
fix: make detail install panels full width
Make skill and plugin detail hero action panels span the full content width, moving scans/install above long-form detail content.\n\nVerified with local focused tests, lint, targeted formatting, diff check, build, and green PR CI build.
2026-04-30 04:06:52 -05:00
Patrick Erichsen
c3c885ec10
Merge pull request #1940 from openclaw/pe/clawscan
chore: remove clawscan eval corpora
2026-04-30 01:42:24 -07:00
Patrick Erichsen
04492fe196 feat: add prompt evals against 2026-04-30 01:12:04 -07:00
Vincent Koc
292f15dbae
fix(convex): preserve public skill type narrowing 2026-04-30 01:05:08 -07:00
Vincent Koc
6bf8d4b7b7
fix(packages): count package archive downloads 2026-04-30 01:00:32 -07:00
Vincent Koc
b60514b3fe
fix(github): run rescan guidance as app 2026-04-30 00:56:10 -07:00
Vincent Koc
45b9c0e51d
fix(cli): apply source path before GitHub package fetch 2026-04-30 00:55:26 -07:00
Vincent Koc
e3cf29a2bc
fix(security): flag remote recipe execution 2026-04-30 00:52:53 -07:00
Vincent Koc
3deff6efd1
chore(github): soften rescan guidance label 2026-04-30 00:51:06 -07:00
Vincent Koc
c4950b8034
fix(security): flag provider secrets and rclone paths 2026-04-30 00:50:15 -07:00
Vincent Koc
b043065ee5
fix(skills): hide nonpublic duplicate references 2026-04-30 00:49:01 -07:00
Patrick Erichsen
94d358e25b chore: move security eval pipelines out of clawhub 2026-04-30 00:48:14 -07:00
Vincent Koc
b8f04b5bc4
fix(security): flag env cgnat credentials 2026-04-30 00:44:55 -07:00
Vincent Koc
248a3f25e3
fix(security): flag hardcoded operator billing 2026-04-30 00:42:19 -07:00
Vincent Koc
8fb4d01e65
fix(github): classify issue auto-responses 2026-04-30 00:41:57 -07:00
Vincent Koc
26744ba4ef
fix(security): flag autonomous credential egress 2026-04-30 00:39:42 -07:00
Vincent Koc
f1481c4d4e
Reapply "feat(security): merge clawscan ASI analysis"
This reverts commit fa9ab8d620.
2026-04-30 00:35:47 -07:00
Vincent Koc
ec2308c96d
fix(security): flag Python credential posts 2026-04-30 00:34:48 -07:00
Vincent Koc
beb5c27d9e
fix(security): flag plaintext cgnat endpoints 2026-04-30 00:32:12 -07:00
Vincent Koc
18ae25b4c2
fix(security): flag unsafe subprocess file writes 2026-04-30 00:30:50 -07:00
Vincent Koc
1208e86b5f
fix(security): flag unsafe browser file renders 2026-04-30 00:29:27 -07:00
Vincent Koc
cd34538f16
fix(readme): scope relative skill links 2026-04-30 00:29:11 -07:00
Vincent Koc
85db1c60ad
fix(security): flag shell file upload exfiltration 2026-04-30 00:27:43 -07:00
Vincent Koc
8aa7a58a40
fix(security): detect dynamic module execution 2026-04-30 00:25:38 -07:00
Vincent Koc
67739a4a9f
fix(ui): improve runtime requirement contrast 2026-04-30 00:24:37 -07:00
Vincent Koc
fc74a2f6cd
fix(security): flag secret argv exposure 2026-04-30 00:23:53 -07:00
Deepak Jain
52078abd85
docs: clarify optional skill environment variables (#1859)
* Document optional skill env vars

Fixes #1617

* fix: honor nested optional env declarations

* chore: format skill env docs

---------

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
2026-04-30 00:22:17 -07:00
Vincent Koc
933fb94bcf
fix(security): flag browser credential automation 2026-04-30 00:20:10 -07:00
Vincent Koc
0db5ef6224
fix(search): stabilize relevance recall window 2026-04-30 00:19:06 -07:00
Vincent Koc
43d50b8947
fix(security): delete GitHub mirror on skill hide 2026-04-30 00:11:14 -07:00
Vincent Koc
3fea99b8a6
docs(search): explain discoverability ranking 2026-04-30 00:08:06 -07:00
Vincent Koc
b4cfe33659
fix(security): flag platform source patch installs 2026-04-30 00:04:46 -07:00
Vincent Koc
e60bff87e8
fix(cli): surface inspect moderation diagnostics 2026-04-30 00:03:51 -07:00
Vincent Koc
b3c42ddba2
fix(security): detect credential exposure docs 2026-04-30 00:01:41 -07:00
Vincent Koc
adbf4347e7
fix(security): scan code files for hardcoded secrets 2026-04-29 23:59:32 -07:00
Vincent Koc
6595e13a10
fix(search): use nonsuspicious digest indexes 2026-04-29 23:54:11 -07:00
Vincent Koc
2a7b0f0a6f
fix(deploy): harden production smoke checks 2026-04-29 23:53:24 -07:00
Vincent Koc
ed596ba24d
fix(github): run Barnacle with app token 2026-04-29 23:53:00 -07:00
Vincent Koc
2d054fe9ed
Merge branch 'main' of https://github.com/openclaw/clawhub
* 'main' of https://github.com/openclaw/clawhub:
  chore(security-dataset): remove eval runner
2026-04-29 23:50:15 -07:00
Vincent Koc
fa9ab8d620
Revert "feat(security): merge clawscan ASI analysis"
This reverts commit 79eddc0223, reversing
changes made to 33334c5afa.
2026-04-29 23:49:30 -07:00
Vincent Koc
7f220c2108
chore(security-dataset): remove eval runner 2026-04-29 23:48:59 -07:00
Vincent Koc
9b5c9541f8
fix(search): add soul lexical fallback 2026-04-29 23:48:13 -07:00
Vincent Koc
e324fcaae2
feat(api): support created-time skill listing 2026-04-29 23:46:57 -07:00
Patrick Erichsen
57be656406 Merge remote-tracking branch 'origin/main' into pe/clawscan 2026-04-29 23:45:10 -07:00
Vincent Koc
8cab60d64a
feat(github): add barnacle auto-response workflows 2026-04-29 23:44:46 -07:00
Vincent Koc
d4d69d42be
feat(cli): list manual skill directories 2026-04-29 23:43:35 -07:00
Vincent Koc
1461d0f175
feat(cli): show moderation in inspect 2026-04-29 23:41:22 -07:00
Vincent Koc
827fd92c7d
fix(ui): use download icon for public stats 2026-04-29 23:40:27 -07:00
Vincent Koc
cf20e10338
fix(github-backups): scan digest rows for sync 2026-04-29 23:33:25 -07:00
Vincent Koc
79eddc0223
feat(security): merge clawscan ASI analysis 2026-04-29 23:32:32 -07:00
Vincent Koc
33334c5afa
fix(stats): avoid scan fallback for public skill count 2026-04-29 23:29:17 -07:00
Vincent Koc
477aae7c95
fix(schema): allow R source files 2026-04-29 23:25:47 -07:00
Vincent Koc
a535da6dfb
feat(security): verify dependency registries 2026-04-29 23:15:35 -07:00
Vincent Koc
50ee17ce7d
style(security-dataset): format eval CLI test 2026-04-29 23:11:04 -07:00
Vincent Koc
0079d3f09a
test(security-dataset): cover eval CLI outputs 2026-04-29 23:10:41 -07:00
Deepak Jain
bcfe66d7d5
fix(security): narrow crypto swap detection
Fixes #1524

Taken from #1857.
2026-04-29 23:04:49 -07:00
Vincent Koc
f3a1d7fc32
feat(security-dataset): expand eval scanner metrics 2026-04-29 23:02:54 -07:00
Vincent Koc
201713c9ed
chore(deps): hold undici on node20-compatible line 2026-04-29 22:54:52 -07:00
Vincent Koc
2a5638e05b
Revert "chore(deps): update undici to v8"
This reverts commit 8d5e7b2d4d.
2026-04-29 22:54:47 -07:00
Patrick Erichsen
82a85ad21e
Merge pull request #1935 from openclaw/pe/clawhub-unban-moderation-skill
feat: add clawhub unban command
2026-04-29 22:50:28 -07:00
Vincent Koc
f3c060c360
fix(security-dataset): adapt oversized export batches 2026-04-29 22:49:17 -07:00
Vincent Koc
8d5e7b2d4d
chore(deps): update undici to v8 2026-04-29 22:48:47 -07:00
Vincent Koc
2325c21108
fix(security-dataset): include export entry names in parser errors 2026-04-29 22:43:51 -07:00
Vincent Koc
ad53229985
chore(repo): normalize workflow hygiene 2026-04-29 22:41:29 -07:00
Patrick Erichsen
886a38cb8b feat: add clawhub unban command 2026-04-29 22:40:46 -07:00
Vincent Koc
1a9d80a43d
merge: sync testbox setup with latest main
* origin/main:
  fix(security-dataset): compress batched export output
2026-04-29 22:33:39 -07:00
Vincent Koc
0a9f969775
merge: sync testbox setup with main
* origin/main:
  fix(security-dataset): align export batch types
  fix(security-dataset): keep batch output smaller
  test(security-dataset): use node environment for export parser
  fix(security-dataset): batch export pages server-side

# Conflicts:
#	scripts/security-dataset/convexExport.test.ts
2026-04-29 22:33:24 -07:00
Vincent Koc
9917881331
fix(security-dataset): compress batched export output 2026-04-29 22:33:09 -07:00
Vincent Koc
eec9702fa3
fix(security-dataset): align export batch types 2026-04-29 22:30:14 -07:00
Vincent Koc
4a09eafe42
fix(security-dataset): keep batch output smaller 2026-04-29 22:27:58 -07:00
Vincent Koc
076b938724
fix(test): isolate Convex export zip fixture 2026-04-29 22:27:02 -07:00
Vincent Koc
33b921af29
test(security-dataset): use node environment for export parser 2026-04-29 22:24:29 -07:00
Vincent Koc
8230c1e365
fix(security-dataset): batch export pages server-side 2026-04-29 22:22:17 -07:00
Vincent Koc
3bfdbfc004
chore(testbox): add Blacksmith runner setup 2026-04-29 22:21:18 -07:00
Patrick Erichsen
b48b95b0c1 Merge remote-tracking branch 'origin/main' into pe/clawscan 2026-04-29 22:16:24 -07:00
Vincent Koc
9ebf7d7bde
feat(security-dataset): ingest Convex exports locally 2026-04-29 22:14:21 -07:00
Vincent Koc
55dc372ecf
fix(security-dataset): type Convex output parser fallback 2026-04-29 22:08:38 -07:00
Vincent Koc
667c69a28b
feat(security-dataset): add snapshot time windows 2026-04-29 22:08:38 -07:00
Vincent Koc
e8b2aa558c
fix(ui): honor cleared security overrides 2026-04-29 22:06:31 -07:00
Vincent Koc
415c8e182e
fix(security-dataset): parse matching Convex output 2026-04-29 22:03:04 -07:00
Patrick Erichsen
52831cbc2b fix: restore legacy clawscan details 2026-04-29 21:58:10 -07:00
Vincent Koc
05c6409c96
feat(security-dataset): expose dataset lineage query 2026-04-29 21:55:58 -07:00
Vincent Koc
c8585875bd
fix(security-dataset): add snapshot manifest lineage 2026-04-29 21:50:30 -07:00
Vincent Koc
4a9ae92d54
fix(security): tighten destructive delete gate 2026-04-29 21:47:04 -07:00
Vincent Koc
8ac5881b4f
fix(security-dataset): retry invalid export pages 2026-04-29 21:45:16 -07:00
Vincent Koc
ef340c047b
fix(security): reduce execfile scanner noise 2026-04-29 21:44:14 -07:00
Vincent Koc
1e6f9bd44c
fix(security-dataset): enforce sharded export limits 2026-04-29 21:43:00 -07:00
Vincent Koc
0150b384a7
fix(security-dataset): stream sharded exports 2026-04-29 21:37:34 -07:00
Vincent Koc
3989cd8126
docs(security): pin publish workflow examples 2026-04-29 21:35:32 -07:00
Vincent Koc
b90a43adcb
fix(security): flag unsafe moderation patterns 2026-04-29 21:33:32 -07:00
Vincent Koc
94102e28f5
fix(security-dataset): parse large Convex pages 2026-04-29 21:28:21 -07:00
Vincent Koc
463e9b3fa7
fix(security-dataset): handle large export pages 2026-04-29 21:26:16 -07:00
Vincent Koc
09820d0d1c
fix(security-dataset): keep exports internal 2026-04-29 21:19:23 -07:00
Patrick Erichsen
faead0e25c fix: remove unreachable scanner report branches 2026-04-29 21:09:37 -07:00
Patrick Erichsen
0a64b977cb test: update clawscan report expectations 2026-04-29 21:02:50 -07:00
Patrick Erichsen
1e81388560 style: format clawscan UI files 2026-04-29 21:02:50 -07:00
Patrick Erichsen
3ab5762dca feat: ui updates 2026-04-29 21:02:50 -07:00
Patrick Erichsen
e2d187b3d5 feat: clawscan seed + frontend 2026-04-29 21:02:50 -07:00
Patrick Erichsen
afdac4a6a3 feat: preserve SkillTester raw corpus snapshot 2026-04-29 21:02:50 -07:00
Patrick Erichsen
5dfcd896e9 feat: add SkillTester ClawHub corpus 2026-04-29 21:02:50 -07:00
Vincent Koc
d91c4804ce
fix(ci): harden CodeQL light coverage 2026-04-29 21:00:02 -07:00
Vincent Koc
59e28c7831
feat: add security dataset eval runner 2026-04-29 20:59:32 -07:00
Vincent Koc
a0713e1833
feat: add security dataset snapshots 2026-04-29 20:57:06 -07:00
Vincent Koc
ca19f31816
chore(deps): ignore incompatible auth core bumps 2026-04-29 20:56:05 -07:00
Vincent Koc
87da4ec65a
chore(deps): update GitHub Actions pins 2026-04-29 20:50:40 -07:00
Vincent Koc
d9b419b21b
fix(deps): pin undici to ci-compatible line 2026-04-29 20:44:04 -07:00
Vincent Koc
bb94325679
chore(deps): complete major dependency updates 2026-04-29 20:38:54 -07:00
Vincent Koc
88f8ca2d29
chore(deps): update dependency drift 2026-04-29 20:34:20 -07:00
Vincent Koc
54ed3c58a1
ci: add lightweight CodeQL scans 2026-04-29 20:23:59 -07:00
Vincent Koc
ea35420eed
chore(deps): enable dependency update automation 2026-04-29 20:23:44 -07:00
Vincent Koc
2520da134c
fix(deps): remediate vulnerable packages 2026-04-29 20:22:20 -07:00
Vincent Koc
0b2de12e04
chore: add Patrick to secure code ownership 2026-04-29 20:14:44 -07:00
Vincent Koc
a8326517ad
chore: add secops code ownership 2026-04-29 20:11:07 -07:00
Val Alexander
7bef2a0b65
fix: remove card link hover underlines
Some checks failed
CI / build (push) Has been cancelled
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Remove the inherited global hover underline from full-card link surfaces while preserving normal inline link behavior.

Validated with targeted formatter/lint checks and a local browser hover pass across home category, carousel, trending, skills, plugins, and users card/list surfaces.
2026-04-29 03:40:13 -05:00
Val Alexander
22bb94cee2
fix: restore ClawHub public UI
Restore the public header, hero, featured carousel, Trending Now, category grid, footer, and UI design-contract guardrails. Remove tweakcn/custom visual overlay settings and stale density preference plumbing, while preserving reviewed search/typeahead behavior and latest review fixes.
2026-04-29 02:52:34 -05:00
Patrick Erichsen
b3c42b661b
Merge pull request #1882 from openclaw/pe/plugin-management-tools
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Add plugin curation to management
2026-04-28 22:31:44 -07:00
Vincent Koc
f72179f37d
chore(ci): update package publish artifact action
Update package publish artifact upload to the Node 24-ready artifact action and align the stale skills default-sort test with current filter behavior.
2026-04-28 22:31:33 -07:00
Patrick Erichsen
1d79f78426 feat: add plugin curation to management
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 22:30:43 -07:00
Patrick Erichsen
6209fe3fff
Merge pull request #1880 from openclaw/pe/featured-plugin-curation
feat: add featured plugin curation
2026-04-28 22:19:21 -07:00
Patrick Erichsen
a7d1701f5a feat: add featured plugin curation
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 22:18:35 -07:00
Patrick Erichsen
52da4954f6
Merge pull request #1871 from openclaw/pe/skills-plugins-typeahead
[codex] Add skills/plugins search typeahead
2026-04-28 21:24:26 -07:00
Patrick Erichsen
0ee5958f7a merge: sync with origin main
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 21:18:34 -07:00
Patrick Erichsen
5d01b99adb
Merge pull request #1878 from openclaw/pe/clawhub-rescan-guidance
feat: add ClawHub rescan guidance workflow
2026-04-28 20:09:30 -07:00
Patrick Erichsen
5dc834c27e feat: add ClawHub rescan guidance workflow
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 20:07:52 -07:00
Patrick Erichsen
82b9a69dad
Merge pull request #1875 from openclaw/pe/settings-stars
fix: move stars link into settings
2026-04-28 19:50:45 -07:00
Vincent Koc
064804e2d3
fix: make package publish retries idempotent 2026-04-28 19:29:39 -07:00
Patrick Erichsen
6c0163f9f2 feat: add skills plugins search typeahead 2026-04-28 18:33:20 -07:00
Patrick Erichsen
04a862d2b2 fix: move stars link into settings
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 18:32:09 -07:00
Patrick Erichsen
a7fc4bbae2
Merge pull request #1874 from openclaw/pe/oxfmt-pr-check
ci: check oxfmt on pull requests
2026-04-28 18:22:56 -07:00
Patrick Erichsen
4701c555f3 ci: check oxfmt on pull requests
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 18:16:32 -07:00
Patrick Erichsen
c1f167721b
Merge pull request #1873 from openclaw/pe/fix-skill-upload
fix: add skill upload button to header
2026-04-28 17:37:14 -07:00
Patrick Erichsen
1a94744484 Update $name.tsx
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 17:37:02 -07:00
Patrick Erichsen
9a5cfeee85 Update SkillHeader.tsx 2026-04-28 17:29:14 -07:00
Patrick Erichsen
e69b7d4501 fix: add skill upload button to header 2026-04-28 17:25:30 -07:00
Patrick Erichsen
ecf09b868a
Merge pull request #1872 from openclaw/pe/clawhub-cli-0.12.1
chore(release): prepare clawhub cli 0.12.1
2026-04-28 16:53:37 -07:00
Patrick Erichsen
4d16472f5b chore(release): prepare clawhub cli 0.12.0
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 16:52:53 -07:00
Patrick Erichsen
2e5ffdc565
Merge pull request #1861 from openclaw/pe/rescan
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
feat: add owner rescan security surfaces
2026-04-28 16:32:26 -07:00
Peter Steinberger
4e13e729fb ci: narrow ClawSweeper dispatch cancellation
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-04-28 11:53:07 +01:00
Peter Steinberger
d17e100cca
ci: harden clawsweeper dispatch workflow 2026-04-28 11:35:24 +01:00
Peter Steinberger
c732b38569
ci: debounce clawsweeper dispatch metadata 2026-04-28 11:31:51 +01:00
Patrick Erichsen
81ca04662c fix: prevent mobile install copy overlap
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-28 01:20:09 -07:00
Patrick Erichsen
ce69ab6a38 fix: polish mobile rescan security surfaces 2026-04-28 01:17:13 -07:00
Patrick Erichsen
a28d94c345 feat: show in progress scans 2026-04-28 00:52:51 -07:00
Peter Steinberger
3701733797
fix: normalize vt engine stats before caching 2026-04-28 08:47:50 +01:00
Peter Steinberger
b8ba595d06
docs: note vt code insight calibration 2026-04-28 08:40:09 +01:00
Patrick Erichsen
ef2846b2e4 feat: add owner rescan security surfaces 2026-04-28 00:39:44 -07:00
Peter Steinberger
232e429dee
fix: clear uncorroborated vt suspicious state 2026-04-28 08:39:39 +01:00
Deepak Jain
932155cb8f
docs: clarify static scan suppression gate 2026-04-28 08:39:17 +01:00
Deepak Jain
d855d09ab0
fix: calibrate vt code insight moderation
Refs #1830
2026-04-28 08:39:17 +01:00
Peter Steinberger
16e87c147d ci: harden ClawSweeper dispatcher credentials 2026-04-28 06:48:40 +01:00
Peter Steinberger
cbe22e70b9 ci: fix ClawSweeper dispatcher payload 2026-04-28 06:44:28 +01:00
Peter Steinberger
837331c967 ci: add ClawSweeper event dispatcher 2026-04-28 06:43:39 +01:00
Peter Steinberger
6ce443496d
fix: keep package list queries single-page 2026-04-28 06:28:27 +01:00
Peter Steinberger
8fd4f3b051
ci: fix production smoke coverage 2026-04-28 06:18:40 +01:00
Peter Steinberger
a2153909da
fix: avoid plugin catalog query limits 2026-04-28 06:13:57 +01:00
Deepak Jain
75e1b4633e
fix: constrain plugin catalog queries (#1842)
Refs #1699.

Use family-indexed plugin catalog paths instead of broad package scans:
- `/api/v1/plugins` merges separate `code-plugin` and `bundle-plugin` list streams with an endpoint-specific cursor.
- `/api/v1/plugins/search` searches both plugin families directly, dedupes, sorts, and limits results.
- Keeps generic `/api/v1/packages` behavior unchanged.

Validation:
- bunx vitest run convex/httpApiV1.handlers.test.ts convex/packages.public.test.ts
- bunx tsc --noEmit
- bunx tsc -p packages/schema/tsconfig.json --noEmit
- bunx tsc -p packages/clawhub/tsconfig.json --noEmit
- bun run lint
- git diff --check
2026-04-28 06:03:15 +01:00
Patrick Erichsen
4cda4a1fa4 Merge branch 'main' of https://github.com/openclaw/clawhub into pe/rescan
# Conflicts:
#	convex/skills.ts
2026-04-27 21:15:31 -07:00
Patrick Erichsen
5fce3ca2f4
Merge pull request #1850 from openclaw/pe/shadcn-ui-primitives
feat: adopt shadcn-managed ui primitives
2026-04-27 20:52:41 -07:00
Patrick Erichsen
87bca06939 feat: adopt shadcn-managed ui primitives 2026-04-27 20:48:15 -07:00
Deepak Jain
bf25b38c39
fix: tolerate stale auth in star status (#1843)
Refs #1819.

Read-only star status queries now treat stale, missing, deleted, or deactivated auth users as not starred instead of throwing. Star and unstar mutations still require an active authenticated user.

Validated locally:
- bunx vitest run convex/stars.test.ts convex/lib/access.test.ts
- bunx tsc -p packages/schema/tsconfig.json --noEmit
- bunx tsc -p packages/clawhub/tsconfig.json --noEmit
- bunx tsc --noEmit
- git diff --check origin/main...HEAD
2026-04-28 04:23:05 +01:00
Deepak Jain
f20dd624a5
fix: flag exposed secrets in skill docs (#1847)
* fix: flag exposed secrets in skill docs

Refs #1760

* fix: harden secret evidence redaction
2026-04-28 03:39:36 +01:00
Peter Steinberger
8752e4bb7e
chore(release): prepare clawhub cli 0.11.0 2026-04-28 02:26:56 +01:00
Peter Steinberger
bc06c472e4
fix(packages): authenticate repository lookups 2026-04-28 02:20:35 +01:00
Peter Steinberger
09fa7daa7b
docs: clarify skill monetization support 2026-04-28 01:52:37 +01:00
Peter Steinberger
e21ca80a7d
docs: document public catalog reuse 2026-04-28 01:21:46 +01:00
Peter Steinberger
456f4db74d
fix(search): widen lexical fallback coverage 2026-04-27 22:46:22 +01:00
Patrick Erichsen
7014a53fdf
Merge pull request #1837 from openclaw/pe/convex-ai-files
[codex] Add Convex AI guidance and skills
2026-04-27 14:25:54 -07:00
Patrick Erichsen
c6c4481ffd chore: keep only Convex agent skills
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-27 14:20:04 -07:00
Patrick Erichsen
02b7d10af8 chore: refresh agent skill state 2026-04-27 14:12:01 -07:00
Patrick Erichsen
c5a6d2700f chore: track repo agent skills 2026-04-27 14:10:42 -07:00
Val Alexander
205db67e99
feat(packages): expose package soft delete API 2026-04-27 16:07:19 -05:00
Patrick Erichsen
f406c5bf16 ci: use pinned Convex CLI for AI file updates 2026-04-27 13:59:30 -07:00
Patrick Erichsen
add0d13bef ci: run Convex AI update at midnight Pacific 2026-04-27 13:56:48 -07:00
Peter Steinberger
7c5b8b2a20
fix(search): boost exact slug matches 2026-04-27 21:56:24 +01:00
Patrick Erichsen
d3ed4434b9 feat: seed local rescan fixtures 2026-04-27 13:53:14 -07:00
Patrick Erichsen
7f7d6676c3 ci: update Convex AI files weekly 2026-04-27 13:52:27 -07:00
Patrick Erichsen
aff186bbcf chore: add Convex AI guidance files 2026-04-27 13:46:14 -07:00
Patrick Erichsen
1c430cc11d feat: add scanner-specific security pages 2026-04-27 13:44:37 -07:00
Peter Steinberger
e2cb7dfe4e
fix(search): fall back when embeddings fail 2026-04-27 21:34:42 +01:00
Patrick Erichsen
fb3bcbafb1 feat: show owner flagged inventory on dashboard 2026-04-27 13:27:08 -07:00
Patrick Erichsen
e6c3d6ff28 feat: add owner rescan requests 2026-04-27 13:25:59 -07:00
Peter Steinberger
57970579cf
fix(search): widen vector candidate pool 2026-04-27 21:10:13 +01:00
Peter Steinberger
2ddc52c0b0
test: fix root test and typecheck gates
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-04-27 20:51:00 +01:00
Peter Steinberger
e63031c452
fix(skill-install): use openclaw-compatible slug 2026-04-27 20:42:22 +01:00
Peter Steinberger
e3c772d90b
fix(search): preserve expanded candidate scoring 2026-04-27 20:41:20 +01:00
Peter Steinberger
73e26e51c1
fix(moderation): narrow webhook flag 2026-04-27 20:40:00 +01:00
Peter Steinberger
e7a1e9937b
fix(moderation): keep skill status on latest version 2026-04-27 20:39:02 +01:00
Peter Steinberger
d5776f8499
fix(skills): separate historical detail tags 2026-04-27 20:37:41 +01:00
Peter Steinberger
280352d959
docs: thank recent contributors 2026-04-27 20:36:56 +01:00
Deepak Jain
bd375f6a93
fix: reduce env scan false positives
Allow declared env vars used with network API calls without weakening broad env scraping or exfiltration findings.\n\nCloses #1790
2026-04-27 20:36:40 +01:00
Deepak Jain
422f6d4e08
docs: surface package publish flow
Document code-plugin package publish required fields and a minimal manifest.\n\nCloses #1796
2026-04-27 20:34:56 +01:00
Peter Steinberger
dd111cacee
fix(api): restore public skills listing
Fixes #1722
Fixes #1739
2026-04-27 20:33:51 +01:00
Peter Steinberger
ffa83db48a
fix: stabilize package plugin search 2026-04-27 20:26:49 +01:00
Momo
96c7ab1aaa
fix(skills): prevent backport publishes from clobbering the latest tag (#1832)
Merged via squash.

Prepared head SHA: bc2ef2216540537e5a406c4527ffc5ab21436da4
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-04-27 20:33:57 +08:00
Momo
743fa3abba
fix(skill-install): stabilize install surface layout
Some checks failed
CI / build (push) Has been cancelled
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
CSS-only stabilization for the skill install surface.

- neutralize Radix scroll-lock body compensation now that the app reserves scrollbar gutter globally
- make the install surface span the full hero width and keep the two install panels balanced
- reserve stable space for prompt feedback and prompt preview content to avoid toggle reflow

Verified locally:
- bunx tsc -p packages/schema/tsconfig.json --noEmit
- bunx tsc -p packages/clawhub/tsconfig.json --noEmit
- bun run build
2026-04-24 15:31:17 -05:00
Val Alexander
6c079e93c2
fix: satisfy skill install typecheck
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
Repair the skill install surface follow-up typecheck issue after #1800 merged.

- replace the unused local exhaustiveness sentinel in `skillDetailUtils` with a shared `assertNever` helper
- keep the package-manager switch exhaustive without tripping `noUnusedLocals`
2026-04-23 15:36:14 -05:00
Val Alexander
23a109c037
feat: add skill install prompt surface
Add a dedicated skill install surface that pairs OpenClaw prompt-driven install with visible CLI commands.

- add Install with OpenClaw and CLI Commands panels to the skill detail page
- add Copy Prompt modes for Install Only and Install & Setup plus package-manager switching for the ClawHub CLI command
- add regression coverage for the new surface and make the repo build path use the working Vite invocation
2026-04-23 15:29:43 -05:00
Patrick Erichsen
f28c1745f9
Merge pull request #1794 from openclaw/fix/vercel-image-allow-svg
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
fix(security): allow SVGs through image optimizer so badges render
2026-04-22 22:52:27 -07:00
Patrick Erichsen
4fe275eb50 fix(security): enable safe SVG handling so shields.io badges render
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
vercel.json currently allow-lists SVG-only hosts (img.shields.io,
shields.io, badgen.net, flat.badgen.net) while dangerouslyAllowSVG:
false rejects every SVG source. Those two settings are incompatible,
and every badge in every README on production is returning 400
INVALID_IMAGE_OPTIMIZE_REQUEST (e.g. the license badge on
/plugins/@opik/opik-openclaw).

Switch to the pattern Vercel documents for safely serving SVGs in
their NEXTJS_SAFE_SVG_IMAGES conformance rule:

- dangerouslyAllowSVG: true  — lets the optimizer accept SVG inputs
- contentDispositionType: attachment  — forces download instead of
  inline document rendering if someone navigates directly to the
  /_vercel/image URL (the only context where SVG scripts would run)
- contentSecurityPolicy: script-src 'none'; sandbox;  — blocks script
  execution in the response

Defense in depth: browsers already sandbox SVGs loaded through <img>
so scripts don't run there anyway; the CSP + attachment header cover
the edge case of someone opening the optimizer URL directly. Net
security is equivalent to rejecting SVGs, but badges actually render.

Docs: https://vercel.com/docs/conformance/rules/NEXTJS_SAFE_SVG_IMAGES

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 22:50:43 -07:00
Patrick Erichsen
c0d2ac7ac0
Merge pull request #1793 from openclaw/fix/image-proxy-xss
fix(security): proxy README images via Vercel Image Optimization
2026-04-22 22:42:32 -07:00
Patrick Erichsen
d6d4028660 refactor(security): swap ProxiedImg component for rehype plugin
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Replaces the React <img> wrapper with a tiny rehype plugin that rewrites
image srcs in the HAST. Same behavior (external http(s) URLs routed
through /_vercel/image; local/relative/data: URIs pass through), less
surface area:

- One shared plugin wired into both MarkdownPreview and SkillDetailTabs
  via rehypePlugins instead of a components override at each call site
- Dropped ProxiedImg.tsx + its 7 unit tests; the two integration tests
  in MarkdownPreview.test.tsx still assert the proxy URL shape for both
  <img> and ![](url) syntax
- Stopped reading <img width="..."> for the proxy's w= param. Vercel
  requires w to match a value in vercel.json sizes, so arbitrary README
  widths (e.g. width="200") would have been rejected. Always w=1024 now;
  the HTML width attribute still drives layout

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 22:32:21 -07:00
Patrick Erichsen
82ae30d940 fix(security): proxy README images via Vercel Image Optimization
Closes the XSS / IP-leak surface from rendering third-party README
images directly on clawhub.ai. Routes external http(s) <img> sources
through Vercel's /_vercel/image endpoint, which enforces a host
allow-list, rejects SVG by default, and re-encodes rasters to webp.

Docs: https://vercel.com/docs/image-optimization

- vercel.json: add `images` config — host allow-list (raw.githubusercontent,
  shields.io, etc., based on NuGet's published README allow-list),
  dangerouslyAllowSVG=false, formats=[webp], 1d minimum cache TTL.
- src/components/ProxiedImg.tsx: small wrapper that rewrites external
  http(s) src URLs to /_vercel/image?url=...&w=...&q=75. Local paths,
  relative paths, and data: URIs pass through unchanged.
- MarkdownPreview + SkillDetailTabs: pass ProxiedImg as the `img`
  component override to react-markdown — covers both raw HTML <img>
  and markdown ![](url) syntax.
- package.json: drop unused `next` dep (vestigial from staging merge,
  zero imports anywhere; doesn't affect next-themes).

Tests: 1028/1028 (was 1017, added 11 — ProxiedImg unit tests +
markdown integration tests covering proxied vs passthrough paths).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 22:12:40 -07:00
Patrick Erichsen
b53813a5a7
Merge pull request #1792 from openclaw/fix/lint-cleanup-staging-fallout
chore(lint): clean up 70 oxlint errors from staging merge #1573
2026-04-22 21:39:34 -07:00
Patrick Erichsen
87469792d5 fix(typecheck): clear remaining tsc errors on main
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
8 typecheck errors that have been on main alongside the lint debt:

- convex/apiSurface.typecheck.ts: drop two stale @ts-expect-error
  directives. The `increment` references they guarded no longer
  exist (functions renamed to *Internal); runtime internal-only
  enforcement is preserved by `internalMutation`.
- src/components/MarkdownPreview.tsx: cast createHighlighter result
  to AnyHighlighter, narrow loadHighlighter return via the local
  promise variable, type baseRehype + memoized rehypePlugins as
  PluggableList (drops `as const` readonly mismatch with
  ReactMarkdown's prop type).
- src/lib/theme.test.tsx: rename remaining "hub" usages to "claw"
  (theme families collapsed to one in PR #1573 — the last "hub"
  references in the harness button + applyTheme call would never
  compile under the current ThemeName type).
- src/lib/packageApi.test.ts: add `?.` on the nullable result.

Full suite: lint 0, tests 1017/1017, typecheck 0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 21:33:17 -07:00
Patrick Erichsen
9e7407cd84 test: update stale assertions left over from staging merge
Two pre-existing test failures on main, both caused by UI/data
changes in PR #1573 that the tests weren't updated for:

- theme.test.tsx: expected stored theme "hub" to round-trip, but
  the staging merge collapsed all families into a single "claw"
  theme — unknown families now fall back to "claw". Test now
  asserts the legacy fallback behavior it claims to test.
- skill-detail-page.test.tsx: gated on the platform license
  summary text, which was removed from SkillMetadataSidebar in
  4d1a08b. Drop the obsolete assertion; the report-button
  findByRole on the next line provides the same render-wait.

Full suite: 1017/1017 pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 21:23:29 -07:00
Patrick Erichsen
70fd9436cf chore(lint): clean up oxlint errors from staging-merge fallout (#1573)
Fixes 70 oxlint errors that landed in the 2026-04-18 staging merge and
have kept main red ever since. Three rule categories:

- typescript-eslint(no-unnecessary-type-conversion): drop redundant
  String/Number/Boolean wraps + 'as T' casts on values already typed.
- typescript-eslint(consistent-return): unify mixed return paths,
  mostly in useEffect callbacks (early-return vs cleanup-fn) and CLI
  command handlers.
- typescript-eslint(no-unnecessary-type-parameters): drop generics
  used only once in a signature; replace with concrete types.
- Plus a handful of no-unused-vars, no-shadow, and one
  no-redundant-type-constituents (JSX.Element -> ReactNode).

No runtime behavior changes. Full lint clean (0 errors); test suite
shows the same 2 pre-existing failures as main, no new regressions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 21:16:23 -07:00
Vincent Koc
5e7584e032
Merge pull request #1791 from openclaw/fix/markdown-html-passthrough 2026-04-22 20:59:27 -07:00
Patrick Erichsen
ea0824878d fix(markdown): render raw HTML + GFM in MarkdownPreview, add shiki highlighting
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Plugin/soul READMEs that use raw HTML (e.g. centered logos via
<h1 align="center">, <picture>, <br/>) were rendering as escaped
text because @create-markdown/preview escapes all HTML. Swap the
renderer for react-markdown + remark-gfm + rehype-raw +
rehype-sanitize (GitHub's stack), with rehype-shiki-from-highlighter
for fenced code block syntax highlighting.

Sanitize runs before shiki so user HTML is scrubbed, and shiki's
trusted styled output flows through untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 20:33:54 -07:00
Val Alexander
da74b2a382
Update .gitignore
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-04-22 14:22:53 -05:00
Val Alexander
4787be4eb1
Refresh Open Graph image (#1754)
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
* Refresh OG image and bust cache version

- Replace the social preview artwork with a new branded SVG and updated PNG
- Add a versioned og image URL in the root head tags to ensure the new asset is served

* Refresh OG image design

- Redesign the social preview graphic for the new ClawHub branding
- Bump the OG image version so the updated asset is served

* fix: refine clawhub og image

* fix: center og logo layout

* fix: emphasize clawhub branding in og image

* Refresh OG image branding

- Update Open Graph artwork and logo asset
- Adjust root metadata to use the new social preview image

* fix: refine clawhub og image

* fix: tighten og image layout

* fix: remove og logo panel

* fix: reduce og logo scale

* fix: align og image to new comp
2026-04-20 21:59:08 -05:00
Gustavo Madeira Santana
89246f1927
chore(ui): remove gap before hero cycled words
Some checks failed
CI / build (push) Has been cancelled
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-19 13:25:56 -04:00
Val Alexander
f4ddccbead
enchance: mobile skills ux (#1737) 2026-04-18 20:10:03 -05:00
Val Alexander
3cafcbf873
Mobile search icon + system theme on first load
- Initialize root theme data from stored selection before paint
- Hide the search label on mobile and tighten button padding
2026-04-18 18:40:56 -05:00
Val Alexander
13064a7897
Merge pull request #1731 from openclaw/okcode/fix-mobile-search-button
Fix mobile header branding and add Home link
2026-04-18 17:52:29 -05:00
Val Alexander
194c22f4dd
Add branded mobile nav header
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
- Show the logo mark in the mobile drawer title
- Tighten mobile suggestion spacing on small screens
- Add test coverage for the branded mobile nav header
2026-04-18 17:50:21 -05:00
Val Alexander
a693b945fa
Add Home link to mobile header navigation
- Insert a Home entry at the top of the mobile menu
- Update header tests to cover the new menu order
2026-04-18 17:40:22 -05:00
Val Alexander
9bef672541
Merge branch 'okcode/polished-card-icons-logo' 2026-04-18 17:28:51 -05:00
Val Alexander
9551cac37b
Merge pull request #1729 from openclaw/okcode/fix-settings-update
Stabilize preferences sync and keep diff editor mounted
2026-04-18 17:25:52 -05:00
Val Alexander
eb4138fbb3
fix: harden preferences storage sync 2026-04-18 17:24:31 -05:00
Val Alexander
5fbead624b
Update src/lib/preferences.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-18 17:17:18 -05:00
Val Alexander
35094177e6
Keep diff editor mounted when switching view mode
- Remove the diff editor remount on inline vs side-by-side toggles
- Add a regression test to verify the editor stays mounted
2026-04-18 17:04:30 -05:00
Val Alexander
faa5c9f2b5
Polish icons and brand mark styling
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
- Simplify home and settings labels by removing redundant icons
- Swap automation icons to refresh glyphs in sidebars and toolbar
- Add subtle border and shadow treatment to the brand mark
2026-04-18 17:03:35 -05:00
Val Alexander
c3314c2d01
Stabilize preference snapshots and storage sync
- Cache localStorage reads to avoid redundant snapshot churn
- Sync updates across tabs and add coverage for preference re-renders
2026-04-18 17:03:15 -05:00
Val Alexander
7dfa19157c
Merge pull request #1573 from openclaw/staging
chore: merge staging into main
2026-04-18 16:46:00 -05:00
Val Alexander
44acf86ac1
merge: resolve AGENTS.md conflict — keep both convex-ai and stat migration rules
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 16:45:37 -05:00
Val Alexander
a0ebc1b50a
style: spread footer columns evenly across full width
Switch footer grid from auto-sized centered columns to equal 1fr
columns that span the full screen width.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 16:44:28 -05:00
Val Alexander
88dbb69a23
style: adopt darker home-v2 palette globally and unify radius to 8px
Shift all theme variants (claw dark/light, hub dark/light) to the
bolder home-v2 backgrounds (#060608 dark, #faf6f1 light cream).
Harmonize surface, nav-bg, input-bg, and overlay-bg to match.
Set every radius token (--r-lg/md/sm/xs/pill) and home-v2 hardcoded
radii to a single consistent 8px value.
Remove home-v2–specific overrides for app-shell background, navbar
background, footer transparency, and navbar-inner max-width that
previously caused visual divergence between the home page and the
rest of the app.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 16:30:53 -05:00
Val Alexander
df9acd27e4
update: styles 2026-04-18 16:28:00 -05:00
Val Alexander
dbd5d4042c
fix: restore header logo and compact/center footer
Uncomment the brand logo image in the header navbar and reduce footer
vertical padding, gaps, and margins to ~55% of original height while
centering the grid columns and link text.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 16:17:17 -05:00
Val Alexander
ebe82b7e18
Improve about page rejection categories (#1728)
* improve about page rejection categories: add icons, fix grid, polish cards

- Add lucide-react icons to each rejection category card for visual scanning
- Fix unbalanced grid layout by removing featured card sizing, using clean 2/3-col grid
- Fix broken hover transitions (var(--transition-fast) was undefined outside reduced-motion)
- Add lift-on-hover effect and icon glow matching home page card patterns
- Render backtick-wrapped text as styled inline code elements
- Improve description text contrast from ~3.5:1 to ~4.8:1 (WCAG AA)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use h3 for card titles to fix heading hierarchy (WCAG 1.3.1)

Change rejection category card titles from <h2> to <h3> since the parent
section already uses <h2> for "Immediate rejection categories". Updates
the matching CSS selector from .about-rule-card h2 to h3.

Also adds tests for renderWithInlineCode helper covering plain text,
single/multiple code spans, empty input, and code-only strings.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove white backgrounds from all logo assets

- Remove white backgrounds from clawd-logo.png, clawd-mark.png,
  logo192.png, logo512.png — now transparent PNGs
- Convert white strokes to dark (#1a0808) in both PNGs and logo.svg
  so segments separate cleanly on any background
- Defringe antialiased edges to eliminate white halos
- Regenerate favicon.ico from transparent source
- Update manifest.json background_color from #ffffff to #0a0a0a

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* consolidate logo assets: delete SVGs, use only PNGs with transparent bg

- Delete public/logo.svg, public/og.svg, src/logo.svg (dead/unused SVGs)
- Remove logo.svg favicon link from __root.tsx (favicon.ico remains)
- Remove white backgrounds from clawd-logo.png and clawd-mark.png
- Convert white strokes to dark (#1a0808), defringe antialiased edges
- Regenerate logo192.png, logo512.png, favicon.ico from clean sources
- Only canonical logo files are now clawd-logo.png and clawd-mark.png

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: widen navbar search bar and polish hero section

Let the search bar span the full width between brand and theme toggle
by removing the oversized right-column minimum and theme-toggle min-width.
Widen the hero search container, subtitle, and tighten vertical padding
for a sleeker feel.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use :is(h2, h3) selector for about-rule-card headings

The /souls page reuses about-rule-card with <h2> elements. Using
:is(h2, h3) ensures both heading levels get styled consistently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: slot machine Easter egg on hero label triple-click

Triple-clicking "BUILT BY THE COMMUNITY" triggers a casino-style slot
machine across all 3 headline words. Reels spin and stop sequentially
with a 1/13 jackpot chance. Winning fires a confetti celebration with
golden text glow. Auto-resets after the animation completes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add cooldown, longer celebration, and Hack x3 lobster jackpot

- 18s cooldown after a win, 3s after a loss to prevent spam
- Win celebration extended to 10s for screenshot opportunities
- Hack x3 jackpot triggers aquatic theme: cyan/teal text glow,
  ocean-colored confetti with bubble and claw particles, and the
  lobster logo fades in behind the headline

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: tune slot machine odds to 1/25 any jackpot, 1/100 Hack jackpot

Replace pure random picks with controlled probability: 4% chance of
any jackpot per spin, with 25% of jackpots being Hack (= 1% overall).
Non-jackpot spins re-roll accidental triple matches to keep odds exact.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clean up slot timers on unmount, fix about-grid specificity

Add useEffect cleanup to clear slot machine timers/intervals when
the home route unmounts mid-animation. Fix about-grid media query
specificity by including .about-panel-categories .about-grid to
override the higher-specificity base rule.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-18 16:07:48 -05:00
Val Alexander
4c566268a9
fix: hide logo, clean up rejection categories layout (#1727)
Some checks are pending
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
* fix: hide logo, use ClawHub as home link, and clean up rejection categories grid

Comment out the brand logo image for now, rename "Immediate rejection
categories" to "Rejection Categories", remove the featured card variant,
and switch to an auto-fill grid so cards spread evenly at full width.
Add overflow: visible on the categories panel to prevent hover shadow
clipping.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: impose max page width on home page using --page-max (1536px)

Constrain .home-v2-main to max-width: var(--page-max) and center it
with margin-inline: auto. Extend the home page background color to the
full viewport via .app-shell:has(.home-v2-main) for both light and dark
themes so the background bleeds edge-to-edge beyond the content column.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove extra footer padding and ensure full-width nav/footer for boxed layout

Zero out the outer .site-footer padding and set background to transparent
on home-v2 pages so the app-shell background bleeds through edge-to-edge.
Remove the redundant light-mode footer background override (app-shell
background already covers it). Nav and footer now visually span full
viewport width while .home-v2-main content stays boxed at --page-max.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: reduce carousel card hover effect and increase track padding

The carousel cards were getting clipped by the parent overflow:hidden
container. Reduce the hover transform from translateY(-4px) scale(1.01)
to translateY(-2px) and shrink box-shadow spread across all theme
variants. Increase carousel track top padding from 4px to 12px to
accommodate the upward shift without cutoff.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review feedback — mobile brand, category grid, hover drama

- Keep brand name visible on mobile (remove display:none for
  .brand-name-responsive at ≤639px) so the home link is always
  discoverable. Add TODO comment on the commented-out logo block.
- Add .about-panel-categories .about-grid to the ≤640px media query
  so the category grid correctly collapses to single-column on mobile.
- Bump carousel card hover to translateY(-3px) with 0 6px 24px shadow
  for a slightly more dramatic lift — still within the 12px top / 48px
  bottom track padding so nothing clips.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove unused footer divider element

The site-footer-divider was already hidden via CSS (display: none) on
home-v2 pages. Remove the element entirely since it serves no purpose.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-18 14:37:43 -05:00
Val Alexander
e54fc1939a
fix: normalize nav and footer layout 2026-04-18 13:41:36 -05:00
Momo
530e39eedc
refactor: extract readCanonicalStat and add structural guards for stat field migration (#1709)
Some checks failed
CI / build (push) Has been cancelled
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Merged via squash.

Prepared head SHA: e92817f66f44254f78a27df12022bf7a1b1af5b9
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-04-17 17:44:32 +08:00
copilot-swe-agent[bot]
8b87c31a99
Merge remote-tracking branch 'origin/main' into staging
Some checks are pending
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
# Conflicts:
#	src/routes/management.tsx
#	src/routes/settings.tsx

Co-authored-by: BunsDev <68980965+BunsDev@users.noreply.github.com>
2026-04-17 08:49:21 +00:00
Momo
f7bc8b6349
fix(stats): fix skill stat field sync direction and reconcile logic (#1704)
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
Merged via squash.

Prepared head SHA: e814278382322fe834938fbde5428f8110c2fb33
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-04-16 20:04:13 +08:00
davida-ps
5b8f09167a
fix(api): align inspect security snapshot with static scan moderation
Include static scan results in the skill version security snapshot so inspect/API responses reflect the same moderation-relevant signal already used elsewhere. Also add regression coverage for suspicious, malicious, and static-only scan combinations.

Co-authored-by: davida-ps <232346510+davida-ps@users.noreply.github.com>
Co-authored-by: Luke <92253590+ImLukeF@users.noreply.github.com>
2026-04-16 18:39:39 +10:00
hugh
17fbd13bc9
fix(cli): use explorer on Windows to preserve auth URL params
On Windows, opening auth URLs via `cmd /c start` can truncate query parameters because `&` is treated as a command separator. Use `explorer` instead so the browser opener gets the full URL without shell parsing, and cover the Windows spawn args in the CLI UI test.

Co-authored-by: hugh <1012760428@qq.com>
2026-04-16 14:00:03 +10:00
Arthur Katcher
aab7dc9ba4
fix(upload): fall back to octet-stream for empty Content-Type
Handle browser uploads that provide an empty MIME type by falling back to `application/octet-stream` before sending the storage request.

Co-authored-by: Arthur Katcher <192321283+arthurkatcher@users.noreply.github.com>
Co-authored-by: Luke <92253590+ImLukeF@users.noreply.github.com>
2026-04-16 12:54:51 +10:00
ImLukeF
dde8796790
feat: tag skills needing sensitive credentials
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
CI / build (push) Has been cancelled
2026-04-14 20:09:24 +10:00
Val Alexander
acc6d292de
Home v2 styles: layout, theme & navbar tweaks
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Add and refine styles for the Home V2 UI: introduce navbar search/home styles, motto and headline variants, section copy/eyebrow rules, discovery and categories layouts, and responsive grid stacking. Adjust hv2 color variables (text-secondary/tertiary) and move category border to the grid element; update spacing/alignment for carousel and section headers. Add light/dark theme overrides to improve navbar, tabs and search contrast and hover states. Misc minor typographic and spacing refinements for a more cohesive Home V2 appearance.
2026-04-14 00:27:04 -05:00
ImLukeF
2236ed7be1
feat: add org profile editing 2026-04-14 14:10:52 +10:00
ImLukeF
f6fb7ccfc0
Revert "Reapply "feat: allow moderators to transfer skill publishers (#1663)""
This reverts commit b73758c7c8.
2026-04-14 13:39:49 +10:00
ImLukeF
b73758c7c8
Reapply "feat: allow moderators to transfer skill publishers (#1663)"
This reverts commit fbc07c5617.
2026-04-14 13:38:30 +10:00
ImLukeF
fbc07c5617
Revert "feat: allow moderators to transfer skill publishers (#1663)"
This reverts commit 80e5aec577.
2026-04-14 13:37:08 +10:00
Luke
80e5aec577
feat: allow moderators to transfer skill publishers (#1663) 2026-04-14 13:36:28 +10:00
Val Alexander
f869b31ad6
fix: remove leftover theme-family UI remnants
- drop mobile theme-family section in header
- remove unused theme-family settings bindings

Co-authored-by: Nova <nova@openknot.ai>
2026-04-13 22:15:06 -05:00
Val Alexander
9a853f2fcc
chore: update lockfile and favicon
- refresh bun.lock after dependency reinstall
- include favicon update

Co-authored-by: Nova <nova@openknot.ai>
2026-04-13 22:13:03 -05:00
Val Alexander
b4a7540157
feat: homepage redesign + unified theme + UI polish
- Redesign homepage with hero, search, featured carousel, categories, proof bar, trending
- Add cream/peach/tan light mode palette with inset-shadow pattern (dark + light)
- Remove Hub theme — single Claw theme only (light/dark mode toggle remains)
- Semi-rounded radius system (Claw × Hub midpoint: 4/7/10px)
- Consistent button radius site-wide (--r-btn: 4px), zero makeshift buttons
- Add VITE_FEATURE_SOULS env flag (default: false) to gate Souls pages
- Hide Souls from nav, footer, and homepage categories
- Remove theme family toggle from Header + Settings
- Widen page max to screen-2xl (1536px)
- Slow featured carousel 15% (40s → 46s)

Co-authored-by: Nova <nova@openknot.ai>
2026-04-13 21:59:23 -05:00
Val Alexander
0ea1127a2b
fix: refine header and about responsiveness (#1661)
Some checks are pending
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-04-13 13:03:40 -05:00
Val Alexander
aeab23a6d6
Fix dark-mode styling for skills filter chips (#1660)
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
- Add readable dark-surface and active-state colors to filter chips
- Cover the toolbar styling with a jsdom test
2026-04-13 13:01:37 -05:00
Val Alexander
15bc4440cc
style: add claw red accents to hub theme 2026-04-13 10:44:51 -05:00
Val Alexander
11a20f5755
refactor: reduce marketplace themes to claw and hub 2026-04-13 10:38:36 -05:00
Val Alexander
05f8674628
refactor: simplify footer browse links and theme switching 2026-04-13 10:19:15 -05:00
Val Alexander
d2b2252770
Merge branch 'main' into staging 2026-04-13 09:48:02 -05:00
Val Alexander
a2387253ec
style: give knot theme distinct component treatments 2026-04-13 04:22:39 -05:00
Val Alexander
411260767b
style: make knot theme darker and purple 2026-04-13 04:18:42 -05:00
Val Alexander
731d0ce0c5
feat: restyle dash theme to match clawhub v2 2026-04-13 03:59:25 -05:00
Val Alexander
361f2affde
feat: add clawhub theme families and tweakcn import 2026-04-13 03:19:17 -05:00
Val Alexander
a17f7bb07e
feat: modernize clawhub app store (#1655)
* build(deps-dev): bump vite in the npm_and_yarn group across 1 directory (#1561)

Bumps the npm_and_yarn group with 1 update in the / directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 8.0.1 to 8.0.5
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.0.5
  dependency-type: direct:development
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: detect generated-source template injection in skill scans (#1597)

* fix: detect exposed resource identifiers in skill scans (#1598)

* fix: restore ci checks after lockfile drift

* refactor: address actionable review cleanup (#1601)

* fix: prevent starring soft-deleted skills and fix star count reconciliation (#1605)

* feat: Add support for Chinese Japanese and Korean(CJK) skills search (#1596)

Merged via squash.

Prepared head SHA: ab58f01be712cb7b9a6ce6cf52d61afab0022446
Co-authored-by: pq-dong <40668796+pq-dong@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage

* docs: document CLI config paths across platforms (#1252)

* docs: document CLI config paths across platforms

* docs: clarify legacy config fallback

---------

Co-authored-by: ImLukeF <92253590+ImLukeF@users.noreply.github.com>

* fix: point plugin metadata help link to OpenClaw docs (#1399)

* fix: point plugin metadata help link to OpenClaw docs

* fix: open plugin metadata docs in a new tab

* fix(cli-auth): ensure fallback token renders before redirect on Windows/Chrome (#1486)

* fix(cli-auth): ensure fallback token renders before redirect on Windows/Chrome

React batches state updates, so setToken() and window.location.assign()
previously raced: the navigation could fire before React re-rendered the
fallback token UI. On Chrome/Windows this means a failed http:// redirect
(ERR_CONNECTION_REFUSED, HTTPS-first interference) would replace the page
with an error screen before the user ever saw the token.

Use flushSync() to render the token synchronously, then attempt
window.location.assign(). If the redirect fails the token and a "Retry
redirect to CLI" link are already painted on screen.

Fixes #1469

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: cover cli auth fallback redirect

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ImLukeF <92253590+ImLukeF@users.noreply.github.com>

* fix: reduce souls browse overfetch (#1637)

* fix: improve admin user search coverage (#1466)

* fix admin user search coverage

* fix admin user search without full table scan

* feat: include stats in package detail API response

Expose package detail stats through the shared API contract and the app client.

This lands the original package detail stats work and folds in the follow-up cleanup to keep the response shape sourced from the shared schema instead of a hand-maintained app-local type.

Co-authored-by: Saurabh Jain <saurabhjain1592@gmail.com>

* test: cover package detail stats response

* fix: normalize misleading MIME types for text files

* feat: modernize clawhub app store

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Luke <92253590+ImLukeF@users.noreply.github.com>
Co-authored-by: Momo <35096042+momothemage@users.noreply.github.com>
Co-authored-by: pqdong <40668796+pq-dong@users.noreply.github.com>
Co-authored-by: Jholly <xiangjunkong90@gmail.com>
Co-authored-by: loong <46096863+robinspt@users.noreply.github.com>
Co-authored-by: Yaovi <dkpoga@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Saurabh Jain <saurabhjain1592@gmail.com>
2026-04-13 01:36:18 -05:00
ImLukeF
7980788ea6
fix: normalize misleading MIME types for text files 2026-04-13 15:54:42 +10:00
Val Alexander
2d03b827d3
fix: close plugins route loader return 2026-04-13 00:44:04 -05:00
Val Alexander
835094ea2c
Update convex/seedDemo.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-13 00:35:16 -05:00
Val Alexander
7bd7e4c99e
Modern utility store (#1646)
* feat: redesign ClawHub marketplace with modern utility store theme

Update styles.css and index.tsx for new modern design

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* style: polish marketplace UI with modern design updates

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* fix: resolve 500 errors in TanStack Router loaders

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* fix: resolve SSR error bubbling in TanStack Router

Ensure loader errors don't escape SSR and hydrate correctly.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* feat: handle errors in Package API calls to prevent SSR and HMR errors

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* feat: update branding to OpenClaw's black, white, and red color scheme

Implement new color scheme across dark, light themes and interactive elements

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* feat: use Vite's native tsconfig paths

Replace deprecated plugin with native option and remove unused import.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* fix: add defensive checks in fetchPluginCatalog

Ensure proper handling of undefined and unexpected API responses.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* fix: add defensive checks to PluginsIndex for SSR errors

Handle undefined loader data in PluginsIndex component.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* refactor: optimize skill detail page layout

Redesign skill detail page to maximize space, remove sidebar, create metadata bar, and add responsive breakpoints.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* fix: add global overflow protection to detail pages

Add overflow prevention for text elements and links.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* feat: highlight parent tabs with activePathPrefixes

Add activePathPrefixes to NavItem and update navigation to highlight parent tabs.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* feat: add user preferences customization section

Add 'usePreferences' hook and new Switch component; enhance settings page with Customization section.

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* feat: optimize layout and create DESIGN.md

Fix orphan cards, enforce equal card heights, add branding accents, improve visual hierarchy, add responsive breakpoints, create design document

Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>

* Update src/routes/plugins/index.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update src/components/layout/Container.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: v0 <v0[bot]@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-12 23:23:08 -05:00
Val Alexander
1722a48055
fix: resolve review feedback on layout and auth state (#1652) 2026-04-12 23:21:11 -05:00
Val Alexander
29178898bb
fix: resolve review feedback on layout and auth state (#1647) 2026-04-12 22:41:30 -05:00
ImLukeF
b16861f422
test: cover package detail stats response
Some checks are pending
CI / build (push) Waiting to run
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Waiting to run
2026-04-12 22:18:01 +10:00
Saurabh Jain
6896e61fa1
feat: include stats in package detail API response
Expose package detail stats through the shared API contract and the app client.

This lands the original package detail stats work and folds in the follow-up cleanup to keep the response shape sourced from the shared schema instead of a hand-maintained app-local type.

Co-authored-by: Saurabh Jain <saurabhjain1592@gmail.com>
2026-04-12 21:59:38 +10:00
Luke
cf137aa592
fix: improve admin user search coverage (#1466)
* fix admin user search coverage

* fix admin user search without full table scan
2026-04-12 21:18:19 +10:00
Luke
a28d014d4f
fix: reduce souls browse overfetch (#1637) 2026-04-12 21:04:12 +10:00
Yaovi
b2038fc931
fix(cli-auth): ensure fallback token renders before redirect on Windows/Chrome (#1486)
* fix(cli-auth): ensure fallback token renders before redirect on Windows/Chrome

React batches state updates, so setToken() and window.location.assign()
previously raced: the navigation could fire before React re-rendered the
fallback token UI. On Chrome/Windows this means a failed http:// redirect
(ERR_CONNECTION_REFUSED, HTTPS-first interference) would replace the page
with an error screen before the user ever saw the token.

Use flushSync() to render the token synchronously, then attempt
window.location.assign(). If the redirect fails the token and a "Retry
redirect to CLI" link are already painted on screen.

Fixes #1469

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: cover cli auth fallback redirect

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ImLukeF <92253590+ImLukeF@users.noreply.github.com>
2026-04-12 19:58:11 +10:00
loong
4a72b543b2
fix: point plugin metadata help link to OpenClaw docs (#1399)
* fix: point plugin metadata help link to OpenClaw docs

* fix: open plugin metadata docs in a new tab
2026-04-12 18:05:57 +10:00
Jholly
54e99c8cc2
docs: document CLI config paths across platforms (#1252)
* docs: document CLI config paths across platforms

* docs: clarify legacy config fallback

---------

Co-authored-by: ImLukeF <92253590+ImLukeF@users.noreply.github.com>
2026-04-12 18:02:41 +10:00
pqdong
5826001795
feat: Add support for Chinese Japanese and Korean(CJK) skills search (#1596)
Some checks failed
CI / build (push) Has been cancelled
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
Merged via squash.

Prepared head SHA: ab58f01be712cb7b9a6ce6cf52d61afab0022446
Co-authored-by: pq-dong <40668796+pq-dong@users.noreply.github.com>
Co-authored-by: momothemage <35096042+momothemage@users.noreply.github.com>
Reviewed-by: @momothemage
2026-04-10 12:20:40 +08:00
Momo
0708a43fde
fix: prevent starring soft-deleted skills and fix star count reconciliation (#1605)
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-09 22:16:51 +08:00
Nimrod Gutman
9a45c371fc fix(ui): align browse page widths across tabs
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-09 14:19:27 +03:00
Luke
311a123fbe
refactor: address actionable review cleanup (#1601) 2026-04-09 20:00:25 +10:00
ImLukeF
59e93862ed
fix: restore ci checks after lockfile drift
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
2026-04-09 19:45:51 +10:00
Luke
8fcd53f899
fix: detect exposed resource identifiers in skill scans (#1598) 2026-04-09 19:32:55 +10:00
Luke
ba2c73e180
fix: detect generated-source template injection in skill scans (#1597) 2026-04-09 19:10:39 +10:00
Val Alexander
298cbdd6db fix: denormalize user hover stats 2026-04-08 15:16:15 -05:00
dependabot[bot]
fa87dc3509
build(deps-dev): bump vite in the npm_and_yarn group across 1 directory (#1561)
Bumps the npm_and_yarn group with 1 update in the / directory: [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `vite` from 8.0.1 to 8.0.5
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.0.5
  dependency-type: direct:development
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-07 08:21:53 -05:00
Val Alexander
f636b31fca
feat: design foundation sweep (#1570) thanks @BunsDev
Co-authored-by: Nova <nova@openknot.ai>
2026-04-07 07:48:20 -05:00
Val Alexander
b255b5865f
Merge pull request #1567 from openclaw/okcode/conflict-resolution-plan
feat: marketplace UI overhaul with main security fixes
2026-04-07 02:07:33 -05:00
Val Alexander
5003c1bec8
fix: address remaining PR review comments
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
- Re-throw non-rate-limit errors in plugin loader so route error
  boundary handles real failures instead of showing empty results
- Bump requestRef on query clear to invalidate in-flight searches
  and prevent stale results from repopulating
- Replace Promise.all with Promise.allSettled in unified search so
  one failing provider doesn't blank results from other sources
- Log unexpected errors in unified search catch block

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 02:04:19 -05:00
Val Alexander
a8a6242f87
fix: address PR review — reason guard, one-shot fetch, index scan
- Reject empty reason strings in setSoftDeleted calls + re-add .catch()
  for error feedback (both reported-skills and skill-tools sections)
- Replace useQuery with ConvexHttpClient.query() on /users public
  browse page per CLAUDE.md policy
- Add by_active_handle compound index on users table to avoid full
  table scan in queryUsersForPublicList

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 00:55:41 -05:00
Val Alexander
39c0fa2531
merge: integrate origin/main into feat/marketplace-ui-overhaul
Keep feature branch UI overhaul (custom CSS) while incorporating
security/stability fixes from main:
- setSoftDeleted now requires moderation reason (runtime-critical)
- moderationNotes displayed in skill detail when available
- Rate limit handling for plugin catalog
- Tailwind @theme block for auto-merged component compatibility
- Capability tag passthrough to SecurityScanResults
- ALL_CATEGORY_KEYWORDS export for skills browse model

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 00:28:17 -05:00
Val Alexander
9df6fe37c5
Merge pull request #1564 from openclaw/okcodes/fix-lightmode
Add sortable paginated data table components
2026-04-06 20:37:44 -05:00
Val Alexander
d5d806516d
Merge remote-tracking branch 'origin/main' into okcodes/fix-lightmode
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
# Conflicts:
#	src/routes/skills/-SkillsToolbar.tsx
2026-04-06 20:35:54 -05:00
Val Alexander
67c74e10ea
Hide empty data table pagination
- Make pagination conditional on there being rows and pages
- Tighten props so manual pagination requires a page count
2026-04-06 20:29:09 -05:00
Val Alexander
3501af6bb4
update create-markdown packages to 2.0.1 2026-04-06 20:28:47 -05:00
Vincent Koc
051b1dafcd fix: audit fixes — warm colors, lint, ARIA labels
- Replace ~20 remaining warm hex colors in upload/form styles
  (#ffddc9, #9a3a24, #fff3ec, etc.) with monochrome equivalents
- Fix 2 lint errors: remove unused Link import (search.tsx),
  prefix unused parseDir with underscore (souls/index.tsx)
- Add aria-label to PluginListItem and UserListItem for
  screen reader identification
2026-04-06 22:58:12 +01:00
Vincent Koc
383844cacf chore: various component changes 2026-04-06 22:03:49 +01:00
Vincent Koc
b7923edbfd Update .gitignore 2026-04-06 20:41:09 +01:00
Val Alexander
655c914c77
Restore dark-mode styling in skills toolbar
Some checks failed
Security Gate: Secret Scanning / Scan for Verified Secrets (push) Has been cancelled
- Resolve merge markers in `-SkillsToolbar.tsx`
- Keep control surfaces readable in dark mode
- Refresh lockfile for dependency bumps
2026-04-06 13:31:33 -05:00
Val Alexander
b15eeab93f
Improve dark mode contrast in skills toolbar
- Apply dark-themed control surface styles to search, select, and view toggles
- Adjust filter chips and labels for better visibility in dark mode
2026-04-06 13:31:33 -05:00
Val Alexander
a39f07427e
Stop empty public page scans from reporting more pages
- Clear `hasMore` and `nextCursor` when a scan hits the budget but returns no items
- Prevent the client IntersectionObserver from looping on empty auto-load responses
2026-04-06 13:31:33 -05:00
Val Alexander
1a42207879
Update src/__tests__/package-detail-route.test.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-06 13:31:33 -05:00
Val Alexander
9bd3a63edf
Equalize stat widths across listings
- Align skill and soul metric chips to fixed widths
- Rebalance skills table columns for better summary space
- Tighten spacing in soul list rows
2026-04-06 13:31:33 -05:00
Luke
be4c51ed1b
fix: improve auth flow and webkit compatibility (#1555)
* fix: improve auth flow, skill filters, and webkit compatibility

* test: mock auth actions in settings route

* fix: remove stale skills toolbar props

* fix: address skills filter and webkit review feedback

* test: avoid monaco lazy import in skill detail test

* fix: recompute other skills category filter
2026-04-06 13:31:33 -05:00
Val Alexander
8711de4441
Merge pull request #1549 from openclaw/okcode/equalize-widths
Bump create-markdown to 2.0.1 and stabilize package detail tests
2026-04-06 12:18:57 -05:00
Luke
f4db0ee32b
fix: improve auth flow and webkit compatibility (#1555)
* fix: improve auth flow, skill filters, and webkit compatibility

* test: mock auth actions in settings route

* fix: remove stale skills toolbar props

* fix: address skills filter and webkit review feedback

* test: avoid monaco lazy import in skill detail test

* fix: recompute other skills category filter
2026-04-06 21:14:47 +10:00
Val Alexander
f1cf715b89
Merge main into okcode/equalize-widths
Resolve conflicts keeping our fixes:
- convex/skills.ts: retain empty-page hasMore guard for tag filter
- package-detail-route.test.tsx: retain widened MarkdownPreview mock type

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-05 22:38:52 -05:00
Val Alexander
57ed1f62a4
Add TanStack data table pagination and sorting
- Introduce reusable data table, column header, and pagination components
- Add TanStack Table dependency for sortable, paginated table UIs
2026-04-05 21:44:03 -05:00
Val Alexander
bde7529fbe
Widen the About page container
- Switch the About page from `narrow` to `wide` layout
- Align page width with the rest of the app
2026-04-05 21:44:03 -05:00
Val Alexander
5892718a23
Add capability tag filtering to skills browse
- Thread capability tags through skills search and public listing
- Add toolbar tag picker and preserve tag state in routing
- Cover tag-filtered search and browse query behavior with tests
2026-04-05 21:44:03 -05:00
Val Alexander
922eecdbd9
Stop empty public page scans from reporting more pages
- Clear `hasMore` and `nextCursor` when a scan hits the budget but returns no items
- Prevent the client IntersectionObserver from looping on empty auto-load responses
2026-04-05 20:57:34 -05:00
Val Alexander
e53a4433ae
Update src/__tests__/package-detail-route.test.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-05 20:36:35 -05:00
Val Alexander
68870a1dcc
Equalize stat widths across listings
- Align skill and soul metric chips to fixed widths
- Rebalance skills table columns for better summary space
- Tighten spacing in soul list rows
2026-04-05 20:26:51 -05:00
Val Alexander
c6aaf27886
Widen the About page container
- Switch the About page from `narrow` to `wide` layout
- Align page width with the rest of the app
2026-04-05 20:26:51 -05:00
Val Alexander
f71db92fc3
Add capability tag filtering to skills browse
- Thread capability tags through skills search and public listing
- Add toolbar tag picker and preserve tag state in routing
- Cover tag-filtered search and browse query behavior with tests
2026-04-05 20:26:51 -05:00
Val Alexander
07a1da4285
Merge pull request #1548 from openclaw/okcode/align-about-page-width
Add capability tag filtering and widen About page
2026-04-05 20:01:20 -05:00
Val Alexander
cb9d854e92
Widen the About page container
- Switch the About page from `narrow` to `wide` layout
- Align page width with the rest of the app
2026-04-05 19:54:55 -05:00
Val Alexander
74ec5191e0
Mock MarkdownPreview in package detail route tests
- Add a lightweight MarkdownPreview mock for the package detail route test
- Keep the test focused on route behavior instead of markdown rendering
2026-04-05 19:54:50 -05:00
Val Alexander
cf98936105
update create-markdown packages to 2.0.1 2026-04-05 19:54:50 -05:00
Val Alexander
dd9c42be28
Merge pull request #1546 from openclaw/okcode/tags-filter-update
Add capability tag filtering to skills browse
2026-04-05 19:35:24 -05:00
Val Alexander
e8a4e094e7
Add capability tag filtering to skills browse
- Thread capability tags through skills search and public listing
- Add toolbar tag picker and preserve tag state in routing
- Cover tag-filtered search and browse query behavior with tests
2026-04-05 19:31:54 -05:00
Val Alexander
068dd78a05
Mock MarkdownPreview in package detail route tests
- Add a lightweight MarkdownPreview mock for the package detail route test
- Keep the test focused on route behavior instead of markdown rendering
2026-04-05 18:44:04 -05:00
Val Alexander
2f5e17f56e
update create-markdown packages to 2.0.1 2026-04-05 17:50:45 -05:00
Val Alexander
5302bf8598
Merge pull request #1537 from openclaw/okcode/fix-rate-limit-errors
Handle public package API rate limits gracefully
2026-04-05 06:44:32 -05:00
Val Alexander
749c89e77b
Move local hook install to an explicit script
- Add `install:local-hooks` for manual hook setup
- Remove the automatic `prepare` hook install
2026-04-05 06:42:28 -05:00
Val Alexander
fd08d74071
Handle missing push base SHA in secret scan workflow
- Resolve scan range explicitly for pull_request and push events
- Fall back to the default branch when GitHub provides a zero or empty base SHA
2026-04-05 06:39:31 -05:00
Val Alexander
67c8e188e3
Fix rate-limit retry messaging
- share retry delay formatting for plugin pages
- handle secret scan diffs in push and PR workflows
2026-04-05 06:26:42 -05:00
Val Alexander
bad07ff95c
Improve plugin layout responsiveness
- tighten container padding on small screens
- make plugin install and filter controls stack cleanly
- reflow detail sections and metadata for mobile
2026-04-05 06:25:04 -05:00
Val Alexander
a7547eee51
Handle package API rate limits gracefully
- Surface retryable empty states for plugin catalog and detail pages
- Preserve Retry-After metadata in package API errors
- Add staged-secret scanning and auto-installed git hooks
2026-04-05 06:15:53 -05:00
Val Alexander
fddbb35b40
Merge pull request #1536 from openclaw/okcode/modern-ui-ux-refactor
Format codebase with consistent linting and import ordering
2026-04-05 06:02:11 -05:00
Val Alexander
ba313e2d0a
Refactor app UI around shared design system
- Add shared Radix/Tailwind UI primitives and empty/loading/error states
- Modernize route layouts, forms, tabs, nav, and markdown rendering
- Tighten package/auth API behavior and update tests for the new UX
2026-04-05 05:39:40 -05:00
Val Alexander
f6f31ac78d
Merge pull request #1527 from openclaw/okcode/modern-ui-ux-refactor
Refactor app UI with shared design system
2026-04-05 05:39:19 -05:00
Val Alexander
f32500b1b4
Update tests for refreshed skill UI copy and behavior
- Align assertions with skeleton loading states and tab roles
- Update skills index expectations for new copy and ordering
2026-04-05 05:30:37 -05:00
Val Alexander
5bc6402b5e
Add skill capability tag controls to management
- surface capability tags in the management view
- align tests with updated loading and toggle behavior
- relax package API credential assertion for env-dependent URLs
2026-04-05 05:24:00 -05:00
Val Alexander
9721461aca
Handle package fetch credentials by origin
- Send cookies only for same-origin package requests
- Avoid CORS failures on cross-origin Convex site URL fetches
2026-04-05 05:11:06 -05:00
Val Alexander
91f80c264a
Fix API URL routing for local and SSR requests
- Route local browser API calls to the Convex site URL
- Keep SSR loaders on the Convex site URL and remove the Vite proxy
2026-04-05 05:06:47 -05:00
Val Alexander
712302eb30
Widen skills and plugins layouts
- Expand skills and plugins pages to the wide container
- Rename settings test route file to match routing convention
2026-04-05 04:19:58 -05:00
Val Alexander
f044164bae
Prompt for hide and restore reasons
- Ask for a reason before soft-deleting or restoring a skill
- Pass the trimmed reason through to `setSoftDeleted` in both management views
2026-04-05 04:09:36 -05:00
Val Alexander
997da8857a
Add Vite proxy for Convex API requests
- Proxy `/api` to `VITE_CONVEX_SITE_URL` in dev
- Keep the proxy disabled when the env var is unset
2026-04-05 04:05:38 -05:00
Val Alexander
1a31e07332
Use GitHub sign-in for publish skill empty state
- Replace the sign-in link with an auth action
- Trigger GitHub login directly from the publish flow
2026-04-05 03:51:32 -05:00
Val Alexander
cb761a73a9
Wrap management links in button components
- Use `Button asChild` for view/manage actions in management screens
- Import `Button` in the dashboard route
2026-04-05 03:19:41 -05:00
Val Alexander
d4ad4ea489
Refactor links to use Button asChild
- Centralize link-styled actions through the Button component
- Surface publish validation errors in the form state as well as toasts
2026-04-05 03:17:18 -05:00
Val Alexander
d7c9126c4b
Show fallback messages from API error payloads
- Accept unknown error shapes in `ErrorFallback`
- Extract user-facing text from `error.message`, `error.error`, or string values
- Keep a default message when no useful detail is available
2026-04-05 02:51:09 -05:00
Val Alexander
5dc6720d9e
Refine markdown previews and tab/table styling
- Auto-link bare URLs in rendered markdown
- Add shared table primitives and markdown table styles
- Update tabs to use an underlined active state
2026-04-05 02:51:09 -05:00
Val Alexander
087f2c75b0
Add rich markdown preview for skill and plugin docs
- replace react-markdown rendering with a shared Shiki-powered preview
- update skill, soul, and plugin detail views to use the new component
- add markdown preview dependencies and adjust tests
2026-04-05 02:51:09 -05:00
Val Alexander
03c42fa947
Reset root error boundary on route changes
- Add a resetKey prop to `ErrorBoundary` so caught errors clear when navigation changes
- Wrap root children in a pathname-aware boundary to recover from route-level failures
2026-04-05 02:51:09 -05:00
Val Alexander
572110b1d5
Add confirmation dialogs for skill ownership actions
- Confirm rename and merge before submitting
- Remove unused form validation schema helpers
- Silence unhandled Promise.all lint in home route
2026-04-05 02:51:09 -05:00
Val Alexander
812276f1b7
Update src/lib/schemas.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-05 02:51:09 -05:00
Val Alexander
b9676f674c
Refactor app UI with shared design system
- Add reusable Radix-based UI primitives and layout helpers
- Refresh skill, soul, and dashboard pages with new empty/error/loading states
- Update dependencies for the modernized component stack
2026-04-05 02:51:09 -05:00
ImLukeF
32d601aab7
fix: show user-facing moderation override errors 2026-04-05 17:09:50 +10:00
ImLukeF
4aae925e76
fix: keep suspicious package scans from promoting to clean 2026-04-05 13:34:40 +10:00
ImLukeF
e41fd677bb
fix: use full skill description on detail page 2026-04-05 13:13:33 +10:00
ImLukeF
31520ef02a
feat: add manual skill capability tag controls 2026-04-05 12:02:41 +10:00
ImLukeF
e829cb1ae6
fix: use multiline moderation note dialog 2026-04-05 11:23:56 +10:00
ImLukeF
24cc59a425
fix: require moderation notes for hide and restore 2026-04-05 11:08:05 +10:00
Vincent Koc
522fa22026 fix: polish pass — tighter spacing, no empty sections, subtle skeletons
- Home page sections only render when data arrives (no skeleton flash)
- Removed SkeletonRows component from home page (unused)
- Skeleton bars simplified: static gray bars, no shimmer animation
- Skeleton row padding matches list item padding
- Hero padding tightened (48px top → 32px)
- Hero subtitle made concise ("20 skill bundles... Browse, install, publish.")
- Removed redundant explainer paragraph
- Browse results count shows em dash while loading (not "Loading...")
- .section class uses spacing tokens
- .section-title uses monospace font at --fs-lg
- .section-subtitle uses --fs-sm
- results-list removed fadeIn animation (subtle state changes only)
2026-04-04 19:26:44 +09:00
ImLukeF
7e76336ae4
fix: restore banned accounts in management 2026-04-04 20:50:55 +11:00
ImLukeF
e41d5d6314
fix: restore banned accounts in management 2026-04-04 20:49:24 +11:00
Vincent Koc
c62ab8bc96 fix: complete audit cleanup — monochrome purity, touch targets, a11y
Theming (P0):
- Removed all 78 [data-theme="dark"] override selectors (dark is now
  default, these were dead code with conflicting warm colors)
- Replaced 56 instances of rgba(255,107,74,x) warm coral with
  rgba(255,255,255,x) monochrome equivalents
- Replaced hard-coded warm hex colors (#c35640, #ff6b4a, etc.) with
  gray monochrome values
- CSS file reduced from ~6300 to 5909 lines

Touch targets (P1):
- Added min-height: 36px to .btn (was ~24px)
- Added min-height: 36px to .navbar-tab (was ~27px)
- Added min-height: 32px to .sidebar-option and .sidebar-checkbox
- Increased padding on buttons and tabs

Accessibility (P2/P3):
- Comprehensive prefers-reduced-motion: reduce rule — disables all
  animations AND transitions for users who prefer reduced motion
- Covers shimmer, fadeIn, fadeUp, and all CSS transitions
2026-04-04 17:15:28 +09:00
Vincent Koc
6df3a3b9ef feat: monochrome dark TUI aesthetic — complete design pivot
Complete visual redesign to a dark, monochrome, terminal-inspired
aesthetic inspired by Warp, modern TUI tools, and blueprint designs.

Color system:
- Default is now dark (#0a0a0a bg, #e0e0e0 ink, #141414 surface)
- All accent colors removed — monochrome only (white as accent)
- Borders use rgba(255,255,255,0.08) for subtle separation
- Light theme available as optional override via [data-theme="light"]

Typography:
- All fonts now IBM Plex Mono (display, body, code all monospace)
- Brand name is lowercase monospace
- Section titles are uppercase monospace with letter-spacing
- Tags and badges use monospace font

Geometry:
- All border-radius reduced to 1-2px (sharp TUI corners)
- No shadows anywhere (--shadow: none)
- No backdrop-filter blur on navbar
- Cards, buttons, inputs all have sharp edges

Components:
- Buttons: transparent bg with border, monospace text
- Primary buttons: white on black (inverted)
- Tags: border-only, no colored backgrounds
- Cards: dark surface with subtle border
- Brand mark: 24px square instead of 28px circle

Layout:
- Replaced category grid with simple quick links
- Removed all warm color references
- Home section titles are small uppercase labels
- Skill list item names use --ink (no accent color)
2026-04-04 16:08:12 +09:00
Vincent Koc
7452cf6f69 feat: white background, onboarding explainer, filter reset, transitions
- Switch light theme from warm beige (#f8f2ed) to neutral white (#fafafa)
  with neutral gray ink (#1a1a1a) and borders (rgba black)
- Switch dark theme from warm brown to neutral dark (#111111) with
  neutral gray borders (rgba white)
- Replace fake category grid (8 keyword-search cards) with curated
  quick links (Most starred, New this week, Browse plugins, Staff picks)
- Add "What are skills?" explainer paragraph below hero CTAs
- Add fadeIn animation on results list when data arrives
- Add "Clear" button in browse results toolbar when filters are active
- Tighten browse layout gap from 24px to 16px
2026-04-04 15:01:57 +09:00
ImLukeF
8ded548da2
fix: reject invalid capability tag filters 2026-04-04 15:41:26 +11:00
Luke
4ff2d1a0cd
feat: add skill capability tags (#1513)
* feat: add skill capability tags

* fix: address capability tag review feedback

* fix: tighten capability tag review heuristics
2026-04-04 15:17:37 +11:00
Vincent Koc
b4e8a26eb4 chore: remove accidentally committed skill/agent config files 2026-04-04 00:01:59 +09:00
Vincent Koc
7ff601bcb8 fix: resolve all lint errors
- Remove unused imports (v, getRuntimeEnv) from seedDemo.ts and SkillHeader.tsx
- Replace `as any` casts with proper Id<"publishers"> types in seedDemo.ts
- Prefix unused params with underscore (_clawdis, _osLabels, _nixSystems, _listDoneLoading)
- Remove unused convexSiteUrl variable from SkillHeader
2026-04-04 00:01:43 +09:00
Vincent Koc
70af109cb2 fix: address PR review feedback
- Import internalMutation from convex/functions (not _generated/server)
  to get trigger wrapping per CLAUDE.md rules
- Derive activeCategory from current search query so sidebar category
  selection shows correct visual/ARIA state
- Push moderationStatus filter server-side in repairGlobalStats to
  avoid full table scan
- Reset skillCount/pluginCount to 0 in useUnifiedSearch catch block
  to prevent stale badge values after search errors
2026-04-03 23:41:54 +09:00
Vincent Koc
1504708208 feat: marketplace UI overhaul — HuggingFace/npm-style discovery hub
Complete frontend rebuild of ClawHub into a marketplace-style discovery
hub inspired by HuggingFace and npm. No backend changes.

Navigation:
- Two-row header: brand + search bar + user actions on top, content
  type tabs (Skills, Plugins) with count badges below
- Inline search in navbar navigates to /search
- Mobile: search collapses behind icon, tabs scroll horizontally

Home page:
- Value-prop hero with Browse/Publish CTAs (no duplicate search)
- Trending section (8 skills by downloads)
- Recently updated section (8 skills by update time)
- Staff picks grid (6 highlighted skill cards)
- Browse by category grid (8 categories with Lucide icons)
- Skeleton loading rows while data fetches

Browse pages (Skills + Plugins):
- Full-width search bar above sidebar+results grid
- Left sidebar with sort options, categories, filter checkboxes
  (proper ARIA: fieldset/legend, role=radiogroup, aria-checked)
- List view uses compact SkillListItem rows (owner/name/summary/meta)
- Card view with hover border feedback
- View toggle (List/Cards)
- Better empty states with guidance text

Skill detail page:
- README tab as default (was Files)
- Two-column layout: tabs+comments on left, metadata sidebar on right
- Removed duplicate README from Files tab
- Removed duplicate Download button from header (kept in sidebar)
- Removed SkillInstallCard from header (license info in sidebar)
- Nix/config snippets moved inside two-column layout
- Friendly "No README available" instead of raw Convex errors

Unified search (/search):
- Real search results page (was redirect-only)
- Type tabs: All / Skills / Plugins with counts
- useUnifiedSearch hook fires skill search + plugin catalog in parallel
- Consistent SkillListItem rendering for results

Dashboard:
- Welcome state for new users with empty dashboard
- Simplified header copy

Profiles & Footer:
- Richer user profiles: large avatar, stat row, SkillListItem for
  published/starred skills
- Multi-column footer: Browse / Publish / Community / Platform

Design system:
- Spacing tokens: --space-1 (4px) through --space-8 (64px)
- Typography scale: --fs-xs through --fs-3xl (8 values, was 42)
- Radius tokens: --r-lg/md/sm/xs/pill (renamed from --radius-* to
  avoid Tailwind CSS v4 variable collision)
- Flat buttons (killed gradient, removed hover lift/shadow)
- Complete markdown styles: tables, blockquotes, lists, images, hr,
  heading hierarchy with h1/h2 bottom borders (npm-style)
- Removed decorative elements: body gradient backgrounds, card
  shadows, brand mark animation, category card glow

New components:
- SkillListItem — compact HF-style row
- BrowseSidebar — faceted filter sidebar with ARIA
- SkillMetadataSidebar — detail page right sidebar
- useUnifiedSearch — parallel search hook
- timeAgo — relative time formatter
- categories — static skill category taxonomy

Seed data:
- seedDemo.ts with 20 realistic skills, 5 publishers
- repairHighlightedBadges for skillBadges table
- repairGlobalStats for correct count

Test updates:
- Updated 5 test files for new text, class names, and prop changes
- All 122 test files, 915 tests passing
2026-04-03 22:58:30 +09:00
Onur
4af2bd50a7
Fix reusable package publish workflow (#1505)
* fix: harden reusable package publish workflow

* Fix reusable package publish CLI path

* Resolve reusable workflow source via OIDC

* Harden OIDC env lookup in publish workflow
2026-04-03 14:37:29 +02:00
Onur
88fe310ff1
feat: add deploy workflow targets (#1499)
Co-authored-by: Onur <onur@solmaz.io>
2026-04-03 09:43:41 +02:00
Onur
0a31b31f1b
docs: remove production approval step from deploy docs (#1497)
Co-authored-by: Onur <onur@solmaz.io>
2026-04-03 09:26:18 +02:00
Onur
079e390da6
fix: repair auth sign-in typecheck (#1493)
Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 22:22:59 +02:00
Peter Steinberger
15da02c70a
fix: improve banned account sign-in errors 2026-04-03 03:51:23 +09:00
Peter Steinberger
8b289618ba
fix: recover broken auth state for package and user flows 2026-04-03 03:51:18 +09:00
Onur
4ae0406948
refactor: rename internal clawdhub package path (#1490)
* refactor: rename internal clawdhub package path

* fix: update workflow paths after clawhub dir rename

* fix: preserve old tag npm release compatibility

---------

Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 17:50:33 +02:00
Onur
78d0a637fa
fix: make trusted publisher environment optional (#1489)
* fix: make trusted publisher environment optional

* fix: avoid env mismatch on unpinned trusted publishes

---------

Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 17:20:40 +02:00
Onur
65bc5d3335
ops: add guarded npm release workflow for clawhub cli (#1487)
* ops: add guarded npm release workflow for clawhub cli

* ops: limit clawhub cli release flow to stable npm tags

* docs: add clawhub cli release note to package readme

---------

Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 16:45:08 +02:00
Onur
8592272720
feat: add package trusted publishing via GitHub OIDC (#1461)
* feat: add package trusted publishing via GitHub OIDC

* fix: harden trusted publishing flow

* fix: finish trusted publishing rollout

* fix: harden trusted publish fallback

* ci: run package publish workflow from source

* fix: fall back when GitHub OIDC request fails

* test: fix plugin detail route mock

* fix: keep caller checkout pinned in package publish

* fix: restore auth query types

* ci: pin package publish workflow sources

* fix: tighten trusted package publish flow

* fix: add override reason for token fallback publishes

---------

Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 15:53:21 +02:00
Onur
15f5769cda
docs: document manual production release flow (#1485)
Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 15:24:45 +02:00
Onur
bb2a05e501
ops: make production deploy manual only (#1484)
Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 15:13:36 +02:00
Onur
af29ca7f9d
ops: gate production deploys behind environment approval (#1482)
* ops: gate production deploys behind environment approval

* ops: use existing production environment

* fix: retry transient deploy status polling failures

---------

Co-authored-by: Onur <onur@solmaz.io>
2026-04-02 15:05:57 +02:00
Luke
13caea7cfd
fix: resolve admin search exact matches via personal publisher handles (#1467)
* fix admin user search coverage

* support personal publisher handle admin search

* fix admin search fallback pagination

* fix package detail route test mock

* fix users test mock typing
2026-04-02 11:25:34 +11:00
Val Alexander
86259eef42
Merge pull request #1455 from openclaw/feature/mobile-nav-spacing
Improve mobile navigation and plugin detail layouts
2026-04-01 09:29:46 -05:00
Val Alexander
bf2ecab92e
Handle source repo values without URL scheme
- Normalize `verification.sourceRepo` to a full GitHub URL when needed
- Keep the rendered source link text consistent for external links
2026-04-01 09:28:56 -05:00
Val Alexander
70516bd0f5
Improve plugin copy fallback and capability labels
- Add textarea-based clipboard fallback for unsupported contexts
- Show copied and failed states in the copy button
- Add labels and formatting for plugin capability values
2026-04-01 09:24:00 -05:00
Val Alexander
66fca9a286
Update src/routes/plugins/$name.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-01 09:16:29 -05:00
Val Alexander
a03fe3bbd0
Handle clipboard copy failures gracefully
- Ignore clipboard write errors when copy is unavailable
- Keep the copy button from throwing in insecure or denied contexts
2026-04-01 09:12:23 -05:00
Val Alexander
d3d9298389
Refine skill header layout and surface version
- Rework the hero into a two-column grid with sidebar actions and metadata
- Move version, badges, and security scan content into clearer sections
- Add responsive spacing tweaks for mobile navigation
2026-04-01 09:07:46 -05:00
Val Alexander
2fd0aa01d5
Adjust mobile nav trigger spacing and add Node version pin
- Shrink the mobile nav trigger and add a larger hit area
- Add `.nvmrc` to pin Node 22 for local development
2026-04-01 09:03:13 -05:00
Val Alexander
5aad36dab0
Refine plugin navigation and detail layout
- Rework plugin list filters into a more compact mobile-friendly toolbar
- Expand plugin detail pages with install, capability, compatibility, and verification sections
- Tighten shared toggle and scan result spacing for the new layout
2026-04-01 08:47:56 -05:00
Val Alexander
1bab92313f
Tighten mobile navigation spacing and controls
- Reduce navbar padding and gaps on small screens
- Compact toggle group, theme buttons, and user trigger
- Hide extra dashboard summary text on narrow layouts
2026-04-01 08:23:55 -05:00
Val Alexander
f77993d614
Make dashboard list responsive on mobile
- Relax desktop grid column sizing
- Collapse dashboard list headers to three columns below 768px
2026-03-31 20:24:02 -05:00
Val Alexander
3a28344c53
Improve mobile wrapping and layout resilience
- Wrap long URLs, token strings, and changelogs to prevent overflow
- Make key controls and dashboard grids shrink more gracefully on small screens
- Relax textarea and file viewer sizing for better mobile usability
2026-03-31 19:42:30 -05:00
Val Alexander
ea49143fe0
Merge pull request #1439 from openclaw/okcode/mobile-friendly-review
fix: improve mobile touch targets and layout
2026-03-31 19:02:52 -05:00
Val Alexander
bd38fef5c5
Relax ghost button min-height 2026-03-31 18:58:12 -05:00
Val Alexander
b7a015523d
Make management controls responsive on small screens
- Let management inputs shrink to container width
- Remove mobile-only overrides that duplicated desktop layout rules
2026-03-31 17:30:19 -05:00
Valentina Alexander
d36f98faa1
fix: comprehensive mobile-friendly improvements
Bring all interactive elements to 44px WCAG touch target minimum,
fix diff editor horizontal scroll on mobile, stack skills table on
small phones, add 480px breakpoint for tiny devices, and tighten
spacing across dashboard/management/dialog components.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-31 16:18:04 -05:00
George Zhang
eeb0ecd932
fix: address post-merge vercel workflow/docs issues (#1386)
* fix: address post-merge vercel review issues

* fix: address remaining publish flow review findings

* docs: note explicit plugin compatibility requirements
2026-03-29 10:38:27 -07:00
Luke
51beceeb20
feat: add ability to delete version tags from skill detail page (#1380)
* feat: add ability to delete version tags from skill detail page

- Add deleteTags mutation to convex/skills.ts (protects 'latest' tag)
- Add delete button (×) on each tag in SkillHeader (visible to owner/moderator only)
- Wire up onTagDelete prop from SkillDetailPage to SkillHeader
- Add .tag-delete CSS styles

Closes: version tags accumulate across publishes with no way to remove them

* fix: address review feedback on deleteTags PR

- Add window.confirm() before deleting a tag (P2: missing confirmation)
- Skip db.patch when no tags are actually removed (P2: unnecessary write)
- Add test suite for deleteTags mutation covering:
  - Tag deletion with latest protection
  - No-op when only latest is targeted
  - No-op for nonexistent tags
  - Permission check for non-owner
  - Moderator access on other user's skill
  - Skill not found error

* fix: repair deleteTags test harness

* fix: satisfy deleteTags test typecheck

---------

Co-authored-by: Jeff <tjefferson518@gmail.com>
2026-03-29 22:10:38 +11:00
George Zhang
a8a687eba2
[codex] streamline plugin publish flow (#1373)
* fix: streamline plugin publish flow

* fix: stabilize github package publish e2e

* test: fix clawdhub ci portability

* test: make settings route assertion portable

* test: fix ci typechecks
2026-03-29 02:37:58 -07:00
Val Alexander
b9341202c7
Merge pull request #1349 from openclaw/okcode/fix-convex-tokens 2026-03-29 03:15:32 -05:00
George Zhang
d701c2a33b
Merge pull request #1362 from openclaw/codex/fix-handle-resolution
[codex] unify personal publisher handle resolution
2026-03-28 15:58:25 -07:00
George Zhang
79d17a91ed
fix: ignore inactive personal publisher handles 2026-03-28 15:34:31 -07:00
George Zhang
2f15202a68
fix: unify personal publisher handle resolution 2026-03-28 15:14:17 -07:00
Nimrod Gutman
342a2b1ca4
fix(skill-detail): stop owner canonical redirect loop (#1357) 2026-03-28 20:49:41 +03:00
ImLukeF
75915cd2b5
fix: improve Safari compatibility and auth callbacks 2026-03-28 23:59:23 +11:00
Val Alexander
5395d9d159
Update src/routes/settings.test.tsx
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-28 07:36:38 -05:00
Val Alexander
87c236e8ce
Skip token query until settings auth resolves
- avoid loading tokens before `me` is available
- add a test that verifies the query is skipped while auth is pending
2026-03-28 07:31:06 -05:00
ImLukeF
d8db9b99a2
fix: resolve user profiles via personal publisher handles 2026-03-28 22:37:25 +11:00
Luke
d8e1f0daa1
fix: align transferred skill publisher ownership (#1344) 2026-03-28 22:22:09 +11:00
ImLukeF
8a350d953c
fix: preserve exact skill slug search matches
Fixes #1322.

Credit to @Yyang100 for the original report and repro.
2026-03-28 16:27:26 +11:00
Peter Steinberger
3fb3150ee2
fix: allow package downloads while VT scan is pending 2026-03-28 02:54:10 +00:00
Peter Steinberger
7dbc0fc3bb
fix: unblock verified community package scans 2026-03-27 23:44:45 +00:00
Peter Steinberger
fc4f8644eb
refactor: centralize package scan state resolution 2026-03-27 02:10:01 +00:00
Peter Steinberger
dda6d55fbf
fix: unblock package vt scan fallback stalls 2026-03-27 02:06:03 +00:00
Vincent Koc
e014759b40
Merge pull request #1191 from openclaw/vincentkoc-code/fix-plugin-compatibility 2026-03-24 22:55:12 -07:00
Peter Steinberger
2186c41c48
fix: prioritize recent package vt backfills 2026-03-24 10:19:28 -07:00
Peter Steinberger
c30c182478
fix: harden package vt scan scheduling 2026-03-24 10:11:26 -07:00
Peter Steinberger
3080567964
perf: tighten package search candidate reads 2026-03-23 23:23:38 -07:00
Peter Steinberger
f304541561
perf: collapse package readme fallback requests 2026-03-23 23:23:36 -07:00
Peter Steinberger
91224ada13
perf: parallelize plugin detail reads 2026-03-23 23:04:36 -07:00
Peter Steinberger
d5fbaeef81
perf: unify plugin catalog reads 2026-03-23 23:03:01 -07:00
Peter Steinberger
3706018b72
docs: clarify abuse policy for leaked data and account farming 2026-03-23 19:42:20 -07:00
Peter Steinberger
62e616f635
fix: raise http api rate limits 2026-03-23 19:06:49 -07:00
Peter Steinberger
9013d324c8
fix: forward client ip headers for ssr package api 2026-03-23 19:03:12 -07:00
Peter Steinberger
c1363ec8d0
feat: add acceptable usage about page 2026-03-23 17:37:17 -07:00
Peter Steinberger
7e09196f92
feat: backfill static plugin scans 2026-03-23 15:58:02 -07:00
Peter Steinberger
807043b4b0
feat: surface plugin security scans 2026-03-23 15:42:59 -07:00
Vincent Koc
972fe35935 fix(plugins): improve package upload detection UX 2026-03-23 15:04:30 -07:00
Vincent Koc
16ee540f5d fix(vt): recover plugin package scan retries 2026-03-23 14:57:39 -07:00
Vincent Koc
f541882d55 feat(plugins): gate publish form behind upload 2026-03-23 14:49:42 -07:00
Vincent Koc
95bc156747 test(e2e): align header smoke nav labels 2026-03-23 14:46:06 -07:00
Vincent Koc
bf7422022f fix(auth): unblock safari cli github login 2026-03-23 14:28:42 -07:00
Vincent Koc
230e5b91f8 fix(publishing): rename publish routes and clarify import scope 2026-03-23 13:54:47 -07:00
Vincent Koc
70dcf21e37 fix(shell): streamline publisher inventory navigation 2026-03-23 13:45:42 -07:00
Vincent Koc
932a1fb30c fix(dashboard): improve publisher plugin card layout 2026-03-23 13:34:28 -07:00
Vincent Koc
44fe60b701 fix(plugins): keep public package routes public 2026-03-23 13:28:26 -07:00
Vincent Koc
1c057ca9b9 fix(dashboard): streamline publisher plugin workflow 2026-03-23 13:24:46 -07:00
Vincent Koc
9f793d1336 feat(dashboard): add publisher plugin status panel 2026-03-23 13:16:30 -07:00
Vincent Koc
aa9295bea9 fix(plugins): forward browser auth to package api 2026-03-23 13:05:41 -07:00
Vincent Koc
370eea4977 fix(plugins): recover owner package detail after auth 2026-03-23 12:53:35 -07:00
Vincent Koc
59e6819020 fix(publishers): remove leftover conflict marker 2026-03-23 12:37:53 -07:00
Vincent Koc
52b633f4e9 fix(publishers): skip digest scheduling without scheduler 2026-03-23 12:28:22 -07:00
Vincent Koc
6b1e6ca1c9
Merge pull request #1203 from openclaw/vincentkoc-code/fix-personal-publisher-pagination
fix: split publisher digest sync scheduling
2026-03-23 12:21:51 -07:00
Vincent Koc
6c112eccb7 chore: use typed internal digest refs 2026-03-23 12:20:59 -07:00
Vincent Koc
f9e9effcdd fix(publishers): split owner digest sync scheduling 2026-03-23 12:14:35 -07:00
Peter Steinberger
b5d2d0fefa
fix: heal missing personal publishers during publish 2026-03-23 10:43:17 -07:00
Vincent Koc
48d0fc91f3 fix(packages): keep plugin api separate from gateway version 2026-03-23 09:08:49 -07:00
Luke
fefb2340a8
feat: redesign plugins page and skills list view (#1166)
* feat: redesign plugins page and skills list view

Redesigned the plugins page with a cleaner toolbar (pill search,
toggle filter buttons) and simplified card layout. Added a proper
table-style list view for skills with skill name, version, summary,
and author avatar columns. Also polished the sort dropdown with a
chevron indicator, added card shadows for better separation, and
tightened up the theme toggle and sign-in button.

* feat: add publisher org ownership

* feat: migrate legacy publisher handles to orgs

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-03-23 03:44:17 -07:00
Peter Steinberger
22287558b9
feat: migrate legacy publisher handles to orgs 2026-03-23 03:39:28 -07:00
Peter Steinberger
f6ce8f9e1e
feat: add publisher org ownership 2026-03-23 03:27:08 -07:00
Peter Steinberger
b5cdee50a9
feat(packages): support shared publisher owners 2026-03-23 02:14:38 -07:00
Vincent Koc
b5fdee1c13 feat(plugins): clarify pending verification and add backfill 2026-03-23 02:07:05 -07:00
Vincent Koc
2530beaf51 fix(plugins): hide pending and malicious packages 2026-03-23 02:04:00 -07:00
Vincent Koc
c627b202f7 fix(plugins): add async malware scan parity 2026-03-23 02:01:04 -07:00
Vincent Koc
5cbeb54c8e fix(upload): cap published skill and plugin file sizes 2026-03-23 01:57:59 -07:00
Vincent Koc
eb8136b7fb fix(ui): keep query when filtering plugins 2026-03-23 01:47:31 -07:00
Peter Steinberger
a166c95eb0
fix: avoid skill scans on verified plugin browse 2026-03-23 01:14:40 -07:00
scoootscooob
86dc196e9b
fix: tolerate stale auth on public package reads (#1157) 2026-03-23 01:06:32 -07:00
Peter Steinberger
649c14a43f
feat: rename Packages to Plugins, filter out skills, add verified badge
- Rename /packages route to /plugins with /packages redirecting
- Nav links now say "Plugins" and point to /plugins
- Plugins page only shows code-plugin and bundle-plugin (no skills)
- "Official only" → "Verified only"; blue checkmark badge for verified publishers
- Compact card footer: "by author · v1.2.3" inline with verified badge
- Remove duplicate Skill/Skill tag bubbles
- Update tests to match new routes and behavior
2026-03-23 01:02:15 -07:00
Peter Steinberger
7cbf0434b0
ci: wait for vercel git deploy status 2026-03-23 00:58:21 -07:00
Peter Steinberger
76944f0e32
chore: prepare 0.9.0 release 2026-03-23 00:45:56 -07:00
Peter Steinberger
6ead1da08b
fix: use read rate limits for package files 2026-03-23 00:44:31 -07:00
Val Alexander
22413f4ff8
Revert "chore: update .gitignore and enhance Convex documentation in AGENTS.md and CLAUDE.md"
This reverts commit 3326c235da.
2026-03-22 20:17:58 -05:00
Val Alexander
3326c235da
chore: update .gitignore and enhance Convex documentation in AGENTS.md and CLAUDE.md 2026-03-22 20:03:04 -05:00
Peter Steinberger
2b97c13191
feat: reserve official handles and package archives 2026-03-22 12:48:47 -07:00
Peter Steinberger
17977c1248
fix: support nested package upload files 2026-03-22 11:14:58 -07:00
Peter Steinberger
aea0127663
fix: add package cleanup mutation 2026-03-22 11:09:31 -07:00
Peter Steinberger
9ea750852f
fix: stabilize package catalog search and contract verify 2026-03-22 11:04:20 -07:00
Peter Steinberger
801cc550ab
fix: narrow skill package catalog search 2026-03-22 11:03:17 -07:00
Peter Steinberger
522512c9d2
fix: use paginator for skill package catalog queries 2026-03-22 11:00:18 -07:00
Peter Steinberger
1ceb5d2bad
fix: tolerate prefixed convex function-spec output 2026-03-22 10:51:04 -07:00
Peter Steinberger
c7b3b2a30e
ci: restore bun in deploy web job 2026-03-22 10:49:21 -07:00
Peter Steinberger
f1e3629a99
ci: deploy web via vercel direct build 2026-03-22 10:47:49 -07:00
Peter Steinberger
931843aeac
ci: fail deploy preflight when secrets are missing 2026-03-22 10:43:36 -07:00
Peter Steinberger
1a56fc5427
test: isolate package api env fallback 2026-03-22 10:22:25 -07:00
Peter Steinberger
03a19f6497
fix: normalize skill and soul slugs 2026-03-22 10:20:48 -07:00
Peter Steinberger
f9087fc6e0
Merge pull request #1093 from openclaw/plugin
feat: add native package and plugin registry flows
2026-03-22 10:18:11 -07:00
Peter Steinberger
5429c28667
Merge remote-tracking branch 'origin/main' into plugin
# Conflicts:
#	bun.lock
#	package.json
2026-03-22 10:15:03 -07:00
Peter Steinberger
c234009b7c
fix: expose private packages to owners 2026-03-22 10:13:16 -07:00
Peter Steinberger
962171a590 feat: unify skills into package catalog 2026-03-22 16:04:56 +00:00
Peter Steinberger
852b54a722 fix: preserve package publish outputs and site fallbacks 2026-03-22 15:33:23 +00:00
Peter Steinberger
dff83a52bf fix: tighten package auth boundaries 2026-03-22 07:27:25 +00:00
Peter Steinberger
ec8601c0eb fix: tighten package fallback and readme errors 2026-03-22 04:09:46 +00:00
Peter Steinberger
ee6a5ab037 test: increase package API coverage 2026-03-22 03:56:42 +00:00
Peter Steinberger
4814ce39d6 fix: remove dead skill family from package catalog 2026-03-22 03:54:11 +00:00
Peter Steinberger
ccace09242 fix: preserve package latest metadata and rename publish route 2026-03-22 03:34:42 +00:00
Peter Steinberger
8f9c9ffba2 fix: tighten package filter indexes and private UI states 2026-03-22 03:20:29 +00:00
Peter Steinberger
7460c084a5 test: cover package publish route plugin uploads 2026-03-22 02:53:30 +00:00
Peter Steinberger
e7ef691013 fix: filter ignored package uploads and block skill publishes 2026-03-22 02:50:05 +00:00
Peter Steinberger
da76bf7554
test: ignore generated vercel output in vitest 2026-03-21 19:35:46 -07:00
Peter Steinberger
3959a7ea85 fix: preserve package cursor tails and latest tags 2026-03-22 02:29:59 +00:00
Peter Steinberger
08da7eb477
fix: remove runtime devtools from app shell 2026-03-21 19:27:43 -07:00
Peter Steinberger
958a37fa6b
fix: respect runtime env in prod SSR 2026-03-21 19:25:11 -07:00
Peter Steinberger
91d963cf13 fix: repoint packages to the right fallback release 2026-03-22 02:23:09 +00:00
Peter Steinberger
31191b7771
fix: lazy-load devtools on client 2026-03-21 19:21:01 -07:00
Peter Steinberger
371367f01d
chore: update dependencies 2026-03-21 19:13:19 -07:00
Peter Steinberger
2ace8f9ab4 fix: freeze code plugin runtime ids 2026-03-22 02:11:59 +00:00
Peter Steinberger
914fa895c5
build: add China mirror redirect entrypoint 2026-03-21 19:08:17 -07:00
Peter Steinberger
93e398e7a5 fix: harden package catalog scans and downloads 2026-03-22 02:06:28 +00:00
Peter Steinberger
043f029f2c fix: resync package latest pointers on release hides 2026-03-22 01:56:54 +00:00
Peter Steinberger
de90f80d72 fix: tighten package summary and digest sync 2026-03-22 01:29:34 +00:00
Peter Steinberger
cc9de82034 fix: tighten package pagination and version indexing 2026-03-22 01:17:38 +00:00
Peter Steinberger
05924a36e1 fix: keep private package flows on app origin 2026-03-22 00:42:26 +00:00
Peter Steinberger
ed9dcc39b7 fix: restore package auth and download metadata 2026-03-22 00:20:36 +00:00
Peter Steinberger
1511aa2082 fix: tighten package latest visibility and filter indexes 2026-03-21 23:09:06 +00:00
Peter Steinberger
f88f3ef925 fix: tighten package release serving and uploads 2026-03-21 22:55:23 +00:00
Peter Steinberger
adce8ac1b5 fix: preserve skill family filters across package listings 2026-03-21 22:40:39 +00:00
Peter Steinberger
ddafee7e0e fix: address final package registry review issues 2026-03-21 17:38:07 +00:00
Peter Steinberger
877f37d052 fix: resolve package publish and search regressions 2026-03-21 17:28:30 +00:00
Peter Steinberger
5210966a19 test: add package registry regression coverage 2026-03-20 18:30:50 +00:00
Peter Steinberger
4ccfb9d3f5 fix: address package registry review regressions 2026-03-20 18:27:52 +00:00
Peter Steinberger
fc7d4dadca feat: add native package and plugin registry flows 2026-03-20 18:00:15 +00:00
Peter Steinberger
fb86e790b8
docs: add plugin hosting plan 2026-03-20 10:29:47 -07:00
Val Alexander
deb592d4ce
docs: update repository guidelines and improve formatting across multiple files
- Enhanced AGENTS.md with clearer project structure and development commands.
- Updated CHANGELOG.md to reflect recent fixes and additions.
- Improved formatting in CONTRIBUTING.md for better readability.
- Adjusted package.json and configuration files for consistent command structure.
- Refined README.md and VISION.md for clarity and organization.
- Standardized code formatting in various TypeScript files for consistency.

These changes aim to enhance documentation clarity and maintainability across the repository.
2026-03-18 21:56:01 -05:00
Peter Steinberger
8030c554e8
test: fix ci for download regression coverage 2026-03-17 09:59:58 -07:00
Peter Steinberger
472e1e875d
test: harden download counter regression coverage 2026-03-17 09:56:49 -07:00
Peter Steinberger
ba9cdde703
fix: internalize download counter mutations 2026-03-17 09:38:11 -07:00
magicseth
8d5a64b599
Merge pull request #965 from sethconvex/fix/github-backup-retry-on-conflict
fix: retry GitHub backup push on fast-forward conflict
2026-03-17 00:07:44 -07:00
Seth Raphael
351dfd48c3 fix: retry GitHub backup push on fast-forward conflict
When a publish-time backup and the cron backup push concurrently, the
second push fails with "not a fast forward" because the branch moved.

Split backupSkillToGitHub into two phases:
1. Create blobs (storage downloads) — done once
2. Fetch ref, build tree, commit, push — retried up to 3x on conflict

Same retry applied to deleteGitHubSkillBackup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:16:22 -07:00
magicseth
0e2a7e1bba
Merge pull request #956 from sethconvex/feat/listpublic-v4-staged-release
feat: switch /skills and homepage to listPublicPageV4
2026-03-16 18:11:10 -07:00
Seth Raphael
54c9d57a22 fix: add authTables to @convex-dev/auth/server mocks in all test files
The schema import in skills.ts (needed for getPage) transitively pulls
in authTables from @convex-dev/auth/server. All test files that import
from skills.ts need this export in their mock.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 18:07:34 -07:00
Seth Raphael
343836a026 fix: use skillBadges index for highlightedOnly filter in V4
Instead of scanning 500+ rows of the sort index and filtering for
highlighted skills in JS (which fails when highlighted skills are
sparse among 25K+ rows), query the skillBadges table via by_kind_at
index to find highlighted skill IDs directly, then look up their
digests. Also simplifies the non-highlighted path to a single getPage
call since the multi-round loop was only needed for highlighted.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 17:59:58 -07:00
Seth Raphael
ac86be7e14 chore: remove useV4 flag — V4 is now the only code path
Remove the V3 fallback branch and useV4 parameter from
useSkillsBrowseModel since V4 is verified and the only path used.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 17:38:07 -07:00
Seth Raphael
6ef8843f1c feat: switch /skills and homepage to V4, remove test routes
- /skills browse page now uses listPublicPageV4 via useV4 flag
- Homepage popular skills section uses listPublicPageV4
- Remove /skillsv4 and /test-v4 temporary test routes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 17:33:24 -07:00
magicseth
33a7cec1af
Merge pull request #955 from sethconvex/feat/listpublic-v4-staged-release
feat: V4 staged release — /skillsv4 test route
2026-03-16 17:30:54 -07:00
Seth Raphael
81e734b1fc fix: guard against hasMore=true with nextCursor=null in V4 path
When V4 returns hasMore=true but nextCursor=null, the next load-more
call would pass cursor=null, triggering the replace branch instead of
append. Treat this edge case as 'done' to prevent silent list reset.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 17:30:12 -07:00
Seth Raphael
b3dd6c1219 feat: V4 staged release — fix getPage schema, add /skillsv4 test route
Root cause of V4 returning empty in production: `getPage()` ignores the
`indexFields` property at runtime and always calls `getIndexFields(table,
index, schema)`. Without `schema`, it threw "schema is required" silently.

Fixes:
- Pass `schema` instead of `indexFields` to `getPage()`
- Use `absoluteMaxRows` instead of `targetMaxRows` (ignored when
  `endIndexKey` is provided)
- Remove unused `DIGEST_INDEX_FIELDS` constant

Staged release:
- Add `/skillsv4` route (same UI as `/skills` but using V4 backend)
- Add `/test-v4` debug page for raw V4 API testing
- Add `useV4` flag to `useSkillsBrowseModel` hook
- Keep `/skills` on V3 until V4 is verified in production

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 17:23:47 -07:00
magicseth
ecf71e8664
Merge pull request #953 from sethconvex/feat/listpublic-v4-backend-only
feat: add listPublicPageV4 backend query (frontend stays on V3)
2026-03-16 16:58:45 -07:00
Seth Raphael
1d170cf634 fix: revert homepage to V3 — V4 failing in production
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:57:47 -07:00
magicseth
c19917e9a4
Merge pull request #950 from sethconvex/feat/listpublic-v4-deterministic-cursors
feat: listPublicPageV4 with cacheable deterministic cursors
2026-03-16 16:55:55 -07:00
Seth Raphael
9922c22291 fix: use indexFields instead of schema import to avoid auth dep in tests
- Replace `import schema from './schema'` with inline DIGEST_INDEX_FIELDS
  lookup, avoiding @convex-dev/auth/server transitive dependency in tests
- Gut V1 test to match gutted handler (single stub verification)
- Fix load-more test to use V4 response shape

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:51:16 -07:00
Seth Raphael
f6c0735238 fix: update tests for V4 response shape and gutted V2
- Replace V2 test suite with single stub verification (no DB reads)
- Update skills-index tests: paginationOpts → cursor/numItems,
  isDone/continueCursor → hasMore/nextCursor
- Update default convexHttpMock to return V4 shape

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:47:57 -07:00
Seth Raphael
1d2a822f2a fix: lint — remove dead code and fix variable shadowing
- Remove sortToIndex and getTrendingEntries (only used by gutted V1)
- Remove unused leaderboard imports
- Rename shadowed 'v' parameter to 'val' in encode/decodeIndexKey

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:45:03 -07:00
Seth Raphael
92e640a46b fix: address codex review — cursor safety, pagination stall, stale doc
- Use tagged object encoding for undefined index key values instead of
  plain string sentinel to avoid collisions
- Add try/catch in decodeIndexKey, treat malformed cursors as first page
- When highlightedOnly filters out all fetched rows, advance nextCursor
  to last fetched position instead of returning null (prevents restart loop)
- Update V3 JSDoc to reflect its current role

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:39:57 -07:00
Seth Raphael
618cb3141c fix: bound getPage with equality prefix to exclude soft-deleted rows
getPage walks the entire index unless bounded. Without constraining
startIndexKey/endIndexKey to the equality prefix ([undefined] for base,
[undefined, false] for nonsuspicious), desc order returns soft-deleted
items first, producing empty pages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:36:45 -07:00
Seth Raphael
49d0e0246a feat: add listPublicPageV4 with cacheable deterministic cursors
Use convex-helpers getPage() instead of .paginate() so that page cursors
are derived from actual index field values. Two users requesting the same
page now produce identical query args, enabling shared query caching.

- Add listPublicPageV4 with IndexKey-based cursor encoding
- Gut listPublicPage (V1) and listPublicPageV2 to return empty results
- Keep listPublicPageV3 intact for any remaining subscribers
- Switch frontend browse model and homepage to V4

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 16:24:01 -07:00
magicseth
0a4e542ba9
Merge pull request #949 from sethconvex/chore/listpublic-v3-stale-tab-check
chore: add listPublicPageV3 to identify stale-tab websocket traffic
2026-03-16 15:53:49 -07:00
Seth Raphael
0721c57fae chore: add listPublicPageV3 to distinguish new clients from stale tabs
Duplicate of listPublicPageV2 as a separate Convex function. Frontend
switched to V3 so any remaining V2 calls in the dashboard are from
stale browser tabs with old bundles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 15:52:02 -07:00
magicseth
57cec34d53
Merge pull request #948 from sethconvex/fix/listpublic-v2-test-typecheck
fix: typecheck error in listPublicPageV2 test
2026-03-16 15:13:23 -07:00
Seth Raphael
c20f836d71 fix: typecheck error in listPublicPageV2 test
Cast page item to Record to access latestVersion property that isn't
on the narrow return type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 15:12:20 -07:00
magicseth
390f52ade0
Merge pull request #947 from sethconvex/fix/listpublic-v2-test-update
test: update listPublicPageV2 tests for digest-only behavior
2026-03-16 15:07:12 -07:00
Seth Raphael
1f30223b54 test: update listPublicPageV2 tests to match digest-only behavior
Remove fallback expectations — pre-backfill rows without owner fields
are now skipped, and missing latestVersionSummary returns null instead
of fetching from skillVersions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 15:06:33 -07:00
magicseth
b7d273daea
Merge pull request #945 from sethconvex/perf/listpublic-v2-digest-only
perf: listPublicPageV2 reads only from digest, no extra table joins
2026-03-16 14:37:06 -07:00
Seth Raphael
8226e418f4 refactor: use filteredMap to avoid redundant digestToHydratableSkill call
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 14:22:51 -07:00
Seth Raphael
48849dec89 perf: build listPublicPageV2 response directly from digest, no extra table reads
listPublicPageV2 was calling buildPublicSkillEntries which fell back to
ctx.db.get() for owners and versions, adding skills/users/skillVersions
to the query's read set. Now that digest rows have owner fields and
latestVersionSummary backfilled, we can construct the full response from
skillSearchDigest alone — writes to other tables no longer bust the cache.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 14:09:38 -07:00
Peter Steinberger
864b24fe03
test: add prod http smoke coverage 2026-03-15 22:22:22 -07:00
Peter Steinberger
26e6a435f2
test: add ssr and og regression coverage 2026-03-15 20:49:18 -07:00
Peter Steinberger
f8104fd759
fix: package og assets for server rendering 2026-03-15 20:36:12 -07:00
Peter Steinberger
ef2cd7a1f4
fix: restore ssr-compatible web stack 2026-03-15 20:30:29 -07:00
Peter Steinberger
ee2d06e622
feat: server render public skill pages 2026-03-15 20:12:53 -07:00
Nimrod Gutman
69e68e696e
fix(web): default compare diff to inline on narrow screens (#898)
* fix: isolate diff editor rendering and styling

* fix: scrollbar style

* fix(web): default compare diff to inline on narrow screens

---------

Co-authored-by: geoffrey <1377499035@qq.com>
2026-03-15 18:53:57 +02:00
magicseth
64b32d88c0
Merge pull request #872 from sethconvex/perf/search-skip-users-and-compound-indexes
perf: skip users table in search, use compound indexes in listPublicPageV2
2026-03-14 09:37:19 -07:00
Seth Raphael
a0d0ec0e1e test: update assertions for compound index and digest owner changes
- search.test: expect users lookup NOT called when digest has owner data
- listPublicPageV2.test: expect by_nonsuspicious_* indexes when nonSuspiciousOnly

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:34:13 -07:00
Seth Raphael
da578cfcad fix: handle null-owner digest fallback and safe compound index migration
- P2: When digestToOwnerInfo returns { owner: null } (deactivated/deleted
  user), fall back to live users table lookup instead of dropping the skill
  from results. Applies to hydrateResults, lexicalFallbackSkills, and
  buildPublicSkillEntries.

- P1: If compound index returns zero results on the first page (isSuspicious
  not yet backfilled), fall back to base index with JS filtering so the
  homepage isn't empty during migration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:28:02 -07:00
Seth Raphael
14e4ab59cb docs: add CLAUDE.md with Convex performance rules
Encodes the patterns learned from bandwidth optimization so AI coding
assistants get them right from the start — digest owner fields over
users table reads, compound indexes over JS filtering, one-shot fetches
for public pages, change detection in triggers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:14:29 -07:00
Seth Raphael
7c48828d69 perf: skip users table in search, use compound indexes in listPublicPageV2
Three bandwidth fixes:

1. search.hydrateResults / lexicalFallbackSkills: use digestToOwnerInfo()
   to resolve owner data from the digest instead of ctx.db.get(ownerUserId)
   on the users table. Eliminates users from the read set for ~99% of
   search calls (8+ GB in 72h).

2. skills.listPublicPageV2: use compound by_nonsuspicious_* indexes when
   nonSuspiciousOnly is true, filtering isSuspicious at the DB level
   instead of scanning and discarding in JS (30+ GB in 72h).

3. maintenance.backfillDigestIsSuspicious: targeted backfill that sets
   isSuspicious on digest rows where it's undefined, using the digest's
   own moderationFlags/moderationReason. Must run before compound indexes
   take effect.

Deploy sequence:
  1. Deploy functions
  2. npx convex run maintenance:backfillDigestIsSuspicious --prod
  3. Compound indexes work immediately for backfilled rows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 09:13:40 -07:00
Nimrod Gutman
674ca01a3b
fix(cli): validate forced install version before rm (#864)
* test(cli): add tests for install --force rm ordering

Add three tests to verify that --force install does not delete the local
skill directory before pre-download checks have passed:

- rm not called when skill is malware-blocked
- rm not called when API fetch fails (skill not found)
- rm called before download when all checks pass (happy path)

* fix(cli): move rm after checks in install --force

Previously, install --force deleted the local skill directory before
fetching metadata or running moderation checks. If any check failed
(skill deleted, malware-blocked, not found), the local copy was lost.

This is inconsistent with cmdUpdate, which already checks before
deleting. Move rm to after all checks pass, just before download.

This does not alter the meaning of --force; it narrows the window in
which data is removed before the command has confirmed the replacement
is viable.

* fix(cli): validate forced install version before rm

---------

Co-authored-by: Jonathan Deamer <202770+jonathandeamer@users.noreply.github.com>
2026-03-14 13:33:33 +02:00
Wangnov
f0b6335966
fix: default skills search to relevance (#802)
* fix: default skills search to relevance

* test: cover explicit search sort guard

---------

Co-authored-by: wangnov <1694546283@qq.com>
2026-03-14 13:10:13 +02:00
Vincent Koc
165f132613
fix: guard direct version file reads (#797)
* fix: guard soul version file reads

* test: cover version file access guards

* fix(convex): tighten version file access guards

---------

Co-authored-by: Nimrod Gutman <nimrod.gutman@gmail.com>
2026-03-14 12:46:16 +02:00
Vincent Koc
da0448923b
fix: sanitize public soul version queries (#796)
* fix: sanitize public soul version queries

* test: cover public soul version sanitization

* fix(api): preserve soul file downloads after sanitization

---------

Co-authored-by: Nimrod Gutman <nimrod.gutman@gmail.com>
2026-03-14 12:14:27 +02:00
Vincent Koc
cca8b4421b
fix: sanitize public skill version queries 2026-03-14 11:42:56 +02:00
magicseth
07df1bcec3
Merge pull request #844 from sethconvex/perf/oneshot-browse-page
perf: replace reactive browse/home queries with one-shot fetches
2026-03-13 22:09:34 -07:00
Seth Raphael
0c5dc63bf5 fix: mock convexHttp in skills-route-default-sort test for CI
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 22:08:02 -07:00
Seth Raphael
a9699684ce merge: resolve conflict with origin/main in index.tsx
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 22:06:16 -07:00
Seth Raphael
1bc3ba98ed fix: allow retry on load-more fetch failure instead of hiding control
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 20:58:18 -07:00
Seth Raphael
08284d8f3d test: update tests for one-shot fetch and preResolved owner short-circuit
- Replace usePaginatedQuery mocks with convexHttp.query mocks in
  browse page tests
- Update load-more test to use convexHttp instead of loadMorePaginated
- Update backend tests to reflect new behavior: getOwnerInfo skips
  db.get when digest has pre-resolved owner data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 20:53:30 -07:00
Seth Raphael
1848fcb819 fix: add error handling and unmount cleanup to home page fetches
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 20:48:21 -07:00
Seth Raphael
6224c05e82 perf: replace reactive browse/home queries with one-shot HTTP fetches
listPublicPageV2 is the #1 DB bandwidth consumer (31GB+ spikes) because
usePaginatedQuery creates reactive subscriptions. Any write to
skillSearchDigest invalidates all active subscribers simultaneously —
a thundering herd. This replaces reactive subscriptions with one-shot
ConvexHttpClient.query() calls on both the /skills browse page and the
home page, eliminating the reactive read set entirely.

Also short-circuits getOwnerInfo() to return pre-resolved owner data
from the digest before hitting ctx.db.get(ownerUserId), removing the
users table from the reactive read set for listPublicPageV2.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 20:28:23 -07:00
magicseth
68fc915444
Merge pull request #843 from sethconvex/perf/home-page-oneshot-queries
perf: replace reactive subscriptions with one-shot fetches on home page
2026-03-13 17:44:47 -07:00
Seth Raphael
7cd68cc3b5 fix: add .catch() handlers to one-shot home page queries
Prevents unhandled promise rejections on transient network/backend
failures. Empty state is an acceptable fallback for the home page.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 17:43:10 -07:00
Seth Raphael
e37ccd95d1 perf: replace reactive subscriptions with one-shot fetches on home page
The home page used useQuery for highlighted and popular skills, creating
live reactive subscriptions that re-executed on every skillSearchDigest
write (crons, triggers). Since the home page doesn't need live updates,
switch to one-shot convex.query() fetches on mount to eliminate unnecessary
reactive invalidation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 17:33:52 -07:00
magicseth
871e89e0b7
Merge pull request #841 from sethconvex/feat/backfill-digest-version-summary
feat: backfill latestVersionSummary into skillSearchDigest rows
2026-03-13 16:33:43 -07:00
Seth Raphael
78077028fe feat: add backfillDigestVersionSummary to patch latestVersionSummary into digest rows
Existing skillSearchDigest rows created before the latestVersionSummary
denormalization lack the field, causing listPublicPageV2 to fall back to
reading full skillVersions docs (~6KB each, 14MB per call).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 16:20:22 -07:00
magicseth
5d0f5155cf
Merge pull request #840 from sethconvex/fix/stable-pagination-global-stats
fix: use stable _creationTime pagination in countPublicDigestPageInternal
2026-03-13 15:48:44 -07:00
Seth Raphael
946ca4febd fix: use stable _creationTime pagination in countPublicDigestPageInternal
The by_active_updated index orders by updatedAt which is mutable —
rows can shift during a paginated scan causing double-counting or
skipping. Default _creationTime ordering is immutable and stable.
isPublicSkillDoc already filters softDeletedAt in JS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:48:07 -07:00
magicseth
593e05f72e
Merge pull request #837 from sethconvex/perf/aggregate-global-stats
perf: split global stats recount into batched action
2026-03-13 15:39:17 -07:00
Seth Raphael
faaedb17ea perf: skip digest write when no fields changed to prevent reactive thundering herd
The skills trigger unconditionally wrote to skillSearchDigest on every skill
mutation, even when only stat fields were updated with identical values.
This caused every cron that patches skills (stat sync, backfill) to invalidate
all active listPublicPageV2 subscriptions, triggering massive re-execution
storms (55 GB bandwidth spikes).

Now upsertSkillSearchDigest compares new fields against the existing row and
skips the write when nothing changed. This prevents unnecessary reactive
invalidation while still keeping the digest current for real changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:24:07 -07:00
Peter Steinberger
0ab23862a9
feat: harden skill moderation and canonicalization 2026-03-13 21:35:39 +00:00
Seth Raphael
7b03a44a9c docs: clarify deprecated mutation is a manual fallback only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:37:59 -07:00
Seth Raphael
50715d6467 perf: split global stats recount into batched action to avoid bytes-read limit
The daily updateGlobalStatsInternal cron read all ~19K skillSearchDigest docs
(17.8 MB) in a single mutation, exceeding the Convex bytes-read limit.

Switch to an action-based approach that pages through the table in ~1000-doc
queries (each ~900 KB), then writes the result in a separate mutation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 13:37:59 -07:00
magicseth
0bb513c19a
Merge pull request #836 from sethconvex/perf/denormalize-owner-into-digest
feat: add start/stop/status controls to digest owner backfill
2026-03-13 13:18:55 -07:00
Peter Steinberger
536c252310
fix: hide banned-owner skills from public surfaces 2026-03-13 20:07:06 +00:00
Seth Raphael
d1b2ca0a29 feat: add start/stop/status controls to digest owner backfill
Add delayMs param, stop flag via skillStatBackfillState, and status
query so backfill speed can be adjusted without redeploying.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 12:47:16 -07:00
magicseth
9730ba65bc
Merge pull request #831 from sethconvex/perf/denormalize-owner-into-digest
perf: denormalize owner fields into skillSearchDigest
2026-03-13 12:25:20 -07:00
DangerouslyShip
247580c456 fix: preserve owner profile data for handle-less visible users
digestToOwnerInfo now checks for profile data (name/displayName/image)
in addition to handle when deciding whether to return an owner object.
Handle-less visible users get their full profile; deactivated users
(no handle AND no profile data) correctly get owner: null.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:53:39 -07:00
DangerouslyShip
114cca2049 fix: use empty string sentinel for handle-less owners in digest
Write ownerHandle: '' (not undefined) for visible users without a
handle and for deactivated users, so digestToOwnerInfo can distinguish
"not backfilled" (undefined → fallback to DB) from "backfilled but
no handle" ('' → use userId fallback, skip DB read).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:08:53 -07:00
DangerouslyShip
928a65a29c fix: gate digest owner fields on deletedAt/deactivatedAt
Prevent baking deactivated/deleted user info into the digest.
The trigger and backfill now write undefined for owner fields
when the owner is not visible, matching the live query path.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:52:24 -07:00
DangerouslyShip
1a88a06948 perf: denormalize owner fields into skillSearchDigest to eliminate users reads
listPublicPageV2 (419 GB) and search.hydrateResults (1.76 TB) both read
full users docs for every unique owner. Denormalize ownerHandle, ownerName,
ownerDisplayName, ownerImage into the digest so query paths skip ctx.db.get
entirely. One extra read per skill mutation (rare) vs eliminating reads on
every query (very frequent).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:41:38 -07:00
Peter Steinberger
44638b73d4
chore(release): 0.8.0 2026-03-13 13:33:03 +00:00
Peter Steinberger
db15896a6b
ci: pin setup-bun to Node 24 action commit 2026-03-13 13:22:20 +00:00
Peter Steinberger
bf160445dd
ci: opt GitHub actions into Node 24 2026-03-13 13:19:08 +00:00
Peter Steinberger
d2956bc64b
test: fix lint and coverage compatibility after upgrades 2026-03-13 13:11:05 +00:00
Peter Steinberger
2486159e96
build(deps): update workspace dependencies and workflows 2026-03-13 13:11:05 +00:00
Nimrod Gutman
b461dcb2bd
fix(convex): avoid trending leaderboard read limit (#821)
* fix(convex): avoid trending leaderboard read limit

* test(convex): exercise trending cold start path

* docs: update convex query guidance

* fix(convex): preserve shim return contract
2026-03-13 14:57:03 +02:00
Peter Steinberger
1f474b68ce
docs(changelog): reconstruct 0.7.0 release notes 2026-03-13 12:39:20 +00:00
Peter Steinberger
e6ec1ec060
fix(ci): harden release verification 2026-03-13 12:39:20 +00:00
Peter Steinberger
1b038d55a2
fix(clawhub): inline packaged license exports 2026-03-13 12:39:04 +00:00
Nimrod Gutman
f4fd8fe6f1 fix(convex): fix leaderboard test typecheck 2026-03-13 14:25:51 +02:00
Nimrod Gutman
e9f731b57f
feat(api): add scan security verification endpoint and non-suspicious filters (#820)
* feat(api): add scan verification endpoint and non-suspicious filters

* fix(api): dedupe bool query parsing and backfill trending non-suspicious

* fix(api): preserve llm dimension warnings in security snapshot

* fix(api): clarify scan result semantics

* fix(api): clarify scan version context

* docs(api): clarify filtered pagination behavior

* fix(api): restore safe non-suspicious behavior

---------

Co-authored-by: VAC <vac@vacs-mac-mini.localdomain>
Co-authored-by: VACInc <3279061+VACInc@users.noreply.github.com>
2026-03-13 14:17:35 +02:00
Nimrod Gutman
e2fb0355df fix(convex): align public skills test typing 2026-03-13 10:47:15 +02:00
Nimrod Gutman
4057431f63
fix(api): allow legacy cli publish payloads (#815) 2026-03-13 10:41:29 +02:00
Vincent Koc
0dcfa0be81
Merge pull request #793 from neeravmakwana/fix/public-skill-owner-leak
fix: sanitize public skill owners
2026-03-12 20:36:54 -04:00
Neerav Makwana
3348e87cb7 fix: sanitize public skill owners
Use the public user serializer for `skills.getBySlug` so Convex query responses no longer expose private account metadata like email addresses from public endpoints.

Made-with: Cursor
2026-03-12 20:19:13 -04:00
DangerouslyShip
b32a499774 test: add coverage for fallback when latestVersionSummary is absent
Verifies that old digest rows without latestVersionSummary correctly
fall back to ctx.db.get(latestVersionId) for version data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:28:44 -07:00
DangerouslyShip
576588b70d perf: denormalize latestVersionSummary into skillSearchDigest
Eliminates ~9MB of skillVersions reads per listPublicPageV2 call by
copying latestVersionSummary from skills into the digest via the
existing trigger. Old rows without the field fall back to fetching
the full version doc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 08:14:35 -07:00
Trek90s
d725a381d7
fix: allow ownership healing when previous owner is deleted/banned (#689)
* fix: allow ownership healing when previous owner is deleted/banned

The GitHub identity check in `publishOrUpdateSkillInternal` and
`checkSlugAvailability` was unreachable when the skill owner's account
was deleted or deactivated. This created a permanent deadlock: the
original owner could not sign in, and no one (not even the same GitHub
user with a new Convex Auth record) could reclaim the slug.

Move the `canHealSkillOwnershipByGitHubProviderAccountId` check before
the deleted/deactivated early-exit so ownership healing still works for
duplicate Convex Auth user records where the old record was later banned.

When healing is not possible (different GitHub identity or missing
auth records), show a message directing the user to contact
security@openclaw.ai instead of a generic "Slug is already taken".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: drop unused skills sort index map

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Nimrod Gutman <nimrod.gutman@gmail.com>
2026-03-12 11:53:52 +02:00
Nimrod Gutman
83f5e07f9d
chore(convex): upgrade convex to 1.32.0 (#766) 2026-03-12 11:51:10 +02:00
Trek90s
a9556ee3f0
fix: surface auth errors from OAuth callback to the UI (#688)
* fix: surface auth errors from OAuth callback to the UI

When `afterUserCreatedOrUpdated` throws a `ConvexError` (e.g. banned or
deleted account), the error was silently discarded — the user was
redirected back to the sign-in page with no feedback.

Parse the error from the OAuth callback URL hash fragment in the
`ConvexAuthProvider` `replaceURL` callback and expose it via a
lightweight `useAuthError` hook (backed by `useSyncExternalStore`).
Display the error next to the sign-in button in both Header and the
CLI auth page, and add `.catch()` to `signIn()` calls to avoid
unhandled promise rejections.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): handle oauth callback sign-in errors

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Nimrod Gutman <nimrod.gutman@gmail.com>
2026-03-12 11:33:47 +02:00
magicseth
041d8b4b92
Merge pull request #749 from sethconvex/fix/revert-nonsuspicious-index-query
fix: disable nonsuspicious index queries until backfill
2026-03-11 18:13:27 -07:00
DangerouslyShip
2058a53c1d fix: disable nonsuspicious index queries until isSuspicious is backfilled
Most skillSearchDigest rows have isSuspicious: undefined (not false),
so eq('isSuspicious', false) returns zero results. Revert to regular
indexes with JS filtering until the field is backfilled.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 18:12:12 -07:00
magicseth
123fecf04a
Merge pull request #748 from sethconvex/fix/digest-nonsuspicious-indexes
fix: add nonsuspicious indexes to skillSearchDigest
2026-03-11 18:04:19 -07:00
DangerouslyShip
b321025921 fix: use nonsuspicious indexes on skillSearchDigest for filtered pagination
Add by_nonsuspicious_* indexes to skillSearchDigest (matching the
existing ones on the skills table) so nonSuspiciousOnly filtering
happens at the index level instead of in JS. This eliminates empty
filtered pages without needing a multi-paginate loop.

TODO: once deployed and stable, remove the duplicate by_nonsuspicious_*
indexes from the skills table (no longer queried by listPublicPageV2).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 18:01:27 -07:00
magicseth
c1115b1491
Merge pull request #746 from sethconvex/fix/listpublicpagev2-multi-paginate
fix: remove multi-paginate loop in listPublicPageV2
2026-03-11 17:51:19 -07:00
DangerouslyShip
cd6403fec8 fix: remove multi-paginate loop in listPublicPageV2
Convex only allows a single .paginate() call per query function.
The while loop that skipped empty filtered pages violated this
constraint, causing "ran multiple paginated queries" errors on prod.

Remove the loop — clients will handle empty filtered pages by
requesting the next page via continueCursor.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 17:46:09 -07:00
magicseth
b7528760b5
Merge pull request #743 from sethconvex/fix/trending-leaderboard-doc-limit
fix: split trending leaderboard rebuild to avoid 32K doc read limit
2026-03-11 15:55:38 -07:00
DangerouslyShip
d47c774f8c fix: use Id<'skills'> cast instead of unsafe as any
Addresses review feedback from Greptile.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 15:42:21 -07:00
DangerouslyShip
8bf9414387 fix: split trending leaderboard rebuild to avoid 32K document read limit
The rebuildTrendingLeaderboardInternal mutation queries ~31,500
skillDailyStats docs (7 days × ~4,500/day) in a single transaction,
hitting Convex's 32K document read limit on prod (71 errors/72h).

Split into action → query → mutation pattern so each day's query runs
in its own transaction with its own 32K budget:
- getDailyStats (internalQuery): reads one day's stats
- writeTrendingLeaderboard (internalMutation): writes leaderboard + prunes
- rebuildTrendingLeaderboardAction (internalAction): orchestrates the above

The old single-mutation path is kept as a fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 15:31:33 -07:00
magicseth
0c6c71d167
Merge pull request #741 from sethconvex/perf/lexical-fallback-digest
perf: switch listPublicPageV2 and countPublicSkills to skillSearchDigest
2026-03-11 14:33:21 -07:00
Shakker
e3b80a848c fix: narrow moderation external override 2026-03-11 21:24:52 +00:00
Shakker
9b33abc0ea fix: harden moderation state reconciliation 2026-03-11 21:24:52 +00:00
Linfang Wang
d68facae8e fix: reduce false-positive skill moderation flags and enable recovery
Problem:
Skills using legitimate API integrations (process.env + fetch) were
permanently flagged as malicious due to CREDENTIAL_HARVEST being
classified as a malicious-level reason code. Once flagged, skills could
not recover to normal status even after clean VT and OpenClaw scans,
because:
1. syncModerationReasons used a partial-update path that only patched
   moderationReason without reconciling moderationFlags, moderationStatus,
   or moderationVerdict.
2. Static scan "you are now a/an" regex over-matched common skill
   preambles, adding spurious INJECTION_INSTRUCTIONS flags.
3. No mechanism existed for external scanner results (VT/LLM) to
   override static suspicious findings when both independently
   confirmed the skill as safe.

Solution:
- Downgrade CREDENTIAL_HARVEST from malicious.env_harvesting to
  suspicious.env_credential_access — env+network is suspicious, not
  malicious, for API integration skills (moderationReasonCodes.ts).
- Remove "you are now a/an" regex from markdown scanning to stop
  false INJECTION_INSTRUCTIONS flags (moderationEngine.ts).
- Add external scanner override in buildModerationSnapshot: when both
  VT and LLM report clean/benign, demote suspicious.* static codes
  from verdict calculation while preserving malicious.* codes and
  keeping all findings in evidence for transparency (moderationEngine.ts).
- Route syncModerationReasons through approveSkillByHashInternal for
  rows with sha256hash, ensuring full moderation state reconciliation.
  For legacy no-hash rows: malicious → escalateSkillByIdInternal
  (immediate hide); clean/suspicious → updateSkillModerationReasonInternal
  (partial fix, matches pre-existing behavior) (vt.ts, skills.ts).
- Add escalateSkillByIdInternal mutation for atomic emergency
  escalation by skillId (sets moderationReason, moderationFlags,
  moderationStatus, hiddenAt, isSuspicious) (skills.ts).
- Ensure approveSkillByHashInternal explicitly hides malicious skills
  by setting moderationStatus to 'hidden' (skills.ts).
- Bump MODERATION_ENGINE_VERSION to v2.1.0.

Frontend:
- Add StaticAnalysisDetail component to display static scan findings
  with severity-aware styling (SkillSecurityScanResults.tsx).
- getStaticGuidance now accepts vtStatus/llmStatus and shows "Confirmed
  safe by external scanners" (benign/green) when both are clean, instead
  of always showing yellow "Patterns worth reviewing" for critical
  severity findings.
- Render SecurityScanResults and disclaimer when only static findings
  are present (SkillHeader.tsx).

Testing:
- 7 new unit tests in moderationEngine.test.ts covering:
  - CREDENTIAL_HARVEST downgrade (suspicious, not malicious)
  - "you are now" no longer flagged in markdown
  - "ignore previous instructions" still flagged
  - buildModerationSnapshot: VT+LLM clean demotes suspicious codes
  - buildModerationSnapshot: malicious codes preserved despite clean VT+LLM
  - Single-scanner-clean does not demote suspicious codes
  - VT suspicious + LLM clean does not demote suspicious codes
- All existing tests pass with engine version bump to v2.1.0.

Follow-up needed (not in this commit):
- One-time backfill for already-misflagged skills (cursor-based,
  re-run approveSkillByHashInternal on isSuspicious=true + clean VT).

Made-with: Cursor
2026-03-11 21:24:52 +00:00
DangerouslyShip
73dfb7b2ba perf: switch listPublicPageV2 and countPublicSkills to skillSearchDigest
Both functions were hitting Bytes Read Limit errors scanning the full
skills table (~1.9KB/doc × 9K docs ≈ 17MB). Switch to the lightweight
skillSearchDigest table (~800 bytes/row) which carries all fields
needed by toPublicSkill/isPublicSkillDoc/isSkillSuspicious.

- Add 5 sort indexes to skillSearchDigest matching the ones used by
  SORT_INDEXES (by_active_created, by_active_name, by_active_stats_*)
- listPublicPageV2: query skillSearchDigest, map via digestToHydratableSkill
- countPublicSkillsForGlobalStats: query skillSearchDigest
- Widen buildPublicSkillEntries/filterPublicSkillPage to HydratableSkill[]
- Guard latestVersionSummary access (digest rows don't carry it)
- Update test mocks to expect skillSearchDigest table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 13:46:36 -07:00
Nimrod Gutman
e689a33a09 fix: resolve typescript errors blocking deploy 2026-03-11 21:21:39 +02:00
DangerouslyShip
d68bcc43dd perf: use skillSearchDigest for lexical fallback scan
Switches the 500-row lexicalFallbackSkills scan from full skill docs
(~3-5KB each) to lightweight digest rows (~800 bytes each), reducing
DB read bandwidth by ~75%.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:15:04 -07:00
magicseth
9861da02d5
Merge pull request #735 from sethconvex/feat/skill-search-digest
perf: add skillSearchDigest table to reduce search hydration bandwidth
2026-03-11 12:12:57 -07:00
DangerouslyShip
94c805d0f5 Merge remote-tracking branch 'origin/main' into feat/skill-search-digest
# Conflicts:
#	convex/skills.ts
2026-03-11 12:02:39 -07:00
Nimrod Gutman
487ecb3890
Merge pull request #682 from openclaw/feat/moderation-override-audit-tools
feat(moderation): add manual override audit tools
2026-03-11 20:46:53 +02:00
DangerouslyShip
869c45b5c0 perf: track all attempted embedding IDs to avoid redundant hydration
Build the seen-ID set from vector results rather than only successful
hydrations, so soft-deleted and suspicious embeddings aren't re-hydrated
on each candidate-limit expansion loop iteration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:34:51 -07:00
DangerouslyShip
5433c66200 refactor: adopt convex-helpers Triggers for automatic digest sync
Replace ~28 manual syncSkillSearchDigest/upsertSkillSearchDigest calls
across 4 files with a single Triggers handler in convex/functions.ts
that fires automatically on every skills table write. This eliminates
the risk of new mutations silently breaking digest consistency.

- Add convex-helpers as direct dependency
- Create convex/functions.ts wrapping mutation/internalMutation with triggers
- Update all 39 convex modules to import from ./functions
- Remove syncSkillSearchDigest from lib (no longer needed)
- Add normalizeId mock to test db objects for trigger wrapper compat

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:07:52 -07:00
DangerouslyShip
c74419d834 fix: add missing maintenance.ts sync hooks, type-safe digest hydration
- Add syncSkillSearchDigest calls to 6 maintenance mutations that patch
  digest-relevant fields without syncing (applyEmptySkillCleanup,
  applySkillBadgeBackfillPatch, upsertSkillBadgeRecord,
  backfillDenormalizedBadges, backfillIsSuspicious, applySkillBackfillPatch)
- Replace unsafe `as unknown as Doc<'skills'>` cast with typed
  HydratableSkill interface and digestToHydratableSkill mapper — compiler
  now catches field drift between digest and skill doc
- DRY up extractDigestFields/digestToHydratableSkill with shared
  SHARED_KEYS array and pick() helper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 10:16:43 -07:00
DangerouslyShip
e2c48d893c fix: clean up orphaned digest rows on hard-delete and fix reclaim test mock
When a skill is hard-deleted, syncSkillSearchDigest now removes the
corresponding digest row instead of silently no-oping. Also adds
skillSearchDigest table handling to reclaim test mock.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 10:09:22 -07:00
DangerouslyShip
b360de5291 refactor: extract shared validators between skills and skillSearchDigest tables
DRY up duplicated validator definitions (forkOf, badges, stats, moderationStatus)
into shared constants reused by both tables to prevent schema drift.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 10:02:53 -07:00
DangerouslyShip
ed2ecab0a2 fix: add missing digest sync hooks and address PR review feedback
- Remove redundant digest write for new skills (Greptile review)
- Add sync to hardDeleteSkillStep init/canonical/forks phases (Codex review)
- Add sync to patchStructuredModerationFromVersion (LLM analysis path)
- Add sync to report mutation (auto-hide path)
- Add sync to applyBanToOwnedSkillsBatchInternal (bulk ban)
- Add sync to restoreOwnedSkillsForUnbanBatchInternal (bulk unban)
- Add sync to transferSkillOwnershipAndEmbeddings (slug reclaim)
- Add sync to setSkillSoftDeletedInternal (internal soft-delete)
- Make digest test fixture derive from makeSkillDoc to avoid fragility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 09:58:44 -07:00
DangerouslyShip
872045681b perf: add skillSearchDigest table to reduce search hydration bandwidth
hydrateResults reads full skill docs (~3-5KB each) but only needs ~800
bytes for toPublicSkill/isPublicSkillDoc/isSkillSuspicious. Add a
lightweight skillSearchDigest projection table that is kept in sync by
all skill mutation paths.

Also fix the searchSkills while loop to incrementally hydrate only new
embedding IDs on each expansion instead of re-hydrating all candidates
from scratch (475 → 250 reads per search).

Expected impact: ~7x bandwidth reduction for hydrateResults
(495 GB → ~70 GB at current traffic).

Post-deploy: npx convex run maintenance:backfillSkillSearchDigestInternal --prod

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 09:34:55 -07:00
Nimrod Gutman
2528c1c35a test(skills): align public list pagination assertions 2026-03-11 11:03:59 +02:00
Nimrod Gutman
e93f9411f3 fix(moderation): tighten override safety guards 2026-03-11 10:49:09 +02:00
Nimrod Gutman
4049a3b58a feat(moderation): add manual override audit tools 2026-03-11 10:32:18 +02:00
magicseth
6318a74adf
Merge pull request #709 from sethconvex/fix/pre-existing-type-errors
fix: resolve type errors and update OG image branding
2026-03-10 23:42:05 -07:00
DangerouslyShip
e7101f155e fix: shrink OG subtitle to fit within card bounds
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 23:38:20 -07:00
DangerouslyShip
0f1c7536ba fix: regenerate og.png with correct ClawHub branding
The static OG image still said "ClawdHub" and "clawdhub.com" — regenerated
from the already-correct og.svg source.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 23:36:25 -07:00
DangerouslyShip
dc89ab643e fix: resolve pre-existing type errors in test files
- Add missing sha256 field to file mocks in moderation.test.ts
- Accept softDeletedAt param in makeSkillDoc in search.test.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 23:24:16 -07:00
magicseth
0a400437ff
Merge pull request #697 from sethconvex/fix/nonsuspicious-index-path
fix: use nonsuspicious indexes in listPublicPageV2

Should reduce bandwidth significantly
2026-03-10 23:20:50 -07:00
DangerouslyShip
023a01f411 fix: use nonsuspicious index for combined highlightedOnly + nonSuspiciousOnly
When both filters are active, use the nonsuspicious index for isSuspicious
and apply highlightedOnly as a JS filter on top, instead of scanning the
full table with both filters in JS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:32:46 -07:00
DangerouslyShip
2c42bf9900 fix: remove backfill fallback — isSuspicious backfill is complete
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 12:26:35 -07:00
DangerouslyShip
523b65e443 fix: use nonsuspicious indexes in listPublicPageV2 to avoid full table scans
Restore the NONSUSPICIOUS_SORT_INDEXES map and index branching logic that
was lost during the PR #572 merge. When nonSuspiciousOnly is set, queries
now use by_nonsuspicious_* indexes with isSuspicious=false in the predicate
instead of scanning the full table and filtering in JS — eliminating
bytesReadLimit errors under load.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 12:12:44 -07:00
Peter Steinberger
e07198ad41 test: cover key site workflows in playwright 2026-03-09 05:18:18 +00:00
Peter Steinberger
be6761526a test: strengthen playwright smoke error assertions 2026-03-09 05:18:15 +00:00
Peter Steinberger
72b6c5ede6 fix: fail deploy workflow clearly without secrets 2026-03-09 05:10:27 +00:00
Peter Steinberger
8dddcea5c4 fix: unblock deploy workflow smoke gate 2026-03-09 05:08:55 +00:00
Peter Steinberger
0e8c00a8eb fix: stabilize deployment drift query subscription 2026-03-09 04:27:25 +00:00
Peter Steinberger
0aa702fa70 fix: tolerate missing deployment info query 2026-03-09 04:23:38 +00:00
Ayaan Zaidi
4d72506b1c fix: isolate deployment drift banner failures 2026-03-09 09:52:04 +05:30
Peter Steinberger
2be9b67e74 ci: harden deploy pipeline against web/backend drift 2026-03-08 21:58:37 +00:00
Peter Steinberger
c617ef124a test: fix timeout mock typing 2026-03-08 03:36:51 +00:00
Peter Steinberger
114e480388 fix: expose structured moderation API (#334) (thanks @ArthurzKV) 2026-03-08 03:18:57 +00:00
Peter Steinberger
e31a8e9d32 fix: add structured moderation snapshots (#333) (thanks @ArthurzKV) 2026-03-08 03:13:13 +00:00
Peter Steinberger
460ad3c13d feat: add skill transfer API and CLI 2026-03-07 22:51:49 +00:00
Peter Steinberger
2687d671a0 feat: enforce MIT-0 skill licensing 2026-03-07 22:46:28 +00:00
Peter Steinberger
deb216e3b4 docs: fix explore flag list formatting (#601) (thanks @gandli) 2026-03-07 21:15:53 +00:00
gandli
e122569d2c docs(cli): fix indentation of --limit flag in explore command
The --limit flag under the 'explore' command's Flags section was
missing the proper two-space indentation, making it inconsistent
with other flag lists in the document.
2026-03-07 21:15:53 +00:00
Peter Steinberger
65649bc032 docs: clarify local dev setup workflow (#584) (thanks @jack-piplabs) 2026-03-07 21:14:45 +00:00
Jack Chan
cf59b41790 docs: fix local dev setup instructions in CONTRIBUTING.md
- Add Node.js v18/20/22/24 prerequisite (Convex backend rejects v25+)
- Remove duplicate CONVEX_SITE_URL from .env.local example
- Reorder steps so Convex backend starts before auth/JWT setup
- Add "Set backend environment variables" section (bunx convex env set)
- Clarify that AUTH_GITHUB_ID/SECRET and SITE_URL must be set on the
  Convex backend, not just in .env.local
- Make frontend port explicit (bun run dev -- --port 3000)
- Add updateGlobalStatsInternal step after seeding
2026-03-07 21:14:45 +00:00
Peter Steinberger
09054bb053 fix: add soft-delete search regression coverage (#552) (thanks @MunemHashmi) 2026-03-07 21:12:55 +00:00
Munem Hashmi
ea0b14dca5 test(search): add soft-delete filtering tests for vector and lexical paths (#29)
Verify that soft-deleted skills are excluded from both vector search
hydration and lexical fallback exact-slug matching.
2026-03-07 21:12:55 +00:00
Peter Steinberger
4a80758357 fix: update manifest branding changelog (#569) (thanks @Glucksberg) 2026-03-07 18:44:11 +00:00
Glucksberg
efd0f50d56 fix: update manifest.json with correct app name
Updates manifest.json from TanStack defaults to ClawHub branding.
This fixes the app name shown when installing as PWA.
2026-03-07 18:44:11 +00:00
Peter Steinberger
a45c5b91f3 fix: stabilize browse pagination during safety backfill (#572) (thanks @sethconvex) 2026-03-07 18:42:53 +00:00
Seth Raphael
fc22bfb2a1 fix: keep pagination cursor on same index path during backfill fallback
The nonsuspicious index fallback now fires on any page (not just the
first) and reuses the client's cursor via stale-cursor recovery. This
prevents pagination from breaking when a SORT_INDEXES cursor is sent
back to the NONSUSPICIOUS_SORT_INDEXES path on page 2+.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Seth Raphael
2152879cd7 fix: address PR review comments for bandwidth reduction
- Fix P1: remove !result.isDone guard from listPublicPageV2 backfill
  fallback so it fires when the nonsuspicious index is empty (isDone=true)
- Fix updateTags to update latestVersionSummary when repointing latest tag
- Parallelize leaderboard daily queries with Promise.all
- Over-fetch stale-reason candidates (2x limit) before VT filtering
- Reconcile existing latestVersionSummary in backfill instead of skipping
- Add _creationTime approximation comment
- Rebuild schema dist to include author field on ClawdisSkillMetadata

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Seth Raphael
f8d57f7c70 fix(schema): add compile-time guard for ClawdisSkillMetadata drift
The explicit ClawdisSkillMetadata interface (needed because ArkType's
[inferred] doesn't resolve all fields) now has a keyof-based type guard
that triggers a compile error if the interface keys drift from the schema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Seth Raphael
a3fe5cbc43 fix: resolve all 26 pre-existing typecheck errors
- Replace ArkType `[inferred]` type alias for ClawdisSkillMetadata with
  an explicit interface so TS can see envVars/dependencies/author/links
- Extract listBySkillHandler from comments.ts so tests can call it
  directly without accessing private _handler property
- Rebuild packages/schema dist output

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Seth Raphael
a90608d240 docs: use convex CLI for insights instead of MCP
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Seth Raphael
8d938e3b44 docs: tell agents to check Convex insights before writing queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Seth Raphael
7b7e2b3dc4 perf: reduce DB bandwidth ~1.5 TB/day with indexes, denormalization, and query rewrites
Phase 1: Add `by_moderation` compound index and rewrite 8 cron query
functions to use `.withIndex()` instead of `.filter().collect()` full
table scans (~6 GB/day saved).

Phase 2: Denormalize `isSuspicious` onto skills table with 6 compound
indexes so `listPublicPageV2` can filter at the index level instead of
paginating the entire table (~1 TB/day saved). Includes backfill
mutation and write-path updates across all moderation mutations.

Phase 3: Add `latestVersionSummary` denormalization to avoid reading
full ~6.4 KB `skillVersions` docs on list pages (~500 GB/day saved).

Phase 4: Split trending leaderboard query to one day at a time to stay
under 32K doc limit. Reduce global stats recount from hourly to daily
since delta tracking handles real-time accuracy (~400 MB/day saved).

Phase 5: Add "Convex Query & Bandwidth Rules" section to AGENTS.md.

Backfill commands (run after deploy):
  bunx convex run maintenance:backfillIsSuspiciousInternal
  bunx convex run maintenance:backfillLatestVersionSummaryInternal

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:42:53 +00:00
Peter Steinberger
8f2c86a878 fix: relax moderation false positives for auth skills (#273) (thanks @superlowburn) 2026-03-07 18:37:44 +00:00
Steve
06a528c5d9 fix: resolve biome lint errors in moderation test
- Fix import ordering (alphabetical: describe, expect, test)
- Add biome-ignore comments for test mock `as any` casts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:37:44 +00:00
Steve
bb1636f255 fix: reduce false positives in suspicious pattern detection for OAuth skills
Fixes #209 by removing overly broad regex patterns that flag legitimate
authentication and payment integration skills.

## Problem

Skills like openbotauth (OAuth identity verification) were being flagged
as suspicious because they mention "token", "api key", or "password" in
their description or metadata. The regex scanner was too aggressive,
catching legitimate auth flows alongside actual threats.

## Solution

Removed three overly broad patterns:
- `suspicious.secrets` - flagged ANY mention of token/api key/password
- `suspicious.crypto` - flagged ANY mention of wallet/seed phrase/crypto

These are common in legitimate skills:
- OAuth skills mention "token" for authentication flows
- API integrations mention "api key" for service credentials
- Database skills mention "password" for connections
- Crypto wallet skills mention "seed phrase" for key management

The LLM evaluator already handles credential proportionality analysis
(section 4 of security prompt). The regex scan should only catch
ACTUAL malicious patterns, not keywords that appear in legitimate contexts.

## What Still Gets Flagged

Kept patterns that catch real threats:
- `suspicious.keyword` - malware, stealer, phishing, keylogger
- `suspicious.webhook` - discord/slack webhooks (data exfiltration)
- `suspicious.script` - curl | bash (arbitrary code execution)
- `suspicious.url_shortener` - bit.ly etc (URL obfuscation)

## Testing

- Added 18 comprehensive tests for pattern detection
- Verified OAuth skills (openbotauth, trello) are NOT flagged
- Verified malicious patterns ARE still flagged
- All 418 existing tests pass

## Security Impact

This does NOT weaken security:
- LLM evaluator still analyzes credential proportionality
- Actual malicious patterns (webhooks, curl|bash, etc) still caught
- Only removes false positives on legitimate auth keywords

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 18:37:44 +00:00
Neerav Makwana
619489a93b fix: debounce URL navigation in skills search to reduce input lag
onQueryChange called navigate() on every keystroke to sync the query
to the URL. Each navigate triggers history.replaceState, TanStack
Router re-evaluation, and useSearch() invalidation, causing multiple
re-renders per keystroke.

Keep setQuery() immediate so the controlled input stays responsive,
but debounce the navigate() call at 220 ms (matching the existing
search-action debounce). Cancel the pending timer when search.q
changes externally (browser back/forward) to prevent the debounced
navigate from overwriting the external URL change.

Made-with: Cursor
2026-03-07 18:35:10 +00:00
Munem Hashmi
0f0086591c fix(ui): persist folder upload input across hydration and re-renders (#58)
Replace the useEffect + useRef approach for setting webkitdirectory/
directory attributes with a ref callback that sets the attributes
every time the input element is mounted. This ensures folder selection
mode persists after page refresh, where React hydration could strip
the non-standard attributes.

Also removes the @ts-expect-error JSX props since the attributes are
now set imperatively via the ref callback.
2026-03-07 18:33:03 +00:00
Peter Steinberger
531dcc8d26 fix: avoid auth crash in slug availability preflight 2026-03-07 18:29:01 +00:00
Tristan Manchester
b1c710e1ea fix: add dedicated slug availability preflight 2026-03-07 15:38:01 +00:00
Tristan Manchester
a4a9fc62bc fix: address review comments for slug-collision error handling 2026-03-07 15:38:01 +00:00
Tristan Manchester
e58fbc8d1f fix: surface slug-collision publish errors and block conflicts preflight 2026-03-07 15:38:01 +00:00
Peter Steinberger
b4b4f266a8 fix: align VT engine fallback verdict mapping (#591) (thanks @Shuai-DaiDai) 2026-03-07 15:33:30 +00:00
帅小呆1号
f9d35cc5e6 fix(vt): sync scan status from AV engines when Code Insight unavailable
When VirusTotal returns scan results with AV engine stats but no Code Insight
AI analysis, the skill status was stuck on 'Pending'. This fix adds fallback
logic to check last_analysis_stats (malicious/suspicious/harmless/undetected)
to determine scan status.

Functions updated:
- pollPendingScans: Check AV engines before requesting rescan
- backfillPendingScans: Check AV engines before marking as no results
- rescanActiveSkills: Check AV engines before keeping as pending
- backfillActiveSkillsVTCache: Check AV engines before skipping

Fixes #33435
2026-03-07 15:33:30 +00:00
Peter Steinberger
1892f72a13 test: lock multipart upload timeout behavior (#550) (thanks @MunemHashmi) 2026-03-07 15:31:20 +00:00
Munem Hashmi
fdbc184e0a fix(cli): improve publish timeout handling and error messages (#533)
- Increase upload timeout from 15s to 120s for multipart form uploads
  (apiRequestForm and curl-based form upload). Regular API requests
  remain at 15s.
- Improve timeout error message from bare "Timeout" to
  "Request timed out after Ns" so users know what happened.
- Normalize non-Error throws (e.g. DOMException from AbortController
  across runtimes) into proper Error instances, preventing the
  misleading "Non-error was thrown" message from p-retry.
- Preserve the original error as `cause` on the wrapped Error.
2026-03-07 15:31:20 +00:00
Peter Steinberger
18cbfc6788 test: cover auth token forwarding for search/explore (#608) (thanks @artdaal) 2026-03-07 15:29:37 +00:00
Артемов Даниил Алексеевич
c821b0ffc4 fix: pass auth token in search and explore commands
cmdSearch and cmdExplore were not calling getOptionalAuthToken()
and did not pass the token to apiRequest, unlike install/update/uninstall.
This caused 'missing API token' errors on registries that require auth
(e.g. private Hermit instances).
2026-03-07 15:29:37 +00:00
Peter Steinberger
9833a5038d docs: note top-level frontmatter metadata parsing fix (#548) (thanks @MunemHashmi) 2026-03-07 15:28:12 +00:00
Munem Hashmi
40a89e02d5 fix: extract requires.env and homepage from top-level frontmatter (#522)
parseFrontmatterLevelDeclarations did not handle the requires block
(env, bins, anyBins, config) or primaryEnv when declared at the
top level of SKILL.md frontmatter without a metadata.openclaw wrapper.
This caused the security scanner to always show "Required env vars: none"
for skills using that format, triggering false-positive suspicious flags.

Also extends the evalCtx.homepage fallback chain to check
clawdis.homepage and clawdis.links.homepage so skills declaring
homepage inside the metadata block are picked up by the scanner.
2026-03-07 15:28:12 +00:00
Timothy Jordan
bc06dbffd0
chore: add Vercel attribution in footer (#557)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:06:35 +00:00
Agent
f5fa23e0c1 chore: add convex attribution in footer 2026-02-27 23:10:26 +01:00
Vincent Koc
e8c3947b21
Merge pull request #547 from openclaw/fix/secret-scan-trufflehog-ref
fix(ci): restore secret scan action reference
2026-02-27 10:39:18 -08:00
Vincent Koc
ef26ee0d1f fix(ci): pin trufflehog to published v3.93.6 tag 2026-02-27 10:38:31 -08:00
Vincent Koc
3d006ec663 fix(ci): use resolvable trufflehog action ref 2026-02-27 09:47:17 -08:00
Peter Steinberger
52590e84dd chore: commit local pending changes 2026-02-26 13:03:25 +01:00
Peter Steinberger
e3a1c95851 docs(agents): reject skill-in-source PRs; require CLI publish 2026-02-26 13:03:25 +01:00
Mahsum Aktaş
db4540743f
feat(registry): support env vars, dependencies, author, and links in skill manifest (#360)
* feat(registry): support env vars, dependencies, author, and links in skill manifest

Closes #350

Add structured declarations for environment variables, package
dependencies, author identity, and project links to the skill
registry manifest. These fields can be declared in the clawdis
metadata block or as top-level frontmatter keys.

Changes:
- schema: add EnvVarDeclaration, DependencyDeclaration, SkillLinks
  types to ClawdisSkillMetadata
- parser: extract envVars, dependencies, author, links from both
  clawdis block and top-level frontmatter (fallback for skills
  without a clawdis block)
- UI: render env vars with required/optional badges and descriptions,
  dependencies with type/version/links, and project links in the
  skill detail page install card
- security: update evaluator prompt to recognize envVars alongside
  requires.env and primaryEnv
- tests: 7 new test cases covering all declaration formats

* fix(ui): handle unspecified env required state and stable keys

* docs(changelog): credit metadata manifest expansion (#360) (thanks @mahsumaktas)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 12:02:25 +00:00
David Abutbul
0cb0963c2b
feat(api): expose security evaluation results (#362)
* feat(api): expose security evaluation results

- Add security field to skill version API responses
- Map llmAnalysis database field to public API format
- Display security info in CLI inspect command
- Enable security tools like clawsec-clawhub-checker to access internal security checks

Security field includes:
- status: clean|suspicious|malicious|pending|error
- hasWarnings: boolean
- checkedAt: timestamp
- model: evaluation model name

Backward compatible: optional field, no breaking changes.

* fix: ensure hasWarnings is always boolean

- Add ?? false to coerce undefined to false when dimensions is undefined
- Fixes Greptile comment: hasWarnings can be undefined instead of boolean
- Ensures SecurityStatusSchema validation passes on client side

* Update convex/httpApiV1/skillsV1.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(api-cli): harden security inspect output + tests (#362) (thanks @abutbul)

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 12:01:23 +00:00
David Aronchick
beae065794
fix(cli): handle missing browser opener gracefully (#163)
* fix(cli): handle missing browser opener gracefully

On headless Linux servers without xdg-open, 'clawhub login' crashes with
ENOENT error. This change catches the error and prints the URL for manual
copy-paste instead of crashing.

Fixes crash on:
- VPS/cloud servers
- Docker containers
- CI environments
- WSL without browser integration

* fix(cli): test browser-opener fallback messaging (#163) (thanks @aronchick)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 11:59:24 +00:00
Vassiliy Lakhonin
46c5637dae
Improve error handling in GitHub import (#512)
* Improve error handling in GitHub import

- Add detailed error messages for storage failures
- Wrap publishVersionForUser in try/catch with helpful context
- Include file size and path in storage error messages
- Guide users to check skill format and slug availability

* fix(github-import): improve failure messaging + coverage (#512) (thanks @vassiliylakhonin)

---------

Co-authored-by: Vassiliy Lakhonin <vassiliy.lakhonin@example.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-26 11:30:01 +00:00
Tristan Manchester
2217a327e7
fix(upload): ignore macOS junk files during publish (#526) 2026-02-26 11:28:27 +00:00
Jason
45d8f0d217
feat: surface platform/architecture labels on skill cards and API (#499)
* feat: surface platform/architecture labels on skill cards and API

Expose existing `os` and `nix.systems` metadata from skill frontmatter
through the HTTP API and render as compact tags on browse/search views.

- Widen `PublicSkillListVersion` and `SkillListEntry` types to include
  `os` and `nix.systems` fields (data already flows through, types were
  artificially narrow)
- Add `metadata: { os, systems }` to `/api/v1/skills/{slug}` and
  `/api/v1/skills` list responses
- Add `formatSystemsList` and `getPlatformLabels` helpers to map nix
  system strings to human-readable labels (e.g. aarch64-darwin → macOS ARM64)
- Add `platformLabels` prop to `SkillCard`, render as `.tag .tag-compact`
- Show platform labels in both card grid and list views
- Update HTTP API docs with new `metadata` field

Coded by Claude Opus 4.6 (Claude Code)
Reviewed and tested by Jason (@asyncjason)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove redundant optional chaining on clawdis

Address greptile-apps review comment — clawdis is already confirmed
truthy by the ternary condition, so `?.` is unnecessary.

Coded by Claude Opus 4.6 (Claude Code)
Reviewed and tested by Jason (@asyncjason)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: include version data in listPublicPageV2 for platform labels

The browse listing passed includeVersion: false, causing latestVersion
to always be null and platform/arch labels to never render.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Jason Separovic <jason@wilma.dog>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 11:27:20 +00:00
Nicolas Grenié
add5d83014
docs: add CONTRIBUTING.md and refresh README header (#400)
* docs: add CONTRIBUTING.md and refresh README header

Add a comprehensive CONTRIBUTING.md covering local Convex setup,
env var configuration, GitHub OAuth, JWT keys, database seeding,
CLI development, PR guidelines, and AI-generated code policy.

Refresh the README with a centered logo, quick links row, and
clickable doc references. Condense the Local dev section to link
to CONTRIBUTING.md for full setup details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add #clawhub discord channel

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 11:25:51 +00:00
Abdul B.
9751199231
fix: prevent filtered skills pagination flicker (#372)
* fix: prevent filtered skills pagination flicker

Skip fully filtered-out pages in public skills pagination so highlighted/non-suspicious filtering doesn't return empty pages with more cursor state, which caused repeated loading-more flicker.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: rely on inferred Convex paginate result type

Remove the custom runPaginate annotation so TypeScript infers the exact Convex paginate result shape and preserves stronger type-safety.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-26 11:25:15 +00:00
Luke
883221f8ec
fix(cli): clarify owner delete permissions in command text (#417) 2026-02-26 11:23:53 +00:00
Peter Steinberger
3e45d67e0d fix: delete and hide comments from banned users 2026-02-26 05:52:09 +01:00
Peter Steinberger
df346aeea9 feat: require 14-day GitHub age for publish and comments 2026-02-26 05:35:44 +01:00
Peter Steinberger
4317369480 fix: stabilize comment moderation tests without env keys 2026-02-26 02:18:00 +01:00
Peter Steinberger
e04d16bdae feat: add ai comment scam backfill and auto-ban flow 2026-02-26 02:16:31 +01:00
Peter Steinberger
cb66d8d6f3 feat: add abuse-resistant comment reporting 2026-02-26 01:28:22 +01:00
Peter Steinberger
14a2fa80f6 test: lock 5xx retry behavior in HTTP client (#457) (thanks @YonghaoZhao722) 2026-02-25 13:03:16 +00:00
Peter Steinberger
ee788b7af3 test: pin Retry-After relative-delay behavior (#421) (thanks @apoorvdarshan) 2026-02-25 12:19:10 +00:00
Apoorv Darshan
3956ca7e55 fix: use relative delay for Retry-After header on 429 responses
Retry-After was set to an absolute Unix epoch timestamp (e.g. 1771404540),
which violates RFC 9110 §10.2.3. Clients treating it as delay-seconds
would wait ~56 years. Now emits the actual seconds until reset.

Closes #407
2026-02-25 12:19:10 +00:00
Peter Steinberger
bc37ec7156 fix: complete registry-url migration with test coverage (#486) (thanks @Liknox) 2026-02-25 12:17:30 +00:00
Nazar Koval
f07408eb81 test: url formatter 2026-02-25 12:17:30 +00:00
Nazar Koval
cdf5baef7f ref: url entity utilization 2026-02-25 12:17:30 +00:00
Nazar Koval
65b154f36c feat: url formatter entity 2026-02-25 12:17:30 +00:00
Peter Steinberger
6ea7a0792d fix: finalize proxy env support + changelog credits (#363) (thanks @kerrypotter) 2026-02-25 12:14:00 +00:00
Jarvis
ed961e459f fix: use EnvHttpProxyAgent for proper proxy support
Address review feedback:
- Use undici's EnvHttpProxyAgent instead of ProxyAgent. This properly
  handles HTTPS_PROXY vs HTTP_PROXY per-scheme, respects NO_PROXY,
  and uses connect.timeout instead of requestTls.
- Update docs to mention NO_PROXY support.
2026-02-25 12:14:00 +00:00
Jarvis
8b5f242f73 fix: respect HTTP_PROXY/HTTPS_PROXY environment variables
The CLI creates a custom undici Agent via setGlobalDispatcher() which
overrides any proxy configuration. Since Node.js native fetch (backed
by undici) does not automatically respect HTTP_PROXY/HTTPS_PROXY env
vars, the CLI fails with 'fetch failed' on systems that require a
proxy for outbound connections.

Import ProxyAgent from undici and use it when any of the standard proxy
environment variables (HTTPS_PROXY, HTTP_PROXY, https_proxy, http_proxy)
is set. When no proxy variable is present, behavior is unchanged.

Also adds proxy documentation to cli.md and a troubleshooting entry.
2026-02-25 12:14:00 +00:00
Phineas1500
311bf1a88a fix: guard de-escalation on clean status and fix alreadyFlagged blocking clean verdicts
Address review feedback:
- Guard rescan de-escalation with `status === 'clean'` so pending/unknown
  verdicts don't accidentally clear the suspicious flag
- Fix approveSkillByHashInternal where `alreadyFlagged` in the condition
  `(isSuspicious || alreadyFlagged) && !bypassSuspicious` prevented clean
  verdicts from reaching the isClean branch that properly checks whether
  a different scanner set the flag
2026-02-25 12:09:41 +00:00
Phineas1500
8343f0bb23 fix: clear stale suspicious flag when VT verdict improves to clean
The daily VT rescan updated vtAnalysis on the version but only called
escalateByVtInternal for suspicious/malicious verdicts. When a verdict
improved from suspicious to clean, the version's vtAnalysis was updated
(website shows "Benign") but the skill's moderationFlags kept the stale
"flagged.suspicious" entry (CLI warns "suspicious"). Now the rescan
calls approveSkillByHashInternal to clear the flag on de-escalation.
2026-02-25 12:09:41 +00:00
Peter Steinberger
3b73a09d36 fix: keep denormalized badge reads consistent (#441) (thanks @sethconvex) 2026-02-25 03:00:34 +00:00
Seth Raphael
412249d2d1 fix: use destructuring instead of undefined assignment in removeSkillBadge
Avoids leaving an explicit undefined key in the badges object which
could fail Convex validation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 03:00:34 +00:00
Seth Raphael
480125d859 feat: add backfill for denormalized skill badges, clean up fallback
- Add backfillDenormalizedBadgesInternal: syncs skillBadges table →
  skill.badges field so listing/search reads are correct
- Simplify hydrateResults fallback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 03:00:34 +00:00
Seth Raphael
0ab1d1e051 fix: restore fallback in hydrateResults for un-backfilled embeddings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 03:00:34 +00:00
Seth Raphael
85374fa44b perf: eliminate redundant badge reads and add embedding lookup table
- Remove badge table queries from listing and search paths (~200 queries
  per page load eliminated). Use denormalized skill.badges field instead.
- Sync skill.badges when badges are mutated (upsertSkillBadge/removeSkillBadge).
- Add embeddingSkillMap lookup table (~100 bytes/doc) so search hydration
  can skip reading full skillEmbeddings docs (~12KB each with vector).
- Remove dead badge query exports from search module.
- Reduce lexical fallback scan limit from 1200 to 500.
- Add backfill mutation for embeddingSkillMap with graceful fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 03:00:34 +00:00
Seth Raphael
c319e46c8b perf: cap badge query to 10 records per skill
Skills should never have more than a handful of badge records.
Using .take(10) instead of .collect() avoids unbounded reads.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 03:00:34 +00:00
Seth Raphael
f17087d1f4 perf: reduce db bandwidth by splitting stat processing into two frequencies
The 5-minute stat event processor was patching skill documents on every run,
which invalidated listPublicPageV2 reactive queries for ALL subscribers —
causing a thundering herd responsible for ~17 TB (59%) of the 28.65 TB
monthly db bandwidth.

Split into two paths:
- Daily stats (15-min cron): writes to skillDailyStats only, no skill doc patches
- Skill doc sync (6-hour cron): patches skill documents with accumulated deltas

Also skip reading version docs in listPublicPageV2 and search hydration
(version data is only needed on detail pages, not listings).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 03:00:34 +00:00
LK
da4469e1e0 fix(rate-limit): skip ip consumption for authenticated requests 2026-02-25 02:56:20 +00:00
LK
42a4648475 refactor(http): parse rate-limit headers once per error 2026-02-25 02:56:20 +00:00
Luke
edc8ec274b Update docs/http-api.md
fix header precedence in docs to match code

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-25 02:56:20 +00:00
LK
15b1a05fee fix(rate-limit): make limiter auth-aware for shared proxy ips 2026-02-25 02:56:20 +00:00
LK
1e216c03c6 fix(rate-limit): standardize retry-after and improve 429 ux 2026-02-25 02:56:20 +00:00
ɐʞsǝs
0a0b2e6cb1 pin the th version for better stability / security 2026-02-25 02:50:17 +00:00
ɐʞsǝs
ac6770acff Update .github/workflows/secret-scan.yml
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-25 02:50:17 +00:00
ɐʞsǝs
bfc87e5932 refined message to give better guidance 2026-02-25 02:50:17 +00:00
ɐʞsǝs
3e1bd19a45 Add secret scanning workflow using TruffleHog
This ensures that a given PR will immediately error on vulnerable, live credentials found by trufflehog
2026-02-25 02:50:17 +00:00
Peter Steinberger
d71b747d1c fix: harden VT fallback activation rules (#300) (thanks @superlowburn) 2026-02-25 02:48:57 +00:00
Steve
4d211fcf73 fix: check moderation reason before activating skills
- Prevent activating skills with quality.low moderation reason
- Add skill lookup and moderationReason check in 3 locations where skills are activated
- This ensures quality gate quarantine is not bypassed when VT scan is unavailable or stale

Resolves review comments on #300
2026-02-25 02:48:57 +00:00
Steve
bc5ab8f3e1 fix: activate skills when VT scan is unavailable or stales out
Published skills stay permanently hidden in search when VirusTotal
cannot produce a verdict. Three code paths leave moderationStatus as
'hidden' with no recovery:

1. VT_API_KEY not configured — scan skipped, skill stays hidden
2. VT hash not found after 10 poll attempts — marked stale, stays hidden
3. VT hash found but no Code Insight after 10 attempts — same

Fix: call setSkillModerationStatusActiveInternal in all three paths so
the skill becomes searchable. If VT later returns a malicious verdict,
approveSkillByHashInternal will correctly re-hide and flag it.

Closes #139

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 02:48:57 +00:00
Peter Steinberger
bb528ea4b9 fix: dedupe OpenAI response parsing (#502) (thanks @ianalloway) 2026-02-25 02:46:35 +00:00
Nimrod Gutman
464a04c1e5
fix(vt): prevent pending scan starvation and retry unresolved results (#468)
* fix(vt): prevent pending scan starvation and retry unresolved results

* fix(vt): remove exhaustive pending-scan clamp
2026-02-23 21:26:47 -06:00
Peter Steinberger
f4f8e7276f docs: clarify skill delete/undelete permissions 2026-02-18 17:24:09 +01:00
Peter Steinberger
3ccf2e05f5 fix: make users reclaim transfer root slug ownership in place 2026-02-18 17:10:08 +01:00
Shadow
0ee2872f5b
fix: prune deleted skill backups 2026-02-17 11:02:57 -06:00
Peter Steinberger
5112d1b215 docs: cross-link README and vision 2026-02-17 17:35:28 +01:00
Peter Steinberger
275a170f15 docs: clarify MCP support policy in vision 2026-02-17 17:33:54 +01:00
Peter Steinberger
17aa24baf9 docs: refine vision priorities and security stance 2026-02-17 17:31:24 +01:00
Peter Steinberger
f01476757a fix(ui): remove upvote-style metric and clarify installs icon 2026-02-17 15:08:47 +01:00
Peter Steinberger
fe011d00fd test(e2e): update search menu route expectation 2026-02-17 15:08:42 +01:00
Peter Steinberger
30ae099825 fix: avoid paginated fallback in skills count query (#76) 2026-02-17 00:32:22 +01:00
Peter Steinberger
812641342d fix: harden /skills count when globalStats is missing (#76) 2026-02-17 00:25:46 +01:00
Peter Steinberger
88848c224c fix: keep public skill counts consistent after moderation/visibility changes (#76) (thanks @rknoche6) 2026-02-17 00:25:46 +01:00
rknoche6
c107adabac display total skills count on /skills page (#76)
Co-authored-by: rknoche <richard.knoche@holidaycheck.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-17 00:25:46 +01:00
Peter Steinberger
38c4a673da
Revert "display total skills count on /skills page (#76)" (#359)
This reverts commit 89933951f5.
2026-02-16 17:47:19 +01:00
rknoche6
89933951f5
display total skills count on /skills page (#76)
Co-authored-by: rknoche <richard.knoche@holidaycheck.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-16 16:06:45 +01:00
Peter Steinberger
e3523093b1 chore: bump clawhub to 0.7.0 2026-02-16 06:00:49 +01:00
Peter Steinberger
1faf3ee5ed test: boost cli http and user branch coverage 2026-02-16 05:58:20 +01:00
Peter Steinberger
286c76a05f test: add even more user auth and ranking regressions 2026-02-16 05:44:12 +01:00
Peter Steinberger
5745b5a096 test: add more users search and auth regressions 2026-02-16 05:40:30 +01:00
Peter Steinberger
a060ae3b15 test: expand users list and search coverage 2026-02-16 05:33:44 +01:00
Peter Steinberger
77982c5d8e test: harden users search coverage for malformed data 2026-02-16 05:31:44 +01:00
Peter Steinberger
4cb84df36a fix: bound users list scans for management 2026-02-16 05:19:05 +01:00
Peter Steinberger
d82c8f66c2 test: add regressions for pagination edge cases 2026-02-16 05:09:05 +01:00
Peter Steinberger
43fd834d23 refactor: simplify pagination + compact stat formatting 2026-02-16 05:07:18 +01:00
Peter Steinberger
9b2fc48a55 fix: harden listPublicPageV2 cursor recovery 2026-02-16 05:03:57 +01:00
Peter Steinberger
a4dad5dc9d refactor: split skills page model and centralize stat rendering 2026-02-16 04:46:13 +01:00
Peter Steinberger
84830a268a feat: compact-format skill and soul stats 2026-02-16 04:35:01 +01:00
Peter Steinberger
37ef3eb7c5 test: add listPublicPageV2 regression coverage 2026-02-16 04:32:15 +01:00
Peter Steinberger
7f987fcc26 fix: prevent skills pagination dead-end and flicker (#339) (thanks @Marvae) 2026-02-16 04:24:24 +01:00
Hongwei Ma
a0ea45c9a6 test: add coverage for search empty state 2026-02-16 04:24:24 +01:00
Hongwei Ma
1f5a782ecd fix: show empty state immediately for search results 2026-02-16 04:24:24 +01:00
Hongwei Ma
c300d4b447 test: add edge case coverage for LoadingMore with empty results 2026-02-16 04:24:24 +01:00
Hongwei Ma
c3a6cd7356 fix: keep load more visible during pagination
Prevents flicker when loading more pages by showing the load more
area during LoadingMore status instead of hiding it completely.
2026-02-16 04:24:24 +01:00
Hongwei Ma
b75e25c4d6 fix: prevent loading state flicker on skills page
- Show 'Loading skills…' instead of 'No skills match' when pagination is not exhausted
- Hide 'Scroll to load more' when results are empty
- Add tests for both cases
2026-02-16 04:24:24 +01:00
Peter Steinberger
54383665d8 refactor: split skill detail page and optimize lazy loading 2026-02-16 03:52:24 +01:00
Kevin Kern
697cc1a08f
feat: add skill file viewer (#44)
* fix: return proper HTTP status codes for delete/undelete errors

The delete and undelete handlers for skills and souls were catching all
errors and returning 401 Unauthorized, even for errors like:
- 'Skill not found' (should be 404)
- 'Forbidden' (should be 403)
- Other validation errors (should be 400)

This change updates the error handling to return appropriate status codes:
- 401 Unauthorized: authentication failures
- 403 Forbidden: authorization failures (not owner/admin/moderator)
- 404 Not Found: skill/soul/user not found
- 400 Bad Request: other errors with descriptive message

Fixes #34

* fix(cli): use proper Error objects in abort timeouts

When AbortController.abort() receives a string instead of an Error,
the string itself is thrown. pRetry then wraps it in a confusing
message: 'Non-error was thrown: Timeout'

Changed all 3 occurrences in http.ts:
- apiRequest (line 57)
- apiRequestForm (line 106)
- downloadZip (line 141)

Now timeouts will surface as proper Error objects with clear messages.

* test: add e2e test for delete error handling

Verifies that deleting a non-existent skill returns a proper 'not found'
error instead of a generic 'Unauthorized' message.

* fix: use Error for timeout abort in e2e helper

* feat: add skill file viewer

* fix: prevent file viewer state updates after unmount

* feat: add ban reasons to moderation

* chore: release 0.6.0

* docs: reset changelog for next release

* feat: add LLM security evaluation at publish time

Add OpenClaw LLM-based security evaluator that runs alongside VirusTotal
when skills are published. Reads SKILL.md prose, metadata, install specs,
and file manifest, then assesses coherence across 5 dimensions to catch
social engineering vectors that VT/regex miss (e.g. instruction-only skills
with no code files).

- convex/lib/securityPrompt.ts: system prompt, message assembly, response
  parsing, injection pattern detection
- convex/llmEval.ts: evaluateWithLlm action, evaluateBySlug convenience
  action, backfillLlmEval for existing skills
- convex/schema.ts: llmAnalysis field on skillVersions
- convex/skills.ts: updateVersionLlmAnalysisInternal mutation,
  getActiveSkillBatchForLlmBackfillInternal query, defense-in-depth
  multi-scanner flag merging in approveSkillByHashInternal
- convex/lib/skillPublish.ts: schedule LLM eval alongside VT scan
- SkillDetailPage.tsx: OpenClaw row, LlmAnalysisDetail expandable
  component with 5 dimension rows, guidance panel, findings section
- styles.css: analysis detail styles from mockup

* fix: collapse OpenClaw analysis by default, fix row spacing, switch to gpt-5-mini

* fix: add retry with backoff for OpenAI rate limits, fix JSON mode requirement

* fix: increase max_output_tokens for reasoning model, fix backfill error retry

* feat: recognize metadata.openclaw as valid frontmatter namespace

* fix: eval assembler falls back to metadata.openclaw for requirements

* feat: evaluator reads all file contents, not just SKILL.md

Reads all files from storage and includes their full source in the eval
prompt so the LLM can detect malicious code hidden behind clean READMEs.
Injection detection now scans all content. Per-file cap 10K chars, total
cap 50K chars.

* feat: add skill metadata docs, suspicious appeal banner for owners

- Document full frontmatter metadata reference in docs/skill-format.md
- Add metadata section + quick example to README
- Show appeal message on suspicious skills (owner-only) linking to GitHub issues
- Accept metadata.openclaw alias in README docs
- Re-evaluate all skills with full file content reading (backfill in progress)

* fix: trailing comma tolerance in JSON metadata, tone down persistence flags

- Strip trailing commas in frontmatter JSON before parsing (silent failure fix)
- Stop flagging disable-model-invocation default as a concern (it's the normal default)
- Stop flagging skills configuring themselves as privilege escalation
- Add MITRE ATLAS AML.T0051 context for when autonomous invocation actually matters
- Show actual defaults in assembled eval message instead of "not set"

* chore: fix lint issues (#213)

* perf: lazy-load diff viewer (Monaco) (#212)

* chore: fix review comments

* fix: VT scan sync race condition + LLM-first moderation model

VT no longer overwrites LLM moderation verdicts. LLM is the primary
moderation authority; VT only escalates (hides + flags) for malicious/
suspicious content via new escalateByVtInternal mutation. Stale VT polls
write vtAnalysis marker instead of overwriting moderationReason. Query
pools expanded to include LLM-evaluated skills awaiting VT results.
Ban message now references malicious skills and security@openclaw.ai.

* fix: handle GitHub API rate limits in account age check (#246)

* fix: handle GitHub API rate limits in account age check

The GitHub account lookup uses unauthenticated requests (60 req/hr
per IP). Since this runs server-side in Convex, all users share the
same IP and quickly exhaust the rate limit, causing "GitHub account
lookup failed" errors during skill publish.

- Detect 403/429 responses and surface a clear rate-limit message
- Support optional GITHUB_TOKEN env var for authenticated requests
  (5,000 req/hr)

Fixes #155

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: stabilize GitHub account gate tests and docs

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* docs: thank @superlowburn for PR #246

* fix: prioritize relevant skills in search

* fix: add lexical fallback for skill search recall

* test: add search fallback coverage

* test: fix search test handler typing

* fix(http): remove allowH2 from undici Agent — causes fetch failed on Node.js 22+ (#245)

* Remove allowH2 option from global dispatcher

fix/remove-allowH2-undici-node22-compat

* fix(http): remove allowH2 from e2e dispatcher

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* docs: add 0.6.1 unreleased changelog from post-0.6.0 commits

* fix: allow soft-deleted users to re-authenticate

Fixes Issue #32 where users who soft-deleted their accounts were unable to sign back in because the re-auth logic was only triggering when an existingUserId was passed by the auth provider, which doesn't happen during a standard fresh login flow.

* test: update auth tests for direct deletedAt check

* fix: restore existingUserId check for type safety

* fix: update tests to include required existingUserId parameter

* fix: resolve final lint error in auth tests

* fix: ensure reactivation only matches soft-deleted user (prevents bypass)

* fix: allow re-auth when existingUserId is null

* fix: use valid crons.interval and set to 1 minute

* test: add missing coverage for fresh-login reactivation and identity mismatch guard

* fix: scope reauth fix; keep banned users blocked (#177) (thanks @tanujbhaud)

* fix: include comment deltas in action-based stat processing & add stats reconciliation (#194)

Bug 1: applyAggregatedStatsAndUpdateCursor was missing 'comments' in both
the guard condition and the applySkillStatDeltas call. This caused comment
count deltas to be silently dropped during cron-based event processing,
while stars/downloads/installs were processed correctly.

Bug 2: No reconciliation mechanism existed. If events were missed due to
cursor issues or processing errors, skill stats (stars, comments) would
remain stale with no way to recover. Added reconcileSkillStarCounts
maintenance mutation that counts actual records in the stars and comments
tables and patches any out-of-sync skill stats.

Fixes #193

Co-authored-by: Limitless2023 <limitless@users.noreply.github.com>

* fix: prevent horizontal overflow from long code blocks in skill pages (#183)

* Fix: Prevent horizontal overflow from long code blocks in skill pages

- Add max-width: 100% to .file-list-body and .file-row
- Prevents page-wide overflow when skills contain long code examples
- Markdown pre blocks already have overflow-x: auto, but parent containers were expanding infinitely
- Fixes issue where skills with 400+ char lines (e.g. browser automation commands) cause horizontal scrolling

Affected: Skills with long inline code in markdown (browser act commands, etc.)

* fix: add max-width to .file-list container to prevent overflow

- Also ensures .file-list-body constraint is inherited properly
- Prevents long code blocks from expanding file list container

* fix: add max-width to all markdown containers and pre tags

- Add max-width: 100% to .markdown, .tab-body, .markdown pre
- Ensures code blocks are constrained and show horizontal scrollbar
- Prevents content from expanding parent containers beyond viewport

* fix: add overflow-x to parent containers for horizontal scroll

Adds overflow-x: auto to .skill-detail-stack, .tab-card, and .tab-body
to ensure long code blocks are scrollable within the content area
instead of causing page-wide horizontal overflow.

Fixes horizontal overflow issue on skill pages with long code examples
(e.g., browser automation commands with 400+ character lines).

Tested on zepto skill page - page now stays within viewport (1200px)
and code blocks are accessible via horizontal scrollbar in tab area.

* docs: note code-block overflow fix in changelog (#183) (thanks @bewithgaurav)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* chore(release): 0.6.1

* fix: prevent infinite loading loop on skills page  (#90)

* fix: prevent infinite loading loop on skills pageAdd isLoadingMore guard to IntersectionObserver useEffect to preventcontinuous WebSocket queries when user is idle at bottom of page.The observer now won't set up while a request is in progress, breakingthe infinite loop cycle.Fixes: Related to #89

* fix: prevent repeated skills auto-load requests (#90) (thanks @xcqtnr)

* fix: resolve PR merge conflicts and keep observer regression test (#90) (thanks @xcqtnr)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* fix(cli): secure config file permissions (#164)

* fix(cli): secure config file permissions and reduce duplication

Security:
- Config files now created with 0600 permissions (owner read/write only)
- Config directories created with 0700 permissions
- Protects API tokens from other users on shared systems

Maintainability:
- Extract resolveConfigPath() helper to reduce code duplication
- Same legacy fallback logic (clawhub -> clawdhub) now in one place

* fix(cli): tolerate unsupported chmod errors for config

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* fix: make /search host-aware in SSR (#257)

* fix: make /search mode-aware

Notes:\n- Medium: /search now depends on getSiteMode() during beforeLoad. On server-side routing, if VITE_SITE_MODE isn’t set and VITE_SOULHUB_SITE_URL is set (as in .env.local), getSiteMode() will resolve to souls and redirect /search to / even on the ClawdHub deployment. This is a regression risk vs the old always-/skills redirect. Confirm deployment envs guarantee correct mode. src/routes/search.tsx:9-31

* fix: make /search host-aware in SSR

* chore: fix lint and route tree for /search route

---------

Co-authored-by: Sash Zats <sash@zats.io>

* fix(vt): explicit return types and missing undici dependency (#255)

* fix(vt): explicit return types and missing undici dependency

Refactor action handlers in convex/vt.ts to use explicit return types, resolving circular type inference (TS7022). Also add undici to devDependencies for E2E tests.

* fix: add root undici devDependency for e2e (#255) (thanks @tanujbhaud)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* Fix initial skill sorting (#92)

* fix: initial skill sorting

* chore: update unit test

* fix: use correct indexes for skill sorting

* chore: cleanup

* fix: land skill sorting update (#92) (thanks @bpk9)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* fix: harden download rate limiting and dedupe (#43) (thanks @regenrek)

- add download-specific rate limit tier\n- add per-IP/day dedupe + daily pruning\n- keep moderation gating + deterministic zips\n- add optional forwarded-IP trust via TRUST_FORWARDED_IPS

* fix: harden skill listing and rate limiting under load

* fix: replace skill report prompt with modal

* fix: add skill publish anti-spam caps and quarantine

* docs: add git local-branch cleanup fallback

* fix: enforce quality gate and trust-tier spam checks

* fix: prevent autobanned users from self-reactivating

* test: expand reauth ban regression coverage

* feat: add empty-skill cleanup backfill with ban nominations

* fix: make empty-skill cleanup resumable

* feat: add non-suspicious skills filter toggle

* style: polish selected states in skills toolbar

* feat: default skills sort to downloads

* fix: enforce downloads as canonical default skills sort

* fix: force canonical downloads sort in skills browse mode

* fix: bypass suspicious flags for privileged owners and polish comment delete UI

* fix: add privileged-owner suspicious flag reconciler

* fix: force auth redirects and registry to canonical clawhub host

* feat: auto-generate missing skill summaries

* fix: make skill summary backfill resumable

* feat: add self-scheduling skill summary backfill job

* perf: short-circuit empty skill summary generation

* style: polish upload page layout and actions

* feat: show popular non-suspicious skills on homepage

* fix: normalize legacy skill stats to prevent homepage crash

* fix: render homepage popular cards from nested skill entries

* style: refine global UI theme, borders, and spacing

* fix: resolve search timeout and improve skills page UI alignment (#53)

* fix: resolve search timeout and improve skills page UI alignment

- Added a 10s timeout to OpenAI embedding requests to prevent hanging searches.
- Fixed a TypeScript error in search.ts regarding entry hydration.
- Restructured skills page layout and CSS to ensure consistent alignment between the search toolbar and skill cards.

* fix: resolve search timeout and improve skills page UI alignment

- Added a 10s timeout to OpenAI embedding requests to prevent hanging searches.
- Fixed a TypeScript error in search.ts regarding entry hydration.
- Restructured skills page layout and CSS to ensure consistent alignment between the search toolbar and skill cards.

* style: format skills index layout block

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* docs: thank @GhadiSaab for #53

* style: shift UI palette to cool blue tones

* style: remove remaining warm accent literals

* style: darken hero primary CTA in dark mode

* fix: show stars in popular skill cards

* fix: simplify skills CTA label

* fix: dedupe download metrics hourly by user-or-ip identity (#278)

* style: restore brown palette and dark-mode CTA tone

* fix(comments): stop updating skills.updatedAt on comment add/remove (#55)

* fix(comments): stop updating skills.updatedAt on comment add/remove

Comments are not content changes, so they shouldn't invalidate skill
list queries that depend on updatedAt. This reduces query invalidation
when users add or remove comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(comments): add updatedAt invalidation regression coverage

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* refactor(comments): extract handlers and harden mutation tests

* feat: make account deletion irreversible and migrate lint to oxlint

* chore: add oxfmt config

* fix(cli): throw Error for all timeout aborts (#283)

* fix(cli): throw Error on timeout aborts

Users have seen an elevated number of:\n  clawdhub search image\n  ✖ Non-error was thrown: "Timeout". You should only throw errors.\n\nInvestigation shows we were aborting with a string instead of an Error. Switching to controller.abort(new Error('Timeout')) makes retries/formatting treat it as a real error and clears the message.\n\nExample after change:\n  clawdhub search image\n  table-image v1.0.0  Table Image  (0.332)\n  nano-banana-pro v1.0.1  Nano Banana Pro  (0.319)\n  vap-media v1.0.1  AI media generation API - Flux2pro, Veo3.1, Suno Ai  (0.281)\n  clawdbot-meshyai-skill v0.1.0  Meshy AI  (0.276)\n  venice-ai-media v1.0.0  Venice AI Media  (0.274)\n  daily-recap v1.0.2  Daily Recap  (0.260)\n  openai-image-gen v1.0.1  Openai Image Gen  (0.260)\n  bible-votd v1.0.1  Bible Verse of the Day  (0.248)\n  orf v1.0.1  ORF  (0.224)\n  smalltalk v1.0.1  Smalltalk  (0.161)

* fix(http): wrap fetch calls in try-finally to prevent timer leaks

Addresses Vercel review comment: clearTimeout was not called on error paths when fetch throws an exception.

* fix(cli): unify timeout abort handling

---------

Co-authored-by: Sash Zats <sash@zats.io>

* refactor(cli): centralize HTTP status errors and timeout tests (#286)

* fix: keep new skill versions pending until VT verdict

* style: remove residual blue accents and warm base palette

* fix: add retry logic for OpenAI embedding API failures (#272)

* fix: add retry logic for OpenAI embedding API failures

Fixes #149

When importing or uploading skills, the OpenAI embedding API call could
fail with transient errors (rate limits, timeouts, network issues),
causing the entire import to fail with a generic "Server Error".

This adds retry logic with exponential backoff (1s, 2s, 4s delays):
- Retries on 429 (rate limit) and 5xx server errors
- Retries on network/fetch errors
- Logs warnings for debugging
- Max 3 retries before failing with clear error message

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct retry count and broaden network error catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address retry loop off-by-one, broaden error catch, preserve original error

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden embeddings retry semantics

* style: format embeddings retry changes

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* fix: sync handle on user ensure

* fix: sync handle on user ensure (#293) (thanks @christianhpoe)

* feat: improve moderation/admin UX + language-aware quality gate

- API: owner-visible responses for hidden/soft-deleted skills\n- Admin: add unban user mutations + docs\n- Quality: Intl.Segmenter tokenization + CJK signal to reduce false rejects\n- Jobs: skill-stat-events interval 15m -> 5m\n- Tests: add coverage for owner-visible states + non-Latin docs\n- Changelog: add Unreleased entry

* refactor: simplify user ensure updates

* fix(cors): complete CORS + tokenized CLI reads (#296)

* fix(cors): add Access-Control-Allow-Origin headers to API and downloads

* fix: add CORS to error/raw paths & add CLI install auth

* fix: add OPTIONS handler for CORS preflight

* fix(cors): complete CORS + tokenized CLI reads

* test(cli): fix config mock typing

---------

Co-authored-by: Grenghis-Khan <63885013+Grenghis-Khan@users.noreply.github.com>

* refactor: centralize CORS + CLI auth token (#297)

* refactor(convex): centralize CORS headers

* refactor(cli): centralize auth token lookup

* fix(skills): keep global sorting across pagination (#98)

* fix: initial skill sorting

* chore: update unit test

* fix: use correct indexes for skill sorting

* chore: cleanup

* fix(skills): preserve server order for paginated sorting

* chore(lint): apply biome formatting fixes

* chore(convex): bump tsconfig lib to ES2022

* fix(skills): add deterministic tie-breaker for search sorting

* fix(skills): stable sorting across pagination (#98) (thanks @CodeBBakGoSu)

---------

Co-authored-by: Brian Kasper <bkasperr@gmail.com>
Co-authored-by: knox-glorang <knox@glorang.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* chore: drop convex-helpers (#302)

* perf: batch tag resolution to reduce action→query round-trips

- Add getVersionsByIds batch query to skills.ts and souls.ts
- Replace per-item tag resolution with batch resolution in httpApiV1.ts
- Reduces N action→query round-trips to 1 for list endpoints

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add null guard and short-circuit for empty tags

- Short-circuit when no version IDs to resolve
- Add null coalescing for runQuery response
- Fixes potential crash when tags are empty or query returns null

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: batch resolve tags in v1 API (#112) (thanks @mkrokosz)

* fix: handle duplicate Convex Auth user records in publish ownership check (#180)

* fix: handle duplicate user records in publish ownership check

* fix: heal publish ownership via GitHub auth identity

---------

Co-authored-by: Emmet Brown <emmet@Emmets-Mac-mini.local>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* fix: gate publish by immutable GitHub account ID

* refactor: simplify GitHub age gate cache

* fix(api): centralize v1 soft-delete error mapping

* chore(cli): align http client with main

* test(api): cover v1 soft-delete error mapping

* test(api): reposition soft-delete mapping test

* fix: default to CF-only client IP parsing

* docs: changelog credit + v1 delete status codes

* fix(cli): clarify logout only affects local config (#166)

* fix(cli): clarify logout only affects local config

Users may assume 'clawhub logout' revokes their token everywhere.
In reality, the token remains valid on the server until explicitly
revoked in the web UI. This could be a security concern on shared
machines.

Update the message to set correct expectations.

* fix(cli): clarify logout revocation scope (#166) (thanks @aronchick)

* chore: sync changelog for merge (#166) (thanks @aronchick)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>

* feat: anti-squatting protection, backup restore, and ban flow improvements (#298)

* feat: anti-squatting protection, backup restore, and ban flow improvements

- Add `reservedSlugs` table with 90-day cooldown to prevent slug squatting
  after skill deletion. Hard-delete finalize phase reserves slugs for the
  original owner; `insertVersion` blocks non-owners during cooldown.

- Change ban flow from hard-delete to soft-delete: `banUserWithActor` now
  sets `moderationReason: 'user.banned'` and syncs embedding visibility.
  `unbanUserWithActor` restores all ban-hidden skills and releases slug
  reservations automatically.

- Align `autobanMalwareAuthorInternal` with the same soft-delete + embedding
  visibility pattern so unban recovery works uniformly.

- Add admin `reclaimSlug` / `reclaimSlugInternal` mutations for reclaiming
  squatted slugs, with audit logging.

- Add GitHub backup restore system (`githubRestore.ts`,
  `githubRestoreMutations.ts`, `githubRestoreHelpers.ts`) that reads from
  the `clawdbot/skills` backup repo and re-creates skill records. Squatter
  eviction runs synchronously in the same transaction as restore to avoid
  async race conditions.

- Add `POST /api/v1/users/restore` and `POST /api/v1/users/reclaim` admin
  HTTP endpoints for bulk operations.

- Add `trustedPublisher` flag on users; trusted publishers bypass the
  `pending.scan` auto-hide for new skill publishes.

- Add `setTrustedPublisher` / `setTrustedPublisherInternal` admin mutations.

Addresses: slug squatting prevention, skill backup/restore, ban recovery,
and trusted publisher workflow improvements.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: harden restore/reclaim + ban flow (#298) (thanks @autogame-17)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* refactor: post-#298 cleanup (#313)

* refactor: consolidate slug + embedding helpers

* refactor: batch ban/unban skill updates

* refactor: report batched ban/unban scheduling

* fix: unblock package typecheck

* refactor: split httpApiV1 + consolidate moderation batches (#315)

* refactor: dedupe v1 file response + unify embedding patches (#316)

* Devin/1771112524 skill metadata update (#312)

* fix: sync GitHub profile on login to handle username renames (#303)

When a user renames their GitHub account, the stored username becomes stale
and causes 'GitHub account lookup failed' errors during skill publishing.

This fix:
- Adds syncGitHubProfile function that fetches current profile using the
  immutable GitHub numeric ID
- Adds syncGitHubProfileInternal mutation to update user's name, handle,
  displayName, and image when they change
- Schedules the sync as a background action on every login via
  afterUserCreatedOrUpdated callback

The sync is best-effort (silently fails if GitHub API unavailable) since
it's not on the critical path. It only updates fields if the username
has actually changed.

Fixes #303

Co-Authored-By: Ian Alloway <adapter_burners.1y@icloud.com>

* fix: allow updating skill summary/description on subsequent publishes (#301)

Previously, the skill summary was only extracted from metadata.description
in the SKILL.md frontmatter. This change also checks for a direct
'description' field in the frontmatter, ensuring that users can update
their skill description by modifying either location.

The fix prioritizes the new description from the current publish over
the existing skill summary, allowing updates to be reflected correctly.

Fixes #301

Co-Authored-By: Ian Alloway <adapter_burners.1y@icloud.com>

* fix: throttle GitHub profile sync

* feat: show skill owner avatars

* fix: avoid nested owner links

* refactor: centralize profile sync + owner lookup

* docs: changelog for #312 (thanks @ianalloway)

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* style: polish markdown code blocks

* feat: show skill owner avatars on home + lists

* feat: sync GitHub profile name

* feat: improve skill card meta layout

* fix: make ghost buttons look like buttons

* fix: match skill hero cta widths

* fix: prefer $HOME over os.homedir() for path resolution (#299)

* fix: prefer $HOME over os.homedir() for path resolution

os.homedir() reads from /etc/passwd which can return a stale path
after a Linux user rename (usermod -l). Prefer the $HOME environment
variable which reflects the current session.

Closes #82

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: normalize resolveHome output

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* UI: allow copying security scan summary text (#322)

* fix(ui): prevent analysis toggle when selecting summary (#324)

* feat: add uninstall command for skills (#241)

* feat: add uninstall command for skills

Implements `clawhub uninstall <slug>` to properly remove installed skills.

Changes:
- Added cmdUninstall function in skills.ts
- Validates skill is installed before removal
- Removes skill directory and lockfile entry
- Supports --yes flag to skip confirmation prompt
- Added comprehensive test coverage

Closes #221

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: require --yes in non-interactive mode and update lockfile before rm

Address review feedback:
- Fail with "Pass --yes (no input)" when running non-interactively
  without --yes flag, matching delete/star/unstar/moderation commands
- Update lockfile before removing directory to avoid inconsistent state
  if rm succeeds but writeLockfile fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden skill uninstall flow (#241) (thanks @superlowburn)

* docs: document uninstall CLI command (#241) (thanks @superlowburn)

* test: fix cmdUninstall mock typing (#241) (thanks @superlowburn)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>

* feat: add skill file viewer

* fix: prevent file viewer state updates after unmount

* fix: lazy-load skill file viewer (#44) (thanks @regenrek)

---------

Co-authored-by: Sergiy Dybskiy <s@serg.tech>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: theonejvo <theonejvo@users.noreply.github.com>
Co-authored-by: Vignesh <vigneshnatarajan92@gmail.com>
Co-authored-by: Steve <superlowburn@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: DColl <david.coll.78@gmail.com>
Co-authored-by: Tanuj Bhaud <tanujbhaud@gmail.com>
Co-authored-by: Limitless <127183162+Limitless2023@users.noreply.github.com>
Co-authored-by: Limitless2023 <limitless@users.noreply.github.com>
Co-authored-by: Gaurav Sharma <sharmag@microsoft.com>
Co-authored-by: xcqtnr <xcqtnr0.0@gmail.com>
Co-authored-by: David Aronchick <aronchick@gmail.com>
Co-authored-by: Sash Zats <sash@zats.io>
Co-authored-by: Tanuj Bhaud <128238320+tanujbhaud@users.noreply.github.com>
Co-authored-by: Brian Kasper <brian@bkasper.com>
Co-authored-by: ghadi saab <ghadisaab21@gmail.com>
Co-authored-by: sethconvex <seth@convex.dev>
Co-authored-by: ChristianHPoe <chpoensgen@me.com>
Co-authored-by: Grenghis-Khan <63885013+Grenghis-Khan@users.noreply.github.com>
Co-authored-by: CodeBBakGoSu <127713112+CodeBBakGoSu@users.noreply.github.com>
Co-authored-by: Brian Kasper <bkasperr@gmail.com>
Co-authored-by: knox-glorang <knox@glorang.com>
Co-authored-by: Matthew Krokosz <mattkrokosz@gmail.com>
Co-authored-by: emmet-bot <emmet@universaleverything.io>
Co-authored-by: Emmet Brown <emmet@Emmets-Mac-mini.local>
Co-authored-by: autogame-17 <166480271+autogame-17@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Ian Alloway <adapter_burners.1y@icloud.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: CleanApp <165804662+borisolver@users.noreply.github.com>
2026-02-16 03:29:30 +01:00
Steve
652beef9c1
feat: add uninstall command for skills (#241)
* feat: add uninstall command for skills

Implements `clawhub uninstall <slug>` to properly remove installed skills.

Changes:
- Added cmdUninstall function in skills.ts
- Validates skill is installed before removal
- Removes skill directory and lockfile entry
- Supports --yes flag to skip confirmation prompt
- Added comprehensive test coverage

Closes #221

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: require --yes in non-interactive mode and update lockfile before rm

Address review feedback:
- Fail with "Pass --yes (no input)" when running non-interactively
  without --yes flag, matching delete/star/unstar/moderation commands
- Update lockfile before removing directory to avoid inconsistent state
  if rm succeeds but writeLockfile fails

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden skill uninstall flow (#241) (thanks @superlowburn)

* docs: document uninstall CLI command (#241) (thanks @superlowburn)

* test: fix cmdUninstall mock typing (#241) (thanks @superlowburn)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-15 16:03:06 +01:00
Peter Steinberger
146df7b166
fix(ui): prevent analysis toggle when selecting summary (#324) 2026-02-15 14:42:03 +01:00
CleanApp
8f23eb5ee8
UI: allow copying security scan summary text (#322) 2026-02-15 14:39:13 +01:00
Steve
30b263c27c
fix: prefer $HOME over os.homedir() for path resolution (#299)
* fix: prefer $HOME over os.homedir() for path resolution

os.homedir() reads from /etc/passwd which can return a stale path
after a Linux user rename (usermod -l). Prefer the $HOME environment
variable which reflects the current session.

Closes #82

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: normalize resolveHome output

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-15 14:26:00 +01:00
Peter Steinberger
10b704278a fix: match skill hero cta widths 2026-02-15 05:28:59 +01:00
Peter Steinberger
a289f9cbd9 fix: make ghost buttons look like buttons 2026-02-15 05:27:55 +01:00
Peter Steinberger
97d68a1be5 feat: improve skill card meta layout 2026-02-15 05:26:08 +01:00
Peter Steinberger
57e0d39cdc feat: sync GitHub profile name 2026-02-15 05:26:03 +01:00
Peter Steinberger
1c033868e7 feat: show skill owner avatars on home + lists 2026-02-15 05:06:23 +01:00
Peter Steinberger
4532366009 style: polish markdown code blocks 2026-02-15 05:00:18 +01:00
Ian Alloway
8e9fa44fc2
Devin/1771112524 skill metadata update (#312)
* fix: sync GitHub profile on login to handle username renames (#303)

When a user renames their GitHub account, the stored username becomes stale
and causes 'GitHub account lookup failed' errors during skill publishing.

This fix:
- Adds syncGitHubProfile function that fetches current profile using the
  immutable GitHub numeric ID
- Adds syncGitHubProfileInternal mutation to update user's name, handle,
  displayName, and image when they change
- Schedules the sync as a background action on every login via
  afterUserCreatedOrUpdated callback

The sync is best-effort (silently fails if GitHub API unavailable) since
it's not on the critical path. It only updates fields if the username
has actually changed.

Fixes #303

Co-Authored-By: Ian Alloway <adapter_burners.1y@icloud.com>

* fix: allow updating skill summary/description on subsequent publishes (#301)

Previously, the skill summary was only extracted from metadata.description
in the SKILL.md frontmatter. This change also checks for a direct
'description' field in the frontmatter, ensuring that users can update
their skill description by modifying either location.

The fix prioritizes the new description from the current publish over
the existing skill summary, allowing updates to be reflected correctly.

Fixes #301

Co-Authored-By: Ian Alloway <adapter_burners.1y@icloud.com>

* fix: throttle GitHub profile sync

* feat: show skill owner avatars

* fix: avoid nested owner links

* refactor: centralize profile sync + owner lookup

* docs: changelog for #312 (thanks @ianalloway)

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-15 04:55:49 +01:00
Peter Steinberger
11a66ea148
refactor: dedupe v1 file response + unify embedding patches (#316) 2026-02-15 03:43:30 +01:00
Peter Steinberger
f94e20d4c3
refactor: split httpApiV1 + consolidate moderation batches (#315) 2026-02-15 03:29:53 +01:00
Peter Steinberger
71c74f61e2
refactor: post-#298 cleanup (#313)
* refactor: consolidate slug + embedding helpers

* refactor: batch ban/unban skill updates

* refactor: report batched ban/unban scheduling

* fix: unblock package typecheck
2026-02-15 02:18:54 +01:00
autogame-17
e2592684ed
feat: anti-squatting protection, backup restore, and ban flow improvements (#298)
* feat: anti-squatting protection, backup restore, and ban flow improvements

- Add `reservedSlugs` table with 90-day cooldown to prevent slug squatting
  after skill deletion. Hard-delete finalize phase reserves slugs for the
  original owner; `insertVersion` blocks non-owners during cooldown.

- Change ban flow from hard-delete to soft-delete: `banUserWithActor` now
  sets `moderationReason: 'user.banned'` and syncs embedding visibility.
  `unbanUserWithActor` restores all ban-hidden skills and releases slug
  reservations automatically.

- Align `autobanMalwareAuthorInternal` with the same soft-delete + embedding
  visibility pattern so unban recovery works uniformly.

- Add admin `reclaimSlug` / `reclaimSlugInternal` mutations for reclaiming
  squatted slugs, with audit logging.

- Add GitHub backup restore system (`githubRestore.ts`,
  `githubRestoreMutations.ts`, `githubRestoreHelpers.ts`) that reads from
  the `clawdbot/skills` backup repo and re-creates skill records. Squatter
  eviction runs synchronously in the same transaction as restore to avoid
  async race conditions.

- Add `POST /api/v1/users/restore` and `POST /api/v1/users/reclaim` admin
  HTTP endpoints for bulk operations.

- Add `trustedPublisher` flag on users; trusted publishers bypass the
  `pending.scan` auto-hide for new skill publishes.

- Add `setTrustedPublisher` / `setTrustedPublisherInternal` admin mutations.

Addresses: slug squatting prevention, skill backup/restore, ban recovery,
and trusted publisher workflow improvements.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: harden restore/reclaim + ban flow (#298) (thanks @autogame-17)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-15 01:16:46 +01:00
David Aronchick
79c9381201
fix(cli): clarify logout only affects local config (#166)
* fix(cli): clarify logout only affects local config

Users may assume 'clawhub logout' revokes their token everywhere.
In reality, the token remains valid on the server until explicitly
revoked in the web UI. This could be a security concern on shared
machines.

Update the message to set correct expectations.

* fix(cli): clarify logout revocation scope (#166) (thanks @aronchick)

* chore: sync changelog for merge (#166) (thanks @aronchick)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 22:56:10 +01:00
Peter Steinberger
6a5712fdb6
Merge pull request #309 from openclaw/chore/merge-all
docs: changelog credit + v1 delete status codes
2026-02-14 22:21:56 +01:00
Peter Steinberger
a85faf76ac docs: changelog credit + v1 delete status codes 2026-02-14 22:21:37 +01:00
Peter Steinberger
c0a04210e9 fix: default to CF-only client IP parsing 2026-02-14 22:20:58 +01:00
Peter Steinberger
5adb334cb2
Merge pull request #35 from sergical/fix/delete-error-handling
fix: return proper HTTP status codes for delete/undelete errors
2026-02-14 22:11:04 +01:00
Peter Steinberger
182ec8741f test(api): reposition soft-delete mapping test 2026-02-14 22:08:05 +01:00
Peter Steinberger
bafd17b00a test(api): cover v1 soft-delete error mapping 2026-02-14 22:06:02 +01:00
Peter Steinberger
802ee58054 chore(cli): align http client with main 2026-02-14 22:04:02 +01:00
Peter Steinberger
0e83ba00b9 fix(api): centralize v1 soft-delete error mapping 2026-02-14 21:59:14 +01:00
Peter Steinberger
3326a5c838
refactor: simplify GitHub age gate cache 2026-02-14 20:53:05 +01:00
Matt Krokosz
f05dd556db
fix: gate publish by immutable GitHub account ID 2026-02-14 20:25:15 +01:00
emmet-bot
964893a622
fix: handle duplicate Convex Auth user records in publish ownership check (#180)
* fix: handle duplicate user records in publish ownership check

* fix: heal publish ownership via GitHub auth identity

---------

Co-authored-by: Emmet Brown <emmet@Emmets-Mac-mini.local>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 19:39:04 +01:00
Peter Steinberger
9a804b951f refactor: batch resolve tags in v1 API (#112) (thanks @mkrokosz) 2026-02-14 19:01:02 +01:00
Matthew Krokosz
d699087786 fix: add null guard and short-circuit for empty tags
- Short-circuit when no version IDs to resolve
- Add null coalescing for runQuery response
- Fixes potential crash when tags are empty or query returns null

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 19:01:02 +01:00
Matthew Krokosz
26b42727fe perf: batch tag resolution to reduce action→query round-trips
- Add getVersionsByIds batch query to skills.ts and souls.ts
- Replace per-item tag resolution with batch resolution in httpApiV1.ts
- Reduces N action→query round-trips to 1 for list endpoints

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-14 19:01:02 +01:00
Peter Steinberger
8756b78a4e
chore: drop convex-helpers (#302) 2026-02-14 17:54:24 +01:00
CodeBBakGoSu
e9c771d55d
fix(skills): keep global sorting across pagination (#98)
* fix: initial skill sorting

* chore: update unit test

* fix: use correct indexes for skill sorting

* chore: cleanup

* fix(skills): preserve server order for paginated sorting

* chore(lint): apply biome formatting fixes

* chore(convex): bump tsconfig lib to ES2022

* fix(skills): add deterministic tie-breaker for search sorting

* fix(skills): stable sorting across pagination (#98) (thanks @CodeBBakGoSu)

---------

Co-authored-by: Brian Kasper <bkasperr@gmail.com>
Co-authored-by: knox-glorang <knox@glorang.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 17:44:07 +01:00
Peter Steinberger
a58f0166fa
refactor: centralize CORS + CLI auth token (#297)
* refactor(convex): centralize CORS headers

* refactor(cli): centralize auth token lookup
2026-02-14 15:31:03 +01:00
Peter Steinberger
4328d4d700
fix(cors): complete CORS + tokenized CLI reads (#296)
* fix(cors): add Access-Control-Allow-Origin headers to API and downloads

* fix: add CORS to error/raw paths & add CLI install auth

* fix: add OPTIONS handler for CORS preflight

* fix(cors): complete CORS + tokenized CLI reads

* test(cli): fix config mock typing

---------

Co-authored-by: Grenghis-Khan <63885013+Grenghis-Khan@users.noreply.github.com>
2026-02-14 14:45:15 +01:00
Peter Steinberger
28ee2618c1 refactor: simplify user ensure updates 2026-02-14 13:56:15 +01:00
Peter Steinberger
a4b850ec33 feat: improve moderation/admin UX + language-aware quality gate
- API: owner-visible responses for hidden/soft-deleted skills\n- Admin: add unban user mutations + docs\n- Quality: Intl.Segmenter tokenization + CJK signal to reduce false rejects\n- Jobs: skill-stat-events interval 15m -> 5m\n- Tests: add coverage for owner-visible states + non-Latin docs\n- Changelog: add Unreleased entry
2026-02-14 13:54:03 +01:00
Peter Steinberger
7e0b21f7c8 fix: sync handle on user ensure (#293) (thanks @christianhpoe) 2026-02-14 13:48:56 +01:00
ChristianHPoe
71c6705ab1 fix: sync handle on user ensure 2026-02-14 13:48:56 +01:00
Steve
a57769771f
fix: add retry logic for OpenAI embedding API failures (#272)
* fix: add retry logic for OpenAI embedding API failures

Fixes #149

When importing or uploading skills, the OpenAI embedding API call could
fail with transient errors (rate limits, timeouts, network issues),
causing the entire import to fail with a generic "Server Error".

This adds retry logic with exponential backoff (1s, 2s, 4s delays):
- Retries on 429 (rate limit) and 5xx server errors
- Retries on network/fetch errors
- Logs warnings for debugging
- Max 3 retries before failing with clear error message

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct retry count and broaden network error catch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address retry loop off-by-one, broaden error catch, preserve original error

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: harden embeddings retry semantics

* style: format embeddings retry changes

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 04:54:18 +01:00
Peter Steinberger
6a2c131a8a style: remove residual blue accents and warm base palette 2026-02-14 04:40:50 +01:00
Peter Steinberger
67ac157545 fix: keep new skill versions pending until VT verdict 2026-02-14 02:53:02 +01:00
Peter Steinberger
ef36cfd698
refactor(cli): centralize HTTP status errors and timeout tests (#286) 2026-02-14 02:39:42 +01:00
Peter Steinberger
e0637ad6aa
fix(cli): throw Error for all timeout aborts (#283)
* fix(cli): throw Error on timeout aborts

Users have seen an elevated number of:\n  clawdhub search image\n  ✖ Non-error was thrown: "Timeout". You should only throw errors.\n\nInvestigation shows we were aborting with a string instead of an Error. Switching to controller.abort(new Error('Timeout')) makes retries/formatting treat it as a real error and clears the message.\n\nExample after change:\n  clawdhub search image\n  table-image v1.0.0  Table Image  (0.332)\n  nano-banana-pro v1.0.1  Nano Banana Pro  (0.319)\n  vap-media v1.0.1  AI media generation API - Flux2pro, Veo3.1, Suno Ai  (0.281)\n  clawdbot-meshyai-skill v0.1.0  Meshy AI  (0.276)\n  venice-ai-media v1.0.0  Venice AI Media  (0.274)\n  daily-recap v1.0.2  Daily Recap  (0.260)\n  openai-image-gen v1.0.1  Openai Image Gen  (0.260)\n  bible-votd v1.0.1  Bible Verse of the Day  (0.248)\n  orf v1.0.1  ORF  (0.224)\n  smalltalk v1.0.1  Smalltalk  (0.161)

* fix(http): wrap fetch calls in try-finally to prevent timer leaks

Addresses Vercel review comment: clearTimeout was not called on error paths when fetch throws an exception.

* fix(cli): unify timeout abort handling

---------

Co-authored-by: Sash Zats <sash@zats.io>
2026-02-14 02:27:34 +01:00
Peter Steinberger
09a21a07ff chore: add oxfmt config 2026-02-14 02:15:12 +01:00
Peter Steinberger
65a14dcef3 feat: make account deletion irreversible and migrate lint to oxlint 2026-02-14 02:15:01 +01:00
Peter Steinberger
97c12b2327 refactor(comments): extract handlers and harden mutation tests 2026-02-14 01:56:34 +01:00
sethconvex
a290c81a75
fix(comments): stop updating skills.updatedAt on comment add/remove (#55)
* fix(comments): stop updating skills.updatedAt on comment add/remove

Comments are not content changes, so they shouldn't invalidate skill
list queries that depend on updatedAt. This reduces query invalidation
when users add or remove comments.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(comments): add updatedAt invalidation regression coverage

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 01:53:40 +01:00
Peter Steinberger
cc5d5cfee5 style: restore brown palette and dark-mode CTA tone 2026-02-14 01:18:29 +01:00
Peter Steinberger
03cd710abc
fix: dedupe download metrics hourly by user-or-ip identity (#278) 2026-02-14 01:13:52 +01:00
Peter Steinberger
287f639fbc fix: simplify skills CTA label 2026-02-14 00:56:52 +01:00
Peter Steinberger
aecf66981c fix: show stars in popular skill cards 2026-02-14 00:55:24 +01:00
Peter Steinberger
db8090f287 style: darken hero primary CTA in dark mode 2026-02-14 00:50:52 +01:00
Peter Steinberger
ae8614fa98 style: remove remaining warm accent literals 2026-02-14 00:38:21 +01:00
Peter Steinberger
e7f78ea5a3 style: shift UI palette to cool blue tones 2026-02-14 00:31:55 +01:00
Peter Steinberger
b9355f7a0c docs: thank @GhadiSaab for #53 2026-02-14 00:16:24 +01:00
ghadi saab
9530676f8a
fix: resolve search timeout and improve skills page UI alignment (#53)
* fix: resolve search timeout and improve skills page UI alignment

- Added a 10s timeout to OpenAI embedding requests to prevent hanging searches.
- Fixed a TypeScript error in search.ts regarding entry hydration.
- Restructured skills page layout and CSS to ensure consistent alignment between the search toolbar and skill cards.

* fix: resolve search timeout and improve skills page UI alignment

- Added a 10s timeout to OpenAI embedding requests to prevent hanging searches.
- Fixed a TypeScript error in search.ts regarding entry hydration.
- Restructured skills page layout and CSS to ensure consistent alignment between the search toolbar and skill cards.

* style: format skills index layout block

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-14 00:15:47 +01:00
Peter Steinberger
ef23520d22 style: refine global UI theme, borders, and spacing 2026-02-13 23:50:24 +01:00
Peter Steinberger
e67a6e6400 fix: render homepage popular cards from nested skill entries 2026-02-13 22:52:21 +01:00
Peter Steinberger
e6871b86e1 fix: normalize legacy skill stats to prevent homepage crash 2026-02-13 22:45:34 +01:00
Peter Steinberger
75937e8b53 feat: show popular non-suspicious skills on homepage 2026-02-13 21:38:56 +01:00
Peter Steinberger
d2919791d1 style: polish upload page layout and actions 2026-02-13 21:35:25 +01:00
Peter Steinberger
9019cd8462 perf: short-circuit empty skill summary generation 2026-02-13 21:21:09 +01:00
Peter Steinberger
5e58bd459e feat: add self-scheduling skill summary backfill job 2026-02-13 21:10:51 +01:00
Peter Steinberger
3badf0668f fix: make skill summary backfill resumable 2026-02-13 20:43:46 +01:00
Peter Steinberger
c719297d70 feat: auto-generate missing skill summaries 2026-02-13 20:35:03 +01:00
Peter Steinberger
e19cd23be2 fix: force auth redirects and registry to canonical clawhub host 2026-02-13 20:22:08 +01:00
Peter Steinberger
9266fb7c20 fix: add privileged-owner suspicious flag reconciler 2026-02-13 20:01:39 +01:00
Peter Steinberger
99645d2c27 fix: bypass suspicious flags for privileged owners and polish comment delete UI 2026-02-13 19:56:53 +01:00
Peter Steinberger
a1ad7fac85 fix: force canonical downloads sort in skills browse mode 2026-02-13 19:33:26 +01:00
Peter Steinberger
ebd2f12cc4 fix: enforce downloads as canonical default skills sort 2026-02-13 19:27:27 +01:00
Peter Steinberger
e2ee7b164c feat: default skills sort to downloads 2026-02-13 19:15:19 +01:00
Peter Steinberger
36ed062739 style: polish selected states in skills toolbar 2026-02-13 19:12:48 +01:00
Peter Steinberger
d12d6e3926 feat: add non-suspicious skills filter toggle 2026-02-13 19:05:46 +01:00
Peter Steinberger
1851a9c01f fix: make empty-skill cleanup resumable 2026-02-13 17:54:47 +01:00
Peter Steinberger
bbeb0be343 feat: add empty-skill cleanup backfill with ban nominations 2026-02-13 17:48:58 +01:00
Peter Steinberger
9c22fb7e54 test: expand reauth ban regression coverage 2026-02-13 17:33:58 +01:00
Peter Steinberger
ef2403179b fix: prevent autobanned users from self-reactivating 2026-02-13 17:28:00 +01:00
Peter Steinberger
df178d4bfc fix: enforce quality gate and trust-tier spam checks 2026-02-13 17:16:53 +01:00
Peter Steinberger
318cdd33c5 docs: add git local-branch cleanup fallback 2026-02-13 17:03:38 +01:00
Peter Steinberger
9087b037dd fix: add skill publish anti-spam caps and quarantine 2026-02-13 16:58:18 +01:00
Peter Steinberger
6991569a1c fix: replace skill report prompt with modal 2026-02-13 16:47:16 +01:00
Peter Steinberger
32bc600be4 fix: harden skill listing and rate limiting under load 2026-02-13 16:39:57 +01:00
Kevin Kern
a52a37d08c
fix: harden download rate limiting and dedupe (#43) (thanks @regenrek)
- add download-specific rate limit tier\n- add per-IP/day dedupe + daily pruning\n- keep moderation gating + deterministic zips\n- add optional forwarded-IP trust via TRUST_FORWARDED_IPS
2026-02-13 16:22:05 +01:00
Brian Kasper
dd58dd0815
Fix initial skill sorting (#92)
* fix: initial skill sorting

* chore: update unit test

* fix: use correct indexes for skill sorting

* chore: cleanup

* fix: land skill sorting update (#92) (thanks @bpk9)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 16:01:37 +01:00
Tanuj Bhaud
37a35c955a
fix(vt): explicit return types and missing undici dependency (#255)
* fix(vt): explicit return types and missing undici dependency

Refactor action handlers in convex/vt.ts to use explicit return types, resolving circular type inference (TS7022). Also add undici to devDependencies for E2E tests.

* fix: add root undici devDependency for e2e (#255) (thanks @tanujbhaud)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 15:25:03 +01:00
Peter Steinberger
ddddb431c2
fix: make /search host-aware in SSR (#257)
* fix: make /search mode-aware

Notes:\n- Medium: /search now depends on getSiteMode() during beforeLoad. On server-side routing, if VITE_SITE_MODE isn’t set and VITE_SOULHUB_SITE_URL is set (as in .env.local), getSiteMode() will resolve to souls and redirect /search to / even on the ClawdHub deployment. This is a regression risk vs the old always-/skills redirect. Confirm deployment envs guarantee correct mode. src/routes/search.tsx:9-31

* fix: make /search host-aware in SSR

* chore: fix lint and route tree for /search route

---------

Co-authored-by: Sash Zats <sash@zats.io>
2026-02-13 14:26:31 +01:00
David Aronchick
78c27579a3
fix(cli): secure config file permissions (#164)
* fix(cli): secure config file permissions and reduce duplication

Security:
- Config files now created with 0600 permissions (owner read/write only)
- Config directories created with 0700 permissions
- Protects API tokens from other users on shared systems

Maintainability:
- Extract resolveConfigPath() helper to reduce code duplication
- Same legacy fallback logic (clawhub -> clawdhub) now in one place

* fix(cli): tolerate unsupported chmod errors for config

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 13:57:03 +01:00
xcqtnr
9aebc35d86
fix: prevent infinite loading loop on skills page (#90)
* fix: prevent infinite loading loop on skills pageAdd isLoadingMore guard to IntersectionObserver useEffect to preventcontinuous WebSocket queries when user is idle at bottom of page.The observer now won't set up while a request is in progress, breakingthe infinite loop cycle.Fixes: Related to #89

* fix: prevent repeated skills auto-load requests (#90) (thanks @xcqtnr)

* fix: resolve PR merge conflicts and keep observer regression test (#90) (thanks @xcqtnr)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 05:44:36 +01:00
Peter Steinberger
fc63f47ffa chore(release): 0.6.1 2026-02-13 05:14:16 +01:00
Gaurav Sharma
0b83ea6ff3
fix: prevent horizontal overflow from long code blocks in skill pages (#183)
* Fix: Prevent horizontal overflow from long code blocks in skill pages

- Add max-width: 100% to .file-list-body and .file-row
- Prevents page-wide overflow when skills contain long code examples
- Markdown pre blocks already have overflow-x: auto, but parent containers were expanding infinitely
- Fixes issue where skills with 400+ char lines (e.g. browser automation commands) cause horizontal scrolling

Affected: Skills with long inline code in markdown (browser act commands, etc.)

* fix: add max-width to .file-list container to prevent overflow

- Also ensures .file-list-body constraint is inherited properly
- Prevents long code blocks from expanding file list container

* fix: add max-width to all markdown containers and pre tags

- Add max-width: 100% to .markdown, .tab-body, .markdown pre
- Ensures code blocks are constrained and show horizontal scrollbar
- Prevents content from expanding parent containers beyond viewport

* fix: add overflow-x to parent containers for horizontal scroll

Adds overflow-x: auto to .skill-detail-stack, .tab-card, and .tab-body
to ensure long code blocks are scrollable within the content area
instead of causing page-wide horizontal overflow.

Fixes horizontal overflow issue on skill pages with long code examples
(e.g., browser automation commands with 400+ character lines).

Tested on zepto skill page - page now stays within viewport (1200px)
and code blocks are accessible via horizontal scrollbar in tab area.

* docs: note code-block overflow fix in changelog (#183) (thanks @bewithgaurav)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 05:09:31 +01:00
Limitless
191b5763ec
fix: include comment deltas in action-based stat processing & add stats reconciliation (#194)
Bug 1: applyAggregatedStatsAndUpdateCursor was missing 'comments' in both
the guard condition and the applySkillStatDeltas call. This caused comment
count deltas to be silently dropped during cron-based event processing,
while stars/downloads/installs were processed correctly.

Bug 2: No reconciliation mechanism existed. If events were missed due to
cursor issues or processing errors, skill stats (stars, comments) would
remain stale with no way to recover. Added reconcileSkillStarCounts
maintenance mutation that counts actual records in the stars and comments
tables and patches any out-of-sync skill stats.

Fixes #193

Co-authored-by: Limitless2023 <limitless@users.noreply.github.com>
2026-02-13 04:55:16 +01:00
Peter Steinberger
5397a8e5e0 fix: scope reauth fix; keep banned users blocked (#177) (thanks @tanujbhaud) 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
7949402888 test: add missing coverage for fresh-login reactivation and identity mismatch guard 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
2e3920d41a fix: use valid crons.interval and set to 1 minute 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
dd4fc823f6 fix: allow re-auth when existingUserId is null 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
135b9ea9b0 fix: ensure reactivation only matches soft-deleted user (prevents bypass) 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
17a106cefe fix: resolve final lint error in auth tests 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
496da99392 fix: update tests to include required existingUserId parameter 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
107486adfb fix: restore existingUserId check for type safety 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
ef9e7f0e57 test: update auth tests for direct deletedAt check 2026-02-13 04:34:46 +01:00
Tanuj Bhaud
ecc8ad3833 fix: allow soft-deleted users to re-authenticate
Fixes Issue #32 where users who soft-deleted their accounts were unable to sign back in because the re-auth logic was only triggering when an existingUserId was passed by the auth provider, which doesn't happen during a standard fresh login flow.
2026-02-13 04:34:46 +01:00
Peter Steinberger
93c2b23b72 docs: add 0.6.1 unreleased changelog from post-0.6.0 commits 2026-02-13 04:14:16 +01:00
DColl
2e492a5b87
fix(http): remove allowH2 from undici Agent — causes fetch failed on Node.js 22+ (#245)
* Remove allowH2 option from global dispatcher

fix/remove-allowH2-undici-node22-compat

* fix(http): remove allowH2 from e2e dispatcher

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 03:02:57 +01:00
Peter Steinberger
91c87d1322 test: fix search test handler typing 2026-02-13 02:48:16 +01:00
Peter Steinberger
91b8f160f8 test: add search fallback coverage 2026-02-13 02:44:53 +01:00
Peter Steinberger
32f3ce45e9 fix: add lexical fallback for skill search recall 2026-02-13 02:22:32 +01:00
Peter Steinberger
19bfe48a67 fix: prioritize relevant skills in search 2026-02-13 02:15:19 +01:00
Peter Steinberger
19951fccf7 docs: thank @superlowburn for PR #246 2026-02-13 01:22:38 +01:00
Steve
7dcada9122
fix: handle GitHub API rate limits in account age check (#246)
* fix: handle GitHub API rate limits in account age check

The GitHub account lookup uses unauthenticated requests (60 req/hr
per IP). Since this runs server-side in Convex, all users share the
same IP and quickly exhaust the rate limit, causing "GitHub account
lookup failed" errors during skill publish.

- Detect 403/429 responses and surface a clear rate-limit message
- Support optional GITHUB_TOKEN env var for authenticated requests
  (5,000 req/hr)

Fixes #155

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: stabilize GitHub account gate tests and docs

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-13 01:21:30 +01:00
theonejvo
dab307cb6d fix: VT scan sync race condition + LLM-first moderation model
VT no longer overwrites LLM moderation verdicts. LLM is the primary
moderation authority; VT only escalates (hides + flags) for malicious/
suspicious content via new escalateByVtInternal mutation. Stale VT polls
write vtAnalysis marker instead of overwriting moderationReason. Query
pools expanded to include LLM-evaluated skills awaiting VT results.
Ban message now references malicious skills and security@openclaw.ai.
2026-02-13 01:05:52 +11:00
vignesh07
e96eb4781c chore: fix review comments 2026-02-11 10:38:26 -08:00
Vignesh
f64b098fcc
perf: lazy-load diff viewer (Monaco) (#212) 2026-02-11 12:31:07 -06:00
Vignesh
243432e04e
chore: fix lint issues (#213) 2026-02-11 12:31:01 -06:00
theonejvo
9f7b9b92cd fix: trailing comma tolerance in JSON metadata, tone down persistence flags
- Strip trailing commas in frontmatter JSON before parsing (silent failure fix)
- Stop flagging disable-model-invocation default as a concern (it's the normal default)
- Stop flagging skills configuring themselves as privilege escalation
- Add MITRE ATLAS AML.T0051 context for when autonomous invocation actually matters
- Show actual defaults in assembled eval message instead of "not set"
2026-02-11 18:56:19 +11:00
theonejvo
3402f0e735 feat: add skill metadata docs, suspicious appeal banner for owners
- Document full frontmatter metadata reference in docs/skill-format.md
- Add metadata section + quick example to README
- Show appeal message on suspicious skills (owner-only) linking to GitHub issues
- Accept metadata.openclaw alias in README docs
- Re-evaluate all skills with full file content reading (backfill in progress)
2026-02-11 18:36:45 +11:00
theonejvo
3e39651074 feat: evaluator reads all file contents, not just SKILL.md
Reads all files from storage and includes their full source in the eval
prompt so the LLM can detect malicious code hidden behind clean READMEs.
Injection detection now scans all content. Per-file cap 10K chars, total
cap 50K chars.
2026-02-11 17:23:19 +11:00
theonejvo
741301848f fix: eval assembler falls back to metadata.openclaw for requirements 2026-02-11 14:39:02 +11:00
theonejvo
b593dedba1 feat: recognize metadata.openclaw as valid frontmatter namespace 2026-02-11 14:29:11 +11:00
theonejvo
e104b8030e fix: increase max_output_tokens for reasoning model, fix backfill error retry 2026-02-11 04:48:08 +11:00
theonejvo
e74e879e12 fix: add retry with backoff for OpenAI rate limits, fix JSON mode requirement 2026-02-11 02:48:22 +11:00
theonejvo
1f1d93ded9 fix: collapse OpenClaw analysis by default, fix row spacing, switch to gpt-5-mini 2026-02-11 02:33:17 +11:00
theonejvo
9c31462f15 feat: add LLM security evaluation at publish time
Add OpenClaw LLM-based security evaluator that runs alongside VirusTotal
when skills are published. Reads SKILL.md prose, metadata, install specs,
and file manifest, then assesses coherence across 5 dimensions to catch
social engineering vectors that VT/regex miss (e.g. instruction-only skills
with no code files).

- convex/lib/securityPrompt.ts: system prompt, message assembly, response
  parsing, injection pattern detection
- convex/llmEval.ts: evaluateWithLlm action, evaluateBySlug convenience
  action, backfillLlmEval for existing skills
- convex/schema.ts: llmAnalysis field on skillVersions
- convex/skills.ts: updateVersionLlmAnalysisInternal mutation,
  getActiveSkillBatchForLlmBackfillInternal query, defense-in-depth
  multi-scanner flag merging in approveSkillByHashInternal
- convex/lib/skillPublish.ts: schedule LLM eval alongside VT scan
- SkillDetailPage.tsx: OpenClaw row, LlmAnalysisDetail expandable
  component with 5 dimension rows, guidance panel, findings section
- styles.css: analysis detail styles from mockup
2026-02-11 02:19:03 +11:00
Peter Steinberger
f53be5c8d2 docs: reset changelog for next release 2026-02-10 13:22:09 +01:00
Peter Steinberger
03164fe8b1 chore: release 0.6.0 2026-02-10 13:20:16 +01:00
Peter Steinberger
81b53f0b20 feat: add ban reasons to moderation 2026-02-10 13:11:47 +01:00
theonejvo
75474075b1 fix(vt): self-sustaining VT scanning + working stats
- getStatsInternal: derive VT stats from moderationReason instead of
  N+1 version lookups that hit the 16MB byte limit
- UI: read cached vtAnalysis from version docs instead of hitting the
  live VT API on every page view
- Backfill: add vt-cache-backfill cron (30min) with self-scheduling to
  drain the backlog of skills missing cached vtAnalysis
- Daily rescan: cursor-based batching (100/batch) with self-scheduling
  instead of loading all skills in one shot
2026-02-10 20:39:27 +11:00
theonejvo
5946a08267 fix(convex): reduce write conflicts across hot paths
- downloads:increment: remove unnecessary db.get that added skill doc
  to read set, causing conflicts with the stat processing cron
- users:ensure: only patch when there are real field changes, skip
  unconditional updatedAt bump that forced a write on every call
- comments: route stats through event sourcing (insertStatEvent) instead
  of synchronous read-modify-write on the skill doc
- rateLimits: split into query-first check + conditional mutation so
  denied requests are conflict-free reads
- skillStatEvents: reduce MAX_SKILLS_PER_RUN from 500 to 50 to shrink
  the write set and lower conflict probability with concurrent mutations

Co-Authored-By: theonejvo <theonejvo@users.noreply.github.com>
2026-02-10 17:58:43 +11:00
theonejvo
b997f5e749 fix(security): block downloads for unscanned/moderated skills
The download endpoint now checks moderation status before serving zips:

- Pending scan (423): "This skill is pending a security scan by VirusTotal. Please try again in a few minutes."
- Malicious (403): "Blocked: this skill has been flagged as malicious by VirusTotal and cannot be downloaded."
- Removed (410): "This skill has been removed by a moderator."
- Hidden (403): "This skill is currently unavailable."

Closes the supply chain gap where a newly published version could be
downloaded before VT scanning completed.
2026-02-08 23:49:23 +11:00
theonejvo
27f300bda5 chore: trigger CI 2026-02-08 16:55:41 +11:00
theonejvo
557f11985d chore: fix lint warnings (unused vars, formatting) 2026-02-08 16:49:43 +11:00
theonejvo
40f9ba5697 chore(clawhub): bump version to 0.5.1 2026-02-08 16:43:45 +11:00
theonejvo
26a353efc7 feat(cli): enforce VT Code Insight moderation on install/update
CLI now checks moderation status before installing skills:

**Suspicious skills** - Shows warning and requires confirmation:
```
⚠️  Warning: "skill-name" is flagged as suspicious by VirusTotal Code Insight.
   This skill may contain risky patterns (crypto keys, external APIs, eval, etc.)
   Review the skill code before use.

? Install anyway? (y/N)
```
Non-interactive mode requires --force flag.

**Malicious skills** - Blocked entirely:
```
✖ Blocked: skill-name is flagged as malicious
Error: This skill has been flagged as malware and cannot be installed.
```

Changes:
- API now returns `moderation` field with `isSuspicious` and `isMalwareBlocked`
- CLI schema updated to expect moderation field
- cmdInstall and cmdUpdate enforce moderation checks

Thanks to @zackkorman for raising this issue.
2026-02-08 16:36:55 +11:00
theonejvo
990d3d730d feat: VT backfill infrastructure and 99.7% scan coverage
- Add getQuickStatsInternal for fast dashboard stats
- Fix getStatsInternal to include null moderationStatus skills
- Add syncModerationReasons to sync vtAnalysis → moderationReason
- Add requestReanalysisForPending to push stuck skills to VT
- Add backfillActiveSkillsVTCache improvements for efficiency
- Add fixNullModerationStatus for legacy skill cleanup
- Add getPendingVTSkillsInternal for monitoring
- Fix getActiveSkillsMissingVTCacheInternal to avoid read limits

Backfilled 5,000+ skills to 99.7% VT Code Insight coverage:
- Clean: 3,537 (70.5%)
- Suspicious: 1,336 (26.6%)
- Malicious: 123 (2.5%)
- Pending: 17 (0.3%)
2026-02-08 15:37:11 +11:00
Peter Steinberger
75f7a93fe8 fix: restore soft-deleted users on reauth (#106) (thanks @mkrokosz) 2026-02-06 16:58:19 -08:00
Matthew Krokosz
bc51ab4b5f fix: Convert userId to string for targetId comparison
The auditLogs.targetId field is v.string() in the schema, so explicitly
convert the Id<'users'> to string to ensure type-safe comparison.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 16:58:19 -08:00
Matthew Krokosz
82fce7f34f fix: Restore soft-deleted users on re-authentication (with ban check)
Users who deleted their account were unable to sign in again with the
same GitHub account. The OAuth flow would complete but the user would
remain logged out due to the `deletedAt` field being set.

This fix adds a `createOrUpdateUser` callback that:
1. Detects soft-deleted users during OAuth
2. Checks audit logs to determine if user was BANNED vs SELF-DELETED
3. If banned → throws error "This account has been suspended"
4. If self-deleted → clears `deletedAt` to restore account

Security: Both `deleteAccount` and `banUser` set the same `deletedAt`
field. This fix ensures banned users cannot restore their accounts.

Performance: The callback runs on every sign-in, but the audit log
query ONLY executes for soft-deleted users (rare edge case). Normal
active users just hit a single `if` check - no extra queries. When
the audit log query does run, it uses the `by_target` index for
efficient lookup.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-06 16:58:19 -08:00
Peter Steinberger
5f4fa02b42 fix: update footer branding to OpenClaw (#122) (thanks @jontsai) 2026-02-06 16:52:01 -08:00
Jonathan Tsai
cfef87059b fix: update footer branding from Clawdbot to OpenClaw
- Change 'A Clawdbot project' to 'An OpenClaw project'
- Update link from clawd.bot to openclaw.ai
2026-02-06 16:52:01 -08:00
Peter Steinberger
b948b216c6 fix: backfill empty handles in ensure (#158) (thanks @adlai88) 2026-02-06 16:24:28 -08:00
adlai88
f156b909e7 fix: handle empty-string handle in ensure() fallback
`??` (nullish coalescing) treats `""` as a valid value, so users
with `handle=""` never get a fallback derived from name/email.
Change to `||` so empty strings fall through to the next candidate.

This affects any user whose handle field is an empty string rather
than null/undefined — e.g. early accounts or migration artifacts.
2026-02-06 16:24:28 -08:00
nikniknikbbb
0b46210f61
Update README.md (#140) 2026-02-06 16:12:04 -08:00
theonejvo
03eee4b6de fix: self-healing VT scan queue
- Batch size 100 (was 10) - process more per run
- Skip skills checked in last 60 min - avoid hammering same hashes
- After 10 failed checks, mark as pending.scan.stale - drops from queue
- Track scanLastCheckedAt and scanCheckCount on skills
- Add TODO for webhook/notification setup
2026-02-07 00:20:49 +11:00
theonejvo
09ffaba89b fix: randomize VT scan queue + add health monitoring
- Fetch 5x batch size and shuffle to avoid queue head-blocking
- Add getScanQueueHealthInternal to monitor queue status
- Log warnings when queue is unhealthy (>50 pending or >24h stale)
- Return health stats from pollPendingScans
2026-02-07 00:06:03 +11:00
theonejvo
b39f203b12 fix: use yellow styling for suspicious code insight blocks
- Malicious verdicts keep red border/background
- Suspicious verdicts now use yellow/amber to match badge color
2026-02-06 20:40:41 +11:00
theonejvo
4002bb615d chore: fix lint errors 2026-02-06 16:39:27 +11:00
theonejvo
583e391d83 chore: add friendly wait message for pending scans 2026-02-06 16:37:24 +11:00
theonejvo
55ae6e1eb7 feat: add VT rescan trigger and backfill function
- Add requestRescan() to trigger Code Insight via /analyse endpoint
- Update pollPendingScans to request rescan when no Code Insight
- Add backfillPendingScans for one-time backlog clearing
2026-02-06 16:36:49 +11:00
theonejvo
b4fe685a46 chore: fix lint formatting 2026-02-06 16:01:53 +11:00
theonejvo
a93de9cf92 feat: add cron job to poll VT for pending scan results
- Add pollPendingScans action to vt.ts that checks VT for Code Insight verdicts
- Add getPendingScanSkillsInternal query to get skills awaiting scan
- Add vt-pending-scans cron job running every 5 minutes
- Updates skill moderation status when VT analysis is complete
2026-02-06 15:51:24 +11:00
theonejvo
74bf5946e1 chore: fix lint formatting 2026-02-06 15:11:43 +11:00
theonejvo
04c99aafc2 chore: remove debug console.log 2026-02-06 15:09:08 +11:00
theonejvo
ee81325cb3 feat: VT Code Insight visibility matrix for malicious/suspicious skills
- Malicious skills: visible for transparency, downloads blocked via moderationFlags
- Suspicious skills: visible with warning banner, downloads allowed
- Neither appears in search/listings (not indexed)
- Add isSuspicious flag to moderation info
- Update approveSkillByHashInternal to set moderationFlags properly
- Add warning banner CSS variant for suspicious skills
2026-02-06 15:08:52 +11:00
theonejvo
bdc6348b1a fix: show verdict labels (Benign/Suspicious/Malicious) instead of engine stats 2026-02-06 12:52:54 +11:00
theonejvo
68abaa3642 debug: add logging to getBySlug query 2026-02-05 20:36:43 +11:00
theonejvo
0c3acda26e fix: remove unused isModerated variable 2026-02-05 20:27:33 +11:00
theonejvo
650090d298 fix: allow owners to see all moderated skills, not just pending.scan 2026-02-05 20:26:29 +11:00
theonejvo
70220f9487 merge: resolve conflicts with main 2026-02-05 20:12:55 +11:00
theonejvo
aad1fbe2c4 fix: use computed badges in pending skill response 2026-02-05 20:08:26 +11:00
theonejvo
2d7914859c feat: show Code Insight analysis for malicious skills
- Add `source` field to VT results to indicate code_insight vs engines
- Display Code Insight analysis text when AI detects malicious patterns
- Only show "X/Y engines" when traditional AV detection triggers
- Add styled analysis block with red accent for malicious verdicts
2026-02-05 19:58:31 +11:00
Jamieson O'Reilly
ba6a99a65f
fix: show pending skill page to owners instead of "Skill not found" (#136)
* fix: show pending skill page to owners instead of "Skill not found"

When a skill owner uploads a skill that's pending VirusTotal scan,
they now see their skill page with a pending banner instead of
"Skill not found". The banner explains the scan is in progress.

Changes:
- Modified getBySlug query to return skill data for owners even when
  moderationStatus is 'hidden' with reason 'pending.scan'
- Added pendingReview flag to query response
- Added pending banner component to SkillDetailPage
- Added CSS for pending banner using existing ClawHub gold theme

* fix: show pending skills on owner's dashboard

Extended the list query to include pending skills when the requester
is viewing their own dashboard. Added "Scanning" badge with gold theme
to indicate skills pending VirusTotal review.

* fix: show all moderation states to owners with appropriate UI

- Owners see their blocked/removed skills with explanatory banners
- Red banner for malware-blocked and removed skills
- Gold banner for pending scan
- Download button hidden for blocked/removed skills
- Added security disclaimer: "Like a lobster shell, security has layers"
- Fixed badges bug (use computed badges, not stale skill.badges)

* feat: make malware-blocked skills publicly visible

Blocked skills are now visible to everyone via direct URL:
- Shows red banner with "security issue detected"
- Displays VT scan results
- No download button
- Still hidden from listings/search

Sends a strong transparency signal about security enforcement.

* fix: allow owners to view pending scan skills (#136)

* fix: make deterministic zip date timezone-safe

* chore: update convex api types

* fix: update changelog for pending scan visibility (#136) (thanks @orlyjamie)

---------

Co-authored-by: theonejvo <theonejvo@users.noreply.github.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-02-04 23:26:49 -08:00
Peter Steinberger
4ac63c2373 fix: update changelog for pending scan visibility (#136) (thanks @orlyjamie) 2026-02-04 23:25:41 -08:00
Peter Steinberger
8d0f32443a chore: update convex api types 2026-02-04 23:23:13 -08:00
Peter Steinberger
9e75090a39 fix: make deterministic zip date timezone-safe 2026-02-04 23:19:58 -08:00
Peter Steinberger
5cff71104c fix: allow owners to view pending scan skills (#136) 2026-02-04 23:19:53 -08:00
theonejvo
5ef8526874 feat: make malware-blocked skills publicly visible
Blocked skills are now visible to everyone via direct URL:
- Shows red banner with "security issue detected"
- Displays VT scan results
- No download button
- Still hidden from listings/search

Sends a strong transparency signal about security enforcement.
2026-02-05 17:55:29 +11:00
theonejvo
46faafc413 fix: show all moderation states to owners with appropriate UI
- Owners see their blocked/removed skills with explanatory banners
- Red banner for malware-blocked and removed skills
- Gold banner for pending scan
- Download button hidden for blocked/removed skills
- Added security disclaimer: "Like a lobster shell, security has layers"
- Fixed badges bug (use computed badges, not stale skill.badges)
2026-02-05 17:47:04 +11:00
theonejvo
0710b99ac4 fix: show pending skills on owner's dashboard
Extended the list query to include pending skills when the requester
is viewing their own dashboard. Added "Scanning" badge with gold theme
to indicate skills pending VirusTotal review.
2026-02-05 14:51:13 +11:00
theonejvo
9ebe1b6da8 fix: show pending skill page to owners instead of "Skill not found"
When a skill owner uploads a skill that's pending VirusTotal scan,
they now see their skill page with a pending banner instead of
"Skill not found". The banner explains the scan is in progress.

Changes:
- Modified getBySlug query to return skill data for owners even when
  moderationStatus is 'hidden' with reason 'pending.scan'
- Added pendingReview flag to query response
- Added pending banner component to SkillDetailPage
- Added CSS for pending banner using existing ClawHub gold theme
2026-02-05 14:41:00 +11:00
Shakker
98dd49c651 style: fix badges test import formatting
Format import statement to single line per Biome rules.
2026-02-05 03:04:26 +00:00
Shakker
a115bb0d65 docs: update changelog for coverage improvements
Add entries for new tests and coverage configuration changes.
2026-02-05 03:04:26 +00:00
Shakker
f1625cd5ff chore: add skillZip.ts to coverage tracking
Include convex/lib/skillZip.ts in coverage reports to track
the deterministic ZIP building utility added in PR #130.
2026-02-05 03:04:26 +00:00
Shakker
010ca1677f test: add 4+ errors truncation test for ark schema
Test the formatArkErrors truncation logic when there are more than
3 validation errors, ensuring the "+N more" message is displayed.
2026-02-05 03:04:26 +00:00
Shakker
62d62f976c test: add expandDroppedItems tests for uploadFiles
Add jsdom tests for the expandDroppedItems function:
- Handles null/empty DataTransferItemList
- Collects files via getAsFile fallback
- Collects files via webkitGetAsEntry for file entries
- Recursively collects files from directory entries
- Skips non-file/non-directory entries

Improves branch coverage for uploadFiles.ts.
2026-02-05 03:04:26 +00:00
Shakker
24ad575a4e test: add skillZip module tests
Add tests for the deterministic ZIP building utility from PR #130:
- buildSkillMeta function
- buildDeterministicZip with various scenarios
- Verifies deterministic output and _meta.json inclusion

Achieves 100% coverage for skillZip.ts.
2026-02-05 03:04:26 +00:00
Shakker
1cd6a22284 test: add badges module tests
Add comprehensive tests for the badges utility functions:
- isSkillHighlighted, isSkillOfficial, isSkillDeprecated
- getSkillBadges with all badge combinations

Improves branch coverage from 50% to 100% for badges.ts.
2026-02-05 03:04:26 +00:00
Peter Steinberger
343e065292 fix: stabilize VT scans and UI fetches (#130) (thanks @aleph8) 2026-02-04 17:34:35 -08:00
Alejandro García Peláez
2849a864e0
VirusTotal Integration on ClawHub (#130)
* feat: implementation of dynamic VirusTotal integration and deterministic ZIPs

* fix: do not show security scan results if hash is missing

* ui: show 'Loading...' instead of 'Pending' while fetching VT results

* security: restrict auto-approval to explicit benign verdicts only

* fix: prioritize AI verdict in results and refine stats fallback
2026-02-04 17:33:48 -08:00
Peter Steinberger
ae0338e469 feat: add fuzzy user search for moderation CLI 2026-02-04 03:44:54 -08:00
Peter Steinberger
ca1ef737cb fix: resolve typecheck errors in api and config 2026-02-04 03:32:09 -08:00
Peter Steinberger
589e46353b fix: management user search and totals 2026-02-04 03:30:07 -08:00
Peter Steinberger
7edbc03494 fix: skill list pagination and footer branding 2026-02-04 03:18:07 -08:00
Peter Steinberger
f359071d96 feat(moderation): add set-role 2026-02-02 04:40:53 -08:00
Peter Steinberger
57bae9859e chore(release): 0.5.0 2026-02-02 04:25:11 -08:00
Peter Steinberger
96e9ffdcdc fix(convex): batch hard delete skills 2026-02-02 04:15:22 -08:00
Peter Steinberger
eb4601141e fix(cli): honor registry from auth login 2026-02-02 03:25:14 -08:00
Peter Steinberger
39686b3b8d feat(management): add report and user filters 2026-02-02 03:02:52 -08:00
Peter Steinberger
a24d3e9809 feat(cli): add inspect and moderation tools 2026-02-02 02:55:56 -08:00
Peter Steinberger
405c74a4ef feat: require report reasons 2026-02-02 00:57:45 -08:00
Peter Steinberger
ee828046b8 chore: update dependencies 2026-02-02 00:31:54 -08:00
Peter Steinberger
789082bc00 chore: suppress empty chunk warnings 2026-02-02 00:27:01 -08:00
Peter Steinberger
3e4c2450cd chore: suppress nitro build warnings 2026-02-02 00:25:28 -08:00
Peter Steinberger
ea2f51d2ba chore: quiet build warnings 2026-02-02 00:22:04 -08:00
Peter Steinberger
7b2bdbd08f feat: harden moderation and upload safety 2026-02-02 00:17:34 -08:00
Peter Steinberger
f654dc9325 fix: allow legacy skill fields in schema 2026-01-31 12:45:35 +01:00
Peter Steinberger
d78c105570 feat: add admin user ban 2026-01-31 11:44:31 +01:00
Peter Steinberger
a32498ea7d fix: use ClawHub branding for registry 2026-01-31 11:31:46 +01:00
Shadow
5d6ee7adf3
trigger new deploy 2026-01-30 13:05:02 -06:00
Peter Steinberger
ffa25f47da chore: rebrand user-facing to OpenClaw 2026-01-30 07:02:35 +01:00
Peter Steinberger
f866dc05d1 style: format moderation flags 2026-01-30 05:27:19 +01:00
Peter Steinberger
69436fd79b fix: allow moltbot parsed data 2026-01-30 05:26:09 +01:00
Peter Steinberger
f185ca6f55 feat: release 0.4.0 2026-01-30 05:23:12 +01:00
Peter Steinberger
f3fc8d62b6 Revert "chore: rename molthub branding"
This reverts commit 8d68b55333.
2026-01-30 05:23:12 +01:00
Shakker
71f94a774f
Merge pull request #70 from moltbot/remove-clawd-authenticator-tool
chore(moderation): block ClawdAuthenticatorTool (suspected malware)
2026-01-29 17:18:09 +00:00
vignesh07
07349e6107 chore(moderation): block ClawdAuthenticatorTool listing 2026-01-29 09:14:41 -08:00
Vignesh
5ef00e8c6a
chore(security): harden file endpoints CSP + XFO + svg detection (#67) 2026-01-28 23:21:16 -06:00
Jamieson O'Reilly
c5e5e657dd
fix: add CSP headers and Content-Disposition to prevent SVG XSS (#61)
Co-authored-by: theonejvo <theonejvo@users.noreply.github.com>
2026-01-28 22:21:39 -06:00
Jamie Turner
69e1e5c507
A few fixes for search. (#64) 2026-01-28 22:21:26 -06:00
Josh Palmer
481f1b9188
Merge pull request #66 from moltbot/fix/ci-lint
fix: restore lint compliance
2026-01-28 21:27:21 +01:00
Josh Palmer
d4f5832554 🤖 chore: merge main into fix/ci-lint
- resolve conflicts in search/skill publish files
- apply Biome formatting updates from main

Tests: bun run lint:biome; bun run lint:oxlint
2026-01-28 21:23:50 +01:00
Josh Palmer
ca3275ba92 🤖 fix: restore lint compliance
- apply Biome formatting and import ordering across linted files
- fix management useEffect dependencies flagged by Biome

Tests: bun run lint:biome; bun run lint:oxlint
2026-01-28 21:18:19 +01:00
Shadow
d577add8a0
fix: unblock convex deploy typecheck 2026-01-28 13:15:18 -06:00
Shadow
90bd065f7e
fix: correct public skill entries build 2026-01-28 13:12:41 -06:00
Shadow
219f05b160
fix: sanitize public skill and soul data 2026-01-28 13:03:19 -06:00
Shadow
c8091ee8a8
chore: remove unauthenticated badge backfill 2026-01-28 08:33:14 -06:00
Shadow
63f367e1d6
chore: add unauthenticated badge backfill 2026-01-28 08:31:04 -06:00
Shadow
e701d7b713
feat: add skill badges table 2026-01-28 01:11:46 -06:00
Shadow
79bd1f1d6c
fix: query highlighted skills by batch index 2026-01-28 00:34:33 -06:00
Shadow
ac51cc0236
fix: rely on highlighted badge 2026-01-27 23:27:36 -06:00
Shadow
b9c23dc00e
feat: add reports dashboard for moderation 2026-01-27 22:27:20 -06:00
Shadow
f4e96995bc
fix: allow clawdis parsed metadata 2026-01-27 21:39:23 -06:00
Shadow
33165db598
feat: unify skill routes with owner slugs 2026-01-27 21:28:46 -06:00
Shadow
4c10e4847b
feat: add moderation management and backfill 2026-01-27 21:11:37 -06:00
Shadow
8d68b55333
chore: rename molthub branding 2026-01-27 18:25:02 -06:00
Jamie Turner
251de1f540
Performance optimizations for /skills page 2026-01-27 15:18:21 -06:00
Shadow
123f60fa93
Revert "Performance optimizations for /skills page" temporarily until we can deploy
This reverts commit 643faf71c8.
2026-01-27 12:20:15 -06:00
Jamie Turner
643faf71c8
Performance optimizations for /skills page 2026-01-27 12:09:57 -06:00
Sergiy Dybskiy
eb9a67f2af fix: use Error for timeout abort in e2e helper 2026-01-26 12:41:29 +00:00
Sergiy Dybskiy
1ae0498595 test: add e2e test for delete error handling
Verifies that deleting a non-existent skill returns a proper 'not found'
error instead of a generic 'Unauthorized' message.
2026-01-26 12:40:39 +00:00
Sergiy Dybskiy
f1a5254755 fix(cli): use proper Error objects in abort timeouts
When AbortController.abort() receives a string instead of an Error,
the string itself is thrown. pRetry then wraps it in a confusing
message: 'Non-error was thrown: Timeout'

Changed all 3 occurrences in http.ts:
- apiRequest (line 57)
- apiRequestForm (line 106)
- downloadZip (line 141)

Now timeouts will surface as proper Error objects with clear messages.
2026-01-25 21:39:54 +00:00
Sergiy Dybskiy
de2542e391 fix: return proper HTTP status codes for delete/undelete errors
The delete and undelete handlers for skills and souls were catching all
errors and returning 401 Unauthorized, even for errors like:
- 'Skill not found' (should be 404)
- 'Forbidden' (should be 403)
- Other validation errors (should be 400)

This change updates the error handling to return appropriate status codes:
- 401 Unauthorized: authentication failures
- 403 Forbidden: authorization failures (not owner/admin/moderator)
- 404 Not Found: skill/soul/user not found
- 400 Bad Request: other errors with descriptive message

Fixes #34
2026-01-25 21:12:02 +00:00
Aaron Ng
a2c46fbb5d
Search Fixes (#30)
* more search fixes

* update tests

* comments

* fix: tune search filters and limits (#30) (thanks @aaronn)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-01-25 00:08:29 +00:00
Peter Steinberger
f51e0a087d test: fix lockfile mock version 2026-01-24 22:56:39 +00:00
emiliano
d9108b0948
feat: show published skills on user profile (#20)
* fix: resolve typecheck and lint errors

* fix: stabilize publish paths and token types

* feat: show published skills on user profile

* fix: document profile published skills (#20) (thanks @njoylab)

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-01-24 22:55:18 +00:00
Peter Steinberger
d7a017e1c3 fix: add update lookup test (#22) (thanks @daveonkels) 2026-01-24 22:30:09 +00:00
Dave Onkels
fffdf82540
fix: use path instead of url for skill metadata API call (#22)
The `cmdUpdate` function was passing a relative path to `apiRequest`
using the `url` property, but `url` expects a full URL. When `url` is
provided, it's used as-is without combining with the registry base URL.

This caused "Failed to parse URL from /api/v1/skills/<slug>" errors
when updating skills that don't have a local fingerprint match.

Changed to use `path` property which correctly combines with the
registry base URL via `new URL(args.path, registry)`.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-24 22:28:54 +00:00
Ahmed Fuad Mire
a16e624766
fix: relax search token matching to require at least one match (#27)
* fix: relax search token matching to require at least one match

The search was requiring ALL query tokens to exist in the skill's
displayName, slug, or summary. This was too strict and caused valid
results to be filtered out. For example, searching "HTTP API client"
would fail to match skills about "HTTP API" that didn't mention "client".

Changed from `.every()` to `.some()` so at least one token must match,
allowing the vector similarity to determine relevance for the rest.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update matchesExactTokens to require prefix matching for query tokens

* more inclusive token check

* Update convex/lib/searchText.ts

Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

---------

Co-authored-by: Ahmed <ahmed.mire@kaluza.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
2026-01-24 21:23:03 +00:00
Peter Steinberger
decce1d35c fix: skip missing skills in search hydration (#28) (thanks @aaronn) 2026-01-24 21:11:46 +00:00
Aaron Ng
468832af3f
fix search (#28) 2026-01-24 21:11:06 +00:00
814 changed files with 174579 additions and 21419 deletions

View File

@ -0,0 +1,347 @@
---
name: blacksmith-testbox
description: Run Blacksmith Testbox for ClawHub CI-parity checks, hosted services, broad Bun gates, or builds local cannot reproduce without hurting developer machines.
---
# Blacksmith Testbox
## Scope
Use Testbox when you need remote CI parity, injected secrets, hosted services,
or an OS/runtime image that your local machine cannot provide cheaply.
Do not default to Testbox for every local test/build loop. If the repo has
documented local commands for normal iteration, use those first so you keep
warm caches, local build state, and fast feedback.
Testbox is the expensive path. Reach for it deliberately.
ClawHub maintainers can opt into Testbox-first validation by setting
`CLAWHUB_TESTBOX=1` in their environment or standing agent rules. This mode is
maintainers-only and requires Blacksmith access.
When `CLAWHUB_TESTBOX=1` is set in ClawHub:
- Pre-warm a Testbox early for longer, wider, or uncertain work.
- Prefer Testbox for broad Bun gates, e2e, Convex-ish deploy parity, package
proof, and expensive validation.
- Reuse the same Testbox ID for every run command in the same task/session.
- Use local commands only when the task explicitly sets
`CLAWHUB_LOCAL_CHECK_MODE=throttled|full`, or when the user asks for local
proof.
## Install The CLI
If `blacksmith` is not installed, install it:
```bash
curl -fsSL https://get.blacksmith.sh | sh
```
For the canary channel:
```bash
BLACKSMITH_CHANNEL=canary sh -c 'curl -fsSL https://get.blacksmith.sh | sh'
```
Then authenticate:
```bash
blacksmith auth login
```
## Agent-Triggered Browser Auth
When an agent needs to ensure the user is authenticated before running Testbox
commands, use browser-based auth with non-interactive mode. This opens the
browser for the user to sign in; the agent does not interact with the browser.
`--organization` is required with `--non-interactive`:
```bash
blacksmith auth login --non-interactive --organization <org-slug>
```
The org slug can come from `BLACKSMITH_ORG` or the `--org` global flag. Do not
use `--api-token` for this browser flow; that is for headless/token auth.
## Decide First: Local Or Testbox
Before warming anything up, check the repo's own instructions.
Prefer local commands when:
- the repo documents a supported local test/build workflow
- you are iterating on unit tests, lint, typecheck, formatting, or other
local-only validation
- the value comes from warm local caches and fast repeat runs
- the command does not need remote secrets, hosted services, or CI-only images
Prefer Testbox when:
- `CLAWHUB_TESTBOX=1` is set by the user, agent environment, or standing rules
- the repo explicitly requires CI-parity or remote validation
- the command needs secrets, service containers, or provisioned infra
- you are reproducing CI-only failures
- you need the exact workflow image/job environment from GitHub Actions
For ClawHub specifically, normal local iteration stays local unless maintainer
Testbox mode is enabled with `CLAWHUB_TESTBOX=1`:
- `bun run format:check`
- `bun run lint`
- `bun run test`
- `bun run coverage`
- `bunx tsc --noEmit`
- `bun run build`
If `CLAWHUB_TESTBOX=1` is enabled, run those same repo commands inside the warm
Testbox. If the user wants laptop-friendly local proof for one command, use the
explicit escape hatch `CLAWHUB_LOCAL_CHECK_MODE=throttled`.
In `.codex` worktrees without a `node_modules` symlink, do not run
`bun install` just to validate locally. Use syntax checks or Testbox.
## Setup: Warmup Before Coding
If you decided Testbox is warranted, warm one up early. This returns an ID
instantly and boots the CI environment in the background while you work:
```bash
blacksmith testbox warmup ci-check-testbox.yml --ref main --idle-timeout 90
# -> tbx_01jkz5b3t9...
```
Save this ID in the current session. You need it for every `run` command.
Treat `blacksmith testbox list` as diagnostics, not a reusable work queue.
Listed boxes can be visible at the org/repo level while still being unusable or
stale for the current local agent lane.
For ClawHub maintainer Testbox mode, claim the ID in the current checkout:
```bash
bun run testbox:claim -- --id <ID>
```
Warmup dispatches `.github/workflows/ci-check-testbox.yml`, which provisions a
VM with Bun, Node, dependency install/cache, and a clean checkout of the repo at
the chosen ref.
Bootstrap note: GitHub only exposes `workflow_dispatch` workflows through the
Actions API after the workflow file exists on the default branch. If a brand-new
Testbox workflow exists only on a feature branch, `blacksmith testbox warmup
ci-check-testbox.yml --ref <branch>` can return a GitHub 404 even though the
file exists on that branch. Land the workflow bootstrap first, then dispatch
branch refs normally.
Options:
```text
--ref <branch|tag> Git ref to dispatch against
--job <name> Specific job within the workflow, if it has multiple
--idle-timeout <min> Idle timeout in minutes
```
## Critical: Always Run From The Repo Root
Always invoke `blacksmith testbox` commands from the root of the git
repository. The CLI syncs the current working directory to the testbox using
rsync with `--delete`. If you run from a subdirectory, rsync mirrors only that
subdirectory and can delete everything else on the testbox.
Correct:
```bash
blacksmith testbox run --id <ID> "bun run test"
blacksmith testbox run --id <ID> "cd packages/clawhub && bun run verify"
```
Wrong:
```bash
cd packages/clawhub && blacksmith testbox run --id <ID> "bun run verify"
```
If your shell is in a subdirectory, move back first:
```bash
cd "$(git rev-parse --show-toplevel)"
```
## Running Commands
Raw Blacksmith form:
```bash
blacksmith testbox run --id <ID> "<command>"
```
The `run` command waits for the testbox to become ready if it is still booting,
so you can call `run` immediately after warmup.
In ClawHub, prefer the guarded runner wrapper so stale/reused ids fail before
the Blacksmith CLI spends time syncing or emits a confusing missing-key error:
```bash
bun run testbox:run -- --id <ID> -- bun run lint
bun run testbox:run -- --id <ID> -- bun run test
bun run testbox:run -- --id <ID> -- bun run build
```
The wrapper refuses to run when the local per-Testbox key is missing or when
the id was not claimed by this ClawHub checkout with:
```bash
bun run testbox:claim -- --id <ID>
```
Treat that as the expected remediation, not as a GitHub account or normal
SSH-key problem. A local key alone is not enough; a ready box may still carry
stale rsync state from another lane.
If the agent crashes, the remote box relies on Blacksmith's idle timeout. The
local ClawHub claim marker is not deleted automatically, so the wrapper treats
claims older than 12 hours as stale. Override only for intentional long-running
work with:
```bash
CLAWHUB_TESTBOX_CLAIM_TTL_MINUTES=<minutes>
```
Before spending a broad gate on a manually assembled command, run:
```bash
bun run testbox:sanity -- --id <ID>
```
## Downloading Files From A Testbox
Use the `download` command to retrieve files or directories from a running
testbox to your local machine. This is useful for fetching build artifacts,
test results, coverage reports, or any output generated on the testbox.
```bash
blacksmith testbox download --id <ID> <remote-path> [local-path]
```
The remote path is relative to the testbox working directory. If no local path
is specified, the file is saved to the current directory using the same base
name.
Examples:
```bash
blacksmith testbox download --id <ID> coverage/lcov-report/ ./coverage/
blacksmith testbox download --id <ID> test-results/ ./test-results/
blacksmith testbox download --id <ID> dist/ ./dist/
```
## How File Sync Works
Understanding this model is critical for using Testbox correctly.
When you call `run`, the CLI performs a delta sync of your local changes to the
remote testbox before executing your command:
1. The testbox VM starts from a clean checkout at the warmup ref. The workflow
setup steps run during warmup and populate dependency directories on the
remote VM.
2. On each `run`, the CLI uses git to detect which files changed locally since
the last sync. It syncs only tracked files and untracked non-ignored files.
3. `.gitignore`'d directories are never synced. `node_modules/`, `.bun/`,
`.vite/`, `dist/`, `.output/`, `.nitro/`, and coverage outputs stay local.
The testbox uses its own copies populated by the warmup workflow.
4. If nothing has changed since the last sync, the sync is skipped.
Why this matters:
- If you modify `package.json` or `bun.lock`, re-run install on the testbox:
```bash
bun run testbox:run -- --id <ID> -- bun install --frozen-lockfile
```
- If tests depend on generated/build output, re-run the build on the testbox.
- New untracked files sync as long as they are not gitignored.
- Deleted files are also deleted on the remote testbox.
## Critical: Do Not Ban Local Tests
Do not assume local validation is forbidden. Many repos intentionally invest in
fast, warm local loops, and forcing every run through Testbox destroys that
advantage.
Use Testbox for checks that actually need it: remote parity, secrets, services,
CI-only runners, expensive broad gates, or reproducibility against the workflow
image.
ClawHub maintainer exception: if `CLAWHUB_TESTBOX=1` is set by the user or
agent environment, treat Testbox as the normal validation path for this repo.
Use `CLAWHUB_LOCAL_CHECK_MODE=throttled|full` as the explicit local escape
hatch.
## Workflow
1. Decide whether the repo's local loop is the right default. For ClawHub,
`CLAWHUB_TESTBOX=1` makes Testbox the maintainer default.
2. If Testbox is warranted, warm up early:
`blacksmith testbox warmup ci-check-testbox.yml --ref main --idle-timeout 90`.
3. Save the ID, then claim it:
`bun run testbox:claim -- --id <ID>`.
4. Write code while the testbox boots in the background.
5. Run sanity before broad checks:
`bun run testbox:sanity -- --id <ID>`.
6. Run the remote command:
`bun run testbox:run -- --id <ID> -- bun run lint`.
7. If tests fail, fix code and re-run against the same warm box.
8. If dependency manifests changed, run install in the box before testing.
9. If you need artifacts, download them with `blacksmith testbox download`.
10. Stop the box when done if it is no longer needed:
`blacksmith testbox stop --id <ID>`.
## ClawHub Broad Gate
For a broad ClawHub proof in maintainer Testbox mode, use the repo package
manager and keep the commands explicit:
```bash
bun run testbox:run -- --id <ID> -- bun run format:check
bun run testbox:run -- --id <ID> -- bun run lint
bun run testbox:run -- --id <ID> -- bun run test
bun run testbox:run -- --id <ID> -- bunx tsc --noEmit
bun run testbox:run -- --id <ID> -- bunx tsc -p packages/schema/tsconfig.json --noEmit
bun run testbox:run -- --id <ID> -- bunx tsc -p packages/clawhub/tsconfig.json --noEmit
bun run testbox:run -- --id <ID> -- bun run build
```
For e2e:
```bash
bun run testbox:run -- --id <ID> -- bun run test:e2e
bun run testbox:run -- --id <ID> -- bun run test:pw
```
## Waiting For Readiness
The `run` command automatically waits for the testbox, so explicit waiting is
usually unnecessary. If you do need to check readiness separately, use
`--wait`. Do not use a sleep-and-recheck loop.
```bash
blacksmith testbox status --id <ID> --wait --wait-timeout 5m
```
## Managing Testboxes
```bash
blacksmith testbox status --id <ID>
blacksmith testbox list
blacksmith testbox stop --id <ID>
```
Testboxes automatically shut down after being idle. For ClawHub maintainer
work, use 90 minutes for long-running sessions:
```bash
blacksmith testbox warmup ci-check-testbox.yml --idle-timeout 90
```

View File

@ -0,0 +1,288 @@
---
name: convex-create-component
description: Builds reusable Convex components with isolated tables and app-facing APIs. Use for new components, reusable backend modules, integrations, or component boundary work.
---
# Convex Create Component
Create reusable Convex components with clear boundaries and a small app-facing API.
## When to Use
- Creating a new Convex component in an existing app
- Extracting reusable backend logic into a component
- Building a third-party integration that should own its own tables and workflows
- Packaging Convex functionality for reuse across multiple apps
## When Not to Use
- One-off business logic that belongs in the main app
- Thin utilities that do not need Convex tables or functions
- App-level orchestration that should stay in `convex/`
- Cases where a normal TypeScript library is enough
## Workflow
1. Ask the user what they are building and what the end goal is. If the repo already makes the answer obvious, say so and confirm before proceeding.
2. Choose the shape using the decision tree below and read the matching reference file.
3. Decide whether a component is justified. Prefer normal app code or a regular library if the feature does not need isolated tables, backend functions, or reusable persistent state.
4. Make a short plan for:
- what tables the component owns
- what public functions it exposes
- what data must be passed in from the app (auth, env vars, parent IDs)
- what stays in the app as wrappers or HTTP mounts
5. Create the component structure with `convex.config.ts`, `schema.ts`, and function files.
6. Implement functions using the component's own `./_generated/server` imports, not the app's generated files.
7. Wire the component into the app with `app.use(...)`. If the app does not already have `convex/convex.config.ts`, create it.
8. Call the component from the app through `components.<name>` using `ctx.runQuery`, `ctx.runMutation`, or `ctx.runAction`.
9. If React clients, HTTP callers, or public APIs need access, create wrapper functions in the app instead of exposing component functions directly.
10. Run `npx convex dev` and fix codegen, type, or boundary issues before finishing.
## Choose the Shape
Ask the user, then pick one path:
| Goal | Shape | Reference |
| ------------------------------------------------- | ---------------- | ----------------------------------- |
| Component for this app only | Local | `references/local-components.md` |
| Publish or share across apps | Packaged | `references/packaged-components.md` |
| User explicitly needs local + shared library code | Hybrid | `references/hybrid-components.md` |
| Not sure | Default to local | `references/local-components.md` |
Read exactly one reference file before proceeding.
## Default Approach
Unless the user explicitly wants an npm package, default to a local component:
- Put it under `convex/components/<componentName>/`
- Define it with `defineComponent(...)` in its own `convex.config.ts`
- Install it from the app's `convex/convex.config.ts` with `app.use(...)`
- Let `npx convex dev` generate the component's own `_generated/` files
## Component Skeleton
A minimal local component with a table and two functions, plus the app wiring.
```ts
// convex/components/notifications/convex.config.ts
import { defineComponent } from "convex/server";
export default defineComponent("notifications");
```
```ts
// convex/components/notifications/schema.ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
notifications: defineTable({
userId: v.string(),
message: v.string(),
read: v.boolean(),
}).index("by_user", ["userId"]),
});
```
```ts
// convex/components/notifications/lib.ts
import { v } from "convex/values";
import { mutation, query } from "./_generated/server.js";
export const send = mutation({
args: { userId: v.string(), message: v.string() },
returns: v.id("notifications"),
handler: async (ctx, args) => {
return await ctx.db.insert("notifications", {
userId: args.userId,
message: args.message,
read: false,
});
},
});
export const listUnread = query({
args: { userId: v.string() },
returns: v.array(
v.object({
_id: v.id("notifications"),
_creationTime: v.number(),
userId: v.string(),
message: v.string(),
read: v.boolean(),
}),
),
handler: async (ctx, args) => {
return await ctx.db
.query("notifications")
.withIndex("by_user", (q) => q.eq("userId", args.userId))
.filter((q) => q.eq(q.field("read"), false))
.collect();
},
});
```
```ts
// convex/convex.config.ts
import { defineApp } from "convex/server";
import notifications from "./components/notifications/convex.config.js";
const app = defineApp();
app.use(notifications);
export default app;
```
```ts
// convex/notifications.ts (app-side wrapper)
import { v } from "convex/values";
import { mutation, query } from "./_generated/server";
import { components } from "./_generated/api";
import { getAuthUserId } from "@convex-dev/auth/server";
export const sendNotification = mutation({
args: { message: v.string() },
returns: v.null(),
handler: async (ctx, args) => {
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("Not authenticated");
await ctx.runMutation(components.notifications.lib.send, {
userId,
message: args.message,
});
return null;
},
});
export const myUnread = query({
args: {},
handler: async (ctx) => {
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("Not authenticated");
return await ctx.runQuery(components.notifications.lib.listUnread, {
userId,
});
},
});
```
Note the reference path shape: a function in `convex/components/notifications/lib.ts` is called as `components.notifications.lib.send` from the app.
## Critical Rules
- Keep authentication in the app, because `ctx.auth` is not available inside components.
- Keep environment access in the app, because component functions cannot read `process.env`.
- Pass parent app IDs across the boundary as strings, because `Id` types become plain strings in the app-facing `ComponentApi`.
- Do not use `v.id("parentTable")` for app-owned tables inside component args or schema, because the component has no access to the app's table namespace.
- Import `query`, `mutation`, and `action` from the component's own `./_generated/server`, not the app's generated files.
- Do not expose component functions directly to clients. Create app wrappers when client access is needed, because components are internal and need auth/env wiring the app provides.
- If the component defines HTTP handlers, mount the routes in the app's `convex/http.ts`, because components cannot register their own HTTP routes.
- If the component needs pagination, use `paginator` from `convex-helpers` instead of built-in `.paginate()`, because `.paginate()` does not work across the component boundary.
- Add `args` and `returns` validators to all public component functions, because the component boundary requires explicit type contracts.
## Patterns
### Authentication and environment access
```ts
// Bad: component code cannot rely on app auth or env
const identity = await ctx.auth.getUserIdentity();
const apiKey = process.env.OPENAI_API_KEY;
```
```ts
// Good: the app resolves auth and env, then passes explicit values
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("Not authenticated");
await ctx.runAction(components.translator.translate, {
userId,
apiKey: process.env.OPENAI_API_KEY,
text: args.text,
});
```
### Client-facing API
```ts
// Bad: assuming a component function is directly callable by clients
export const send = components.notifications.send;
```
```ts
// Good: re-export through an app mutation or query
export const sendNotification = mutation({
args: { message: v.string() },
returns: v.null(),
handler: async (ctx, args) => {
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("Not authenticated");
await ctx.runMutation(components.notifications.lib.send, {
userId,
message: args.message,
});
return null;
},
});
```
### IDs across the boundary
```ts
// Bad: parent app table IDs are not valid component validators
args: {
userId: v.id("users");
}
```
```ts
// Good: treat parent-owned IDs as strings at the boundary
args: {
userId: v.string();
}
```
### Advanced Patterns
For additional patterns including function handles for callbacks, deriving validators from schema, static configuration with a globals table, and class-based client wrappers, see `references/advanced-patterns.md`.
## Validation
Try validation in this order:
1. `npx convex codegen --component-dir convex/components/<name>`
2. `npx convex codegen`
3. `npx convex dev`
Important:
- Fresh repos may fail these commands until `CONVEX_DEPLOYMENT` is configured.
- Until codegen runs, component-local `./_generated/*` imports and app-side `components.<name>...` references will not typecheck.
- If validation blocks on Convex login or deployment setup, stop and ask the user for that exact step instead of guessing.
## Reference Files
Read exactly one of these after the user confirms the goal:
- `references/local-components.md`
- `references/packaged-components.md`
- `references/hybrid-components.md`
Official docs: [Authoring Components](https://docs.convex.dev/components/authoring)
## Checklist
- [ ] Asked the user what they want to build and confirmed the shape
- [ ] Read the matching reference file
- [ ] Confirmed a component is the right abstraction
- [ ] Planned tables, public API, boundaries, and app wrappers
- [ ] Component lives under `convex/components/<name>/` (or package layout if publishing)
- [ ] Component imports from its own `./_generated/server`
- [ ] Auth, env access, and HTTP routes stay in the app
- [ ] Parent app IDs cross the boundary as `v.string()`
- [ ] Public functions have `args` and `returns` validators
- [ ] Ran `npx convex dev` and fixed codegen or type issues

View File

@ -0,0 +1,10 @@
interface:
display_name: "Convex Create Component"
short_description: "Design and build reusable Convex components with clear boundaries."
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#14B8A6"
default_prompt: "Help me create a Convex component for this feature. First check that a component is actually justified, then design the tables, API surface, and app-facing wrappers before implementing it."
policy:
allow_implicit_invocation: true

View File

@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" aria-hidden="true" data-slot="icon">
<path stroke-linecap="round" stroke-linejoin="round" d="m21 7.5-2.25-1.313M21 7.5v2.25m0-2.25-2.25 1.313M3 7.5l2.25-1.313M3 7.5l2.25 1.313M3 7.5v2.25m9 3 2.25-1.313M12 12.75l-2.25-1.313M12 12.75V15m0 6.75 2.25-1.313M12 21.75V19.5m0 2.25-2.25-1.313m0-16.875L12 2.25l2.25 1.313M21 14.25v2.25l-2.25 1.313m-13.5 0L3 16.5v-2.25"/>
</svg>

After

Width:  |  Height:  |  Size: 485 B

View File

@ -0,0 +1,134 @@
# Advanced Component Patterns
Additional patterns for Convex components that go beyond the basics covered in the main skill file.
## Function Handles for callbacks
When the app needs to pass a callback function to the component, use function handles. This is common for components that run app-defined logic on a schedule or in a workflow.
```ts
// App side: create a handle and pass it to the component
import { createFunctionHandle } from "convex/server";
export const startJob = mutation({
handler: async (ctx) => {
const handle = await createFunctionHandle(internal.myModule.processItem);
await ctx.runMutation(components.workpool.enqueue, {
callback: handle,
});
},
});
```
```ts
// Component side: accept and invoke the handle
import { v } from "convex/values";
import type { FunctionHandle } from "convex/server";
import { mutation } from "./_generated/server.js";
export const enqueue = mutation({
args: { callback: v.string() },
handler: async (ctx, args) => {
const handle = args.callback as FunctionHandle<"mutation">;
await ctx.scheduler.runAfter(0, handle, {});
},
});
```
## Deriving validators from schema
Instead of manually repeating field types in return validators, extend the schema validator:
```ts
import { v } from "convex/values";
import schema from "./schema.js";
const notificationDoc = schema.tables.notifications.validator.extend({
_id: v.id("notifications"),
_creationTime: v.number(),
});
export const getLatest = query({
args: {},
returns: v.nullable(notificationDoc),
handler: async (ctx) => {
return await ctx.db.query("notifications").order("desc").first();
},
});
```
## Static configuration with a globals table
A common pattern for component configuration is a single-document "globals" table:
```ts
// schema.ts
export default defineSchema({
globals: defineTable({
maxRetries: v.number(),
webhookUrl: v.optional(v.string()),
}),
// ... other tables
});
```
```ts
// lib.ts
export const configure = mutation({
args: { maxRetries: v.number(), webhookUrl: v.optional(v.string()) },
returns: v.null(),
handler: async (ctx, args) => {
const existing = await ctx.db.query("globals").first();
if (existing) {
await ctx.db.patch(existing._id, args);
} else {
await ctx.db.insert("globals", args);
}
return null;
},
});
```
## Class-based client wrappers
For components with many functions or configuration options, a class-based client provides a cleaner API. This pattern is common in published components.
```ts
// src/client/index.ts
import type { GenericMutationCtx, GenericDataModel } from "convex/server";
import type { ComponentApi } from "../component/_generated/component.js";
type MutationCtx = Pick<GenericMutationCtx<GenericDataModel>, "runMutation">;
export class Notifications {
constructor(
private component: ComponentApi,
private options?: { defaultChannel?: string },
) {}
async send(ctx: MutationCtx, args: { userId: string; message: string }) {
return await ctx.runMutation(this.component.lib.send, {
...args,
channel: this.options?.defaultChannel ?? "default",
});
}
}
```
```ts
// App usage
import { Notifications } from "@convex-dev/notifications";
import { components } from "./_generated/api";
const notifications = new Notifications(components.notifications, {
defaultChannel: "alerts",
});
export const send = mutation({
args: { message: v.string() },
handler: async (ctx, args) => {
const userId = await getAuthUserId(ctx);
await notifications.send(ctx, { userId, message: args.message });
},
});
```

View File

@ -0,0 +1,37 @@
# Hybrid Convex Components
Read this file only when the user explicitly wants a hybrid setup.
## What This Means
A hybrid component combines a local Convex component with shared library code.
This can help when:
- the user wants a local install but also shared package logic
- the component needs extension points or override hooks
- some logic should live in normal TypeScript code outside the component boundary
## Default Advice
Treat hybrid as an advanced option, not the default.
Before choosing it, ask:
- Why is a plain local component not enough?
- Why is a packaged component not enough?
- What exactly needs to stay overridable or shared?
If the answer is vague, fall back to local or packaged.
## Risks
- More moving parts
- Harder upgrades and backwards compatibility
- Easier to blur the component boundary
## Checklist
- [ ] User explicitly needs hybrid behavior
- [ ] Local-only and packaged-only options were considered first
- [ ] The extension points are clearly defined before coding

View File

@ -0,0 +1,38 @@
# Local Convex Components
Read this file when the component should live inside the current app and does not need to be published as an npm package.
## When to Choose This
- The user wants the simplest path
- The component only needs to work in this repo
- The goal is extracting app logic into a cleaner boundary
## Default Layout
Use this structure unless the repo already has a clear alternative pattern:
```text
convex/
convex.config.ts
components/
<name>/
convex.config.ts
schema.ts
<feature>.ts
```
## Workflow Notes
- Define the component with `defineComponent("<name>")`
- Install it from the app with `defineApp()` and `app.use(...)`
- Keep auth, env access, public API wrappers, and HTTP route mounting in the app
- Let the component own isolated tables and reusable backend workflows
- Add app wrappers if clients need to call into the component
## Checklist
- [ ] Component is inside `convex/components/<name>/`
- [ ] App installs it with `app.use(...)`
- [ ] Component owns only its own tables
- [ ] App wrappers handle client-facing calls when needed

View File

@ -0,0 +1,51 @@
# Packaged Convex Components
Read this file when the user wants a reusable npm package or a component shared across multiple apps.
## When to Choose This
- The user wants to publish the component
- The user wants a stable reusable package boundary
- The component will be shared across multiple apps or teams
## Default Approach
- Prefer starting from `npx create-convex@latest --component` when possible
- Keep the official authoring docs as the source of truth for package layout and exports
- Validate the bundled package through an example app, not just the source files
## Build Flow
When building a packaged component, make sure the bundled output exists before the example app tries to consume it.
Recommended order:
1. `npx convex codegen --component-dir ./path/to/component`
2. Run the package build command
3. Run `npx convex dev --typecheck-components` in the example app
Do not assume normal app codegen is enough for packaged component workflows.
## Package Exports
If publishing to npm, make sure the package exposes the entry points apps need:
- package root for client helpers, types, or classes
- `./convex.config.js` for installing the component
- `./_generated/component.js` for the app-facing `ComponentApi` type
- `./test` for testing helpers when applicable
## Testing
- Use `convex-test` for component logic
- Register the component schema and modules with the test instance
- Test app-side wrapper code from an example app that installs the package
- Export a small helper from `./test` if consumers need easy test registration
## Checklist
- [ ] Packaging is actually required
- [ ] Build order avoids bundle and codegen races
- [ ] Package exports include install and typing entry points
- [ ] Example app exercises the packaged component
- [ ] Core behavior is covered by tests

View File

@ -0,0 +1,149 @@
---
name: convex-migration-helper
description: Plans Convex schema and data migrations with widen-migrate-narrow and @convex-dev/migrations. Use for breaking schema changes, backfills, table reshaping, or zero-downtime rollouts.
---
# Convex Migration Helper
Safely migrate Convex schemas and data when making breaking changes.
## When to Use
- Adding new required fields to existing tables
- Changing field types or structure
- Splitting or merging tables
- Renaming or deleting fields
- Migrating from nested to relational data
## When Not to Use
- Greenfield schema with no existing data in production or dev
- Adding optional fields that do not need backfilling
- Adding new tables with no existing data to migrate
- Adding or removing indexes with no correctness concern
- Questions about Convex schema design without a migration need
## Key Concepts
### Schema Validation Drives the Workflow
Convex will not let you deploy a schema that does not match the data at rest. This is the fundamental constraint that shapes every migration:
- You cannot add a required field if existing documents don't have it
- You cannot change a field's type if existing documents have the old type
- You cannot remove a field from the schema if existing documents still have it
This means migrations follow a predictable pattern: **widen the schema, migrate the data, narrow the schema**.
### Online Migrations
Convex migrations run online, meaning the app continues serving requests while data is updated asynchronously in batches. During the migration window, your code must handle both old and new data formats.
### Prefer New Fields Over Changing Types
When changing the shape of data, create a new field rather than modifying an existing one. This makes the transition safer and easier to roll back.
### Don't Delete Data
Unless you are certain, prefer deprecating fields over deleting them. Mark the field as `v.optional` and add a code comment explaining it is deprecated and why it existed.
## Safe Changes (No Migration Needed)
### Adding Optional Field
```typescript
// Before
users: defineTable({
name: v.string(),
});
// After - safe, new field is optional
users: defineTable({
name: v.string(),
bio: v.optional(v.string()),
});
```
### Adding New Table
```typescript
posts: defineTable({
userId: v.id("users"),
title: v.string(),
}).index("by_user", ["userId"]);
```
### Adding Index
```typescript
users: defineTable({
name: v.string(),
email: v.string(),
}).index("by_email", ["email"]);
```
## Breaking Changes: The Deployment Workflow
Every breaking migration follows the same multi-deploy pattern:
**Deploy 1 - Widen the schema:**
1. Update schema to allow both old and new formats (e.g., add optional new field)
2. Update code to handle both formats when reading
3. Update code to write the new format for new documents
4. Deploy
**Between deploys - Migrate data:**
5. Run migration to backfill existing documents
6. Verify all documents are migrated
**Deploy 2 - Narrow the schema:**
7. Update schema to require the new format only
8. Remove code that handles the old format
9. Deploy
## Using the Migrations Component
For any non-trivial migration, use the [`@convex-dev/migrations`](https://www.convex.dev/components/migrations) component. It handles batching, cursor-based pagination, state tracking, resume from failure, dry runs, and progress monitoring.
See `references/migrations-component.md` for installation, setup, defining and running migrations, dry runs, status monitoring, and configuration options.
## Common Migration Patterns
See `references/migration-patterns.md` for complete patterns with code examples covering:
- Adding a required field
- Deleting a field
- Changing a field type
- Splitting nested data into a separate table
- Cleaning up orphaned documents
- Zero-downtime strategies (dual write, dual read)
- Small table shortcut (single internalMutation without the component)
- Verifying a migration is complete
## Common Pitfalls
1. **Making a field required before migrating data**: Convex rejects the deploy because existing documents lack the field. Always widen the schema first.
2. **Using `.collect()` on large tables**: Hits transaction limits or causes timeouts. Use the migrations component for proper batched pagination. `.collect()` is only safe for tables you know are small.
3. **Not writing the new format before migrating**: Documents created during the migration window will be missed, leaving unmigrated data after the migration "completes."
4. **Skipping the dry run**: Use `dryRun: true` to validate migration logic before committing changes to production data. Catches bugs before they touch real documents.
5. **Deleting fields prematurely**: Prefer deprecating with `v.optional` and a comment. Only delete after you are confident the data is no longer needed and no code references it.
6. **Using crons for migration batches**: The migrations component handles batching via recursive scheduling internally. Crons require manual cleanup and an extra deploy to remove.
## Migration Checklist
- [ ] Identify the breaking change and plan the multi-deploy workflow
- [ ] Update schema to allow both old and new formats
- [ ] Update code to handle both formats when reading
- [ ] Update code to write the new format for new documents
- [ ] Deploy widened schema and updated code
- [ ] Define migration using the `@convex-dev/migrations` component
- [ ] Test with `dryRun: true`
- [ ] Run migration and monitor status
- [ ] Verify all documents are migrated
- [ ] Update schema to require new format only
- [ ] Clean up code that handled old format
- [ ] Deploy final schema and code
- [ ] Remove migration code once confirmed stable

View File

@ -0,0 +1,10 @@
interface:
display_name: "Convex Migration Helper"
short_description: "Plan and run safe Convex schema and data migrations."
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#8B5CF6"
default_prompt: "Help me plan and execute this Convex migration safely. Start by identifying the schema change, the existing data shape, and the widen-migrate-narrow path before making edits."
policy:
allow_implicit_invocation: true

View File

@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" aria-hidden="true" data-slot="icon">
<path stroke-linecap="round" stroke-linejoin="round" d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0 3.181 3.183a8.25 8.25 0 0 0 13.803-3.7M4.031 9.865a8.25 8.25 0 0 1 13.803-3.7l3.181 3.182m0-4.991v4.99"/>
</svg>

After

Width:  |  Height:  |  Size: 386 B

View File

@ -0,0 +1,231 @@
# Migration Patterns Reference
Common migration patterns, zero-downtime strategies, and verification techniques for Convex schema and data migrations.
## Adding a Required Field
```typescript
// Deploy 1: Schema allows both states
users: defineTable({
name: v.string(),
role: v.optional(v.union(v.literal("user"), v.literal("admin"))),
});
// Migration: backfill the field
export const addDefaultRole = migrations.define({
table: "users",
migrateOne: async (ctx, user) => {
if (user.role === undefined) {
await ctx.db.patch(user._id, { role: "user" });
}
},
});
// Deploy 2: After migration completes, make it required
users: defineTable({
name: v.string(),
role: v.union(v.literal("user"), v.literal("admin")),
});
```
## Deleting a Field
Mark the field optional first, migrate data to remove it, then remove from schema:
```typescript
// Deploy 1: Make optional
// isPro: v.boolean() --> isPro: v.optional(v.boolean())
// Migration
export const removeIsPro = migrations.define({
table: "teams",
migrateOne: async (ctx, team) => {
if (team.isPro !== undefined) {
await ctx.db.patch(team._id, { isPro: undefined });
}
},
});
// Deploy 2: Remove isPro from schema entirely
```
## Changing a Field Type
Prefer creating a new field. You can combine adding and deleting in one migration:
```typescript
// Deploy 1: Add new field, keep old field optional
// isPro: v.boolean() --> isPro: v.optional(v.boolean()), plan: v.optional(...)
// Migration: convert old field to new field
export const convertToEnum = migrations.define({
table: "teams",
migrateOne: async (ctx, team) => {
if (team.plan === undefined) {
await ctx.db.patch(team._id, {
plan: team.isPro ? "pro" : "basic",
isPro: undefined,
});
}
},
});
// Deploy 2: Remove isPro from schema, make plan required
```
## Splitting Nested Data Into a Separate Table
```typescript
export const extractPreferences = migrations.define({
table: "users",
migrateOne: async (ctx, user) => {
if (user.preferences === undefined) return;
const existing = await ctx.db
.query("userPreferences")
.withIndex("by_user", (q) => q.eq("userId", user._id))
.first();
if (!existing) {
await ctx.db.insert("userPreferences", {
userId: user._id,
...user.preferences,
});
}
await ctx.db.patch(user._id, { preferences: undefined });
},
});
```
Make sure your code is already writing to the new `userPreferences` table for new users before running this migration, so you don't miss documents created during the migration window.
## Cleaning Up Orphaned Documents
```typescript
export const deleteOrphanedEmbeddings = migrations.define({
table: "embeddings",
migrateOne: async (ctx, doc) => {
const chunk = await ctx.db
.query("chunks")
.withIndex("by_embedding", (q) => q.eq("embeddingId", doc._id))
.first();
if (!chunk) {
await ctx.db.delete(doc._id);
}
},
});
```
## Zero-Downtime Strategies
During the migration window, your app must handle both old and new data formats. There are two main strategies.
### Dual Write (Preferred)
Write to both old and new structures. Read from the old structure until migration is complete.
1. Deploy code that writes both formats, reads old format
2. Run migration on existing data
3. Deploy code that reads new format, still writes both
4. Deploy code that only reads and writes new format
This is preferred because you can safely roll back at any point, the old format is always up to date.
```typescript
// Bad: only writing to new structure before migration is done
export const createTeam = mutation({
args: { name: v.string(), isPro: v.boolean() },
handler: async (ctx, args) => {
await ctx.db.insert("teams", {
name: args.name,
plan: args.isPro ? "pro" : "basic",
});
},
});
// Good: writing to both structures during migration
export const createTeam = mutation({
args: { name: v.string(), isPro: v.boolean() },
handler: async (ctx, args) => {
const plan = args.isPro ? "pro" : "basic";
await ctx.db.insert("teams", {
name: args.name,
isPro: args.isPro,
plan,
});
},
});
```
### Dual Read
Read both formats. Write only the new format.
1. Deploy code that reads both formats (preferring new), writes only new format
2. Run migration on existing data
3. Deploy code that reads and writes only new format
This avoids duplicating writes, which is useful when having two copies of data could cause inconsistencies. The downside is that rolling back to before step 1 is harder, since new documents only have the new format.
```typescript
// Good: reading both formats, preferring new
function getTeamPlan(team: Doc<"teams">): "basic" | "pro" {
if (team.plan !== undefined) return team.plan;
return team.isPro ? "pro" : "basic";
}
```
## Small Table Shortcut
For small tables (a few thousand documents at most), you can migrate in a single `internalMutation` without the component:
```typescript
import { internalMutation } from "./_generated/server";
export const backfillSmallTable = internalMutation({
handler: async (ctx) => {
const docs = await ctx.db.query("smallConfig").collect();
for (const doc of docs) {
if (doc.newField === undefined) {
await ctx.db.patch(doc._id, { newField: "default" });
}
}
},
});
```
```bash
npx convex run migrations:backfillSmallTable
```
Only use `.collect()` when you are certain the table is small. For anything larger, use the migrations component.
## Verifying a Migration
Query to check remaining unmigrated documents:
```typescript
import { query } from "./_generated/server";
export const verifyMigration = query({
handler: async (ctx) => {
const remaining = await ctx.db
.query("users")
.filter((q) => q.eq(q.field("role"), undefined))
.take(10);
return {
complete: remaining.length === 0,
sampleRemaining: remaining.map((u) => u._id),
};
},
});
```
Or use the component's built-in status monitoring:
```bash
npx convex run --component migrations lib:getStatus --watch
```

View File

@ -0,0 +1,169 @@
# Migrations Component Reference
Complete guide to the [`@convex-dev/migrations`](https://www.convex.dev/components/migrations) component for batched, resumable Convex data migrations.
## Installation
```bash
npm install @convex-dev/migrations
```
## Setup
```typescript
// convex/convex.config.ts
import { defineApp } from "convex/server";
import migrations from "@convex-dev/migrations/convex.config.js";
const app = defineApp();
app.use(migrations);
export default app;
```
```typescript
// convex/migrations.ts
import { Migrations } from "@convex-dev/migrations";
import { components } from "./_generated/api.js";
import { DataModel } from "./_generated/dataModel.js";
export const migrations = new Migrations<DataModel>(components.migrations);
export const run = migrations.runner();
```
The `DataModel` type parameter is optional but provides type safety for migration definitions.
## Define a Migration
The `migrateOne` function processes a single document. The component handles batching and pagination automatically.
```typescript
// convex/migrations.ts
export const addDefaultRole = migrations.define({
table: "users",
migrateOne: async (ctx, user) => {
if (user.role === undefined) {
await ctx.db.patch(user._id, { role: "user" });
}
},
});
```
Shorthand: if you return an object, it is applied as a patch automatically.
```typescript
export const clearDeprecatedField = migrations.define({
table: "users",
migrateOne: () => ({ legacyField: undefined }),
});
```
## Run a Migration
From the CLI:
```bash
# Define a one-off runner in convex/migrations.ts:
# export const runIt = migrations.runner(internal.migrations.addDefaultRole);
npx convex run migrations:runIt
# Or use the general-purpose runner
npx convex run migrations:run '{"fn": "migrations:addDefaultRole"}'
```
Programmatically from another Convex function:
```typescript
await migrations.runOne(ctx, internal.migrations.addDefaultRole);
```
## Run Multiple Migrations in Order
```typescript
export const runAll = migrations.runner([
internal.migrations.addDefaultRole,
internal.migrations.clearDeprecatedField,
internal.migrations.normalizeEmails,
]);
```
```bash
npx convex run migrations:runAll
```
If one fails, it stops and will not continue to the next. Call it again to retry from where it left off. Completed migrations are skipped automatically.
## Dry Run
Test a migration before committing changes:
```bash
npx convex run migrations:runIt '{"dryRun": true}'
```
This runs one batch and then rolls back, so you can see what it would do without changing any data.
## Check Migration Status
```bash
npx convex run --component migrations lib:getStatus --watch
```
## Cancel a Running Migration
```bash
npx convex run --component migrations lib:cancel '{"name": "migrations:addDefaultRole"}'
```
Or programmatically:
```typescript
await migrations.cancel(ctx, internal.migrations.addDefaultRole);
```
## Run Migrations on Deploy
Chain migration execution after deploying:
```bash
npx convex deploy --cmd 'npm run build' && npx convex run migrations:runAll --prod
```
## Configuration Options
### Custom Batch Size
If documents are large or the table has heavy write traffic, reduce the batch size to avoid transaction limits or OCC conflicts:
```typescript
export const migrateHeavyTable = migrations.define({
table: "largeDocuments",
batchSize: 10,
migrateOne: async (ctx, doc) => {
// migration logic
},
});
```
### Migrate a Subset Using an Index
Process only matching documents instead of the full table:
```typescript
export const fixEmptyNames = migrations.define({
table: "users",
customRange: (query) => query.withIndex("by_name", (q) => q.eq("name", "")),
migrateOne: () => ({ name: "<unknown>" }),
});
```
### Parallelize Within a Batch
By default each document in a batch is processed serially. Enable parallel processing if your migration logic does not depend on ordering:
```typescript
export const clearField = migrations.define({
table: "myTable",
parallelize: true,
migrateOne: () => ({ optionalField: undefined }),
});
```

View File

@ -0,0 +1,143 @@
---
name: convex-performance-audit
description: Audits Convex performance for reads, subscriptions, write contention, and function limits. Use for slow features, insights findings, OCC conflicts, or read amplification.
---
# Convex Performance Audit
Diagnose and fix performance problems in Convex applications, one problem class at a time.
## When to Use
- A Convex page or feature feels slow or expensive
- `npx convex insights --details` reports high bytes read, documents read, or OCC conflicts
- Low-freshness read paths are using reactivity where point-in-time reads would do
- OCC conflict errors or excessive mutation retries
- High subscription count or slow UI updates
- Functions approaching execution or transaction limits
- The same performance pattern needs fixing across sibling functions
## When Not to Use
- Initial Convex setup, auth setup, or component extraction
- Pure schema migrations with no performance goal
- One-off micro-optimizations without a user-visible or deployment-visible problem
## Guardrails
- Prefer simpler code when scale is small, traffic is modest, or the available signals are weak
- Do not recommend digest tables, document splitting, fetch-strategy changes, or migration-heavy rollouts unless there is a measured signal, a clearly unbounded path, or a known hot read/write path
- In Convex, a simple scan on a small table is often acceptable. Do not invent structural work just because a pattern is not ideal at large scale
## First Step: Gather Signals
Start with the strongest signal available:
1. If deployment Health insights are already available from the user or the current context, treat them as a first-class source of performance signals.
2. If CLI insights are available, run `npx convex insights --details`. Use `--prod`, `--preview-name`, or `--deployment-name` when needed.
- If the local repo's Convex CLI is too old to support `insights`, try `npx -y convex@latest insights --details` before giving up.
3. If the repo already uses `convex-doctor`, you may treat its findings as hints. Do not require it, and do not treat it as the source of truth.
4. If runtime signals are unavailable, audit from code anyway, but keep the guardrails above in mind. Lack of insights is not proof of health, but it is also not proof that a large refactor is warranted.
## Signal Routing
After gathering signals, identify the problem class and read the matching reference file.
| Signal | Reference |
| -------------------------------------------------------------- | ----------------------------------------- |
| High bytes or documents read, JS filtering, unnecessary joins | `references/hot-path-rules.md` |
| OCC conflict errors, write contention, mutation retries | `references/occ-conflicts.md` |
| High subscription count, slow UI updates, excessive re-renders | `references/subscription-cost.md` |
| Function timeouts, transaction size errors, large payloads | `references/function-budget.md` |
| General "it's slow" with no specific signal | Start with `references/hot-path-rules.md` |
Multiple problem classes can overlap. Read the most relevant reference first, then check the others if symptoms remain.
## Escalate Larger Fixes
If the likely fix is invasive, cross-cutting, or migration-heavy, stop and present options before editing.
Examples:
- introducing digest or summary tables across multiple flows
- splitting documents to isolate frequently-updated fields
- reworking pagination or fetch strategy across several screens
- switching to a new index or denormalized field that needs migration-safe rollout
When correctness depends on handling old and new states during a rollout, consult `skills/convex-migration-helper/SKILL.md` for the migration workflow.
## Workflow
### 1. Scope the problem
Pick one concrete user flow from the actual project. Look at the codebase, client pages, and API surface to find the flow that matches the symptom.
Write down:
- entrypoint functions
- client callsites using `useQuery`, `usePaginatedQuery`, or `useMutation`
- tables read
- tables written
- whether the path is high-read, high-write, or both
### 2. Trace the full read and write set
For each function in the path:
1. Trace every `ctx.db.get()` and `ctx.db.query()`
2. Trace every `ctx.db.patch()`, `ctx.db.replace()`, and `ctx.db.insert()`
3. Note foreign-key lookups, JS-side filtering, and full-document reads
4. Identify all sibling functions touching the same tables
5. Identify reactive stats, aggregates, or widgets rendered on the same page
In Convex, every extra read increases transaction work, and every write can invalidate reactive subscribers. Treat read amplification and invalidation amplification as first-class problems.
### 3. Apply fixes from the relevant reference
Read the reference file matching your problem class. Each reference includes specific patterns, code examples, and a recommended fix order.
Do not stop at the single function named by an insight. Trace sibling readers and writers touching the same tables.
### 4. Fix sibling functions together
When one function touching a table has a performance bug, audit sibling functions for the same pattern.
After finding one problem, inspect both sibling readers and sibling writers for the same table family, including companion digest or summary tables.
Examples:
- If one list query switches from full docs to a digest table, inspect the other list queries for that table
- If one mutation isolates a frequently-updated field or splits a hot document, inspect the other writers to the same table
- If one read path needs a migration-safe rollout for an unbackfilled field, inspect sibling reads for the same rollout risk
Do not leave one path fixed and another path on the old pattern unless there is a clear product reason.
### 5. Verify before finishing
Confirm all of these:
1. Results are the same as before, no dropped records
2. Eliminated reads or writes are no longer in the path where expected
3. Fallback behavior works when denormalized or indexed fields are missing
4. Frequently-updated fields are isolated from widely-read documents where needed
5. Every relevant sibling reader and writer was inspected, not just the original function
## Reference Files
- `references/hot-path-rules.md` - Read amplification, invalidation, denormalization, indexes, digest tables
- `references/occ-conflicts.md` - Write contention, OCC resolution, hot document splitting
- `references/subscription-cost.md` - Reactive query cost, subscription granularity, point-in-time reads
- `references/function-budget.md` - Execution limits, transaction size, large documents, payload size
Also check the official [Convex Best Practices](https://docs.convex.dev/understanding/best-practices/) page for additional patterns covering argument validation, access control, and code organization that may surface during the audit.
## Checklist
- [ ] Gathered signals from insights, dashboard, or code audit
- [ ] Identified the problem class and read the matching reference
- [ ] Scoped one concrete user flow or function path
- [ ] Traced every read and write in that path
- [ ] Identified sibling functions touching the same tables
- [ ] Applied fixes from the reference, following the recommended fix order
- [ ] Fixed sibling functions consistently
- [ ] Verified behavior and confirmed no regressions

View File

@ -0,0 +1,10 @@
interface:
display_name: "Convex Performance Audit"
short_description: "Audit slow Convex reads, subscriptions, OCC conflicts, and limits."
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#EF4444"
default_prompt: "Audit this Convex app for performance issues. Start with the strongest signal available, identify the problem class, and suggest the smallest high-impact fix before proposing bigger structural changes."
policy:
allow_implicit_invocation: true

View File

@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" aria-hidden="true" data-slot="icon">
<path stroke-linecap="round" stroke-linejoin="round" d="M8.25 3v1.5M4.5 8.25H3m18 0h-1.5M4.5 12H3m18 0h-1.5m-15 3.75H3m18 0h-1.5M8.25 19.5V21M12 3v1.5m0 15V21m3.75-18v1.5m0 15V21m-9-1.5h10.5a2.25 2.25 0 0 0 2.25-2.25V6.75a2.25 2.25 0 0 0-2.25-2.25H6.75A2.25 2.25 0 0 0 4.5 6.75v10.5a2.25 2.25 0 0 0 2.25 2.25Zm.75-12h9v9h-9v-9Z"/>
</svg>

After

Width:  |  Height:  |  Size: 490 B

View File

@ -0,0 +1,232 @@
# Function Budget
Use these rules when functions are hitting execution limits, transaction size errors, or returning excessively large payloads to the client.
## Core Principle
Convex functions run inside transactions with budgets for time, reads, and writes. Staying well within these limits is not just about avoiding errors, it reduces latency and contention.
## Limits to Know
These are the current values from the [Convex limits docs](https://docs.convex.dev/production/state/limits). Check that page for the latest numbers.
| Resource | Limit |
| --------------------------------- | ----------------------------------------------------- |
| Query/mutation execution time | 1 second (user code only, excludes DB operations) |
| Action execution time | 10 minutes |
| Data read per transaction | 16 MiB |
| Data written per transaction | 16 MiB |
| Documents scanned per transaction | 32,000 (includes documents filtered out by `.filter`) |
| Index ranges read per transaction | 4,096 (each `db.get` and `db.query` call) |
| Documents written per transaction | 16,000 |
| Individual document size | 1 MiB |
| Function return value size | 16 MiB |
## Symptoms
- "Function execution took too long" errors
- "Transaction too large" or read/write set size errors
- Slow queries that read many documents
- Client receiving large payloads that slow down page load
- `npx convex insights --details` showing high bytes read
## Common Causes
### Unbounded collection
A query that calls `.collect()` on a table without a reasonable limit. As the table grows, the query reads more and more documents.
### Large document reads on hot paths
Reading documents with large fields (rich text, embedded media references, long arrays) when only a small subset of the data is needed for the current view.
### Mutation doing too much work
A single mutation that updates hundreds of documents, backfills data, or rebuilds derived state in one transaction.
### Returning too much data to the client
A query returning full documents when the client only needs a few fields.
## Fix Order
### 1. Bound your reads
Never `.collect()` without a limit on a table that can grow unbounded.
```ts
// Bad: unbounded read, breaks as the table grows
const messages = await ctx.db.query("messages").collect();
```
```ts
// Good: paginate or limit
const messages = await ctx.db
.query("messages")
.withIndex("by_channel", (q) => q.eq("channelId", channelId))
.order("desc")
.take(50);
```
### 2. Read smaller shapes
If the list page only needs title, author, and date, do not read full documents with rich content fields.
Use digest or summary tables for hot list pages. See `hot-path-rules.md` for the digest table pattern.
### 3. Break large mutations into batches
If a mutation needs to update hundreds of documents, split it into a self-scheduling chain.
```ts
// Bad: one mutation updating every row
export const backfillAll = internalMutation({
handler: async (ctx) => {
const docs = await ctx.db.query("items").collect();
for (const doc of docs) {
await ctx.db.patch(doc._id, { newField: computeValue(doc) });
}
},
});
```
```ts
// Good: cursor-based batch processing
export const backfillBatch = internalMutation({
args: { cursor: v.optional(v.string()), batchSize: v.optional(v.number()) },
handler: async (ctx, args) => {
const batchSize = args.batchSize ?? 100;
const result = await ctx.db
.query("items")
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
for (const doc of result.page) {
if (doc.newField === undefined) {
await ctx.db.patch(doc._id, { newField: computeValue(doc) });
}
}
if (!result.isDone) {
await ctx.scheduler.runAfter(0, internal.items.backfillBatch, {
cursor: result.continueCursor,
batchSize,
});
}
},
});
```
### 4. Move heavy work to actions
Queries and mutations run inside Convex's transactional runtime with strict budgets. If you need to do CPU-intensive computation, call external APIs, or process large files, use an action instead.
Actions run outside the transaction and can call mutations to write results back.
```ts
// Bad: heavy computation inside a mutation
export const processUpload = mutation({
handler: async (ctx, args) => {
const result = expensiveComputation(args.data);
await ctx.db.insert("results", result);
},
});
```
```ts
// Good: action for heavy work, mutation for the write
export const processUpload = action({
handler: async (ctx, args) => {
const result = expensiveComputation(args.data);
await ctx.runMutation(internal.results.store, { result });
},
});
```
### 5. Trim return values
Only return what the client needs. If a query fetches full documents but the component only renders a few fields, map the results before returning.
```ts
// Bad: returns full documents including large content fields
export const list = query({
handler: async (ctx) => {
return await ctx.db.query("articles").take(20);
},
});
```
```ts
// Good: project to only the fields the client needs
export const list = query({
handler: async (ctx) => {
const articles = await ctx.db.query("articles").take(20);
return articles.map((a) => ({
_id: a._id,
title: a.title,
author: a.author,
createdAt: a._creationTime,
}));
},
});
```
### 6. Replace `ctx.runQuery` and `ctx.runMutation` with helper functions
Inside queries and mutations, `ctx.runQuery` and `ctx.runMutation` have overhead compared to calling a plain TypeScript helper function. They run in the same transaction but pay extra per-call cost.
```ts
// Bad: unnecessary overhead from ctx.runQuery inside a mutation
export const createProject = mutation({
handler: async (ctx, args) => {
const user = await ctx.runQuery(api.users.getCurrentUser);
await ctx.db.insert("projects", { ...args, ownerId: user._id });
},
});
```
```ts
// Good: plain helper function, no extra overhead
export const createProject = mutation({
handler: async (ctx, args) => {
const user = await getCurrentUser(ctx);
await ctx.db.insert("projects", { ...args, ownerId: user._id });
},
});
```
Exception: components require `ctx.runQuery`/`ctx.runMutation`. Use them there, but prefer helpers everywhere else.
### 7. Avoid unnecessary `runAction` calls
`runAction` from within an action creates a separate function invocation with its own memory and CPU budget. The parent action just sits idle waiting. Replace with a plain TypeScript function call unless you need a different runtime (e.g. calling Node.js code from the Convex runtime).
```ts
// Bad: runAction overhead for no reason
export const processItems = action({
handler: async (ctx, args) => {
for (const item of args.items) {
await ctx.runAction(internal.items.processOne, { item });
}
},
});
```
```ts
// Good: plain function call
export const processItems = action({
handler: async (ctx, args) => {
for (const item of args.items) {
await processOneItem(ctx, { item });
}
},
});
```
## Verification
1. No function execution or transaction size errors
2. `npx convex insights --details` shows reduced bytes read
3. Large mutations are batched and self-scheduling
4. Client payloads are reasonably sized for the UI they serve
5. `ctx.runQuery`/`ctx.runMutation` in queries and mutations replaced with helpers where possible
6. Sibling functions with similar patterns were checked

View File

@ -0,0 +1,368 @@
# Hot Path Rules
Use these rules when the top-level workflow points to read amplification, denormalization, index rollout, reactive query cost, or invalidation-heavy writes.
## Contents
- Core Principle
- Consistency Rule
- 1. Push Filters To Storage (indexes, migration rule, redundant indexes)
- 2. Minimize Data Sources (denormalization, fallback rule)
- 3. Minimize Row Size (digest tables)
- 4. Skip No-Op Writes
- 5. Match Consistency To Read Patterns (high-read/low-write, high-read/high-write)
- Convex-Specific Notes (reactive queries, point-in-time reads, triggers, aggregates, backfills)
- Verification
## Core Principle
Every byte read or written multiplies with concurrency.
Think:
`cost x calls_per_second x 86400`
In Convex, every write can also fan out into reactive invalidation, replication work, and downstream sync.
## Consistency Rule
If you fix a hot-path pattern for one function, audit sibling functions touching the same tables for the same pattern.
Do this especially for:
- multiple list queries over the same table
- multiple writers to the same table
- public browse and search queries over the same records
- helper functions reused by more than one endpoint
## 1. Push Filters To Storage
Both JavaScript `.filter()` and the Convex query `.filter()` method after a DB scan mean you already paid for the read. The Convex `.filter()` method has the same performance as filtering in JS, it does not push the predicate to the storage layer. Only `.withIndex()` and `.withSearchIndex()` actually reduce the documents scanned.
Prefer:
- `withIndex(...)`
- `.withSearchIndex(...)` for text search
- narrower tables
- summary tables
before accepting a scan-plus-filter pattern.
```ts
// Bad: scans then filters in JavaScript
export const listOpen = query({
args: {},
handler: async (ctx) => {
const tasks = await ctx.db.query("tasks").collect();
return tasks.filter((task) => task.status === "open");
},
});
```
```ts
// Also bad: Convex .filter() does not push to storage either
export const listOpen = query({
args: {},
handler: async (ctx) => {
return await ctx.db
.query("tasks")
.filter((q) => q.eq(q.field("status"), "open"))
.collect();
},
});
```
```ts
// Good: use an index so storage does the filtering
export const listOpen = query({
args: {},
handler: async (ctx) => {
return await ctx.db
.query("tasks")
.withIndex("by_status", (q) => q.eq("status", "open"))
.collect();
},
});
```
### Migration rule for indexes
New indexes on partially backfilled fields can create correctness bugs during rollout.
Important Convex detail:
`undefined !== false`
If an older document is missing a field entirely, it will not match a compound index entry that expects `false`.
Do not trust old comments saying a field is "not backfilled" or "already backfilled". Verify.
If correctness depends on handling old and new states during rollout, do not improvise a partial-backfill workaround in the hot path. Use a migration-safe rollout and consult `skills/convex-migration-helper/SKILL.md`.
```ts
// Bad: optional booleans can miss older rows where the field is undefined
const projects = await ctx.db
.query("projects")
.withIndex("by_archived_and_updated", (q) => q.eq("isArchived", false))
.order("desc")
.take(20);
```
```ts
// Good: switch hot-path reads only after the rollout is migration-safe
// See the migration helper skill for dual-read / backfill / cutover patterns.
```
### Check for redundant indexes
Indexes like `by_foo` and `by_foo_and_bar` are usually redundant. You only need `by_foo_and_bar`, since you can query it with just the `foo` condition and omit `bar`. Extra indexes add storage cost and write overhead on every insert, patch, and delete.
```ts
// Bad: two indexes where one would do
defineTable({ team: v.id("teams"), user: v.id("users") })
.index("by_team", ["team"])
.index("by_team_and_user", ["team", "user"]);
```
```ts
// Good: single compound index serves both query patterns
defineTable({ team: v.id("teams"), user: v.id("users") }).index("by_team_and_user", [
"team",
"user",
]);
```
Exception: `.index("by_foo", ["foo"])` is really an index on `foo` + `_creationTime`, while `.index("by_foo_and_bar", ["foo", "bar"])` is on `foo` + `bar` + `_creationTime`. If you need results sorted by `foo` then `_creationTime`, you need the single-field index because the compound one would sort by `bar` first.
## 2. Minimize Data Sources
Trace every read.
If a function resolves a foreign key for a tiny display field and a denormalized copy already exists, prefer the denormalized field on the hot path.
### When to denormalize
Denormalize when all of these are true:
- the path is hot
- the joined document is much larger than the field you need
- many readers are paying that join cost repeatedly
Useful mental model:
`join_cost = rows_per_page x foreign_doc_size x pages_per_second`
Small-table joins are often fine. Large-document joins for tiny fields on hot list pages are usually not.
### Fallback rule
Denormalized data is an optimization. Live data is the correctness path.
Rules:
- If the denormalized field is missing or null, fall back to the live read
- Do not show placeholders instead of falling back
- In lookup maps, only include fully populated entries
```ts
// Bad: missing denormalized data becomes a placeholder and blocks correctness
const ownerName = project.ownerName ?? "Unknown owner";
```
```ts
// Good: denormalized data is an optimization, not the only source of truth
const ownerName = project.ownerName ?? (await ctx.db.get(project.ownerId))?.name ?? null;
```
Bad lookup map pattern:
```ts
const ownersById = {
[project.ownerId]: { ownerName: null },
};
```
That blocks fallback because the map says "I have data" when it does not.
Good lookup map pattern:
```ts
const ownersById =
project.ownerName !== undefined && project.ownerName !== null
? { [project.ownerId]: { ownerName: project.ownerName } }
: {};
```
### No denormalized copy yet
Prefer adding fields to an existing summary, companion, or digest table instead of bloating the primary hot-path table.
If introducing the new field or table requires a staged rollout, backfill, or old/new-shape handling, use the migration helper skill for the rollout plan.
Rollout order:
1. Update schema
2. Update write path
3. Backfill
4. Switch read path
## 3. Minimize Row Size
Hot list pages should read the smallest document shape that still answers the UI.
Prefer summary or digest tables over full source tables when:
- the list page only needs a subset of fields
- source documents are large
- the query is high volume
An 800 byte summary row is materially cheaper than a 3 KB full document on a hot page.
Digest tables are a tradeoff, not a default:
- Worth it when the path is clearly hot, the source rows are much larger than the UI needs, or many readers are repeatedly paying the same join and payload cost
- Probably not worth it when an indexed read on the source table is already cheap enough, the table is still small, or the extra write and migration complexity would dominate the benefit
```ts
// Bad: list page reads source docs, then joins owner data per row
const projects = await ctx.db
.query("projects")
.withIndex("by_public", (q) => q.eq("isPublic", true))
.collect();
```
```ts
// Good: list page reads the smaller digest shape first
const projects = await ctx.db
.query("projectDigests")
.withIndex("by_public_and_updated", (q) => q.eq("isPublic", true))
.order("desc")
.take(20);
```
## 4. Isolate Frequently-Updated Fields
Convex already no-ops unchanged writes. The invalidation problem here is real writes hitting documents that many queries subscribe to.
Move high-churn fields like `lastSeen`, counters, presence, or ephemeral status off widely-read documents when most readers do not need them.
Apply this across sibling writers too. Splitting one write path does not help much if three other mutations still update the same widely-read document.
```ts
// Bad: every presence heartbeat invalidates subscribers to the whole profile
await ctx.db.patch(user._id, {
name: args.name,
avatarUrl: args.avatarUrl,
lastSeen: Date.now(),
});
```
```ts
// Good: keep profile reads stable, move heartbeat updates to a separate document
await ctx.db.patch(user._id, {
name: args.name,
avatarUrl: args.avatarUrl,
});
await ctx.db.patch(presence._id, {
lastSeen: Date.now(),
});
```
## 5. Match Consistency To Read Patterns
Choose read strategy based on traffic shape.
### High-read, low-write
Examples:
- public browse pages
- search results
- landing pages
- directory listings
Prefer:
- point-in-time reads where appropriate
- explicit refresh
- local state for pagination
- caching where appropriate
Do not treat subscriptions as automatically wrong here. Prefer point-in-time reads only when the product does not need live freshness and the reactive cost is material. See `subscription-cost.md` for detailed patterns.
### High-read, high-write
Examples:
- collaborative editors
- live dashboards
- presence-heavy views
Reactive queries may be worth the ongoing cost.
## Convex-Specific Notes
### Reactive queries
Every `ctx.db.get()` and `ctx.db.query()` contributes to the invalidation set for the query.
On the client:
- `useQuery` creates a live subscription
- `usePaginatedQuery` creates a live subscription per page
For low-freshness flows, consider a point-in-time read instead of a live subscription only when the product does not need updates pushed automatically.
### Point-in-time reads
Framework helpers, server-rendered fetches, or one-shot client reads can avoid ongoing subscription cost when live updates are not useful.
Use them for:
- aggregate snapshots
- reports
- low-churn listings
- pages where explicit refresh is fine
### Triggers and fan-out
Triggers fire on every write, including writes that did not materially change the document.
When a write exists only to keep derived state in sync:
- diff before patching
- move expensive non-blocking work to `ctx.scheduler.runAfter` when appropriate
### Aggregates
Reactive global counts invalidate frequently on busy tables.
Prefer:
- one-shot aggregate fetches
- periodic recomputation
- precomputed summary rows
for global stats that do not need live updates every second.
### Backfills
For larger backfills, use cursor-based, self-scheduling `internalMutation` jobs or the migrations component.
Deploy code that can handle both states before running the backfill.
During the gap:
- writes should populate the new shape
- reads should fall back safely
## Verification
Before closing the audit, confirm:
1. Same results as before, no dropped records
2. The removed table or lookup is no longer in the hot-path read set
3. Tests or validation cover fallback behavior
4. Migration safety is preserved while fields or indexes are unbackfilled
5. Sibling functions were fixed consistently

View File

@ -0,0 +1,114 @@
# OCC Conflict Resolution
Use these rules when insights, logs, or dashboard health show OCC (Optimistic Concurrency Control) conflicts, mutation retries, or write contention on hot tables.
## Core Principle
Convex uses optimistic concurrency control. When two transactions read or write overlapping data, one succeeds and the other retries automatically. High contention means wasted work and increased latency.
## Symptoms
- OCC conflict errors in deployment logs or health page
- Mutations retrying multiple times before succeeding
- User-visible latency spikes on write-heavy pages
- `npx convex insights --details` showing high conflict rates
## Common Causes
### Hot documents
Multiple mutations writing to the same document concurrently. Classic examples: a global counter, a shared settings row, or a "last updated" timestamp on a parent record.
### Broad read sets causing false conflicts
A query that scans a large table range creates a broad read set. If any write touches that range, the query's transaction conflicts even if the specific document the query cared about was not modified.
### Fan-out from triggers or cascading writes
A single user action triggers multiple mutations that all touch related documents. Each mutation competes with the others.
Database triggers (e.g. from `convex-helpers`) run inside the same transaction as the mutation that caused them. If a trigger does heavy work, reads extra tables, or writes to many documents, it extends the transaction's read/write set and increases the window for conflicts. Keep trigger logic minimal, or move expensive derived work to a scheduled function.
### Write-then-read chains
A mutation writes a document, then a reactive query re-reads it, then another mutation writes it again. Under load, these chains stack up.
## Fix Order
### 1. Reduce read set size
Narrower reads mean fewer false conflicts.
```ts
// Bad: broad scan creates a wide conflict surface
const allTasks = await ctx.db.query("tasks").collect();
const mine = allTasks.filter((t) => t.ownerId === userId);
```
```ts
// Good: indexed query touches only relevant documents
const mine = await ctx.db
.query("tasks")
.withIndex("by_owner", (q) => q.eq("ownerId", userId))
.collect();
```
### 2. Split hot documents
When many writers target the same document, split the contention point.
```ts
// Bad: every vote increments the same counter document
const counter = await ctx.db.get(pollCounterId);
await ctx.db.patch(pollCounterId, { count: counter!.count + 1 });
```
```ts
// Good: shard the counter across multiple documents, aggregate on read
const shardIndex = Math.floor(Math.random() * SHARD_COUNT);
const shardId = shardIds[shardIndex];
const shard = await ctx.db.get(shardId);
await ctx.db.patch(shardId, { count: shard!.count + 1 });
```
Aggregate the shards in a query or scheduled job when you need the total.
### 3. Move non-critical work to scheduled functions
If a mutation does primary work plus secondary bookkeeping (analytics, non-critical notifications, cache warming), the bookkeeping extends the transaction's lifetime and read/write set.
```ts
// Bad: canonical write and derived work happen in the same transaction
await ctx.db.patch(userId, { name: args.name });
await ctx.db.insert("userUpdateAnalytics", {
userId,
kind: "name_changed",
name: args.name,
});
```
```ts
// Good: keep the primary write small, defer the analytics work
await ctx.db.patch(userId, { name: args.name });
await ctx.scheduler.runAfter(0, internal.users.recordNameChangeAnalytics, {
userId,
name: args.name,
});
```
### 4. Combine competing writes
If two mutations must update the same document atomically, consider whether they can be combined into a single mutation call from the client, reducing round trips and conflict windows.
Do not introduce artificial locks or queues unless the above steps have been tried first.
## Related: Invalidation Scope
Splitting hot documents also reduces subscription invalidation, not just OCC contention. If a document is written frequently and read by many queries, those queries re-run on every write even when the fields they care about have not changed. See `subscription-cost.md` section 4 ("Isolate frequently-updated fields") for that pattern.
## Verification
1. OCC conflict rate has dropped in insights or dashboard
2. Mutation latency is lower and more consistent
3. No data correctness regressions from splitting or scheduling changes
4. Sibling writers to the same hot documents were fixed consistently

View File

@ -0,0 +1,249 @@
# Subscription Cost
Use these rules when the problem is too many reactive subscriptions, queries invalidating too frequently, or React components re-rendering excessively due to Convex state changes.
## Core Principle
Every `useQuery` and `usePaginatedQuery` call creates a live subscription. The server tracks the query's read set and re-executes the query whenever any document in that read set changes. Subscription cost scales with:
`subscriptions x invalidation_frequency x query_cost`
Subscriptions are not inherently bad. Convex reactivity is often the right default. The goal is to reduce unnecessary invalidation work, not to eliminate subscriptions on principle.
## Symptoms
- Dashboard shows high active subscription count
- UI feels sluggish or laggy despite fast individual queries
- React profiling shows frequent re-renders from Convex state
- Pages with many components each running their own `useQuery`
- Paginated lists where every loaded page stays subscribed
## Common Causes
### Reactive queries on low-freshness flows
Some user flows are read-heavy and do not need live updates every time the underlying data changes. In those cases, ongoing subscriptions may cost more than they are worth.
### Overly broad queries
A query that returns a large result set invalidates whenever any document in that set changes. The broader the query, the more frequent the invalidation.
### Too many subscriptions per page
A page with 20 list items, each running its own `useQuery` to fetch related data, creates 20+ subscriptions per visitor.
### Paginated queries keeping all pages live
`usePaginatedQuery` with `loadMore` keeps every loaded page subscribed. On a page where a user has scrolled through 10 pages, all 10 stay reactive.
### Frequently-updated fields on widely-read documents
A document that many queries touch gets a frequently-updated field (like `lastSeen`, `lastActiveAt`, or a counter). Every write to that field invalidates every subscription that reads the document, even if those subscriptions never use the field. This is different from OCC conflicts (see `occ-conflicts.md`), which are write-vs-write contention. This is write-vs-subscription: the write succeeds fine, but it forces hundreds of queries to re-run for no reason.
## Fix Order
### 1. Use point-in-time reads when live updates are not valuable
Keep `useQuery` and `usePaginatedQuery` by default when the product benefits from fresh live data.
Consider a point-in-time read instead when all of these are true:
- the flow is high-read
- the underlying data changes less often than users need to see
- explicit refresh, periodic refresh, or a fresh read on navigation is acceptable
Possible implementations depend on environment:
- a server-rendered fetch
- a framework helper like `fetchQuery`
- a point-in-time client read such as `ConvexHttpClient.query()`
```ts
// Reactive by default when fresh live data matters
function TeamPresence() {
const presence = useQuery(api.teams.livePresence, { teamId });
return <PresenceList users={presence} />;
}
```
```ts
// Point-in-time read when explicit refresh is acceptable
import { ConvexHttpClient } from "convex/browser";
const client = new ConvexHttpClient(import.meta.env.VITE_CONVEX_URL);
function SnapshotView() {
const [items, setItems] = useState<Item[]>([]);
useEffect(() => {
client.query(api.items.snapshot).then(setItems);
}, []);
return <ItemGrid items={items} />;
}
```
Good candidates for point-in-time reads:
- aggregate snapshots
- reports
- low-churn listings
- flows where explicit refresh is already acceptable
Keep reactive for:
- collaborative editing
- live dashboards
- presence-heavy views
- any surface where users expect fresh changes to appear automatically
### 2. Batch related data into fewer queries
Instead of N components each fetching their own related data, fetch it in a single query.
```ts
// Bad: each card fetches its own author
function ProjectCard({ project }: { project: Project }) {
const author = useQuery(api.users.get, { id: project.authorId });
return <Card title={project.name} author={author?.name} />;
}
```
```ts
// Good: parent query returns projects with author names included
function ProjectList() {
const projects = useQuery(api.projects.listWithAuthors);
return projects?.map((p) => (
<Card key={p._id} title={p.name} author={p.authorName} />
));
}
```
This can use denormalized fields or server-side joins in the query handler. Either way, it is one subscription instead of N.
This is not automatically better. If the combined query becomes much broader and invalidates much more often, several narrower subscriptions may be the better tradeoff. Optimize for total invalidation cost, not raw subscription count.
### 3. Use skip to avoid unnecessary subscriptions
The `"skip"` value prevents a subscription from being created when the arguments are not ready.
```ts
// Bad: subscribes with undefined args, wastes a subscription slot
const profile = useQuery(api.users.getProfile, { userId: selectedId! });
```
```ts
// Good: skip when there is nothing to fetch
const profile = useQuery(api.users.getProfile, selectedId ? { userId: selectedId } : "skip");
```
### 4. Isolate frequently-updated fields into separate documents
If a document is widely read but has a field that changes often, move that field to a separate document. Queries that do not need the field will no longer be invalidated by its writes.
```ts
// Bad: lastSeen lives on the user doc, every heartbeat invalidates
// every query that reads this user
const users = defineTable({
name: v.string(),
email: v.string(),
lastSeen: v.number(),
});
```
```ts
// Good: lastSeen lives in a separate heartbeat doc
const users = defineTable({
name: v.string(),
email: v.string(),
heartbeatId: v.id("heartbeats"),
});
const heartbeats = defineTable({
lastSeen: v.number(),
});
```
Queries that only need `name` and `email` no longer re-run on every heartbeat. Queries that actually need online status fetch the heartbeat document explicitly.
For an even further optimization, if you only need a coarse online/offline boolean rather than the exact `lastSeen` timestamp, add a separate presence document with an `isOnline` flag. Update it immediately when a user comes online, and use a cron to batch-mark users offline when their heartbeat goes stale. This way the presence query only invalidates when online status actually changes, not on every heartbeat.
### 5. Use the aggregate component for counts and sums
Reactive global counts (`SELECT COUNT(*)` equivalent) invalidate on every insert or delete to the table. The [`@convex-dev/aggregate`](https://www.npmjs.com/package/@convex-dev/aggregate) component maintains denormalized COUNT, SUM, and MAX values efficiently so you do not need a reactive query scanning the full table.
Use it for leaderboards, totals, "X items" badges, or any stat that would otherwise require scanning many rows reactively.
If the aggregate component is not appropriate, prefer point-in-time reads for global stats, or precomputed summary rows updated by a cron or trigger, over reactive queries that scan large tables.
### 6. Narrow query read sets
Queries that return less data and touch fewer documents invalidate less often.
```ts
// Bad: returns all fields, invalidates on any field change
export const list = query({
handler: async (ctx) => {
return await ctx.db.query("projects").collect();
},
});
```
```ts
// Good: use a digest table with only the fields the list needs
export const listDigests = query({
handler: async (ctx) => {
return await ctx.db.query("projectDigests").collect();
},
});
```
Writes to fields not in the digest table do not invalidate the digest query.
### 7. Remove `Date.now()` from queries
Using `Date.now()` inside a query defeats Convex's query cache. The cache is invalidated frequently to avoid showing stale time-dependent results, which increases database work even when the underlying data has not changed.
```ts
// Bad: Date.now() defeats query caching and causes frequent re-evaluation
const releasedPosts = await ctx.db
.query("posts")
.withIndex("by_released_at", (q) => q.lte("releasedAt", Date.now()))
.take(100);
```
```ts
// Good: use a boolean field updated by a scheduled function
const releasedPosts = await ctx.db
.query("posts")
.withIndex("by_is_released", (q) => q.eq("isReleased", true))
.take(100);
```
If the query must compare against a time value, pass it as an explicit argument from the client and round it to a coarse interval (e.g. the most recent minute) so requests within that window share the same cache entry.
### 8. Consider pagination strategy
For long lists where users scroll through many pages:
- If the data does not need live updates, use point-in-time fetching with manual "load more"
- If it does need live updates, accept the subscription cost but limit the number of loaded pages
- Consider whether older pages can be unloaded as the user scrolls forward
### 9. Separate backend cost from UI churn
If the main problem is loading flash or UI churn when query arguments change, stabilizing the reactive UI behavior may be better than replacing reactivity altogether.
Treat this as a UX problem first when:
- the underlying query is already reasonably cheap
- the complaint is flicker, loading flashes, or re-render churn
- live updates are still desirable once fresh data arrives
## Verification
1. Subscription count in dashboard is lower for the affected pages
2. UI responsiveness has improved
3. React profiling shows fewer unnecessary re-renders
4. Surfaces that do not need live updates are not paying for persistent subscriptions unnecessarily
5. Sibling pages with similar patterns were updated consistently

View File

@ -0,0 +1,341 @@
---
name: convex-quickstart
description: Creates or adds Convex to an app. Use for new Convex projects, npm create convex@latest, frontend setup, env vars, or the first npx convex dev run.
---
# Convex Quickstart
Set up a working Convex project as fast as possible.
## When to Use
- Starting a brand new project with Convex
- Adding Convex to an existing React, Next.js, Vue, Svelte, or other app
- Scaffolding a Convex app for prototyping
## When Not to Use
- The project already has Convex installed and `convex/` exists - just start building
- You only need to add auth to an existing Convex app - use the `convex-setup-auth` skill
## Workflow
1. Determine the starting point: new project or existing app
2. If new project, pick a template and scaffold with `npm create convex@latest`
3. If existing app, install `convex` and wire up the provider
4. Run `npx convex dev` to connect a deployment and start the dev loop
5. Verify the setup works
## Path 1: New Project (Recommended)
Use the official scaffolding tool. It creates a complete project with the frontend framework, Convex backend, and all config wired together.
### Pick a template
| Template | Stack |
| -------------------------- | ----------------------------------------- |
| `react-vite-shadcn` | React + Vite + Tailwind + shadcn/ui |
| `nextjs-shadcn` | Next.js App Router + Tailwind + shadcn/ui |
| `react-vite-clerk-shadcn` | React + Vite + Clerk auth + shadcn/ui |
| `nextjs-clerk` | Next.js + Clerk auth |
| `nextjs-convexauth-shadcn` | Next.js + Convex Auth + shadcn/ui |
| `nextjs-lucia-shadcn` | Next.js + Lucia auth + shadcn/ui |
| `bare` | Convex backend only, no frontend |
If the user has not specified a preference, default to `react-vite-shadcn` for simple apps or `nextjs-shadcn` for apps that need SSR or API routes.
You can also use any GitHub repo as a template:
```bash
npm create convex@latest my-app -- -t owner/repo
npm create convex@latest my-app -- -t owner/repo#branch
```
### Scaffold the project
Always pass the project name and template flag to avoid interactive prompts:
```bash
npm create convex@latest my-app -- -t react-vite-shadcn
cd my-app
npm install
```
The scaffolding tool creates files but does not run `npm install`, so you must run it yourself.
To scaffold in the current directory (if it is empty):
```bash
npm create convex@latest . -- -t react-vite-shadcn
npm install
```
### Start the dev loop
`npx convex dev` is a long-running watcher process that syncs backend code to a Convex deployment on every save. It also requires authentication on first run (browser-based OAuth). Both of these make it unsuitable for an agent to run directly.
**Ask the user to run this themselves:**
Tell the user to run `npx convex dev` in their terminal. On first run it will prompt them to log in or develop anonymously. Once running, it will:
- Create a Convex project and dev deployment
- Write the deployment URL to `.env.local`
- Create the `convex/` directory with generated types
- Watch for changes and sync continuously
The user should keep `npx convex dev` running in the background while you work on code. The watcher will automatically pick up any files you create or edit in `convex/`.
**Exception - cloud or headless agents:** Environments that cannot open a browser for interactive login should use Agent Mode (see below) to run anonymously without user interaction.
### Start the frontend
The user should also run the frontend dev server in a separate terminal:
```bash
npm run dev
```
Vite apps serve on `http://localhost:5173`, Next.js on `http://localhost:3000`.
### What you get
After scaffolding, the project structure looks like:
```
my-app/
convex/ # Backend functions and schema
_generated/ # Auto-generated types (check this into git)
schema.ts # Database schema (if template includes one)
src/ # Frontend code (or app/ for Next.js)
package.json
.env.local # CONVEX_URL / VITE_CONVEX_URL / NEXT_PUBLIC_CONVEX_URL
```
The template already has:
- `ConvexProvider` wired into the app root
- Correct env var names for the framework
- Tailwind and shadcn/ui ready (for shadcn templates)
- Auth provider configured (for auth templates)
Proceed to adding schema, functions, and UI.
## Path 2: Add Convex to an Existing App
Use this when the user already has a frontend project and wants to add Convex as the backend.
### Install
```bash
npm install convex
```
### Initialize and start dev loop
Ask the user to run `npx convex dev` in their terminal. This handles login, creates the `convex/` directory, writes the deployment URL to `.env.local`, and starts the file watcher. See the notes in Path 1 about why the agent should not run this directly.
### Wire up the provider
The Convex client must wrap the app at the root. The setup varies by framework.
Create the `ConvexReactClient` at module scope, not inside a component:
```tsx
// Bad: re-creates the client on every render
function App() {
const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL as string);
return <ConvexProvider client={convex}>...</ConvexProvider>;
}
// Good: created once at module scope
const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL as string);
function App() {
return <ConvexProvider client={convex}>...</ConvexProvider>;
}
```
#### React (Vite)
```tsx
// src/main.tsx
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import { ConvexProvider, ConvexReactClient } from "convex/react";
import App from "./App";
const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL as string);
createRoot(document.getElementById("root")!).render(
<StrictMode>
<ConvexProvider client={convex}>
<App />
</ConvexProvider>
</StrictMode>,
);
```
#### Next.js (App Router)
```tsx
// app/ConvexClientProvider.tsx
"use client";
import { ConvexProvider, ConvexReactClient } from "convex/react";
import { ReactNode } from "react";
const convex = new ConvexReactClient(process.env.NEXT_PUBLIC_CONVEX_URL!);
export function ConvexClientProvider({ children }: { children: ReactNode }) {
return <ConvexProvider client={convex}>{children}</ConvexProvider>;
}
```
```tsx
// app/layout.tsx
import { ConvexClientProvider } from "./ConvexClientProvider";
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="en">
<body>
<ConvexClientProvider>{children}</ConvexClientProvider>
</body>
</html>
);
}
```
#### Other frameworks
For Vue, Svelte, React Native, TanStack Start, Remix, and others, follow the matching quickstart guide:
- [Vue](https://docs.convex.dev/quickstart/vue)
- [Svelte](https://docs.convex.dev/quickstart/svelte)
- [React Native](https://docs.convex.dev/quickstart/react-native)
- [TanStack Start](https://docs.convex.dev/quickstart/tanstack-start)
- [Remix](https://docs.convex.dev/quickstart/remix)
- [Node.js (no frontend)](https://docs.convex.dev/quickstart/nodejs)
### Environment variables
The env var name depends on the framework:
| Framework | Variable |
| ------------ | ------------------------ |
| Vite | `VITE_CONVEX_URL` |
| Next.js | `NEXT_PUBLIC_CONVEX_URL` |
| Remix | `CONVEX_URL` |
| React Native | `EXPO_PUBLIC_CONVEX_URL` |
`npx convex dev` writes the correct variable to `.env.local` automatically.
## Agent Mode (Cloud and Headless Agents)
When running in a cloud or headless agent environment where interactive browser login is not possible, set `CONVEX_AGENT_MODE=anonymous` to use a local anonymous deployment.
Add `CONVEX_AGENT_MODE=anonymous` to `.env.local`, or set it inline:
```bash
CONVEX_AGENT_MODE=anonymous npx convex dev
```
This runs a local Convex backend on the VM without requiring authentication, and avoids conflicting with the user's personal dev deployment.
## Verify the Setup
After setup, confirm everything is working:
1. The user confirms `npx convex dev` is running without errors
2. The `convex/_generated/` directory exists and has `api.ts` and `server.ts`
3. `.env.local` contains the deployment URL
## Writing Your First Function
Once the project is set up, create a schema and a query to verify the full loop works.
`convex/schema.ts`:
```ts
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
tasks: defineTable({
text: v.string(),
completed: v.boolean(),
}),
});
```
`convex/tasks.ts`:
```ts
import { query, mutation } from "./_generated/server";
import { v } from "convex/values";
export const list = query({
args: {},
handler: async (ctx) => {
return await ctx.db.query("tasks").collect();
},
});
export const create = mutation({
args: { text: v.string() },
handler: async (ctx, args) => {
await ctx.db.insert("tasks", { text: args.text, completed: false });
},
});
```
Use in a React component (adjust the import path based on your file location relative to `convex/`):
```tsx
import { useQuery, useMutation } from "convex/react";
import { api } from "../convex/_generated/api";
function Tasks() {
const tasks = useQuery(api.tasks.list);
const create = useMutation(api.tasks.create);
return (
<div>
<button onClick={() => create({ text: "New task" })}>Add</button>
{tasks?.map((t) => (
<div key={t._id}>{t.text}</div>
))}
</div>
);
}
```
## Development vs Production
Always use `npx convex dev` during development. It runs against your personal dev deployment and syncs code on save.
When ready to ship, deploy to production:
```bash
npx convex deploy
```
This pushes to the production deployment, which is separate from dev. Do not use `deploy` during development.
## Next Steps
- Add authentication: use the `convex-setup-auth` skill
- Design your schema: see [Schema docs](https://docs.convex.dev/database/schemas)
- Build components: use the `convex-create-component` skill
- Plan a migration: use the `convex-migration-helper` skill
- Add file storage: see [File Storage docs](https://docs.convex.dev/file-storage)
- Set up cron jobs: see [Scheduling docs](https://docs.convex.dev/scheduling)
## Checklist
- [ ] Determined starting point: new project or existing app
- [ ] If new project: scaffolded with `npm create convex@latest` using appropriate template
- [ ] If existing app: installed `convex` and wired up the provider
- [ ] User has `npx convex dev` running and connected to a deployment
- [ ] `convex/_generated/` directory exists with types
- [ ] `.env.local` has the deployment URL
- [ ] Verified a basic query/mutation round-trip works

View File

@ -0,0 +1,10 @@
interface:
display_name: "Convex Quickstart"
short_description: "Start a new Convex app or add Convex to an existing frontend."
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#F97316"
default_prompt: "Set up Convex for this project as fast as possible. First decide whether this is a new app or an existing app, then scaffold or integrate Convex and verify the setup works."
policy:
allow_implicit_invocation: true

View File

@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" aria-hidden="true" data-slot="icon">
<path stroke-linecap="round" stroke-linejoin="round" d="M21 12a9 9 0 1 1-18 0 9 9 0 0 1 18 0Z"/>
<path stroke-linecap="round" stroke-linejoin="round" d="M15.91 11.672a.375.375 0 0 1 0 .656l-5.603 3.113a.375.375 0 0 1-.557-.328V8.887c0-.286.307-.466.557-.327l5.603 3.112Z"/>
</svg>

After

Width:  |  Height:  |  Size: 435 B

View File

@ -0,0 +1,148 @@
---
name: convex-setup-auth
description: Sets up Convex auth, identity mapping, and access control. Use for login, auth providers, users tables, protected functions, or roles in a Convex app.
---
# Convex Authentication Setup
Implement secure authentication in Convex with user management and access control.
## When to Use
- Setting up authentication for the first time
- Implementing user management (users table, identity mapping)
- Creating authentication helper functions
- Setting up auth providers (Convex Auth, Clerk, WorkOS AuthKit, Auth0, custom JWT)
## When Not to Use
- Auth for a non-Convex backend
- Pure OAuth/OIDC documentation without a Convex implementation
- Debugging unrelated bugs that happen to surface near auth code
- The auth provider is already fully configured and the user only needs a one-line fix
## First Step: Choose the Auth Provider
Convex supports multiple authentication approaches. Do not assume a provider.
Before writing setup code:
1. Ask the user which auth solution they want, unless the repository already makes it obvious
2. If the repo already uses a provider, continue with that provider unless the user wants to switch
3. If the user has not chosen a provider and the repo does not make it obvious, ask before proceeding
Common options:
- [Convex Auth](https://docs.convex.dev/auth/convex-auth) - good default when the user wants auth handled directly in Convex
- [Clerk](https://docs.convex.dev/auth/clerk) - use when the app already uses Clerk or the user wants Clerk's hosted auth features
- [WorkOS AuthKit](https://docs.convex.dev/auth/authkit/) - use when the app already uses WorkOS or the user wants AuthKit specifically
- [Auth0](https://docs.convex.dev/auth/auth0) - use when the app already uses Auth0
- Custom JWT provider - use when integrating an existing auth system not covered above
Look for signals in the repo before asking:
- Dependencies such as `@clerk/*`, `@workos-inc/*`, `@auth0/*`, or Convex Auth packages
- Existing files such as `convex/auth.config.ts`, auth middleware, provider wrappers, or login components
- Environment variables that clearly point at a provider
## After Choosing a Provider
Read the provider's official guide and the matching local reference file:
- Convex Auth: [official docs](https://docs.convex.dev/auth/convex-auth), then `references/convex-auth.md`
- Clerk: [official docs](https://docs.convex.dev/auth/clerk), then `references/clerk.md`
- WorkOS AuthKit: [official docs](https://docs.convex.dev/auth/authkit/), then `references/workos-authkit.md`
- Auth0: [official docs](https://docs.convex.dev/auth/auth0), then `references/auth0.md`
The local reference files contain the concrete workflow, expected files and env vars, gotchas, and validation checks.
Use those sources for:
- package installation
- client provider wiring
- environment variables
- `convex/auth.config.ts` setup
- login and logout UI patterns
- framework-specific setup for React, Vite, or Next.js
For shared auth behavior, use the official Convex docs as the source of truth:
- [Auth in Functions](https://docs.convex.dev/auth/functions-auth) for `ctx.auth.getUserIdentity()`
- [Storing Users in the Convex Database](https://docs.convex.dev/auth/database-auth) for optional app-level user storage
- [Authentication](https://docs.convex.dev/auth) for general auth and authorization guidance
- [Convex Auth Authorization](https://labs.convex.dev/auth/authz) when the provider is Convex Auth
Prefer official docs over recalled steps, because provider CLIs and Convex Auth internals change between versions. Inventing setup from memory risks outdated patterns.
For third-party providers, only add app-level user storage if the app actually needs user documents in Convex. Not every app needs a `users` table.
For Convex Auth, follow the Convex Auth docs and built-in auth tables rather than adding a parallel `users` table plus `storeUser` flow, because Convex Auth already manages user records internally.
After running provider initialization commands, verify generated files and complete the post-init wiring steps the provider reference calls out. Initialization commands rarely finish the entire integration.
## Core Pattern: Protecting Backend Functions
The most common auth task is checking identity in Convex functions.
```ts
// Bad: trusting a client-provided userId
export const getMyProfile = query({
args: { userId: v.id("users") },
handler: async (ctx, args) => {
return await ctx.db.get(args.userId);
},
});
```
```ts
// Good: verifying identity server-side
export const getMyProfile = query({
args: {},
handler: async (ctx) => {
const identity = await ctx.auth.getUserIdentity();
if (!identity) throw new Error("Not authenticated");
return await ctx.db
.query("users")
.withIndex("by_tokenIdentifier", (q) => q.eq("tokenIdentifier", identity.tokenIdentifier))
.unique();
},
});
```
## Workflow
1. Determine the provider, either by asking the user or inferring from the repo
2. Ask whether the user wants local-only setup or production-ready setup now
3. Read the matching provider reference file
4. Follow the official provider docs for current setup details
5. Follow the official Convex docs for shared backend auth behavior, user storage, and authorization patterns
6. Only add app-level user storage if the docs and app requirements call for it
7. Add authorization checks for ownership, roles, or team access only where the app needs them
8. Verify login state, protected queries, environment variables, and production configuration if requested
If the flow blocks on interactive provider or deployment setup, ask the user explicitly for the exact human step needed, then continue after they complete it.
For UI-facing auth flows, offer to validate the real sign-up or sign-in flow after setup is done.
If the environment has browser automation tools, you can use them.
If it does not, give the user a short manual validation checklist instead.
## Reference Files
### Provider References
- `references/convex-auth.md`
- `references/clerk.md`
- `references/workos-authkit.md`
- `references/auth0.md`
## Checklist
- [ ] Chosen the correct auth provider before writing setup code
- [ ] Read the relevant provider reference file
- [ ] Asked whether the user wants local-only setup or production-ready setup
- [ ] Used the official provider docs for provider-specific wiring
- [ ] Used the official Convex docs for shared auth behavior and authorization patterns
- [ ] Only added app-level user storage if the app actually needs it
- [ ] Did not invent a cross-provider `users` table or `storeUser` flow for Convex Auth
- [ ] Added authentication checks in protected backend functions
- [ ] Added authorization checks where the app actually needs them
- [ ] Clear error messages ("Not authenticated", "Unauthorized")
- [ ] Client auth provider configured for the chosen provider
- [ ] If requested, production auth setup is covered too

View File

@ -0,0 +1,10 @@
interface:
display_name: "Convex Setup Auth"
short_description: "Set up Convex auth, user identity mapping, and access control."
icon_small: "./assets/icon.svg"
icon_large: "./assets/icon.svg"
brand_color: "#2563EB"
default_prompt: "Set up authentication for this Convex app. Figure out the provider first, then wire up the user model, identity mapping, and access control with the smallest solid implementation."
policy:
allow_implicit_invocation: true

View File

@ -0,0 +1,3 @@
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" aria-hidden="true" data-slot="icon">
<path stroke-linecap="round" stroke-linejoin="round" d="M16.5 10.5V6.75a4.5 4.5 0 1 0-9 0v3.75m-.75 11.25h10.5a2.25 2.25 0 0 0 2.25-2.25v-6.75a2.25 2.25 0 0 0-2.25-2.25H6.75a2.25 2.25 0 0 0-2.25 2.25v6.75a2.25 2.25 0 0 0 2.25 2.25Z"/>
</svg>

After

Width:  |  Height:  |  Size: 394 B

View File

@ -0,0 +1,116 @@
# Auth0
Official docs:
- https://docs.convex.dev/auth/auth0
- https://auth0.github.io/auth0-cli/
- https://auth0.github.io/auth0-cli/auth0_apps_create.html
Use this when the app already uses Auth0 or the user wants Auth0 specifically.
## Workflow
1. Confirm the user wants Auth0
2. Determine the app framework and whether Auth0 is already partly set up
3. Ask whether the user wants local-only setup or production-ready setup now
4. Read the official Convex and Auth0 guides before making changes
5. Ask whether they want the fastest setup path by installing the Auth0 CLI
6. If they agree, install the Auth0 CLI and do as much of the Auth0 app setup as possible through the CLI
7. If they do not want the CLI path, use the Auth0 dashboard path instead
8. Complete the relevant Auth0 frontend quickstart if the app does not already have Auth0 wired up
9. Configure `convex/auth.config.ts` with the Auth0 domain and client ID
10. Set environment variables for local and production environments
11. Wrap the app with `Auth0Provider` and `ConvexProviderWithAuth0`
12. Gate Convex-backed UI with Convex auth state
13. Try to verify Convex reports the user as authenticated after Auth0 login
14. If the refresh-token path fails, stop improvising and send the user back to the official docs
15. If the user wants production-ready setup, make sure the production Auth0 tenant and env vars are also covered
## What To Do
- Read the official Convex and Auth0 guide before writing setup code
- Prefer the Auth0 CLI path for mechanical setup if the user is willing to install it, but do not present it as a fully validated end-to-end path yet
- Ask the user directly: "The fastest path is to install the Auth0 CLI so I can do more of this for you. If you want, I can install it and then only ask you to log in when needed. Would you like me to do that?"
- Make sure the app has already completed the relevant Auth0 quickstart for its frontend
- Use the official examples for `Auth0Provider` and `ConvexProviderWithAuth0`
- If the Auth0 login or refresh flow starts failing in a way that is not clearly explained by the docs, say that plainly and fall back to the official docs instead of pretending the flow is validated
## Key Setup Areas
- install the Auth0 SDK for the app's framework
- configure `convex/auth.config.ts` with the Auth0 domain and client ID
- set environment variables for local and production environments
- wrap the app with `Auth0Provider` and `ConvexProviderWithAuth0`
- use Convex auth state when gating Convex-backed UI
## Files and Env Vars To Expect
- `convex/auth.config.ts`
- frontend app entry or provider wrapper
- Auth0 CLI install docs: `https://auth0.github.io/auth0-cli/`
- Auth0 environment variables commonly include:
- `AUTH0_DOMAIN`
- `AUTH0_CLIENT_ID`
- `VITE_AUTH0_DOMAIN`
- `VITE_AUTH0_CLIENT_ID`
## Concrete Steps
1. Start by reading `https://docs.convex.dev/auth/auth0` and the relevant Auth0 quickstart for the app's framework
2. Ask whether the user wants the Auth0 CLI path
3. If yes, install Auth0 CLI and have the user authenticate it with `auth0 login`
4. Use `auth0 apps create` with SPA settings, callback URL, logout URL, and web origins if creating a new app
5. If not using the CLI path, complete the relevant Auth0 frontend quickstart and create the Auth0 app in the dashboard
6. Get the Auth0 domain and client ID from the CLI output or the Auth0 dashboard
7. Install the Auth0 SDK for the app's framework
8. Create or update `convex/auth.config.ts` with the Auth0 domain and client ID
9. Set frontend and backend environment variables
10. Wrap the app in `Auth0Provider`
11. Replace plain `ConvexProvider` wiring with `ConvexProviderWithAuth0`
12. Run the normal Convex dev or deploy flow after backend config changes
13. Try the official provider config shown in the Convex docs
14. If login works but Convex auth or token refresh fails in a way you cannot clearly resolve, stop and tell the user to follow the official docs manually for now
15. Only claim success if the user can sign in and Convex recognizes the authenticated session
16. If the user wants production-ready setup, configure the production Auth0 tenant values and production environment variables too
## Gotchas
- The Convex docs assume the Auth0 side is already set up, so do not skip the Auth0 quickstart if the app is starting from scratch
- The Auth0 CLI is often the fastest path for a fresh setup, but it still requires the user to authenticate the CLI to their Auth0 tenant
- If the user agrees to install the Auth0 CLI, do the mechanical setup yourself instead of bouncing them through the dashboard
- If login succeeds but Convex still reports unauthenticated, double-check `convex/auth.config.ts` and whether the backend config was synced
- We were able to automate Auth0 app creation and Convex config wiring, but we did not fully validate the refresh-token path end to end
- In validation, the documented `useRefreshTokens={true}` and `cacheLocation="localstorage"` setup hit refresh-token failures, so do not present that path as settled
- If you hit Auth0 errors like `Unknown or invalid refresh token`, do not keep inventing fixes indefinitely, send the user back to the official docs and explain that this path is still under investigation
- Keep dev and prod tenants separate if the project uses different Auth0 environments
- Do not confuse "Auth0 login works" with "Convex can validate the Auth0 token". Both need to work.
- If the repo already uses Auth0, preserve existing redirect and tenant configuration unless the user asked to change it.
- Do not assume the local Auth0 tenant settings match production. Verify the production domain, client ID, and callback URLs separately.
- For local dev, make sure the Auth0 app settings match the app's real local port for callback URLs, logout URLs, and web origins
## Production
- Ask whether the user wants dev-only setup or production-ready setup
- If the answer is production-ready, make sure the production Auth0 tenant values, callback URLs, and Convex deployment config are all covered
- Verify production environment variables and redirect settings before calling the task complete
- Do not silently write a notes file into the repo by default. If the user wants rollout or handoff docs, create one explicitly.
## Validation
- Verify the user can complete the Auth0 login flow
- Verify Convex-authenticated UI renders only after Convex auth state is ready
- Verify protected Convex queries succeed after login
- Verify `ctx.auth.getUserIdentity()` is non-null in protected backend functions
- Verify the Auth0 app settings match the real local callback and logout URLs during development
- If the Auth0 refresh-token path fails, mark the setup as not fully validated and direct the user to the official docs instead of claiming the skill completed successfully
- If production-ready setup was requested, verify the production Auth0 configuration is also covered
## Checklist
- [ ] Confirm the user wants Auth0
- [ ] Ask whether the user wants local-only setup or production-ready setup
- [ ] Complete the relevant Auth0 frontend setup
- [ ] Configure `convex/auth.config.ts`
- [ ] Set environment variables
- [ ] Verify Convex authenticated state after login, or explicitly tell the user this path is still under investigation and send them to the official docs
- [ ] If requested, configure the production deployment too

View File

@ -0,0 +1,113 @@
# Clerk
Official docs:
- https://docs.convex.dev/auth/clerk
- https://clerk.com/docs/guides/development/integrations/databases/convex
Use this when the app already uses Clerk or the user wants Clerk's hosted auth features.
## Workflow
1. Confirm the user wants Clerk
2. Make sure the user has a Clerk account and a Clerk application
3. Determine the app framework:
- React
- Next.js
- TanStack Start
4. Ask whether the user wants local-only setup or production-ready setup now
5. Gather the Clerk keys and the Clerk Frontend API URL
6. Follow the correct framework section in the official docs
7. Complete the backend and client wiring
8. Verify Convex reports the user as authenticated after login
9. If the user wants production-ready setup, make sure the production Clerk config is also covered
## What To Do
- Read the official Convex and Clerk guide before writing setup code
- If the user does not already have Clerk set up, send them to `https://dashboard.clerk.com/sign-up` to create an account and `https://dashboard.clerk.com/apps/new` to create an application
- Send the user to `https://dashboard.clerk.com/apps/setup/convex` if the Convex integration is not already active
- Match the guide to the app's framework, usually React, Next.js, or TanStack Start
- Use the official examples for `ConvexProviderWithClerk`, `ClerkProvider`, and `useAuth`
## Key Setup Areas
- install the Clerk SDK for the framework in use
- configure `convex/auth.config.ts` with the Clerk issuer domain
- set the required Clerk environment variables
- wrap the app with `ClerkProvider` and `ConvexProviderWithClerk`
- use Convex auth-aware UI patterns such as `Authenticated`, `Unauthenticated`, and `AuthLoading`
## Files and Env Vars To Expect
- `convex/auth.config.ts`
- React or Vite client entry such as `src/main.tsx`
- Next.js client wrapper for Convex if using App Router
- Clerk account sign-up page: `https://dashboard.clerk.com/sign-up`
- Clerk app creation page: `https://dashboard.clerk.com/apps/new`
- Clerk Convex integration page: `https://dashboard.clerk.com/apps/setup/convex`
- Clerk API keys page: `https://dashboard.clerk.com/last-active?path=api-keys`
- Clerk environment variables:
- `CLERK_JWT_ISSUER_DOMAIN` for Convex backend validation in the Convex docs
- `CLERK_FRONTEND_API_URL` in the Clerk docs
- `VITE_CLERK_PUBLISHABLE_KEY` for Vite apps
- `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` for Next.js apps
- `CLERK_SECRET_KEY` for Next.js server-side Clerk setup where required
`CLERK_JWT_ISSUER_DOMAIN` and `CLERK_FRONTEND_API_URL` refer to the same Clerk Frontend API URL value. Do not treat them as two different URLs.
## Concrete Steps
1. If needed, create a Clerk account at `https://dashboard.clerk.com/sign-up`
2. If needed, create a Clerk application at `https://dashboard.clerk.com/apps/new`
3. Open `https://dashboard.clerk.com/last-active?path=api-keys` and copy the publishable key, plus the secret key for Next.js where needed
4. Open `https://dashboard.clerk.com/apps/setup/convex`
5. Activate the Convex integration in Clerk if it is not already active
6. Copy the Clerk Frontend API URL shown there
7. Install the Clerk package for the app's framework
8. Create or update `convex/auth.config.ts` so Convex validates Clerk tokens
9. Set the publishable key in the frontend environment
10. Set the issuer domain or Frontend API URL so Convex can validate the JWT
11. Replace plain `ConvexProvider` wiring with `ConvexProviderWithClerk`
12. Wrap the app in `ClerkProvider`
13. Use Convex auth helpers for authenticated rendering
14. Run the normal Convex dev or deploy flow after updating backend auth config
15. If the user wants production-ready setup, configure the production Clerk values and production issuer domain too
## Gotchas
- Prefer `useConvexAuth()` over raw Clerk auth state when deciding whether Convex-authenticated UI can render
- For Next.js, keep server and client boundaries in mind when creating the Convex provider wrapper
- After changing `convex/auth.config.ts`, run the normal Convex dev or deploy flow so the backend picks up the new config
- Do not stop at "Clerk login works". The important check is that Convex also sees the session and can authenticate requests.
- If the repo already uses Clerk, preserve its existing auth flow unless the user asked to change it.
- Do not assume the same Clerk values work for both dev and production. Check the production issuer domain and publishable key separately.
- The Convex setup page is where you get the Clerk Frontend API URL for Convex. Keep using the Clerk API keys page for the publishable key and the secret key.
- If Convex says no auth provider matched the token, first confirm the Clerk Convex integration was activated at `https://dashboard.clerk.com/apps/setup/convex`
- After activating the Clerk Convex integration, sign out completely and sign back in before retesting. An old Clerk session can keep using a token that Convex rejects.
## Production
- Ask whether the user wants dev-only setup or production-ready setup
- If the answer is production-ready, make sure production Clerk keys and issuer configuration are included
- Verify production redirect URLs and any production Clerk domain values before calling the task complete
- Do not silently write a notes file into the repo by default. If the user wants rollout or handoff docs, create one explicitly.
## Validation
- Verify the user can sign in with Clerk
- If the Clerk integration was just activated, verify after a full Clerk sign-out and fresh sign-in
- Verify `useConvexAuth()` reaches the authenticated state after Clerk login
- Verify protected Convex queries run successfully inside authenticated UI
- Verify `ctx.auth.getUserIdentity()` is non-null in protected backend functions
- If production-ready setup was requested, verify the production Clerk configuration is also covered
## Checklist
- [ ] Confirm the user wants Clerk
- [ ] Ask whether the user wants local-only setup or production-ready setup
- [ ] Follow the correct framework section in the official guide
- [ ] Set Clerk environment variables
- [ ] Configure `convex/auth.config.ts`
- [ ] Verify Convex authenticated state after login
- [ ] If requested, configure the production deployment too

View File

@ -0,0 +1,143 @@
# Convex Auth
Official docs: https://docs.convex.dev/auth/convex-auth
Setup guide: https://labs.convex.dev/auth/setup
Use this when the user wants auth handled directly in Convex rather than through a third-party provider.
## Workflow
1. Confirm the user wants Convex Auth specifically
2. Determine which sign-in methods the app needs:
- magic links or OTPs
- OAuth providers
- passwords and password reset
3. Ask whether the user wants local-only setup or production-ready setup now
4. Read the Convex Auth setup guide before writing code
5. Make sure the project has a configured Convex deployment:
- run `npx convex dev` first if `CONVEX_DEPLOYMENT` is not set
- if CLI configuration requires interactive human input, stop and ask the user to complete that step before continuing
6. Install the auth packages:
- `npm install @convex-dev/auth @auth/core@0.37.0`
7. Run the initialization command:
- `npx @convex-dev/auth`
8. Confirm the initializer created:
- `convex/auth.config.ts`
- `convex/auth.ts`
- `convex/http.ts`
9. Add the required `authTables` to `convex/schema.ts`
10. Replace plain `ConvexProvider` wiring with `ConvexAuthProvider`
11. Configure at least one auth method in `convex/auth.ts`
12. Run `npx convex dev --once` or the normal dev flow to push the updated schema and generated code
13. Verify the client can sign in successfully
14. Verify Convex receives authenticated identity in backend functions
15. If the user wants production-ready setup, make sure the same auth setup is configured for the production deployment as well
16. Only add a `users` table and `storeUser` flow if the app needs app-level user records inside Convex
## What This Reference Is For
- choosing Convex Auth as the default provider for a new Convex app
- understanding whether the app wants magic links, OTPs, OAuth, or passwords
- keeping the setup provider-specific while using the official Convex Auth docs for identity and authorization behavior
## What To Do
- Read the Convex Auth setup guide before writing setup code
- Follow the setup flow from the docs rather than recreating it from memory
- If the app is new, consider starting from the official starter flow instead of hand-wiring everything
- Treat `npx @convex-dev/auth` as a required initialization step for existing apps, not an optional extra
## Concrete Steps
1. Install `@convex-dev/auth` and `@auth/core@0.37.0`
2. Run `npx convex dev` if the project does not already have a configured deployment
3. If `npx convex dev` blocks on interactive setup, ask the user explicitly to finish configuring the Convex deployment
4. Run `npx @convex-dev/auth`
5. Confirm the generated auth setup is present before continuing:
- `convex/auth.config.ts`
- `convex/auth.ts`
- `convex/http.ts`
6. Add `authTables` to `convex/schema.ts`
7. Replace `ConvexProvider` with `ConvexAuthProvider` in the app entry
8. Configure the selected auth methods in `convex/auth.ts`
9. Run `npx convex dev --once` or the normal dev flow so the updated schema and auth files are pushed
10. Verify login locally
11. If the user wants production-ready setup, repeat the required auth configuration against the production deployment
## Expected Files and Decisions
- `convex/schema.ts`
- frontend app entry such as `src/main.tsx` or the framework-equivalent provider file
- generated Convex Auth setup produced by `npx @convex-dev/auth`
- an existing configured Convex deployment, or the ability to create one with `npx convex dev`
- `convex/auth.ts` starts with `providers: []` until the app configures actual sign-in methods
- Decide whether the user is creating a new app or adding auth to an existing app
- For a new app, prefer the official starter flow instead of rebuilding setup by hand
- Decide which auth methods the app needs:
- magic links or OTPs
- OAuth providers
- passwords
- Decide whether the user wants local-only setup or production-ready setup now
- Decide whether the app actually needs a `users` table inside Convex, or whether provider identity alone is enough
## Gotchas
- Do not assume a specific sign-in method. Ask which methods the app needs before wiring UI and backend behavior.
- `npx @convex-dev/auth` is important because it initializes the auth setup, including the key material. Do not skip it when adding Convex Auth to an existing project.
- `npx @convex-dev/auth` will fail if the project does not already have a configured `CONVEX_DEPLOYMENT`.
- `npx convex dev` may require interactive setup for deployment creation or project selection. If that happens, ask the user explicitly for that human step instead of guessing.
- `npx @convex-dev/auth` does not finish the whole integration by itself. You still need to add `authTables`, swap in `ConvexAuthProvider`, and configure at least one auth method.
- A project can still build even if `convex/auth.ts` still has `providers: []`, so do not treat a successful build as proof that sign-in is fully configured.
- Convex Auth does not mean every app needs a `users` table. If the app only needs authentication gates, `ctx.auth.getUserIdentity()` may be enough.
- If the app is greenfield, starting from the official starter flow is usually better than partially recreating it by hand.
- Do not stop at local dev setup if the user expects production-ready auth. The production deployment needs the auth setup too.
- Keep provider-specific setup and Convex Auth authorization behavior in the official docs instead of inventing shared patterns from memory.
## Production
- Ask whether the user wants dev-only setup or production-ready setup
- If the answer is production-ready, make sure the auth configuration is applied to the production deployment, not just the dev deployment
- Verify production-specific redirect URLs, auth method configuration, and deployment settings before calling the task complete
- Do not silently write a notes file into the repo by default. If the user wants rollout or handoff docs, create one explicitly.
## Human Handoff
If `npx convex dev` or deployment setup requires human input:
- stop and explain exactly what the user needs to do
- say why that step is required
- resume the auth setup immediately after the user confirms it is done
## Validation
- Verify the user can complete a sign-in flow
- Offer to validate sign up, sign out, and sign back in with the configured auth method
- If browser automation is available in the environment, you can do this directly
- If browser automation is not available, give the user a short manual validation checklist instead
- Verify `ctx.auth.getUserIdentity()` returns an identity in protected backend functions
- Verify protected UI only renders after Convex-authenticated state is ready
- Verify environment variables and redirect settings match the current app environment
- Verify `convex/auth.ts` no longer has an empty `providers: []` configuration once the app is meant to support real sign-in
- Run `npx convex dev --once` or the normal dev flow after setup changes and confirm Convex codegen and push succeed
- If production-ready setup was requested, verify the production deployment is also configured correctly
## Checklist
- [ ] Confirm the user wants Convex Auth specifically
- [ ] Ask whether the user wants local-only setup or production-ready setup
- [ ] Ensure a Convex deployment is configured before running auth initialization
- [ ] Install `@convex-dev/auth` and `@auth/core@0.37.0`
- [ ] Run `npx convex dev` first if needed
- [ ] Run `npx @convex-dev/auth`
- [ ] Confirm `convex/auth.config.ts`, `convex/auth.ts`, and `convex/http.ts` were created
- [ ] Follow the setup guide for package install and wiring
- [ ] Add `authTables` to `convex/schema.ts`
- [ ] Replace `ConvexProvider` with `ConvexAuthProvider`
- [ ] Configure at least one auth method in `convex/auth.ts`
- [ ] Run `npx convex dev --once` or the normal dev flow after setup changes
- [ ] Confirm which sign-in methods the app needs
- [ ] Verify the client can sign in and the backend receives authenticated identity
- [ ] Offer end-to-end validation of sign up, sign out, and sign back in
- [ ] If requested, configure the production deployment too
- [ ] Only add extra `users` table sync if the app needs app-level user records

View File

@ -0,0 +1,114 @@
# WorkOS AuthKit
Official docs:
- https://docs.convex.dev/auth/authkit/
- https://docs.convex.dev/auth/authkit/add-to-app
- https://docs.convex.dev/auth/authkit/auto-provision
Use this when the app already uses WorkOS or the user wants AuthKit specifically.
## Workflow
1. Confirm the user wants WorkOS AuthKit
2. Determine whether they want:
- a Convex-managed WorkOS team
- an existing WorkOS team
3. Ask whether the user wants local-only setup or production-ready setup now
4. Read the official Convex and WorkOS AuthKit guide
5. Create or update `convex.json` for the app's framework and real local port
6. Follow the correct branch of the setup flow based on that choice
7. Configure the required WorkOS environment variables
8. Configure `convex/auth.config.ts` for WorkOS-issued JWTs
9. Wire the client provider and callback flow
10. Verify authenticated requests reach Convex
11. If the user wants production-ready setup, make sure the production WorkOS configuration is covered too
12. Only add `storeUser` or a `users` table if the app needs first-class user rows inside Convex
## What To Do
- Read the official Convex and WorkOS AuthKit guide before writing setup code
- Determine whether the user wants a Convex-managed WorkOS team or an existing WorkOS team
- Treat `convex.json` as a first-class part of the AuthKit setup, not an optional extra
- Follow the current setup flow from the docs instead of relying on older examples
## Key Setup Areas
- package installation for the app's framework
- `convex.json` with the `authKit` section for dev, and preview or prod if needed
- environment variables such as `WORKOS_CLIENT_ID`, `WORKOS_API_KEY`, and redirect configuration
- `convex/auth.config.ts` wiring for WorkOS-issued JWTs
- client provider setup and token flow into Convex
- login callback and redirect configuration
## Files and Env Vars To Expect
- `convex.json`
- `convex/auth.config.ts`
- frontend auth provider wiring
- callback or redirect route setup where the framework requires it
- WorkOS environment variables commonly include:
- `WORKOS_CLIENT_ID`
- `WORKOS_API_KEY`
- `WORKOS_COOKIE_PASSWORD`
- `VITE_WORKOS_CLIENT_ID`
- `VITE_WORKOS_REDIRECT_URI`
- `NEXT_PUBLIC_WORKOS_REDIRECT_URI`
For a managed WorkOS team, `convex dev` can provision the AuthKit environment and write local env vars such as `VITE_WORKOS_CLIENT_ID` and `VITE_WORKOS_REDIRECT_URI` into `.env.local` for Vite apps.
## Concrete Steps
1. Choose Convex-managed or existing WorkOS team
2. Create or update `convex.json` with the `authKit` section for the framework in use
3. Make sure the dev `redirectUris`, `appHomepageUrl`, `corsOrigins`, and local redirect env vars match the app's actual local port
4. For a managed WorkOS team, run `npx convex dev` and follow the interactive onboarding flow
5. For an existing WorkOS team, get `WORKOS_CLIENT_ID` and `WORKOS_API_KEY` from the WorkOS dashboard and set them with `npx convex env set`
6. Create or update `convex/auth.config.ts` for WorkOS JWT validation
7. Run the normal Convex dev or deploy flow so backend config is synced
8. Wire the WorkOS client provider in the app
9. Configure callback and redirect handling
10. Verify the user can sign in and return to the app
11. Verify Convex sees the authenticated user after login
12. If the user wants production-ready setup, configure the production client ID, API key, redirect URI, and deployment settings too
## Gotchas
- The docs split setup between Convex-managed and existing WorkOS teams, so ask which path the user wants if it is not obvious
- Keep dev and prod WorkOS configuration separate where the docs call for different client IDs or API keys
- Only add `storeUser` or a `users` table if the app needs first-class user rows inside Convex
- Do not mix dev and prod WorkOS credentials or redirect URIs
- If the repo already contains WorkOS setup, preserve the current tenant model unless the user wants to change it
- For managed WorkOS setup, `convex dev` is interactive the first time. In non-interactive terminals, stop and ask the user to complete the onboarding prompts.
- `convex.json` is not optional for the managed AuthKit flow. It drives redirect URI, homepage URL, CORS configuration, and local env var generation.
- If the frontend starts on a different port than the one in `convex.json`, the hosted WorkOS sign-in flow will point to the wrong callback URL. Update `convex.json`, update the local redirect env var, and run `npx convex dev` again.
- Vite can fall off `5173` if other apps are already running. Do not assume the default port still matches the generated AuthKit config.
- A successful WorkOS sign-in should redirect back to the local callback route and then reach a Convex-authenticated state. Do not stop at "the hosted WorkOS page loaded."
## Production
- Ask whether the user wants dev-only setup or production-ready setup
- If the answer is production-ready, make sure the production WorkOS client ID, API key, redirect URI, and Convex deployment config are all covered
- Verify the production redirect and callback settings before calling the task complete
- Do not silently write a notes file into the repo by default. If the user wants rollout or handoff docs, create one explicitly.
## Validation
- Verify the user can complete the login flow and return to the app
- Verify the callback URL matches the real frontend port in local dev
- Verify Convex receives authenticated requests after login
- Verify `convex.json` matches the framework and chosen WorkOS setup path
- Verify `convex/auth.config.ts` matches the chosen WorkOS setup path
- Verify environment variables differ correctly between local and production where needed
- If production-ready setup was requested, verify the production WorkOS configuration is also covered
## Checklist
- [ ] Confirm the user wants WorkOS AuthKit
- [ ] Ask whether the user wants local-only setup or production-ready setup
- [ ] Choose Convex-managed or existing WorkOS team
- [ ] Create or update `convex.json`
- [ ] Configure WorkOS environment variables
- [ ] Configure `convex/auth.config.ts`
- [ ] Verify authenticated requests reach Convex after login
- [ ] If requested, configure the production deployment too

View File

@ -0,0 +1,47 @@
---
name: convex
description: Routes general Convex requests to the right project skill. Use when the user asks which Convex skill to use or gives an underspecified Convex app task.
---
# Convex
Use this as the routing skill for Convex work in this repo.
If a more specific Convex skill clearly matches the request, use that instead.
## Start Here
If the project does not already have Convex AI guidance installed, or the existing guidance looks stale, strongly recommend installing it first.
Preferred:
```bash
npx convex ai-files install
```
This installs or refreshes the managed Convex AI files. It is the recommended starting point for getting the official Convex guidelines in place and following the current Convex AI setup described in the docs:
- [Convex AI docs](https://docs.convex.dev/ai)
Simple fallback:
- [convex_rules.txt](https://convex.link/convex_rules.txt)
Prefer `npx convex ai-files install` over copying rules by hand when possible.
## Route to the Right Skill
After that, use the most specific Convex skill for the task:
- New project or adding Convex to an app: `convex-quickstart`
- Authentication setup: `convex-setup-auth`
- Building a reusable Convex component: `convex-create-component`
- Planning or running a migration: `convex-migration-helper`
- Investigating performance issues: `convex-performance-audit`
If one of those clearly matches the user's goal, switch to it instead of staying in this skill.
## When Not to Use
- The user has already named a more specific Convex workflow
- Another Convex skill obviously fits the request better

View File

@ -0,0 +1,21 @@
# THIS IS AUTOGENERATED. DO NOT EDIT MANUALLY
version = 1
name = "ClawHub"
[setup]
script = "bun install"
[[actions]]
name = "Run"
icon = "run"
command = "bun run dev:worktree"
[[actions]]
name = "Convex Dev"
icon = "tool"
command = "bun run setup:worktree -- --quiet && bunx convex dev --typecheck=disable"
[[actions]]
name = "Seed Dev DB"
icon = "tool"
command = "bun run seed:dev"

15
.editorconfig Normal file
View File

@ -0,0 +1,15 @@
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
[*.md]
trim_trailing_whitespace = false
[Makefile]
indent_style = tab

12
.gitattributes vendored Normal file
View File

@ -0,0 +1,12 @@
* text=auto eol=lf
*.avif binary
*.gif binary
*.ico binary
*.jpg binary
*.jpeg binary
*.png binary
*.webp binary
*.woff binary
*.woff2 binary

118
.github/CODEOWNERS vendored Normal file
View File

@ -0,0 +1,118 @@
# Protect the ownership rules themselves.
/.github/CODEOWNERS @openclaw/openclaw-secops
# WARNING: GitHub CODEOWNERS uses last-match-wins semantics.
# If you add overlapping rules below the secops block, include @openclaw/openclaw-secops
# on those entries too or you can silently remove required secops review.
# Security-sensitive code, config, workflows, and docs require secops review.
/.github/actions/ @openclaw/openclaw-secops
/.github/actionlint.yaml @openclaw/openclaw-secops
/.github/codeql/ @openclaw/openclaw-secops
/.github/dependabot.yml @openclaw/openclaw-secops
/.github/workflows/ @openclaw/openclaw-secops
/scripts/check-staged-secrets.mjs @openclaw/openclaw-secops
/scripts/clawhub-cli-npm-publish.sh @openclaw/openclaw-secops
/scripts/clawhub-cli-npm-release-check.mjs @openclaw/openclaw-secops
/scripts/github/clawhub-rescan-auto-response.mjs @openclaw/openclaw-secops
# Backend auth, API, publish, upload, moderation, and scan enforcement.
/convex/schema.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/auth.config.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/auth.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/commentModeration.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/http.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/httpApi.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/httpApiV1/ @openclaw/openclaw-secops @Patrick-Erichsen
/convex/packagePublishTokens.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/packages.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/publishers.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/rateLimits.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/rescanRequests.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/skills.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/skillTransfers.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/tokens.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/uploads.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/vt.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/webhooks.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/access.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/apiTokenAuth.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/commentScamPrompt.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/githubActionsOidc.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/httpHeaders.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/httpRateLimit.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/manualOverrides.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/moderation.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/moderationEngine.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/moderationReasonCodes.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/packageRegistry.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/packageSecurity.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/publishers.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/publishLimits.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/reporting.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/securityPrompt.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/skillCapabilityTags.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/skillPublish.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/skillSafety.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/staticPublishScan.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/tokens.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/lib/webhooks.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/model/packages/rescans.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/model/rescans/policy.ts @openclaw/openclaw-secops @Patrick-Erichsen
/convex/model/skills/rescans.ts @openclaw/openclaw-secops @Patrick-Erichsen
# Frontend auth, admin, publish, upload, and security-review surfaces.
/src/lib/packageApi.ts @openclaw/openclaw-secops @BunsDev
/src/lib/packageUpload.ts @openclaw/openclaw-secops @BunsDev
/src/lib/roles.ts @openclaw/openclaw-secops @BunsDev
/src/lib/uploadFiles.ts @openclaw/openclaw-secops @BunsDev
/src/lib/uploadUtils.ts @openclaw/openclaw-secops @BunsDev
/src/routes/admin.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/cli/auth.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/packages/new.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/plugins/publish.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/publish-plugin.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/publish-skill.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/skills/publish.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/upload.tsx @openclaw/openclaw-secops @BunsDev
/src/routes/upload/ @openclaw/openclaw-secops @BunsDev
/src/routes/$owner/$slug/security/ @openclaw/openclaw-secops @BunsDev
/src/routes/plugins/$name/security/ @openclaw/openclaw-secops @BunsDev
# CLI auth, admin, publishing, ownership, and package-contract surfaces.
/packages/clawhub/src/browserAuth.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/http.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/adminHelp.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/authToken.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/clawdbotConfig.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/auth.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/delete.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/github.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/moderation.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/ownership.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/packages.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/publish.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/rescan.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/transfer.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/commands/sync.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/cli/scanSkills.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/schema/openclawContract.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/schema/packages.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/schema/routes.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/schema/schemas.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/clawhub/src/schema/textFiles.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/schema/src/openclawContract.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/schema/src/packages.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/schema/src/routes.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/schema/src/schemas.ts @openclaw/openclaw-secops @Patrick-Erichsen
/packages/schema/src/textFiles.ts @openclaw/openclaw-secops @Patrick-Erichsen
# Security, auth, API, webhook, and deployment documentation.
/docs/acceptable-usage.md @openclaw/openclaw-secops @Patrick-Erichsen
/docs/api.md @openclaw/openclaw-secops @Patrick-Erichsen
/docs/auth.md @openclaw/openclaw-secops @Patrick-Erichsen
/docs/deploy.md @openclaw/openclaw-secops @Patrick-Erichsen
/docs/http-api.md @openclaw/openclaw-secops @Patrick-Erichsen
/docs/security.md @openclaw/openclaw-secops @Patrick-Erichsen
/docs/webhook.md @openclaw/openclaw-secops @Patrick-Erichsen
/specs/github-import.md @openclaw/openclaw-secops @Patrick-Erichsen
/public/api/v1/openapi.json @openclaw/openclaw-secops @Patrick-Erichsen

104
.github/ISSUE_TEMPLATE/rfc.yml vendored Normal file
View File

@ -0,0 +1,104 @@
name: RFC
description: Propose a ClawHub policy, product, trust, or interface decision for feedback.
title: "RFC: "
labels:
- "type: rfc"
- "status: review"
body:
- type: markdown
attributes:
value: |
Use RFCs for decisions that need visible feedback before they become policy, product behavior, or public API contract. Accepted repo RFC files live under `rfcs/`, not `docs/`, so draft/decision records do not publish to the docs site. Keep sensitive enforcement details, private reports, exploit specifics, and scanner thresholds out of the public issue.
- type: dropdown
id: area
attributes:
label: Area
description: Pick the primary area this RFC affects.
options:
- Moderation / policy
- Security / trust
- Product / UX
- API / CLI
- Documentation
- Other
validations:
required: true
- type: textarea
id: context
attributes:
label: Context
description: What problem, decision, or ambiguity does this RFC address?
placeholder: |
ClawHub needs a clearer policy for...
validations:
required: true
- type: textarea
id: goals
attributes:
label: Goals
description: What should this RFC achieve?
placeholder: |
- Make enforcement expectations understandable to users.
- Give moderators a consistent decision boundary.
validations:
required: true
- type: textarea
id: non_goals
attributes:
label: Non-goals
description: What is intentionally out of scope?
placeholder: |
- This RFC does not expose internal scanner thresholds.
- This RFC does not decide implementation details for every moderation tool.
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposal
description: Describe the proposed policy, behavior, or decision.
placeholder: |
ClawHub should...
validations:
required: true
- type: textarea
id: examples
attributes:
label: Examples
description: Give concrete allowed, not allowed, or edge-case examples.
placeholder: |
Allowed:
- Defensive security review with explicit scope and evidence.
Not allowed:
- Account takeover, evasion, or non-consensual surveillance workflows.
Edge cases:
- ...
- type: textarea
id: user_impact
attributes:
label: User impact
description: How does this affect authors, users, moderators, API consumers, or external contributors?
placeholder: |
Authors will...
Users will...
Moderators will...
- type: textarea
id: open_questions
attributes:
label: Open questions
description: What feedback would be most useful before a decision?
placeholder: |
- Should appeals be handled in-product, through GitHub, or both?
- What examples would make this clearer?
validations:
required: true
- type: input
id: feedback_deadline
attributes:
label: Feedback deadline
description: Use an absolute date. Normal RFCs should stay open for 7-14 days unless urgent.
placeholder: "YYYY-MM-DD"
validations:
required: true

10
.github/actionlint.yaml vendored Normal file
View File

@ -0,0 +1,10 @@
# actionlint configuration
# https://github.com/rhysd/actionlint/blob/main/docs/config.md
self-hosted-runner:
labels:
# Blacksmith CI runners
- blacksmith-4vcpu-ubuntu-2404
- blacksmith-8vcpu-ubuntu-2404
- blacksmith-16vcpu-ubuntu-2404
- blacksmith-32vcpu-ubuntu-2404

13
.github/actions/setup-bun/action.yml vendored Normal file
View File

@ -0,0 +1,13 @@
name: Setup Bun
description: Install the pinned Bun runtime and workspace dependencies.
runs:
using: composite
steps:
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: 1.3.10
- name: Install dependencies
shell: bash
run: bun install --frozen-lockfile

View File

@ -0,0 +1,16 @@
name: clawhub-codeql-actions-security
disable-default-queries: true
queries:
- uses: security-extended
query-filters:
- include:
precision:
- high
- very-high
tags contain: security
paths:
- .github/workflows

View File

@ -0,0 +1,76 @@
name: clawhub-codeql-backend-api-security
disable-default-queries: true
queries:
- uses: security-extended
query-filters:
- include:
precision:
- high
- very-high
tags contain: security
security-severity: /([7-9]|10)\.(\d)+/
paths:
- convex/auth.config.ts
- convex/auth.ts
- convex/commentModeration.ts
- convex/http.ts
- convex/httpApi.ts
- convex/httpApiV1
- convex/packagePublishTokens.ts
- convex/packages.ts
- convex/publishers.ts
- convex/rateLimits.ts
- convex/rescanRequests.ts
- convex/skills.ts
- convex/skillTransfers.ts
- convex/tokens.ts
- convex/uploads.ts
- convex/vt.ts
- convex/webhooks.ts
- convex/lib/access.ts
- convex/lib/apiTokenAuth.ts
- convex/lib/commentScamPrompt.ts
- convex/lib/githubActionsOidc.ts
- convex/lib/httpHeaders.ts
- convex/lib/httpRateLimit.ts
- convex/lib/httpUtils.ts
- convex/lib/manualOverrides.ts
- convex/lib/moderation.ts
- convex/lib/moderationEngine.ts
- convex/lib/moderationReasonCodes.ts
- convex/lib/packageRegistry.ts
- convex/lib/packageSecurity.ts
- convex/lib/publishers.ts
- convex/lib/publishLimits.ts
- convex/lib/reporting.ts
- convex/lib/securityPrompt.ts
- convex/lib/skillPublish.ts
- convex/lib/skillSafety.ts
- convex/lib/staticPublishScan.ts
- convex/lib/tokens.ts
- convex/lib/webhooks.ts
- convex/model/packages/rescans.ts
- convex/model/rescans/policy.ts
- convex/model/skills/rescans.ts
paths-ignore:
- "**/node_modules"
- "**/coverage"
- "**/dist"
- "**/dist/**"
- "**/*.generated.ts"
- "**/*.bundle.js"
- "**/*.test.ts"
- "**/*.test.tsx"
- "**/*.e2e.test.ts"
- "**/*.e2e.test.tsx"
- "**/*test-support*"
- "**/*test-helper*"
- "**/*mock*"
- "**/*fixture*"
- "**/*bench*"
- "convex/_generated/**"

View File

@ -0,0 +1,59 @@
name: clawhub-codeql-cli-package-security
disable-default-queries: true
queries:
- uses: security-extended
query-filters:
- include:
precision:
- high
- very-high
tags contain: security
security-severity: /([7-9]|10)\.(\d)+/
paths:
- packages/clawhub/src/browserAuth.ts
- packages/clawhub/src/http.ts
- packages/clawhub/src/cli/adminHelp.ts
- packages/clawhub/src/cli/authToken.ts
- packages/clawhub/src/cli/clawdbotConfig.ts
- packages/clawhub/src/cli/commands/auth.ts
- packages/clawhub/src/cli/commands/delete.ts
- packages/clawhub/src/cli/commands/github.ts
- packages/clawhub/src/cli/commands/moderation.ts
- packages/clawhub/src/cli/commands/ownership.ts
- packages/clawhub/src/cli/commands/packages.ts
- packages/clawhub/src/cli/commands/publish.ts
- packages/clawhub/src/cli/commands/rescan.ts
- packages/clawhub/src/cli/commands/sync.ts
- packages/clawhub/src/cli/commands/transfer.ts
- packages/clawhub/src/cli/scanSkills.ts
- packages/clawhub/src/schema/openclawContract.ts
- packages/clawhub/src/schema/packages.ts
- packages/clawhub/src/schema/routes.ts
- packages/clawhub/src/schema/schemas.ts
- packages/clawhub/src/schema/textFiles.ts
- packages/schema/src/openclawContract.ts
- packages/schema/src/packages.ts
- packages/schema/src/routes.ts
- packages/schema/src/schemas.ts
- packages/schema/src/textFiles.ts
paths-ignore:
- "**/node_modules"
- "**/coverage"
- "**/dist"
- "**/dist/**"
- "**/*.generated.ts"
- "**/*.bundle.js"
- "**/*.test.ts"
- "**/*.test.tsx"
- "**/*.e2e.test.ts"
- "**/*.e2e.test.tsx"
- "**/*test-support*"
- "**/*test-helper*"
- "**/*mock*"
- "**/*fixture*"
- "**/*bench*"

View File

@ -0,0 +1,59 @@
name: clawhub-codeql-frontend-publish-security
disable-default-queries: true
queries:
- uses: security-extended
query-filters:
- include:
precision:
- high
- very-high
tags contain: security
security-severity: /([7-9]|10)\.(\d)+/
paths:
- src/components/DetailSecuritySummary.tsx
- src/components/MarkdownPreview.tsx
- src/components/PackageSourceChooser.tsx
- src/components/SecurityScannerPage.tsx
- src/components/SkillSecurityScanResults.tsx
- src/lib/authErrorMessage.ts
- src/lib/packageApi.ts
- src/lib/packageUpload.ts
- src/lib/pluginPublishPrefill.ts
- src/lib/rehypeProxyImages.ts
- src/lib/roles.ts
- src/lib/uploadFiles.ts
- src/lib/uploadUtils.ts
- src/lib/useAuthError.ts
- src/lib/useAuthStatus.ts
- src/routes/admin.tsx
- src/routes/cli/auth.tsx
- src/routes/packages/new.tsx
- src/routes/plugins/publish.tsx
- src/routes/publish-plugin.tsx
- src/routes/publish-skill.tsx
- src/routes/skills/publish.tsx
- src/routes/upload.tsx
- src/routes/upload
- src/routes/$owner/$slug/security
- src/routes/plugins/$name/security
paths-ignore:
- "**/node_modules"
- "**/coverage"
- "**/dist"
- "**/dist/**"
- "**/*.generated.ts"
- "**/*.bundle.js"
- "**/*.test.ts"
- "**/*.test.tsx"
- "**/*.e2e.test.ts"
- "**/*.e2e.test.tsx"
- "**/*test-support*"
- "**/*test-helper*"
- "**/*mock*"
- "**/*fixture*"
- "**/*bench*"

View File

@ -0,0 +1,39 @@
name: clawhub-codeql-repository-automation-security
disable-default-queries: true
queries:
- uses: security-extended
query-filters:
- include:
precision:
- high
- very-high
tags contain: security
security-severity: /([7-9]|10)\.(\d)+/
paths:
- scripts/check-staged-secrets.mjs
- scripts/clawhub-cli-npm-release-check.mjs
- scripts/github
- scripts/verify-convex-contract.ts
- scripts/copy-og-assets.ts
- scripts/check-peer-deps.ts
paths-ignore:
- "**/node_modules"
- "**/coverage"
- "**/dist"
- "**/dist/**"
- "**/*.generated.ts"
- "**/*.bundle.js"
- "**/*.test.ts"
- "**/*.test.tsx"
- "**/*.e2e.test.ts"
- "**/*.e2e.test.tsx"
- "**/*test-support*"
- "**/*test-helper*"
- "**/*mock*"
- "**/*fixture*"
- "**/*bench*"

41
.github/dependabot.yml vendored Normal file
View File

@ -0,0 +1,41 @@
version: 2
updates:
- package-ecosystem: "bun"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
time: "09:00"
timezone: "America/Los_Angeles"
open-pull-requests-limit: 10
ignore:
- dependency-name: "@auth/core"
update-types:
- "version-update:semver-minor"
- "version-update:semver-major"
- dependency-name: "undici"
update-types:
- "version-update:semver-major"
groups:
production-minor-and-patch:
dependency-type: "production"
update-types:
- "minor"
- "patch"
development-minor-and-patch:
dependency-type: "development"
update-types:
- "minor"
- "patch"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
day: "monday"
time: "09:00"
timezone: "America/Los_Angeles"
groups:
github-actions:
patterns:
- "*"

32
.github/pull_request_template.md vendored Normal file
View File

@ -0,0 +1,32 @@
## Summary
- What changed:
- Why:
## Linked Issue
- Closes #
- Related #
## Screenshots
For website/UI changes, attach screenshots or recordings from the real app. Include mobile/narrow views when layout changes.
- [ ] Screenshots/recordings attached, or `N/A`
## Security / Trust Impact
- [ ] No security/trust impact
- [ ] Security/trust impact explained
## Data / Deploy Impact
- [ ] No data/deploy impact
- [ ] Data/deploy impact explained
## Verification
- [ ] `bun run format:check`
- [ ] `bun run lint`
- [ ] `bun run test`
- [ ] Other:

59
.github/workflows/auto-response.yml vendored Normal file
View File

@ -0,0 +1,59 @@
name: Auto response
on:
issues:
types: [opened, edited, labeled]
issue_comment:
types: [created]
pull_request_target: # zizmor: ignore[dangerous-triggers] trusted base checkout only; no untrusted PR code execution
types: [opened, edited, synchronize, reopened, labeled]
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.event.issue.number || github.ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request_target' }}
permissions: {}
jobs:
auto-response:
permissions:
contents: read
issues: write
pull-requests: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.sha }}
persist-credentials: false
- uses: actions/create-github-app-token@v3
id: app-token
continue-on-error: true
with:
app-id: "2729701"
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- uses: actions/create-github-app-token@v3
id: app-token-fallback
continue-on-error: true
if: steps.app-token.outcome == 'failure'
with:
app-id: "2971289"
private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }}
- name: Run Barnacle auto-response
uses: actions/github-script@v9
with:
github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token || github.token }}
script: |
const { pathToFileURL } = require("node:url");
const moduleUrl = pathToFileURL(
`${process.env.GITHUB_WORKSPACE}/scripts/github/barnacle-auto-response.mjs`,
);
const { runBarnacleAutoResponse } = await import(moduleUrl.href);
await runBarnacleAutoResponse({ github, context, core });

80
.github/workflows/ci-check-testbox.yml vendored Normal file
View File

@ -0,0 +1,80 @@
name: Blacksmith Testbox
on:
workflow_dispatch:
inputs:
testbox_id:
type: string
description: "Testbox session ID"
required: true
permissions:
contents: read
env:
BUN_VERSION: "1.3.10"
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
jobs:
check:
name: "check"
runs-on: blacksmith-8vcpu-ubuntu-2404
timeout-minutes: 30
steps:
- name: Begin Testbox
uses: useblacksmith/begin-testbox@d0e04585c26905fdd92c94a09c159544c7ee1b67
with:
testbox_id: ${{ inputs.testbox_id }}
- uses: actions/checkout@v6
with:
fetch-depth: 50
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Restore Bun install cache
id: bun-cache
uses: actions/cache/restore@v5
with:
path: ~/.bun/install/cache
key: ${{ runner.os }}-bun-${{ env.BUN_VERSION }}-${{ hashFiles('bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-${{ env.BUN_VERSION }}-
- name: Install
run: bun install --frozen-lockfile
- name: Save Bun install cache
if: steps.bun-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v5
continue-on-error: true
with:
path: ~/.bun/install/cache
key: ${{ steps.bun-cache.outputs.cache-primary-key }}
- name: Prepare Testbox shell
shell: bash
run: |
set -euo pipefail
git fetch --no-tags --depth=50 origin "+refs/heads/main:refs/remotes/origin/main"
bun_bin="$(command -v bun)"
sudo ln -sf "$bun_bin" /usr/local/bin/bun
if command -v bunx >/dev/null 2>&1; then
sudo ln -sf "$(command -v bunx)" /usr/local/bin/bunx
fi
node_bin="$(dirname "$(node -p 'process.execPath')")"
sudo ln -sf "$node_bin/node" /usr/local/bin/node
sudo ln -sf "$node_bin/npm" /usr/local/bin/npm
sudo ln -sf "$node_bin/npx" /usr/local/bin/npx
- name: Run Testbox
uses: useblacksmith/run-testbox@5ca05834db1d3813554d1dd109e5f2087a8d7cbc
if: always()
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"

View File

@ -4,37 +4,106 @@ on:
push:
branches: [main]
pull_request:
workflow_dispatch:
concurrency:
group: ci-${{ github.event_name == 'pull_request' && github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
permissions:
contents: read
env:
VITE_CONVEX_URL: https://example.invalid
jobs:
build:
static:
name: static
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
- uses: actions/checkout@v6
with:
bun-version: 1.3.6
fetch-depth: 0
- name: Install
run: bun install --frozen-lockfile
- name: Peer deps
run: bun run check:peers
- uses: ./.github/actions/setup-bun
- name: Lint
run: bun run lint
- name: Static checks
run: bun run ci:static
- name: Test
run: bun run test
unit:
name: unit
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/setup-bun
- name: Coverage
run: bun run coverage
run: bun run ci:unit
- name: Typecheck packages
run: |
bunx tsc -p packages/schema/tsconfig.json --noEmit
bunx tsc -p packages/clawdhub/tsconfig.json --noEmit
packages:
name: packages
runs-on: ubuntu-latest
timeout-minutes: 15
- name: Build
run: bun run build
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/setup-bun
- name: Package checks
run: bun run ci:packages
types-build:
name: types-build
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/setup-bun
- name: Typecheck and build
run: bun run ci:types-build
e2e-http:
name: e2e-http
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/setup-bun
- name: HTTP e2e
run: bun run ci:e2e-http
playwright-smoke:
name: playwright-smoke
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/setup-bun
- name: Install Playwright browsers
run: bunx playwright install --with-deps chromium
- name: Browser e2e
run: bun run ci:playwright-smoke
- name: Upload Playwright report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: playwright-report
path: playwright-report/
if-no-files-found: ignore

View File

@ -0,0 +1,317 @@
name: ClawHub CLI NPM Release
on:
workflow_dispatch:
inputs:
tag:
description: Release tag to publish, for example v0.10.0
required: true
type: string
preflight_only:
description: Run validation/build only and skip the gated publish job
required: true
default: false
type: boolean
preflight_run_id:
description: Existing successful preflight workflow run id to promote without rebuilding
required: false
type: string
concurrency:
group: clawhub-cli-npm-release-${{ inputs.tag }}
cancel-in-progress: false
permissions: {}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
NODE_VERSION: "24.x"
BUN_VERSION: "1.3.10"
jobs:
preflight_clawhub_cli_npm:
if: ${{ inputs.preflight_only }}
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Forbid preflight artifact promotion on validation-only runs
if: ${{ inputs.preflight_run_id != '' }}
run: |
echo "preflight_run_id is only valid for real publish runs."
exit 1
- name: Checkout
uses: actions/checkout@v6
with:
ref: refs/tags/${{ inputs.tag }}
fetch-depth: 0
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
registry-url: https://registry.npmjs.org
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Resolve CLI package directory
run: |
set -euo pipefail
if [[ -d "packages/clawhub" ]]; then
echo "PACKAGE_DIR=packages/clawhub" >> "$GITHUB_ENV"
elif [[ -d "packages/clawdhub" ]]; then
echo "PACKAGE_DIR=packages/clawdhub" >> "$GITHUB_ENV"
else
echo "Unable to find clawhub CLI package directory." >&2
exit 1
fi
- name: Ensure version is not already published
env:
PREFLIGHT_ONLY: ${{ inputs.preflight_only }}
run: |
set -euo pipefail
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
import { readFileSync } from "node:fs";
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
process.stdout.write(String(pkg.version ?? "").trim());
EOF
)"
if npm view "clawhub@${PACKAGE_VERSION}" version >/dev/null 2>&1; then
if [[ "${PREFLIGHT_ONLY}" == "true" ]]; then
echo "clawhub@${PACKAGE_VERSION} is already published on npm; continuing because preflight_only=true."
exit 0
fi
echo "clawhub@${PACKAGE_VERSION} is already published on npm."
exit 1
fi
echo "Publishing clawhub@${PACKAGE_VERSION}"
- name: Validate release tag and package metadata
env:
RELEASE_TAG: ${{ inputs.tag }}
RELEASE_MAIN_REF: origin/main
run: |
set -euo pipefail
RELEASE_SHA="$(git rev-parse HEAD)"
export RELEASE_SHA
git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
node scripts/clawhub-cli-npm-release-check.mjs
- name: Verify CLI package
run: bun run --cwd "$PACKAGE_DIR" verify
- name: Pack prepared npm tarball
id: packed_tarball
env:
RELEASE_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
pushd "$PACKAGE_DIR" >/dev/null
PACK_JSON="$(npm pack --json --ignore-scripts)"
echo "$PACK_JSON"
PACK_PATH="$(printf '%s\n' "$PACK_JSON" | node --input-type=module -e 'const chunks=[]; process.stdin.on("data", (chunk) => chunks.push(chunk)); process.stdin.on("end", () => { const parsed = JSON.parse(Buffer.concat(chunks).toString("utf8")); const first = Array.isArray(parsed) ? parsed[0] : null; if (!first || typeof first.filename !== "string" || !first.filename) process.exit(1); process.stdout.write(first.filename); });')"
popd >/dev/null
if [[ -z "${PACK_PATH}" || ! -f "${PACKAGE_DIR}/${PACK_PATH}" ]]; then
echo "npm pack did not produce a tarball file." >&2
exit 1
fi
RELEASE_SHA="$(git rev-parse HEAD)"
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
import { readFileSync } from "node:fs";
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
process.stdout.write(String(pkg.version ?? "").trim());
EOF
)"
ARTIFACT_DIR="$RUNNER_TEMP/clawhub-cli-npm-preflight"
rm -rf "$ARTIFACT_DIR"
mkdir -p "$ARTIFACT_DIR"
cp "${PACKAGE_DIR}/${PACK_PATH}" "$ARTIFACT_DIR/"
printf '%s\n' "$RELEASE_TAG" > "$ARTIFACT_DIR/release-tag.txt"
printf '%s\n' "$RELEASE_SHA" > "$ARTIFACT_DIR/release-sha.txt"
printf '%s\n' "$PACKAGE_VERSION" > "$ARTIFACT_DIR/package-version.txt"
echo "dir=$ARTIFACT_DIR" >> "$GITHUB_OUTPUT"
- name: Upload prepared npm publish bundle
uses: actions/upload-artifact@v7
with:
name: clawhub-cli-npm-preflight-${{ inputs.tag }}
path: ${{ steps.packed_tarball.outputs.dir }}
if-no-files-found: error
validate_publish_request:
if: ${{ !inputs.preflight_only }}
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Require main workflow ref for publish
env:
WORKFLOW_REF: ${{ github.ref }}
run: |
set -euo pipefail
if [[ "${WORKFLOW_REF}" != "refs/heads/main" ]]; then
echo "Real publish runs must be dispatched from main. Use preflight_only=true for branch validation."
exit 1
fi
- name: Require preflight artifact promotion on real publish
env:
PREFLIGHT_RUN_ID: ${{ inputs.preflight_run_id }}
run: |
set -euo pipefail
if [[ -z "${PREFLIGHT_RUN_ID}" ]]; then
echo "Real publish requires preflight_run_id from a successful npm preflight run." >&2
exit 1
fi
publish_clawhub_cli_npm:
needs: [validate_publish_request]
if: ${{ !inputs.preflight_only }}
runs-on: ubuntu-latest
environment: npm-release
permissions:
actions: read
contents: read
id-token: write
steps:
- name: Checkout
uses: actions/checkout@v6
with:
ref: refs/tags/${{ inputs.tag }}
fetch-depth: 0
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
registry-url: https://registry.npmjs.org
- name: Resolve CLI package directory
run: |
set -euo pipefail
if [[ -d "packages/clawhub" ]]; then
echo "PACKAGE_DIR=packages/clawhub" >> "$GITHUB_ENV"
elif [[ -d "packages/clawdhub" ]]; then
echo "PACKAGE_DIR=packages/clawdhub" >> "$GITHUB_ENV"
else
echo "Unable to find clawhub CLI package directory." >&2
exit 1
fi
- name: Ensure version is not already published
run: |
set -euo pipefail
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
import { readFileSync } from "node:fs";
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
process.stdout.write(String(pkg.version ?? "").trim());
EOF
)"
if npm view "clawhub@${PACKAGE_VERSION}" version >/dev/null 2>&1; then
echo "clawhub@${PACKAGE_VERSION} is already published on npm."
exit 1
fi
echo "Publishing clawhub@${PACKAGE_VERSION}"
- name: Verify preflight run metadata
env:
GH_TOKEN: ${{ github.token }}
PREFLIGHT_RUN_ID: ${{ inputs.preflight_run_id }}
run: |
set -euo pipefail
RUN_JSON="$(gh run view "$PREFLIGHT_RUN_ID" --repo "$GITHUB_REPOSITORY" --json workflowName,headBranch,event,conclusion,url)"
# shellcheck disable=SC2016
printf '%s' "$RUN_JSON" | node --input-type=module -e 'const chunks=[]; process.stdin.on("data", (chunk) => chunks.push(chunk)); process.stdin.on("end", () => { const run = JSON.parse(Buffer.concat(chunks).toString("utf8")); const checks = [["workflowName", "ClawHub CLI NPM Release"], ["headBranch", "main"], ["event", "workflow_dispatch"], ["conclusion", "success"]]; for (const [key, expected] of checks) { if (run[key] !== expected) { console.error(`Referenced npm preflight run ${process.env.PREFLIGHT_RUN_ID} must have ${key}=${expected}, got ${run[key] ?? "<missing>"}.`); process.exit(1); } } console.log(`Using npm preflight run ${process.env.PREFLIGHT_RUN_ID}: ${run.url}`); });'
- name: Download prepared npm tarball
uses: actions/download-artifact@v8
with:
name: clawhub-cli-npm-preflight-${{ inputs.tag }}
path: preflight-tarball
repository: ${{ github.repository }}
run-id: ${{ inputs.preflight_run_id }}
github-token: ${{ github.token }}
- name: Validate release tag and package metadata
env:
RELEASE_TAG: ${{ inputs.tag }}
RELEASE_MAIN_REF: origin/main
run: |
set -euo pipefail
RELEASE_SHA="$(git rev-parse HEAD)"
export RELEASE_SHA
git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
node scripts/clawhub-cli-npm-release-check.mjs
- name: Verify prepared tarball provenance
env:
RELEASE_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
EXPECTED_RELEASE_SHA="$(git rev-parse HEAD)"
EXPECTED_PACKAGE_VERSION="$(node --input-type=module <<'EOF'
import { readFileSync } from "node:fs";
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
process.stdout.write(String(pkg.version ?? "").trim());
EOF
)"
TAG_FILE="preflight-tarball/release-tag.txt"
SHA_FILE="preflight-tarball/release-sha.txt"
VERSION_FILE="preflight-tarball/package-version.txt"
if [[ ! -f "$TAG_FILE" || ! -f "$SHA_FILE" || ! -f "$VERSION_FILE" ]]; then
echo "Prepared preflight metadata is missing." >&2
ls -la preflight-tarball >&2 || true
exit 1
fi
ARTIFACT_RELEASE_TAG="$(tr -d '\r\n' < "$TAG_FILE")"
ARTIFACT_RELEASE_SHA="$(tr -d '\r\n' < "$SHA_FILE")"
ARTIFACT_PACKAGE_VERSION="$(tr -d '\r\n' < "$VERSION_FILE")"
if [[ "$ARTIFACT_RELEASE_TAG" != "$RELEASE_TAG" ]]; then
echo "Prepared preflight tag mismatch: expected $RELEASE_TAG, got $ARTIFACT_RELEASE_TAG" >&2
exit 1
fi
if [[ "$ARTIFACT_RELEASE_SHA" != "$EXPECTED_RELEASE_SHA" ]]; then
echo "Prepared preflight SHA mismatch: expected $EXPECTED_RELEASE_SHA, got $ARTIFACT_RELEASE_SHA" >&2
exit 1
fi
if [[ "$ARTIFACT_PACKAGE_VERSION" != "$EXPECTED_PACKAGE_VERSION" ]]; then
echo "Prepared preflight package version mismatch: expected $EXPECTED_PACKAGE_VERSION, got $ARTIFACT_PACKAGE_VERSION" >&2
exit 1
fi
- name: Resolve publish tarball
id: publish_tarball
run: |
set -euo pipefail
TARBALL_PATH="$(find preflight-tarball -type f -name '*.tgz' -print | sort | tail -n 1)"
if [[ -z "$TARBALL_PATH" ]]; then
echo "Prepared preflight tarball not found." >&2
ls -la preflight-tarball >&2 || true
exit 1
fi
echo "path=$TARBALL_PATH" >> "$GITHUB_OUTPUT"
- name: Publish
run: |
set -euo pipefail
publish_target="${{ steps.publish_tarball.outputs.path }}"
if [[ -n "${publish_target}" ]]; then
publish_target="./${publish_target}"
fi
bash scripts/clawhub-cli-npm-publish.sh --publish "${publish_target}"

View File

@ -0,0 +1,168 @@
name: ClawHub Moderator CLI Release
on:
workflow_dispatch:
inputs:
tag:
description: Moderator release tag, for example clawhub-mod-v0.1.0
required: true
type: string
concurrency:
group: clawhub-mod-release-${{ inputs.tag }}
cancel-in-progress: false
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
NODE_VERSION: "24.x"
BUN_VERSION: "1.3.10"
PACKAGE_DIR: packages/clawhub-mod
jobs:
release_clawhub_mod_cli:
runs-on: ubuntu-latest
environment: moderator-release
permissions:
contents: write
steps:
- name: Require main workflow ref
env:
WORKFLOW_REF: ${{ github.ref }}
run: |
set -euo pipefail
if [[ "${WORKFLOW_REF}" != "refs/heads/main" ]]; then
echo "Moderator release runs must be dispatched from main."
exit 1
fi
- name: Checkout release tag
uses: actions/checkout@v6
with:
ref: refs/tags/${{ inputs.tag }}
fetch-depth: 0
- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Validate tag and package metadata
env:
RELEASE_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
RELEASE_SHA="$(git rev-parse HEAD)"
git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
if ! git merge-base --is-ancestor "$RELEASE_SHA" origin/main; then
echo "Moderator release tag must point at a commit reachable from main." >&2
exit 1
fi
node --input-type=module <<'EOF'
import { readFileSync } from "node:fs";
const tag = process.env.RELEASE_TAG;
const pkg = JSON.parse(readFileSync(`${process.env.PACKAGE_DIR}/package.json`, "utf8"));
const version = String(pkg.version ?? "").trim();
const expectedTag = `clawhub-mod-v${version}`;
if (!/^clawhub-mod-v\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?$/.test(tag)) {
throw new Error(`Moderator release tag must look like clawhub-mod-vX.Y.Z, got ${tag}`);
}
if (tag !== expectedTag) {
throw new Error(`Moderator release tag ${tag} must match ${expectedTag} from package.json`);
}
if (pkg.private !== true) {
throw new Error("@openclaw/clawhub-mod must remain private: true");
}
if (pkg.name !== "@openclaw/clawhub-mod") {
throw new Error(`Unexpected package name ${pkg.name}`);
}
console.log(`Preparing ${pkg.name}@${version} from ${tag}`);
EOF
- name: Verify moderator CLI package
run: bun run --cwd "$PACKAGE_DIR" verify
- name: Pack moderator CLI tarball
id: pack
env:
RELEASE_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
ARTIFACT_DIR="$RUNNER_TEMP/clawhub-mod-release"
rm -rf "$ARTIFACT_DIR"
mkdir -p "$ARTIFACT_DIR"
pushd "$PACKAGE_DIR" >/dev/null
PACK_JSON="$(npm pack --json --ignore-scripts)"
echo "$PACK_JSON"
PACK_PATH="$(printf '%s\n' "$PACK_JSON" | node --input-type=module -e 'const chunks=[]; process.stdin.on("data", (chunk) => chunks.push(chunk)); process.stdin.on("end", () => { const parsed = JSON.parse(Buffer.concat(chunks).toString("utf8")); const first = Array.isArray(parsed) ? parsed[0] : null; if (!first || typeof first.filename !== "string" || !first.filename) process.exit(1); process.stdout.write(first.filename); });')"
popd >/dev/null
if [[ -z "${PACK_PATH}" || ! -f "${PACKAGE_DIR}/${PACK_PATH}" ]]; then
echo "npm pack did not produce a tarball file." >&2
exit 1
fi
cp "${PACKAGE_DIR}/${PACK_PATH}" "$ARTIFACT_DIR/"
cp scripts/install-clawhub-mod.sh "$ARTIFACT_DIR/"
(cd "$ARTIFACT_DIR" && shasum -a 256 ./* > SHA256SUMS.txt)
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
import { readFileSync } from "node:fs";
const pkg = JSON.parse(readFileSync(`${process.env.PACKAGE_DIR}/package.json`, "utf8"));
process.stdout.write(String(pkg.version ?? "").trim());
EOF
)"
RELEASE_SHA="$(git rev-parse HEAD)"
NOTES_FILE="$ARTIFACT_DIR/release-notes.md"
cat > "$NOTES_FILE" <<EOF
Moderator-only ClawHub operator CLI release.
- Package: @openclaw/clawhub-mod@${PACKAGE_VERSION}
- Commit: ${RELEASE_SHA}
- Install/upgrade:
\`\`\`bash
bash scripts/install-clawhub-mod.sh --tag ${RELEASE_TAG}
\`\`\`
This release is intentionally distributed as a GitHub Release asset, not npm.
EOF
echo "artifact_dir=$ARTIFACT_DIR" >> "$GITHUB_OUTPUT"
echo "notes_file=$NOTES_FILE" >> "$GITHUB_OUTPUT"
echo "package_version=$PACKAGE_VERSION" >> "$GITHUB_OUTPUT"
- name: Create or update draft GitHub release
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ inputs.tag }}
RELEASE_TITLE: ClawHub Moderator CLI ${{ steps.pack.outputs.package_version }}
ARTIFACT_DIR: ${{ steps.pack.outputs.artifact_dir }}
NOTES_FILE: ${{ steps.pack.outputs.notes_file }}
run: |
set -euo pipefail
if gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
IS_DRAFT="$(gh release view "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" --json isDraft --jq .isDraft)"
if [[ "$IS_DRAFT" != "true" ]]; then
echo "Refusing to mutate non-draft moderator release ${RELEASE_TAG}." >&2
exit 1
fi
gh release upload "$RELEASE_TAG" "$ARTIFACT_DIR"/* --repo "$GITHUB_REPOSITORY" --clobber
else
gh release create "$RELEASE_TAG" "$ARTIFACT_DIR"/* \
--repo "$GITHUB_REPOSITORY" \
--draft \
--verify-tag \
--title "$RELEASE_TITLE" \
--notes-file "$NOTES_FILE"
fi

View File

@ -0,0 +1,57 @@
name: ClawHub Rescan Guidance
on:
issues:
types: [labeled]
workflow_dispatch:
inputs:
issue:
description: "Issue number to check"
required: true
type: string
permissions:
contents: read
issues: write
concurrency:
group: clawhub-rescan-guidance-${{ github.event.issue.number || github.event.inputs.issue }}
cancel-in-progress: false
jobs:
rescan-guidance:
runs-on: ubuntu-latest
if: "${{ github.event_name == 'workflow_dispatch' || github.event.label.name == 'r: rescan-guidance' }}"
env:
CLAWHUB_RESCAN_GUIDANCE_APPLY: "1"
ISSUE_NUMBER: ${{ github.event.issue.number || github.event.inputs.issue }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.sha }}
persist-credentials: false
- uses: actions/create-github-app-token@v3
id: app-token
continue-on-error: true
with:
app-id: "2729701"
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- uses: actions/create-github-app-token@v3
id: app-token-fallback
continue-on-error: true
if: steps.app-token.outcome == 'failure'
with:
app-id: "2971289"
private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }}
- name: Comment when rescan guidance label is present
env:
GH_TOKEN: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token || github.token }}
run: |
node scripts/github/clawhub-rescan-auto-response.mjs \
--repo "$GITHUB_REPOSITORY" \
--issue "$ISSUE_NUMBER" \
--comment-for-labeled-issue \
--apply

View File

@ -0,0 +1,62 @@
name: ClawSweeper Dispatch
on:
issues:
types: [opened, reopened, edited, labeled, unlabeled]
pull_request_target: # zizmor: ignore[dangerous-triggers] maintainer-owned external dispatch; no checkout or untrusted PR code execution
types: [opened, reopened, synchronize, ready_for_review, edited, labeled, unlabeled]
permissions:
contents: read
concurrency:
group: clawsweeper-dispatch-${{ github.repository }}-${{ github.event.issue.number || github.event.pull_request.number || github.run_id }}
cancel-in-progress: ${{ github.event.action == 'edited' || github.event.action == 'synchronize' || github.event.action == 'ready_for_review' }}
jobs:
dispatch:
runs-on: ubuntu-latest
if: ${{ !(endsWith(github.actor, '[bot]') && (github.event.action == 'labeled' || github.event.action == 'unlabeled')) }}
env:
HAS_CLAWSWEEPER_APP_PRIVATE_KEY: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY != '' }}
CLAWSWEEPER_APP_CLIENT_ID: Iv23liOECG0slfuhz093
SUPERSEDES_IN_PROGRESS: ${{ (github.event.action == 'edited' || github.event.action == 'synchronize' || github.event.action == 'ready_for_review') && 'true' || 'false' }}
steps:
- name: Debounce bursty metadata events
if: ${{ github.event.action == 'labeled' || github.event.action == 'unlabeled' }}
run: sleep 20
- name: Create ClawSweeper dispatch token
id: token
if: ${{ env.HAS_CLAWSWEEPER_APP_PRIVATE_KEY == 'true' }}
uses: actions/create-github-app-token@1b10c78c7865c340bc4f6099eb2f838309f1e8c3 # v3.1.1
with:
client-id: ${{ env.CLAWSWEEPER_APP_CLIENT_ID }}
private-key: ${{ secrets.CLAWSWEEPER_APP_PRIVATE_KEY }}
owner: openclaw
repositories: clawsweeper
- name: Dispatch exact ClawSweeper review
env:
GH_TOKEN: ${{ steps.token.outputs.token || secrets.OPENCLAW_GH_TOKEN }}
TARGET_REPO: ${{ github.repository }}
ITEM_NUMBER: ${{ github.event.issue.number || github.event.pull_request.number }}
ITEM_KIND: ${{ github.event_name == 'pull_request_target' && 'pull_request' || 'issue' }}
SOURCE_EVENT: ${{ github.event_name }}
SOURCE_ACTION: ${{ github.event.action }}
run: |
if [ -z "$GH_TOKEN" ]; then
echo "::notice::Skipping ClawSweeper dispatch because no dispatch credential is configured."
exit 0
fi
payload="$(jq -nc \
--arg target_repo "$TARGET_REPO" \
--argjson item_number "$ITEM_NUMBER" \
--arg item_kind "$ITEM_KIND" \
--arg source_event "$SOURCE_EVENT" \
--arg source_action "$SOURCE_ACTION" \
--argjson supersedes_in_progress "$SUPERSEDES_IN_PROGRESS" \
'{event_type:"clawsweeper_item",client_payload:{target_repo:$target_repo,item_number:$item_number,item_kind:$item_kind,source_event:$source_event,source_action:$source_action,supersedes_in_progress:$supersedes_in_progress}}')"
gh api repos/openclaw/clawsweeper/dispatches \
--method POST \
--input - <<< "$payload"

100
.github/workflows/codeql-light.yml vendored Normal file
View File

@ -0,0 +1,100 @@
name: CodeQL Light
on:
workflow_dispatch:
inputs:
profile:
description: CodeQL light profile to run
required: false
default: all
type: choice
options:
- all
- backend-api
- frontend-publish
- cli-package
- repository-automation
- actions
push:
branches: [main]
paths:
- ".github/codeql/**"
- ".github/workflows/**"
- "convex/**"
- "packages/clawhub/**"
- "packages/schema/**"
- "scripts/**"
- "src/**"
- "bun.lock"
- "package.json"
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths:
- ".github/codeql/**"
- ".github/workflows/**"
- "convex/**"
- "packages/clawhub/**"
- "packages/schema/**"
- "scripts/**"
- "src/**"
- "bun.lock"
- "package.json"
schedule:
- cron: "17 7 * * *"
concurrency:
group: codeql-light-${{ github.event_name == 'workflow_dispatch' && github.run_id || github.event_name == 'pull_request' && github.event.pull_request.number || github.sha }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
permissions:
actions: read
contents: read
security-events: write
jobs:
analyze:
name: Analyze (${{ matrix.category }})
if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.draft }}
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
- language: javascript-typescript
category: backend-api
config_file: ./.github/codeql/codeql-backend-api-security.yml
- language: javascript-typescript
category: frontend-publish
config_file: ./.github/codeql/codeql-frontend-publish-security.yml
- language: javascript-typescript
category: cli-package
config_file: ./.github/codeql/codeql-cli-package-security.yml
- language: javascript-typescript
category: repository-automation
config_file: ./.github/codeql/codeql-repository-automation-security.yml
- language: actions
category: actions
config_file: ./.github/codeql/codeql-actions-security.yml
steps:
- name: Checkout
if: ${{ github.event_name != 'workflow_dispatch' || inputs.profile == 'all' || inputs.profile == matrix.category }}
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
submodules: false
- name: Initialize CodeQL
if: ${{ github.event_name != 'workflow_dispatch' || inputs.profile == 'all' || inputs.profile == matrix.category }}
uses: github/codeql-action/init@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4
with:
languages: ${{ matrix.language }}
config-file: ${{ matrix.config_file }}
- name: Analyze
if: ${{ github.event_name != 'workflow_dispatch' || inputs.profile == 'all' || inputs.profile == matrix.category }}
uses: github/codeql-action/analyze@95e58e9a2cdfd71adc6e0353d5c52f41a045d225 # v4
with:
category: "/codeql-light/${{ matrix.category }}"

187
.github/workflows/deploy-staging.yml vendored Normal file
View File

@ -0,0 +1,187 @@
name: Deploy Staging
on:
push:
branches: [main]
workflow_dispatch:
inputs:
reset_seed:
description: "Reset seeded staging fixtures before smoke tests"
required: true
default: false
type: boolean
allow_deleting_large_indexes:
description: "Allow Convex to delete large indexes"
required: true
default: false
type: boolean
concurrency:
group: deploy-staging
cancel-in-progress: true
permissions:
contents: read
jobs:
deploy-staging:
name: deploy-staging
runs-on: ubuntu-latest
timeout-minutes: 45
environment:
name: Staging
url: https://staging.hub.openclaw.ai
env:
CONVEX_DEPLOY_KEY: ${{ secrets.CONVEX_DEPLOY_KEY }}
PLAYWRIGHT_BASE_URL: https://staging.hub.openclaw.ai
RESET_STAGING_SEED: ${{ github.event_name == 'workflow_dispatch' && inputs.reset_seed || false }}
STAGING_CONVEX_SITE_URL: ${{ vars.STAGING_CONVEX_SITE_URL }}
STAGING_CONVEX_URL: ${{ vars.STAGING_CONVEX_URL }}
STAGING_SITE_URL: https://staging.hub.openclaw.ai
VERCEL_CLI_VERSION: 52.0.0
VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }}
VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
ALLOW_DELETING_LARGE_INDEXES: ${{ github.event_name == 'workflow_dispatch' && inputs.allow_deleting_large_indexes || false }}
steps:
- name: Check staging deploy configuration
id: config
run: |
set -euo pipefail
missing=()
for name in \
CONVEX_DEPLOY_KEY \
STAGING_CONVEX_SITE_URL \
STAGING_CONVEX_URL \
VERCEL_ORG_ID \
VERCEL_PROJECT_ID \
VERCEL_TOKEN
do
if [[ -z "${!name}" ]]; then
missing+=("$name")
fi
done
if (( ${#missing[@]} > 0 )); then
echo "configured=false" >> "$GITHUB_OUTPUT"
printf '::notice::Skipping staging deploy; missing Staging environment values: %s\n' "${missing[*]}"
exit 0
fi
echo "configured=true" >> "$GITHUB_OUTPUT"
echo "Deploying staging site: $STAGING_SITE_URL"
echo "Using staging Convex site URL: $STAGING_CONVEX_SITE_URL"
- uses: actions/checkout@v6
if: steps.config.outputs.configured == 'true'
- uses: ./.github/actions/setup-bun
if: steps.config.outputs.configured == 'true'
- name: Stamp Convex staging metadata
if: steps.config.outputs.configured == 'true'
run: |
set -euo pipefail
bunx convex env set APP_BUILD_SHA "$GITHUB_SHA"
bunx convex env set APP_DEPLOYED_AT "$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
bunx convex env set SITE_URL "$STAGING_SITE_URL"
bunx convex env set VITE_SITE_URL "$STAGING_SITE_URL"
bunx convex env set CONVEX_SITE_URL "$STAGING_CONVEX_SITE_URL"
- name: Deploy Convex staging backend
if: steps.config.outputs.configured == 'true'
run: |
set -euo pipefail
if [[ "$ALLOW_DELETING_LARGE_INDEXES" == "true" ]]; then
bunx convex deploy --typecheck=disable --yes --allow-deleting-large-indexes
else
bun run convex:deploy
fi
- name: Verify Convex staging contract
if: steps.config.outputs.configured == 'true'
run: bun run verify:convex-contract
- name: Seed staging fixtures
if: steps.config.outputs.configured == 'true'
run: |
set -euo pipefail
if [[ "$RESET_STAGING_SEED" == "true" ]]; then
bunx convex run --no-push devSeed:seedNixSkills '{"reset":true}'
else
bunx convex run --no-push devSeed:seedNixSkills
fi
bunx convex run --no-push statsMaintenance:updateGlobalStatsAction
- name: Prepare staging deploy config
if: steps.config.outputs.configured == 'true'
run: |
bun run deploy:prepare-config -- \
--target staging \
--site-url "$STAGING_SITE_URL" \
--convex-site-url "$STAGING_CONVEX_SITE_URL"
- name: Pull Vercel staging project settings
if: steps.config.outputs.configured == 'true'
run: |
bunx "vercel@$VERCEL_CLI_VERSION" pull \
--yes \
--environment=staging \
--token "$VERCEL_TOKEN"
- name: Sync Vercel staging public env
if: steps.config.outputs.configured == 'true'
run: |
set -euo pipefail
set_vercel_env() {
local name="$1"
local value="$2"
bunx "vercel@$VERCEL_CLI_VERSION" env add "$name" staging \
--force \
--yes \
--value "$value" \
--token "$VERCEL_TOKEN"
}
set_vercel_env VITE_CONVEX_URL "$STAGING_CONVEX_URL"
set_vercel_env VITE_CONVEX_SITE_URL "$STAGING_CONVEX_SITE_URL"
set_vercel_env CONVEX_SITE_URL "$STAGING_CONVEX_SITE_URL"
set_vercel_env SITE_URL "$STAGING_SITE_URL"
set_vercel_env VITE_SITE_URL "$STAGING_SITE_URL"
set_vercel_env VITE_APP_BUILD_SHA "$GITHUB_SHA"
- name: Deploy Vercel staging frontend
id: vercel
if: steps.config.outputs.configured == 'true'
run: |
set -euo pipefail
deployment_url="$(bunx "vercel@$VERCEL_CLI_VERSION" deploy \
--target=staging \
--yes \
--token "$VERCEL_TOKEN")"
echo "deployment_url=$deployment_url" >> "$GITHUB_OUTPUT"
echo "Vercel staging deployment: $deployment_url"
- name: Smoke test staging HTTP
if: steps.config.outputs.configured == 'true'
env:
CLAWHUB_E2E_SITE: https://staging.hub.openclaw.ai
CLAWHUB_E2E_SKILL_OWNER: local
CLAWHUB_E2E_SKILL_SLUG: padel
run: bun run test:e2e:prod-http
- name: Install Playwright browser
if: steps.config.outputs.configured == 'true'
run: bunx playwright install --with-deps chromium
- name: Smoke test staging UI
if: steps.config.outputs.configured == 'true'
run: bunx playwright test --workers=1 --project=chromium e2e/menu-smoke.pw.test.ts
- name: Report unconfigured staging
if: steps.config.outputs.configured != 'true'
run: |
echo "Staging deploy is wired in the repo but not configured yet."
echo "Add the Staging environment secrets and variables documented in docs/deploy.md."

250
.github/workflows/deploy.yml vendored Normal file
View File

@ -0,0 +1,250 @@
name: Deploy
on:
workflow_dispatch:
inputs:
target:
description: "What to deploy"
required: true
default: full
type: choice
options:
- full
- backend
- frontend
allow_deleting_large_indexes:
description: "Allow Convex to delete large indexes"
required: true
default: false
type: boolean
concurrency:
group: deploy-production
cancel-in-progress: true
permissions:
contents: write
statuses: read
jobs:
validate-deploy-request:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
deploy_backend: ${{ steps.mode.outputs.deploy_backend }}
deploy_frontend: ${{ steps.mode.outputs.deploy_frontend }}
run_smoke: ${{ steps.mode.outputs.run_smoke }}
target: ${{ steps.mode.outputs.target }}
steps:
- name: Require main ref for production deploy
run: |
set -euo pipefail
if [[ "${GITHUB_REF}" != "refs/heads/main" ]]; then
echo "Production deploys must run from main."
exit 1
fi
- name: Resolve deploy mode
id: mode
run: |
set -euo pipefail
target="${{ inputs.target }}"
case "$target" in
full)
echo "deploy_backend=true" >> "$GITHUB_OUTPUT"
echo "deploy_frontend=true" >> "$GITHUB_OUTPUT"
echo "run_smoke=true" >> "$GITHUB_OUTPUT"
;;
backend)
echo "deploy_backend=true" >> "$GITHUB_OUTPUT"
echo "deploy_frontend=false" >> "$GITHUB_OUTPUT"
echo "run_smoke=true" >> "$GITHUB_OUTPUT"
;;
frontend)
echo "deploy_backend=false" >> "$GITHUB_OUTPUT"
echo "deploy_frontend=true" >> "$GITHUB_OUTPUT"
echo "run_smoke=true" >> "$GITHUB_OUTPUT"
;;
*)
echo "Unsupported deploy target: $target" >&2
exit 1
;;
esac
echo "target=$target" >> "$GITHUB_OUTPUT"
deploy-production:
runs-on: ubuntu-latest
timeout-minutes: 45
needs: validate-deploy-request
environment:
name: Production
url: https://clawhub.ai
env:
CONVEX_DEPLOY_KEY: ${{ secrets.CONVEX_DEPLOY_KEY }}
PLAYWRIGHT_AUTH_STORAGE_STATE_JSON: ${{ secrets.PLAYWRIGHT_AUTH_STORAGE_STATE_JSON }}
PLAYWRIGHT_BASE_URL: https://clawhub.ai
steps:
- name: Check deploy configuration
run: |
set -euo pipefail
missing=()
if [[ "${{ needs.validate-deploy-request.outputs.deploy_backend }}" == "true" && -z "$CONVEX_DEPLOY_KEY" ]]; then
missing+=("CONVEX_DEPLOY_KEY")
fi
if (( ${#missing[@]} > 0 )); then
echo "::error::Missing required production environment secrets: ${missing[*]}"
exit 1
fi
echo "Deploy target: ${{ needs.validate-deploy-request.outputs.target }}"
echo "Allow deleting large Convex indexes: ${{ inputs.allow_deleting_large_indexes }}"
if [[ -z "$PLAYWRIGHT_AUTH_STORAGE_STATE_JSON" ]]; then
echo "PLAYWRIGHT_AUTH_STORAGE_STATE_JSON not set; authenticated smoke will be skipped."
fi
- uses: actions/checkout@v6
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: 1.3.10
- name: Install
run: bun install --frozen-lockfile
- name: Stamp Convex build SHA
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
run: bunx convex env set APP_BUILD_SHA "${GITHUB_SHA}" --prod
- name: Stamp Convex deploy time
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
run: bunx convex env set APP_DEPLOYED_AT "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" --prod
- name: Deploy Convex
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
run: |
set -euo pipefail
if [[ "${{ inputs.allow_deleting_large_indexes }}" == "true" ]]; then
bunx convex deploy --typecheck=disable --yes --allow-deleting-large-indexes
else
bun run convex:deploy
fi
- name: Verify Convex contract
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
run: bun run verify:convex-contract -- --prod
- name: Wait for Vercel production deployment
id: vercel
if: needs.validate-deploy-request.outputs.deploy_frontend == 'true'
env:
GH_TOKEN: ${{ github.token }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_SHA: ${{ github.sha }}
VERCEL_STATUS_CONTEXT: Vercel clawhub
run: |
set -euo pipefail
for attempt in {1..90}; do
if ! status_json="$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA/status" \
--jq '.statuses[] | select(.context == env.VERCEL_STATUS_CONTEXT) | {state, target_url, description} | @base64' \
2>/dev/null | head -n1)"; then
echo "GitHub status check failed for $GITHUB_SHA on attempt $attempt; retrying..."
sleep 10
continue
fi
if [[ -z "$status_json" ]]; then
state=""
target_url=""
else
state="$(printf '%s' "$status_json" | base64 -d | jq -r '.state // ""')"
target_url="$(printf '%s' "$status_json" | base64 -d | jq -r '.target_url // ""')"
fi
case "$state" in
success)
echo "Vercel production deployment ready for $GITHUB_SHA"
echo "deployment_url=$target_url" >> "$GITHUB_OUTPUT"
exit 0
;;
failure|error)
echo "::error::Vercel production deployment failed for $GITHUB_SHA"
exit 1
;;
pending)
echo "Vercel deployment pending for $GITHUB_SHA on attempt $attempt; waiting..."
;;
*)
echo "Vercel status for $GITHUB_SHA not published yet on attempt $attempt; waiting..."
;;
esac
sleep 10
done
echo "::error::Timed out waiting for Vercel production deployment for $GITHUB_SHA"
exit 1
- name: Install Playwright browser
if: needs.validate-deploy-request.outputs.run_smoke == 'true' && needs.validate-deploy-request.outputs.deploy_frontend == 'true'
run: bunx playwright install --with-deps chromium webkit
- name: Smoke test production HTTP
if: needs.validate-deploy-request.outputs.run_smoke == 'true'
run: bun run test:e2e:prod-http
- name: Write authenticated storage state
if: needs.validate-deploy-request.outputs.run_smoke == 'true' && needs.validate-deploy-request.outputs.deploy_frontend == 'true' && env.PLAYWRIGHT_AUTH_STORAGE_STATE_JSON != ''
run: |
echo "$PLAYWRIGHT_AUTH_STORAGE_STATE_JSON" > "$RUNNER_TEMP/playwright-auth.json"
echo "PLAYWRIGHT_AUTH_STORAGE_STATE=$RUNNER_TEMP/playwright-auth.json" >> "$GITHUB_ENV"
- name: Smoke test production UI
if: needs.validate-deploy-request.outputs.run_smoke == 'true' && needs.validate-deploy-request.outputs.deploy_frontend == 'true'
run: bunx playwright test --workers=1 e2e/menu-smoke.pw.test.ts e2e/publish-entry-workflows.pw.test.ts e2e/upload-auth-smoke.pw.test.ts
- name: Tag production frontend deployment
if: needs.validate-deploy-request.outputs.deploy_frontend == 'true'
env:
DEPLOY_TARGET: ${{ needs.validate-deploy-request.outputs.target }}
DEPLOYMENT_URL: ${{ steps.vercel.outputs.deployment_url }}
run: |
set -euo pipefail
deployed_at="$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
tag_name="deploy/prod/$(date -u +"%Y%m%d-%H%M%SZ")-${GITHUB_SHA::7}"
version_prefix="prod/v$(date -u +"%Y.%m.%d")."
run_url="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
next_version=1
while IFS= read -r existing_tag; do
existing_tag="${existing_tag#refs/tags/}"
existing_tag="${existing_tag%\^\{\}}"
suffix="${existing_tag##*.}"
if [[ "$existing_tag" == "$version_prefix"* && "$suffix" =~ ^[0-9]+$ && "$suffix" -ge "$next_version" ]]; then
next_version=$((suffix + 1))
fi
done < <(git ls-remote --tags origin "refs/tags/${version_prefix}*" | awk '{print $2}' | sort -u)
version_tag="${version_prefix}${next_version}"
git tag -a "$tag_name" "$GITHUB_SHA" \
-m "Production frontend deploy $tag_name" \
-m "SHA: $GITHUB_SHA" \
-m "Version: $version_tag" \
-m "Deployed at: $deployed_at" \
-m "Target: $DEPLOY_TARGET" \
-m "Vercel: ${DEPLOYMENT_URL:-unknown}" \
-m "Run: $run_url"
git tag -a "$version_tag" "$GITHUB_SHA" \
-m "Production frontend deploy $version_tag" \
-m "SHA: $GITHUB_SHA" \
-m "Timestamp tag: $tag_name" \
-m "Deployed at: $deployed_at" \
-m "Target: $DEPLOY_TARGET" \
-m "Vercel: ${DEPLOYMENT_URL:-unknown}" \
-m "Run: $run_url"
git push origin "refs/tags/$tag_name" "refs/tags/$version_tag"

View File

@ -0,0 +1,36 @@
name: OpenClaw Docs Sync Dispatch
on:
push:
branches:
- main
paths:
- docs/**
- .github/workflows/openclaw-docs-sync-dispatch.yml
workflow_dispatch:
permissions:
contents: read
jobs:
dispatch-openclaw-docs-sync:
runs-on: ubuntu-latest
steps:
- name: Dispatch OpenClaw docs sync
env:
OPENCLAW_DOCS_SYNC_TOKEN: ${{ secrets.OPENCLAW_DOCS_SYNC_TOKEN }}
run: |
set -euo pipefail
if [ -z "${OPENCLAW_DOCS_SYNC_TOKEN:-}" ]; then
echo "::error::OPENCLAW_DOCS_SYNC_TOKEN is required to dispatch openclaw/openclaw docs sync."
exit 1
fi
curl --fail-with-body --silent --show-error \
--request POST \
--header "Authorization: Bearer ${OPENCLAW_DOCS_SYNC_TOKEN}" \
--header "Accept: application/vnd.github+json" \
--header "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/openclaw/openclaw/actions/workflows/docs-sync-publish.yml/dispatches \
--data '{"ref":"main"}'

354
.github/workflows/package-publish.yml vendored Normal file
View File

@ -0,0 +1,354 @@
name: Package Publish
on:
workflow_call:
inputs:
source:
description: Package source to publish. Usually owner/repo, owner/repo@ref, or a GitHub URL.
required: false
type: string
default: ""
ref:
description: Optional ref to append to the source when source is not already pinned.
required: false
type: string
dry_run:
description: Preview only. When true, no publish mutation is performed.
required: false
type: boolean
default: true
json:
description: Emit structured JSON output.
required: false
type: boolean
default: true
registry:
description: ClawHub registry URL.
required: false
type: string
default: https://clawhub.ai
site:
description: ClawHub site URL.
required: false
type: string
default: https://clawhub.ai
owner:
description: Optional owner handle override for org/shared publishing.
required: false
type: string
version:
description: Optional package version override.
required: false
type: string
tags:
description: Optional comma-separated tags override.
required: false
type: string
default: latest
source_repo:
description: Optional source repo override for local-folder publishes.
required: false
type: string
source_commit:
description: Optional source commit override for local-folder publishes.
required: false
type: string
source_ref:
description: Optional source ref override for local-folder publishes.
required: false
type: string
source_path:
description: Optional source path inside the repository for monorepo package publishes.
required: false
type: string
clawhub_version:
description: Legacy npm CLI version input. Kept for compatibility; the workflow now runs the checked-out source.
required: false
type: string
default: latest
secrets:
clawhub_token:
required: false
outputs:
publish_json:
description: Structured JSON output from clawhub package publish.
value: ${{ jobs.publish.outputs.publish_json }}
release_id:
description: Published release id when dry_run is false.
value: ${{ jobs.publish.outputs.release_id }}
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
permissions: {}
jobs:
publish:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
id-token: write
outputs:
publish_json: ${{ steps.capture.outputs.publish_json }}
release_id: ${{ steps.capture.outputs.release_id }}
steps:
- uses: actions/checkout@v6
with:
ref: ${{ github.sha }}
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: 1.3.10
- name: Resolve ClawHub workflow source
id: clawhub_source
run: |
python3 - <<'PY'
import base64
import json
import os
from pathlib import Path
from urllib.request import Request, urlopen
request_token = os.environ.get("ACTIONS_ID_TOKEN_REQUEST_TOKEN", "").strip()
request_url = os.environ.get("ACTIONS_ID_TOKEN_REQUEST_URL", "").strip()
if not request_token or not request_url:
raise SystemExit("GitHub OIDC token request env vars are missing; id-token: write is required.")
audience = "clawhub-workflow-source"
joiner = "&" if "?" in request_url else "?"
token_url = f"{request_url}{joiner}audience={audience}"
request = Request(
token_url,
headers={"Authorization": f"Bearer {request_token}"},
)
with urlopen(request) as response:
payload = json.load(response)
token = str(payload.get("value", "")).strip()
if not token:
raise SystemExit("GitHub OIDC token response did not include a token value.")
try:
encoded_payload = token.split(".")[1]
except IndexError as exc:
raise SystemExit("GitHub OIDC token was not a valid JWT.") from exc
padding = "=" * (-len(encoded_payload) % 4)
claims = json.loads(
base64.urlsafe_b64decode(encoded_payload + padding).decode("utf-8")
)
workflow_ref = str(claims.get("job_workflow_ref", "")).strip()
workflow_sha = str(claims.get("job_workflow_sha", "")).strip()
repo, marker, _ = workflow_ref.partition("/.github/workflows/")
if not marker or not repo or not workflow_sha:
raise SystemExit(
"Unable to resolve reusable workflow source from GitHub OIDC claims: "
f"job_workflow_ref={workflow_ref!r} job_workflow_sha={workflow_sha!r}"
)
output_path = Path(os.environ["GITHUB_OUTPUT"])
with output_path.open("a", encoding="utf-8") as fh:
fh.write(f"repository={repo}\n")
fh.write(f"ref={workflow_sha}\n")
PY
- uses: actions/checkout@v6
with:
repository: ${{ steps.clawhub_source.outputs.repository }}
ref: ${{ steps.clawhub_source.outputs.ref }}
path: clawhub-source
- name: Install ClawHub CLI dependencies
working-directory: clawhub-source
run: bun install --frozen-lockfile
- name: Validate publish mode inputs
env:
DRY_RUN: ${{ inputs.dry_run }}
JSON_MODE: ${{ inputs.json }}
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
GITHUB_EVENT_NAME: ${{ github.event_name }}
run: |
if [[ "$JSON_MODE" != "true" ]]; then
echo "::warning::This reusable workflow always emits JSON output; forcing --json for downstream parsing."
fi
if [[ "$DRY_RUN" == "true" ]]; then
exit 0
fi
if [[ -n "$CLAWHUB_TOKEN" ]]; then
exit 0
fi
if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" && -n "${ACTIONS_ID_TOKEN_REQUEST_URL:-}" && -n "${ACTIONS_ID_TOKEN_REQUEST_TOKEN:-}" ]]; then
echo "No ClawHub token provided; publish will rely on GitHub OIDC trusted publishing."
exit 0
fi
echo "::error::Real publishes need secrets.clawhub_token, or GitHub OIDC on workflow_dispatch runs (permissions.id-token=write)."
exit 1
- name: Write ClawHub config
env:
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
CLAWHUB_REGISTRY: ${{ inputs.registry }}
run: |
if [[ -z "$CLAWHUB_TOKEN" ]]; then
echo "No ClawHub token provided, skipping config file creation."
exit 0
fi
python3 - <<'PY'
import json
import os
from pathlib import Path
path = Path(os.environ["RUNNER_TEMP"]) / "clawhub-config.json"
path.write_text(
json.dumps(
{
"registry": os.environ["CLAWHUB_REGISTRY"],
"token": os.environ["CLAWHUB_TOKEN"],
},
indent=2,
)
+ "\n",
encoding="utf-8",
)
print(path)
PY
echo "CLAWHUB_CONFIG_PATH=$RUNNER_TEMP/clawhub-config.json" >> "$GITHUB_ENV"
- name: Resolve publish command
env:
INPUT_SOURCE: ${{ inputs.source }}
INPUT_REF: ${{ inputs.ref }}
INPUT_DRY_RUN: ${{ inputs.dry_run }}
INPUT_OWNER: ${{ inputs.owner }}
INPUT_VERSION: ${{ inputs.version }}
INPUT_TAGS: ${{ inputs.tags }}
INPUT_SOURCE_REPO: ${{ inputs.source_repo }}
INPUT_SOURCE_COMMIT: ${{ inputs.source_commit }}
INPUT_SOURCE_REF: ${{ inputs.source_ref }}
INPUT_SOURCE_PATH: ${{ inputs.source_path }}
INPUT_SITE: ${{ inputs.site }}
INPUT_REGISTRY: ${{ inputs.registry }}
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
GITHUB_EVENT_NAME: ${{ github.event_name }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_REF: ${{ github.ref }}
GITHUB_SHA: ${{ github.sha }}
run: |
python3 - <<'PY'
import json
import os
import shlex
from pathlib import Path
source = os.environ["INPUT_SOURCE"].strip()
if not source:
source = os.environ["GITHUB_REPOSITORY"]
source_is_current_repo = source == os.environ["GITHUB_REPOSITORY"]
ref = os.environ["INPUT_REF"].strip()
if not ref and source_is_current_repo:
ref = os.environ["GITHUB_SHA"].strip()
is_local_source = source.startswith(".") or source.startswith("/") or Path(source).exists()
if ref and "@" not in source and not source.startswith("http") and not is_local_source:
source = f"{source}@{ref}"
cli_entry = (
Path(os.environ["GITHUB_WORKSPACE"])
/ "clawhub-source"
/ "packages"
/ "clawhub"
/ "src"
/ "cli.ts"
)
if not cli_entry.exists():
raise SystemExit(f"Missing ClawHub CLI entrypoint at {cli_entry}")
cmd = [
"bun",
str(cli_entry),
"package",
"publish",
source,
"--site",
os.environ["INPUT_SITE"],
"--registry",
os.environ["INPUT_REGISTRY"],
]
if os.environ["INPUT_DRY_RUN"] == "true":
cmd.append("--dry-run")
cmd.append("--json")
owner = os.environ["INPUT_OWNER"].strip()
version = os.environ["INPUT_VERSION"].strip()
tags = os.environ["INPUT_TAGS"].strip()
if owner:
cmd += ["--owner", owner]
if version:
cmd += ["--version", version]
if tags:
cmd += ["--tags", tags]
source_repo = os.environ["INPUT_SOURCE_REPO"].strip()
source_commit = os.environ["INPUT_SOURCE_COMMIT"].strip()
source_ref = os.environ["INPUT_SOURCE_REF"].strip()
source_path = os.environ["INPUT_SOURCE_PATH"].strip()
if source_repo:
cmd += ["--source-repo", source_repo]
if source_commit:
cmd += ["--source-commit", source_commit]
if source_ref:
cmd += ["--source-ref", source_ref]
elif source_is_current_repo:
github_ref = os.environ["GITHUB_REF"].strip()
if github_ref:
cmd += ["--source-ref", github_ref]
if source_path:
cmd += ["--source-path", source_path]
if os.environ["INPUT_DRY_RUN"] != "true" and os.environ["CLAWHUB_TOKEN"].strip():
cmd += [
"--manual-override-reason",
f"GitHub Actions {os.environ['GITHUB_EVENT_NAME'].strip()} publish via CLAWHUB_TOKEN",
]
path = Path(os.environ["RUNNER_TEMP"]) / "clawhub-package-publish-command.sh"
shell_line = " ".join(shlex.quote(part) for part in cmd)
path.write_text("#!/usr/bin/env bash\nset -euo pipefail\n" + shell_line + "\n", encoding="utf-8")
path.chmod(0o755)
print(shell_line)
PY
- name: Run package publish
run: |
set -euo pipefail
"$RUNNER_TEMP/clawhub-package-publish-command.sh" | tee "$RUNNER_TEMP/package-publish.json"
- name: Capture workflow outputs
id: capture
run: |
python3 - <<'PY'
import json
import os
from pathlib import Path
output_path = Path(os.environ["RUNNER_TEMP"]) / "package-publish.json"
raw = output_path.read_text(encoding="utf-8").strip()
parsed = json.loads(raw)
github_output = Path(os.environ["GITHUB_OUTPUT"])
with github_output.open("a", encoding="utf-8") as fh:
fh.write("publish_json<<__CLAWHUB_JSON__\n")
fh.write(json.dumps(parsed, indent=2))
fh.write("\n__CLAWHUB_JSON__\n")
release_id = str(parsed.get("releaseId", "") or "")
fh.write(f"release_id={release_id}\n")
PY
- name: Upload publish JSON artifact
uses: actions/upload-artifact@v7
with:
name: clawhub-package-publish-json
path: ${{ runner.temp }}/package-publish.json
if-no-files-found: error

66
.github/workflows/secret-scan.yml vendored Normal file
View File

@ -0,0 +1,66 @@
name: "Security Gate: Secret Scanning"
on:
push:
branches: ["**"]
pull_request:
branches: [main, master]
permissions: {}
jobs:
trufflehog:
name: Scan for Verified Secrets
runs-on: ubuntu-latest
permissions:
contents: read # Required to scan the code in the PR
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0 # necessary to support the scoping requirements below
- name: Resolve scan range
id: scan_range
env:
EVENT_NAME: ${{ github.event_name }}
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PUSH_BASE_SHA: ${{ github.event.before }}
PUSH_HEAD_SHA: ${{ github.sha }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: |
set -euo pipefail
zero_sha="0000000000000000000000000000000000000000"
if [[ "$EVENT_NAME" == "pull_request" ]]; then
base="$PR_BASE_SHA"
head="$PR_HEAD_SHA"
else
base="$PUSH_BASE_SHA"
head="$PUSH_HEAD_SHA"
if [[ -z "$base" || "$base" == "$zero_sha" ]]; then
base="origin/$DEFAULT_BRANCH"
fi
fi
echo "base=$base" >> "$GITHUB_OUTPUT"
echo "head=$head" >> "$GITHUB_OUTPUT"
- name: TruffleHog OSS
id: trufflehog
# Use a concrete released ref that resolves in upstream action registry.
# v3 (major tag) is not published by trufflesecurity/trufflehog.
uses: trufflesecurity/trufflehog@v3.95.2
with:
path: ./
base: ${{ steps.scan_range.outputs.base }}
head: ${{ steps.scan_range.outputs.head }}
extra_args: --only-verified --debug
- name: Notify on Failure
if: steps.trufflehog.outcome == 'failure'
run: |
echo "::error::Verified secrets found! This PR contains live credentials that must be rotated immediately."
echo "::notice::If these secrets are already in the commit history, they cannot be removed via a simple removal commit/push. A repository owner can contact GitHub Support to purge the cached data: https://support.github.com/contact/private-information"
exit 1

171
.github/workflows/stale.yml vendored Normal file
View File

@ -0,0 +1,171 @@
name: Stale
on:
schedule:
- cron: "17 3 * * *"
workflow_dispatch:
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
permissions: {}
jobs:
stale:
permissions:
issues: write
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Mark stale unassigned issues and pull requests
uses: actions/stale@v10
with:
repo-token: ${{ github.token }}
days-before-issue-stale: 14
days-before-issue-close: 7
days-before-pr-stale: 7
days-before-pr-close: 5
stale-issue-label: stale
stale-pr-label: stale
exempt-issue-labels: enhancement,maintainer,pinned,security,no-stale,bad-barnacle
exempt-pr-labels: maintainer,no-stale,bad-barnacle
operations-per-run: 1000
ascending: true
exempt-all-assignees: true
remove-stale-when-updated: true
stale-issue-message: |
This issue has been automatically marked as stale due to inactivity.
Please add updated ClawHub details or it will be closed.
stale-pr-message: |
This pull request has been automatically marked as stale due to inactivity.
Please update it or it will be closed.
close-issue-message: |
Closing due to inactivity.
If this still affects ClawHub, reopen or file a new issue with the current URL, skill/package name, and fresh reproduction details.
close-issue-reason: not_planned
close-pr-message: |
Closing due to inactivity.
If this PR should be revived, reopen it with current context and a fresh validation plan.
- name: Mark stale assigned issues
uses: actions/stale@v10
with:
repo-token: ${{ github.token }}
days-before-issue-stale: 30
days-before-issue-close: 10
days-before-pr-stale: -1
days-before-pr-close: -1
stale-issue-label: stale
exempt-issue-labels: enhancement,maintainer,pinned,security,no-stale,bad-barnacle
operations-per-run: 1000
ascending: true
include-only-assigned: true
remove-stale-when-updated: true
stale-issue-message: |
This assigned issue has been automatically marked as stale after 30 days of inactivity.
Please add an update or it will be closed.
close-issue-message: |
Closing due to inactivity.
If this still affects ClawHub, reopen or file a new issue with current evidence.
close-issue-reason: not_planned
- name: Mark stale assigned pull requests
uses: actions/stale@v10
with:
repo-token: ${{ github.token }}
days-before-issue-stale: -1
days-before-issue-close: -1
days-before-pr-stale: 27
days-before-pr-close: 5
stale-pr-label: stale
exempt-pr-labels: maintainer,no-stale,bad-barnacle
operations-per-run: 1000
ascending: true
include-only-assigned: true
ignore-pr-updates: true
remove-stale-when-updated: true
stale-pr-message: |
This assigned pull request has been automatically marked as stale after being open for 27 days.
Please add an update or it will be closed.
close-pr-message: |
Closing due to inactivity.
If this PR should be revived, reopen it with current context and a fresh validation plan.
lock-closed-issues:
permissions:
issues: write
runs-on: ubuntu-latest
steps:
- name: Lock closed issues after 48h of no comments
uses: actions/github-script@v9
with:
github-token: ${{ github.token }}
script: |
const lockAfterHours = 48;
const lockAfterMs = lockAfterHours * 60 * 60 * 1000;
const cutoffMs = Date.now() - lockAfterMs;
const { owner, repo } = context.repo;
let locked = 0;
let inspected = 0;
let page = 1;
while (true) {
const { data: issues } = await github.rest.issues.listForRepo({
owner,
repo,
state: "closed",
sort: "updated",
direction: "desc",
per_page: 100,
page,
});
if (issues.length === 0) {
break;
}
for (const issue of issues) {
if (issue.pull_request || issue.locked || !issue.closed_at) {
continue;
}
inspected += 1;
const closedAtMs = Date.parse(issue.closed_at);
if (!Number.isFinite(closedAtMs) || closedAtMs > cutoffMs) {
continue;
}
let lastCommentMs = 0;
if (issue.comments > 0) {
const { data: comments } = await github.rest.issues.listComments({
owner,
repo,
issue_number: issue.number,
per_page: 1,
page: 1,
sort: "created",
direction: "desc",
});
if (comments.length > 0) {
lastCommentMs = Date.parse(comments[0].created_at);
}
}
if (Math.max(closedAtMs, lastCommentMs || 0) > cutoffMs) {
continue;
}
await github.rest.issues.lock({
owner,
repo,
issue_number: issue.number,
lock_reason: "resolved",
});
locked += 1;
}
page += 1;
}
core.info(`Inspected ${inspected} closed issues; locked ${locked}.`);

View File

@ -0,0 +1,121 @@
name: Update Convex AI Files
on:
schedule:
# Midnight Pacific during daylight saving time. GitHub cron uses UTC.
- cron: "0 7 * * 1"
workflow_dispatch:
concurrency:
group: update-convex-ai-files
cancel-in-progress: false
permissions: {}
env:
BUN_VERSION: "1.3.10"
UPDATE_BRANCH: automation/update-convex-ai-files
jobs:
update:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: ${{ env.BUN_VERSION }}
- uses: actions/create-github-app-token@v3
id: app-token
continue-on-error: true
with:
app-id: "2729701"
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- uses: actions/create-github-app-token@v3
id: app-token-fallback
continue-on-error: true
if: steps.app-token.outcome == 'failure'
with:
app-id: "2971289"
private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }}
- name: Install dependencies
run: bun install --frozen-lockfile
- name: Update Convex AI files
run: |
"$(bun pm bin)/convex" ai-files update
- name: Check Convex AI files status
run: |
"$(bun pm bin)/convex" ai-files status
- name: Detect changes
id: changes
run: |
set -euo pipefail
if [[ -n "$(git status --porcelain)" ]]; then
echo "changed=true" >> "$GITHUB_OUTPUT"
else
echo "changed=false" >> "$GITHUB_OUTPUT"
fi
- name: Commit and push update branch
if: steps.changes.outputs.changed == 'true'
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -B "$UPDATE_BRANCH"
git add AGENTS.md CLAUDE.md .agents/skills convex/_generated/ai/guidelines.md convex/_generated/ai/ai-files.state.json
git commit -m "chore: update Convex AI files"
git push --force-with-lease origin "$UPDATE_BRANCH"
- name: Open or update pull request
if: steps.changes.outputs.changed == 'true'
env:
GH_TOKEN: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token || github.token }}
run: |
set -euo pipefail
body_file="$(mktemp)"
{
printf '%s\n' '## Summary'
printf '\n'
printf '%s\n' '- refresh Convex-managed AI guidance files'
printf '%s\n' '- keep AGENTS.md / CLAUDE.md Convex sections in sync when Convex updates them'
printf '%s\n' '- update repo-local Convex developer skills under .agents/skills'
printf '\n'
printf '%s\n' '## Validation'
printf '\n'
# shellcheck disable=SC2016
printf '%s\n' '- `$(bun pm bin)/convex ai-files status`'
} > "$body_file"
if gh pr view "$UPDATE_BRANCH" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
gh pr edit "$UPDATE_BRANCH" \
--repo "$GITHUB_REPOSITORY" \
--title "[automation] Update Convex AI files" \
--body-file "$body_file"
else
gh pr create \
--repo "$GITHUB_REPOSITORY" \
--base main \
--head "$UPDATE_BRANCH" \
--title "[automation] Update Convex AI files" \
--body-file "$body_file"
fi

23
.gitignore vendored
View File

@ -2,6 +2,7 @@ node_modules
.DS_Store
.bun-build
*.bun-build
.data/
bin/docs-list
dist
dist-ssr
@ -10,7 +11,9 @@ dist-ssr
*.local
.vercel
count.txt
.env
.env*
!.env.local.example
!.env.example
.nitro
.tanstack
.wrangler
@ -21,6 +24,24 @@ todos.json
.vscode
.env*.local
coverage
eval/cache/
eval/results/
playwright-report
test-results
.playwright
convex/_generated/*
!convex/_generated/ai/
convex/_generated/ai/*
!convex/_generated/ai/guidelines.md
!convex/_generated/ai/ai-files.state.json
skills-lock.json
*/skills/*
!.agents/skills/
!.agents/skills/convex*/
!.agents/skills/convex*/**
!.agents/skills/blacksmith-testbox/
!.agents/skills/blacksmith-testbox/**
skills/*
.codex/*
!.codex/environments/
!.codex/environments/environment.toml

1
.nvmrc Normal file
View File

@ -0,0 +1 @@
22

20
.oxfmtrc.jsonc Normal file
View File

@ -0,0 +1,20 @@
{
"$schema": "./node_modules/oxfmt/configuration_schema.json",
"experimentalSortImports": {
"newlinesBetween": false,
},
"experimentalSortPackageJson": {
"sortScripts": true,
},
"ignorePatterns": [
".output/",
".tanstack/",
"convex/_generated/",
"coverage/",
"dist/",
"node_modules/",
"public/",
"src/routeTree.gen.ts",
"test-results/",
],
}

View File

@ -1,3 +1,38 @@
{
"ignorePatterns": ["node_modules", "dist", "coverage", "convex/_generated", ".tanstack", "public"]
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": ["unicorn", "typescript", "oxc"],
"categories": {
"correctness": "error",
"perf": "error",
"suspicious": "error"
},
"rules": {
"curly": "off",
"eslint-plugin-unicorn/prefer-array-find": "off",
"eslint-plugin-unicorn/no-array-sort": "off",
"eslint/no-await-in-loop": "off",
"eslint/no-underscore-dangle": "off",
"eslint/no-new": "off",
"oxc/no-accumulating-spread": "off",
"oxc/no-async-endpoint-handlers": "off",
"oxc/no-map-spread": "off",
"typescript/no-explicit-any": "error",
"typescript/no-extraneous-class": "off",
"typescript/no-unnecessary-boolean-literal-compare": "off",
"typescript/no-unnecessary-type-assertion": "off",
"typescript/no-unsafe-type-assertion": "off",
"unicorn/consistent-function-scoping": "off",
"unicorn/require-post-message-target-origin": "off"
},
"ignorePatterns": [
".output/",
".tanstack/",
"convex/_generated/",
"coverage/",
"dist/",
"node_modules/",
"public/",
"src/routeTree.gen.ts",
"test-results/"
]
}

View File

@ -1,44 +1,130 @@
# Repository Guidelines
## Project Structure & Module Organization
- `src/` — TanStack Start app code (routes, components, styles).
- `convex/` — Convex backend (schema, queries/mutations/actions, HTTP routes).
- `convex/_generated/` — generated Convex API/types; committed for builds.
- `docs/` — product/spec docs (see `docs/spec.md`).
- `docs/` — publishable public/operator docs for the ClawHub docs tab.
- `specs/` — product specs, plans, regression notes, design history (see `specs/spec.md`).
- `public/` — static assets.
## Durable Intent & Specs
- Use `specs/` to persist system/subsystem intent, invariants, and design rationale that future agents should preserve.
- Keep intended behavior for security-sensitive flows there, especially moderation, upload gating, scanner outcomes, appeals, bans, ownership, package installability, and API trust boundaries.
- If code changes reveal or change how a subsystem is supposed to work, update the relevant spec or add a focused spec note instead of burying the intent only in PR text or public docs.
- Keep `docs/` user/operator-facing: explain current behavior and commands there, but put internal “why this must work this way” context in `specs/`.
## Build, Test, and Development Commands
- `bun run dev` — local app server at `http://localhost:3000`.
- `bun run build` — production build (Vite + Nitro).
- `bun run preview` — preview built app.
- `bunx convex dev` — Convex dev deployment + function watcher.
- `bunx convex codegen` — regenerate `convex/_generated`.
- `bun run format:check` — formatting check.
- `bun run lint` — Biome + oxlint (type-aware).
- `bun run test` — Vitest (unit tests).
- `bun run coverage` — coverage run; keep global >= 80%.
## Coding Style & Naming Conventions
- TypeScript strict; ESM.
- Indentation: 2 spaces, single quotes (Biome).
- Lint/format: Biome + oxlint (type-aware).
- Convex function names: verb-first (`getBySlug`, `publishVersion`).
## Testing Guidelines
- Framework: Vitest 4 + jsdom.
- Tests live in `src/**` and `convex/lib/**`.
- Coverage threshold: 80% global (lines/functions/branches/statements).
- Example: `convex/lib/skills.test.ts`.
## Commit & Pull Request Guidelines
- Commit messages: Conventional Commits (`feat:`, `fix:`, `chore:`, `docs:`…).
- Keep changes scoped; avoid repo-wide search/replace.
- Before commit/PR handoff, run `bun run format:check` and `bun run lint`; include commands run in the PR summary.
- PRs: include summary + test commands run. Add screenshots for UI changes.
- Before merging any PR, verify TypeScript cleanly with `bunx tsc -p packages/schema/tsconfig.json --noEmit` and `bunx tsc -p packages/clawhub/tsconfig.json --noEmit`; if Convex code changed, also run the repo typecheck path used by deploy so `bunx convex deploy` will not fail on `tsc`.
- GitHub comments: for multiline `gh` comments/close messages, use `--body-file`, `--input`, or stdin/heredoc with real newlines; never pass literal `\\n` in shell strings.
- Reject PRs that add skills into source code/repo content directly (for example under `skills/` or seed-only additions intended as published skills). Skills must be uploaded/published via CLI.
- Repo-local Convex developer skills under `.agents/skills/convex*/` are allowed when they support working on this codebase; keep top-level `skills/` reserved for installed/published skill content and ignored by git.
## Production Release
- Production deploys are manual-only. Merging to `main` does **not** deploy.
- To release production, start the GitHub Actions `Deploy` workflow from `main`:
`gh workflow run deploy.yml --repo openclaw/clawhub --ref main`
- The workflow supports `full`, `backend`, and `frontend` targets.
- `frontend` currently means: wait for the Vercel production deploy for the selected `main` SHA, then run production smoke checks. It does not call `vercel deploy` directly yet.
- The workflow uses the GitHub `Production` environment for deploy secrets, but it does not require a separate approval step.
- Prod deploy secrets live on the `Production` environment, not as ordinary repo secrets. Required: `CONVEX_DEPLOY_KEY`. Optional: `PLAYWRIGHT_AUTH_STORAGE_STATE_JSON`.
- CLI npm releases are also manual-only and tag-based. Stable tags only: `vX.Y.Z`. Start `ClawHub CLI NPM Release` from `main`, first with `preflight_only=true`, then rerun it with the same tag and the successful `preflight_run_id`.
- Real CLI publishes wait at the GitHub `npm-release` environment and use npm trusted publishing. Required npm trusted publisher settings: repository `openclaw/clawhub`, workflow `clawhub-cli-npm-release.yml`, environment `npm-release`.
## Git Notes
- If `git branch -d/-D <branch>` is policy-blocked, delete the local ref directly: `git update-ref -d refs/heads/<branch>`.
## URL Quick Reference
- Canonical site: `https://clawhub.ai` (prefer this over legacy domains).
- Skill page URL format: `https://clawhub.ai/<owner>/<slug>` (owner handle preferred; falls back to owner id).
- Skill API detail URL: `https://clawhub.ai/api/v1/skills/<slug>`.
- Skill file URL: `https://clawhub.ai/api/v1/skills/<slug>/file?path=SKILL.md`.
- For “full URL?” requests, return the canonical page URL first, then API URL if useful.
## Configuration & Security
- Local env: `.env.local` (never commit secrets).
- Convex env holds JWT keys; Vercel only needs `VITE_CONVEX_URL` + `VITE_CONVEX_SITE_URL`.
- OAuth: GitHub OAuth App credentials required for login.
## Convex Ops (Gotchas)
- New Convex functions must be pushed before `convex run`: use `bunx convex dev --once` (dev) or `bunx convex deploy --prod` (prod).
- New Convex functions must be pushed before `convex run`: use `bunx convex dev --once` (dev) or `bunx convex deploy` (prod).
- For non-interactive prod deploys, use `bunx convex deploy -y` to skip confirmation.
- If `bunx convex run --env-file .env.local ...` returns `401 MissingAccessToken` despite `bunx convex login`, workaround: omit `--env-file` and use `--deployment-name <name>` / `--prod`.
## Convex Query & Bandwidth Rules
- **Always use `.withIndex()` instead of `.filter()` for fields that can be indexed.** `.filter()` causes full table scans — every doc is read and billed. Even a single `.filter()` on a 16K-row table reads ~16 MB per call.
- **Convex reads entire documents** — no field projections. If you only need a few fields from large docs (~6 KB+), denormalize a lightweight summary onto the parent doc or use a lookup table (see `embeddingSkillMap`, `skill.latestVersionSummary`, `skill.badges` for examples).
- **Denormalization pattern**: persist computed fields so they can be indexed. Every mutation that updates source fields must also update the denormalized field. Always write a cursor-based backfill for new fields (see `backfillIsSuspiciousInternal`, `backfillLatestVersionSummaryInternal`, `backfillDenormalizedBadgesInternal` for examples).
- **Cron jobs must never scan entire tables.** Use indexed queries with equality filters. Use cursor-based pagination for large datasets. Prefer incremental/delta tracking over full recounts.
- **32K document limit per query.** Split `.collect()` calls by a partition field (e.g., one day at a time instead of a 7-day range). See `rebuildTrendingLeaderboardAction` in `convex/leaderboards.ts` for an example.
- **Common mistakes**: `.filter().collect()` without an index; `ctx.db.get()` on large docs in a loop for list views; while loops that paginate the whole table to find filtered results.
- **Before writing or reviewing Convex queries, check deployment health.** Run `bunx convex insights` to check for OCC conflicts, `bytesReadLimit`, and `documentsReadLimit` errors. Run `bunx convex logs --failure` to see individual error messages and stack traces. This helps identify which functions are causing bandwidth issues so you can prioritize fixes.
<!-- convex-ai-start -->
This project uses [Convex](https://convex.dev) as its backend.
When working on Convex code, **always read `convex/_generated/ai/guidelines.md` first** for important guidelines on how to correctly use Convex APIs and patterns. The file contains rules that override what you may have learned about Convex from training data.
Convex agent skills for common tasks can be installed by running `npx convex ai-files install`.
<!-- convex-ai-end -->
## Stat Field Migration Rules
The `skills` table maintains two parallel sets of stat fields as part of an in-progress field migration:
| Legacy (nested, `@deprecated`) | Top-level (source of truth, indexable) |
| ------------------------------ | -------------------------------------- |
| `stats.downloads` | `statsDownloads` |
| `stats.stars` | `statsStars` |
| `stats.installsCurrent` | `statsInstallsCurrent` |
| `stats.installsAllTime` | `statsInstallsAllTime` |
**Rules:**
- **Always use `readCanonicalStat(skill, field)` (`convex/lib/skillStats.ts`) to read** any of the four migrated fields. It prefers the top-level field and falls back to the nested field for pre-migration documents. Never access `skill.stats.downloads` / `.stars` / `.installsCurrent` / `.installsAllTime` directly.
- **Always use `applySkillStatDeltas()` to write** stat deltas. It writes both the top-level and nested fields in the same patch to keep them in sync.
- **Both sets of fields must be written together** in any patch that touches stat values (see the return shape of `applySkillStatDeltas`).
- **Nested-only reads are acceptable only for** `stats.comments` and `stats.versions` — no top-level field exists for these yet.
- The four legacy nested fields are marked `@deprecated` in `statsValidator` (schema.ts). Any IDE access to `skill.stats.downloads` etc. will show a strikethrough warning — treat this as a signal to use `readCanonicalStat()` instead.
- When adding new stat fields, follow the same dual-write pattern and add a cursor-based backfill mutation (see `backfillSkillStatFieldsInternal` for an example).

View File

@ -1,8 +1,357 @@
# Changelog
## 0.12.3 - 2026-05-06
### Fixes
- CLI/API: allow skill publishes to target an org/user publisher with `--owner` / `ownerHandle`, and keep root `SKILL.md` publishable even when broad ignore rules match Markdown files (thanks @deepujain).
- Packages: expose owned plugin/package soft-delete in the CLI and dashboard, keep moderator takedown access, and remove deleted packages from package search surfaces (thanks @Patrick-Erichsen).
- Packages: support monorepo package publishes, infer package owners from scoped names, and keep dry-run publishes metadata-only.
- Packages: validate code-plugin runtime entries against extracted files, allow admin plugin release publishes, and raise trusted-publish/admin API rate limits for legitimate publish bursts.
- API/Search: return lean skill list payloads, route package search through digest indexes, decode scoped package paths, and bound fallback scans to reduce production read pressure.
- Web: restore skill downloads and search paging, canonicalize scoped plugin paths, and improve mobile layout responsiveness.
- Security: add scanner checks for confirmation bypasses and Python file upload exfiltration while reducing generic false-positive package tags.
## 0.12.2 - 2026-05-02
### Fixes
- CLI: publish code plugins as clawpacks and allow legacy package downloads to keep older install flows working.
- API: resolve scoped package routes and accept scoped npm packuments.
- Schema: allow nullable package SHA values in package responses and refresh generated schema artifacts.
## 0.12.1 - 2026-05-02
### Added
- Packages: add clawpack parsing, uploads, mirror artifact routes, artifact downloads, release moderation, reports, appeals, and official migration management across API, dashboard, and CLI.
- Security: add ClawScan security surfaces, owner rescan guidance, scanner-specific report pages, security dataset snapshots, and redacted skill-content exports.
- CLI: add unban support, moderation diagnostics in `inspect`, manual skill-directory listing, package environment filters, and package migration-status commands.
- Web: add skills/plugins search typeahead, featured plugin curation, plugin management tools, skill upload shortcuts, and dashboard pagination.
### Fixes
- API: raise public read rate limits to reduce false-positive 429s from browser pages and production smoke tests (thanks @steipete).
- CLI/moderation: allow `delete`, `hide`, `undelete`, and `unhide` to record moderation reasons in skill notes and audit logs for legal or policy reviews (thanks @steipete).
- Packages: make package publish retries idempotent, constrain catalog queries, keep package list queries single-page, count package archive downloads, and keep beta plugin packages off `latest`.
- Search: add soul lexical fallback, non-suspicious digest indexes, normalized skill prefix recall, and more stable relevance recall windows.
- Security: broaden static scanner coverage for unsafe credential, subprocess, browser-file, provider-secret, and remote-recipe patterns while hardening prompt-boundary handling.
- Deploy/CI: harden production smoke checks, expand PR validation coverage, add dead-code gates, and stabilize CodeQL light coverage.
- Dependencies: pin `undici` on the Node 20-compatible line after reverting the incompatible v8 update.
## 0.12.0 - 2026-04-28
### Added
- Security: add owner rescan requests, owner flagged inventory, scanner-specific security pages, and in-progress scan states.
- UI: adopt shadcn-managed primitives and polish the rescan/security surfaces for mobile.
### Fixes
- Moderation: calibrate VirusTotal Code Insight suspicious verdicts so uncorroborated AI-only findings do not keep otherwise clean skills quarantined (#1830, #1841) (thanks @deepujain).
- Security: flag exposed secrets in skill docs and normalize VirusTotal engine stats before caching.
- Packages: constrain plugin catalog queries and avoid catalog/package-list query limits.
- Auth: tolerate stale auth state when reading star status.
- CI: harden and debounce ClawSweeper dispatch workflows and fix production smoke coverage.
## 0.11.0 - 2026-04-28
### Changed
- Docs: clarify that ClawHub does not support paid skills, per-skill pricing, or paywalled releases (#1752, #1844) (thanks @deepujain).
- API docs: clarify how third-party directories can reuse public ClawHub catalog endpoints while respecting rate limits and canonical links (#1825, #1845) (thanks @deepujain).
- Packages docs: document the required fields for code-plugin package publish flows (#1802) (thanks @deepujain).
- Search: add CJK tokenization support (Chinese/Japanese/Korean) with Intl.Segmenter plus fallback behavior to improve skill query matching (#1596) (thanks @pq-dong).
- Stats: centralize migrated skill stat fallback reads through `readCanonicalStat()` and add schema/agent guardrails to discourage direct legacy nested-field access (#1709) (thanks @momothemage).
### Fixes
- Packages: use the configured `GITHUB_TOKEN` for trusted-publisher repository identity lookups to avoid anonymous GitHub API rate limits during publish setup (#1820, #1846) (thanks @deepujain).
- Packages: keep package search fallback scans bounded, stop scanning after the requested result limit, and keep direct plugin-name matches scoped to the requested package family (OpenClaw #64025).
- Moderation: stop flagging declared env vars sent to their intended API while preserving broad env scraping and exfiltration findings (#1803) (thanks @deepujain).
- Moderation: stop treating generic webhook integration docs as suspicious unless they include explicit Discord or Slack webhook endpoints (#1716) (thanks @langningchen-openclaw).
- Search: increase initial vector candidate pools and align CLI search's default limit with the web UI so high-scoring matches are not missed at small limits (#1375, #1429) (thanks @tjefferson).
- Search: fall back to lexical skill search when embedding generation fails instead of returning empty skill results (#1291) (thanks @goulonghui).
- Search: rank exact slug matches above longer slugs that merely contain all query tokens (#1130) (thanks @QuinnH496).
- Search: widen lexical fallback coverage and scan recently created skills so newly published skills can be found before embeddings rank well (#1185, #1200) (thanks @thirumaleshp).
- Search: preserve vector scores across candidate expansion and require all query tokens to match exact-token filters so relevant skills are not crowded out (#1759, #1762) (thanks @LinPower).
- Stats maintenance: keep skill stat migration fields synchronized by treating top-level stat fields as canonical during backfill/reconcile fallback reads (#1704) (thanks @momothemage).
- Skill install: render OpenClaw CLI commands with the bare slug that the current CLI accepts (#1807).
- Skills: keep historical tags out of public skill detail surfaces while preserving manager visibility (#1804) (thanks @deepujain).
- Skills moderation: keep hash-based scanner callbacks from overwriting skill-level moderation for older versions (#1805) (thanks @deepujain).
- Skills: prevent backport publishes from clobbering `latest` state and guard malformed persisted latest semver values during publish comparisons (#1832) (thanks @momothemage).
## 0.10.0 - 2026-04-05
### Added
- Design system: introduce a shared UI component library (`src/components/ui/`) built on Radix UI primitives — Button, Card, Badge, Tabs, Dialog, Input, Textarea, Label, Select, Avatar, Separator, Tooltip, ScrollArea, Sheet, Skeleton, and Table — following the shadcn/ui pattern with `cn()` + Tailwind utilities.
- Design system: `Button` supports `asChild` via Radix Slot for polymorphic rendering (e.g., wrapping `<Link>` without extra DOM).
- Layout: add `Container` component with `narrow` / `default` / `wide` size presets and `Breadcrumb` component for hierarchical navigation.
- Loading: add skeleton loading states (`SkillCardSkeleton`, `SkillDetailSkeleton`, `DashboardSkeleton`) replacing text-based "Loading..." indicators with animated placeholders.
- Errors: add `ErrorBoundary` with `resetKey` prop that auto-resets on route changes, wired into the root layout.
- Errors: surface fallback messages from Convex API error payloads in mutation/action error toasts.
- UX: add `EmptyState` component with icon, headline, description, and optional CTA action used across dashboard, stars, profile, and publish pages.
- UX: add confirmation dialogs for destructive skill ownership actions (transfer, abandon).
- Markdown: add `MarkdownPreview` component with `react-markdown`, `remark-gfm`, and `react-syntax-highlighter` for rich rendering of skill/plugin READMEs with syntax-highlighted code blocks, GFM tables, and task lists.
- Markdown: render tables with the new `Table` UI primitive for consistent styling across skill docs.
- Navigation: replace DropdownMenu-based mobile nav with a slide-out `Sheet` panel.
- Validation: add Zod schemas (`src/lib/schemas.ts`) for publish-skill, settings, report, and org forms.
- Management: restore capability-tags UI (crypto, requires-wallet, can-make-purchases, etc.) that was silently removed during the initial refactor.
- Management: add `.catch()` error handling with toast feedback on `setSoftDeleted` calls; prompt for hide/restore reasons.
### Changed
- CSS: migrate from a monolithic 5,161-line `styles.css` to Tailwind utilities on components, pruning CSS to ~1,000 lines (81% reduction). Dark mode now uses Tailwind `dark:` variants via a `@variant dark` directive bridging existing CSS custom properties.
- Tailwind: add `@theme` block mapping all CSS design tokens (`--bg`, `--surface`, `--ink`, `--accent`, `--line`, `--radius-*`, etc.) into first-class Tailwind utilities.
- Pages: modernize all route pages (home, skills browse, skill detail, dashboard, settings, publish-skill, publish-plugin, import, about, CLI auth, stars, souls, user profile, org profile, management, plugins browse, plugin detail) from CSS class selectors to Tailwind + UI primitives.
- Skills browse: widen container to `wide` (1400px) for better use of screen space on desktop; same for plugins browse.
- Skills browse: replace text-based filter toggles with pill chips and modernize toolbar layout.
- Skill detail: migrate tab controls from CSS-styled buttons to Radix `Tabs` primitive with proper `role="tab"` accessibility.
- Skill detail: replace inline CSS class-based install card with `SkillInstallCard` using Card + Button primitives.
- Header/Footer: migrate from CSS classes to Tailwind utilities with responsive Sheet-based mobile navigation.
- Dashboard: replace CSS table layout with `Table` UI primitive; add metric cards and skeleton loading.
- Settings: modernize form inputs with `Input`/`Textarea`/`Label` primitives and structured layout.
- Publish: use `Dialog` primitive for modals; inline validation indicators; modernized file list display.
### Fixed
- Auth: `EmptyState` "Sign in" button on publish page now triggers GitHub OAuth via `useAuthActions` instead of linking to non-existent `/signin` route.
- API: fix plugins page dev-mode `{"error":"Only HTML requests are supported here"}` by routing SSR and localhost API fetches directly to the Convex site URL instead of through TanStack Start's request pipeline.
- API: fix CORS error when `credentials: "include"` conflicts with `Access-Control-Allow-Origin: *` by making credentials conditional on same-origin requests.
- API: fix SSR `packageApiUrl` to always use `VITE_CONVEX_SITE_URL` directly, avoiding `getRequestUrl()` failures when SSR request context is unavailable.
- Management: restore `setSoftDeleted` reason parameter for hide/restore actions.
- Tests: rename `settings.test.tsx` to `-settings.test.tsx` to exclude from TanStack Router's file-based route discovery.
- Tests: add `@convex-dev/auth/react` mock for `useAuthActions` in upload route tests.
- Tests: update skill detail tests for Radix tab roles (`role="tab"` instead of `role="button"`), skeleton loading classes (`animate-pulse`), and capability tag data.
- Tests: update skills index tests for refreshed UI copy (placeholder text, empty state wording, loading indicator patterns).
- Tests: update SkillDiffCard tests for Tailwind active-tab class (`shadow-sm` replacing `.is-active`).
- Tests: update packages publish route tests for Tailwind border classes.
- Tests: update packageApi tests for conditional credentials and SSR URL resolution.
## 0.9.0 - 2026-03-23
### Added
- Packages/Plugins: add a first-class OpenClaw package registry across the web app, CLI, and HTTP API. ClawHub now supports package browse/search/detail/version/file/download flows plus `clawhub package explore`, `clawhub package inspect`, and `clawhub package publish` for `skill`, `code-plugin`, and `bundle-plugin` packages. (#1093)
- Packages/Install: package downloads now ship install-ready archives with a `package/` root, support nested files like `dist/index.js`, and work directly with OpenClaw plugin install flows.
- Skills/Web: server-render public skill pages and OG assets for faster first loads, cleaner sharing previews, and better cache behavior.
### Changed
- Browse/Search: rebuild public browse/search around denormalized digests, one-shot HTTP fetches, and deterministic cursors so the homepage and `/skills` are faster, more cacheable, and less likely to hit stale-tab or pagination dead ends.
- Search: default skill search to relevance, keep load-more retryable after fetch failures, and tighten package/skill catalog query paths to reduce inconsistent results under load.
### Fixed
- Packages/Auth: authenticated owners can now list, search, inspect, download, and read files from their own private packages instead of private packages being direct-URL-only. (#1093)
- Packages/API: stabilize package latest-version pointers, cursor pagination, publish outputs, fallback release resolution, and app-origin auth handling so package publish/search/install flows stay reliable.
- Visibility/API: prevent skills owned by deleted/banned users from showing up in public detail pages, browse/search results, or version API routes.
- Skills/API: sanitize public skill and soul version/file reads so hidden or invalid version data does not leak through direct API access.
- Skills/Web: keep Monaco compare layout toggles reliable while defaulting narrow screens to inline mode (#828) (thanks @geoffrey-xiao).
## 0.8.0 - 2026-03-13
### Added
- Skills/Web: show skill owner avatar + handle on skill cards, lists, and detail pages (#312) (thanks @ianalloway).
- Skills/Web: add file viewer for skill version files on detail page (#44) (thanks @regenrek).
- CLI: add `uninstall` command for skills (#241) (thanks @superlowburn).
- Skills/API/CLI: add ownership transfer workflow with request/list/accept/reject/cancel flows.
- Skills/Web/API: surface platform/architecture labels and security evaluation results in v1 + inspect views (#499, #362).
- API: add structured skill moderation responses plus `GET /api/v1/skills/{slug}/moderation` with redacted public evidence and full owner/staff detail (#334) (thanks @ArthurzKV).
- Moderation: persist structured moderation snapshots (static scan + VT/LLM merged verdict, reason codes, and evidence) on skills and versions (#333) (thanks @ArthurzKV).
- API: add scan security verification endpoint and non-suspicious filters (#820).
- Users: add `trustedPublisher` flag and admin mutations to bypass pending-scan auto-hide for trusted publishers (#298) (thanks @autogame-17).
- Moderation: add comment reporting with per-user active report caps, unique reporter/target enforcement, and auto-hide on the 4th unique report.
- Moderation: add AI-driven comment scam backfill (`commentModeration:*`) with persisted verdict/confidence/explainer metadata and strict auto-ban for `certain_scam` + `high` confidence.
- Admin: add manual unban for banned users (clears `deletedAt` + `banReason`, audit log entry). Revoked API tokens stay revoked.
- Admin: bulk restore skills from GitHub backup; reclaim squatted slugs via v1 endpoints + internal tooling (#298) (thanks @autogame-17).
- Moderation/Admin: add manual override audit tools for suspicious-skill review.
- CI/Security: add TruffleHog pull-request scanning for verified leaked credentials (#505) (thanks @akses0).
### Changed
- Skills: make published skill licensing explicit and fixed to MIT-0; require publish consent, surface no-attribution messaging in web/CLI/API, and remove per-skill license metadata.
- Skill metadata: support env vars, dependency declarations, author, and links in parsed manifest metadata + install UI (#360) (thanks @mahsumaktas).
- Rate limiting: apply authenticated quotas by user bucket (vs shared IP), emit delay-based reset headers, and improve CLI 429 guidance/retries (#412) (thanks @lc0rp).
- Skills: reserve deleted slugs for prior owners (90-day cooldown) to prevent squatting; add admin reclaim flow (#298) (thanks @autogame-17).
- Moderation: ban flow soft-deletes owned skills (reversible) and removes them from vector search (#298) (thanks @autogame-17).
- Security/docs: document comment reporting/auto-hide behavior alongside existing skill reporting rules.
- Security/moderation: add bounded explainable auto-ban reasons for scam comments and protect moderator/admin accounts from automated bans.
- Moderation: banning users now also soft-deletes their authored comments (skill + soul), including legacy cleanup on re-ban.
- Quality gate: language-aware word counting (`Intl.Segmenter`) and new `cjkChars` signal to reduce false rejects for non-Latin docs.
- Jobs: run skill stat event processing every 5 minutes (was 15).
- Deploy: add frontend/backend drift detection plus hardened production smoke/deploy checks.
- API performance: batch resolve skill/soul tags in v1 list/get endpoints (fewer action->query round-trips) (#112) (thanks @mkrokosz).
- LLM helpers: centralize OpenAI Responses text extraction for changelog/summary/eval flows (#502) (thanks @ianalloway).
- Search/listing performance: cut embedding hydration and badge read bandwidth via `embeddingSkillMap` + denormalized skill badges; shift stat-doc sync to low-frequency cron (#441) (thanks @sethconvex).
- Search/listing performance: move public browse/search hydration onto `skillSearchDigest`, add non-suspicious index paths, and split trending rebuilds to stay under Convex document limits.
### Fixed
- API: accept legacy CLI publish payloads during the v1 migration (#815).
- Auth/UI: surface OAuth callback failures in the web UI instead of swallowing them (#688).
- Skills: allow ownership healing when the previous owner was deleted/banned, and sanitize owner data in public payloads (#689, #793).
- CLI: validate explicit `install --force --version` targets before removing an existing local skill, preventing data loss when the requested version does not exist (#825) (thanks @jonathandeamer).
- Skills/Web: debounce search URL updates on `/skills` to keep typing responsive, and cancel stale pending navigations on external query changes (#587) (thanks @neeravmakwana).
- Upload: keep folder-picking enabled after page refresh by reapplying `webkitdirectory`/`directory` on the file input ref (#551) (thanks @MunemHashmi).
- CLI publish: use a longer multipart upload timeout and normalize abort rejections into proper Errors (#550) (thanks @MunemHashmi).
- CLI: forward optional auth tokens for `search` and `explore` against authenticated registries (#608) (thanks @artdaal).
- CLI: respect `HTTPS_PROXY`/`HTTP_PROXY`/`NO_PROXY` env vars for outbound registry requests, with troubleshooting docs (#363) (thanks @kerrypotter).
- CLI: preserve registry base paths when composing API URLs for search/inspect/moderation commands (#486) (thanks @Liknox).
- CLI: show manual URL guidance when automatic browser opening is unavailable; add regression tests for opener errors (#163) (thanks @aronchick).
- API/CLI: expose skill security status in version inspect output, with schema wiring and CLI regression coverage (#362) (thanks @abutbul).
- Moderation: remove over-broad keyword flags for common auth/payment/crypto terms so legitimate skills stop tripping regex prefilters (#273) (thanks @superlowburn).
- Skills hard-delete: delete `commentReports` rows during moderation cleanup to avoid orphaned report records.
- Comments: hide entries authored by deleted/deactivated users in `comments:listBySkill`.
- Admin API: `POST /api/v1/users/reclaim` now performs non-destructive root-slug owner transfer
(preserves existing skill versions/stats/metadata) and clears active slug reservations.
- VirusTotal: use shared AV-engine fallback verdict mapping for pending/backfill flows and keep undetected-only results pending (#591) (thanks @Shuai-DaiDai).
- Skills/listing: keep non-suspicious browse pagination on one cursor family during `isSuspicious` backfill, and re-sync stale `latestVersionSummary` metadata fields (#572) (thanks @sethconvex).
- PWA: update `manifest.json` branding so installed apps show the correct ClawHub name (#569) (thanks @Glucksberg).
- Search/tests: cover soft-deleted skill filtering in vector hydration and lexical exact-slug fallback (#552) (thanks @MunemHashmi).
- Docs/dev: fix local setup instructions for Node support, Convex env vars, frontend port, and post-seed stats refresh (#584) (thanks @jack-piplabs).
- Docs/CLI: fix `explore` flag list indentation so `--limit` renders correctly in the command reference (#601) (thanks @gandli).
- Skill metadata: parse top-level `requires.*`, `primaryEnv`, and homepage fallbacks for security review accuracy (#548) (thanks @MunemHashmi).
- Users: sync handle on ensure when GitHub login changes (#293) (thanks @christianhpoe).
- Users/Auth: throttle GitHub profile sync on login; also sync avatar when it changes (#312) (thanks @ianalloway).
- Upload gate: fetch GitHub account age by immutable account ID (prevents username swaps) (#116) (thanks @mkrokosz).
- VT fallback: activate only VT-pending hidden skills when scans are unavailable/stale; keep quality/scanner-blocked skills hidden (#300) (thanks @superlowburn).
- API: return proper status codes for delete/undelete errors (#35) (thanks @sergical).
- API: for owners, return clearer status/messages for hidden/soft-deleted skills instead of a generic 404.
- Web: allow copying OpenClaw scan summary text (thanks @borisolver, #322).
- HTTP/CORS: add preflight handler + include CORS headers on API/download errors; CLI: include auth token for owner-visible installs/updates (#146) (thanks @Grenghis-Khan).
- CLI: clarify `logout` only removes the local token; token remains valid until revoked in the web UI (#166) (thanks @aronchick).
- CLI: validate skill slugs used for filesystem operations (prevents path traversal) (#241) (thanks @superlowburn).
- Skills: keep global sorting across pagination on `/skills` (thanks @CodeBBakGoSu, #98).
- Skills: allow updating skill description/summary from frontmatter on subsequent publishes (#312) (thanks @ianalloway).
- Skills/Web: prevent filtered pagination dead-ends and loading-state flicker on `/skills`; move highlighted browse filtering into server list query (#339) (thanks @Marvae).
- Web: align `/skills` total count with public visibility and format header count (thanks @rknoche6, #76).
- Skills/Web: centralize public visibility checks and keep `globalStats` skill counts in sync incrementally; remove duplicate `/skills` default-sort fallback and share browse test mocks (thanks @rknoche6, #76).
- Moderation: clear stale `flagged.suspicious` flags when VirusTotal rescans improve to clean verdicts (#418) (thanks @Phineas1500).
- API tests: lock `Retry-After` behavior to relative-delay semantics for v1 search 429s (#421) (thanks @apoorvdarshan).
- CLI tests: assert 5xx HTTP responses still perform retry attempts before surfacing final error (#457) (thanks @YonghaoZhao722).
- GitHub import: improve storage/publish failure errors with actionable context; add regression tests for error formatting (#512) (thanks @vassiliylakhonin).
## 0.7.0 - 2026-02-16
Reconstructed from the `clawhub@0.7.0` npm publish timestamp (`2026-02-16T05:02:25Z`) and the repo version bump commit (`e352309`).
### Added
- Skills/Web: show owner avatars/handles across cards, lists, and detail pages (#312) (thanks @ianalloway).
- Skills/Web: add version file viewer on skill detail pages (#44) (thanks @regenrek).
- CLI: add `uninstall` for installed skills (#241) (thanks @superlowburn).
- Skills/Web: add non-suspicious browse filter, downloads-first browse defaults, and popular non-suspicious homepage sections.
- Web: compact-format skill and soul stats, plus split page models for skills/detail rendering.
- Skills: auto-generate missing summaries and add a resumable/self-scheduling summary backfill job.
- Moderation/Admin: add anti-spam publish caps, trust-tier quality checks, empty-skill cleanup tooling, and stronger moderator UX.
### Changed
- HTTP/CLI: centralize CORS handling and allow tokenized owner-visible reads through the CLI (#296, #297).
- API performance: batch resolve tags in v1 list/get flows to cut action-to-query round-trips (#112) (thanks @mkrokosz).
- Quality gate: add language-aware word counting and tighten spam/quarantine handling around publish flows.
### Fixed
- Skills/Web: fix initial sort wiring, keep global ordering across pagination, prevent pagination dead-ends/flicker, and harden cursor recovery (#92, #98, #339).
- CLI: normalize abort/timeout errors, secure config-file permissions, clarify logout semantics, and prefer `$HOME` for path resolution (#164, #166, #283, #286, #299).
- API: return correct delete/undelete status codes and clearer soft-delete/owner-visible error responses (#35) (thanks @sergical).
- Upload/Auth: gate publish ownership by immutable GitHub account ID and handle duplicate auth-user records safely.
- Downloads/Search: harden download dedupe/rate limiting, improve SSR host awareness, and fix homepage/search regressions under legacy data.
## 0.6.1 - 2026-02-13
### Added
- Security: add LLM-based security evaluation during skill publish.
- Parsing: recognize `metadata.openclaw` frontmatter and evaluate all skill files for requirements.
### Changed
- Performance: lazy-load Monaco diff viewer on demand (thanks @alexjcm, #212).
- Search: improve recall/ranking with lexical fallback and relevance prioritization.
- Moderation UX: collapse OpenClaw analysis by default; update spacing and default reasoning model.
### Fixed
- Skills: fix initial `/skills` sort wiring so first page respects selected sort/direction (thanks @bpk9, #92).
- Search/UI: add embedding request timeout and align `/skills` toolbar + list width (thanks @GhadiSaab, #53).
- Upload gate: handle GitHub API rate limits and optional authenticated lookup token (thanks @superlowburn, #246).
- HTTP: remove `allowH2` from Undici agent to prevent `fetch failed` on Node.js 22+ (#245).
- Tests: add root `undici` dev dependency for Node E2E imports (thanks @tanujbhaud, #255).
- Downloads: add download rate limiting + per-IP/day dedupe + scheduled dedupe pruning; preserve moderation gating and deterministic zips (thanks @regenrek, #43).
- VirusTotal: fix scan sync race conditions and retry behavior in scan/backfill paths.
- Metadata: tolerate trailing commas in JSON metadata.
- Auth: allow soft-deleted users to re-authenticate on fresh login, while keeping banned users blocked (thanks @tanujbhaud, #177).
- Web: prevent horizontal overflow from long code blocks in skill pages (thanks @bewithgaurav, #183).
## 0.6.0 - 2026-02-10
### Added
- CLI/API: add `set-role` to change user roles (admin only).
- Security: quarantine skill publishes with VirusTotal scans + UI (thanks @aleph8, #130).
- Testing: add tests for badges, skillZip, uploadFiles expandDroppedItems, and ark schema error truncation.
- Moderation: add ban reasons to API/CLI and show in management UI.
### Changed
- Coverage: track `convex/lib/skillZip.ts` in coverage reports.
### Fixed
- Web: show pending-scan skills to owners without 404 (thanks @orlyjamie, #136).
- Users: backfill empty handles from name/email in ensure (thanks @adlai88, #158).
- Web: update footer branding to OpenClaw (thanks @jontsai, #122).
- Auth: restore soft-deleted users on reauth, block banned users (thanks @mkrokosz, #106).
## 0.5.0 - 2026-02-02
### Added
- Admin: ban users and delete owned skills from management console.
- Moderation: auto-hide skills after 4 unique reports; per-user report cap; moderators can ban users.
- Uploads: require GitHub accounts to be at least 7 days old for skill + soul publish/import.
- CLI: add `inspect` to fetch skill metadata/files without installing.
- CLI: add moderation commands for hide/unhide/delete and ban users.
- Management: add filters for reported skills and users.
### Changed
- Deps: update dependencies to latest available versions.
- Reporting: require reasons, show them in management console, warn about abuse bans.
### Fixed
- Bans: batch hard-delete cleanup to avoid Convex read limits on large skills.
## 0.4.0 - 2026-01-30
### Added
- Web: show published skills on user profiles (thanks @njoylab, #20).
- CLI: include ClawHub + Moltbot fallback skill roots for sync scans.
- CLI: support OpenClaw configuration files (`OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR`).
### Changed
- Brand: rebrand to ClawHub and publish CLI as `clawhub` (legacy `clawdhub` supported).
- Domain: default site/registry now `https://clawhub.ai`; `.well-known/clawhub.json` preferred.
- Theme: persist theme under `clawhub-theme` (legacy key still read).
### Fixed
- Registry: drop missing skills during search hydration (thanks @aaronn, #28).
- CLI: use path-based skill metadata lookup for updates (thanks @daveonkels, #22).
- Search: keep highlighted-only filtering and clamp vector candidates to Convex limits (thanks @aaronn, #30).
## 0.3.0 - 2026-01-19
### Added
- CLI: add `explore` command for latest updates, with limit clamping + tests/docs (thanks @jdrhyne, #14).
- CLI: `explore --json` output + new sorts (`installs`, `installsAllTime`, `trending`) and limit up to 200.
- API: `/api/v1/skills` supports installs + trending sorts (7-day installs).
@ -10,39 +359,44 @@
- Registry: trending leaderboard + daily stats backfill for installs-based sorts.
### Fixed
- Web: keep search mode navigation and state in sync (thanks @NACC96, #12).
## 0.2.0 - 2026-01-13
### Added
- Web: dynamic OG image cards for skills (name, description, version).
- CLI: auto-scan Clawdbot skill roots (per-agent workspaces, shared skills, extraDirs).
- Web: import skills from public GitHub URLs (auto-detect `SKILL.md`, smart file selection, provenance).
- Web/API: SoulHub (SOUL.md registry) with v1 endpoints and first-run auto-seed.
### Fixed
- Web: stabilize skill OG image generation on server runtimes.
- Web: prevent skill OG text overflow outside the card.
- Registry: make SoulHub auto-seed idempotent and non-user-owned.
- Registry: keep GitHub backup state + publish backups intact (thanks @joshp123, #1).
- CLI/Registry: restore fork lineage on sync + clamp bulk list queries (thanks @joshp123, #1).
- CLI: default workdir falls back to Clawdbot workspace (override with `--workdir` / `CLAWDHUB_WORKDIR`).
- CLI: default workdir falls back to Clawdbot workspace (override with `--workdir` / `CLAWHUB_WORKDIR`).
## 0.0.6 - 2026-01-07
### Added
- API: v1 public REST endpoints with rate limits, raw file fetch, and OpenAPI spec.
- Docs: `docs/api.md` and `DEPRECATIONS.md` for the v1 cutover plan.
### Changed
- CLI: publish now uses single multipart `POST /api/v1/skills`.
- Registry: legacy `/api/*` + `/api/cli/*` marked for deprecation (kept for now).
## 0.0.5 - 2026-01-06
### Added
- Telemetry: track installs via `clawdhub sync` (logged-in only), per root, with 120-day staleness.
- Telemetry: track installs via `clawhub sync` (logged-in only), per root, with 120-day staleness.
- Skills: show current + all-time installs; sort by installs.
- Profile: private "Installed" tab with JSON export + delete telemetry controls.
- Docs: add `docs/telemetry.md` (what we track + how to opt out).
@ -50,30 +404,36 @@
- Web: dashboard for managing your published skills (thanks @dbhurley!).
### Changed
- CLI: telemetry opt-out via `CLAWDHUB_DISABLE_TELEMETRY=1`.
- CLI: telemetry opt-out via `CLAWHUB_DISABLE_TELEMETRY=1`.
- Web: move theme picker into mobile menu.
### Fixed
- Web: handle shorthand hex colors in diff theme (thanks @dbhurley!).
## 0.0.5 - 2026-01-06
### Added
- Maintenance: admin backfill to re-parse `SKILL.md` and repair stored summaries/parsed metadata.
### Fixed
- CLI sync: ignore plural `skills.md` docs files when scanning for skills.
- Registry: parse YAML frontmatter (incl multiline `description`) and accept YAML `metadata` objects.
## 0.0.4 - 2026-01-05
### Added
- Web: `/skills` list view with sorting (newest/downloads/stars/name) + quick filter.
- Web: admin/moderator highlight toggle on skill detail.
- Web: canonical skill URLs as `/<owner>/<slug>` (legacy `/skills/<slug>` redirects).
- Web: upload auto-generates a changelog via OpenAI when left blank (marked as auto-generated).
### Fixed
- Web: skill detail shows a loading state instead of flashing "Skill not found".
- Web: user profile shows avatar + loading state (no "User not found" flash).
- Web: improved mobile responsiveness (nav menu, skill detail layout, install command overflow).
@ -82,31 +442,37 @@
- CLI: ignore legacy `auth.clawdhub.com` registry and prefer site discovery.
### Changed
- Web: homepage search now expands into full search mode with live results + highlighted toggle.
- CLI: sync no longer prompts for changelog; registry auto-generates when blank.
## 0.0.3 - 2026-01-04
### Added
- CLI sync: concurrency flag to limit registry checks.
- Home: install command switcher (npm/pnpm/bun).
### Changed
- CLI sync: default `--concurrency` is now 4 (was 8).
- CLI sync: replace boxed notes with plain output for long lists.
### Fixed
- CLI sync: wrap note output to avoid terminal overflow; cap list lengths.
- CLI sync: label fallback scans as fallback locations.
- CLI package: bundle schema internally (no external `clawdhub-schema` publish).
- Repo: mark `clawdhub-schema` as private to prevent publishing.
- CLI package: bundle schema internally (no external `clawhub-schema` publish).
- Repo: mark `clawhub-schema` as private to prevent publishing.
## 0.0.2 - 2026-01-04
### Added
- CLI: delete/undelete commands for soft-deleted skills (owner/admin).
### Fixed
- CLI sync: dedupe duplicate slugs across scan roots; skip duplicates to avoid double-publish errors.
- CLI sync: show parsing progress while hashing local skills.
- CLI sync: prompt only actionable skills; preselect all by default; list synced separately; condensed synced summary when nothing to sync.
@ -119,6 +485,7 @@
## 0.0.1 - 2026-01-04
### Features
- CLI auth: login/logout/whoami; browser loopback auth; token storage; site/registry discovery; config overrides.
- CLI workflow: search, install, update (single/all), list, publish, sync (scan workdir + legacy roots), dry-run, version bumping, tags.
- Registry/API: skills + versions with semver; tags (latest + custom); changelog per version; SKILL.md frontmatter parsing; text-only validation; zip download; hash resolve; stats (downloads/stars/versions/comments).

57
CLAUDE.md Normal file
View File

@ -0,0 +1,57 @@
# ClawHub — Project Rules
## Convex Performance Rules
- For public listing/browse pages, use `ConvexHttpClient.query()` (one-shot fetch),
not `useQuery`/`usePaginatedQuery` (reactive subscription). Reserve reactive
queries for data the user needs to see update in real time.
- Denormalize hot read paths into a single lightweight "digest" table. Every
`ctx.db.get()` join adds a table to the reactive invalidation scope.
- When a `skillSearchDigest` row is available, use `digestToOwnerInfo(digest)`
to resolve owner data. NEVER call `ctx.db.get(ownerUserId)` when digest
owner fields (`ownerHandle`, `ownerName`, `ownerDisplayName`, `ownerImage`)
are already present. Reading from `users` adds the entire table to the
reactive read set and wastes bandwidth.
- Use `convex-helpers` Triggers to sync denormalized tables automatically.
Always add change detection — skip the write if no fields actually changed.
- Use compound indexes instead of JS filtering. If you're filtering docs after
the query, you're scanning documents you'll throw away.
- For search results scored by computed values (vector + lexical + popularity),
fetch all results once and paginate client-side. Don't re-run the full search
pipeline on "load more."
- Backfills on reactively-subscribed tables need `delayMs` between batches.
- Mutations that read >8 MB should use the Action → Query → Mutation pattern
to split reads across transactions.
## Convex Conventions
- All mutations import from `convex/functions.ts` (not `convex/_generated/server`)
to get trigger wrapping. Type imports still come from `convex/_generated/server`.
- NEVER use `--typecheck=disable` on `npx convex deploy`.
- Use `npx convex dev --once` to push functions once (not long-running watcher).
## Production Release
- Production deploys are manual-only. Merging to `main` does **not** deploy.
- Start the GitHub Actions `Deploy` workflow from `main` with `gh workflow run deploy.yml --repo openclaw/clawhub --ref main`.
- The workflow supports `full`, `backend`, and `frontend` targets.
- `frontend` currently waits for the Vercel production deploy on the selected `main` SHA and then runs smoke checks. It does not trigger Vercel directly yet.
- The workflow uses the `Production` environment for deploy secrets, but it does not wait for a separate approval.
- Required prod secret: `CONVEX_DEPLOY_KEY` on the `Production` environment. Optional smoke secret: `PLAYWRIGHT_AUTH_STORAGE_STATE_JSON`.
- CLI npm releases are manual-only and tag-based through `ClawHub CLI NPM Release`. Stable tags only: `vX.Y.Z`. Run a `preflight_only=true` pass first, then rerun with the same tag plus `preflight_run_id` for the real publish.
- Real CLI publishes wait at `npm-release` and rely on npm trusted publishing for `openclaw/clawhub` + `clawhub-cli-npm-release.yml` + `npm-release`.
## Testing
- Tests use `._handler` to call mutation handlers directly with mock `db` objects.
- Mock `db` objects MUST include `normalizeId: vi.fn()` for trigger wrapper compatibility.
<!-- convex-ai-start -->
This project uses [Convex](https://convex.dev) as its backend.
When working on Convex code, **always read `convex/_generated/ai/guidelines.md` first** for important guidelines on how to correctly use Convex APIs and patterns. The file contains rules that override what you may have learned about Convex from training data.
Convex agent skills for common tasks can be installed by running `npx convex ai-files install`.
<!-- convex-ai-end -->

219
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,219 @@
# Contributing to ClawHub
Welcome! ClawHub is the public skill registry for [OpenClaw](https://github.com/openclaw/openclaw). We appreciate bug fixes, documentation improvements, and feature contributions.
- **Questions?** Ask in [#clawhub on Discord](https://discord.gg/clawd).
- **Bug fixes** — PRs are welcome.
- **New features or architectural changes** — please start with a Discord conversation in #clawhub first so we can align on scope.
## Local Development Setup
### Prerequisites
- [Bun](https://bun.sh/) (Convex CLI runs via `bunx`, no global install needed)
- [Node.js](https://nodejs.org/) v18, 20, 22, or 24 (required by the local Convex backend; v25+ is not yet supported)
### Install and configure
```bash
bun install
cp .env.local.example .env.local
```
Edit `.env.local` with the following values for **local Convex**:
```bash
# Frontend
VITE_CONVEX_URL=http://127.0.0.1:3210
VITE_CONVEX_SITE_URL=http://127.0.0.1:3210
SITE_URL=http://localhost:3000
# Deployment used by `bunx convex dev`
CONVEX_DEPLOYMENT=anonymous:anonymous-clawhub
```
### GitHub OAuth App (for login)
1. Go to [github.com/settings/developers](https://github.com/settings/developers) and create a new OAuth App.
2. Set **Homepage URL** to `http://localhost:3000`.
3. Set **Authorization callback URL** to `http://127.0.0.1:3210/api/auth/callback/github`.
4. Copy the Client ID and generate a Client Secret.
### Run the Convex backend
Start the local Convex backend first — other setup steps depend on it:
```bash
bunx convex dev --typecheck=disable
```
### Set backend environment variables
The Convex backend has its own env var store separate from `.env.local`. With the backend running, open a new terminal and set the required variables:
```bash
bunx convex env set AUTH_GITHUB_ID <your-client-id>
bunx convex env set AUTH_GITHUB_SECRET <your-client-secret>
bunx convex env set SITE_URL http://localhost:3000
```
### JWT keys (for Convex Auth)
With the backend still running, generate the signing keys:
```bash
bunx @convex-dev/auth
```
This sets `JWT_PRIVATE_KEY` and `JWKS` on the Convex backend and outputs values you can also save to `.env.local` for reference.
### Run the frontend
```bash
bun run dev -- --port 3000
```
Change the port if 3000 is already in use, and update `SITE_URL` in both `.env.local` and the Convex backend (`bunx convex env set SITE_URL ...`) to match.
### Seed the database
Populate sample data so the UI isn't empty:
```bash
# 3 sample skills (padel, gohome, xuezh)
bunx convex run --no-push devSeed:seedNixSkills
# 50 extra skills for pagination testing (optional)
bunx convex run --no-push devSeedExtra:seedExtraSkillsInternal
# Refresh the cached skills count (required after seeding)
bunx convex run --no-push statsMaintenance:updateGlobalStatsAction
```
To reset and re-seed:
```bash
bunx convex run --no-push devSeed:seedNixSkills '{"reset": true}'
```
### Optional environment variables
These features degrade gracefully without their keys:
| Variable | Purpose |
| ------------------------------------------------------------------------- | --------------------------------------------------------- |
| `OPENAI_API_KEY` | Embeddings and vector search (falls back to zero vectors) |
| `VT_API_KEY` | VirusTotal malware scanning |
| `DISCORD_WEBHOOK_URL` | Discord notifications |
| `GITHUB_APP_ID` / `GITHUB_APP_PRIVATE_KEY` / `GITHUB_APP_INSTALLATION_ID` | GitHub backup sync |
## CLI Development
The CLI source lives in [`packages/clawhub/`](packages/clawhub/). Both `clawhub` and `clawdhub` are registered as bin aliases.
To test the CLI against your local instance:
```bash
CLAWHUB_REGISTRY=http://127.0.0.1:3210 CLAWHUB_SITE=http://localhost:3000 clawhub search "padel"
```
Use the package-local verification contract when working on the CLI:
```bash
bun run --cwd packages/clawhub test
bun run --cwd packages/clawhub verify:build
bun run --cwd packages/clawhub test:artifact
bun run --cwd packages/clawhub verify
```
`bun test packages/clawhub/` is not the supported workflow. Source tests and built-artifact smoke tests are intentionally split.
Manual smoke tests are documented in [`specs/manual-testing.md`](specs/manual-testing.md).
## Skill & Soul Publishing
- Skill format reference: [`docs/skill-format.md`](docs/skill-format.md)
- Soul format reference: [`docs/soul-format.md`](docs/soul-format.md)
- End-to-end walkthrough (search, install, publish, sync): [`docs/quickstart.md`](docs/quickstart.md)
Quick publish:
```bash
clawhub publish <path-to-skill-directory>
```
## Before Submitting a PR
```bash
bun run format:check # oxfmt
bun run lint # oxlint
bun run deadcode:ci # Knip files/deps/exports
bun run test # Vitest (80% coverage threshold)
bun run build # Vite + Nitro
bun run --cwd packages/clawhub verify
```
These are the same checks that run in CI (`.github/workflows/ci.yml`).
### Blacksmith Testbox checks
Maintainers with Blacksmith access can run the same checks in a warmed Testbox
instead of spending local CPU:
```bash
export CLAWHUB_TESTBOX=1
blacksmith testbox warmup ci-check-testbox.yml --ref main --idle-timeout 90
bun run testbox:claim -- --id <tbx_id>
bun run testbox:sanity -- --id <tbx_id>
bun run testbox:run -- --id <tbx_id> -- bun run lint
bun run testbox:run -- --id <tbx_id> -- bun run test
bun run testbox:run -- --id <tbx_id> -- bun run build
```
Use the `tbx_...` id from the current warmup output. The wrapper refuses ids
that are missing the local SSH key or were claimed by a different checkout.
Use `CLAWHUB_LOCAL_CHECK_MODE=throttled` or `CLAWHUB_LOCAL_CHECK_MODE=full` as
the explicit local escape hatch when you intentionally want laptop-side proof.
If Blacksmith auth/org access is missing, report that instead of falling back
to a broad local gate that can bog down a dev machine.
For the initial bootstrap only, the Testbox workflow must land on `main` before
`blacksmith testbox warmup ci-check-testbox.yml --ref <branch>` can dispatch it.
**PR guidelines:**
- Keep PRs focused — one concern per PR.
- Use [Conventional Commits](https://www.conventionalcommits.org/): `feat:`, `fix:`, `chore:`, `docs:`, etc.
- Include test commands and screenshots for UI changes.
- Write a clear description of what changed and why.
## AI-Generated Code
AI-assisted contributions are welcome. When submitting AI-generated or AI-assisted code:
- Note it in the PR description.
- Describe the level of testing you applied.
- Include prompts if useful for reviewers.
- Confirm that you understand and can maintain the code.
## Security Reporting
Report vulnerabilities to **security@openclaw.ai** with:
- Severity assessment
- Technical reproduction steps
- Suggested remediation
See [`docs/security.md`](docs/security.md) for moderation and upload gating details.
## Reading Order for New Contributors
1. This file (local setup)
2. [`docs/clawhub.md`](docs/clawhub.md) — public registry overview
3. [`docs/quickstart.md`](docs/quickstart.md) — end-to-end workflows
4. [`docs/architecture.md`](docs/architecture.md) — system design
5. [`docs/skill-format.md`](docs/skill-format.md) — skill structure
6. [`docs/cli.md`](docs/cli.md) — CLI reference
7. [`docs/http-api.md`](docs/http-api.md) — HTTP endpoints
8. [`docs/auth.md`](docs/auth.md) — authentication
9. [`docs/deploy.md`](docs/deploy.md) — deployment
10. [`docs/troubleshooting.md`](docs/troubleshooting.md) — common issues

362
DESIGN.md Normal file
View File

@ -0,0 +1,362 @@
# ClawHub Design System
This document outlines the design rules, patterns, and guidelines for the ClawHub platform to ensure consistency, accessibility, and maintainability across all components.
---
## Color System
### Brand Palette (OpenClaw)
ClawHub uses a strict **3-5 color palette** based on the OpenClaw brand:
| Token | Light Mode | Dark Mode | Usage |
| --------------- | ---------- | --------- | ----------------------------------------------- |
| `--accent` | `#dc2626` | `#dc2626` | Primary actions, interactive elements, emphasis |
| `--accent-deep` | `#b91c1c` | `#ef4444` | Hover states, secondary emphasis |
| `--ink` | `#0a0a0a` | `#fafafa` | Primary text |
| `--ink-soft` | `#525252` | `#a1a1a1` | Secondary text, descriptions |
| `--surface` | `#ffffff` | `#121212` | Card backgrounds, elevated surfaces |
| `--bg` | `#fafafa` | `#0a0a0a` | Page background |
### Rules
1. **Never exceed 5 colors** without explicit design approval
2. **Never use purple/violet prominently** unless explicitly requested
3. **Always override text color** when changing background color to ensure contrast
4. **Use semantic tokens** (`--accent`, `--ink`, `--surface`) instead of raw colors
---
## Typography
### Font Stack
```css
--font-sans: "Geist", system-ui, sans-serif;
--font-mono: "Geist Mono", monospace;
--font-display: "Geist", system-ui, sans-serif;
```
### Scale
| Token | Size | Usage |
| ----------- | --------------- | ------------------------ |
| `--fs-xs` | 0.75rem (12px) | Labels, badges, metadata |
| `--fs-sm` | 0.875rem (14px) | Body text, descriptions |
| `--fs-base` | 1rem (16px) | Default body text |
| `--fs-md` | 1.125rem (18px) | Subheadings |
| `--fs-lg` | 1.25rem (20px) | Section titles |
| `--fs-xl` | 1.5rem (24px) | Page headings |
### Rules
1. **Maximum 2 font families** per page
2. **Line height 1.4-1.6** for body text (use `leading-relaxed`)
3. **Never use decorative fonts** for body text
4. **Minimum font size: 14px** for readability
5. Use `text-balance` or `text-pretty` for titles
---
## Layout
### Method Priority
Use this hierarchy for layout decisions:
1. **Flexbox** - Default for most layouts
2. **CSS Grid** - Only for complex 2D layouts (cards, galleries)
3. **Never use floats** or absolute positioning unless absolutely necessary
### Spacing Scale
```css
--space-1: 0.25rem /* 4px */ --space-2: 0.5rem /* 8px */ --space-3: 0.75rem /* 12px */
--space-4: 1rem /* 16px */ --space-5: 1.5rem /* 24px */ --space-6: 2rem /* 32px */;
```
### Grid Patterns
#### Auto-fit Grid (Recommended for Cards)
```css
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
```
- Automatically adjusts columns based on container width
- Prevents orphan items on partial rows
- Maintains consistent card widths
#### Fixed Grid (When exact columns needed)
```css
/* 3-column at desktop, 2 at tablet, 1 at mobile */
grid-template-columns: repeat(3, minmax(0, 1fr));
@media (max-width: 860px) {
grid-template-columns: repeat(2, minmax(0, 1fr));
}
@media (max-width: 520px) {
grid-template-columns: 1fr;
}
```
### Container Widths
| Size | Max Width | Usage |
| ------- | ----------------------- | ----------------------- |
| Default | `--page-max` (1200px) | Standard pages |
| Narrow | `--page-narrow` (720px) | Reading content, forms |
| Wide | Full width | Dashboards, data tables |
---
## Components
### Cards
```css
.card {
padding: var(--space-4);
border: 1px solid var(--line);
border-radius: var(--r-md);
background: var(--surface);
}
```
**Rules:**
- Always use `display: flex; flex-direction: column;` for consistent height
- Add `flex: 1` to content area for equal-height cards in grids
- Include hover state with `border-color` and subtle `box-shadow`
### Buttons
| Variant | Usage |
| ------------- | ------------------------------------- |
| `primary` | Main actions (Submit, Save, Download) |
| `secondary` | Alternative actions |
| `ghost` | Tertiary actions, navigation |
| `destructive` | Delete, remove, dangerous actions |
**Rules:**
- Always include visible focus state
- Minimum touch target: 44x44px on mobile
- Include `aria-label` when icon-only
### Form Controls
- Labels above inputs (not inline)
- Error states use `--status-error-fg`
- Focus rings use `--accent` with 0.2 opacity
- Minimum input height: 40px
---
## Responsive Breakpoints
```css
/* Mobile first - base styles for mobile */
@media (min-width: 520px) {
/* Small tablets, large phones */
}
@media (min-width: 640px) {
/* Tablets */
}
@media (min-width: 860px) {
/* Small desktops, landscape tablets */
}
@media (min-width: 1024px) {
/* Desktops */
}
@media (min-width: 1280px) {
/* Large desktops */
}
```
### Rules
1. **Mobile-first approach** - Base styles target mobile
2. **Progressive enhancement** - Add complexity as viewport increases
3. **Test intermediate breakpoints** - Avoid jarring layout jumps
4. **Never hide critical content** on mobile
---
## Accessibility
### Color Contrast
- Normal text: Minimum 4.5:1 ratio
- Large text (18px+): Minimum 3:1 ratio
- Interactive elements: Minimum 3:1 ratio
### Focus States
```css
:focus-visible {
outline: 2px solid var(--accent);
outline-offset: 2px;
border-radius: 2px;
}
```
### Screen Readers
- Use `sr-only` class for visually hidden but accessible text
- Always include `alt` text for images (empty `alt=""` for decorative)
- Use semantic HTML elements (`main`, `nav`, `article`, `section`)
- Proper heading hierarchy (h1 > h2 > h3, no skipping)
### Motion
```css
/* Respect user preference */
@media (prefers-reduced-motion: reduce) {
* {
animation-duration: 0.01ms !important;
transition-duration: 0.01ms !important;
}
}
```
---
## Animation
### Timing
```css
--transition-fast: 150ms;
--transition-base: 200ms;
--transition-slow: 300ms;
```
### Easing
- Use `ease` or `ease-out` for most transitions
- Use `ease-in-out` for enter/exit animations
- Never use `linear` except for continuous animations
### Rules
1. **Subtle by default** - Avoid flashy animations
2. **Purpose-driven** - Animation should provide feedback
3. **Respect preferences** - Support `prefers-reduced-motion`
4. **Performance** - Use `transform` and `opacity` only
---
## Icons
### Usage
- Use Lucide icons consistently
- Standard sizes: 14px, 16px, 20px, 24px
- Include `aria-hidden="true"` for decorative icons
- Never use emojis as icons
### Placement
- Left of labels in buttons and navigation
- Right of labels for external links or dropdowns
- Centered when used alone with `aria-label`
---
## Dark Mode
### Implementation
```css
[data-theme="dark"] {
/* Dark mode overrides */
}
```
### Rules
1. Never use pure white (`#ffffff`) on dark backgrounds
2. Reduce shadow intensity in dark mode
3. Adjust image brightness if needed
4. Test contrast ratios in both modes
---
## Performance
### CSS
1. Use CSS custom properties for theming
2. Avoid deeply nested selectors (max 3 levels)
3. Use `will-change` sparingly
4. Prefer `transform` over `top/left` for animations
### Images
1. Always specify `width` and `height` attributes
2. Use `loading="lazy"` for below-fold images
3. Use appropriate formats (WebP with fallbacks)
4. Include placeholder or skeleton states
---
## Code Style
### CSS Class Naming
```css
/* Component */
.component-name {
}
/* Component modifier */
.component-name.variant {
}
/* Component child */
.component-name-child {
}
/* State */
.component-name.is-active {
}
.component-name[data-state="open"] {
}
```
### File Organization
```
src/
components/
ui/ # Primitive components (Button, Input, Card)
layout/ # Layout components (Container, Header)
styles.css # Global styles and design tokens
lib/
theme.ts # Theme utilities
preferences.ts # User preference management
```
---
## Checklist
Before shipping any UI changes, verify:
- [ ] Color contrast meets WCAG AA standards
- [ ] Focus states are visible
- [ ] Layout works at all breakpoints
- [ ] Animations respect `prefers-reduced-motion`
- [ ] Text is readable at default browser zoom
- [ ] Interactive elements have 44px minimum touch target
- [ ] Semantic HTML is used appropriately
- [ ] Dark mode has been tested

147
README.md
View File

@ -1,33 +1,48 @@
# ClawdHub
<p align="center">
<img src="public/clawd-logo.png" alt="ClawHub" width="120">
</p>
<h1 align="center">ClawHub</h1>
<p align="center">
<a href="https://github.com/clawdbot/clawdhub/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/clawdbot/clawdhub/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://github.com/openclaw/clawhub/actions/workflows/ci.yml?branch=main"><img src="https://img.shields.io/github/actions/workflow/status/openclaw/clawhub/ci.yml?branch=main&style=for-the-badge" alt="CI status"></a>
<a href="https://discord.gg/clawd"><img src="https://img.shields.io/discord/1456350064065904867?label=Discord&logo=discord&logoColor=white&color=5865F2&style=for-the-badge" alt="Discord"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
</p>
ClawdHub is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
Its designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
ClawHub is the **public skill registry for OpenClaw**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
It's designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
It also now exposes a native **OpenClaw package catalog** for code plugins and bundle plugins.
onlycrabs.ai is the **SOUL.md registry**: publish and share system lore the same way you publish skills.
Live: `https://clawdhub.com`
onlycrabs.ai: `https://onlycrabs.ai`
<p align="center">
<a href="https://clawhub.ai">ClawHub</a> ·
<a href="https://onlycrabs.ai">onlycrabs.ai</a> ·
<a href="VISION.md">Vision</a> ·
<a href="docs/clawhub.md">Docs</a> ·
<a href="CONTRIBUTING.md">Contributing</a> ·
<a href="https://discord.gg/clawd">Discord</a>
</p>
## What you can do
## What you can do with it
- Browse skills + render their `SKILL.md`.
- Publish new skill versions with changelogs + tags (including `latest`).
- Rename an owned skill without breaking old links or installs.
- Merge duplicate owned skills into one canonical slug.
- Browse souls + render their `SOUL.md`.
- Publish new soul versions with changelogs + tags.
- Search via embeddings (vector index) instead of brittle keywords.
- Star + comment; admins/mods can curate and approve skills.
- Browse OpenClaw packages with family/trust/capability metadata.
- Publish native code plugins and bundle plugins through `/packages` APIs and CLI flows.
## onlycrabs.ai (SOUL.md registry)
- Entry point is host-based: `onlycrabs.ai`.
- On the onlycrabs.ai host, the home page and nav default to souls.
- On ClawdHub, souls live under `/souls`.
- On ClawHub, souls live under `/souls`.
- Soul bundles only accept `SOUL.md` for now (no extra files).
## How it works (high level)
@ -35,50 +50,76 @@ onlycrabs.ai: `https://onlycrabs.ai`
- Web app: TanStack Start (React, Vite/Nitro).
- Backend: Convex (DB + file storage + HTTP actions) + Convex Auth (GitHub OAuth).
- Search: OpenAI embeddings (`text-embedding-3-small`) + Convex vector search.
- API schema + routes: `packages/schema` (`clawdhub-schema`).
- API schema + routes: `packages/schema` (`clawhub-schema`).
## CLI
Common CLI flows:
- Auth: `clawhub login`, `clawhub whoami`
- Discover: `clawhub search ...`, `clawhub explore`
- Browse unified catalog (skills + plugins): `clawhub package explore`, `clawhub package inspect <name>`
- Manage local installs: `clawhub install <slug>`, `clawhub uninstall <slug>`, `clawhub list`, `clawhub update --all`
- Inspect without installing: `clawhub inspect <slug>`
- Publish/sync skills: `clawhub skill publish <path>`, `clawhub sync`
- Publish plugins: `clawhub package publish <source>`
- Code-plugin manifests must include `openclaw.compat.pluginApi` and `openclaw.build.openclawVersion`; see [`docs/cli.md`](docs/cli.md) for a minimal example.
- Canonicalize owned skills: `clawhub skill rename <slug> <new-slug>`, `clawhub skill merge <source> <target>`
Docs: [`docs/quickstart.md`](docs/quickstart.md), [`docs/cli.md`](docs/cli.md).
### Removal permissions
- `clawhub uninstall <slug>` only removes a local install on your machine.
- Uploaded registry skills use soft-delete/restore (`clawhub delete <slug>` / `clawhub undelete <slug>` or API equivalents).
- Soft-delete/restore is allowed for the skill owner, moderators, and admins.
- Hard delete is admin-only (management tools / ban flows).
- Owner rename keeps the old slug as a redirect alias.
- Owner merge hides the source listing and redirects the old slug to the canonical target.
## Telemetry
ClawdHub tracks minimal **install telemetry** (to compute install counts) when you run `clawdhub sync` while logged in.
ClawHub tracks minimal **install telemetry** (to compute install counts) when you run `clawhub sync` while logged in.
Disable via:
```bash
export CLAWDHUB_DISABLE_TELEMETRY=1
export CLAWHUB_DISABLE_TELEMETRY=1
```
Details: `docs/telemetry.md`.
Details: [`docs/telemetry.md`](docs/telemetry.md).
## Repo layout
- `src/` — TanStack Start app (routes, components, styles).
- `convex/` — schema + queries/mutations/actions + HTTP API routes.
- `packages/schema/` — shared API types/routes for the CLI and app.
- `docs/spec.md` — product + implementation spec (good first read).
- [`docs/`](docs/README.md) — publishable ClawHub public/operator docs for users, publishers, API clients, and deploy operators.
- [`specs/`](specs/README.md) — product specs, plans, regression notes, and design history.
- [`specs/spec.md`](specs/spec.md) — product + implementation spec (good first read for maintainers).
## Local dev
Prereqs: Bun + Convex CLI.
Prereqs: [Bun](https://bun.sh/) (Convex runs via `bunx`, no global install needed).
```bash
bun install
cp .env.local.example .env.local
# edit .env.local — see CONTRIBUTING.md for local Convex values
# terminal A: web app
# terminal A: local Convex backend
bunx convex dev
# terminal B: web app (port 3000)
bun run dev
# terminal B: Convex dev deployment
bunx convex dev
# detached/Codex worktree preview
bun run dev:worktree
# seed sample data
bun run seed:dev
```
## Auth (GitHub OAuth) setup
Create a GitHub OAuth App, set `AUTH_GITHUB_ID` / `AUTH_GITHUB_SECRET`, then:
```bash
bunx auth --deployment-name <deployment> --web-server-url http://localhost:3000
```
This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints values for your local `.env.local`.
For full setup instructions (env vars, GitHub OAuth, JWT keys, database seeding), see [CONTRIBUTING.md](CONTRIBUTING.md).
## Environment
@ -95,7 +136,7 @@ This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints values for y
## Nix plugins (nixmode skills)
ClawdHub can store a nix-clawdbot plugin pointer in SKILL frontmatter so the registry knows which
ClawHub can store a nix-clawdbot plugin pointer in SKILL frontmatter so the registry knows which
Nix package bundle to install. A nix plugin is different from a regular skill pack: it bundles the
skill pack, the CLI binary, and its config flags/requirements together.
@ -105,7 +146,17 @@ Add this to `SKILL.md`:
---
name: peekaboo
description: Capture and automate macOS UI with the Peekaboo CLI.
metadata: {"clawdbot":{"nix":{"plugin":"github:clawdbot/nix-steipete-tools?dir=tools/peekaboo","systems":["aarch64-darwin"]}}}
metadata:
{
"clawdbot":
{
"nix":
{
"plugin": "github:clawdbot/nix-steipete-tools?dir=tools/peekaboo",
"systems": ["aarch64-darwin"],
},
},
}
---
```
@ -123,7 +174,18 @@ You can also declare config requirements + an example snippet:
---
name: padel
description: Check padel court availability and manage bookings via Playtomic.
metadata: {"clawdbot":{"config":{"requiredEnv":["PADEL_AUTH_FILE"],"stateDirs":[".config/padel"],"example":"config = { env = { PADEL_AUTH_FILE = \\\"/run/agenix/padel-auth\\\"; }; };"}}}
metadata:
{
"clawdbot":
{
"config":
{
"requiredEnv": ["PADEL_AUTH_FILE"],
"stateDirs": [".config/padel"],
"example": "config = { env = { PADEL_AUTH_FILE = \\\"/run/agenix/padel-auth\\\"; }; };",
},
},
}
---
```
@ -133,11 +195,34 @@ To show CLI help (recommended for nix plugins), include the `cli --help` output:
---
name: padel
description: Check padel court availability and manage bookings via Playtomic.
metadata: {"clawdbot":{"cliHelp":"padel --help\\nUsage: padel [command]\\n"}}
metadata: { "clawdbot": { "cliHelp": "padel --help\\nUsage: padel [command]\\n" } }
---
```
`metadata.clawdbot` is preferred, but `metadata.clawdis` is accepted as an alias for compatibility.
`metadata.clawdbot` is preferred, but `metadata.clawdis` and `metadata.openclaw` are accepted as aliases.
## Skill metadata
Skills declare their runtime requirements (env vars, binaries, install specs) in the `SKILL.md` frontmatter. ClawHub's security analysis checks these declarations against actual skill behavior.
Full reference: [`docs/skill-format.md`](docs/skill-format.md#frontmatter-metadata)
Quick example:
```yaml
---
name: my-skill
description: Does a thing with an API.
metadata:
openclaw:
requires:
env:
- MY_API_KEY
bins:
- curl
primaryEnv: MY_API_KEY
---
```
## Scripts

98
VISION.md Normal file
View File

@ -0,0 +1,98 @@
## OpenClaw Vision
OpenClaw is the AI that actually does things.
It runs on your devices, in your channels, with your rules.
This document explains the current state and direction of the project.
We are still early, so iteration is fast.
Project overview and developer docs: [`README.md`](README.md)
OpenClaw started as my personal playground to learn AI and build something genuinely useful:
an assistant that can run real tasks on my computer.
It evolved through several names and shells: Warelay -> Clawdbot -> Moltbot -> OpenClaw.
The goal? A personal assistant that's easy to use, supports a wide range of platforms, and respects your privacy and security.
The current focus is:
Priority:
- Security and safe defaults
- Bug fixes and stability
- Setup reliability and first-run UX
Next priorities:
- Supporting all major model providers
- Improving support for major messaging channels (and adding a few high-demand ones)
- Performance and test infrastructure
- Better computer-use and agent harness capabilities
- Ergonomics across CLI and web frontend
- Companion apps on macOS, iOS, Android, Windows, and Linux
## Security
Security in OpenClaw is a deliberate tradeoff: strong defaults without killing capability.
The goal is to stay powerful for real work while making risky paths explicit and operator-controlled.
Canonical security policy and reporting:
- https://github.com/openclaw/openclaw/blob/main/SECURITY.md
We prioritize secure defaults, but we also expose clear knobs for trusted high-power workflows.
## Plugins & Memory
OpenClaw has an extensive plugin API.
Core stays lean; optional capability should usually ship as plugins.
Preferred plugin path is npm package distribution plus local extension loading for development.
If you build a plugin, please host and maintain it in your own repository.
The bar for adding optional plugins to core is intentionally high.
Memory is a special plugin slot where only one memory plugin can be active at a time.
Today we ship multiple memory options; over time we plan to converge on one recommended default path.
### Skills
We still ship some bundled skills for baseline UX.
New skills should be published to ClawHub first (`clawhub.ai`), not added to core by default.
Core skill additions should be rare and require a strong product or security reason.
### MCP Support
OpenClaw supports MCP through `mcporter`: https://github.com/steipete/mcporter
This keeps MCP integration flexible and decoupled from core runtime:
- add or change MCP servers without restarting the gateway
- keep core tool/context surface lean
- reduce MCP churn impact on core stability and security
For now, we prefer this bridge model over building first-class MCP runtime into core.
If there is an MCP server or feature `mcporter` does not support yet, please open an issue there.
### Setup
OpenClaw is currently terminal-first by design.
This keeps setup explicit: users see docs, auth, permissions, and security posture up front.
Long term, we want easier onboarding flows as hardening matures.
We do not want convenience wrappers that hide critical security decisions from users.
### Why TypeScript?
OpenClaw is primarily an orchestration system: prompts, tools, protocols, and integrations.
TypeScript was chosen to keep OpenClaw hackable by default.
It is widely known, fast to iterate in, and easy to read, modify, and extend.
## What We Will Not Merge (For Now)
- New core skills when they can live on ClawHub
- Commercial service integrations that do not clearly fit the model-provider category
- Wrapper channels around already supported channels without a clear capability or security gap
- First-class MCP runtime in core when `mcporter` already provides the integration path
- Heavy orchestration layers that duplicate existing agent and tool infrastructure
This list is a roadmap guardrail, not a law of physics.
Strong user demand and strong technical rationale can change it.

View File

@ -1,41 +0,0 @@
{
"$schema": "https://biomejs.dev/schemas/2.3.11/schema.json",
"files": {
"includes": [
"**",
"!**/.cta.json",
"!**/.vscode",
"!**/node_modules",
"!**/dist",
"!**/.output",
"!**/coverage",
"!**/convex/_generated",
"!**/test-results",
"!**/src/routeTree.gen.ts",
"!**/.tanstack",
"!**/public",
"!**/.devenv",
"!**/.devenv"
]
},
"assist": { "actions": { "source": { "organizeImports": "on" } } },
"formatter": {
"enabled": true,
"indentStyle": "space",
"indentWidth": 2,
"lineWidth": 100
},
"linter": {
"enabled": true,
"rules": {
"recommended": true
}
},
"javascript": {
"formatter": {
"quoteStyle": "single",
"semicolons": "asNeeded",
"trailingCommas": "all"
}
}
}

1110
bun.lock

File diff suppressed because it is too large Load Diff

View File

@ -3,9 +3,10 @@ import { existsSync } from 'node:fs'
import { stat } from 'node:fs/promises'
import { fileURLToPath } from 'node:url'
const distCliUrl = new URL('./packages/clawdhub/dist/cli.js', import.meta.url)
const packageRootPath = fileURLToPath(new URL('./packages/clawhub/', import.meta.url))
const distCliUrl = new URL('./packages/clawhub/dist/cli.js', import.meta.url)
const distCliPath = fileURLToPath(distCliUrl)
const srcRootPath = fileURLToPath(new URL('./packages/clawdhub/src/', import.meta.url))
const srcRootPath = fileURLToPath(new URL('./packages/clawhub/src/', import.meta.url))
const shouldBuild = await (async () => {
if (!existsSync(distCliPath)) return true
@ -19,7 +20,8 @@ const shouldBuild = await (async () => {
})()
if (shouldBuild) {
const proc = Bun.spawn(['bunx', 'tsc', '-p', 'packages/clawdhub/tsconfig.json'], {
const proc = Bun.spawn(['bun', 'run', 'build'], {
cwd: packageRootPath,
stdin: 'inherit',
stdout: 'inherit',
stderr: 'inherit',
@ -34,6 +36,7 @@ async function getLatestMtime(root: string) {
let latest = 0
const glob = new Bun.Glob('**/*.ts')
for await (const rel of glob.scan({ cwd: root, onlyFiles: true })) {
if (rel.endsWith('.test.ts')) continue
const path = `${root}${root.endsWith('/') ? '' : '/'}${rel}`
try {
const entry = await stat(path)

49
clawhub Executable file
View File

@ -0,0 +1,49 @@
#!/usr/bin/env bun
import { existsSync } from 'node:fs'
import { stat } from 'node:fs/promises'
import { fileURLToPath } from 'node:url'
const packageRootPath = fileURLToPath(new URL('./packages/clawhub/', import.meta.url))
const distCliUrl = new URL('./packages/clawhub/dist/cli.js', import.meta.url)
const distCliPath = fileURLToPath(distCliUrl)
const srcRootPath = fileURLToPath(new URL('./packages/clawhub/src/', import.meta.url))
const shouldBuild = await (async () => {
if (!existsSync(distCliPath)) return true
try {
const dist = await stat(distCliPath)
const latestSrcMtime = await getLatestMtime(srcRootPath)
return latestSrcMtime > dist.mtimeMs
} catch {
return true
}
})()
if (shouldBuild) {
const proc = Bun.spawn(['bun', 'run', 'build'], {
cwd: packageRootPath,
stdin: 'inherit',
stdout: 'inherit',
stderr: 'inherit',
})
const code = await proc.exited
if (code !== 0) process.exit(code)
}
await import(distCliUrl.href)
async function getLatestMtime(root: string) {
let latest = 0
const glob = new Bun.Glob('**/*.ts')
for await (const rel of glob.scan({ cwd: root, onlyFiles: true })) {
if (rel.endsWith('.test.ts')) continue
const path = `${root}${root.endsWith('/') ? '' : '/'}${rel}`
try {
const entry = await stat(path)
latest = Math.max(latest, entry.mtimeMs)
} catch {
// ignore
}
}
return latest
}

21
components.json Normal file
View File

@ -0,0 +1,21 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "new-york",
"rsc": false,
"tsx": true,
"tailwind": {
"config": "",
"css": "src/styles.css",
"baseColor": "neutral",
"cssVariables": true,
"prefix": ""
},
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"iconLibrary": "lucide"
}

View File

@ -0,0 +1,14 @@
{
"guidelinesHash": "62d72acb9afcc18f658d88dd772f34b5b1da5fa60ef0402e57a784d97c458e57",
"agentsMdSectionHash": "bbf30bd25ceea0aefd279d62e1cb2b4c207fcb712b69adf26f3d02b296ffc7b2",
"claudeMdHash": "bbf30bd25ceea0aefd279d62e1cb2b4c207fcb712b69adf26f3d02b296ffc7b2",
"agentSkillsSha": "d0fa8085af313029add5740f67198aa42ca60c8d",
"installedSkillNames": [
"convex",
"convex-create-component",
"convex-migration-helper",
"convex-performance-audit",
"convex-quickstart",
"convex-setup-auth"
]
}

View File

@ -0,0 +1,365 @@
# Convex guidelines
## Function guidelines
### Http endpoint syntax
- HTTP endpoints are defined in `convex/http.ts` and require an `httpAction` decorator. For example:
```typescript
import { httpRouter } from "convex/server";
import { httpAction } from "./_generated/server";
const http = httpRouter();
http.route({
path: "/echo",
method: "POST",
handler: httpAction(async (ctx, req) => {
const body = await req.bytes();
return new Response(body, { status: 200 });
}),
});
```
- HTTP endpoints are always registered at the exact path you specify in the `path` field. For example, if you specify `/api/someRoute`, the endpoint will be registered at `/api/someRoute`.
### Validators
- Below is an example of an array validator:
```typescript
import { mutation } from "./_generated/server";
import { v } from "convex/values";
export default mutation({
args: {
simpleArray: v.array(v.union(v.string(), v.number())),
},
handler: async (ctx, args) => {
//...
},
});
```
- Below is an example of a schema with validators that codify a discriminated union type:
```typescript
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
results: defineTable(
v.union(
v.object({
kind: v.literal("error"),
errorMessage: v.string(),
}),
v.object({
kind: v.literal("success"),
value: v.number(),
}),
),
),
});
```
- Here are the valid Convex types along with their respective validators:
Convex Type | TS/JS type | Example Usage | Validator for argument validation and schemas | Notes |
| ----------- | ------------| -----------------------| -----------------------------------------------| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Id | string | `doc._id` | `v.id(tableName)` | |
| Null | null | `null` | `v.null()` | JavaScript's `undefined` is not a valid Convex value. Functions the return `undefined` or do not return will return `null` when called from a client. Use `null` instead. |
| Int64 | bigint | `3n` | `v.int64()` | Int64s only support BigInts between -2^63 and 2^63-1. Convex supports `bigint`s in most modern browsers. |
| Float64 | number | `3.1` | `v.number()` | Convex supports all IEEE-754 double-precision floating point numbers (such as NaNs). Inf and NaN are JSON serialized as strings. |
| Boolean | boolean | `true` | `v.boolean()` |
| String | string | `"abc"` | `v.string()` | Strings are stored as UTF-8 and must be valid Unicode sequences. Strings must be smaller than the 1MB total size limit when encoded as UTF-8. |
| Bytes | ArrayBuffer | `new ArrayBuffer(8)` | `v.bytes()` | Convex supports first class bytestrings, passed in as `ArrayBuffer`s. Bytestrings must be smaller than the 1MB total size limit for Convex types. |
| Array | Array | `[1, 3.2, "abc"]` | `v.array(values)` | Arrays can have at most 8192 values. |
| Object | Object | `{a: "abc"}` | `v.object({property: value})` | Convex only supports "plain old JavaScript objects" (objects that do not have a custom prototype). Objects can have at most 1024 entries. Field names must be nonempty and not start with "$" or "_". |
| Record | Record | `{"a": "1", "b": "2"}` | `v.record(keys, values)` | Records are objects at runtime, but can have dynamic keys. Keys must be only ASCII characters, nonempty, and not start with "$" or "\_". |
### Function registration
- Use `internalQuery`, `internalMutation`, and `internalAction` to register internal functions. These functions are private and aren't part of an app's API. They can only be called by other Convex functions. These functions are always imported from `./_generated/server`.
- Use `query`, `mutation`, and `action` to register public functions. These functions are part of the public API and are exposed to the public Internet. Do NOT use `query`, `mutation`, or `action` to register sensitive internal functions that should be kept private.
- You CANNOT register a function through the `api` or `internal` objects.
- ALWAYS include argument validators for all Convex functions. This includes all of `query`, `internalQuery`, `mutation`, `internalMutation`, `action`, and `internalAction`.
### Function calling
- Use `ctx.runQuery` to call a query from a query, mutation, or action.
- Use `ctx.runMutation` to call a mutation from a mutation or action.
- Use `ctx.runAction` to call an action from an action.
- ONLY call an action from another action if you need to cross runtimes (e.g. from V8 to Node). Otherwise, pull out the shared code into a helper async function and call that directly instead.
- Try to use as few calls from actions to queries and mutations as possible. Queries and mutations are transactions, so splitting logic up into multiple calls introduces the risk of race conditions.
- All of these calls take in a `FunctionReference`. Do NOT try to pass the callee function directly into one of these calls.
- When using `ctx.runQuery`, `ctx.runMutation`, or `ctx.runAction` to call a function in the same file, specify a type annotation on the return value to work around TypeScript circularity limitations. For example,
```
export const f = query({
args: { name: v.string() },
handler: async (ctx, args) => {
return "Hello " + args.name;
},
});
export const g = query({
args: {},
handler: async (ctx, args) => {
const result: string = await ctx.runQuery(api.example.f, { name: "Bob" });
return null;
},
});
```
### Function references
- Use the `api` object defined by the framework in `convex/_generated/api.ts` to call public functions registered with `query`, `mutation`, or `action`.
- Use the `internal` object defined by the framework in `convex/_generated/api.ts` to call internal (or private) functions registered with `internalQuery`, `internalMutation`, or `internalAction`.
- Convex uses file-based routing, so a public function defined in `convex/example.ts` named `f` has a function reference of `api.example.f`.
- A private function defined in `convex/example.ts` named `g` has a function reference of `internal.example.g`.
- Functions can also registered within directories nested within the `convex/` folder. For example, a public function `h` defined in `convex/messages/access.ts` has a function reference of `api.messages.access.h`.
### Pagination
- Define pagination using the following syntax:
```ts
import { v } from "convex/values";
import { query, mutation } from "./_generated/server";
import { paginationOptsValidator } from "convex/server";
export const listWithExtraArg = query({
args: { paginationOpts: paginationOptsValidator, author: v.string() },
handler: async (ctx, args) => {
return await ctx.db
.query("messages")
.withIndex("by_author", (q) => q.eq("author", args.author))
.order("desc")
.paginate(args.paginationOpts);
},
});
```
Note: `paginationOpts` is an object with the following properties:
- `numItems`: the maximum number of documents to return (the validator is `v.number()`)
- `cursor`: the cursor to use to fetch the next page of documents (the validator is `v.union(v.string(), v.null())`)
- A query that ends in `.paginate()` returns an object that has the following properties:
- page (contains an array of documents that you fetches)
- isDone (a boolean that represents whether or not this is the last page of documents)
- continueCursor (a string that represents the cursor to use to fetch the next page of documents)
## Schema guidelines
- Always define your schema in `convex/schema.ts`.
- Always import the schema definition functions from `convex/server`.
- System fields are automatically added to all documents and are prefixed with an underscore. The two system fields that are automatically added to all documents are `_creationTime` which has the validator `v.number()` and `_id` which has the validator `v.id(tableName)`.
- Always include all index fields in the index name. For example, if an index is defined as `["field1", "field2"]`, the index name should be "by_field1_and_field2".
- Index fields must be queried in the same order they are defined. If you want to be able to query by "field1" then "field2" and by "field2" then "field1", you must create separate indexes.
- Do not store unbounded lists as an array field inside a document (e.g. `v.array(v.object({...}))`). As the array grows it will hit the 1MB document size limit, and every update rewrites the entire document. Instead, create a separate table for the child items with a foreign key back to the parent.
- Separate high-churn operational data (e.g. heartbeats, online status, typing indicators) from stable profile data. Storing frequently updated fields on a shared document forces every write to contend with reads of the entire document. Instead, create a dedicated table for the high-churn data with a foreign key back to the parent record.
## Authentication guidelines
- Convex supports JWT-based authentication through `convex/auth.config.ts`. ALWAYS create this file when using authentication. Without it, `ctx.auth.getUserIdentity()` will always return `null`.
- Example `convex/auth.config.ts`:
```typescript
export default {
providers: [
{
domain: "https://your-auth-provider.com",
applicationID: "convex",
},
],
};
```
The `domain` must be the issuer URL of the JWT provider. Convex fetches `{domain}/.well-known/openid-configuration` to discover the JWKS endpoint. The `applicationID` is checked against the JWT `aud` (audience) claim.
- Use `ctx.auth.getUserIdentity()` to get the authenticated user's identity in any query, mutation, or action. This returns `null` if the user is not authenticated, or a `UserIdentity` object with fields like `subject`, `issuer`, `name`, `email`, etc. The `subject` field is the unique user identifier.
- In Convex `UserIdentity`, `tokenIdentifier` is guaranteed and is the canonical stable identifier for the authenticated identity. For any auth-linked database lookup or ownership check, prefer `identity.tokenIdentifier` over `identity.subject`. Do NOT use `identity.subject` alone as a global identity key.
- NEVER accept a `userId` or any user identifier as a function argument for authorization purposes. Always derive the user identity server-side via `ctx.auth.getUserIdentity()`.
- When using an external auth provider with Convex on the client, use `ConvexProviderWithAuth` instead of `ConvexProvider`:
```tsx
import { ConvexProviderWithAuth, ConvexReactClient } from "convex/react";
const convex = new ConvexReactClient(process.env.NEXT_PUBLIC_CONVEX_URL!);
function App({ children }: { children: React.ReactNode }) {
return (
<ConvexProviderWithAuth client={convex} useAuth={useYourAuthHook}>
{children}
</ConvexProviderWithAuth>
);
}
```
The `useAuth` prop must return `{ isLoading, isAuthenticated, fetchAccessToken }`. Do NOT use plain `ConvexProvider` when authentication is needed — it will not send tokens with requests.
## Typescript guidelines
- You can use the helper typescript type `Id` imported from './\_generated/dataModel' to get the type of the id for a given table. For example if there is a table called 'users' you can use `Id<'users'>` to get the type of the id for that table.
- Use `Doc<"tableName">` from `./_generated/dataModel` to get the full document type for a table.
- Use `QueryCtx`, `MutationCtx`, `ActionCtx` from `./_generated/server` for typing function contexts. NEVER use `any` for ctx parameters — always use the proper context type.
- If you need to define a `Record` make sure that you correctly provide the type of the key and value in the type. For example a validator `v.record(v.id('users'), v.string())` would have the type `Record<Id<'users'>, string>`. Below is an example of using `Record` with an `Id` type in a query:
```ts
import { query } from "./_generated/server";
import { Doc, Id } from "./_generated/dataModel";
export const exampleQuery = query({
args: { userIds: v.array(v.id("users")) },
handler: async (ctx, args) => {
const idToUsername: Record<Id<"users">, string> = {};
for (const userId of args.userIds) {
const user = await ctx.db.get("users", userId);
if (user) {
idToUsername[user._id] = user.username;
}
}
return idToUsername;
},
});
```
- Be strict with types, particularly around id's of documents. For example, if a function takes in an id for a document in the 'users' table, take in `Id<'users'>` rather than `string`.
## Full text search guidelines
- A query for "10 messages in channel '#general' that best match the query 'hello hi' in their body" would look like:
const messages = await ctx.db
.query("messages")
.withSearchIndex("search_body", (q) =>
q.search("body", "hello hi").eq("channel", "#general"),
)
.take(10);
## Query guidelines
- Do NOT use `filter` in queries. Instead, define an index in the schema and use `withIndex` instead.
- If the user does not explicitly tell you to return all results from a query you should ALWAYS return a bounded collection instead. So that is instead of using `.collect()` you should use `.take()` or paginate on database queries. This prevents future performance issues when tables grow in an unbounded way.
- Never use `.collect().length` to count rows. Convex has no built-in count operator, so if you need a count that stays efficient at scale, maintain a denormalized counter in a separate document and update it in your mutations.
- Convex queries do NOT support `.delete()`. If you need to delete all documents matching a query, use `.take(n)` to read them in batches, iterate over each batch calling `ctx.db.delete(row._id)`, and repeat until no more results are returned.
- Convex mutations are transactions with limits on the number of documents read and written. If a mutation needs to process more documents than fit in a single transaction (e.g. bulk deletion on a large table), process a batch with `.take(n)` and then call `ctx.scheduler.runAfter(0, api.myModule.myMutation, args)` to schedule itself to continue. This way each invocation stays within transaction limits.
- Use `.unique()` to get a single document from a query. This method will throw an error if there are multiple documents that match the query.
- When using async iteration, don't use `.collect()` or `.take(n)` on the result of a query. Instead, use the `for await (const row of query)` syntax.
### Ordering
- By default Convex always returns documents in ascending `_creationTime` order.
- You can use `.order('asc')` or `.order('desc')` to pick whether a query is in ascending or descending order. If the order isn't specified, it defaults to ascending.
- Document queries that use indexes will be ordered based on the columns in the index and can avoid slow table scans.
## Mutation guidelines
- Use `ctx.db.replace` to fully replace an existing document. This method will throw an error if the document does not exist. Syntax: `await ctx.db.replace('tasks', taskId, { name: 'Buy milk', completed: false })`
- Use `ctx.db.patch` to shallow merge updates into an existing document. This method will throw an error if the document does not exist. Syntax: `await ctx.db.patch('tasks', taskId, { completed: true })`
## Action guidelines
- Always add `"use node";` to the top of files containing actions that use Node.js built-in modules.
- Never add `"use node";` to a file that also exports queries or mutations. Only actions can run in the Node.js runtime; queries and mutations must stay in the default Convex runtime. If you need Node.js built-ins alongside queries or mutations, put the action in a separate file.
- `fetch()` is available in the default Convex runtime. You do NOT need `"use node";` just to use `fetch()`.
- Never use `ctx.db` inside of an action. Actions don't have access to the database.
- Below is an example of the syntax for an action:
```ts
import { action } from "./_generated/server";
export const exampleAction = action({
args: {},
handler: async (ctx, args) => {
console.log("This action does not return anything");
return null;
},
});
```
## Scheduling guidelines
### Cron guidelines
- Only use the `crons.interval` or `crons.cron` methods to schedule cron jobs. Do NOT use the `crons.hourly`, `crons.daily`, or `crons.weekly` helpers.
- Both cron methods take in a FunctionReference. Do NOT try to pass the function directly into one of these methods.
- Define crons by declaring the top-level `crons` object, calling some methods on it, and then exporting it as default. For example,
```ts
import { cronJobs } from "convex/server";
import { internal } from "./_generated/api";
import { internalAction } from "./_generated/server";
const empty = internalAction({
args: {},
handler: async (ctx, args) => {
console.log("empty");
},
});
const crons = cronJobs();
// Run `internal.crons.empty` every two hours.
crons.interval("delete inactive users", { hours: 2 }, internal.crons.empty, {});
export default crons;
```
- You can register Convex functions within `crons.ts` just like any other file.
- If a cron calls an internal function, always import the `internal` object from '\_generated/api', even if the internal function is registered in the same file.
## Testing guidelines
- Use `convex-test` with `vitest` and `@edge-runtime/vm` to test Convex functions. Always install the latest versions of these packages. Configure vitest with `environment: "edge-runtime"` in `vitest.config.ts`.
Test files go inside the `convex/` directory. You must pass a module map from `import.meta.glob` to `convexTest`:
```typescript
/// <reference types="vite/client" />
import { convexTest } from "convex-test";
import { expect, test } from "vitest";
import { api } from "./_generated/api";
import schema from "./schema";
const modules = import.meta.glob("./**/*.ts");
test("some behavior", async () => {
const t = convexTest(schema, modules);
await t.mutation(api.messages.send, { body: "Hi!", author: "Sarah" });
const messages = await t.query(api.messages.list);
expect(messages).toMatchObject([{ body: "Hi!", author: "Sarah" }]);
});
```
The `modules` argument is required so convex-test can discover and load function files. The `/// <reference types="vite/client" />` directive is needed for TypeScript to recognize `import.meta.glob`.
## File storage guidelines
- The `ctx.storage.getUrl()` method returns a signed URL for a given file. It returns `null` if the file doesn't exist.
- Do NOT use the deprecated `ctx.storage.getMetadata` call for loading a file's metadata.
Instead, query the `_storage` system table. For example, you can use `ctx.db.system.get` to get an `Id<"_storage">`.
```
import { query } from "./_generated/server";
import { Id } from "./_generated/dataModel";
type FileMetadata = {
_id: Id<"_storage">;
_creationTime: number;
contentType?: string;
sha256: string;
size: number;
}
export const exampleQuery = query({
args: { fileId: v.id("_storage") },
handler: async (ctx, args) => {
const metadata: FileMetadata | null = await ctx.db.system.get("_storage", args.fileId);
console.log(metadata);
return null;
},
});
```
- Convex storage stores items as `Blob` objects. You must convert all items to/from a `Blob` when using Convex storage.

View File

@ -8,42 +8,113 @@
* @module
*/
import type * as appMeta from "../appMeta.js";
import type * as auth from "../auth.js";
import type * as commentModeration from "../commentModeration.js";
import type * as comments from "../comments.js";
import type * as crons from "../crons.js";
import type * as depRegistryScan from "../depRegistryScan.js";
import type * as devSeed from "../devSeed.js";
import type * as devSeedExtra from "../devSeedExtra.js";
import type * as downloads from "../downloads.js";
import type * as functions from "../functions.js";
import type * as githubBackups from "../githubBackups.js";
import type * as githubBackupsNode from "../githubBackupsNode.js";
import type * as githubIdentity from "../githubIdentity.js";
import type * as githubImport from "../githubImport.js";
import type * as githubRestore from "../githubRestore.js";
import type * as githubRestoreMutations from "../githubRestoreMutations.js";
import type * as githubSoulBackups from "../githubSoulBackups.js";
import type * as githubSoulBackupsNode from "../githubSoulBackupsNode.js";
import type * as http from "../http.js";
import type * as httpApi from "../httpApi.js";
import type * as httpApiV1 from "../httpApiV1.js";
import type * as httpApiV1_docsSessionV1 from "../httpApiV1/docsSessionV1.js";
import type * as httpApiV1_packagesV1 from "../httpApiV1/packagesV1.js";
import type * as httpApiV1_shared from "../httpApiV1/shared.js";
import type * as httpApiV1_skillsV1 from "../httpApiV1/skillsV1.js";
import type * as httpApiV1_soulsV1 from "../httpApiV1/soulsV1.js";
import type * as httpApiV1_starsV1 from "../httpApiV1/starsV1.js";
import type * as httpApiV1_transfersV1 from "../httpApiV1/transfersV1.js";
import type * as httpApiV1_usersV1 from "../httpApiV1/usersV1.js";
import type * as httpApiV1_whoamiV1 from "../httpApiV1/whoamiV1.js";
import type * as httpPreflight from "../httpPreflight.js";
import type * as leaderboards from "../leaderboards.js";
import type * as lib_access from "../lib/access.js";
import type * as lib_apiTokenAuth from "../lib/apiTokenAuth.js";
import type * as lib_artifactModeration from "../lib/artifactModeration.js";
import type * as lib_badges from "../lib/badges.js";
import type * as lib_batching from "../lib/batching.js";
import type * as lib_changelog from "../lib/changelog.js";
import type * as lib_clawpack from "../lib/clawpack.js";
import type * as lib_commentScamPrompt from "../lib/commentScamPrompt.js";
import type * as lib_contentTypes from "../lib/contentTypes.js";
import type * as lib_depRegistryScan from "../lib/depRegistryScan.js";
import type * as lib_embeddingVisibility from "../lib/embeddingVisibility.js";
import type * as lib_embeddings from "../lib/embeddings.js";
import type * as lib_githubAccount from "../lib/githubAccount.js";
import type * as lib_githubActionsOidc from "../lib/githubActionsOidc.js";
import type * as lib_githubBackup from "../lib/githubBackup.js";
import type * as lib_githubIdentity from "../lib/githubIdentity.js";
import type * as lib_githubImport from "../lib/githubImport.js";
import type * as lib_githubProfileSync from "../lib/githubProfileSync.js";
import type * as lib_githubRestoreHelpers from "../lib/githubRestoreHelpers.js";
import type * as lib_githubSoulBackup from "../lib/githubSoulBackup.js";
import type * as lib_globalStats from "../lib/globalStats.js";
import type * as lib_httpHeaders from "../lib/httpHeaders.js";
import type * as lib_httpRateLimit from "../lib/httpRateLimit.js";
import type * as lib_httpUtils from "../lib/httpUtils.js";
import type * as lib_leaderboards from "../lib/leaderboards.js";
import type * as lib_manualOverrides from "../lib/manualOverrides.js";
import type * as lib_moderation from "../lib/moderation.js";
import type * as lib_moderationEngine from "../lib/moderationEngine.js";
import type * as lib_moderationReasonCodes from "../lib/moderationReasonCodes.js";
import type * as lib_openaiResponse from "../lib/openaiResponse.js";
import type * as lib_packageRegistry from "../lib/packageRegistry.js";
import type * as lib_packageSearchDigest from "../lib/packageSearchDigest.js";
import type * as lib_packageSecurity from "../lib/packageSecurity.js";
import type * as lib_public from "../lib/public.js";
import type * as lib_publishLimits from "../lib/publishLimits.js";
import type * as lib_publishers from "../lib/publishers.js";
import type * as lib_reporting from "../lib/reporting.js";
import type * as lib_reservedHandles from "../lib/reservedHandles.js";
import type * as lib_reservedSlugs from "../lib/reservedSlugs.js";
import type * as lib_searchText from "../lib/searchText.js";
import type * as lib_securityPrompt from "../lib/securityPrompt.js";
import type * as lib_skillBackfill from "../lib/skillBackfill.js";
import type * as lib_skillCapabilityTags from "../lib/skillCapabilityTags.js";
import type * as lib_skillPublish from "../lib/skillPublish.js";
import type * as lib_skillQuality from "../lib/skillQuality.js";
import type * as lib_skillSafety from "../lib/skillSafety.js";
import type * as lib_skillSearchDigest from "../lib/skillSearchDigest.js";
import type * as lib_skillStats from "../lib/skillStats.js";
import type * as lib_skillSummary from "../lib/skillSummary.js";
import type * as lib_skillZip from "../lib/skillZip.js";
import type * as lib_skills from "../lib/skills.js";
import type * as lib_soulChangelog from "../lib/soulChangelog.js";
import type * as lib_soulPublish from "../lib/soulPublish.js";
import type * as lib_staticPublishScan from "../lib/staticPublishScan.js";
import type * as lib_tokens from "../lib/tokens.js";
import type * as lib_userSearch from "../lib/userSearch.js";
import type * as lib_userSkillStats from "../lib/userSkillStats.js";
import type * as lib_webhooks from "../lib/webhooks.js";
import type * as llmEval from "../llmEval.js";
import type * as maintenance from "../maintenance.js";
import type * as model_packages_rescans from "../model/packages/rescans.js";
import type * as model_rescans_policy from "../model/rescans/policy.js";
import type * as model_skills_rescans from "../model/skills/rescans.js";
import type * as packagePublishTokens from "../packagePublishTokens.js";
import type * as packages from "../packages.js";
import type * as publishers from "../publishers.js";
import type * as rateLimits from "../rateLimits.js";
import type * as rescanRequests from "../rescanRequests.js";
import type * as search from "../search.js";
import type * as securityDataset from "../securityDataset.js";
import type * as securityDatasetNode from "../securityDatasetNode.js";
import type * as seed from "../seed.js";
import type * as seedSouls from "../seedSouls.js";
import type * as skillStatEvents from "../skillStatEvents.js";
import type * as skillTransfers from "../skillTransfers.js";
import type * as skills from "../skills.js";
import type * as soulComments from "../soulComments.js";
import type * as soulDownloads from "../soulDownloads.js";
@ -55,6 +126,7 @@ import type * as telemetry from "../telemetry.js";
import type * as tokens from "../tokens.js";
import type * as uploads from "../uploads.js";
import type * as users from "../users.js";
import type * as vt from "../vt.js";
import type * as webhooks from "../webhooks.js";
import type {
@ -64,42 +136,113 @@ import type {
} from "convex/server";
declare const fullApi: ApiFromModules<{
appMeta: typeof appMeta;
auth: typeof auth;
commentModeration: typeof commentModeration;
comments: typeof comments;
crons: typeof crons;
depRegistryScan: typeof depRegistryScan;
devSeed: typeof devSeed;
devSeedExtra: typeof devSeedExtra;
downloads: typeof downloads;
functions: typeof functions;
githubBackups: typeof githubBackups;
githubBackupsNode: typeof githubBackupsNode;
githubIdentity: typeof githubIdentity;
githubImport: typeof githubImport;
githubRestore: typeof githubRestore;
githubRestoreMutations: typeof githubRestoreMutations;
githubSoulBackups: typeof githubSoulBackups;
githubSoulBackupsNode: typeof githubSoulBackupsNode;
http: typeof http;
httpApi: typeof httpApi;
httpApiV1: typeof httpApiV1;
"httpApiV1/docsSessionV1": typeof httpApiV1_docsSessionV1;
"httpApiV1/packagesV1": typeof httpApiV1_packagesV1;
"httpApiV1/shared": typeof httpApiV1_shared;
"httpApiV1/skillsV1": typeof httpApiV1_skillsV1;
"httpApiV1/soulsV1": typeof httpApiV1_soulsV1;
"httpApiV1/starsV1": typeof httpApiV1_starsV1;
"httpApiV1/transfersV1": typeof httpApiV1_transfersV1;
"httpApiV1/usersV1": typeof httpApiV1_usersV1;
"httpApiV1/whoamiV1": typeof httpApiV1_whoamiV1;
httpPreflight: typeof httpPreflight;
leaderboards: typeof leaderboards;
"lib/access": typeof lib_access;
"lib/apiTokenAuth": typeof lib_apiTokenAuth;
"lib/artifactModeration": typeof lib_artifactModeration;
"lib/badges": typeof lib_badges;
"lib/batching": typeof lib_batching;
"lib/changelog": typeof lib_changelog;
"lib/clawpack": typeof lib_clawpack;
"lib/commentScamPrompt": typeof lib_commentScamPrompt;
"lib/contentTypes": typeof lib_contentTypes;
"lib/depRegistryScan": typeof lib_depRegistryScan;
"lib/embeddingVisibility": typeof lib_embeddingVisibility;
"lib/embeddings": typeof lib_embeddings;
"lib/githubAccount": typeof lib_githubAccount;
"lib/githubActionsOidc": typeof lib_githubActionsOidc;
"lib/githubBackup": typeof lib_githubBackup;
"lib/githubIdentity": typeof lib_githubIdentity;
"lib/githubImport": typeof lib_githubImport;
"lib/githubProfileSync": typeof lib_githubProfileSync;
"lib/githubRestoreHelpers": typeof lib_githubRestoreHelpers;
"lib/githubSoulBackup": typeof lib_githubSoulBackup;
"lib/globalStats": typeof lib_globalStats;
"lib/httpHeaders": typeof lib_httpHeaders;
"lib/httpRateLimit": typeof lib_httpRateLimit;
"lib/httpUtils": typeof lib_httpUtils;
"lib/leaderboards": typeof lib_leaderboards;
"lib/manualOverrides": typeof lib_manualOverrides;
"lib/moderation": typeof lib_moderation;
"lib/moderationEngine": typeof lib_moderationEngine;
"lib/moderationReasonCodes": typeof lib_moderationReasonCodes;
"lib/openaiResponse": typeof lib_openaiResponse;
"lib/packageRegistry": typeof lib_packageRegistry;
"lib/packageSearchDigest": typeof lib_packageSearchDigest;
"lib/packageSecurity": typeof lib_packageSecurity;
"lib/public": typeof lib_public;
"lib/publishLimits": typeof lib_publishLimits;
"lib/publishers": typeof lib_publishers;
"lib/reporting": typeof lib_reporting;
"lib/reservedHandles": typeof lib_reservedHandles;
"lib/reservedSlugs": typeof lib_reservedSlugs;
"lib/searchText": typeof lib_searchText;
"lib/securityPrompt": typeof lib_securityPrompt;
"lib/skillBackfill": typeof lib_skillBackfill;
"lib/skillCapabilityTags": typeof lib_skillCapabilityTags;
"lib/skillPublish": typeof lib_skillPublish;
"lib/skillQuality": typeof lib_skillQuality;
"lib/skillSafety": typeof lib_skillSafety;
"lib/skillSearchDigest": typeof lib_skillSearchDigest;
"lib/skillStats": typeof lib_skillStats;
"lib/skillSummary": typeof lib_skillSummary;
"lib/skillZip": typeof lib_skillZip;
"lib/skills": typeof lib_skills;
"lib/soulChangelog": typeof lib_soulChangelog;
"lib/soulPublish": typeof lib_soulPublish;
"lib/staticPublishScan": typeof lib_staticPublishScan;
"lib/tokens": typeof lib_tokens;
"lib/userSearch": typeof lib_userSearch;
"lib/userSkillStats": typeof lib_userSkillStats;
"lib/webhooks": typeof lib_webhooks;
llmEval: typeof llmEval;
maintenance: typeof maintenance;
"model/packages/rescans": typeof model_packages_rescans;
"model/rescans/policy": typeof model_rescans_policy;
"model/skills/rescans": typeof model_skills_rescans;
packagePublishTokens: typeof packagePublishTokens;
packages: typeof packages;
publishers: typeof publishers;
rateLimits: typeof rateLimits;
rescanRequests: typeof rescanRequests;
search: typeof search;
securityDataset: typeof securityDataset;
securityDatasetNode: typeof securityDatasetNode;
seed: typeof seed;
seedSouls: typeof seedSouls;
skillStatEvents: typeof skillStatEvents;
skillTransfers: typeof skillTransfers;
skills: typeof skills;
soulComments: typeof soulComments;
soulDownloads: typeof soulDownloads;
@ -111,6 +254,7 @@ declare const fullApi: ApiFromModules<{
tokens: typeof tokens;
uploads: typeof uploads;
users: typeof users;
vt: typeof vt;
webhooks: typeof webhooks;
}>;

View File

@ -0,0 +1,7 @@
import { internal } from "./_generated/api";
// Asserts that the internal-only download counters remain internal-only.
// Public exposure is prevented at runtime by `internalMutation`; this file
// just pins the public references that *should* exist.
void internal.downloads.recordDownloadInternal;
void internal.soulDownloads.incrementInternal;

14
convex/appMeta.ts Normal file
View File

@ -0,0 +1,14 @@
import { query } from "./functions";
function normalizeEnv(value: string | undefined) {
const normalized = value?.trim();
return normalized ? normalized : null;
}
export const getDeploymentInfo = query({
args: {},
handler: async () => ({
appBuildSha: normalizeEnv(process.env.APP_BUILD_SHA),
deployedAt: normalizeEnv(process.env.APP_DEPLOYED_AT),
}),
});

View File

@ -2,7 +2,7 @@ export default {
providers: [
{
domain: process.env.CONVEX_SITE_URL,
applicationID: 'convex',
applicationID: "convex",
},
],
}
};

141
convex/auth.test.ts Normal file
View File

@ -0,0 +1,141 @@
import { describe, expect, it, vi } from "vitest";
import type { Id } from "./_generated/dataModel";
import {
BANNED_REAUTH_MESSAGE,
DELETED_ACCOUNT_REAUTH_MESSAGE,
handleDeletedUserSignIn,
} from "./auth";
function makeCtx({
user,
banRecords,
}: {
user: {
deletedAt?: number;
deactivatedAt?: number;
purgedAt?: number;
banReason?: string;
} | null;
banRecords?: Array<Record<string, unknown>>;
}) {
const query = {
withIndex: vi.fn().mockReturnValue({
collect: vi.fn().mockResolvedValue(banRecords ?? []),
}),
};
const ctx = {
db: {
get: vi.fn().mockResolvedValue(user),
patch: vi.fn().mockResolvedValue(null),
query: vi.fn().mockReturnValue(query),
},
};
return { ctx, query };
}
describe("handleDeletedUserSignIn", () => {
const userId = "users:1" as Id<"users">;
it("skips when user not found", async () => {
const { ctx } = makeCtx({ user: null });
await handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId });
expect(ctx.db.get).toHaveBeenCalledWith(userId);
expect(ctx.db.query).not.toHaveBeenCalled();
});
it("skips active users", async () => {
const { ctx } = makeCtx({ user: { deletedAt: undefined, deactivatedAt: undefined } });
await handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId });
expect(ctx.db.query).not.toHaveBeenCalled();
expect(ctx.db.patch).not.toHaveBeenCalled();
});
it("blocks sign-in for deactivated users", async () => {
const { ctx } = makeCtx({ user: { deactivatedAt: 123, purgedAt: 123 } });
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(DELETED_ACCOUNT_REAUTH_MESSAGE);
expect(ctx.db.query).not.toHaveBeenCalled();
expect(ctx.db.patch).not.toHaveBeenCalled();
});
it("migrates legacy self-deleted users and blocks sign-in", async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecords: [] });
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(DELETED_ACCOUNT_REAUTH_MESSAGE);
expect(ctx.db.patch).toHaveBeenCalledWith(userId, {
deletedAt: undefined,
deactivatedAt: 123,
purgedAt: 123,
updatedAt: expect.any(Number),
});
});
it("migrates legacy users on fresh login (existingUserId is null)", async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecords: [] });
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: null }),
).rejects.toThrow(DELETED_ACCOUNT_REAUTH_MESSAGE);
expect(ctx.db.patch).toHaveBeenCalledWith(userId, {
deletedAt: undefined,
deactivatedAt: 123,
purgedAt: 123,
updatedAt: expect.any(Number),
});
});
it("skips mutation when existingUserId does not match userId", async () => {
const otherUserId = "users:999" as Id<"users">;
const { ctx } = makeCtx({ user: { deletedAt: 123 } });
await handleDeletedUserSignIn(ctx as never, { userId, existingUserId: otherUserId });
expect(ctx.db.query).not.toHaveBeenCalled();
expect(ctx.db.patch).not.toHaveBeenCalled();
});
it("blocks banned users with a custom message", async () => {
const { ctx } = makeCtx({ user: { deletedAt: 123 }, banRecords: [{ action: "user.ban" }] });
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(BANNED_REAUTH_MESSAGE);
expect(ctx.db.patch).not.toHaveBeenCalled();
});
it("blocks users auto-banned for malware", async () => {
const { ctx } = makeCtx({
user: { deletedAt: 123, banReason: "malware auto-ban" },
banRecords: [{ action: "user.autoban.malware" }],
});
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(BANNED_REAUTH_MESSAGE);
expect(ctx.db.patch).not.toHaveBeenCalled();
});
it("includes the moderator ban reason in the sign-in error", async () => {
const { ctx } = makeCtx({
user: { deletedAt: 123, banReason: "Chargeback fraud" },
banRecords: [{ action: "user.ban" }],
});
await expect(
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
).rejects.toThrow(`${BANNED_REAUTH_MESSAGE} Reason: Chargeback fraud`);
});
});

View File

@ -1,19 +1,116 @@
import GitHub from '@auth/core/providers/github'
import { convexAuth } from '@convex-dev/auth/server'
import GitHub from "@auth/core/providers/github";
import { convexAuth } from "@convex-dev/auth/server";
import type { GenericMutationCtx } from "convex/server";
import { ConvexError } from "convex/values";
import { internal } from "./_generated/api";
import type { DataModel, Id } from "./_generated/dataModel";
import { shouldScheduleGitHubProfileSync } from "./lib/githubProfileSync";
export const BANNED_REAUTH_MESSAGE =
"This account has been banned and cannot sign in. If you believe this is a mistake, please contact security@openclaw.ai and we will review it.";
export const DELETED_ACCOUNT_REAUTH_MESSAGE =
"This account has been permanently deleted and cannot be restored.";
const REAUTH_BLOCKING_BAN_ACTIONS = new Set(["user.ban", "user.autoban.malware"]);
function getBannedReauthMessage(reason: string | undefined) {
const normalizedReason = reason?.trim();
if (!normalizedReason || normalizedReason.toLowerCase() === "malware auto-ban") {
return BANNED_REAUTH_MESSAGE;
}
return `${BANNED_REAUTH_MESSAGE} Reason: ${normalizedReason}`;
}
export async function handleDeletedUserSignIn(
ctx: GenericMutationCtx<DataModel>,
args: { userId: Id<"users">; existingUserId: Id<"users"> | null },
userOverride?: {
deletedAt?: number;
deactivatedAt?: number;
purgedAt?: number;
banReason?: string;
} | null,
) {
const user = userOverride !== undefined ? userOverride : await ctx.db.get(args.userId);
if (!user?.deletedAt && !user?.deactivatedAt) return;
// Verify that the incoming identity matches the existing account to prevent bypass.
if (args.existingUserId && args.existingUserId !== args.userId) {
return;
}
if (user.deactivatedAt) {
throw new ConvexError(DELETED_ACCOUNT_REAUTH_MESSAGE);
}
const userId = args.userId;
const deletedAt = user.deletedAt ?? Date.now();
const banRecords = await ctx.db
.query("auditLogs")
.withIndex("by_target", (q) => q.eq("targetType", "user").eq("targetId", userId.toString()))
.collect();
const hasBlockingBan = banRecords.some((record) =>
REAUTH_BLOCKING_BAN_ACTIONS.has(record.action),
);
if (hasBlockingBan) {
throw new ConvexError(getBannedReauthMessage(user.banReason));
}
// Migrate legacy self-deleted accounts (stored in deletedAt) to the new
// irreversible state and reject sign-in.
await ctx.db.patch(userId, {
deletedAt: undefined,
deactivatedAt: deletedAt,
purgedAt: user.purgedAt ?? deletedAt,
updatedAt: Date.now(),
});
throw new ConvexError(DELETED_ACCOUNT_REAUTH_MESSAGE);
}
export const { auth, signIn, signOut, store, isAuthenticated } = convexAuth({
providers: [
GitHub({
clientId: process.env.AUTH_GITHUB_ID ?? '',
clientSecret: process.env.AUTH_GITHUB_SECRET ?? '',
clientId: process.env.AUTH_GITHUB_ID ?? "",
clientSecret: process.env.AUTH_GITHUB_SECRET ?? "",
profile(profile) {
return {
id: String(profile.id),
name: profile.login,
email: profile.email ?? undefined,
image: profile.avatar_url,
}
};
},
}),
],
})
callbacks: {
/**
* Block sign-in for deleted/deactivated users and sync GitHub profile.
*
* Performance note: This callback runs on every OAuth sign-in, but the
* audit log query ONLY executes when a legacy deleted user attempts to sign
* in (user.deletedAt is set). For active users, this is a single field check.
*
* The GitHub profile sync is scheduled as a background action to handle
* the case where a user renames their GitHub account (fixes #303).
*/
async afterUserCreatedOrUpdated(ctx, args) {
const user = await ctx.db.get(args.userId);
await handleDeletedUserSignIn(ctx, args, user);
await ctx.scheduler.runAfter(0, internal.publishers.ensurePersonalPublisherInternal, {
userId: args.userId,
});
// Schedule GitHub profile sync to handle username renames (fixes #303)
// This runs as a background action so it doesn't block sign-in
const now = Date.now();
if (shouldScheduleGitHubProfileSync(user, now)) {
await ctx.scheduler.runAfter(0, internal.users.syncGitHubProfileAction, {
userId: args.userId,
});
}
},
},
});

View File

@ -0,0 +1,290 @@
/* @vitest-environment node */
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
vi.mock("./_generated/api", () => ({
internal: {
commentModeration: {
getCommentScamBackfillPageInternal: Symbol(
"commentModeration.getCommentScamBackfillPageInternal",
),
applyCommentScamResultInternal: Symbol("commentModeration.applyCommentScamResultInternal"),
backfillCommentScamModerationInternal: Symbol(
"commentModeration.backfillCommentScamModerationInternal",
),
continueCommentScamModerationJobInternal: Symbol(
"commentModeration.continueCommentScamModerationJobInternal",
),
},
llmEval: {
evaluateCommentForScam: Symbol("llmEval.evaluateCommentForScam"),
},
users: {
banUserInternal: Symbol("users.banUserInternal"),
},
},
}));
const { applyCommentScamResultInternalHandler, backfillCommentScamModerationInternalHandler } =
await import("./commentModeration");
const { internal } = await import("./_generated/api");
const previousOpenAiApiKey = process.env.OPENAI_API_KEY;
beforeEach(() => {
process.env.OPENAI_API_KEY = "test-key";
});
afterEach(() => {
if (previousOpenAiApiKey === undefined) {
delete process.env.OPENAI_API_KEY;
return;
}
process.env.OPENAI_API_KEY = previousOpenAiApiKey;
});
describe("commentModeration backfill", () => {
it("evaluates comments and bans on certain/high scams", async () => {
const runQuery = vi.fn().mockResolvedValueOnce({
items: [
{
commentId: "comments:1",
skillId: "skills:1",
userId: "users:2",
body: 'echo "mal" | base64 -D | bash',
softDeletedAt: undefined,
scamScanCheckedAt: undefined,
},
],
cursor: null,
isDone: true,
});
const runAction = vi.fn().mockResolvedValue({
ok: true,
model: "gpt-5-mini",
verdict: "certain_scam",
confidence: "high",
explanation: "Obfuscated shell execution payload.",
evidence: ["base64 decode piped to bash"],
});
const runMutation = vi.fn().mockResolvedValue({
ok: true,
shouldBan: true,
banned: true,
alreadyBanned: false,
protectedRole: false,
wouldBan: false,
});
const result = await backfillCommentScamModerationInternalHandler(
{ runQuery, runAction, runMutation } as never,
{
actorUserId: "users:admin",
dryRun: false,
batchSize: 10,
maxBatches: 1,
} as never,
);
expect(result.ok).toBe(true);
expect(result.stats.commentsScanned).toBe(1);
expect(result.stats.commentsEvaluated).toBe(1);
expect(result.stats.certainScams).toBe(1);
expect(result.stats.banCandidates).toBe(1);
expect(result.stats.usersBanned).toBe(1);
expect(runAction).toHaveBeenCalledWith(internal.llmEval.evaluateCommentForScam, {
commentId: "comments:1",
skillId: "skills:1",
userId: "users:2",
body: 'echo "mal" | base64 -D | bash',
});
expect(runMutation).toHaveBeenCalledWith(
internal.commentModeration.applyCommentScamResultInternal,
{
actorUserId: "users:admin",
commentId: "comments:1",
verdict: "certain_scam",
confidence: "high",
explanation: "Obfuscated shell execution payload.",
evidence: ["base64 decode piped to bash"],
model: "gpt-5-mini",
checkedAt: expect.any(Number),
dryRun: false,
},
);
});
it("skips previously scanned comments unless rescan=true", async () => {
const runQuery = vi.fn().mockResolvedValue({
items: [
{
commentId: "comments:1",
skillId: "skills:1",
userId: "users:2",
body: "something",
softDeletedAt: undefined,
scamScanCheckedAt: 123,
},
],
cursor: null,
isDone: true,
});
const runAction = vi.fn();
const runMutation = vi.fn();
const result = await backfillCommentScamModerationInternalHandler(
{ runQuery, runAction, runMutation } as never,
{
actorUserId: "users:admin",
batchSize: 10,
maxBatches: 1,
} as never,
);
expect(result.stats.commentsScanned).toBe(1);
expect(result.stats.skippedAlreadyScanned).toBe(1);
expect(runAction).not.toHaveBeenCalled();
expect(runMutation).not.toHaveBeenCalled();
});
it("tracks dry-run ban candidates without banning", async () => {
const runQuery = vi.fn().mockResolvedValue({
items: [
{
commentId: "comments:9",
skillId: "skills:7",
userId: "users:5",
body: "run this update installer from random domain",
softDeletedAt: undefined,
scamScanCheckedAt: undefined,
},
],
cursor: null,
isDone: true,
});
const runAction = vi.fn().mockResolvedValue({
ok: true,
model: "gpt-5-mini",
verdict: "certain_scam",
confidence: "high",
explanation: "Social-engineering install command.",
evidence: ["unknown update domain"],
});
const runMutation = vi.fn().mockResolvedValue({
ok: true,
shouldBan: true,
banned: false,
alreadyBanned: false,
protectedRole: false,
wouldBan: true,
});
const result = await backfillCommentScamModerationInternalHandler(
{ runQuery, runAction, runMutation } as never,
{
actorUserId: "users:admin",
dryRun: true,
batchSize: 10,
maxBatches: 1,
} as never,
);
expect(result.stats.usersBanned).toBe(0);
expect(result.stats.usersWouldBeBanned).toBe(1);
});
});
describe("applyCommentScamResultInternalHandler", () => {
it("persists scan metadata and triggers ban with bounded reason", async () => {
const get = vi
.fn()
.mockResolvedValueOnce({
_id: "comments:1",
skillId: "skills:1",
userId: "users:2",
})
.mockResolvedValueOnce({
_id: "users:2",
role: "user",
});
const patch = vi.fn();
const insert = vi.fn();
const runMutation = vi
.fn()
.mockResolvedValue({ ok: true, alreadyBanned: false, deletedSkills: 0 });
const result = await applyCommentScamResultInternalHandler(
{ db: { get, patch, insert }, runMutation } as never,
{
actorUserId: "users:admin",
commentId: "comments:1",
verdict: "certain_scam",
confidence: "high",
explanation: "X".repeat(700),
evidence: ["Y".repeat(280), "Z".repeat(280)],
model: "gpt-5-mini",
checkedAt: 123,
} as never,
);
expect(result.banned).toBe(true);
expect(insert).toHaveBeenCalledWith("auditLogs", {
actorUserId: "users:admin",
action: "comment.scam_scan",
targetType: "comment",
targetId: "comments:1",
metadata: {
skillId: "skills:1",
commentAuthorId: "users:2",
verdict: "certain_scam",
confidence: "high",
shouldBan: true,
model: "gpt-5-mini",
},
createdAt: 123,
});
const banCall = runMutation.mock.calls.find(
(call) => call[0] === internal.users.banUserInternal,
);
expect(banCall).toBeTruthy();
if (!banCall) throw new Error("Expected ban mutation to be called");
expect((banCall[1] as { reason: string }).reason.length).toBeLessThanOrEqual(500);
expect(patch).toHaveBeenCalledWith("comments:1", {
scamBanTriggeredAt: 123,
});
});
it("skips banning moderator/admin accounts", async () => {
const get = vi
.fn()
.mockResolvedValueOnce({
_id: "comments:2",
skillId: "skills:2",
userId: "users:staff",
})
.mockResolvedValueOnce({
_id: "users:staff",
role: "moderator",
});
const patch = vi.fn();
const insert = vi.fn();
const runMutation = vi.fn();
const result = await applyCommentScamResultInternalHandler(
{ db: { get, patch, insert }, runMutation } as never,
{
actorUserId: "users:admin",
commentId: "comments:2",
verdict: "certain_scam",
confidence: "high",
explanation: "Malicious command spam.",
evidence: ["base64|bash"],
model: "gpt-5-mini",
checkedAt: 300,
} as never,
);
expect(result.protectedRole).toBe(true);
expect(runMutation).not.toHaveBeenCalled();
});
});

479
convex/commentModeration.ts Normal file
View File

@ -0,0 +1,479 @@
import { ConvexError, v } from "convex/values";
import { internal } from "./_generated/api";
import type { Id } from "./_generated/dataModel";
import type { ActionCtx, MutationCtx } from "./_generated/server";
import { action, internalAction, internalMutation, internalQuery } from "./functions";
import { assertRole, requireUserFromAction } from "./lib/access";
import {
buildCommentScamBanReason,
isCertainScam,
type CommentScamConfidence,
type CommentScamVerdict,
} from "./lib/commentScamPrompt";
const DEFAULT_BATCH_SIZE = 25;
const MAX_BATCH_SIZE = 100;
const DEFAULT_MAX_BATCHES = 10;
const MAX_MAX_BATCHES = 200;
type CommentBackfillPageItem = {
commentId: Id<"comments">;
skillId: Id<"skills">;
userId: Id<"users">;
body: string;
softDeletedAt?: number;
scamScanCheckedAt?: number;
};
type CommentBackfillPageResult = {
items: CommentBackfillPageItem[];
cursor: string | null;
isDone: boolean;
};
type ApplyCommentScamResult = {
ok: true;
shouldBan: boolean;
banned: boolean;
alreadyBanned: boolean;
protectedRole: boolean;
wouldBan: boolean;
};
export type CommentScamBackfillStats = {
commentsScanned: number;
commentsEvaluated: number;
certainScams: number;
banCandidates: number;
usersBanned: number;
usersAlreadyBanned: number;
usersWouldBeBanned: number;
protectedRoleSkips: number;
skippedSoftDeleted: number;
skippedAlreadyScanned: number;
skippedEmptyBody: number;
evalErrors: number;
};
export type CommentScamBackfillActionArgs = {
actorUserId: Id<"users">;
dryRun?: boolean;
batchSize?: number;
maxBatches?: number;
cursor?: string;
rescan?: boolean;
includeSoftDeleted?: boolean;
};
export type CommentScamBackfillActionResult = {
ok: true;
stats: CommentScamBackfillStats;
isDone: boolean;
cursor: string | null;
};
export const getCommentScamBackfillPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args): Promise<CommentBackfillPageResult> => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
const { page, isDone, continueCursor } = await ctx.db
.query("comments")
.order("asc")
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
return {
items: page.map((comment) => ({
commentId: comment._id,
skillId: comment.skillId,
userId: comment.userId,
body: comment.body,
softDeletedAt: comment.softDeletedAt,
scamScanCheckedAt: comment.scamScanCheckedAt,
})),
cursor: continueCursor,
isDone,
};
},
});
export async function applyCommentScamResultInternalHandler(
ctx: MutationCtx,
args: {
actorUserId: Id<"users">;
commentId: Id<"comments">;
verdict: CommentScamVerdict;
confidence: CommentScamConfidence;
explanation: string;
evidence: string[];
model: string;
checkedAt: number;
dryRun?: boolean;
},
): Promise<ApplyCommentScamResult> {
const comment = await ctx.db.get(args.commentId);
if (!comment) {
throw new ConvexError("Comment not found");
}
const user = await ctx.db.get(comment.userId);
if (!user) {
throw new ConvexError("Comment author not found");
}
const dryRun = Boolean(args.dryRun);
const shouldBan = isCertainScam({
verdict: args.verdict,
confidence: args.confidence,
});
const explanation = args.explanation.trim().slice(0, 1200);
const evidence = args.evidence
.map((item) => item.trim())
.filter(Boolean)
.slice(0, 5);
if (!dryRun) {
await ctx.db.patch(comment._id, {
scamScanVerdict: args.verdict,
scamScanConfidence: args.confidence,
scamScanExplanation: explanation,
scamScanEvidence: evidence,
scamScanModel: args.model,
scamScanCheckedAt: args.checkedAt,
});
await ctx.db.insert("auditLogs", {
actorUserId: args.actorUserId,
action: "comment.scam_scan",
targetType: "comment",
targetId: comment._id,
metadata: {
skillId: comment.skillId,
commentAuthorId: comment.userId,
verdict: args.verdict,
confidence: args.confidence,
shouldBan,
model: args.model,
},
createdAt: args.checkedAt,
});
}
if (!shouldBan) {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: false,
protectedRole: false,
wouldBan: false,
};
}
if (user.role === "admin" || user.role === "moderator") {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: false,
protectedRole: true,
wouldBan: false,
};
}
if (user.deletedAt || user.deactivatedAt) {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: true,
protectedRole: false,
wouldBan: false,
};
}
if (dryRun) {
return {
ok: true,
shouldBan,
banned: false,
alreadyBanned: false,
protectedRole: false,
wouldBan: true,
};
}
const reason = buildCommentScamBanReason({
commentId: String(comment._id),
skillId: String(comment.skillId),
explanation,
evidence,
});
const banResult = await ctx.runMutation(internal.users.banUserInternal, {
actorUserId: args.actorUserId,
targetUserId: comment.userId,
reason,
});
if (!banResult.alreadyBanned) {
await ctx.db.patch(comment._id, {
scamBanTriggeredAt: args.checkedAt,
});
}
return {
ok: true,
shouldBan,
banned: !banResult.alreadyBanned,
alreadyBanned: banResult.alreadyBanned,
protectedRole: false,
wouldBan: false,
};
}
export const applyCommentScamResultInternal = internalMutation({
args: {
actorUserId: v.id("users"),
commentId: v.id("comments"),
verdict: v.union(v.literal("not_scam"), v.literal("likely_scam"), v.literal("certain_scam")),
confidence: v.union(v.literal("low"), v.literal("medium"), v.literal("high")),
explanation: v.string(),
evidence: v.array(v.string()),
model: v.string(),
checkedAt: v.number(),
dryRun: v.optional(v.boolean()),
},
handler: applyCommentScamResultInternalHandler,
});
export async function backfillCommentScamModerationInternalHandler(
ctx: ActionCtx,
args: CommentScamBackfillActionArgs,
): Promise<CommentScamBackfillActionResult> {
if (!process.env.OPENAI_API_KEY) {
throw new ConvexError("OPENAI_API_KEY not configured");
}
const dryRun = Boolean(args.dryRun);
const rescan = Boolean(args.rescan);
const includeSoftDeleted = Boolean(args.includeSoftDeleted);
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES);
let cursor: string | null = args.cursor ?? null;
let isDone = false;
const stats: CommentScamBackfillStats = {
commentsScanned: 0,
commentsEvaluated: 0,
certainScams: 0,
banCandidates: 0,
usersBanned: 0,
usersAlreadyBanned: 0,
usersWouldBeBanned: 0,
protectedRoleSkips: 0,
skippedSoftDeleted: 0,
skippedAlreadyScanned: 0,
skippedEmptyBody: 0,
evalErrors: 0,
};
for (let i = 0; i < maxBatches; i++) {
const page = (await ctx.runQuery(
internal.commentModeration.getCommentScamBackfillPageInternal,
{
cursor: cursor ?? undefined,
batchSize,
},
)) as CommentBackfillPageResult;
cursor = page.cursor;
isDone = page.isDone;
for (const comment of page.items) {
stats.commentsScanned++;
if (!includeSoftDeleted && comment.softDeletedAt) {
stats.skippedSoftDeleted++;
continue;
}
if (!rescan && comment.scamScanCheckedAt) {
stats.skippedAlreadyScanned++;
continue;
}
const body = comment.body.trim();
if (!body) {
stats.skippedEmptyBody++;
continue;
}
const evalResult = (await ctx.runAction(internal.llmEval.evaluateCommentForScam, {
commentId: comment.commentId,
skillId: comment.skillId,
userId: comment.userId,
body,
})) as
| {
ok: true;
model: string;
verdict: CommentScamVerdict;
confidence: CommentScamConfidence;
explanation: string;
evidence: string[];
}
| { ok: false; error: string };
if (!evalResult.ok) {
stats.evalErrors++;
continue;
}
stats.commentsEvaluated++;
const shouldBan = isCertainScam(evalResult);
if (evalResult.verdict === "certain_scam") {
stats.certainScams++;
}
if (shouldBan) {
stats.banCandidates++;
}
const applyResult = (await ctx.runMutation(
internal.commentModeration.applyCommentScamResultInternal,
{
actorUserId: args.actorUserId,
commentId: comment.commentId,
verdict: evalResult.verdict,
confidence: evalResult.confidence,
explanation: evalResult.explanation,
evidence: evalResult.evidence,
model: evalResult.model,
checkedAt: Date.now(),
dryRun,
},
)) as ApplyCommentScamResult;
if (applyResult.banned) stats.usersBanned++;
if (applyResult.alreadyBanned) stats.usersAlreadyBanned++;
if (applyResult.wouldBan) stats.usersWouldBeBanned++;
if (applyResult.protectedRole) stats.protectedRoleSkips++;
}
if (isDone) break;
}
return {
ok: true,
stats,
isDone,
cursor,
};
}
export const backfillCommentScamModerationInternal = internalAction({
args: {
actorUserId: v.id("users"),
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
cursor: v.optional(v.string()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: backfillCommentScamModerationInternalHandler,
});
export const backfillCommentScamModeration: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
cursor: v.optional(v.string()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<CommentScamBackfillActionResult> => {
const { user } = await requireUserFromAction(ctx);
assertRole(user, ["admin", "moderator"]);
return ctx.runAction(internal.commentModeration.backfillCommentScamModerationInternal, {
actorUserId: user._id,
...args,
}) as Promise<CommentScamBackfillActionResult>;
},
});
export const continueCommentScamModerationJobInternal = internalAction({
args: {
actorUserId: v.id("users"),
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
cursor: v.optional(v.string()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: async (ctx, args) => {
const result = await backfillCommentScamModerationInternalHandler(ctx, {
actorUserId: args.actorUserId,
dryRun: args.dryRun,
batchSize: args.batchSize,
cursor: args.cursor,
maxBatches: 1,
rescan: args.rescan,
includeSoftDeleted: args.includeSoftDeleted,
});
if (!result.isDone && result.cursor) {
await ctx.scheduler.runAfter(
2_000,
internal.commentModeration.continueCommentScamModerationJobInternal,
{
actorUserId: args.actorUserId,
dryRun: Boolean(args.dryRun),
batchSize: args.batchSize ?? DEFAULT_BATCH_SIZE,
cursor: result.cursor,
rescan: Boolean(args.rescan),
includeSoftDeleted: Boolean(args.includeSoftDeleted),
},
);
}
return result;
},
});
export const scheduleCommentScamModeration: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
rescan: v.optional(v.boolean()),
includeSoftDeleted: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<{ ok: true }> => {
const { user } = await requireUserFromAction(ctx);
assertRole(user, ["admin", "moderator"]);
await ctx.scheduler.runAfter(
0,
internal.commentModeration.continueCommentScamModerationJobInternal,
{
actorUserId: user._id,
dryRun: Boolean(args.dryRun),
batchSize: clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE),
cursor: undefined,
rescan: Boolean(args.rescan),
includeSoftDeleted: Boolean(args.includeSoftDeleted),
},
);
return { ok: true as const };
},
});
function clampInt(value: number, min: number, max: number) {
return Math.min(Math.max(Math.trunc(value), min), max);
}

151
convex/comments.handlers.ts Normal file
View File

@ -0,0 +1,151 @@
import type { Id } from "./_generated/dataModel";
import type { MutationCtx } from "./_generated/server";
import { assertModerator, requireUser } from "./lib/access";
import { requireGitHubAccountAge } from "./lib/githubAccount";
import {
AUTO_HIDE_REPORT_THRESHOLD,
MAX_ACTIVE_REPORTS_PER_USER,
MAX_REPORT_REASON_LENGTH,
} from "./lib/reporting";
import { insertStatEvent } from "./skillStatEvents";
export async function addHandler(ctx: MutationCtx, args: { skillId: Id<"skills">; body: string }) {
const { userId } = await requireUser(ctx);
await requireGitHubAccountAge(ctx, userId);
const body = args.body.trim();
if (!body) throw new Error("Comment body required");
const skill = await ctx.db.get(args.skillId);
if (!skill) throw new Error("Skill not found");
await ctx.db.insert("comments", {
skillId: args.skillId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
});
await insertStatEvent(ctx, { skillId: skill._id, kind: "comment" });
}
export async function removeHandler(ctx: MutationCtx, args: { commentId: Id<"comments"> }) {
const { user } = await requireUser(ctx);
const comment = await ctx.db.get(args.commentId);
if (!comment) throw new Error("Comment not found");
if (comment.softDeletedAt) return;
const isOwner = comment.userId === user._id;
if (!isOwner) {
assertModerator(user);
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
});
await insertStatEvent(ctx, { skillId: comment.skillId, kind: "uncomment" });
await ctx.db.insert("auditLogs", {
actorUserId: user._id,
action: "comment.delete",
targetType: "comment",
targetId: comment._id,
metadata: { skillId: comment.skillId },
createdAt: Date.now(),
});
}
async function countActiveReportsForUser(ctx: MutationCtx, userId: Id<"users">) {
const reports = await ctx.db
.query("commentReports")
.withIndex("by_user", (q) => q.eq("userId", userId))
.collect();
let count = 0;
for (const report of reports) {
const comment = await ctx.db.get(report.commentId);
if (!comment || comment.softDeletedAt) continue;
const skill = await ctx.db.get(comment.skillId);
if (!skill || skill.softDeletedAt || skill.moderationStatus === "removed") continue;
const owner = await ctx.db.get(comment.userId);
if (!owner || owner.deletedAt || owner.deactivatedAt) continue;
count += 1;
if (count >= MAX_ACTIVE_REPORTS_PER_USER) break;
}
return count;
}
export async function reportHandler(
ctx: MutationCtx,
args: { commentId: Id<"comments">; reason: string },
) {
const { userId } = await requireUser(ctx);
const comment = await ctx.db.get(args.commentId);
if (!comment || comment.softDeletedAt) {
throw new Error("Comment not found");
}
const skill = await ctx.db.get(comment.skillId);
if (!skill || skill.softDeletedAt || skill.moderationStatus === "removed") {
throw new Error("Comment not found");
}
const reason = args.reason.trim();
if (!reason) {
throw new Error("Report reason required.");
}
const existing = await ctx.db
.query("commentReports")
.withIndex("by_comment_user", (q) => q.eq("commentId", args.commentId).eq("userId", userId))
.unique();
if (existing) return { ok: true as const, reported: false, alreadyReported: true };
const activeReports = await countActiveReportsForUser(ctx, userId);
if (activeReports >= MAX_ACTIVE_REPORTS_PER_USER) {
throw new Error("Report limit reached. Please wait for moderation before reporting more.");
}
const now = Date.now();
await ctx.db.insert("commentReports", {
commentId: args.commentId,
skillId: comment.skillId,
userId,
reason: reason.slice(0, MAX_REPORT_REASON_LENGTH),
createdAt: now,
});
const nextReportCount = (comment.reportCount ?? 0) + 1;
const shouldAutoHide = nextReportCount > AUTO_HIDE_REPORT_THRESHOLD && !comment.softDeletedAt;
const updates: {
reportCount: number;
lastReportedAt: number;
softDeletedAt?: number;
} = {
reportCount: nextReportCount,
lastReportedAt: now,
};
if (shouldAutoHide) {
updates.softDeletedAt = now;
}
await ctx.db.patch(comment._id, updates);
if (shouldAutoHide) {
await insertStatEvent(ctx, { skillId: comment.skillId, kind: "uncomment" });
await ctx.db.insert("auditLogs", {
actorUserId: userId,
action: "comment.auto_hide",
targetType: "comment",
targetId: comment._id,
metadata: { skillId: comment.skillId, reportCount: nextReportCount },
createdAt: now,
});
}
return { ok: true as const, reported: true, alreadyReported: false };
}

View File

@ -0,0 +1,127 @@
/* @vitest-environment node */
import { describe, expect, it } from "vitest";
import { listBySkillHandler } from "./comments";
function makeCtx(args: {
comments: Array<Record<string, unknown>>;
usersById: Record<string, Record<string, unknown> | null>;
}) {
const get = async (id: string) => args.usersById[id] ?? null;
const take = async () => args.comments;
const order = () => ({ take });
const withIndex = () => ({ order });
const query = () => ({ withIndex });
return { db: { get, query } } as never;
}
describe("comments.listBySkill", () => {
it("skips soft-deleted comments", async () => {
const ctx = makeCtx({
comments: [
{
_id: "comments:live",
skillId: "skills:1",
userId: "users:live",
body: "hello",
},
{
_id: "comments:deleted",
skillId: "skills:1",
userId: "users:live",
body: "bye",
softDeletedAt: 123,
},
],
usersById: {
"users:live": {
_id: "users:live",
_creationTime: 1,
handle: "live",
name: "live",
displayName: "Live",
image: null,
bio: null,
},
},
});
const result = await listBySkillHandler(ctx, {
skillId: "skills:1",
limit: 50,
} as never);
expect(result).toHaveLength(1);
expect(result[0]?.comment._id).toBe("comments:live");
});
it("skips comments whose author is deleted/deactivated/missing", async () => {
const ctx = makeCtx({
comments: [
{
_id: "comments:ok",
skillId: "skills:1",
userId: "users:ok",
body: "ok",
},
{
_id: "comments:deleted-user",
skillId: "skills:1",
userId: "users:deleted",
body: "hidden",
},
{
_id: "comments:deactivated-user",
skillId: "skills:1",
userId: "users:deactivated",
body: "hidden",
},
{
_id: "comments:missing-user",
skillId: "skills:1",
userId: "users:missing",
body: "hidden",
},
],
usersById: {
"users:ok": {
_id: "users:ok",
_creationTime: 1,
handle: "ok",
name: "ok",
displayName: "Ok",
image: null,
bio: null,
},
"users:deleted": {
_id: "users:deleted",
_creationTime: 1,
handle: "deleted",
name: "deleted",
displayName: "Deleted",
image: null,
bio: null,
deletedAt: 123,
},
"users:deactivated": {
_id: "users:deactivated",
_creationTime: 1,
handle: "deactivated",
name: "deactivated",
displayName: "Deactivated",
image: null,
bio: null,
deactivatedAt: 456,
},
},
});
const result = await listBySkillHandler(ctx, {
skillId: "skills:1",
limit: 50,
} as never);
expect(result).toHaveLength(1);
expect(result[0]?.comment._id).toBe("comments:ok");
expect(result[0]?.user._id).toBe("users:ok");
});
});

614
convex/comments.test.ts Normal file
View File

@ -0,0 +1,614 @@
/* @vitest-environment node */
import { afterEach, describe, expect, it, vi } from "vitest";
vi.mock("./lib/access", () => ({
assertModerator: vi.fn(),
requireUser: vi.fn(),
}));
vi.mock("./skillStatEvents", () => ({
insertStatEvent: vi.fn(),
}));
vi.mock("./lib/githubAccount", () => ({
requireGitHubAccountAge: vi.fn(),
}));
const { requireUser, assertModerator } = await import("./lib/access");
const { insertStatEvent } = await import("./skillStatEvents");
const { requireGitHubAccountAge } = await import("./lib/githubAccount");
const { addHandler, removeHandler, reportHandler } = await import("./comments.handlers");
describe("comments mutations", () => {
afterEach(() => {
vi.mocked(assertModerator).mockReset();
vi.mocked(requireUser).mockReset();
vi.mocked(insertStatEvent).mockReset();
vi.mocked(requireGitHubAccountAge).mockReset();
vi.restoreAllMocks();
});
it("add avoids direct skill patch and records stat event", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
vi.mocked(requireGitHubAccountAge).mockResolvedValue(undefined as never);
const get = vi.fn().mockResolvedValue({
_id: "skills:1",
});
const insert = vi.fn();
const patch = vi.fn();
const ctx = { db: { get, insert, patch } } as never;
await addHandler(ctx, { skillId: "skills:1", body: " hello " } as never);
expect(requireGitHubAccountAge).toHaveBeenCalledWith(ctx, "users:1");
expect(patch).not.toHaveBeenCalled();
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: "skills:1",
kind: "comment",
});
});
it("add blocks new comments when github account age gate fails", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:new",
user: { _id: "users:new", role: "user" },
} as never);
vi.mocked(requireGitHubAccountAge).mockRejectedValue(
new Error(
"GitHub account must be at least 14 days old to upload skills. Try again in 3 days.",
),
);
const get = vi.fn();
const insert = vi.fn();
const patch = vi.fn();
const ctx = { db: { get, insert, patch } } as never;
await expect(addHandler(ctx, { skillId: "skills:1", body: "hello" } as never)).rejects.toThrow(
/at least 14 days old/i,
);
expect(get).not.toHaveBeenCalled();
expect(insert).not.toHaveBeenCalled();
expect(patch).not.toHaveBeenCalled();
expect(insertStatEvent).not.toHaveBeenCalled();
});
it("remove keeps comment soft-delete patch free of updatedAt", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:2",
user: { _id: "users:2", role: "moderator" },
} as never);
const comment = {
_id: "comments:1",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:1") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const ctx = { db: { get, insert, patch } } as never;
await removeHandler(ctx, { commentId: "comments:1" } as never);
expect(patch).toHaveBeenCalledTimes(1);
const deletePatch = vi.mocked(patch).mock.calls[0]?.[1] as Record<string, unknown>;
expect(deletePatch.updatedAt).toBeUndefined();
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: "skills:1",
kind: "uncomment",
});
});
it("remove rejects non-owner without moderator permission", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:3",
user: { _id: "users:3", role: "user" },
} as never);
vi.mocked(assertModerator).mockImplementation(() => {
throw new Error("Moderator role required");
});
const comment = {
_id: "comments:2",
skillId: "skills:2",
userId: "users:9",
softDeletedAt: undefined,
};
const get = vi.fn().mockResolvedValue(comment);
const insert = vi.fn();
const patch = vi.fn();
const ctx = { db: { get, insert, patch } } as never;
await expect(removeHandler(ctx, { commentId: "comments:2" } as never)).rejects.toThrow(
"Moderator role required",
);
expect(patch).not.toHaveBeenCalled();
expect(insertStatEvent).not.toHaveBeenCalled();
});
it("remove no-ops for soft-deleted comment", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:4",
user: { _id: "users:4", role: "moderator" },
} as never);
const comment = {
_id: "comments:3",
skillId: "skills:3",
userId: "users:4",
softDeletedAt: 123,
};
const get = vi.fn().mockResolvedValue(comment);
const insert = vi.fn();
const patch = vi.fn();
const ctx = { db: { get, insert, patch } } as never;
await removeHandler(ctx, { commentId: "comments:3" } as never);
expect(patch).not.toHaveBeenCalled();
expect(insert).not.toHaveBeenCalled();
expect(insertStatEvent).not.toHaveBeenCalled();
});
it("report increments count and stores reason", async () => {
vi.spyOn(Date, "now").mockReturnValue(1_700_000_000_000);
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:1",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 1,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:1") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn((table: string) => {
if (table === "commentReports") {
return {
withIndex: (index: string) => {
if (index === "by_comment_user") {
return { unique: vi.fn().mockResolvedValue(null) };
}
if (index === "by_user") {
return { collect: vi.fn().mockResolvedValue([]) };
}
throw new Error(`Unexpected index ${index}`);
},
};
}
throw new Error(`Unexpected table ${table}`);
});
const ctx = { db: { get, insert, patch, query } } as never;
const result = await reportHandler(ctx, {
commentId: "comments:1",
reason: " spam ",
} as never);
expect(result).toEqual({ ok: true, reported: true, alreadyReported: false });
expect(insert).toHaveBeenCalledWith("commentReports", {
commentId: "comments:1",
skillId: "skills:1",
userId: "users:1",
reason: "spam",
createdAt: 1_700_000_000_000,
});
expect(patch).toHaveBeenCalledWith("comments:1", {
reportCount: 2,
lastReportedAt: 1_700_000_000_000,
});
expect(insertStatEvent).not.toHaveBeenCalled();
});
it("report returns alreadyReported for duplicate reporter/comment pair", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:dup",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 0,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:dup") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn((table: string) => {
if (table !== "commentReports") throw new Error(`Unexpected table ${table}`);
return {
withIndex: (index: string) => {
if (index === "by_comment_user") {
return { unique: vi.fn().mockResolvedValue({ _id: "commentReports:existing" }) };
}
throw new Error(`Unexpected index ${index}`);
},
};
});
const ctx = { db: { get, insert, patch, query } } as never;
const result = await reportHandler(ctx, { commentId: "comments:dup", reason: "spam" } as never);
expect(result).toEqual({ ok: true, reported: false, alreadyReported: true });
expect(insert).not.toHaveBeenCalled();
expect(patch).not.toHaveBeenCalled();
});
it("report rejects empty reason", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:empty",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 0,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:empty") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn();
const ctx = { db: { get, insert, patch, query } } as never;
await expect(
reportHandler(ctx, { commentId: "comments:empty", reason: " " } as never),
).rejects.toThrow("Report reason required.");
expect(query).not.toHaveBeenCalled();
expect(insert).not.toHaveBeenCalled();
expect(patch).not.toHaveBeenCalled();
});
it("report rejects comment when parent skill is hidden/removed", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:hidden-parent",
skillId: "skills:hidden",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 0,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:hidden-parent") return comment;
if (id === "skills:hidden") {
return { _id: "skills:hidden", softDeletedAt: 123, moderationStatus: "removed" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn();
const ctx = { db: { get, insert, patch, query } } as never;
await expect(
reportHandler(ctx, { commentId: "comments:hidden-parent", reason: "abuse" } as never),
).rejects.toThrow("Comment not found");
expect(query).not.toHaveBeenCalled();
expect(insert).not.toHaveBeenCalled();
expect(patch).not.toHaveBeenCalled();
});
it("report truncates long reason to 500 chars", async () => {
vi.spyOn(Date, "now").mockReturnValue(1_700_000_000_050);
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:long",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 0,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:long") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn((table: string) => {
if (table !== "commentReports") throw new Error(`Unexpected table ${table}`);
return {
withIndex: (index: string) => {
if (index === "by_comment_user") return { unique: vi.fn().mockResolvedValue(null) };
if (index === "by_user") return { collect: vi.fn().mockResolvedValue([]) };
throw new Error(`Unexpected index ${index}`);
},
};
});
const ctx = { db: { get, insert, patch, query } } as never;
await reportHandler(ctx, { commentId: "comments:long", reason: "x".repeat(700) } as never);
const reportInsert = vi.mocked(insert).mock.calls.find((call) => call[0] === "commentReports");
expect(reportInsert?.[1]).toMatchObject({
commentId: "comments:long",
reason: "x".repeat(500),
});
});
it("report active-count filter ignores stale/non-active report targets", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:target2",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 0,
};
const reports = [
{
_id: "commentReports:1",
commentId: "comments:deleted",
userId: "users:1",
skillId: "skills:1",
},
{
_id: "commentReports:2",
commentId: "comments:removed-skill",
userId: "users:1",
skillId: "skills:removed",
},
{
_id: "commentReports:3",
commentId: "comments:deleted-owner",
userId: "users:1",
skillId: "skills:active",
},
];
const get = vi.fn(async (id: string) => {
if (id === "comments:target2") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
if (id === "comments:deleted") {
return {
_id: "comments:deleted",
softDeletedAt: 123,
skillId: "skills:1",
userId: "users:2",
};
}
if (id === "comments:removed-skill") {
return {
_id: "comments:removed-skill",
softDeletedAt: undefined,
skillId: "skills:removed",
userId: "users:2",
};
}
if (id === "skills:removed") {
return { _id: "skills:removed", softDeletedAt: undefined, moderationStatus: "removed" };
}
if (id === "comments:deleted-owner") {
return {
_id: "comments:deleted-owner",
softDeletedAt: undefined,
skillId: "skills:active",
userId: "users:deleted-owner",
};
}
if (id === "skills:active") {
return { _id: "skills:active", softDeletedAt: undefined, moderationStatus: "active" };
}
if (id === "users:deleted-owner") {
return { _id: "users:deleted-owner", deletedAt: 1, deactivatedAt: undefined };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn((table: string) => {
if (table !== "commentReports") throw new Error(`Unexpected table ${table}`);
return {
withIndex: (index: string) => {
if (index === "by_comment_user") return { unique: vi.fn().mockResolvedValue(null) };
if (index === "by_user") return { collect: vi.fn().mockResolvedValue(reports) };
throw new Error(`Unexpected index ${index}`);
},
};
});
const ctx = { db: { get, insert, patch, query } } as never;
const result = await reportHandler(ctx, {
commentId: "comments:target2",
reason: "still allowed",
} as never);
expect(result).toEqual({ ok: true, reported: true, alreadyReported: false });
expect(insert).toHaveBeenCalledWith(
"commentReports",
expect.objectContaining({ commentId: "comments:target2", userId: "users:1" }),
);
});
it("report rejects when active report limit is reached", async () => {
vi.mocked(requireUser).mockResolvedValue({
userId: "users:1",
user: { _id: "users:1", role: "user" },
} as never);
const comment = {
_id: "comments:target",
skillId: "skills:1",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 0,
};
const reportedComment = {
_id: "comments:reported",
skillId: "skills:active",
userId: "users:owner",
softDeletedAt: undefined,
};
const reports = Array.from({ length: 20 }, (_, i) => ({
_id: `commentReports:${i + 1}`,
commentId: `comments:reported-${i + 1}`,
userId: "users:1",
skillId: "skills:active",
createdAt: i + 1,
}));
const get = vi.fn(async (id: string) => {
if (id === "comments:target") return comment;
if (id === "skills:1") {
return { _id: "skills:1", softDeletedAt: undefined, moderationStatus: "active" };
}
if (id.startsWith("comments:reported-")) return reportedComment;
if (id === "skills:active") {
return { _id: "skills:active", softDeletedAt: undefined, moderationStatus: "active" };
}
if (id === "users:owner") {
return { _id: "users:owner", deletedAt: undefined, deactivatedAt: undefined };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn((table: string) => {
if (table === "commentReports") {
return {
withIndex: (index: string) => {
if (index === "by_comment_user") {
return { unique: vi.fn().mockResolvedValue(null) };
}
if (index === "by_user") {
return { collect: vi.fn().mockResolvedValue(reports) };
}
throw new Error(`Unexpected index ${index}`);
},
};
}
throw new Error(`Unexpected table ${table}`);
});
const ctx = { db: { get, insert, patch, query } } as never;
await expect(
reportHandler(ctx, { commentId: "comments:target", reason: "abuse" } as never),
).rejects.toThrow("Report limit reached. Please wait for moderation before reporting more.");
expect(insert).not.toHaveBeenCalled();
expect(patch).not.toHaveBeenCalled();
});
it("report auto-hides comment after fourth unique report", async () => {
vi.spyOn(Date, "now").mockReturnValue(1_700_000_000_100);
vi.mocked(requireUser).mockResolvedValue({
userId: "users:3",
user: { _id: "users:3", role: "user" },
} as never);
const comment = {
_id: "comments:4",
skillId: "skills:9",
userId: "users:2",
softDeletedAt: undefined,
reportCount: 3,
};
const get = vi.fn(async (id: string) => {
if (id === "comments:4") return comment;
if (id === "skills:9") {
return { _id: "skills:9", softDeletedAt: undefined, moderationStatus: "active" };
}
return null;
});
const insert = vi.fn();
const patch = vi.fn();
const query = vi.fn((table: string) => {
if (table === "commentReports") {
return {
withIndex: (index: string) => {
if (index === "by_comment_user") {
return { unique: vi.fn().mockResolvedValue(null) };
}
if (index === "by_user") {
return { collect: vi.fn().mockResolvedValue([]) };
}
throw new Error(`Unexpected index ${index}`);
},
};
}
throw new Error(`Unexpected table ${table}`);
});
const ctx = { db: { get, insert, patch, query } } as never;
const result = await reportHandler(ctx, {
commentId: "comments:4",
reason: " hate ",
} as never);
expect(result).toEqual({ ok: true, reported: true, alreadyReported: false });
expect(patch).toHaveBeenCalledWith("comments:4", {
reportCount: 4,
lastReportedAt: 1_700_000_000_100,
softDeletedAt: 1_700_000_000_100,
});
expect(insertStatEvent).toHaveBeenCalledWith(ctx, {
skillId: "skills:9",
kind: "uncomment",
});
expect(insert).toHaveBeenCalledWith("auditLogs", {
actorUserId: "users:3",
action: "comment.auto_hide",
targetType: "comment",
targetId: "comments:4",
metadata: { skillId: "skills:9", reportCount: 4 },
createdAt: 1_700_000_000_100,
});
});
});

View File

@ -1,87 +1,49 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { assertRole, requireUser } from './lib/access'
import { v } from "convex/values";
import type { Doc } from "./_generated/dataModel";
import { addHandler, removeHandler, reportHandler } from "./comments.handlers";
import { mutation, query } from "./functions";
import { type PublicUser, toPublicUser } from "./lib/public";
export const listBySkill = query({
args: { skillId: v.id('skills'), limit: v.optional(v.number()) },
handler: async (ctx, args) => {
const limit = args.limit ?? 50
const comments = await ctx.db
.query('comments')
.withIndex('by_skill', (q) => q.eq('skillId', args.skillId))
.order('desc')
.take(limit)
args: { skillId: v.id("skills"), limit: v.optional(v.number()) },
handler: listBySkillHandler,
});
const results: Array<{ comment: Doc<'comments'>; user: Doc<'users'> | null }> = []
for (const comment of comments) {
if (comment.softDeletedAt) continue
const user = await ctx.db.get(comment.userId)
results.push({ comment, user })
}
return results
},
})
export async function listBySkillHandler(
ctx: import("./_generated/server").QueryCtx,
args: { skillId: import("./_generated/dataModel").Id<"skills">; limit?: number },
) {
const limit = args.limit ?? 50;
const comments = await ctx.db
.query("comments")
.withIndex("by_skill", (q) => q.eq("skillId", args.skillId))
.order("desc")
.take(limit);
const rows = await Promise.all(
comments.map(
async (comment): Promise<{ comment: Doc<"comments">; user: PublicUser } | null> => {
if (comment.softDeletedAt) return null;
const user = toPublicUser(await ctx.db.get(comment.userId));
if (!user) return null;
return { comment, user };
},
),
);
return rows.filter((row): row is { comment: Doc<"comments">; user: PublicUser } => row !== null);
}
export const add = mutation({
args: { skillId: v.id('skills'), body: v.string() },
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
const skill = await ctx.db.get(args.skillId)
if (!skill) throw new Error('Skill not found')
await ctx.db.insert('comments', {
skillId: args.skillId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
})
await ctx.db.patch(skill._id, {
stats: { ...skill.stats, comments: skill.stats.comments + 1 },
updatedAt: Date.now(),
})
},
})
args: { skillId: v.id("skills"), body: v.string() },
handler: addHandler,
});
export const remove = mutation({
args: { commentId: v.id('comments') },
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment) throw new Error('Comment not found')
if (comment.softDeletedAt) return
args: { commentId: v.id("comments") },
handler: removeHandler,
});
const isOwner = comment.userId === user._id
if (!isOwner) {
assertRole(user, ['admin', 'moderator'])
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
})
const skill = await ctx.db.get(comment.skillId)
if (skill) {
await ctx.db.patch(skill._id, {
stats: { ...skill.stats, comments: Math.max(0, skill.stats.comments - 1) },
updatedAt: Date.now(),
})
}
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
action: 'comment.delete',
targetType: 'comment',
targetId: comment._id,
metadata: { skillId: comment.skillId },
createdAt: Date.now(),
})
},
})
export const report = mutation({
args: { commentId: v.id("comments"), reason: v.string() },
handler: reportHandler,
});

View File

@ -1,27 +1,85 @@
import { cronJobs } from 'convex/server'
import { internal } from './_generated/api'
import { cronJobs } from "convex/server";
import { internal } from "./_generated/api";
const crons = cronJobs()
const crons = cronJobs();
crons.interval(
'github-backup-sync',
"github-backup-sync",
{ minutes: 30 },
internal.githubBackupsNode.syncGitHubBackupsInternal,
{ batchSize: 50, maxBatches: 5 },
)
);
crons.interval(
'trending-leaderboard',
"trending-leaderboard",
{ minutes: 60 },
internal.leaderboards.rebuildTrendingLeaderboardInternal,
internal.leaderboards.rebuildTrendingLeaderboardAction,
{ limit: 200 },
)
);
crons.interval(
'skill-stats-backfill',
{ minutes: 10 },
"skill-stats-backfill",
{ hours: 6 },
internal.statsMaintenance.runSkillStatBackfillInternal,
{ batchSize: 200, maxBatches: 5 },
)
);
export default crons
// Runs frequently to keep dailyStats/trending accurate,
// but does NOT patch skill documents (only writes to skillDailyStats).
crons.interval(
"skill-stat-events",
{ minutes: 15 },
internal.skillStatEvents.processSkillStatEventsAction,
{},
);
crons.interval(
"package-stat-events",
{ minutes: 15 },
internal.packages.processPackageStatEventsInternal,
{ batchSize: 500 },
);
// Syncs accumulated stat deltas to skill documents every 6 hours.
// Runs infrequently to avoid thundering-herd reactive query invalidation.
// Uses processedAt field to track progress (independent of the action cursor).
crons.interval(
"skill-doc-stat-sync",
{ hours: 6 },
internal.skillStatEvents.processSkillStatEventsInternal,
{ batchSize: 100 },
);
crons.interval(
"global-stats-update",
{ hours: 24 },
internal.statsMaintenance.updateGlobalStatsAction,
{},
);
crons.interval("vt-pending-scans", { minutes: 5 }, internal.vt.pollPendingScans, {
batchSize: 100,
});
crons.interval("vt-cache-backfill", { minutes: 30 }, internal.vt.backfillActiveSkillsVTCache, {
batchSize: 100,
});
crons.interval(
"package-scan-backfill",
{ minutes: 30 },
internal.packages.backfillPackageReleaseScansInternal,
{ batchSize: 100 },
);
// Daily re-scan of all active skills at 3am UTC
crons.daily("vt-daily-rescan", { hourUTC: 3, minuteUTC: 0 }, internal.vt.rescanActiveSkills, {});
crons.interval(
"download-dedupe-prune",
{ hours: 24 },
internal.downloads.pruneDownloadDedupesInternal,
{},
);
export default crons;

270
convex/depRegistryScan.ts Normal file
View File

@ -0,0 +1,270 @@
import { v } from "convex/values";
import { internal } from "./_generated/api";
import type { Doc, Id } from "./_generated/dataModel";
import type { ActionCtx } from "./_generated/server";
import { internalAction, internalMutation, internalQuery } from "./functions";
import {
dedupeDeps,
depRegistryUrl,
parseDependencyFile,
SUPPORTED_DEP_REGISTRIES,
summarizeDepRegistryChecks,
type DepEntry,
type DepRegistryResult,
type DepRegistryUnresolved,
type SupportedDepRegistry,
} from "./lib/depRegistryScan";
import { readStorageText } from "./lib/packageRegistry";
const REQUEST_TIMEOUT_MS = 8_000;
const MAX_RETRIES = 2;
const BACKOFF_BASE_MS = 750;
const INTER_REQUEST_DELAY_MS = 100;
const MAX_DEPENDENCIES_PER_SCAN = 120;
const CACHE_TTL_EXISTS_MS = 30 * 24 * 60 * 60 * 1_000;
const CACHE_TTL_NOT_EXISTS_MS = 7 * 24 * 60 * 60 * 1_000;
const registryValidator = v.union(v.literal("pypi"), v.literal("npm"), v.literal("cargo"));
type RegistryCheck =
| { kind: "found"; httpStatus: number }
| { kind: "missing"; httpStatus: number }
| { kind: "unresolved"; reason: string };
function isSupportedRegistry(value: string): value is SupportedDepRegistry {
return (SUPPORTED_DEP_REGISTRIES as readonly string[]).includes(value);
}
async function wait(ms: number) {
await new Promise((resolve) => setTimeout(resolve, ms));
}
async function checkRegistry(dep: DepEntry): Promise<RegistryCheck> {
const headers: Record<string, string> = { Accept: "application/json" };
if (dep.registry === "cargo") {
headers["User-Agent"] = "ClawHub-DepRegistryScan/1.0 (https://clawhub.ai)";
}
let lastStatus: number | undefined;
for (let attempt = 0; attempt <= MAX_RETRIES; attempt += 1) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
try {
const response = await fetch(depRegistryUrl(dep.registry, dep.name), {
method: "GET",
headers,
signal: controller.signal,
});
clearTimeout(timeout);
lastStatus = response.status;
if (response.status === 200) return { kind: "found", httpStatus: response.status };
if (response.status === 404) return { kind: "missing", httpStatus: response.status };
if (response.status !== 429 && response.status < 500) {
return {
kind: "unresolved",
reason: `unexpected HTTP ${response.status}`,
};
}
} catch (error) {
clearTimeout(timeout);
if (attempt === MAX_RETRIES) {
return {
kind: "unresolved",
reason: error instanceof Error ? error.message : "network error",
};
}
}
if (attempt < MAX_RETRIES) {
await wait(2 ** attempt * BACKOFF_BASE_MS);
}
}
return {
kind: "unresolved",
reason: lastStatus ? `HTTP ${lastStatus}` : "network error",
};
}
async function extractDependencies(ctx: Pick<ActionCtx, "storage">, version: Doc<"skillVersions">) {
const entries: DepEntry[] = [];
for (const file of version.files) {
const basename = file.path.split("/").pop()?.toLowerCase() ?? "";
if (
basename !== "requirements.txt" &&
basename !== "requirements-dev.txt" &&
basename !== "requirements_dev.txt" &&
basename !== "requirements-test.txt" &&
basename !== "requirements_test.txt" &&
basename !== "package.json" &&
basename !== "cargo.toml" &&
basename !== "pyproject.toml"
) {
continue;
}
const content = await readStorageText(ctx, file.storageId);
entries.push(...parseDependencyFile(file.path, content));
}
return dedupeDeps(entries);
}
export const lookupCacheInternal = internalQuery({
args: {
registry: registryValidator,
name: v.string(),
},
handler: async (ctx, args): Promise<Doc<"depRegistryCache"> | null> => {
return ctx.db
.query("depRegistryCache")
.withIndex("by_registry_name", (q) => q.eq("registry", args.registry).eq("name", args.name))
.unique();
},
});
export const upsertCacheInternal = internalMutation({
args: {
registry: registryValidator,
name: v.string(),
exists: v.boolean(),
httpStatus: v.number(),
checkedAt: v.number(),
},
handler: async (ctx, args) => {
const existing = await ctx.db
.query("depRegistryCache")
.withIndex("by_registry_name", (q) => q.eq("registry", args.registry).eq("name", args.name))
.unique();
const patch = {
registry: args.registry,
name: args.name,
exists: args.exists,
httpStatus: args.httpStatus,
checkedAt: args.checkedAt,
};
if (existing) {
await ctx.db.patch(existing._id, patch);
} else {
await ctx.db.insert("depRegistryCache", patch);
}
},
});
export const getRetryableVersionIdsInternal = internalQuery({
args: {
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const limit = Math.min(Math.max(args.limit ?? 25, 1), 100);
const versions = await ctx.db
.query("skillVersions")
.withIndex("by_dep_registry_scan_status_and_created", (q) =>
q.eq("depRegistryScanStatus", "error"),
)
.order("desc")
.take(limit);
return versions.map((version) => version._id);
},
});
async function checkWithCache(ctx: ActionCtx, dep: DepEntry) {
const now = Date.now();
const cached = (await ctx.runQuery(internal.depRegistryScan.lookupCacheInternal, {
registry: dep.registry,
name: dep.name,
})) as Doc<"depRegistryCache"> | null;
if (cached) {
const ttl = cached.exists ? CACHE_TTL_EXISTS_MS : CACHE_TTL_NOT_EXISTS_MS;
if (now - cached.checkedAt < ttl) {
return cached.exists
? ({ kind: "found", httpStatus: cached.httpStatus } as const)
: ({ kind: "missing", httpStatus: cached.httpStatus } as const);
}
}
const check = await checkRegistry(dep);
if (check.kind !== "unresolved") {
await ctx.runMutation(internal.depRegistryScan.upsertCacheInternal, {
registry: dep.registry,
name: dep.name,
exists: check.kind === "found",
httpStatus: check.httpStatus,
checkedAt: now,
});
}
return check;
}
export const checkDependencyRegistries = internalAction({
args: { versionId: v.id("skillVersions") },
handler: async (ctx, args) => {
const version = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
versionId: args.versionId,
})) as Doc<"skillVersions"> | null;
if (!version) return null;
if (version.depRegistryAnalysis && version.depRegistryAnalysis.status !== "error") {
return version.depRegistryAnalysis;
}
const deps = await extractDependencies(ctx, version);
const checkableDeps = deps.slice(0, MAX_DEPENDENCIES_PER_SCAN);
const deferredDeps = deps.slice(MAX_DEPENDENCIES_PER_SCAN);
const results: DepRegistryResult[] = [];
const unresolved: DepRegistryUnresolved[] = deferredDeps.map((dep) => ({
...dep,
reason: "dependency scan limit reached",
}));
for (const dep of checkableDeps) {
if (!isSupportedRegistry(dep.registry)) continue;
const check = await checkWithCache(ctx, dep);
if (check.kind === "unresolved") {
unresolved.push({ ...dep, reason: check.reason });
} else {
results.push({
...dep,
exists: check.kind === "found",
httpStatus: check.httpStatus,
});
}
await wait(INTER_REQUEST_DELAY_MS);
}
const analysis = summarizeDepRegistryChecks({
results,
unresolved,
checkedAt: Date.now(),
});
await ctx.runMutation(internal.skills.updateVersionDepRegistryAnalysisInternal, {
versionId: args.versionId,
depRegistryAnalysis: analysis,
});
return analysis;
},
});
export const rescanErrorDepRegistryVersions = internalAction({
args: {
batchSize: v.optional(v.number()),
},
handler: async (ctx, args) => {
const versionIds = (await ctx.runQuery(
internal.depRegistryScan.getRetryableVersionIdsInternal,
{ limit: args.batchSize ?? 25 },
)) as Id<"skillVersions">[];
let scheduled = 0;
for (const versionId of versionIds) {
await ctx.scheduler.runAfter(
scheduled * 2_000,
internal.depRegistryScan.checkDependencyRegistries,
{
versionId,
},
);
scheduled += 1;
}
return { scheduled };
},
});

View File

@ -0,0 +1,175 @@
import { describe, expect, it } from "vitest";
import { seedRescanUxFixturesHandler } from "./devSeed";
import { MAX_OWNER_RESCAN_REQUESTS_PER_RELEASE } from "./model/rescans/policy";
function chainEq(constraints: Record<string, unknown>) {
return {
eq(field: string, value: unknown) {
constraints[field] = value;
return chainEq(constraints);
},
};
}
function matches(doc: Record<string, unknown>, constraints: Record<string, unknown>) {
return Object.entries(constraints).every(([key, value]) => doc[key] === value);
}
function createDb() {
const tables: Record<string, Array<Record<string, unknown> & { _id: string }>> = {};
const counters: Record<string, number> = {};
const list = (table: string) => {
tables[table] ??= [];
return tables[table];
};
const db = {
get: async (id: string) => {
const table = id.split(":")[0] ?? "";
return list(table).find((doc) => doc._id === id) ?? null;
},
insert: async (table: string, doc: Record<string, unknown>) => {
counters[table] = (counters[table] ?? 0) + 1;
const inserted = {
_id: `${table}:${counters[table]}`,
_creationTime: counters[table],
...doc,
};
list(table).push(inserted);
return inserted._id;
},
patch: async (id: string, patch: Record<string, unknown>) => {
const table = id.split(":")[0] ?? "";
const doc = list(table).find((candidate) => candidate._id === id);
if (doc) Object.assign(doc, patch);
},
delete: async (id: string) => {
const table = id.split(":")[0] ?? "";
const rows = list(table);
const index = rows.findIndex((doc) => doc._id === id);
if (index !== -1) rows.splice(index, 1);
},
query: (table: string) => ({
withIndex: (_name: string, build: (q: ReturnType<typeof chainEq>) => unknown) => {
const constraints: Record<string, unknown> = {};
build(chainEq(constraints));
const matched = () =>
list(table).filter((doc) => matches(doc as Record<string, unknown>, constraints));
return {
collect: async () => matched(),
unique: async () => matched()[0] ?? null,
order: () => ({
collect: async () => matched(),
}),
};
},
}),
};
return { db, tables };
}
describe("devSeed rescan UX fixtures", () => {
it("seeds flagged local owner inventory and deterministic rescan counts idempotently", async () => {
const { db, tables } = createDb();
const args = {
flaggedSkillStorageId: "storage:skill",
flaggedSkillMd: "# Flagged skill",
scannedSkillStorageId: "storage:scanned-skill",
scannedSkillMd: "# Scanned skill",
flaggedPluginStorageId: "storage:plugin",
flaggedPluginReadme: "# Flagged plugin",
scannedPluginStorageId: "storage:scanned-plugin",
scannedPluginReadme: "# Scanned plugin",
};
await seedRescanUxFixturesHandler({ db } as never, args as never);
await seedRescanUxFixturesHandler({ db } as never, args as never);
await seedRescanUxFixturesHandler({ db } as never, { ...args, reset: true } as never);
expect(tables.users).toHaveLength(1);
expect(tables.users?.[0]).toEqual(expect.objectContaining({ handle: "local" }));
expect(tables.publishers).toHaveLength(1);
expect(tables.skills).toHaveLength(2);
expect(tables.skills?.find((skill) => skill.slug === "local-flagged-wallet-sync")).toEqual(
expect.objectContaining({
ownerUserId: tables.users?.[0]?._id,
ownerPublisherId: tables.publishers?.[0]?._id,
moderationStatus: "hidden",
moderationVerdict: "malicious",
}),
);
expect(tables.skills?.find((skill) => skill.slug === "local-agentic-risk-demo")).toEqual(
expect.objectContaining({
ownerUserId: tables.users?.[0]?._id,
ownerPublisherId: tables.publishers?.[0]?._id,
moderationStatus: "active",
moderationVerdict: "suspicious",
}),
);
expect(tables.packages).toHaveLength(2);
expect(tables.packages?.find((pkg) => pkg.name === "local-flagged-runtime-plugin")).toEqual(
expect.objectContaining({
ownerUserId: tables.users?.[0]?._id,
ownerPublisherId: tables.publishers?.[0]?._id,
scanStatus: "malicious",
}),
);
expect(tables.packages?.find((pkg) => pkg.name === "local-scanned-runtime-plugin")).toEqual(
expect.objectContaining({
ownerUserId: tables.users?.[0]?._id,
ownerPublisherId: tables.publishers?.[0]?._id,
scanStatus: "suspicious",
}),
);
const scannedPackage = tables.packages?.find(
(pkg) => pkg.name === "local-scanned-runtime-plugin",
);
const scannedRelease = tables.packageReleases?.find(
(release) => release.packageId === scannedPackage?._id,
);
expect(scannedRelease).toEqual(
expect.objectContaining({
sha256hash: "seeded-scanned-plugin-hash",
vtAnalysis: expect.objectContaining({ status: "clean" }),
llmAnalysis: expect.objectContaining({ status: "suspicious" }),
staticScan: expect.objectContaining({ status: "suspicious" }),
}),
);
const scannedSkill = tables.skills?.find((skill) => skill.slug === "local-agentic-risk-demo");
const scannedSkillVersion = tables.skillVersions?.find(
(version) => version.skillId === scannedSkill?._id,
);
expect(scannedSkillVersion).toEqual(
expect.objectContaining({
sha256hash: "seeded-agentic-risk-skill-hash",
vtAnalysis: expect.objectContaining({ status: "clean" }),
llmAnalysis: expect.objectContaining({
status: "suspicious",
riskSummary: expect.objectContaining({
sensitive_data_protection: expect.objectContaining({ status: "concern" }),
}),
agenticRiskFindings: expect.arrayContaining([
expect.objectContaining({
categoryId: "ASI06",
riskBucket: "sensitive_data_protection",
status: "concern",
evidence: expect.objectContaining({ path: "SKILL.md" }),
}),
]),
}),
staticScan: expect.objectContaining({ status: "suspicious" }),
}),
);
const skillRequests =
tables.rescanRequests?.filter((request) => request.targetKind === "skill") ?? [];
const pluginRequests =
tables.rescanRequests?.filter((request) => request.targetKind === "plugin") ?? [];
expect(skillRequests).toHaveLength(1);
expect(pluginRequests).toHaveLength(MAX_OWNER_RESCAN_REQUESTS_PER_RELEASE);
});
});

File diff suppressed because it is too large Load Diff

557
convex/devSeedExtra.ts Normal file
View File

@ -0,0 +1,557 @@
/**
* Extra seed skills for pagination testing.
*
* This file contains 50 placeholder skills to test pagination behavior.
* Run with: bunx convex run internal.devSeedExtra.seedExtraSkillsInternal
* Or with reset: bunx convex run internal.devSeedExtra.seedExtraSkillsInternal '{"reset": true}'
*/
import { v } from "convex/values";
import { internal } from "./_generated/api";
import type { Id } from "./_generated/dataModel";
import type { ActionCtx } from "./_generated/server";
import { internalAction, internalMutation } from "./functions";
import { parseClawdisMetadata, parseFrontmatter } from "./lib/skills";
type SeedSkillSpec = {
slug: string;
displayName: string;
summary: string;
version: string;
metadata: Record<string, unknown>;
rawSkillMd: string;
};
function makeSkill(
slug: string,
displayName: string,
summary: string,
envVars: string[] = [],
commands: string[] = ["help", "status", "run"],
): SeedSkillSpec {
const cliHelp = `${slug} - ${summary}
Usage:
${slug} [command]
Commands:
${commands.map((cmd) => ` ${cmd.padEnd(12)} Run ${cmd} operation`).join("\n")}
Flags:
-h, --help Show help
--json Output as JSON
`;
const rawSkillMd = `---
name: ${slug}
description: ${summary}
---
# ${displayName}
## CLI
\`\`\`bash
${commands.map((cmd) => `${slug} ${cmd}`).join("\n")}
\`\`\`
## Usage
Use this skill to ${summary.toLowerCase()}.
`;
return {
slug,
displayName,
summary,
version: "0.1.0",
metadata: {
clawdbot: {
nix: {
plugin: `github:example/${slug}`,
systems: ["aarch64-darwin", "x86_64-linux"],
},
config: {
requiredEnv: envVars,
},
cliHelp,
},
},
rawSkillMd,
};
}
// 50 placeholder skills for pagination testing
const EXTRA_SEED_SKILLS: SeedSkillSpec[] = [
// DevOps & Infrastructure (10)
makeSkill(
"kubectl-helper",
"Kubectl Helper",
"Simplified kubectl commands for common Kubernetes operations.",
["KUBECONFIG"],
["pods", "logs", "exec", "describe", "apply"],
),
makeSkill(
"terraform-runner",
"Terraform Runner",
"Execute Terraform plans and applies with safety checks.",
["TF_VAR_region", "AWS_PROFILE"],
["plan", "apply", "destroy", "output", "state"],
),
makeSkill(
"ansible-exec",
"Ansible Exec",
"Run Ansible playbooks and ad-hoc commands.",
["ANSIBLE_INVENTORY"],
["playbook", "adhoc", "inventory", "facts", "vault"],
),
makeSkill(
"docker-compose-mgr",
"Docker Compose Manager",
"Manage Docker Compose stacks and services.",
["DOCKER_HOST"],
["up", "down", "logs", "ps", "restart"],
),
makeSkill(
"k9s-wrapper",
"K9s Wrapper",
"Interactive Kubernetes cluster management via K9s.",
["KUBECONFIG"],
["launch", "contexts", "namespaces", "pods", "logs"],
),
makeSkill(
"helm-charts",
"Helm Charts",
"Manage Helm chart deployments and releases.",
["KUBECONFIG", "HELM_REPO"],
["install", "upgrade", "rollback", "list", "search"],
),
makeSkill(
"prometheus-alerts",
"Prometheus Alerts",
"Query Prometheus metrics and manage alerting rules.",
["PROMETHEUS_URL"],
["query", "alerts", "rules", "targets", "status"],
),
makeSkill(
"grafana-dash",
"Grafana Dashboards",
"Create and manage Grafana dashboards programmatically.",
["GRAFANA_URL", "GRAFANA_API_KEY"],
["list", "export", "import", "create", "delete"],
),
makeSkill(
"nginx-config",
"Nginx Config",
"Generate and validate Nginx configuration files.",
["NGINX_CONF_DIR"],
["generate", "validate", "reload", "test", "sites"],
),
makeSkill(
"jenkins-jobs",
"Jenkins Jobs",
"Manage Jenkins jobs and pipelines.",
["JENKINS_URL", "JENKINS_TOKEN"],
["list", "build", "status", "logs", "config"],
),
// Productivity (8)
makeSkill(
"todoist-sync",
"Todoist Sync",
"Sync and manage Todoist tasks from the command line.",
["TODOIST_API_TOKEN"],
["list", "add", "complete", "projects", "labels"],
),
makeSkill(
"notion-backup",
"Notion Backup",
"Export and backup Notion workspaces.",
["NOTION_TOKEN"],
["export", "backup", "restore", "pages", "databases"],
),
makeSkill(
"gcal-manager",
"Google Calendar Manager",
"Manage Google Calendar events and schedules.",
["GOOGLE_CREDENTIALS_FILE"],
["events", "create", "delete", "calendars", "reminders"],
),
makeSkill(
"time-tracker",
"Time Tracker",
"Track time spent on projects and tasks.",
["TIMETRACK_DB"],
["start", "stop", "status", "report", "projects"],
),
makeSkill(
"email-digest",
"Email Digest",
"Generate email digests and summaries.",
["IMAP_SERVER", "IMAP_USER"],
["fetch", "digest", "search", "folders", "unread"],
),
makeSkill(
"habit-tracker",
"Habit Tracker",
"Track daily habits and streaks.",
["HABITS_DB"],
["log", "streak", "stats", "habits", "remind"],
),
makeSkill(
"bookmark-sync",
"Bookmark Sync",
"Sync bookmarks across browsers and devices.",
["BOOKMARKS_DIR"],
["sync", "export", "import", "search", "tags"],
),
makeSkill(
"notes-export",
"Notes Export",
"Export notes to various formats.",
["NOTES_DIR"],
["export", "convert", "search", "list", "tags"],
),
// Media & Entertainment (6)
makeSkill(
"spotify-ctl",
"Spotify Control",
"Control Spotify playback from the terminal.",
["SPOTIFY_CLIENT_ID", "SPOTIFY_CLIENT_SECRET"],
["play", "pause", "next", "prev", "search"],
),
makeSkill(
"plex-manager",
"Plex Manager",
"Manage Plex media libraries and playback.",
["PLEX_URL", "PLEX_TOKEN"],
["libraries", "scan", "search", "play", "sessions"],
),
makeSkill(
"ytdl-wrapper",
"YouTube Downloader",
"Download videos from YouTube and other platforms.",
["YTDL_OUTPUT_DIR"],
["download", "info", "playlist", "audio", "formats"],
),
makeSkill(
"podcast-dl",
"Podcast Downloader",
"Download and manage podcast episodes.",
["PODCAST_DIR"],
["subscribe", "download", "list", "play", "search"],
),
makeSkill(
"audiobook-player",
"Audiobook Player",
"Manage and play audiobook collections.",
["AUDIOBOOK_DIR"],
["play", "pause", "bookmark", "list", "progress"],
),
makeSkill(
"music-lib",
"Music Library",
"Organize and query local music libraries.",
["MUSIC_DIR"],
["scan", "search", "play", "playlist", "stats"],
),
// Smart Home (8)
makeSkill(
"hass-control",
"Home Assistant Control",
"Control Home Assistant entities and automations.",
["HASS_URL", "HASS_TOKEN"],
["entities", "services", "automations", "scenes", "history"],
),
makeSkill(
"zigbee-mqtt",
"Zigbee2MQTT",
"Manage Zigbee devices via MQTT.",
["MQTT_BROKER", "ZIGBEE_TOPIC"],
["devices", "pair", "remove", "rename", "groups"],
),
makeSkill(
"tasmota-ctl",
"Tasmota Control",
"Control Tasmota-flashed devices.",
["TASMOTA_HOSTS"],
["status", "power", "config", "update", "backup"],
),
makeSkill(
"esphome-mgr",
"ESPHome Manager",
"Manage ESPHome device configurations.",
["ESPHOME_DIR"],
["compile", "upload", "logs", "dashboard", "config"],
),
makeSkill(
"mqtt-broker",
"MQTT Broker",
"Interact with MQTT brokers for IoT messaging.",
["MQTT_BROKER", "MQTT_USER"],
["pub", "sub", "topics", "clients", "stats"],
),
makeSkill(
"hue-lights",
"Philips Hue",
"Control Philips Hue lights and scenes.",
["HUE_BRIDGE_IP", "HUE_API_KEY"],
["lights", "scenes", "groups", "schedules", "sensors"],
),
makeSkill(
"smart-thermo",
"Smart Thermostat",
"Control smart thermostats and HVAC systems.",
["THERMOSTAT_API_KEY"],
["status", "set", "schedule", "history", "zones"],
),
makeSkill(
"cam-viewer",
"Camera Viewer",
"View and manage security camera feeds.",
["CAMERA_URLS"],
["list", "snapshot", "stream", "record", "events"],
),
// Finance (5)
makeSkill(
"budget-track",
"Budget Tracker",
"Track budgets and spending across categories.",
["BUDGET_DB"],
["summary", "add", "categories", "report", "goals"],
),
makeSkill(
"crypto-watch",
"Crypto Watcher",
"Monitor cryptocurrency prices and portfolios.",
["CRYPTO_API_KEY"],
["prices", "portfolio", "alerts", "history", "convert"],
),
makeSkill(
"stock-alerts",
"Stock Alerts",
"Set up stock price alerts and notifications.",
["STOCK_API_KEY"],
["quote", "watch", "alerts", "portfolio", "news"],
),
makeSkill(
"expense-cat",
"Expense Categorizer",
"Automatically categorize expenses.",
["EXPENSE_DB"],
["import", "categorize", "report", "rules", "export"],
),
makeSkill(
"invoice-gen",
"Invoice Generator",
"Generate and manage invoices.",
["INVOICE_DIR", "COMPANY_INFO"],
["create", "list", "send", "paid", "overdue"],
),
// Communication (5)
makeSkill(
"slack-bot",
"Slack Bot",
"Interact with Slack channels and messages.",
["SLACK_TOKEN"],
["send", "channels", "users", "search", "files"],
),
makeSkill(
"discord-mgr",
"Discord Manager",
"Manage Discord servers and messages.",
["DISCORD_TOKEN"],
["send", "servers", "channels", "members", "roles"],
),
makeSkill(
"telegram-bot",
"Telegram Bot",
"Send and receive Telegram messages.",
["TELEGRAM_BOT_TOKEN"],
["send", "receive", "chats", "files", "inline"],
),
makeSkill(
"matrix-cli",
"Matrix CLI",
"Interact with Matrix chat rooms.",
["MATRIX_HOMESERVER", "MATRIX_TOKEN"],
["send", "rooms", "join", "leave", "sync"],
),
makeSkill(
"irc-bridge",
"IRC Bridge",
"Bridge IRC channels to other platforms.",
["IRC_SERVER", "IRC_NICK"],
["connect", "join", "send", "channels", "users"],
),
// Data & Analytics (5)
makeSkill(
"pg-queries",
"PostgreSQL Queries",
"Execute PostgreSQL queries and manage databases.",
["DATABASE_URL"],
["query", "tables", "schema", "backup", "restore"],
),
makeSkill(
"clickhouse-ql",
"ClickHouse Queries",
"Run ClickHouse analytics queries.",
["CLICKHOUSE_URL"],
["query", "tables", "insert", "system", "optimize"],
),
makeSkill(
"redis-cli",
"Redis CLI",
"Interact with Redis cache and data structures.",
["REDIS_URL"],
["get", "set", "keys", "info", "flush"],
),
makeSkill(
"elastic-search",
"Elasticsearch",
"Search and manage Elasticsearch indices.",
["ELASTICSEARCH_URL"],
["search", "index", "mapping", "cluster", "aliases"],
),
makeSkill(
"mongo-shell",
"MongoDB Shell",
"Query and manage MongoDB collections.",
["MONGODB_URI"],
["find", "insert", "update", "delete", "aggregate"],
),
// Security (3)
makeSkill(
"vault-secrets",
"Vault Secrets",
"Manage secrets in HashiCorp Vault.",
["VAULT_ADDR", "VAULT_TOKEN"],
["read", "write", "list", "delete", "seal"],
),
makeSkill(
"gpg-keys",
"GPG Keys",
"Manage GPG keys and encryption.",
["GNUPGHOME"],
["list", "generate", "export", "import", "encrypt"],
),
makeSkill(
"ssh-rotate",
"SSH Key Rotator",
"Rotate and manage SSH keys.",
["SSH_KEY_DIR"],
["generate", "rotate", "deploy", "list", "revoke"],
),
// CJK Language Support (2)
makeSkill(
"nihongo-check",
"日本語チェッカー",
"日本語文章の文法チェックと翻訳支援ツール。Japanese grammar checker and translation assistant.",
["NIHONGO_API_KEY"],
["check", "translate", "kanji", "grammar", "vocabulary"],
),
makeSkill(
"hangukgeo-helper",
"한국어 도우미",
"한국어 학습 보조 도구입니다. Korean language learning assistant with vocabulary and grammar support.",
["HANGUL_API_KEY"],
["learn", "quiz", "vocabulary", "grammar", "pronunciation"],
),
];
function injectMetadata(rawSkillMd: string, metadata: Record<string, unknown>) {
const frontmatterEnd = rawSkillMd.indexOf("\n---", 3);
if (frontmatterEnd === -1) return rawSkillMd;
return `${rawSkillMd.slice(0, frontmatterEnd)}\nmetadata: ${JSON.stringify(
metadata,
)}${rawSkillMd.slice(frontmatterEnd)}`;
}
function randomStats() {
return {
downloads: Math.floor(Math.random() * 5000),
stars: Math.floor(Math.random() * 500),
installsCurrent: Math.floor(Math.random() * 200),
installsAllTime: Math.floor(Math.random() * 1000),
};
}
export const applyRandomStats = internalMutation({
args: {
skillId: v.id("skills"),
stats: v.object({
downloads: v.number(),
stars: v.number(),
installsCurrent: v.number(),
installsAllTime: v.number(),
}),
},
handler: async (ctx, args) => {
await ctx.db.patch(args.skillId, {
statsDownloads: args.stats.downloads,
statsStars: args.stats.stars,
statsInstallsCurrent: args.stats.installsCurrent,
statsInstallsAllTime: args.stats.installsAllTime,
stats: {
downloads: args.stats.downloads,
stars: args.stats.stars,
installsCurrent: args.stats.installsCurrent,
installsAllTime: args.stats.installsAllTime,
versions: 1,
comments: 0,
},
});
},
});
export const seedExtraSkillsInternal = internalAction({
args: {
reset: v.optional(v.boolean()),
},
handler: async (ctx: ActionCtx, args) => {
const results: Array<{ slug: string; ok: boolean; skipped?: boolean }> = [];
for (const spec of EXTRA_SEED_SKILLS) {
const skillMd = injectMetadata(spec.rawSkillMd, spec.metadata);
const frontmatter = parseFrontmatter(skillMd);
const clawdis = parseClawdisMetadata(frontmatter);
const storageId = await ctx.storage.store(new Blob([skillMd], { type: "text/markdown" }));
const result = (await ctx.runMutation(internal.devSeed.seedSkillMutation, {
reset: args.reset,
storageId,
metadata: spec.metadata,
frontmatter,
clawdis,
skillMd,
slug: spec.slug,
displayName: spec.displayName,
summary: spec.summary,
version: spec.version,
})) as { ok: boolean; skipped?: boolean; skillId?: string };
// Apply random stats after creation (only if not skipped)
if (result.skillId && !result.skipped) {
const stats = randomStats();
await ctx.runMutation(internal.devSeedExtra.applyRandomStats, {
skillId: result.skillId as Id<"skills">,
stats,
});
}
results.push({ slug: spec.slug, ok: result.ok, skipped: result.skipped });
}
const created = results.filter((r) => !r.skipped).length;
const skipped = results.filter((r) => r.skipped).length;
return { ok: true, total: results.length, created, skipped };
},
});

144
convex/downloads.test.ts Normal file
View File

@ -0,0 +1,144 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import type { ActionCtx } from "./_generated/server";
import { __test, downloadZipHandler } from "./downloads";
type RateLimitArgs = { key: string; limit: number; windowMs: number };
function isRateLimitArgs(args: unknown): args is RateLimitArgs {
if (!args || typeof args !== "object") return false;
const value = args as Record<string, unknown>;
return (
typeof value.key === "string" &&
typeof value.limit === "number" &&
typeof value.windowMs === "number"
);
}
const okRate = () => ({
allowed: true,
remaining: 10,
limit: 100,
resetAt: Date.now() + 60_000,
});
describe("downloads helpers", () => {
afterEach(() => {
vi.unstubAllEnvs();
vi.unstubAllGlobals();
});
it("calculates hour start boundaries", () => {
const hour = 3_600_000;
expect(__test.getHourStart(0)).toBe(0);
expect(__test.getHourStart(hour - 1)).toBe(0);
expect(__test.getHourStart(hour)).toBe(hour);
expect(__test.getHourStart(hour + 1)).toBe(hour);
});
it("prefers user identity when token user exists", () => {
const request = new Request("https://example.com", {
headers: { "cf-connecting-ip": "1.2.3.4" },
});
expect(__test.getDownloadIdentityValue(request, "users_123")).toBe("user:users_123");
});
it("uses cf-connecting-ip for anonymous identity", () => {
const request = new Request("https://example.com", {
headers: { "cf-connecting-ip": "1.2.3.4" },
});
expect(__test.getDownloadIdentityValue(request, null)).toBe("ip:1.2.3.4");
});
it("falls back to forwarded ip when explicitly enabled", () => {
vi.stubEnv("TRUST_FORWARDED_IPS", "true");
const request = new Request("https://example.com", {
headers: { "x-forwarded-for": "10.0.0.1, 10.0.0.2" },
});
expect(__test.getDownloadIdentityValue(request, null)).toBe("ip:10.0.0.1");
});
it("returns null when user and ip are missing", () => {
const request = new Request("https://example.com");
expect(__test.getDownloadIdentityValue(request, null)).toBeNull();
});
it("schedules zip download stats outside the response path", async () => {
class MockResponse {
status: number;
headers: Headers;
constructor(_body?: BodyInit | null, init?: ResponseInit) {
this.status = init?.status ?? 200;
this.headers = new Headers(init?.headers);
}
}
vi.stubGlobal("Response", MockResponse as unknown as typeof Response);
const runQuery = vi.fn(async (_query: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
if ("slug" in args) {
return {
skill: {
_id: "skills:1",
ownerUserId: "users:1",
slug: "demo",
tags: {},
latestVersionId: "skillVersions:1",
},
moderationInfo: null,
};
}
if ("versionId" in args) {
return {
_id: "skillVersions:1",
version: "1.0.0",
createdAt: 3,
files: [{ path: "SKILL.md", storageId: "_storage:1" }],
softDeletedAt: undefined,
};
}
return null;
});
const runMutation = vi.fn(async (mutation: unknown, args: Record<string, unknown>) => {
if (isRateLimitArgs(args)) return okRate();
return { mutation, args };
});
const runAfter = vi.fn();
const storageGet = vi.fn().mockResolvedValue(new Blob(["hello"], { type: "text/markdown" }));
const response = await downloadZipHandler(
{
runQuery,
runMutation,
scheduler: { runAfter },
storage: { get: storageGet },
} as unknown as ActionCtx,
new Request("https://example.com/api/v1/download?slug=demo", {
headers: { "cf-connecting-ip": "1.2.3.4" },
}),
);
expect(response.status).toBe(200);
expect(response.headers.get("Content-Type")).toBe("application/zip");
expect(storageGet).toHaveBeenCalledWith("_storage:1");
const recordCalls = runAfter.mock.calls.filter(([, , args]) => {
if (!args || typeof args !== "object") return false;
const value = args as Record<string, unknown>;
return (
value.skillId === "skills:1" &&
typeof value.identityHash === "string" &&
typeof value.hourStart === "number"
);
});
expect(recordCalls).toHaveLength(1);
expect(recordCalls[0]?.[0]).toEqual(expect.any(Number));
expect(recordCalls[0]?.[0]).toBeGreaterThanOrEqual(0);
expect(recordCalls[0]?.[0]).toBeLessThan(60_000);
expect(recordCalls[0]?.[2]).toEqual({
skillId: "skills:1",
identityHash: expect.any(String),
hourStart: expect.any(Number),
});
});
});

View File

@ -1,81 +1,226 @@
import { v } from 'convex/values'
import { zipSync } from 'fflate'
import { api } from './_generated/api'
import { httpAction, mutation } from './_generated/server'
import { applySkillStatDeltas, bumpDailySkillStats } from './lib/skillStats'
import { v } from "convex/values";
import { api, internal } from "./_generated/api";
import { httpAction, internalMutation } from "./functions";
import { getOptionalApiTokenUserId } from "./lib/apiTokenAuth";
import { corsHeaders, mergeHeaders } from "./lib/httpHeaders";
import { applyRateLimit, getClientIp } from "./lib/httpRateLimit";
import { buildDeterministicZip } from "./lib/skillZip";
import { hashToken } from "./lib/tokens";
import { insertStatEvent } from "./skillStatEvents";
export const downloadZip = httpAction(async (ctx, request) => {
const url = new URL(request.url)
const slug = url.searchParams.get('slug')?.trim().toLowerCase()
const versionParam = url.searchParams.get('version')?.trim()
const tagParam = url.searchParams.get('tag')?.trim()
const HOUR_MS = 3_600_000;
const DEDUPE_RETENTION_MS = 7 * 24 * HOUR_MS;
const PRUNE_BATCH_SIZE = 200;
const PRUNE_MAX_BATCHES = 50;
const DOWNLOAD_STAT_JITTER_MS = 60_000;
export async function downloadZipHandler(
ctx: Parameters<Parameters<typeof httpAction>[0]>[0],
request: Request,
) {
const url = new URL(request.url);
const slug = url.searchParams.get("slug")?.trim().toLowerCase();
const versionParam = url.searchParams.get("version")?.trim();
const tagParam = url.searchParams.get("tag")?.trim();
if (!slug) {
return new Response('Missing slug', { status: 400 })
return new Response("Missing slug", {
status: 400,
headers: corsHeaders(),
});
}
const skillResult = await ctx.runQuery(api.skills.getBySlug, { slug })
const rate = await applyRateLimit(ctx, request, "download");
if (!rate.ok) return rate.response;
const skillResult = await ctx.runQuery(api.skills.getBySlug, { slug });
if (!skillResult?.skill) {
return new Response('Skill not found', { status: 404 })
return new Response("Skill not found", {
status: 404,
headers: mergeHeaders(rate.headers, corsHeaders()),
});
}
const skill = skillResult.skill
let version = skillResult.latestVersion
// Block downloads based on moderation status.
const mod = skillResult.moderationInfo;
if (mod?.isMalwareBlocked) {
return new Response(
"Blocked: this skill has been flagged as malicious by VirusTotal and cannot be downloaded.",
{
status: 403,
headers: mergeHeaders(rate.headers, corsHeaders()),
},
);
}
if (mod?.isPendingScan) {
return new Response(
"This skill is pending a security scan by VirusTotal. Please try again in a few minutes.",
{
status: 423,
headers: mergeHeaders(rate.headers, corsHeaders()),
},
);
}
if (mod?.isRemoved) {
return new Response("This skill has been removed by a moderator.", {
status: 410,
headers: mergeHeaders(rate.headers, corsHeaders()),
});
}
if (mod?.isHiddenByMod) {
return new Response("This skill is currently unavailable.", {
status: 403,
headers: mergeHeaders(rate.headers, corsHeaders()),
});
}
const skill = skillResult.skill;
let version = skill.latestVersionId
? await ctx.runQuery(internal.skills.getVersionByIdInternal, {
versionId: skill.latestVersionId,
})
: null;
if (versionParam) {
version = await ctx.runQuery(api.skills.getVersionBySkillAndVersion, {
version = await ctx.runQuery(internal.skills.getVersionBySkillAndVersionInternal, {
skillId: skill._id,
version: versionParam,
})
});
} else if (tagParam) {
const versionId = skill.tags[tagParam]
const versionId = skill.tags[tagParam];
if (versionId) {
version = await ctx.runQuery(api.skills.getVersionById, { versionId })
version = await ctx.runQuery(internal.skills.getVersionByIdInternal, { versionId });
}
}
if (!version) {
return new Response('Version not found', { status: 404 })
return new Response("Version not found", {
status: 404,
headers: mergeHeaders(rate.headers, corsHeaders()),
});
}
if (version.softDeletedAt) {
return new Response('Version not available', { status: 410 })
return new Response("Version not available", {
status: 410,
headers: mergeHeaders(rate.headers, corsHeaders()),
});
}
const files: Record<string, Uint8Array> = {}
const entries: Array<{ path: string; bytes: Uint8Array }> = [];
for (const file of version.files) {
const blob = await ctx.storage.get(file.storageId)
if (!blob) continue
const buffer = new Uint8Array(await blob.arrayBuffer())
files[file.path] = buffer
const blob = await ctx.storage.get(file.storageId);
if (!blob) continue;
const buffer = new Uint8Array(await blob.arrayBuffer());
entries.push({ path: file.path, bytes: buffer });
}
const zipArray = buildDeterministicZip(entries, {
ownerId: String(skill.ownerUserId),
slug: skill.slug,
version: version.version,
publishedAt: version.createdAt,
});
const zipBlob = new Blob([zipArray], { type: "application/zip" });
const zipData = zipSync(files, { level: 6 })
const zipArray = Uint8Array.from(zipData)
const zipBlob = new Blob([zipArray], { type: 'application/zip' })
await ctx.runMutation(api.downloads.increment, { skillId: skill._id })
try {
const userId = await getOptionalApiTokenUserId(ctx, request);
const identity = getDownloadIdentityValue(request, userId ? String(userId) : null);
if (identity) {
await ctx.scheduler.runAfter(
Math.floor(Math.random() * DOWNLOAD_STAT_JITTER_MS),
internal.downloads.recordDownloadInternal,
{
skillId: skill._id,
identityHash: await hashToken(identity),
hourStart: getHourStart(Date.now()),
},
);
}
} catch {
// Best-effort metric path; do not fail downloads.
}
return new Response(zipBlob, {
status: 200,
headers: {
'Content-Type': 'application/zip',
'Content-Disposition': `attachment; filename="${slug}-${version.version}.zip"`,
'Cache-Control': 'private, max-age=60',
},
})
})
headers: mergeHeaders(
rate.headers,
{
"Content-Type": "application/zip",
"Content-Disposition": `attachment; filename="${slug}-${version.version}.zip"`,
"Cache-Control": "private, max-age=60",
},
corsHeaders(),
),
});
}
export const increment = mutation({
args: { skillId: v.id('skills') },
handler: async (ctx, args) => {
const skill = await ctx.db.get(args.skillId)
if (!skill) return
const now = Date.now()
const patch = applySkillStatDeltas(skill, { downloads: 1 })
await ctx.db.patch(skill._id, {
...patch,
updatedAt: now,
})
await bumpDailySkillStats(ctx, { skillId: skill._id, now, downloads: 1 })
export const downloadZip = httpAction(downloadZipHandler);
export const recordDownloadInternal = internalMutation({
args: {
skillId: v.id("skills"),
identityHash: v.string(),
hourStart: v.number(),
},
})
handler: async (ctx, args) => {
const existing = await ctx.db
.query("downloadDedupes")
.withIndex("by_skill_identity_hour", (q) =>
q
.eq("skillId", args.skillId)
.eq("identityHash", args.identityHash)
.eq("hourStart", args.hourStart),
)
.first();
if (existing) return;
await ctx.db.insert("downloadDedupes", {
skillId: args.skillId,
identityHash: args.identityHash,
hourStart: args.hourStart,
createdAt: Date.now(),
});
await insertStatEvent(ctx, {
skillId: args.skillId,
kind: "download",
});
},
});
export const pruneDownloadDedupesInternal = internalMutation({
args: {},
handler: async (ctx) => {
const cutoff = Date.now() - DEDUPE_RETENTION_MS;
for (let batches = 0; batches < PRUNE_MAX_BATCHES; batches += 1) {
const stale = await ctx.db
.query("downloadDedupes")
.withIndex("by_hour", (q) => q.lt("hourStart", cutoff))
.take(PRUNE_BATCH_SIZE);
if (stale.length === 0) break;
for (const entry of stale) {
await ctx.db.delete(entry._id);
}
if (stale.length < PRUNE_BATCH_SIZE) break;
}
},
});
export function getHourStart(timestamp: number) {
return Math.floor(timestamp / HOUR_MS) * HOUR_MS;
}
export function getDownloadIdentityValue(request: Request, userId: string | null) {
if (userId) return `user:${userId}`;
const ip = getClientIp(request);
if (!ip) return null;
return `ip:${ip}`;
}
export const __test = {
getHourStart,
getDownloadIdentityValue,
};

663
convex/functions.test.ts Normal file
View File

@ -0,0 +1,663 @@
/* @vitest-environment node */
import { describe, expect, it, vi } from "vitest";
import { internal } from "./_generated/api";
import {
isGitHubMirrorEligibleSkillDoc,
repointPackageLatestRelease,
scheduleGitHubBackupDeletionForSkill,
scheduleOwnerPublisherDigestSync,
syncPackageSearchDigestForPackageId,
syncPackageSearchDigestsForOwnerPublisherId,
syncPackageSearchDigestsForOwnerUserId,
syncSkillSearchDigestsForOwnerPublisherId,
} from "./functions";
describe("package digest sync", () => {
it("identifies GitHub mirror eligibility from skill visibility fields", () => {
expect(isGitHubMirrorEligibleSkillDoc({ softDeletedAt: undefined })).toBe(true);
expect(
isGitHubMirrorEligibleSkillDoc({
softDeletedAt: undefined,
moderationStatus: "active",
}),
).toBe(true);
expect(
isGitHubMirrorEligibleSkillDoc({
softDeletedAt: undefined,
moderationStatus: "hidden",
}),
).toBe(false);
expect(
isGitHubMirrorEligibleSkillDoc({
softDeletedAt: undefined,
moderationStatus: "removed",
}),
).toBe(false);
expect(isGitHubMirrorEligibleSkillDoc({ softDeletedAt: 123 })).toBe(false);
});
it("schedules GitHub mirror deletion for a skill using the owner handle", async () => {
const ctx = {
db: {
get: vi.fn(async (id: string) => {
if (id === "users:owner") {
return {
_id: "users:owner",
handle: "alice",
deletedAt: undefined,
deactivatedAt: undefined,
};
}
return null;
}),
query: vi.fn(() => ({
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
})),
})),
},
scheduler: {
runAfter: vi.fn(),
},
};
await scheduleGitHubBackupDeletionForSkill(
ctx as never,
{
slug: "hidden-skill",
ownerUserId: "users:owner",
ownerPublisherId: undefined,
softDeletedAt: 123,
moderationStatus: "hidden",
} as never,
);
expect(ctx.scheduler.runAfter).toHaveBeenCalledWith(
0,
internal.githubBackupsNode.deleteGitHubBackupForSlugInternal,
{
ownerHandle: "alice",
slug: "hidden-skill",
},
);
});
it("clears latestVersion when the current package release is soft-deleted", async () => {
const pkg = {
_id: "packages:demo",
name: "demo-plugin",
normalizedName: "demo-plugin",
displayName: "Demo Plugin",
family: "code-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:owner",
summary: "demo",
capabilityTags: ["tools"],
executesCode: true,
runtimeId: null,
softDeletedAt: undefined,
createdAt: 1,
updatedAt: 2,
latestReleaseId: "packageReleases:demo-2",
latestVersionSummary: { version: "2.0.0" },
verification: { tier: "community" },
};
const latestRelease = {
_id: "packageReleases:demo-2",
version: "2.0.0",
softDeletedAt: 10,
};
const owner = {
_id: "users:owner",
handle: "owner",
deletedAt: undefined,
deactivatedAt: undefined,
};
const ctx = {
db: {
get: vi.fn(async (id: string) => {
if (id === "packages:demo") return pkg;
if (id === "packageReleases:demo-2") return latestRelease;
if (id === "users:owner") return owner;
return null;
}),
query: vi.fn(() => ({
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
})),
patch: vi.fn(),
insert: vi.fn(),
delete: vi.fn(),
},
};
await syncPackageSearchDigestForPackageId(ctx as never, "packages:demo" as never);
expect(ctx.db.insert).toHaveBeenCalledWith(
"packageSearchDigest",
expect.objectContaining({
packageId: "packages:demo",
latestVersion: undefined,
ownerHandle: "owner",
}),
);
});
it("preserves latestVersion when the current package release is active", async () => {
const pkg = {
_id: "packages:demo",
name: "demo-plugin",
normalizedName: "demo-plugin",
displayName: "Demo Plugin",
family: "code-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:owner",
summary: "demo",
capabilityTags: ["tools"],
executesCode: true,
runtimeId: null,
softDeletedAt: undefined,
createdAt: 1,
updatedAt: 2,
latestReleaseId: "packageReleases:demo-2",
latestVersionSummary: { version: "2.0.0" },
verification: { tier: "community" },
};
const latestRelease = {
_id: "packageReleases:demo-2",
version: "2.0.0",
};
const owner = {
_id: "users:owner",
handle: "owner",
deletedAt: undefined,
deactivatedAt: undefined,
};
const ctx = {
db: {
get: vi.fn(async (id: string) => {
if (id === "packages:demo") return pkg;
if (id === "packageReleases:demo-2") return latestRelease;
if (id === "users:owner") return owner;
return null;
}),
query: vi.fn(() => ({
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
})),
patch: vi.fn(),
insert: vi.fn(),
delete: vi.fn(),
},
};
await syncPackageSearchDigestForPackageId(ctx as never, "packages:demo" as never);
expect(ctx.db.insert).toHaveBeenCalledWith(
"packageSearchDigest",
expect.objectContaining({
packageId: "packages:demo",
latestVersion: "2.0.0",
ownerHandle: "owner",
}),
);
});
it("repoints packages to the highest-version active release and restores its summary", async () => {
const pkg = {
_id: "packages:demo",
_creationTime: 1,
name: "demo-plugin",
normalizedName: "demo-plugin",
displayName: "Demo Plugin",
family: "code-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:owner",
summary: "latest summary",
tags: {
latest: "packageReleases:demo-2",
stable: "packageReleases:demo-2",
},
latestReleaseId: "packageReleases:demo-2",
latestVersionSummary: { version: "2.0.0" },
capabilityTags: ["new"],
executesCode: true,
compatibility: { openclaw: "^2.0.0" },
capabilities: { capabilityTags: ["new"], executesCode: true },
verification: { tier: "community" },
runtimeId: null,
softDeletedAt: undefined,
createdAt: 1,
updatedAt: 2,
};
const fallbackRelease = {
_id: "packageReleases:demo-1",
_creationTime: 10,
packageId: "packages:demo",
version: "1.0.0",
changelog: "old stable",
summary: "stable summary",
compatibility: { openclaw: "^1.0.0" },
capabilities: { capabilityTags: ["stable"], executesCode: false },
verification: { tier: "verified" },
distTags: ["stable"],
createdAt: 10,
softDeletedAt: undefined,
};
const legacyHotfixRelease = {
_id: "packageReleases:demo-legacy",
_creationTime: 20,
packageId: "packages:demo",
version: "0.9.9",
changelog: "legacy hotfix",
summary: "legacy summary",
compatibility: { openclaw: "^0.9.0" },
capabilities: { capabilityTags: ["legacy"], executesCode: false },
verification: { tier: "verified" },
distTags: ["legacy"],
createdAt: 20,
softDeletedAt: undefined,
};
const owner = {
_id: "users:owner",
handle: "owner",
deletedAt: undefined,
deactivatedAt: undefined,
};
const ctx = {
db: {
get: vi.fn(async (id: string) => {
if (id === "packages:demo") return pkg;
if (id === "packageReleases:demo-1") return fallbackRelease;
if (id === "users:owner") return owner;
return null;
}),
query: vi.fn((table: string) => {
if (table === "packageReleases") {
return {
withIndex: vi.fn(() => ({
order: vi.fn(() => ({
paginate: vi.fn().mockResolvedValue({
page: [legacyHotfixRelease, fallbackRelease],
isDone: true,
continueCursor: "",
}),
})),
})),
};
}
if (table === "packageSearchDigest") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
};
}
if (table === "packageCapabilitySearchDigest") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
};
}
throw new Error(`Unexpected table ${table}`);
}),
patch: vi.fn(),
insert: vi.fn(),
delete: vi.fn(),
},
};
await repointPackageLatestRelease(
ctx as never,
"packages:demo" as never,
"packageReleases:demo-2" as never,
);
expect(ctx.db.patch).toHaveBeenCalledWith("packageReleases:demo-1", {
distTags: ["stable", "latest"],
});
expect(ctx.db.patch).toHaveBeenCalledWith(
"packages:demo",
expect.objectContaining({
latestReleaseId: "packageReleases:demo-1",
tags: { latest: "packageReleases:demo-1" },
latestVersionSummary: expect.objectContaining({ version: "1.0.0" }),
summary: "stable summary",
capabilityTags: ["stable"],
executesCode: false,
}),
);
expect(ctx.db.insert).toHaveBeenCalledWith(
"packageSearchDigest",
expect.objectContaining({
latestVersion: "1.0.0",
ownerHandle: "owner",
}),
);
});
it("repoints bundle packages to the newest surviving release, not semver-looking versions", async () => {
const pkg = {
_id: "packages:bundle",
_creationTime: 1,
name: "demo-bundle",
normalizedName: "demo-bundle",
displayName: "Demo Bundle",
family: "bundle-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:owner",
summary: "latest summary",
tags: {
latest: "packageReleases:bundle-latest",
},
latestReleaseId: "packageReleases:bundle-latest",
latestVersionSummary: { version: "latest" },
capabilityTags: ["new"],
executesCode: false,
compatibility: { hosts: ["openclaw"] },
capabilities: { capabilityTags: ["new"], executesCode: false },
verification: { tier: "community" },
runtimeId: "bundle.runtime",
softDeletedAt: undefined,
createdAt: 1,
updatedAt: 2,
};
const semverLookingRelease = {
_id: "packageReleases:bundle-semver",
_creationTime: 10,
packageId: "packages:bundle",
version: "2.0.0",
changelog: "older semver",
summary: "older semver summary",
compatibility: { hosts: ["openclaw"] },
capabilities: { capabilityTags: ["semver"], executesCode: false },
verification: { tier: "verified" },
distTags: ["legacy"],
createdAt: 10,
softDeletedAt: undefined,
};
const newestRelease = {
_id: "packageReleases:bundle-newest",
_creationTime: 20,
packageId: "packages:bundle",
version: "2024-12",
changelog: "newest bundle build",
summary: "newest bundle summary",
compatibility: { hosts: ["openclaw"] },
capabilities: { capabilityTags: ["bundle"], executesCode: false },
verification: { tier: "verified" },
distTags: ["release-2024-12"],
createdAt: 20,
softDeletedAt: undefined,
};
const owner = {
_id: "users:owner",
handle: "owner",
deletedAt: undefined,
deactivatedAt: undefined,
};
const ctx = {
db: {
get: vi.fn(async (id: string) => {
if (id === "packages:bundle") return pkg;
if (id === "packageReleases:bundle-newest") return newestRelease;
if (id === "users:owner") return owner;
return null;
}),
query: vi.fn((table: string) => {
if (table === "packageReleases") {
return {
withIndex: vi.fn(() => ({
order: vi.fn(() => ({
paginate: vi.fn().mockResolvedValue({
page: [newestRelease, semverLookingRelease],
isDone: true,
continueCursor: "",
}),
})),
})),
};
}
if (table === "packageSearchDigest") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
};
}
if (table === "packageCapabilitySearchDigest") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
};
}
throw new Error(`Unexpected table ${table}`);
}),
patch: vi.fn(),
insert: vi.fn(),
delete: vi.fn(),
},
};
await repointPackageLatestRelease(
ctx as never,
"packages:bundle" as never,
"packageReleases:bundle-latest" as never,
);
expect(ctx.db.patch).toHaveBeenCalledWith("packageReleases:bundle-newest", {
distTags: ["release-2024-12", "latest"],
});
expect(ctx.db.patch).toHaveBeenCalledWith(
"packages:bundle",
expect.objectContaining({
latestReleaseId: "packageReleases:bundle-newest",
tags: { latest: "packageReleases:bundle-newest" },
latestVersionSummary: expect.objectContaining({ version: "2024-12" }),
summary: "newest bundle summary",
}),
);
expect(ctx.db.insert).toHaveBeenCalledWith(
"packageSearchDigest",
expect.objectContaining({
latestVersion: "2024-12",
ownerHandle: "owner",
}),
);
});
it("re-syncs package digests when an owner handle changes", async () => {
const owner = {
_id: "users:owner",
handle: "renamed",
deletedAt: undefined,
deactivatedAt: undefined,
};
const pkg = {
_id: "packages:demo",
_creationTime: 1,
name: "demo-plugin",
normalizedName: "demo-plugin",
displayName: "Demo Plugin",
family: "code-plugin",
channel: "community",
isOfficial: false,
ownerUserId: "users:owner",
summary: "demo",
tags: {},
latestReleaseId: undefined,
latestVersionSummary: undefined,
capabilityTags: [],
executesCode: false,
runtimeId: null,
softDeletedAt: undefined,
createdAt: 1,
updatedAt: 2,
verification: undefined,
};
const paginate = vi.fn().mockResolvedValueOnce({
page: [pkg],
isDone: true,
continueCursor: "",
});
const ctx = {
db: {
get: vi.fn(async (id: string) => {
if (id === "users:owner") return owner;
return null;
}),
query: vi.fn((table: string) => {
if (table === "packages") {
return {
withIndex: vi.fn(() => ({
paginate,
})),
};
}
if (table === "packageSearchDigest") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
};
}
if (table === "packageCapabilitySearchDigest") {
return {
withIndex: vi.fn(() => ({
unique: vi.fn().mockResolvedValue(null),
collect: vi.fn().mockResolvedValue([]),
})),
};
}
throw new Error(`Unexpected table ${table}`);
}),
patch: vi.fn(),
insert: vi.fn(),
delete: vi.fn(),
},
};
await syncPackageSearchDigestsForOwnerUserId(ctx as never, "users:owner" as never);
expect(paginate).toHaveBeenCalledWith({ cursor: null, numItems: 100 });
expect(ctx.db.insert).toHaveBeenCalledWith(
"packageSearchDigest",
expect.objectContaining({
packageId: "packages:demo",
ownerHandle: "renamed",
}),
);
});
});
describe("publisher digest scheduling", () => {
it("schedules package and skill digest sync in separate background mutations", async () => {
const ctx = {
scheduler: {
runAfter: vi.fn().mockResolvedValue(undefined),
},
};
await scheduleOwnerPublisherDigestSync(ctx as never, "publishers:demo" as never);
expect(ctx.scheduler.runAfter).toHaveBeenCalledTimes(2);
expect(ctx.scheduler.runAfter).toHaveBeenNthCalledWith(
1,
0,
internal.functions.syncPackageSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId: "publishers:demo" },
);
expect(ctx.scheduler.runAfter).toHaveBeenNthCalledWith(
2,
0,
internal.functions.syncSkillSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId: "publishers:demo" },
);
});
it("skips scheduling when the trigger context has no scheduler", async () => {
await expect(
scheduleOwnerPublisherDigestSync({} as never, "publishers:demo" as never),
).resolves.toBeUndefined();
});
it("continues owner-publisher package digest sync one page at a time", async () => {
const paginate = vi.fn().mockResolvedValue({
page: [],
isDone: false,
continueCursor: "next-packages",
});
const ctx = {
db: {
query: vi.fn(() => ({
withIndex: vi.fn(() => ({ paginate })),
})),
},
scheduler: {
runAfter: vi.fn().mockResolvedValue(undefined),
},
};
await syncPackageSearchDigestsForOwnerPublisherId(
ctx as never,
"publishers:demo" as never,
"current-packages",
);
expect(paginate).toHaveBeenCalledTimes(1);
expect(paginate).toHaveBeenCalledWith({ cursor: "current-packages", numItems: 100 });
expect(ctx.scheduler.runAfter).toHaveBeenCalledWith(
0,
internal.functions.syncPackageSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId: "publishers:demo", cursor: "next-packages" },
);
});
it("continues owner-publisher skill digest sync one page at a time", async () => {
const paginate = vi.fn().mockResolvedValue({
page: [],
isDone: false,
continueCursor: "next-skills",
});
const ctx = {
db: {
query: vi.fn(() => ({
withIndex: vi.fn(() => ({ paginate })),
})),
},
scheduler: {
runAfter: vi.fn().mockResolvedValue(undefined),
},
};
await syncSkillSearchDigestsForOwnerPublisherId(
ctx as never,
"publishers:demo" as never,
"current-skills",
);
expect(paginate).toHaveBeenCalledTimes(1);
expect(paginate).toHaveBeenCalledWith({ cursor: "current-skills", numItems: 100 });
expect(ctx.scheduler.runAfter).toHaveBeenCalledWith(
0,
internal.functions.syncSkillSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId: "publishers:demo", cursor: "next-skills" },
);
});
});

439
convex/functions.ts Normal file
View File

@ -0,0 +1,439 @@
import { customCtx, customMutation } from "convex-helpers/server/customFunctions";
import { Triggers } from "convex-helpers/server/triggers";
import { v } from "convex/values";
import semver from "semver";
import { internal } from "./_generated/api";
import type { DataModel, Doc, Id } from "./_generated/dataModel";
import {
mutation as rawMutation,
internalMutation as rawInternalMutation,
query,
internalQuery,
action,
internalAction,
httpAction,
} from "./_generated/server";
import type { MutationCtx } from "./_generated/server";
import {
deletePackageSearchDigests,
extractPackageDigestFields,
upsertPackageSearchDigest,
} from "./lib/packageSearchDigest";
import { getOwnerPublisher } from "./lib/publishers";
import { extractDigestFields, upsertSkillSearchDigest } from "./lib/skillSearchDigest";
const triggers = new Triggers<DataModel>();
function isMissingTableError(error: unknown, table: string) {
return (
error instanceof Error &&
new RegExp(`unexpected (query )?table:? ${table}`, "i").test(error.message)
);
}
type PackageDigestSyncCtx = Pick<MutationCtx, "db">;
type OwnerPublisherDigestScheduleCtx = Pick<Partial<MutationCtx>, "scheduler">;
type GitHubBackupDeletionCtx = Pick<MutationCtx, "db" | "scheduler">;
const OWNER_PUBLISHER_DIGEST_PAGE_SIZE = 100;
type LatestPackageRelease = Pick<
Doc<"packageReleases">,
| "_id"
| "createdAt"
| "version"
| "changelog"
| "summary"
| "compatibility"
| "capabilities"
| "verification"
| "distTags"
> & {
scanStatus?: Doc<"packages">["scanStatus"];
};
function toPackageLatestVersionSummary(
release: LatestPackageRelease | null,
): Doc<"packages">["latestVersionSummary"] {
if (!release) return undefined;
return {
version: release.version,
createdAt: release.createdAt,
changelog: release.changelog,
compatibility: release.compatibility,
capabilities: release.capabilities,
verification: release.verification,
};
}
function compareFallbackReleases(
family: Doc<"packages">["family"],
a: LatestPackageRelease,
b: LatestPackageRelease,
) {
if (family === "bundle-plugin") {
if (a.createdAt !== b.createdAt) return a.createdAt - b.createdAt;
return a._id.localeCompare(b._id);
}
const aSemver = semver.valid(a.version);
const bSemver = semver.valid(b.version);
if (aSemver && bSemver) return semver.compare(aSemver, bSemver);
if (aSemver) return 1;
if (bSemver) return -1;
if (a.createdAt !== b.createdAt) return a.createdAt - b.createdAt;
return a._id.localeCompare(b._id);
}
async function getPreferredFallbackPackageRelease(
ctx: PackageDigestSyncCtx,
packageId: Id<"packages">,
family: Doc<"packages">["family"],
): Promise<LatestPackageRelease | null> {
let cursor: string | null = null;
let best: LatestPackageRelease | null = null;
while (true) {
const page = await ctx.db
.query("packageReleases")
.withIndex("by_package_active_created", (q) =>
q.eq("packageId", packageId).eq("softDeletedAt", undefined),
)
.order("desc")
.paginate({ cursor, numItems: 100 });
for (const release of page.page) {
const candidate: LatestPackageRelease = {
_id: release._id,
createdAt: release.createdAt,
version: release.version,
changelog: release.changelog,
summary: release.summary,
compatibility: release.compatibility,
capabilities: release.capabilities,
verification: release.verification,
scanStatus: release.verification?.scanStatus,
distTags: release.distTags,
};
if (!best || compareFallbackReleases(family, candidate, best) > 0) best = candidate;
}
if (page.isDone) return best;
cursor = page.continueCursor;
}
}
async function syncPackageSearchDigest(
ctx: PackageDigestSyncCtx,
pkg: Doc<"packages"> | null | undefined,
) {
if (!pkg) return;
const latestRelease = pkg.latestReleaseId ? await ctx.db.get(pkg.latestReleaseId) : null;
const fields = extractPackageDigestFields(pkg);
const owner = await getOwnerPublisher(ctx, {
ownerPublisherId: pkg.ownerPublisherId,
ownerUserId: pkg.ownerUserId,
});
await upsertPackageSearchDigest(ctx, {
...fields,
latestVersion:
latestRelease && !latestRelease.softDeletedAt ? latestRelease.version : undefined,
ownerHandle: owner?.handle ?? "",
ownerKind: owner?.kind,
});
}
export async function syncPackageSearchDigestForPackageId(
ctx: PackageDigestSyncCtx,
packageId: Id<"packages"> | null | undefined,
) {
if (!packageId) return;
const pkg = await ctx.db.get(packageId);
if (!pkg) return;
await syncPackageSearchDigest(ctx, pkg);
}
export async function syncPackageSearchDigestsForOwnerUserId(
ctx: PackageDigestSyncCtx,
ownerUserId: Id<"users"> | null | undefined,
) {
if (!ownerUserId) return;
let cursor: string | null = null;
try {
while (true) {
const page = await ctx.db
.query("packages")
.withIndex("by_owner", (q) => q.eq("ownerUserId", ownerUserId))
.paginate({ cursor, numItems: 100 });
for (const pkg of page.page) {
await syncPackageSearchDigest(ctx, pkg);
}
if (page.isDone) break;
cursor = page.continueCursor;
}
} catch (error) {
if (isMissingTableError(error, "packages")) return;
throw error;
}
}
export async function syncPackageSearchDigestsForOwnerPublisherId(
ctx: PackageDigestSyncCtx & OwnerPublisherDigestScheduleCtx,
ownerPublisherId: Id<"publishers"> | null | undefined,
cursor: string | null = null,
) {
if (!ownerPublisherId) return;
try {
const page = await ctx.db
.query("packages")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", ownerPublisherId))
.paginate({ cursor, numItems: OWNER_PUBLISHER_DIGEST_PAGE_SIZE });
for (const pkg of page.page) {
await syncPackageSearchDigest(ctx, pkg);
}
if (!page.isDone && ctx.scheduler && page.continueCursor) {
await ctx.scheduler.runAfter(
0,
internal.functions.syncPackageSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId, cursor: page.continueCursor },
);
}
} catch (error) {
if (isMissingTableError(error, "packages")) return;
throw error;
}
}
async function syncSkillSearchDigestForSkill(
ctx: PackageDigestSyncCtx,
skill: Doc<"skills"> | null | undefined,
) {
if (!skill) return;
const fields = extractDigestFields(skill);
const owner = await getOwnerPublisher(ctx, {
ownerPublisherId: skill.ownerPublisherId,
ownerUserId: skill.ownerUserId,
});
await upsertSkillSearchDigest(ctx, {
...fields,
ownerHandle: owner?.handle ?? "",
ownerKind: owner?.kind,
ownerName: owner?.linkedUserId ? owner.handle : undefined,
ownerDisplayName: owner?.displayName,
ownerImage: owner?.image,
});
}
export function isGitHubMirrorEligibleSkillDoc(
skill: Pick<Doc<"skills">, "softDeletedAt" | "moderationStatus"> | null | undefined,
) {
if (!skill || skill.softDeletedAt) return false;
return (
skill.moderationStatus === undefined ||
skill.moderationStatus === null ||
skill.moderationStatus === "active"
);
}
export async function scheduleGitHubBackupDeletionForSkill(
ctx: GitHubBackupDeletionCtx,
skill: Pick<
Doc<"skills">,
"slug" | "ownerPublisherId" | "ownerUserId" | "softDeletedAt" | "moderationStatus"
>,
) {
const owner = await getOwnerPublisher(ctx, {
ownerPublisherId: skill.ownerPublisherId,
ownerUserId: skill.ownerUserId,
});
const ownerHandle = owner?.handle ?? String(skill.ownerPublisherId ?? skill.ownerUserId);
await ctx.scheduler.runAfter(0, internal.githubBackupsNode.deleteGitHubBackupForSlugInternal, {
ownerHandle,
slug: skill.slug,
});
}
export async function syncSkillSearchDigestsForOwnerPublisherId(
ctx: PackageDigestSyncCtx & OwnerPublisherDigestScheduleCtx,
ownerPublisherId: Id<"publishers"> | null | undefined,
cursor: string | null = null,
) {
if (!ownerPublisherId) return;
try {
const page = await ctx.db
.query("skills")
.withIndex("by_owner_publisher", (q) => q.eq("ownerPublisherId", ownerPublisherId))
.paginate({ cursor, numItems: OWNER_PUBLISHER_DIGEST_PAGE_SIZE });
for (const skill of page.page) {
await syncSkillSearchDigestForSkill(ctx, skill);
}
if (!page.isDone && ctx.scheduler && page.continueCursor) {
await ctx.scheduler.runAfter(
0,
internal.functions.syncSkillSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId, cursor: page.continueCursor },
);
}
} catch (error) {
if (isMissingTableError(error, "skills")) return;
throw error;
}
}
export async function scheduleOwnerPublisherDigestSync(
ctx: OwnerPublisherDigestScheduleCtx,
ownerPublisherId: Id<"publishers"> | null | undefined,
) {
if (!ownerPublisherId || !ctx.scheduler) return;
await ctx.scheduler.runAfter(
0,
internal.functions.syncPackageSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId },
);
await ctx.scheduler.runAfter(
0,
internal.functions.syncSkillSearchDigestsForOwnerPublisherIdInternal,
{ ownerPublisherId },
);
}
export const syncPackageSearchDigestsForOwnerPublisherIdInternal = rawInternalMutation({
args: {
ownerPublisherId: v.id("publishers"),
cursor: v.optional(v.union(v.string(), v.null())),
},
handler: async (ctx, args) => {
await syncPackageSearchDigestsForOwnerPublisherId(
ctx,
args.ownerPublisherId,
args.cursor ?? null,
);
},
});
export const syncSkillSearchDigestsForOwnerPublisherIdInternal = rawInternalMutation({
args: {
ownerPublisherId: v.id("publishers"),
cursor: v.optional(v.union(v.string(), v.null())),
},
handler: async (ctx, args) => {
await syncSkillSearchDigestsForOwnerPublisherId(
ctx,
args.ownerPublisherId,
args.cursor ?? null,
);
},
});
export async function repointPackageLatestRelease(
ctx: PackageDigestSyncCtx,
packageId: Id<"packages"> | null | undefined,
affectedReleaseId: Id<"packageReleases"> | null | undefined,
) {
if (!packageId || !affectedReleaseId) return;
const pkg = await ctx.db.get(packageId);
if (!pkg) return;
const nextTags = Object.fromEntries(
Object.entries(pkg.tags).filter(([, releaseId]) => releaseId !== affectedReleaseId),
) as Doc<"packages">["tags"];
const latestPointerAffected =
pkg.latestReleaseId === affectedReleaseId || pkg.tags.latest === affectedReleaseId;
if (!latestPointerAffected && Object.keys(nextTags).length === Object.keys(pkg.tags).length) {
return;
}
const nextLatest = latestPointerAffected
? await getPreferredFallbackPackageRelease(ctx, packageId, pkg.family)
: null;
if (latestPointerAffected && nextLatest && !(nextLatest.distTags ?? []).includes("latest")) {
await ctx.db.patch(nextLatest._id, {
distTags: [...(nextLatest.distTags ?? []), "latest"],
});
}
const patch: Partial<Doc<"packages">> = {
tags: latestPointerAffected && nextLatest ? { ...nextTags, latest: nextLatest._id } : nextTags,
updatedAt: Date.now(),
};
if (latestPointerAffected) {
patch.latestReleaseId = nextLatest?._id;
patch.latestVersionSummary = toPackageLatestVersionSummary(nextLatest);
patch.summary = nextLatest?.summary;
patch.capabilityTags = nextLatest?.capabilities?.capabilityTags;
patch.executesCode =
typeof nextLatest?.capabilities?.executesCode === "boolean"
? nextLatest.capabilities.executesCode
: undefined;
patch.compatibility = nextLatest?.compatibility;
patch.capabilities = nextLatest?.capabilities;
patch.verification = nextLatest?.verification;
patch.scanStatus = nextLatest?.scanStatus;
}
await ctx.db.patch(pkg._id, patch);
await syncPackageSearchDigest(ctx, { ...pkg, ...patch });
}
triggers.register("skills", async (ctx, change) => {
if (change.operation === "delete") {
await scheduleGitHubBackupDeletionForSkill(ctx, change.oldDoc);
const existing = await ctx.db
.query("skillSearchDigest")
.withIndex("by_skill", (q) => q.eq("skillId", change.id))
.unique();
if (existing) await ctx.db.delete(existing._id);
} else {
if (
change.operation === "update" &&
isGitHubMirrorEligibleSkillDoc(change.oldDoc) &&
!isGitHubMirrorEligibleSkillDoc(change.newDoc)
) {
await scheduleGitHubBackupDeletionForSkill(ctx, change.oldDoc);
}
await syncSkillSearchDigestForSkill(ctx, change.newDoc);
}
});
triggers.register("packages", async (ctx, change) => {
if (change.operation === "delete") {
await deletePackageSearchDigests(ctx, change.id);
return;
}
await syncPackageSearchDigest(ctx, change.newDoc);
});
triggers.register("packageReleases", async (ctx, change) => {
if (change.operation === "insert") return;
if (
change.operation === "update" &&
change.oldDoc.softDeletedAt === change.newDoc.softDeletedAt
) {
return;
}
const packageId =
change.operation === "delete" ? change.oldDoc.packageId : change.newDoc.packageId;
const affectedReleaseId = change.operation === "delete" ? change.oldDoc._id : change.newDoc._id;
if (change.operation === "delete" || change.newDoc.softDeletedAt) {
await repointPackageLatestRelease(ctx, packageId, affectedReleaseId);
return;
}
await syncPackageSearchDigestForPackageId(ctx, packageId);
});
triggers.register("users", async (ctx, change) => {
if (
change.operation === "update" &&
change.oldDoc.handle === change.newDoc.handle &&
change.oldDoc.deletedAt === change.newDoc.deletedAt &&
change.oldDoc.deactivatedAt === change.newDoc.deactivatedAt
) {
return;
}
const ownerUserId = change.operation === "delete" ? change.id : change.newDoc._id;
await syncPackageSearchDigestsForOwnerUserId(ctx, ownerUserId);
});
triggers.register("publishers", async (ctx, change) => {
const ownerPublisherId = change.operation === "delete" ? change.id : change.newDoc._id;
await scheduleOwnerPublisherDigestSync(ctx, ownerPublisherId);
});
export const mutation = customMutation(rawMutation, customCtx(triggers.wrapDB));
export const internalMutation = customMutation(rawInternalMutation, customCtx(triggers.wrapDB));
export { query, internalQuery, action, internalAction, httpAction };

View File

@ -0,0 +1,204 @@
import { describe, expect, it, vi } from "vitest";
import { getGitHubBackupPageInternal } from "./githubBackups";
const handler = (getGitHubBackupPageInternal as unknown as { _handler: Function })._handler;
describe("githubBackups page filtering", () => {
it("skips non-public digests (soft-deleted, hidden, removed)", async () => {
const activeDigest = {
_id: "skillSearchDigest:active",
skillId: "skills:active",
slug: "active-skill",
displayName: "Active Skill",
ownerUserId: "users:active",
ownerHandle: "alice",
latestVersionId: "skillVersions:active",
latestVersionSummary: {
version: "1.0.0",
createdAt: 1_700_000_000_000,
changelog: "init",
},
softDeletedAt: undefined,
moderationStatus: "active",
};
const hiddenDigest = {
_id: "skillSearchDigest:hidden",
skillId: "skills:hidden",
slug: "hidden-skill",
displayName: "Hidden Skill",
ownerUserId: "users:hidden",
ownerHandle: "bob",
latestVersionId: "skillVersions:hidden",
latestVersionSummary: {
version: "1.0.0",
createdAt: 1_700_000_000_000,
changelog: "init",
},
softDeletedAt: undefined,
moderationStatus: "hidden",
};
const removedDigest = {
_id: "skillSearchDigest:removed",
skillId: "skills:removed",
slug: "removed-skill",
displayName: "Removed Skill",
ownerUserId: "users:removed",
ownerHandle: "carol",
latestVersionId: "skillVersions:removed",
latestVersionSummary: {
version: "1.0.0",
createdAt: 1_700_000_000_000,
changelog: "init",
},
softDeletedAt: undefined,
moderationStatus: "removed",
};
const softDeletedDigest = {
_id: "skillSearchDigest:soft",
skillId: "skills:soft",
slug: "soft-skill",
displayName: "Soft Skill",
ownerUserId: "users:soft",
ownerHandle: "dave",
latestVersionId: "skillVersions:soft",
latestVersionSummary: {
version: "1.0.0",
createdAt: 1_700_000_000_000,
changelog: "init",
},
softDeletedAt: 1,
moderationStatus: "active",
};
const paginate = vi.fn().mockResolvedValue({
page: [activeDigest, hiddenDigest, removedDigest, softDeletedDigest],
isDone: true,
continueCursor: null,
});
const order = vi.fn().mockReturnValue({ paginate });
const query = vi.fn().mockReturnValue({ order });
const result = await handler(
{
db: { query },
} as never,
{ batchSize: 50 },
);
expect(query).toHaveBeenCalledWith("skillSearchDigest");
expect(result).toMatchObject({
isDone: true,
cursor: null,
items: [
{
kind: "ok",
slug: "active-skill",
ownerHandle: "alice",
version: "1.0.0",
},
],
});
});
it("keeps legacy digests with undefined moderationStatus eligible", async () => {
const legacyDigest = {
_id: "skillSearchDigest:legacy",
skillId: "skills:legacy",
slug: "legacy-skill",
displayName: "Legacy Skill",
ownerUserId: "users:legacy",
ownerHandle: "",
latestVersionId: "skillVersions:legacy",
latestVersionSummary: {
version: "2.0.0",
createdAt: 1_700_000_000_100,
changelog: "update",
},
softDeletedAt: undefined,
moderationStatus: undefined,
};
const paginate = vi.fn().mockResolvedValue({
page: [legacyDigest],
isDone: true,
continueCursor: null,
});
const order = vi.fn().mockReturnValue({ paginate });
const query = vi.fn().mockReturnValue({ order });
const result = await handler(
{
db: { query },
} as never,
{},
);
expect(result.items).toHaveLength(1);
expect(result.items[0]).toMatchObject({
kind: "ok",
slug: "legacy-skill",
ownerHandle: "users:legacy",
version: "2.0.0",
});
});
it("skips digests without ownerHandle or latestVersionSummary", async () => {
const noOwnerHandle = {
_id: "skillSearchDigest:no-owner",
skillId: "skills:no-owner",
slug: "no-owner",
displayName: "No Owner",
ownerUserId: "users:no-owner",
ownerHandle: undefined,
latestVersionId: "skillVersions:no-owner",
latestVersionSummary: { version: "1.0.0", createdAt: 1, changelog: "init" },
softDeletedAt: undefined,
moderationStatus: "active",
};
const noVersion = {
_id: "skillSearchDigest:no-version",
skillId: "skills:no-version",
slug: "no-version",
displayName: "No Version",
ownerUserId: "users:no-version",
ownerHandle: "frank",
latestVersionId: undefined,
latestVersionSummary: undefined,
softDeletedAt: undefined,
moderationStatus: "active",
};
const paginate = vi.fn().mockResolvedValue({
page: [noOwnerHandle, noVersion],
isDone: true,
continueCursor: null,
});
const order = vi.fn().mockReturnValue({ paginate });
const query = vi.fn().mockReturnValue({ order });
const result = await handler({ db: { query } } as never, {});
expect(result.items).toEqual([
{ kind: "missingOwner", skillId: "skills:no-owner", ownerUserId: "users:no-owner" },
{ kind: "missingLatestVersion", skillId: "skills:no-version" },
]);
});
it("resets stale skills-table cursors after switching to digest pagination", async () => {
const paginate = vi
.fn()
.mockRejectedValueOnce(new Error("cursor is from a different query"))
.mockResolvedValueOnce({ page: [], isDone: true, continueCursor: null });
const order = vi.fn().mockReturnValue({ paginate });
const query = vi.fn().mockReturnValue({ order });
const result = await handler({ db: { query } } as never, { cursor: "stale-cursor" });
expect(result).toMatchObject({ items: [], isDone: true, cursor: null });
expect(paginate).toHaveBeenNthCalledWith(1, { cursor: "stale-cursor", numItems: 50 });
expect(paginate).toHaveBeenNthCalledWith(2, { cursor: null, numItems: 50 });
});
});

Some files were not shown because too many files have changed in this diff Show More