feat: SoulHub registry + auto-seed

SoulHub SOUL.md registry (souls table, versions, search, OG) + first-run auto-seed; fixes seed concurrency and GitHub backup owner handle.
This commit is contained in:
Josh Palmer 2026-01-10 19:25:11 +01:00 committed by GitHub
parent cc0027a094
commit 0cc0bdcd50
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
53 changed files with 4846 additions and 345 deletions

View File

@ -1,6 +1,9 @@
# Frontend
VITE_CONVEX_URL=
VITE_CONVEX_SITE_URL=
VITE_SOULHUB_SITE_URL=
VITE_SOULHUB_HOST=
VITE_SITE_MODE=
SITE_URL=http://localhost:3000
CONVEX_SITE_URL=

View File

@ -6,6 +6,7 @@
- Web: dynamic OG image cards for skills (name, description, version).
- CLI: auto-scan Clawdbot skill roots (per-agent workspaces, shared skills, extraDirs).
- Web: import skills from public GitHub URLs (auto-detect `SKILL.md`, smart file selection, provenance).
- Web/API: SoulHub (SOUL.md registry) with v1 endpoints and first-run auto-seed.
### Fixed
- Web: stabilize skill OG image generation on server runtimes.

View File

@ -9,14 +9,26 @@
ClawdHub is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
Its designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
onlycrabs.ai is the **SOUL.md registry**: publish and share system lore the same way you publish skills.
Live: `https://clawdhub.com`
onlycrabs.ai: `https://onlycrabs.ai`
## What you can do
- Browse skills + render their `SKILL.md`.
- Publish new versions with changelogs + tags (including `latest`).
- Publish new skill versions with changelogs + tags (including `latest`).
- Browse souls + render their `SOUL.md`.
- Publish new soul versions with changelogs + tags.
- Search via embeddings (vector index) instead of brittle keywords.
- Star + comment; admins/mods can curate and approve.
- Star + comment; admins/mods can curate and approve skills.
## onlycrabs.ai (SOUL.md registry)
- Entry point is host-based: `onlycrabs.ai`.
- On the onlycrabs.ai host, the home page and nav default to souls.
- On ClawdHub, souls live under `/souls`.
- Soul bundles only accept `SOUL.md` for now (no extra files).
## How it works (high level)
@ -72,6 +84,9 @@ This writes `JWT_PRIVATE_KEY` + `JWKS` to the deployment and prints values for y
- `VITE_CONVEX_URL`: Convex deployment URL (`https://<deployment>.convex.cloud`).
- `VITE_CONVEX_SITE_URL`: Convex site URL (`https://<deployment>.convex.site`).
- `VITE_SOULHUB_SITE_URL`: onlycrabs.ai site URL (`https://onlycrabs.ai`).
- `VITE_SOULHUB_HOST`: onlycrabs.ai host match (`onlycrabs.ai`).
- `VITE_SITE_MODE`: Optional override (`skills` or `souls`) for SSR builds.
- `CONVEX_SITE_URL`: same as `VITE_CONVEX_SITE_URL` (auth + cookies).
- `SITE_URL`: App URL (local: `http://localhost:3000`).
- `AUTH_GITHUB_ID` / `AUTH_GITHUB_SECRET`: GitHub OAuth App.

View File

@ -15,6 +15,8 @@ import type * as downloads from "../downloads.js";
import type * as githubBackups from "../githubBackups.js";
import type * as githubBackupsNode from "../githubBackupsNode.js";
import type * as githubImport from "../githubImport.js";
import type * as githubSoulBackups from "../githubSoulBackups.js";
import type * as githubSoulBackupsNode from "../githubSoulBackupsNode.js";
import type * as http from "../http.js";
import type * as httpApi from "../httpApi.js";
import type * as httpApiV1 from "../httpApiV1.js";
@ -24,15 +26,23 @@ import type * as lib_changelog from "../lib/changelog.js";
import type * as lib_embeddings from "../lib/embeddings.js";
import type * as lib_githubBackup from "../lib/githubBackup.js";
import type * as lib_githubImport from "../lib/githubImport.js";
import type * as lib_githubSoulBackup from "../lib/githubSoulBackup.js";
import type * as lib_skillBackfill from "../lib/skillBackfill.js";
import type * as lib_skillPublish from "../lib/skillPublish.js";
import type * as lib_skills from "../lib/skills.js";
import type * as lib_soulChangelog from "../lib/soulChangelog.js";
import type * as lib_soulPublish from "../lib/soulPublish.js";
import type * as lib_tokens from "../lib/tokens.js";
import type * as lib_webhooks from "../lib/webhooks.js";
import type * as maintenance from "../maintenance.js";
import type * as rateLimits from "../rateLimits.js";
import type * as search from "../search.js";
import type * as seed from "../seed.js";
import type * as skills from "../skills.js";
import type * as soulComments from "../soulComments.js";
import type * as soulDownloads from "../soulDownloads.js";
import type * as soulStars from "../soulStars.js";
import type * as souls from "../souls.js";
import type * as stars from "../stars.js";
import type * as telemetry from "../telemetry.js";
import type * as tokens from "../tokens.js";
@ -54,6 +64,8 @@ declare const fullApi: ApiFromModules<{
githubBackups: typeof githubBackups;
githubBackupsNode: typeof githubBackupsNode;
githubImport: typeof githubImport;
githubSoulBackups: typeof githubSoulBackups;
githubSoulBackupsNode: typeof githubSoulBackupsNode;
http: typeof http;
httpApi: typeof httpApi;
httpApiV1: typeof httpApiV1;
@ -63,15 +75,23 @@ declare const fullApi: ApiFromModules<{
"lib/embeddings": typeof lib_embeddings;
"lib/githubBackup": typeof lib_githubBackup;
"lib/githubImport": typeof lib_githubImport;
"lib/githubSoulBackup": typeof lib_githubSoulBackup;
"lib/skillBackfill": typeof lib_skillBackfill;
"lib/skillPublish": typeof lib_skillPublish;
"lib/skills": typeof lib_skills;
"lib/soulChangelog": typeof lib_soulChangelog;
"lib/soulPublish": typeof lib_soulPublish;
"lib/tokens": typeof lib_tokens;
"lib/webhooks": typeof lib_webhooks;
maintenance: typeof maintenance;
rateLimits: typeof rateLimits;
search: typeof search;
seed: typeof seed;
skills: typeof skills;
soulComments: typeof soulComments;
soulDownloads: typeof soulDownloads;
soulStars: typeof soulStars;
souls: typeof souls;
stars: typeof stars;
telemetry: typeof telemetry;
tokens: typeof tokens;

170
convex/githubSoulBackups.ts Normal file
View File

@ -0,0 +1,170 @@
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import { action, internalMutation, internalQuery } from './_generated/server'
import { assertRole, requireUserFromAction } from './lib/access'
const DEFAULT_BATCH_SIZE = 50
const MAX_BATCH_SIZE = 200
const SYNC_STATE_KEY = 'souls'
type BackupPageItem =
| {
kind: 'ok'
soulId: Id<'souls'>
versionId: Id<'soulVersions'>
slug: string
displayName: string
version: string
ownerHandle: string
files: Doc<'soulVersions'>['files']
publishedAt: number
}
| { kind: 'missingLatestVersion'; soulId: Id<'souls'> }
| { kind: 'missingVersionDoc'; soulId: Id<'souls'>; versionId: Id<'soulVersions'> }
| { kind: 'missingOwner'; soulId: Id<'souls'>; ownerUserId: Id<'users'> }
type BackupPageResult = {
items: BackupPageItem[]
cursor: string | null
isDone: boolean
}
type BackupSyncState = {
cursor: string | null
}
export type SyncGitHubSoulBackupsResult = {
stats: {
soulsScanned: number
soulsSkipped: number
soulsBackedUp: number
soulsMissingVersion: number
soulsMissingOwner: number
errors: number
}
cursor: string | null
isDone: boolean
}
export const getGitHubSoulBackupPageInternal = internalQuery({
args: {
cursor: v.optional(v.string()),
batchSize: v.optional(v.number()),
},
handler: async (ctx, args): Promise<BackupPageResult> => {
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const { page, isDone, continueCursor } = await ctx.db
.query('souls')
.order('asc')
.paginate({ cursor: args.cursor ?? null, numItems: batchSize })
const items: BackupPageItem[] = []
for (const soul of page) {
if (soul.softDeletedAt) continue
if (!soul.latestVersionId) {
items.push({ kind: 'missingLatestVersion', soulId: soul._id })
continue
}
const version = await ctx.db.get(soul.latestVersionId)
if (!version) {
items.push({
kind: 'missingVersionDoc',
soulId: soul._id,
versionId: soul.latestVersionId,
})
continue
}
const owner = await ctx.db.get(soul.ownerUserId)
if (!owner || owner.deletedAt) {
items.push({ kind: 'missingOwner', soulId: soul._id, ownerUserId: soul.ownerUserId })
continue
}
items.push({
kind: 'ok',
soulId: soul._id,
versionId: version._id,
slug: soul.slug,
displayName: soul.displayName,
version: version.version,
ownerHandle: owner.handle ?? owner._id,
files: version.files,
publishedAt: version.createdAt,
})
}
return { items, cursor: continueCursor, isDone }
},
})
export const getGitHubSoulBackupSyncStateInternal = internalQuery({
args: {},
handler: async (ctx): Promise<BackupSyncState> => {
const state = await ctx.db
.query('githubBackupSyncState')
.withIndex('by_key', (q) => q.eq('key', SYNC_STATE_KEY))
.unique()
return { cursor: state?.cursor ?? null }
},
})
export const setGitHubSoulBackupSyncStateInternal = internalMutation({
args: {
cursor: v.optional(v.string()),
},
handler: async (ctx, args) => {
const now = Date.now()
const state = await ctx.db
.query('githubBackupSyncState')
.withIndex('by_key', (q) => q.eq('key', SYNC_STATE_KEY))
.unique()
if (!state) {
await ctx.db.insert('githubBackupSyncState', {
key: SYNC_STATE_KEY,
cursor: args.cursor,
updatedAt: now,
})
return { ok: true as const }
}
await ctx.db.patch(state._id, {
cursor: args.cursor,
updatedAt: now,
})
return { ok: true as const }
},
})
export const syncGitHubSoulBackups: ReturnType<typeof action> = action({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
resetCursor: v.optional(v.boolean()),
},
handler: async (ctx, args): Promise<SyncGitHubSoulBackupsResult> => {
const { user } = await requireUserFromAction(ctx)
assertRole(user, ['admin'])
if (args.resetCursor && !args.dryRun) {
await ctx.runMutation(internal.githubSoulBackups.setGitHubSoulBackupSyncStateInternal, {
cursor: undefined,
})
}
return ctx.runAction(internal.githubSoulBackupsNode.syncGitHubSoulBackupsInternal, {
dryRun: args.dryRun,
batchSize: args.batchSize,
maxBatches: args.maxBatches,
}) as Promise<SyncGitHubSoulBackupsResult>
},
})
function clampInt(value: number, min: number, max: number) {
return Math.max(min, Math.min(max, Math.floor(value)))
}

View File

@ -0,0 +1,186 @@
'use node'
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc } from './_generated/dataModel'
import type { ActionCtx } from './_generated/server'
import { internalAction } from './_generated/server'
import {
backupSoulToGitHub,
fetchGitHubSoulMeta,
getGitHubSoulBackupContext,
isGitHubSoulBackupConfigured,
} from './lib/githubSoulBackup'
const DEFAULT_BATCH_SIZE = 50
const MAX_BATCH_SIZE = 200
const DEFAULT_MAX_BATCHES = 5
const MAX_MAX_BATCHES = 200
type BackupPageItem =
| {
kind: 'ok'
slug: string
version: string
displayName: string
ownerHandle: string
files: Doc<'soulVersions'>['files']
publishedAt: number
}
| { kind: 'missingLatestVersion' }
| { kind: 'missingVersionDoc' }
| { kind: 'missingOwner' }
export type GitHubSoulBackupSyncStats = {
soulsScanned: number
soulsSkipped: number
soulsBackedUp: number
soulsMissingVersion: number
soulsMissingOwner: number
errors: number
}
export type SyncGitHubSoulBackupsInternalArgs = {
dryRun?: boolean
batchSize?: number
maxBatches?: number
}
export type SyncGitHubSoulBackupsInternalResult = {
stats: GitHubSoulBackupSyncStats
cursor: string | null
isDone: boolean
}
export const backupSoulForPublishInternal = internalAction({
args: {
slug: v.string(),
version: v.string(),
displayName: v.string(),
ownerHandle: v.string(),
files: v.array(
v.object({
path: v.string(),
size: v.number(),
storageId: v.id('_storage'),
sha256: v.string(),
contentType: v.optional(v.string()),
}),
),
publishedAt: v.number(),
},
handler: async (ctx, args) => {
if (!isGitHubSoulBackupConfigured()) {
return { skipped: true as const }
}
await backupSoulToGitHub(ctx, args)
return { skipped: false as const }
},
})
export async function syncGitHubSoulBackupsInternalHandler(
ctx: ActionCtx,
args: SyncGitHubSoulBackupsInternalArgs,
): Promise<SyncGitHubSoulBackupsInternalResult> {
const dryRun = Boolean(args.dryRun)
const stats: GitHubSoulBackupSyncStats = {
soulsScanned: 0,
soulsSkipped: 0,
soulsBackedUp: 0,
soulsMissingVersion: 0,
soulsMissingOwner: 0,
errors: 0,
}
if (!isGitHubSoulBackupConfigured()) {
return { stats, cursor: null, isDone: true }
}
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE)
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
const context = await getGitHubSoulBackupContext()
const state = dryRun
? { cursor: null as string | null }
: ((await ctx.runQuery(
internal.githubSoulBackups.getGitHubSoulBackupSyncStateInternal,
{},
)) as {
cursor: string | null
})
let cursor: string | null = state.cursor
let isDone = false
for (let batch = 0; batch < maxBatches; batch++) {
const page = (await ctx.runQuery(internal.githubSoulBackups.getGitHubSoulBackupPageInternal, {
cursor: cursor ?? undefined,
batchSize,
})) as { items: BackupPageItem[]; cursor: string | null; isDone: boolean }
cursor = page.cursor
isDone = page.isDone
for (const item of page.items) {
if (item.kind !== 'ok') {
if (item.kind === 'missingLatestVersion' || item.kind === 'missingVersionDoc') {
stats.soulsMissingVersion += 1
} else if (item.kind === 'missingOwner') {
stats.soulsMissingOwner += 1
}
continue
}
stats.soulsScanned += 1
try {
const meta = await fetchGitHubSoulMeta(context, item.ownerHandle, item.slug)
if (meta?.latest?.version === item.version) {
stats.soulsSkipped += 1
continue
}
if (!dryRun) {
await backupSoulToGitHub(
ctx,
{
slug: item.slug,
version: item.version,
displayName: item.displayName,
ownerHandle: item.ownerHandle,
files: item.files,
publishedAt: item.publishedAt,
},
context,
)
stats.soulsBackedUp += 1
}
} catch (error) {
console.error('GitHub soul backup sync failed', error)
stats.errors += 1
}
}
if (!dryRun) {
await ctx.runMutation(internal.githubSoulBackups.setGitHubSoulBackupSyncStateInternal, {
cursor: isDone ? undefined : (cursor ?? undefined),
})
}
if (isDone) break
}
return { stats, cursor, isDone }
}
export const syncGitHubSoulBackupsInternal = internalAction({
args: {
dryRun: v.optional(v.boolean()),
batchSize: v.optional(v.number()),
maxBatches: v.optional(v.number()),
},
handler: syncGitHubSoulBackupsInternalHandler,
})
function clampInt(value: number, min: number, max: number) {
return Math.max(min, Math.min(max, Math.floor(value)))
}

View File

@ -15,12 +15,17 @@ import {
} from './httpApi'
import {
listSkillsV1Http,
listSoulsV1Http,
publishSkillV1Http,
publishSoulV1Http,
resolveSkillVersionV1Http,
searchSkillsV1Http,
skillsDeleteRouterV1Http,
skillsGetRouterV1Http,
skillsPostRouterV1Http,
soulsDeleteRouterV1Http,
soulsGetRouterV1Http,
soulsPostRouterV1Http,
whoamiV1Http,
} from './httpApiV1'
@ -82,6 +87,36 @@ http.route({
handler: whoamiV1Http,
})
http.route({
path: ApiRoutes.souls,
method: 'GET',
handler: listSoulsV1Http,
})
http.route({
pathPrefix: `${ApiRoutes.souls}/`,
method: 'GET',
handler: soulsGetRouterV1Http,
})
http.route({
path: ApiRoutes.souls,
method: 'POST',
handler: publishSoulV1Http,
})
http.route({
pathPrefix: `${ApiRoutes.souls}/`,
method: 'POST',
handler: soulsPostRouterV1Http,
})
http.route({
pathPrefix: `${ApiRoutes.souls}/`,
method: 'DELETE',
handler: soulsDeleteRouterV1Http,
})
// TODO: remove legacy /api routes after deprecation window.
http.route({
path: LegacyApiRoutes.download,

View File

@ -273,6 +273,7 @@ function parsePublishBody(body: unknown) {
version: parsed.version,
changelog: parsed.changelog,
tags,
source: parsed.source ?? undefined,
forkOf: parsed.forkOf
? {
slug: parsed.forkOf.slug,

View File

@ -6,6 +6,7 @@ import { httpAction } from './_generated/server'
import { requireApiTokenUser } from './lib/apiTokenAuth'
import { hashToken } from './lib/tokens'
import { publishVersionForUser } from './skills'
import { publishSoulVersionForUser } from './souls'
const RATE_LIMIT_WINDOW_MS = 60_000
const RATE_LIMITS = {
@ -76,6 +77,57 @@ type ListVersionsResult = {
nextCursor: string | null
}
type ListSoulsResult = {
items: Array<{
soul: {
_id: Id<'souls'>
slug: string
displayName: string
summary?: string
tags: Record<string, Id<'soulVersions'>>
stats: unknown
createdAt: number
updatedAt: number
latestVersionId?: Id<'soulVersions'>
}
latestVersion: { version: string; createdAt: number; changelog: string } | null
}>
nextCursor: string | null
}
type GetSoulBySlugResult = {
soul: {
_id: Id<'souls'>
slug: string
displayName: string
summary?: string
tags: Record<string, Id<'soulVersions'>>
stats: unknown
createdAt: number
updatedAt: number
} | null
latestVersion: Doc<'soulVersions'> | null
owner: { handle?: string; displayName?: string; image?: string } | null
} | null
type ListSoulVersionsResult = {
items: Array<{
version: string
createdAt: number
changelog: string
changelogSource?: 'auto' | 'user'
files: Array<{
path: string
size: number
storageId: Id<'_storage'>
sha256: string
contentType?: string
}>
softDeletedAt?: number
}>
nextCursor: string | null
}
async function searchSkillsV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'read')
if (!rate.ok) return rate.response
@ -512,6 +564,7 @@ async function parseMultipartPublish(
version: payload.version,
changelog: typeof payload.changelog === 'string' ? payload.changelog : '',
tags: Array.isArray(payload.tags) ? payload.tags : undefined,
...(payload.source ? { source: payload.source } : {}),
files,
...(payload.forkOf === undefined ? {} : { forkOf: payload.forkOf }),
}
@ -529,6 +582,7 @@ function parsePublishBody(body: unknown) {
version: parsed.version,
changelog: parsed.changelog,
tags,
source: parsed.source ?? undefined,
forkOf: parsed.forkOf
? {
slug: parsed.forkOf.slug,
@ -542,6 +596,20 @@ function parsePublishBody(body: unknown) {
}
}
async function resolveSoulTags(
ctx: ActionCtx,
tags: Record<string, Id<'soulVersions'>>,
): Promise<Record<string, string>> {
const resolved: Record<string, string> = {}
for (const [tag, versionId] of Object.entries(tags)) {
const version = await ctx.runQuery(api.souls.getVersionById, { versionId })
if (version && !version.softDeletedAt) {
resolved[tag] = version.version
}
}
return resolved
}
async function resolveTags(
ctx: ActionCtx,
tags: Record<string, Id<'skillVersions'>>,
@ -696,6 +764,288 @@ function toHex(bytes: Uint8Array) {
return out
}
async function listSoulsV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'read')
if (!rate.ok) return rate.response
const url = new URL(request.url)
const limit = toOptionalNumber(url.searchParams.get('limit'))
const cursor = url.searchParams.get('cursor')?.trim() || undefined
const result = (await ctx.runQuery(api.souls.listPublicPage, {
limit,
cursor,
})) as ListSoulsResult
const items = await Promise.all(
result.items.map(async (item) => {
const tags = await resolveSoulTags(ctx, item.soul.tags)
return {
slug: item.soul.slug,
displayName: item.soul.displayName,
summary: item.soul.summary ?? null,
tags,
stats: item.soul.stats,
createdAt: item.soul.createdAt,
updatedAt: item.soul.updatedAt,
latestVersion: item.latestVersion
? {
version: item.latestVersion.version,
createdAt: item.latestVersion.createdAt,
changelog: item.latestVersion.changelog,
}
: null,
}
}),
)
return json({ items, nextCursor: result.nextCursor ?? null }, 200, rate.headers)
}
export const listSoulsV1Http = httpAction(listSoulsV1Handler)
async function soulsGetRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'read')
if (!rate.ok) return rate.response
const segments = getPathSegments(request, '/api/v1/souls/')
if (segments.length === 0) return text('Missing slug', 400, rate.headers)
const slug = segments[0]?.trim().toLowerCase() ?? ''
const second = segments[1]
const third = segments[2]
if (segments.length === 1) {
const result = (await ctx.runQuery(api.souls.getBySlug, { slug })) as GetSoulBySlugResult
if (!result?.soul) return text('Soul not found', 404, rate.headers)
const tags = await resolveSoulTags(ctx, result.soul.tags)
return json(
{
soul: {
slug: result.soul.slug,
displayName: result.soul.displayName,
summary: result.soul.summary ?? null,
tags,
stats: result.soul.stats,
createdAt: result.soul.createdAt,
updatedAt: result.soul.updatedAt,
},
latestVersion: result.latestVersion
? {
version: result.latestVersion.version,
createdAt: result.latestVersion.createdAt,
changelog: result.latestVersion.changelog,
}
: null,
owner: result.owner
? {
handle: result.owner.handle ?? null,
displayName: result.owner.displayName ?? null,
image: result.owner.image ?? null,
}
: null,
},
200,
rate.headers,
)
}
if (second === 'versions' && segments.length === 2) {
const soul = await ctx.runQuery(internal.souls.getSoulBySlugInternal, { slug })
if (!soul || soul.softDeletedAt) return text('Soul not found', 404, rate.headers)
const url = new URL(request.url)
const limit = toOptionalNumber(url.searchParams.get('limit'))
const cursor = url.searchParams.get('cursor')?.trim() || undefined
const result = (await ctx.runQuery(api.souls.listVersionsPage, {
soulId: soul._id,
limit,
cursor,
})) as ListSoulVersionsResult
const items = result.items
.filter((version) => !version.softDeletedAt)
.map((version) => ({
version: version.version,
createdAt: version.createdAt,
changelog: version.changelog,
changelogSource: version.changelogSource ?? null,
}))
return json({ items, nextCursor: result.nextCursor ?? null }, 200, rate.headers)
}
if (second === 'versions' && third && segments.length === 3) {
const soul = await ctx.runQuery(internal.souls.getSoulBySlugInternal, { slug })
if (!soul || soul.softDeletedAt) return text('Soul not found', 404, rate.headers)
const version = await ctx.runQuery(api.souls.getVersionBySoulAndVersion, {
soulId: soul._id,
version: third,
})
if (!version) return text('Version not found', 404, rate.headers)
if (version.softDeletedAt) return text('Version not available', 410, rate.headers)
return json(
{
soul: { slug: soul.slug, displayName: soul.displayName },
version: {
version: version.version,
createdAt: version.createdAt,
changelog: version.changelog,
changelogSource: version.changelogSource ?? null,
files: version.files.map((file) => ({
path: file.path,
size: file.size,
sha256: file.sha256,
contentType: file.contentType ?? null,
})),
},
},
200,
rate.headers,
)
}
if (second === 'file' && segments.length === 2) {
const url = new URL(request.url)
const path = url.searchParams.get('path')?.trim()
if (!path) return text('Missing path', 400, rate.headers)
const versionParam = url.searchParams.get('version')?.trim()
const tagParam = url.searchParams.get('tag')?.trim()
const soulResult = (await ctx.runQuery(api.souls.getBySlug, {
slug,
})) as GetSoulBySlugResult
if (!soulResult?.soul) return text('Soul not found', 404, rate.headers)
let version = soulResult.latestVersion
if (versionParam) {
version = await ctx.runQuery(api.souls.getVersionBySoulAndVersion, {
soulId: soulResult.soul._id,
version: versionParam,
})
} else if (tagParam) {
const versionId = soulResult.soul.tags[tagParam]
if (versionId) {
version = await ctx.runQuery(api.souls.getVersionById, { versionId })
}
}
if (!version) return text('Version not found', 404, rate.headers)
if (version.softDeletedAt) return text('Version not available', 410, rate.headers)
const normalized = path.trim()
const normalizedLower = normalized.toLowerCase()
const file =
version.files.find((entry) => entry.path === normalized) ??
version.files.find((entry) => entry.path.toLowerCase() === normalizedLower)
if (!file) return text('File not found', 404, rate.headers)
if (file.size > MAX_RAW_FILE_BYTES) return text('File exceeds 200KB limit', 413, rate.headers)
const blob = await ctx.storage.get(file.storageId)
if (!blob) return text('File missing in storage', 410, rate.headers)
const textContent = await blob.text()
void ctx.runMutation(api.soulDownloads.increment, { soulId: soulResult.soul._id })
const headers = mergeHeaders(rate.headers, {
'Content-Type': file.contentType
? `${file.contentType}; charset=utf-8`
: 'text/plain; charset=utf-8',
'Cache-Control': 'private, max-age=60',
ETag: file.sha256,
'X-Content-SHA256': file.sha256,
'X-Content-Size': String(file.size),
})
return new Response(textContent, { status: 200, headers })
}
return text('Not found', 404, rate.headers)
}
export const soulsGetRouterV1Http = httpAction(soulsGetRouterV1Handler)
async function publishSoulV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'write')
if (!rate.ok) return rate.response
try {
if (!parseBearerToken(request)) return text('Unauthorized', 401, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
}
const { userId } = await requireApiTokenUser(ctx, request)
const contentType = request.headers.get('content-type') ?? ''
try {
if (contentType.includes('application/json')) {
const body = await request.json()
const payload = parsePublishBody(body)
const result = await publishSoulVersionForUser(ctx, userId, payload)
return json({ ok: true, ...result }, 200, rate.headers)
}
if (contentType.includes('multipart/form-data')) {
const payload = await parseMultipartPublish(ctx, request)
const result = await publishSoulVersionForUser(ctx, userId, payload)
return json({ ok: true, ...result }, 200, rate.headers)
}
} catch (error) {
const message = error instanceof Error ? error.message : 'Publish failed'
return text(message, 400, rate.headers)
}
return text('Unsupported content type', 415, rate.headers)
}
export const publishSoulV1Http = httpAction(publishSoulV1Handler)
async function soulsPostRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'write')
if (!rate.ok) return rate.response
const segments = getPathSegments(request, '/api/v1/souls/')
if (segments.length !== 2 || segments[1] !== 'undelete') {
return text('Not found', 404, rate.headers)
}
const slug = segments[0]?.trim().toLowerCase() ?? ''
try {
const { userId } = await requireApiTokenUser(ctx, request)
await ctx.runMutation(internal.souls.setSoulSoftDeletedInternal, {
userId,
slug,
deleted: false,
})
return json({ ok: true }, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
}
}
export const soulsPostRouterV1Http = httpAction(soulsPostRouterV1Handler)
async function soulsDeleteRouterV1Handler(ctx: ActionCtx, request: Request) {
const rate = await applyRateLimit(ctx, request, 'write')
if (!rate.ok) return rate.response
const segments = getPathSegments(request, '/api/v1/souls/')
if (segments.length !== 1) return text('Not found', 404, rate.headers)
const slug = segments[0]?.trim().toLowerCase() ?? ''
try {
const { userId } = await requireApiTokenUser(ctx, request)
await ctx.runMutation(internal.souls.setSoulSoftDeletedInternal, {
userId,
slug,
deleted: true,
})
return json({ ok: true }, 200, rate.headers)
} catch {
return text('Unauthorized', 401, rate.headers)
}
}
export const soulsDeleteRouterV1Http = httpAction(soulsDeleteRouterV1Handler)
export const __handlers = {
searchSkillsV1Handler,
resolveSkillVersionV1Handler,
@ -704,5 +1054,10 @@ export const __handlers = {
publishSkillV1Handler,
skillsPostRouterV1Handler,
skillsDeleteRouterV1Handler,
listSoulsV1Handler,
soulsGetRouterV1Handler,
publishSoulV1Handler,
soulsPostRouterV1Handler,
soulsDeleteRouterV1Handler,
whoamiV1Handler,
}

View File

@ -1,9 +1,16 @@
export const EMBEDDING_MODEL = 'text-embedding-3-small'
export const EMBEDDING_DIMENSIONS = 1536
function emptyEmbedding() {
return Array.from({ length: EMBEDDING_DIMENSIONS }, () => 0)
}
export async function generateEmbedding(text: string) {
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) throw new Error('OPENAI_API_KEY is not configured')
if (!apiKey) {
console.warn('OPENAI_API_KEY is not configured; using zero embeddings')
return emptyEmbedding()
}
const response = await fetch('https://api.openai.com/v1/embeddings', {
method: 'POST',

View File

@ -0,0 +1,443 @@
'use node'
import { createPrivateKey, createSign } from 'node:crypto'
import type { Id } from '../_generated/dataModel'
import type { ActionCtx } from '../_generated/server'
const GITHUB_API = 'https://api.github.com'
const DEFAULT_REPO = 'clawdbot/souls'
const DEFAULT_ROOT = 'souls'
const META_FILENAME = '_meta.json'
const USER_AGENT = 'clawdhub/souls-backup'
type BackupFile = {
path: string
size: number
storageId: Id<'_storage'>
sha256: string
contentType?: string
}
type BackupParams = {
slug: string
version: string
displayName: string
ownerHandle: string
files: BackupFile[]
publishedAt: number
}
type RepoInfo = {
default_branch?: string
}
type GitRef = {
object: { sha: string }
}
type GitCommit = {
sha: string
tree: { sha: string }
}
type GitTreeEntry = {
path?: string
type?: string
}
type GitTree = {
tree?: GitTreeEntry[]
}
type MetaFile = {
owner: string
slug: string
displayName: string
latest: {
version: string
publishedAt: number
commit: string | null
}
history: Array<{
version: string
publishedAt: number
commit: string
}>
}
export type GitHubBackupContext = {
token: string
repo: string
repoOwner: string
repoName: string
branch: string
root: string
}
export function isGitHubSoulBackupConfigured() {
return Boolean(
process.env.GITHUB_APP_ID &&
process.env.GITHUB_APP_PRIVATE_KEY &&
process.env.GITHUB_APP_INSTALLATION_ID,
)
}
export async function getGitHubSoulBackupContext(): Promise<GitHubBackupContext> {
const repo = process.env.GITHUB_SOULS_REPO ?? DEFAULT_REPO
const root = process.env.GITHUB_SOULS_ROOT ?? DEFAULT_ROOT
const [repoOwner, repoName] = parseRepo(repo)
const token = await createInstallationToken()
const repoInfo = await githubGet<RepoInfo>(token, `/repos/${repoOwner}/${repoName}`)
const branch = repoInfo.default_branch ?? 'main'
return { token, repo, repoOwner, repoName, branch, root }
}
export async function fetchGitHubSoulMeta(
context: GitHubBackupContext,
ownerHandle: string,
slug: string,
): Promise<MetaFile | null> {
const soulRoot = buildSoulRoot(context.root, ownerHandle, slug)
return fetchMetaFile(
context.token,
context.repoOwner,
context.repoName,
`${soulRoot}/${META_FILENAME}`,
context.branch,
)
}
export async function backupSoulToGitHub(
ctx: ActionCtx,
params: BackupParams,
context?: GitHubBackupContext,
) {
if (!isGitHubSoulBackupConfigured()) return
const resolved = context ?? (await getGitHubSoulBackupContext())
const soulRoot = buildSoulRoot(resolved.root, params.ownerHandle, params.slug)
const ref = await githubGet<GitRef>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/ref/heads/${resolved.branch}`,
)
const baseCommitSha = ref.object.sha
const baseCommit = await githubGet<GitCommit>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits/${baseCommitSha}`,
)
const baseTreeSha = baseCommit.tree.sha
const existingTree = await githubGet<GitTree>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees/${baseTreeSha}?recursive=1`,
)
const prefix = `${soulRoot}/`
const existingPaths = new Set(
(existingTree.tree ?? [])
.filter((entry) => entry.type === 'blob' && entry.path?.startsWith(prefix))
.map((entry) => entry.path ?? ''),
)
const newPaths = new Set<string>()
const treeEntries: Array<{
path: string
mode: '100644'
type: 'blob'
sha: string | null
}> = []
for (const file of params.files) {
const content = await fetchStorageBase64(ctx, file.storageId)
const blobSha = await createBlob(resolved.token, resolved.repoOwner, resolved.repoName, content)
const path = `${soulRoot}/${file.path}`
newPaths.add(path)
treeEntries.push({ path, mode: '100644', type: 'blob', sha: blobSha })
}
const existingMeta = await fetchMetaFile(
resolved.token,
resolved.repoOwner,
resolved.repoName,
`${soulRoot}/${META_FILENAME}`,
resolved.branch,
)
const metaPath = `${soulRoot}/${META_FILENAME}`
const metaDraft = buildMetaFile(params, existingMeta, resolved.repo, baseCommitSha, null)
const metaDraftContent = `${JSON.stringify(metaDraft, null, 2)}\n`
const metaDraftSha = await createBlob(
resolved.token,
resolved.repoOwner,
resolved.repoName,
toBase64(metaDraftContent),
)
newPaths.add(metaPath)
treeEntries.push({ path: metaPath, mode: '100644', type: 'blob', sha: metaDraftSha })
for (const path of existingPaths) {
if (newPaths.has(path)) continue
treeEntries.push({ path, mode: '100644', type: 'blob', sha: null })
}
const newTree = await githubPost<{ sha: string }>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees`,
{
base_tree: baseTreeSha,
tree: treeEntries,
},
)
const commit = await githubPost<GitCommit>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits`,
{
message: `soul: ${params.slug} v${params.version}`,
tree: newTree.sha,
parents: [baseCommitSha],
},
)
const metaFinal = buildMetaFile(params, existingMeta, resolved.repo, baseCommitSha, commit.sha)
const metaFinalContent = `${JSON.stringify(metaFinal, null, 2)}\n`
const metaFinalSha = await createBlob(
resolved.token,
resolved.repoOwner,
resolved.repoName,
toBase64(metaFinalContent),
)
const metaTree = await githubPost<{ sha: string }>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/trees`,
{
base_tree: commit.tree.sha,
tree: [{ path: metaPath, mode: '100644', type: 'blob', sha: metaFinalSha }],
},
)
const metaCommit = await githubPost<GitCommit>(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/commits`,
{
message: `meta: ${params.slug} v${params.version}`,
tree: metaTree.sha,
parents: [commit.sha],
},
)
await githubPatch(
resolved.token,
`/repos/${resolved.repoOwner}/${resolved.repoName}/git/refs/heads/${resolved.branch}`,
{
sha: metaCommit.sha,
},
)
}
function buildMetaFile(
params: BackupParams,
existing: MetaFile | null,
repo: string,
baseCommitSha: string,
latestCommitSha: string | null,
): MetaFile {
let history = [...(existing?.history ?? [])]
if (existing?.latest?.version) {
const previousCommit = existing.latest.commit ?? commitUrl(repo, baseCommitSha)
const previous = {
version: existing.latest.version,
publishedAt: existing.latest.publishedAt,
commit: previousCommit,
}
history = [previous, ...history.filter((entry) => entry.version !== previous.version)]
}
return {
owner: normalizeOwner(params.ownerHandle),
slug: params.slug,
displayName: params.displayName,
latest: {
version: params.version,
publishedAt: params.publishedAt,
commit: latestCommitSha ? commitUrl(repo, latestCommitSha) : null,
},
history: history.slice(0, 200),
}
}
async function fetchMetaFile(
token: string,
repoOwner: string,
repoName: string,
path: string,
branch: string,
): Promise<MetaFile | null> {
try {
const response = await githubGet<{ content?: string }>(
token,
`/repos/${repoOwner}/${repoName}/contents/${encodePath(path)}?ref=${branch}`,
)
if (!response.content) return null
const raw = fromBase64(response.content)
return JSON.parse(raw) as MetaFile
} catch (error) {
if (isNotFoundError(error)) return null
throw error
}
}
async function fetchStorageBase64(ctx: ActionCtx, storageId: Id<'_storage'>) {
const blob = await ctx.storage.get(storageId)
if (!blob) throw new Error('File missing in storage')
const buffer = Buffer.from(await blob.arrayBuffer())
return buffer.toString('base64')
}
async function createInstallationToken() {
const appId = process.env.GITHUB_APP_ID
const installationId = process.env.GITHUB_APP_INSTALLATION_ID
if (!appId || !installationId) {
throw new Error('GitHub App credentials missing')
}
const jwt = createAppJwt(appId)
const response = await fetch(`${GITHUB_API}/app/installations/${installationId}/access_tokens`, {
method: 'POST',
headers: buildHeaders(jwt, true),
})
if (!response.ok) {
const message = await response.text()
throw new Error(`GitHub App token failed: ${message}`)
}
const payload = (await response.json()) as { token?: string }
if (!payload.token) throw new Error('GitHub App token missing')
return payload.token
}
function createAppJwt(appId: string) {
const privateKey = loadPrivateKey()
const now = Math.floor(Date.now() / 1000)
const header = { alg: 'RS256', typ: 'JWT' }
const payload = { iat: now - 60, exp: now + 9 * 60, iss: appId }
const encodedHeader = base64Url(JSON.stringify(header))
const encodedPayload = base64Url(JSON.stringify(payload))
const signingInput = `${encodedHeader}.${encodedPayload}`
const sign = createSign('RSA-SHA256')
sign.update(signingInput)
sign.end()
const signature = sign.sign(privateKey)
return `${signingInput}.${base64Url(signature)}`
}
function loadPrivateKey() {
const raw = process.env.GITHUB_APP_PRIVATE_KEY
if (!raw) throw new Error('GITHUB_APP_PRIVATE_KEY is not configured')
const normalized = raw.replace(/\\n/g, '\n')
return createPrivateKey(normalized)
}
async function createBlob(token: string, repoOwner: string, repoName: string, content: string) {
const result = await githubPost<{ sha: string }>(
token,
`/repos/${repoOwner}/${repoName}/git/blobs`,
{
content,
encoding: 'base64',
},
)
if (!result.sha) throw new Error('GitHub blob missing sha')
return result.sha
}
async function githubGet<T>(token: string, path: string): Promise<T> {
const response = await fetch(`${GITHUB_API}${path}`, {
headers: buildHeaders(token),
})
if (!response.ok) {
const message = await response.text()
throw new Error(`GitHub GET ${path} failed: ${message}`)
}
return (await response.json()) as T
}
async function githubPost<T>(token: string, path: string, body: unknown): Promise<T> {
const response = await fetch(`${GITHUB_API}${path}`, {
method: 'POST',
headers: buildHeaders(token),
body: JSON.stringify(body),
})
if (!response.ok) {
const message = await response.text()
throw new Error(`GitHub POST ${path} failed: ${message}`)
}
return (await response.json()) as T
}
async function githubPatch(token: string, path: string, body: unknown) {
const response = await fetch(`${GITHUB_API}${path}`, {
method: 'PATCH',
headers: buildHeaders(token),
body: JSON.stringify(body),
})
if (!response.ok) {
const message = await response.text()
throw new Error(`GitHub PATCH ${path} failed: ${message}`)
}
}
function buildHeaders(token: string, isAppJwt = false) {
return {
Authorization: `${isAppJwt ? 'Bearer' : 'token'} ${token}`,
Accept: 'application/vnd.github+json',
'User-Agent': USER_AGENT,
}
}
function parseRepo(repo: string) {
const [owner, name] = repo.split('/')
if (!owner || !name) throw new Error('GITHUB_SOULS_REPO must be owner/repo')
return [owner, name] as const
}
function normalizeOwner(value: string) {
const normalized = value
.trim()
.toLowerCase()
.replace(/[^a-z0-9-]/g, '-')
.replace(/-+/g, '-')
.replace(/^-+|-+$/g, '')
return normalized || 'unknown'
}
function commitUrl(repo: string, sha: string) {
return `https://github.com/${repo}/commit/${sha}`
}
function buildSoulRoot(root: string, ownerHandle: string, slug: string) {
const ownerSegment = normalizeOwner(ownerHandle)
return `${root}/${ownerSegment}/${slug}`
}
function encodePath(path: string) {
return path
.split('/')
.map((segment) => encodeURIComponent(segment))
.join('/')
}
function base64Url(value: string | Buffer) {
const buffer = typeof value === 'string' ? Buffer.from(value) : value
return buffer.toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/g, '')
}
function toBase64(value: string) {
return Buffer.from(value).toString('base64')
}
function fromBase64(value: string) {
return Buffer.from(value, 'base64').toString('utf8')
}
function isNotFoundError(error: unknown) {
return (
error instanceof Error && (error.message.includes('404') || error.message.includes('Not Found'))
)
}

View File

@ -157,12 +157,15 @@ export async function publishVersionForUser(
embedding,
})) as PublishResult
const owner = (await ctx.runQuery(api.users.getById, { userId })) as Doc<'users'> | null
const ownerHandle = owner?.handle ?? owner?.displayName ?? owner?.name ?? 'unknown'
void ctx.scheduler
.runAfter(0, internal.githubBackupsNode.backupSkillForPublishInternal, {
slug,
version,
displayName,
ownerHandle: userId,
ownerHandle,
files: sanitizedFiles,
publishedAt: Date.now(),
})

273
convex/lib/soulChangelog.ts Normal file
View File

@ -0,0 +1,273 @@
import { internal } from '../_generated/api'
import type { Doc } from '../_generated/dataModel'
import type { ActionCtx } from '../_generated/server'
const CHANGELOG_MODEL = process.env.OPENAI_CHANGELOG_MODEL ?? 'gpt-4.1'
const MAX_README_CHARS = 8_000
const MAX_PATHS_IN_PROMPT = 30
type FileMeta = { path: string; sha256?: string }
type FileDiffSummary = {
added: string[]
removed: string[]
changed: string[]
}
function clampText(value: string, maxChars: number) {
const trimmed = value.trim()
if (trimmed.length <= maxChars) return trimmed
return `${trimmed.slice(0, maxChars).trimEnd()}\n…`
}
function summarizeFileDiff(oldFiles: FileMeta[], nextFiles: FileMeta[]): FileDiffSummary {
const oldByPath = new Map(oldFiles.map((f) => [f.path, f] as const))
const nextByPath = new Map(nextFiles.map((f) => [f.path, f] as const))
const added: string[] = []
const removed: string[] = []
const changed: string[] = []
for (const [path, file] of nextByPath.entries()) {
const prev = oldByPath.get(path)
if (!prev) {
added.push(path)
continue
}
if (file.sha256 && prev.sha256 && file.sha256 !== prev.sha256) changed.push(path)
}
for (const path of oldByPath.keys()) {
if (!nextByPath.has(path)) removed.push(path)
}
added.sort()
removed.sort()
changed.sort()
return { added, removed, changed }
}
function formatDiffSummary(diff: FileDiffSummary) {
const parts: string[] = []
if (diff.added.length) parts.push(`${diff.added.length} added`)
if (diff.changed.length) parts.push(`${diff.changed.length} changed`)
if (diff.removed.length) parts.push(`${diff.removed.length} removed`)
return parts.join(', ') || 'no file changes detected'
}
function pickPaths(values: string[]) {
if (values.length <= MAX_PATHS_IN_PROMPT) return values
return values.slice(0, MAX_PATHS_IN_PROMPT)
}
function extractResponseText(payload: unknown) {
if (!payload || typeof payload !== 'object') return null
const output = (payload as { output?: unknown }).output
if (!Array.isArray(output)) return null
const chunks: string[] = []
for (const item of output) {
if (!item || typeof item !== 'object') continue
if ((item as { type?: unknown }).type !== 'message') continue
const content = (item as { content?: unknown }).content
if (!Array.isArray(content)) continue
for (const part of content) {
if (!part || typeof part !== 'object') continue
if ((part as { type?: unknown }).type !== 'output_text') continue
const text = (part as { text?: unknown }).text
if (typeof text === 'string' && text.trim()) chunks.push(text)
}
}
const joined = chunks.join('\n').trim()
return joined || null
}
async function generateWithOpenAI(args: {
slug: string
version: string
oldReadme: string | null
nextReadme: string
fileDiff: FileDiffSummary | null
}) {
const apiKey = process.env.OPENAI_API_KEY
if (!apiKey) return null
const oldReadme = args.oldReadme ? clampText(args.oldReadme, MAX_README_CHARS) : ''
const nextReadme = clampText(args.nextReadme, MAX_README_CHARS)
const fileDiff = args.fileDiff
const diffSummary = fileDiff ? formatDiffSummary(fileDiff) : 'unknown'
const changedPaths = fileDiff ? pickPaths(fileDiff.changed) : []
const addedPaths = fileDiff ? pickPaths(fileDiff.added) : []
const removedPaths = fileDiff ? pickPaths(fileDiff.removed) : []
const input = [
`Soul: ${args.slug}`,
`Version: ${args.version}`,
`File changes: ${diffSummary}`,
changedPaths.length ? `Changed files (sample): ${changedPaths.join(', ')}` : null,
addedPaths.length ? `Added files (sample): ${addedPaths.join(', ')}` : null,
removedPaths.length ? `Removed files (sample): ${removedPaths.join(', ')}` : null,
oldReadme ? `Previous SOUL.md:\n${oldReadme}` : null,
`New SOUL.md:\n${nextReadme}`,
]
.filter(Boolean)
.join('\n\n')
const response = await fetch('https://api.openai.com/v1/responses', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: CHANGELOG_MODEL,
instructions:
'Write a concise changelog for this soul version. Audience: everyone. Output plain text. Prefer 26 bullet points. If it is a big change, include a short 1-line summary first, then bullets. Dont mention that you are AI. Dont invent details; only use the inputs.',
input,
max_output_tokens: 220,
}),
})
if (!response.ok) return null
const payload = (await response.json()) as unknown
return extractResponseText(payload)
}
function generateFallback(args: {
slug: string
version: string
oldReadme: string | null
nextReadme: string
fileDiff: FileDiffSummary | null
}) {
const lines: string[] = []
if (!args.oldReadme) {
lines.push(`- Initial release.`)
return lines.join('\n')
}
const diff = args.fileDiff
if (diff) {
const parts: string[] = []
if (diff.added.length) parts.push(`added ${diff.added.length}`)
if (diff.changed.length) parts.push(`updated ${diff.changed.length}`)
if (diff.removed.length) parts.push(`removed ${diff.removed.length}`)
if (parts.length) lines.push(`- ${parts.join(', ')} file(s).`)
}
lines.push(`- Updated SOUL.md.`)
return lines.join('\n')
}
export async function generateSoulChangelogForPublish(
ctx: ActionCtx,
args: { slug: string; version: string; readmeText: string; files: FileMeta[] },
): Promise<string> {
try {
const soul = (await ctx.runQuery(internal.souls.getSoulBySlugInternal, {
slug: args.slug,
})) as Doc<'souls'> | null
const previous: Doc<'soulVersions'> | null =
soul?.latestVersionId && !soul.softDeletedAt
? ((await ctx.runQuery(internal.souls.getVersionByIdInternal, {
versionId: soul.latestVersionId,
})) as Doc<'soulVersions'> | null)
: null
const oldReadmeText: string | null = previous
? await readReadmeFromVersion(ctx, previous)
: null
const oldFiles = previous
? previous.files.map((file) => ({ path: file.path, sha256: file.sha256 }))
: []
const fileDiff = previous ? summarizeFileDiff(oldFiles, args.files) : null
const ai = await generateWithOpenAI({
slug: args.slug,
version: args.version,
oldReadme: oldReadmeText,
nextReadme: args.readmeText,
fileDiff,
}).catch(() => null)
return (
ai ??
generateFallback({
slug: args.slug,
version: args.version,
oldReadme: oldReadmeText,
nextReadme: args.readmeText,
fileDiff,
})
)
} catch {
return '- Updated soul.'
}
}
export async function generateSoulChangelogPreview(
ctx: ActionCtx,
args: {
slug: string
version: string
readmeText: string
filePaths?: string[]
},
): Promise<string> {
try {
const soul = (await ctx.runQuery(internal.souls.getSoulBySlugInternal, {
slug: args.slug,
})) as Doc<'souls'> | null
const previous: Doc<'soulVersions'> | null =
soul?.latestVersionId && !soul.softDeletedAt
? ((await ctx.runQuery(internal.souls.getVersionByIdInternal, {
versionId: soul.latestVersionId,
})) as Doc<'soulVersions'> | null)
: null
const oldReadmeText: string | null = previous
? await readReadmeFromVersion(ctx, previous)
: null
const oldPaths = previous ? previous.files.map((file) => file.path) : []
const nextPaths = args.filePaths ?? []
const diff = previous ? summarizeFileDiffFromPaths(oldPaths, nextPaths) : null
const ai = await generateWithOpenAI({
slug: args.slug,
version: args.version,
oldReadme: oldReadmeText,
nextReadme: args.readmeText,
fileDiff: diff,
}).catch(() => null)
return (
ai ??
generateFallback({
slug: args.slug,
version: args.version,
oldReadme: oldReadmeText,
nextReadme: args.readmeText,
fileDiff: diff,
})
)
} catch {
return '- Updated soul.'
}
}
async function readReadmeFromVersion(ctx: ActionCtx, version: Doc<'soulVersions'>) {
const file = version.files.find((entry) => entry.path.toLowerCase() === 'soul.md')
if (!file) return null
const blob = await ctx.storage.get(file.storageId)
if (!blob) return null
return blob.text()
}
function summarizeFileDiffFromPaths(oldPaths: string[], nextPaths: string[]) {
const oldFiles = oldPaths.map((path) => ({ path }))
const nextFiles = nextPaths.map((path) => ({ path }))
return summarizeFileDiff(oldFiles, nextFiles)
}
export const __test = {
summarizeFileDiff,
}

234
convex/lib/soulPublish.ts Normal file
View File

@ -0,0 +1,234 @@
import { ConvexError } from 'convex/values'
import semver from 'semver'
import { api, internal } from '../_generated/api'
import type { Doc, Id } from '../_generated/dataModel'
import type { ActionCtx } from '../_generated/server'
import { generateEmbedding } from './embeddings'
import {
buildEmbeddingText,
getFrontmatterMetadata,
getFrontmatterValue,
hashSkillFiles,
isTextFile,
parseFrontmatter,
sanitizePath,
} from './skills'
import { generateSoulChangelogForPublish } from './soulChangelog'
const MAX_TOTAL_BYTES = 50 * 1024 * 1024
const MAX_SUMMARY_LENGTH = 160
function deriveSoulSummary(readmeText: string) {
const lines = readmeText.split(/\r?\n/)
let inFrontmatter = false
for (const raw of lines) {
const trimmed = raw.trim()
if (!trimmed) continue
if (!inFrontmatter && trimmed === '---') {
inFrontmatter = true
continue
}
if (inFrontmatter) {
if (trimmed === '---') {
inFrontmatter = false
}
continue
}
const cleaned = trimmed.replace(/^#+\s*/, '')
if (!cleaned) continue
if (cleaned.length > MAX_SUMMARY_LENGTH) {
return `${cleaned.slice(0, MAX_SUMMARY_LENGTH - 3).trimEnd()}...`
}
return cleaned
}
return undefined
}
export type PublishResult = {
soulId: Id<'souls'>
versionId: Id<'soulVersions'>
embeddingId: Id<'soulEmbeddings'>
}
export type PublishVersionArgs = {
slug: string
displayName: string
version: string
changelog: string
tags?: string[]
source?: {
kind: 'github'
url: string
repo: string
ref: string
commit: string
path: string
importedAt: number
}
files: Array<{
path: string
size: number
storageId: Id<'_storage'>
sha256: string
contentType?: string
}>
}
export async function publishSoulVersionForUser(
ctx: ActionCtx,
userId: Id<'users'>,
args: PublishVersionArgs,
): Promise<PublishResult> {
const version = args.version.trim()
const slug = args.slug.trim().toLowerCase()
const displayName = args.displayName.trim()
if (!slug || !displayName) throw new ConvexError('Slug and display name required')
if (!/^[a-z0-9][a-z0-9-]*$/.test(slug)) {
throw new ConvexError('Slug must be lowercase and url-safe')
}
if (!semver.valid(version)) {
throw new ConvexError('Version must be valid semver')
}
const suppliedChangelog = args.changelog.trim()
const changelogSource = suppliedChangelog ? ('user' as const) : ('auto' as const)
const sanitizedFiles = args.files.map((file) => {
const path = sanitizePath(file.path)
if (!path) throw new ConvexError('Invalid file paths')
if (!isTextFile(path, file.contentType ?? undefined)) {
throw new ConvexError('Only text-based files are allowed')
}
return { ...file, path }
})
const totalBytes = sanitizedFiles.reduce((sum, file) => sum + file.size, 0)
if (totalBytes > MAX_TOTAL_BYTES) {
throw new ConvexError('Soul bundle exceeds 50MB limit')
}
const isSoulFile = (path: string) => path.toLowerCase() === 'soul.md'
const readmeFile = sanitizedFiles.find((file) => isSoulFile(file.path))
if (!readmeFile) throw new ConvexError('SOUL.md is required')
const nonSoulFiles = sanitizedFiles.filter((file) => !isSoulFile(file.path))
if (nonSoulFiles.length > 0) {
throw new ConvexError('Only SOUL.md is allowed for soul bundles')
}
const readmeText = await fetchText(ctx, readmeFile.storageId)
const frontmatter = parseFrontmatter(readmeText)
const summary = getFrontmatterValue(frontmatter, 'description') ?? deriveSoulSummary(readmeText)
const metadata = mergeSourceIntoMetadata(getFrontmatterMetadata(frontmatter), args.source)
const embeddingText = buildEmbeddingText({
frontmatter,
readme: readmeText,
otherFiles: [],
})
const fingerprint = await hashSkillFiles(
sanitizedFiles.map((file) => ({
path: file.path ?? '',
sha256: file.sha256,
})),
)
const changelogPromise =
changelogSource === 'user'
? Promise.resolve(suppliedChangelog)
: generateSoulChangelogForPublish(ctx, {
slug,
version,
readmeText,
files: sanitizedFiles.map((file) => ({ path: file.path ?? '', sha256: file.sha256 })),
})
const embeddingPromise = generateEmbedding(embeddingText)
const [changelogText, embedding] = await Promise.all([
changelogPromise,
embeddingPromise.catch((error) => {
throw new ConvexError(formatEmbeddingError(error))
}),
])
const publishResult = (await ctx.runMutation(internal.souls.insertVersion, {
userId,
slug,
displayName,
version,
changelog: changelogText,
changelogSource,
tags: args.tags?.map((tag) => tag.trim()).filter(Boolean),
fingerprint,
files: sanitizedFiles,
parsed: {
frontmatter,
metadata,
},
summary,
embedding,
})) as PublishResult
const owner = (await ctx.runQuery(api.users.getById, { userId })) as Doc<'users'> | null
const ownerHandle = owner?.handle ?? owner?.name ?? userId
void ctx.scheduler
.runAfter(0, internal.githubSoulBackupsNode.backupSoulForPublishInternal, {
slug,
version,
displayName,
ownerHandle,
files: sanitizedFiles,
publishedAt: Date.now(),
})
.catch((error) => {
console.error('GitHub soul backup scheduling failed', error)
})
return publishResult
}
function mergeSourceIntoMetadata(metadata: unknown, source: PublishVersionArgs['source']) {
if (!source) return metadata === undefined ? undefined : metadata
const sourceValue = {
kind: source.kind,
url: source.url,
repo: source.repo,
ref: source.ref,
commit: source.commit,
path: source.path,
importedAt: source.importedAt,
}
if (!metadata) return { source: sourceValue }
if (typeof metadata !== 'object' || Array.isArray(metadata)) return { source: sourceValue }
return { ...(metadata as Record<string, unknown>), source: sourceValue }
}
export async function fetchText(
ctx: { storage: { get: (id: Id<'_storage'>) => Promise<Blob | null> } },
storageId: Id<'_storage'>,
) {
const blob = await ctx.storage.get(storageId)
if (!blob) throw new Error('File missing in storage')
return blob.text()
}
function formatEmbeddingError(error: unknown) {
if (error instanceof Error) {
if (error.message.includes('OPENAI_API_KEY')) {
return 'OPENAI_API_KEY is not configured.'
}
if (error.message.startsWith('Embedding failed')) {
return error.message
}
}
return 'Embedding failed. Please try again.'
}
export const __test = {
getSummary: (frontmatter: Record<string, unknown>) =>
getFrontmatterValue(frontmatter, 'description'),
}

View File

@ -65,6 +65,27 @@ const skills = defineTable({
.index('by_updated', ['updatedAt'])
.index('by_batch', ['batch'])
const souls = defineTable({
slug: v.string(),
displayName: v.string(),
summary: v.optional(v.string()),
ownerUserId: v.id('users'),
latestVersionId: v.optional(v.id('soulVersions')),
tags: v.record(v.string(), v.id('soulVersions')),
softDeletedAt: v.optional(v.number()),
stats: v.object({
downloads: v.number(),
stars: v.number(),
versions: v.number(),
comments: v.number(),
}),
createdAt: v.number(),
updatedAt: v.number(),
})
.index('by_slug', ['slug'])
.index('by_owner', ['ownerUserId'])
.index('by_updated', ['updatedAt'])
const skillVersions = defineTable({
skillId: v.id('skills'),
version: v.string(),
@ -92,6 +113,32 @@ const skillVersions = defineTable({
.index('by_skill', ['skillId'])
.index('by_skill_version', ['skillId', 'version'])
const soulVersions = defineTable({
soulId: v.id('souls'),
version: v.string(),
fingerprint: v.optional(v.string()),
changelog: v.string(),
changelogSource: v.optional(v.union(v.literal('auto'), v.literal('user'))),
files: v.array(
v.object({
path: v.string(),
size: v.number(),
storageId: v.id('_storage'),
sha256: v.string(),
contentType: v.optional(v.string()),
}),
),
parsed: v.object({
frontmatter: v.record(v.string(), v.any()),
metadata: v.optional(v.any()),
}),
createdBy: v.id('users'),
createdAt: v.number(),
softDeletedAt: v.optional(v.number()),
})
.index('by_soul', ['soulId'])
.index('by_soul_version', ['soulId', 'version'])
const skillVersionFingerprints = defineTable({
skillId: v.id('skills'),
versionId: v.id('skillVersions'),
@ -102,6 +149,16 @@ const skillVersionFingerprints = defineTable({
.index('by_fingerprint', ['fingerprint'])
.index('by_skill_fingerprint', ['skillId', 'fingerprint'])
const soulVersionFingerprints = defineTable({
soulId: v.id('souls'),
versionId: v.id('soulVersions'),
fingerprint: v.string(),
createdAt: v.number(),
})
.index('by_version', ['versionId'])
.index('by_fingerprint', ['fingerprint'])
.index('by_soul_fingerprint', ['soulId', 'fingerprint'])
const skillEmbeddings = defineTable({
skillId: v.id('skills'),
versionId: v.id('skillVersions'),
@ -120,6 +177,24 @@ const skillEmbeddings = defineTable({
filterFields: ['visibility'],
})
const soulEmbeddings = defineTable({
soulId: v.id('souls'),
versionId: v.id('soulVersions'),
ownerId: v.id('users'),
embedding: v.array(v.number()),
isLatest: v.boolean(),
isApproved: v.boolean(),
visibility: v.string(),
updatedAt: v.number(),
})
.index('by_soul', ['soulId'])
.index('by_version', ['versionId'])
.vectorIndex('by_embedding', {
vectorField: 'embedding',
dimensions: EMBEDDING_DIMENSIONS,
filterFields: ['visibility'],
})
const comments = defineTable({
skillId: v.id('skills'),
userId: v.id('users'),
@ -131,6 +206,17 @@ const comments = defineTable({
.index('by_skill', ['skillId'])
.index('by_user', ['userId'])
const soulComments = defineTable({
soulId: v.id('souls'),
userId: v.id('users'),
body: v.string(),
createdAt: v.number(),
softDeletedAt: v.optional(v.number()),
deletedBy: v.optional(v.id('users')),
})
.index('by_soul', ['soulId'])
.index('by_user', ['userId'])
const stars = defineTable({
skillId: v.id('skills'),
userId: v.id('users'),
@ -140,6 +226,15 @@ const stars = defineTable({
.index('by_user', ['userId'])
.index('by_skill_user', ['skillId', 'userId'])
const soulStars = defineTable({
soulId: v.id('souls'),
userId: v.id('users'),
createdAt: v.number(),
})
.index('by_soul', ['soulId'])
.index('by_user', ['userId'])
.index('by_soul_user', ['soulId', 'userId'])
const auditLogs = defineTable({
actorUserId: v.id('users'),
action: v.string(),
@ -221,11 +316,17 @@ export default defineSchema({
...authTables,
users,
skills,
souls,
skillVersions,
soulVersions,
skillVersionFingerprints,
soulVersionFingerprints,
skillEmbeddings,
soulEmbeddings,
comments,
soulComments,
stars,
soulStars,
auditLogs,
apiTokens,
rateLimits,

View File

@ -66,3 +66,61 @@ export const hydrateResults = internalQuery({
return entries
},
})
type HydratedSoulEntry = {
embeddingId: Id<'soulEmbeddings'>
soul: Doc<'souls'> | null
version: Doc<'soulVersions'> | null
}
type SoulSearchResult = HydratedSoulEntry & { score: number }
export const searchSouls: ReturnType<typeof action> = action({
args: {
query: v.string(),
limit: v.optional(v.number()),
},
handler: async (ctx, args): Promise<SoulSearchResult[]> => {
const query = args.query.trim()
if (!query) return []
const vector = await generateEmbedding(query)
const results = await ctx.vectorSearch('soulEmbeddings', 'by_embedding', {
vector,
limit: args.limit ?? 10,
filter: (q) => q.or(q.eq('visibility', 'latest'), q.eq('visibility', 'latest-approved')),
})
const hydrated = (await ctx.runQuery(internal.search.hydrateSoulResults, {
embeddingIds: results.map((result) => result._id),
})) as HydratedSoulEntry[]
const scoreById = new Map<Id<'soulEmbeddings'>, number>(
results.map((result) => [result._id, result._score]),
)
return hydrated
.map((entry) => ({
...entry,
score: scoreById.get(entry.embeddingId) ?? 0,
}))
.filter((entry) => entry.soul)
},
})
export const hydrateSoulResults = internalQuery({
args: { embeddingIds: v.array(v.id('soulEmbeddings')) },
handler: async (ctx, args): Promise<HydratedSoulEntry[]> => {
const entries: HydratedSoulEntry[] = []
for (const embeddingId of args.embeddingIds) {
const embedding = await ctx.db.get(embeddingId)
if (!embedding) continue
const soul = await ctx.db.get(embedding.soulId)
if (soul?.softDeletedAt) continue
const version = await ctx.db.get(embedding.versionId)
entries.push({ embeddingId, soul, version })
}
return entries
},
})

224
convex/seed.ts Normal file
View File

@ -0,0 +1,224 @@
import { v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import type { ActionCtx, DatabaseReader } from './_generated/server'
import { action, internalMutation, internalQuery } from './_generated/server'
import { publishSoulVersionForUser } from './lib/soulPublish'
import { SOUL_SEED_DISPLAY_NAME, SOUL_SEED_HANDLE, SOUL_SEED_KEY, SOUL_SEEDS } from './seedSouls'
const SEED_LOCK_STALE_MS = 10 * 60 * 1000
type SeedStateDoc = Doc<'githubBackupSyncState'>
async function getSeedState(ctx: { db: DatabaseReader }): Promise<SeedStateDoc | null> {
const entries = (await ctx.db
.query('githubBackupSyncState')
.withIndex('by_key', (q) => q.eq('key', SOUL_SEED_KEY))
.order('desc')
.take(2)) as SeedStateDoc[]
return entries[0] ?? null
}
export const getSoulSeedStateInternal = internalQuery({
args: {},
handler: async (ctx) => getSeedState(ctx),
})
export const setSoulSeedStateInternal = internalMutation({
args: { status: v.string() },
handler: async (ctx, args) => {
const existing = await getSeedState(ctx)
const now = Date.now()
if (existing) {
await ctx.db.patch(existing._id, { cursor: args.status, updatedAt: now })
return existing._id
}
return ctx.db.insert('githubBackupSyncState', {
key: SOUL_SEED_KEY,
cursor: args.status,
updatedAt: now,
})
},
})
export const tryStartSoulSeedInternal = internalMutation({
args: {},
handler: async (ctx) => {
const now = Date.now()
const existing = await getSeedState(ctx)
const cursor = existing?.cursor ?? null
if (cursor === 'done') return { started: false, reason: 'done' as const }
if (cursor === 'running' && existing && now - existing.updatedAt < SEED_LOCK_STALE_MS) {
return { started: false, reason: 'running' as const }
}
if (existing) {
await ctx.db.patch(existing._id, { cursor: 'running', updatedAt: now })
return { started: true, reason: 'patched' as const }
}
await ctx.db.insert('githubBackupSyncState', {
key: SOUL_SEED_KEY,
cursor: 'running',
updatedAt: now,
})
return { started: true, reason: 'inserted' as const }
},
})
export const hasAnySoulsInternal = internalQuery({
args: {},
handler: async (ctx) => {
const entry = await ctx.db.query('souls').take(1)
return entry.length > 0
},
})
export const ensureSoulSeeds = action({
args: {},
handler: async (ctx) => {
const started = (await ctx.runMutation(internal.seed.tryStartSoulSeedInternal, {})) as {
started: boolean
reason: 'done' | 'running' | 'patched' | 'inserted'
}
if (!started.started) {
if (started.reason === 'done') return { seeded: false, reason: 'already-seeded' as const }
return { seeded: false, reason: 'in-progress' as const }
}
const hasSouls = (await ctx.runQuery(internal.seed.hasAnySoulsInternal, {})) as boolean
if (hasSouls) {
await ctx.runMutation(internal.seed.setSoulSeedStateInternal, { status: 'done' })
return { seeded: false, reason: 'souls-exist' as const }
}
try {
const result = await runSeed(ctx)
await ctx.runMutation(internal.seed.setSoulSeedStateInternal, { status: 'done' })
return { seeded: true, reason: 'seeded' as const, ...result }
} catch (error) {
await ctx.runMutation(internal.seed.setSoulSeedStateInternal, { status: 'error' })
throw error
}
},
})
export const seed = action({
args: {},
handler: async (ctx) => runSeed(ctx),
})
async function runSeed(ctx: ActionCtx) {
const userId = (await ctx.runMutation(internal.seed.ensureSeedUserInternal, {
handle: SOUL_SEED_HANDLE,
displayName: SOUL_SEED_DISPLAY_NAME,
})) as Id<'users'>
const created: string[] = []
const skipped: string[] = []
for (const seedEntry of SOUL_SEEDS) {
const existing = (await ctx.runQuery(internal.souls.getSoulBySlugInternal, {
slug: seedEntry.slug,
})) as Doc<'souls'> | null
if (existing) {
if (existing.softDeletedAt && existing.ownerUserId === userId) {
await ctx.runMutation(internal.souls.setSoulSoftDeletedInternal, {
userId,
slug: seedEntry.slug,
deleted: false,
})
}
skipped.push(seedEntry.slug)
continue
}
const body = seedEntry.readme
if (!body) {
skipped.push(seedEntry.slug)
continue
}
const bytes = new TextEncoder().encode(body)
const sha256 = await sha256Hex(bytes)
const storageId = await ctx.storage.store(new Blob([bytes], { type: 'text/markdown' }))
try {
await publishSoulVersionForUser(ctx, userId, {
slug: seedEntry.slug,
displayName: seedEntry.displayName,
version: seedEntry.version,
changelog: '',
tags: seedEntry.tags,
files: [
{
path: 'SOUL.md',
size: bytes.byteLength,
storageId,
sha256,
contentType: 'text/markdown',
},
],
})
created.push(seedEntry.slug)
} catch (error) {
if (!isExpectedSeedSkipError(error)) throw error
skipped.push(seedEntry.slug)
}
}
return { created, skipped }
}
function isExpectedSeedSkipError(error: unknown) {
const message = error instanceof Error ? error.message : String(error)
return (
message.includes('Version already exists') || message.includes('Only the owner can publish')
)
}
export const ensureSeedUserInternal = internalMutation({
args: {
handle: v.string(),
displayName: v.string(),
},
handler: async (ctx, args) => {
const baseHandle = args.handle.trim()
const displayName = args.displayName.trim()
const candidates = [baseHandle, `${baseHandle}-bot`]
for (let i = 2; i <= 6; i += 1) candidates.push(`${baseHandle}-bot-${i}`)
for (const candidate of candidates) {
const existing = await ctx.db
.query('users')
.withIndex('handle', (q) => q.eq('handle', candidate))
.take(2)
const user = (existing[0] ?? null) as Doc<'users'> | null
if (user) {
if ((user.displayName ?? user.name) === displayName) return user._id
continue
}
return ctx.db.insert('users', {
handle: candidate,
displayName,
createdAt: Date.now(),
updatedAt: Date.now(),
})
}
throw new Error('Unable to allocate seed user handle')
},
})
async function sha256Hex(bytes: Uint8Array) {
const digest = await crypto.subtle.digest('SHA-256', bytes)
return toHex(new Uint8Array(digest))
}
function toHex(bytes: Uint8Array) {
let out = ''
for (const byte of bytes) out += byte.toString(16).padStart(2, '0')
return out
}

111
convex/seedSouls.ts Normal file

File diff suppressed because one or more lines are too long

87
convex/soulComments.ts Normal file
View File

@ -0,0 +1,87 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { assertRole, requireUser } from './lib/access'
export const listBySoul = query({
args: { soulId: v.id('souls'), limit: v.optional(v.number()) },
handler: async (ctx, args) => {
const limit = args.limit ?? 50
const comments = await ctx.db
.query('soulComments')
.withIndex('by_soul', (q) => q.eq('soulId', args.soulId))
.order('desc')
.take(limit)
const results: Array<{ comment: Doc<'soulComments'>; user: Doc<'users'> | null }> = []
for (const comment of comments) {
if (comment.softDeletedAt) continue
const user = await ctx.db.get(comment.userId)
results.push({ comment, user })
}
return results
},
})
export const add = mutation({
args: { soulId: v.id('souls'), body: v.string() },
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx)
const body = args.body.trim()
if (!body) throw new Error('Comment body required')
const soul = await ctx.db.get(args.soulId)
if (!soul) throw new Error('Soul not found')
await ctx.db.insert('soulComments', {
soulId: args.soulId,
userId,
body,
createdAt: Date.now(),
softDeletedAt: undefined,
deletedBy: undefined,
})
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, comments: soul.stats.comments + 1 },
updatedAt: Date.now(),
})
},
})
export const remove = mutation({
args: { commentId: v.id('soulComments') },
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
const comment = await ctx.db.get(args.commentId)
if (!comment) throw new Error('Comment not found')
if (comment.softDeletedAt) return
const isOwner = comment.userId === user._id
if (!isOwner) {
assertRole(user, ['admin', 'moderator'])
}
await ctx.db.patch(comment._id, {
softDeletedAt: Date.now(),
deletedBy: user._id,
})
const soul = await ctx.db.get(comment.soulId)
if (soul) {
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, comments: Math.max(0, soul.stats.comments - 1) },
updatedAt: Date.now(),
})
}
await ctx.db.insert('auditLogs', {
actorUserId: user._id,
action: 'soul.comment.delete',
targetType: 'soulComment',
targetId: comment._id,
metadata: { soulId: comment.soulId },
createdAt: Date.now(),
})
},
})

14
convex/soulDownloads.ts Normal file
View File

@ -0,0 +1,14 @@
import { v } from 'convex/values'
import { mutation } from './_generated/server'
export const increment = mutation({
args: { soulId: v.id('souls') },
handler: async (ctx, args) => {
const soul = await ctx.db.get(args.soulId)
if (!soul) return
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, downloads: soul.stats.downloads + 1 },
updatedAt: Date.now(),
})
},
})

69
convex/soulStars.ts Normal file
View File

@ -0,0 +1,69 @@
import { v } from 'convex/values'
import type { Doc } from './_generated/dataModel'
import { mutation, query } from './_generated/server'
import { requireUser } from './lib/access'
export const isStarred = query({
args: { soulId: v.id('souls') },
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx)
const existing = await ctx.db
.query('soulStars')
.withIndex('by_soul_user', (q) => q.eq('soulId', args.soulId).eq('userId', userId))
.unique()
return Boolean(existing)
},
})
export const toggle = mutation({
args: { soulId: v.id('souls') },
handler: async (ctx, args) => {
const { userId } = await requireUser(ctx)
const soul = await ctx.db.get(args.soulId)
if (!soul) throw new Error('Soul not found')
const existing = await ctx.db
.query('soulStars')
.withIndex('by_soul_user', (q) => q.eq('soulId', args.soulId).eq('userId', userId))
.unique()
if (existing) {
await ctx.db.delete(existing._id)
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, stars: Math.max(0, soul.stats.stars - 1) },
updatedAt: Date.now(),
})
return { starred: false }
}
await ctx.db.insert('soulStars', {
soulId: args.soulId,
userId,
createdAt: Date.now(),
})
await ctx.db.patch(soul._id, {
stats: { ...soul.stats, stars: soul.stats.stars + 1 },
updatedAt: Date.now(),
})
return { starred: true }
},
})
export const listByUser = query({
args: { userId: v.id('users'), limit: v.optional(v.number()) },
handler: async (ctx, args) => {
const limit = args.limit ?? 50
const stars = await ctx.db
.query('soulStars')
.withIndex('by_user', (q) => q.eq('userId', args.userId))
.order('desc')
.take(limit)
const souls: Doc<'souls'>[] = []
for (const star of stars) {
const soul = await ctx.db.get(star.soulId)
if (soul) souls.push(soul)
}
return souls
},
})

554
convex/souls.ts Normal file
View File

@ -0,0 +1,554 @@
import { ConvexError, v } from 'convex/values'
import { internal } from './_generated/api'
import type { Doc, Id } from './_generated/dataModel'
import { action, internalMutation, internalQuery, mutation, query } from './_generated/server'
import { assertRole, requireUser, requireUserFromAction } from './lib/access'
import { getFrontmatterValue, hashSkillFiles } from './lib/skills'
import { generateSoulChangelogPreview } from './lib/soulChangelog'
import { fetchText, type PublishResult, publishSoulVersionForUser } from './lib/soulPublish'
export { publishSoulVersionForUser } from './lib/soulPublish'
type ReadmeResult = { path: string; text: string }
type FileTextResult = { path: string; text: string; size: number; sha256: string }
const MAX_DIFF_FILE_BYTES = 200 * 1024
const MAX_LIST_LIMIT = 50
export const getBySlug = query({
args: { slug: v.string() },
handler: async (ctx, args) => {
const matches = await ctx.db
.query('souls')
.withIndex('by_slug', (q) => q.eq('slug', args.slug))
.order('desc')
.take(2)
const soul = matches[0] ?? null
if (!soul || soul.softDeletedAt) return null
const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
const owner = await ctx.db.get(soul.ownerUserId)
return { soul, latestVersion, owner }
},
})
export const getSoulBySlugInternal = internalQuery({
args: { slug: v.string() },
handler: async (ctx, args) => {
const matches = await ctx.db
.query('souls')
.withIndex('by_slug', (q) => q.eq('slug', args.slug))
.order('desc')
.take(2)
return matches[0] ?? null
},
})
export const list = query({
args: {
ownerUserId: v.optional(v.id('users')),
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const limit = args.limit ?? 24
const ownerUserId = args.ownerUserId
if (ownerUserId) {
const entries = await ctx.db
.query('souls')
.withIndex('by_owner', (q) => q.eq('ownerUserId', ownerUserId))
.order('desc')
.take(limit * 5)
return entries.filter((soul) => !soul.softDeletedAt).slice(0, limit)
}
const entries = await ctx.db
.query('souls')
.order('desc')
.take(limit * 5)
return entries.filter((soul) => !soul.softDeletedAt).slice(0, limit)
},
})
export const listPublicPage = query({
args: {
cursor: v.optional(v.string()),
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const limit = clampInt(args.limit ?? 24, 1, MAX_LIST_LIMIT)
const { page, isDone, continueCursor } = await ctx.db
.query('souls')
.withIndex('by_updated', (q) => q)
.order('desc')
.paginate({ cursor: args.cursor ?? null, numItems: limit })
const items: Array<{ soul: Doc<'souls'>; latestVersion: Doc<'soulVersions'> | null }> = []
for (const soul of page) {
if (soul.softDeletedAt) continue
const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
items.push({ soul, latestVersion })
}
return { items, nextCursor: isDone ? null : continueCursor }
},
})
export const listVersions = query({
args: { soulId: v.id('souls'), limit: v.optional(v.number()) },
handler: async (ctx, args) => {
const limit = args.limit ?? 20
return ctx.db
.query('soulVersions')
.withIndex('by_soul', (q) => q.eq('soulId', args.soulId))
.order('desc')
.take(limit)
},
})
export const listVersionsPage = query({
args: {
soulId: v.id('souls'),
cursor: v.optional(v.string()),
limit: v.optional(v.number()),
},
handler: async (ctx, args) => {
const limit = clampInt(args.limit ?? 20, 1, MAX_LIST_LIMIT)
const { page, isDone, continueCursor } = await ctx.db
.query('soulVersions')
.withIndex('by_soul', (q) => q.eq('soulId', args.soulId))
.order('desc')
.paginate({ cursor: args.cursor ?? null, numItems: limit })
const items = page.filter((version) => !version.softDeletedAt)
return { items, nextCursor: isDone ? null : continueCursor }
},
})
export const getVersionById = query({
args: { versionId: v.id('soulVersions') },
handler: async (ctx, args) => ctx.db.get(args.versionId),
})
export const getVersionByIdInternal = internalQuery({
args: { versionId: v.id('soulVersions') },
handler: async (ctx, args) => ctx.db.get(args.versionId),
})
export const getVersionBySoulAndVersion = query({
args: { soulId: v.id('souls'), version: v.string() },
handler: async (ctx, args) => {
return ctx.db
.query('soulVersions')
.withIndex('by_soul_version', (q) => q.eq('soulId', args.soulId).eq('version', args.version))
.unique()
},
})
export const publishVersion: ReturnType<typeof action> = action({
args: {
slug: v.string(),
displayName: v.string(),
version: v.string(),
changelog: v.string(),
tags: v.optional(v.array(v.string())),
source: v.optional(
v.object({
kind: v.literal('github'),
url: v.string(),
repo: v.string(),
ref: v.string(),
commit: v.string(),
path: v.string(),
importedAt: v.number(),
}),
),
files: v.array(
v.object({
path: v.string(),
size: v.number(),
storageId: v.id('_storage'),
sha256: v.string(),
contentType: v.optional(v.string()),
}),
),
},
handler: async (ctx, args): Promise<PublishResult> => {
const { userId } = await requireUserFromAction(ctx)
return publishSoulVersionForUser(ctx, userId, args)
},
})
export const generateChangelogPreview = action({
args: {
slug: v.string(),
version: v.string(),
readmeText: v.string(),
filePaths: v.optional(v.array(v.string())),
},
handler: async (ctx, args) => {
await requireUserFromAction(ctx)
const changelog = await generateSoulChangelogPreview(ctx, {
slug: args.slug.trim().toLowerCase(),
version: args.version.trim(),
readmeText: args.readmeText,
filePaths: args.filePaths?.map((value) => value.trim()).filter(Boolean),
})
return { changelog, source: 'auto' as const }
},
})
export const getReadme: ReturnType<typeof action> = action({
args: { versionId: v.id('soulVersions') },
handler: async (ctx, args): Promise<ReadmeResult> => {
const version = (await ctx.runQuery(internal.souls.getVersionByIdInternal, {
versionId: args.versionId,
})) as Doc<'soulVersions'> | null
if (!version) throw new ConvexError('Version not found')
const readmeFile = version.files.find((file) => file.path.toLowerCase() === 'soul.md')
if (!readmeFile) throw new ConvexError('SOUL.md not found')
const text = await fetchText(ctx, readmeFile.storageId)
return { path: readmeFile.path, text }
},
})
export const getFileText: ReturnType<typeof action> = action({
args: { versionId: v.id('soulVersions'), path: v.string() },
handler: async (ctx, args): Promise<FileTextResult> => {
const version = (await ctx.runQuery(internal.souls.getVersionByIdInternal, {
versionId: args.versionId,
})) as Doc<'soulVersions'> | null
if (!version) throw new ConvexError('Version not found')
const normalizedPath = args.path.trim()
const normalizedLower = normalizedPath.toLowerCase()
const file =
version.files.find((entry) => entry.path === normalizedPath) ??
version.files.find((entry) => entry.path.toLowerCase() === normalizedLower)
if (!file) throw new ConvexError('File not found')
if (file.size > MAX_DIFF_FILE_BYTES) {
throw new ConvexError('File exceeds 200KB limit')
}
const text = await fetchText(ctx, file.storageId)
return { path: file.path, text, size: file.size, sha256: file.sha256 }
},
})
export const resolveVersionByHash = query({
args: { slug: v.string(), hash: v.string() },
handler: async (ctx, args) => {
const slug = args.slug.trim().toLowerCase()
const hash = args.hash.trim().toLowerCase()
if (!slug || !/^[a-f0-9]{64}$/.test(hash)) return null
const soulMatches = await ctx.db
.query('souls')
.withIndex('by_slug', (q) => q.eq('slug', slug))
.order('desc')
.take(2)
const soul = soulMatches[0] ?? null
if (!soul || soul.softDeletedAt) return null
const latestVersion = soul.latestVersionId ? await ctx.db.get(soul.latestVersionId) : null
const fingerprintMatches = await ctx.db
.query('soulVersionFingerprints')
.withIndex('by_soul_fingerprint', (q) => q.eq('soulId', soul._id).eq('fingerprint', hash))
.take(25)
let match: { version: string } | null = null
if (fingerprintMatches.length > 0) {
const newest = fingerprintMatches.reduce(
(best, entry) => (entry.createdAt > best.createdAt ? entry : best),
fingerprintMatches[0] as (typeof fingerprintMatches)[number],
)
const version = await ctx.db.get(newest.versionId)
if (version && !version.softDeletedAt) {
match = { version: version.version }
}
}
if (!match) {
const versions = await ctx.db
.query('soulVersions')
.withIndex('by_soul', (q) => q.eq('soulId', soul._id))
.order('desc')
.take(200)
for (const version of versions) {
if (version.softDeletedAt) continue
if (typeof version.fingerprint === 'string' && version.fingerprint === hash) {
match = { version: version.version }
break
}
const fingerprint = await hashSkillFiles(
version.files.map((file) => ({ path: file.path, sha256: file.sha256 })),
)
if (fingerprint === hash) {
match = { version: version.version }
break
}
}
}
return {
match,
latestVersion: latestVersion ? { version: latestVersion.version } : null,
}
},
})
export const updateTags = mutation({
args: {
soulId: v.id('souls'),
tags: v.array(v.object({ tag: v.string(), versionId: v.id('soulVersions') })),
},
handler: async (ctx, args) => {
const { user } = await requireUser(ctx)
const soul = await ctx.db.get(args.soulId)
if (!soul) throw new Error('Soul not found')
if (soul.ownerUserId !== user._id) {
assertRole(user, ['admin', 'moderator'])
}
const nextTags = { ...soul.tags }
for (const entry of args.tags) {
nextTags[entry.tag] = entry.versionId
}
const latestEntry = args.tags.find((entry) => entry.tag === 'latest')
await ctx.db.patch(soul._id, {
tags: nextTags,
latestVersionId: latestEntry ? latestEntry.versionId : soul.latestVersionId,
updatedAt: Date.now(),
})
if (latestEntry) {
const embeddings = await ctx.db
.query('soulEmbeddings')
.withIndex('by_soul', (q) => q.eq('soulId', soul._id))
.collect()
for (const embedding of embeddings) {
const isLatest = embedding.versionId === latestEntry.versionId
await ctx.db.patch(embedding._id, {
isLatest,
visibility: visibilityFor(isLatest, embedding.isApproved),
updatedAt: Date.now(),
})
}
}
},
})
export const insertVersion = internalMutation({
args: {
userId: v.id('users'),
slug: v.string(),
displayName: v.string(),
version: v.string(),
changelog: v.string(),
changelogSource: v.optional(v.union(v.literal('auto'), v.literal('user'))),
tags: v.optional(v.array(v.string())),
fingerprint: v.string(),
summary: v.optional(v.string()),
files: v.array(
v.object({
path: v.string(),
size: v.number(),
storageId: v.id('_storage'),
sha256: v.string(),
contentType: v.optional(v.string()),
}),
),
parsed: v.object({
frontmatter: v.record(v.string(), v.any()),
metadata: v.optional(v.any()),
}),
embedding: v.array(v.number()),
},
handler: async (ctx, args) => {
const userId = args.userId
const user = await ctx.db.get(userId)
if (!user || user.deletedAt) throw new Error('User not found')
const soulMatches = await ctx.db
.query('souls')
.withIndex('by_slug', (q) => q.eq('slug', args.slug))
.order('desc')
.take(2)
let soul = soulMatches[0] ?? null
if (soul && soul.ownerUserId !== userId) {
throw new Error('Only the owner can publish updates')
}
const now = Date.now()
if (!soul) {
const summary = args.summary ?? getFrontmatterValue(args.parsed.frontmatter, 'description')
const soulId = await ctx.db.insert('souls', {
slug: args.slug,
displayName: args.displayName,
summary: summary ?? undefined,
ownerUserId: userId,
latestVersionId: undefined,
tags: {},
softDeletedAt: undefined,
stats: {
downloads: 0,
stars: 0,
versions: 0,
comments: 0,
},
createdAt: now,
updatedAt: now,
})
soul = await ctx.db.get(soulId)
}
if (!soul) throw new Error('Soul creation failed')
const existingVersion = await ctx.db
.query('soulVersions')
.withIndex('by_soul_version', (q) => q.eq('soulId', soul._id).eq('version', args.version))
.unique()
if (existingVersion) {
throw new Error('Version already exists')
}
const versionId = await ctx.db.insert('soulVersions', {
soulId: soul._id,
version: args.version,
fingerprint: args.fingerprint,
changelog: args.changelog,
changelogSource: args.changelogSource,
files: args.files,
parsed: args.parsed,
createdBy: userId,
createdAt: now,
softDeletedAt: undefined,
})
const nextTags: Record<string, Id<'soulVersions'>> = { ...soul.tags }
nextTags.latest = versionId
for (const tag of args.tags ?? []) {
nextTags[tag] = versionId
}
const latestBefore = soul.latestVersionId
await ctx.db.patch(soul._id, {
displayName: args.displayName,
summary:
args.summary ?? getFrontmatterValue(args.parsed.frontmatter, 'description') ?? soul.summary,
latestVersionId: versionId,
tags: nextTags,
stats: { ...soul.stats, versions: soul.stats.versions + 1 },
softDeletedAt: undefined,
updatedAt: now,
})
const embeddingId = await ctx.db.insert('soulEmbeddings', {
soulId: soul._id,
versionId,
ownerId: userId,
embedding: args.embedding,
isLatest: true,
isApproved: true,
visibility: visibilityFor(true, true),
updatedAt: now,
})
if (latestBefore) {
const previousEmbedding = await ctx.db
.query('soulEmbeddings')
.withIndex('by_version', (q) => q.eq('versionId', latestBefore))
.unique()
if (previousEmbedding) {
await ctx.db.patch(previousEmbedding._id, {
isLatest: false,
visibility: visibilityFor(false, previousEmbedding.isApproved),
updatedAt: now,
})
}
}
await ctx.db.insert('soulVersionFingerprints', {
soulId: soul._id,
versionId,
fingerprint: args.fingerprint,
createdAt: now,
})
return { soulId: soul._id, versionId, embeddingId }
},
})
export const setSoulSoftDeletedInternal = internalMutation({
args: {
userId: v.id('users'),
slug: v.string(),
deleted: v.boolean(),
},
handler: async (ctx, args) => {
const user = await ctx.db.get(args.userId)
if (!user || user.deletedAt) throw new Error('User not found')
const slug = args.slug.trim().toLowerCase()
if (!slug) throw new Error('Slug required')
const soulMatches = await ctx.db
.query('souls')
.withIndex('by_slug', (q) => q.eq('slug', slug))
.order('desc')
.take(2)
const soul = soulMatches[0] ?? null
if (!soul) throw new Error('Soul not found')
if (soul.ownerUserId !== args.userId) {
assertRole(user, ['admin', 'moderator'])
}
const now = Date.now()
await ctx.db.patch(soul._id, {
softDeletedAt: args.deleted ? now : undefined,
updatedAt: now,
})
const embeddings = await ctx.db
.query('soulEmbeddings')
.withIndex('by_soul', (q) => q.eq('soulId', soul._id))
.collect()
for (const embedding of embeddings) {
await ctx.db.patch(embedding._id, {
visibility: args.deleted
? 'deleted'
: visibilityFor(embedding.isLatest, embedding.isApproved),
updatedAt: now,
})
}
await ctx.db.insert('auditLogs', {
actorUserId: args.userId,
action: args.deleted ? 'soul.delete' : 'soul.undelete',
targetType: 'soul',
targetId: soul._id,
metadata: { slug, softDeletedAt: args.deleted ? now : null },
createdAt: now,
})
return { ok: true as const }
},
})
function visibilityFor(isLatest: boolean, isApproved: boolean) {
if (isLatest && isApproved) return 'latest-approved'
if (isLatest) return 'latest'
if (isApproved) return 'archived-approved'
return 'archived'
}
function clampInt(value: number, min: number, max: number) {
const rounded = Number.isFinite(value) ? Math.round(value) : min
return Math.min(max, Math.max(min, rounded))
}

37
docs/soul-format.md Normal file
View File

@ -0,0 +1,37 @@
---
summary: 'Soul bundle format, required files, limits.'
read_when:
- Publishing souls
- Debugging soul publish failures
---
# Soul format
## On disk
A soul is a single file:
- `SOUL.md` (or `soul.md`)
For now, onlycrabs.ai rejects any extra files.
## `SOUL.md`
- Markdown with optional YAML frontmatter.
- The server extracts metadata from frontmatter during publish.
- `description` is used as the soul summary in the UI/search.
## Limits
- Total bundle size: 50MB.
- Embedding text includes `SOUL.md` only.
## Slugs
- Derived from folder name by default.
- Must be lowercase and URL-safe: `^[a-z0-9][a-z0-9-]*$`.
## Versioning + tags
- Each publish creates a new version (semver).
- Tags are string pointers to a version; `latest` is commonly used.

View File

@ -9,6 +9,7 @@ read_when:
# ClawdHub — product + implementation spec (v1)
## Goals
- onlycrabs.ai mode for sharing `SOUL.md` bundles (host-based entry point).
- Minimal, fast SPA for browsing and publishing agent skills.
- Skills stored in Convex (files + metadata + versions + stats).
- GitHub OAuth login; GitHub App backs up skills to `clawdbot/skills`.
@ -63,6 +64,39 @@ From SKILL.md frontmatter + AgentSkills + Clawdis extensions:
`requires` (`bins`, `anyBins`, `env`, `config`), `install[]`
- `metadata` in frontmatter is YAML (object) preferred; legacy JSON-string accepted.
### Soul
- `slug` (unique)
- `displayName`
- `ownerUserId`
- `summary` (from SOUL.md frontmatter `description`)
- `latestVersionId`
- `tags` map: `{ tag -> versionId }`
- `stats`: `{ downloads, stars, versions, comments }`
- `status`: `active` only (soft-delete on version/comment only)
- `createdAt`, `updatedAt`
### SoulVersion
- `soulId`
- `version` (semver string)
- `tag` (string, optional; `latest` always maintained separately)
- `changelog` (required)
- `files`: list of file metadata (SOUL.md only)
- `path`, `size`, `storageId`, `sha256`
- `parsed` (metadata extracted from SOUL.md)
- `vectorDocId` (if using RAG component) OR `embeddingId`
- `createdBy`, `createdAt`
- `softDeletedAt` (nullable)
### SoulComment
- `soulId`, `userId`, `body`
- `softDeletedAt`, `deletedBy`
- `createdAt`
### SoulStar
- `soulId`, `userId`, `createdAt`
### Comment
- `skillId`, `userId`, `body`
- `softDeletedAt`, `deletedBy`
@ -94,6 +128,9 @@ From SKILL.md frontmatter + AgentSkills + Clawdis extensions:
- version uniqueness
5) Server stores files + metadata, sets `latest` tag, updates stats.
Soul upload flow: same as skills, but only `SOUL.md` is allowed in the bundle.
Seed data lives in `convex/seed.ts` for local dev.
## Versioning + tags
- Each upload is a new `SkillVersion`.
- `latest` tag always points to most recent version unless user re-tags.
@ -101,7 +138,7 @@ From SKILL.md frontmatter + AgentSkills + Clawdis extensions:
- Changelog is optional.
## Search
- Vector search over: SKILL.md + other text files + metadata summary.
- Vector search over: SKILL.md + other text files + metadata summary (souls index SOUL.md).
- Convex embeddings + vector index.
- Filters: tag, owner, `redactionApproved` only, min stars, updatedAt.

View File

@ -16,5 +16,6 @@ export const ApiRoutes = {
resolve: '/api/v1/resolve',
download: '/api/v1/download',
skills: '/api/v1/skills',
souls: '/api/v1/souls',
whoami: '/api/v1/whoami',
} as const

View File

@ -15,5 +15,6 @@ export declare const ApiRoutes: {
readonly resolve: "/api/v1/resolve";
readonly download: "/api/v1/download";
readonly skills: "/api/v1/skills";
readonly souls: "/api/v1/souls";
readonly whoami: "/api/v1/whoami";
};

View File

@ -15,6 +15,7 @@ export const ApiRoutes = {
resolve: '/api/v1/resolve',
download: '/api/v1/download',
skills: '/api/v1/skills',
souls: '/api/v1/souls',
whoami: '/api/v1/whoami',
};
//# sourceMappingURL=routes.js.map

View File

@ -1 +1 @@
{"version":3,"file":"routes.js","sourceRoot":"","sources":["../src/routes.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,eAAe,GAAG;IAC7B,QAAQ,EAAE,eAAe;IACzB,MAAM,EAAE,aAAa;IACrB,KAAK,EAAE,YAAY;IACnB,YAAY,EAAE,oBAAoB;IAClC,SAAS,EAAE,iBAAiB;IAC5B,YAAY,EAAE,qBAAqB;IACnC,UAAU,EAAE,kBAAkB;IAC9B,gBAAgB,EAAE,yBAAyB;IAC3C,cAAc,EAAE,uBAAuB;IACvC,gBAAgB,EAAE,yBAAyB;CACnC,CAAA;AAEV,MAAM,CAAC,MAAM,SAAS,GAAG;IACvB,MAAM,EAAE,gBAAgB;IACxB,OAAO,EAAE,iBAAiB;IAC1B,QAAQ,EAAE,kBAAkB;IAC5B,MAAM,EAAE,gBAAgB;IACxB,MAAM,EAAE,gBAAgB;CAChB,CAAA"}
{"version":3,"file":"routes.js","sourceRoot":"","sources":["../src/routes.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,eAAe,GAAG;IAC7B,QAAQ,EAAE,eAAe;IACzB,MAAM,EAAE,aAAa;IACrB,KAAK,EAAE,YAAY;IACnB,YAAY,EAAE,oBAAoB;IAClC,SAAS,EAAE,iBAAiB;IAC5B,YAAY,EAAE,qBAAqB;IACnC,UAAU,EAAE,kBAAkB;IAC9B,gBAAgB,EAAE,yBAAyB;IAC3C,cAAc,EAAE,uBAAuB;IACvC,gBAAgB,EAAE,yBAAyB;CACnC,CAAA;AAEV,MAAM,CAAC,MAAM,SAAS,GAAG;IACvB,MAAM,EAAE,gBAAgB;IACxB,OAAO,EAAE,iBAAiB;IAC1B,QAAQ,EAAE,kBAAkB;IAC5B,MAAM,EAAE,gBAAgB;IACxB,KAAK,EAAE,eAAe;IACtB,MAAM,EAAE,gBAAgB;CAChB,CAAA"}

View File

@ -57,6 +57,15 @@ export declare const CliPublishFileSchema: import("arktype/internal/variants/obj
contentType?: string | undefined;
}, {}>;
export type CliPublishFile = (typeof CliPublishFileSchema)[inferred];
export declare const PublishSourceSchema: import("arktype/internal/variants/object.ts").ObjectType<{
kind: "github";
url: string;
repo: string;
ref: string;
commit: string;
path: string;
importedAt: number;
}, {}>;
export declare const CliPublishRequestSchema: import("arktype/internal/variants/object.ts").ObjectType<{
slug: string;
displayName: string;
@ -70,6 +79,15 @@ export declare const CliPublishRequestSchema: import("arktype/internal/variants/
contentType?: string | undefined;
}[];
tags?: string[] | undefined;
source?: {
kind: "github";
url: string;
repo: string;
ref: string;
commit: string;
path: string;
importedAt: number;
} | undefined;
forkOf?: {
slug: string;
version?: string | undefined;

View File

@ -53,12 +53,22 @@ export const CliPublishFileSchema = type({
sha256: 'string',
contentType: 'string?',
});
export const PublishSourceSchema = type({
kind: '"github"',
url: 'string',
repo: 'string',
ref: 'string',
commit: 'string',
path: 'string',
importedAt: 'number',
});
export const CliPublishRequestSchema = type({
slug: 'string',
displayName: 'string',
version: 'string',
changelog: 'string',
tags: 'string[]?',
source: PublishSourceSchema.optional(),
forkOf: type({
slug: 'string',
version: 'string?',

File diff suppressed because one or more lines are too long

View File

@ -16,5 +16,6 @@ export const ApiRoutes = {
resolve: '/api/v1/resolve',
download: '/api/v1/download',
skills: '/api/v1/skills',
souls: '/api/v1/souls',
whoami: '/api/v1/whoami',
} as const

View File

@ -36,6 +36,30 @@ describe('clawdhub-schema', () => {
expect(payload.files[0]?.path).toBe('SKILL.md')
})
it('accepts publish payload with github source', () => {
const payload = parseArk(
CliPublishRequestSchema,
{
slug: 'demo',
displayName: 'Demo',
version: '1.0.0',
changelog: '',
source: {
kind: 'github',
url: 'https://github.com/example/demo',
repo: 'example/demo',
ref: 'main',
commit: 'abc123',
path: '.',
importedAt: 123,
},
files: [{ path: 'SKILL.md', size: 1, storageId: 's', sha256: 'x' }],
},
'Publish payload',
)
expect(payload.source?.repo).toBe('example/demo')
})
it('parses well-known config', () => {
expect(
parseArk(WellKnownConfigSchema, { registry: 'https://example.convex.site' }, 'WellKnown'),

View File

@ -67,12 +67,23 @@ export const CliPublishFileSchema = type({
})
export type CliPublishFile = (typeof CliPublishFileSchema)[inferred]
export const PublishSourceSchema = type({
kind: '"github"',
url: 'string',
repo: 'string',
ref: 'string',
commit: 'string',
path: 'string',
importedAt: 'number',
})
export const CliPublishRequestSchema = type({
slug: 'string',
displayName: 'string',
version: 'string',
changelog: 'string',
tags: 'string[]?',
source: PublishSourceSchema.optional(),
forkOf: type({
slug: 'string',
version: 'string?',

View File

@ -0,0 +1,27 @@
export type SoulOgMeta = {
displayName: string | null
summary: string | null
owner: string | null
version: string | null
}
export async function fetchSoulOgMeta(slug: string, apiBase: string): Promise<SoulOgMeta | null> {
try {
const url = new URL(`/api/v1/souls/${encodeURIComponent(slug)}`, apiBase)
const response = await fetch(url.toString(), { headers: { Accept: 'application/json' } })
if (!response.ok) return null
const payload = (await response.json()) as {
soul?: { displayName?: string; summary?: string | null } | null
owner?: { handle?: string | null } | null
latestVersion?: { version?: string | null } | null
}
return {
displayName: payload.soul?.displayName ?? null,
summary: payload.soul?.summary ?? null,
owner: payload.owner?.handle ?? null,
version: payload.latestVersion?.version ?? null,
}
} catch {
return null
}
}

209
server/og/soulOgSvg.ts Normal file
View File

@ -0,0 +1,209 @@
import { FONT_MONO, FONT_SANS } from './ogAssets'
export type SoulOgSvgParams = {
markDataUrl: string
title: string
description: string
ownerLabel: string
versionLabel: string
footer: string
}
function escapeXml(value: string) {
return value
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;')
}
function wrapText(value: string, maxChars: number, maxLines: number) {
const words = value.trim().split(/\s+/).filter(Boolean)
const lines: string[] = []
let current = ''
function pushLine(line: string) {
if (!line) return
lines.push(line)
}
function splitLongWord(word: string) {
if (word.length <= maxChars) return [word]
const parts: string[] = []
let remaining = word
while (remaining.length > maxChars) {
parts.push(`${remaining.slice(0, maxChars - 1)}`)
remaining = remaining.slice(maxChars - 1)
}
if (remaining) parts.push(remaining)
return parts
}
for (const word of words) {
if (word.length > maxChars) {
if (current) {
pushLine(current)
current = ''
if (lines.length >= maxLines - 1) break
}
const parts = splitLongWord(word)
for (const part of parts) {
pushLine(part)
if (lines.length >= maxLines) break
}
current = ''
if (lines.length >= maxLines - 1) break
continue
}
const next = current ? `${current} ${word}` : word
if (next.length <= maxChars) {
current = next
continue
}
pushLine(current)
current = word
if (lines.length >= maxLines - 1) break
}
if (lines.length < maxLines && current) pushLine(current)
if (lines.length > maxLines) lines.length = maxLines
const usedWords = lines.join(' ').split(/\s+/).filter(Boolean).length
if (usedWords < words.length) {
const last = lines.at(-1) ?? ''
const trimmed = last.length > maxChars ? last.slice(0, maxChars) : last
lines[lines.length - 1] = `${trimmed.replace(/\s+$/g, '').replace(/[.。,;:!?]+$/g, '')}`
}
return lines
}
export function buildSoulOgSvg(params: SoulOgSvgParams) {
const rawTitle = params.title.trim() || 'onlycrabs.ai'
const rawDescription = params.description.trim() || 'SOUL.md bundle on onlycrabs.ai.'
const cardX = 72
const cardY = 96
const cardW = 640
const cardH = 456
const cardR = 34
const titleLines = wrapText(rawTitle, 22, 2)
const descLines = wrapText(rawDescription, 42, 3)
const titleFontSize = titleLines.length > 1 || rawTitle.length > 24 ? 72 : 80
const titleY = titleLines.length > 1 ? 258 : 280
const titleLineHeight = 84
const descY = titleLines.length > 1 ? 395 : 380
const descLineHeight = 34
const pillText = `${params.ownerLabel}${params.versionLabel}`
const footerY = cardY + cardH - 18
const titleTspans = titleLines
.map((line, index) => {
const dy = index === 0 ? 0 : titleLineHeight
return `<tspan x="114" dy="${dy}">${escapeXml(line)}</tspan>`
})
.join('')
const descTspans = descLines
.map((line, index) => {
const dy = index === 0 ? 0 : descLineHeight
return `<tspan x="114" dy="${dy}">${escapeXml(line)}</tspan>`
})
.join('')
return `<?xml version="1.0" encoding="UTF-8"?>
<svg width="1200" height="630" viewBox="0 0 1200 630" fill="none" xmlns="http://www.w3.org/2000/svg">
<defs>
<linearGradient id="bg" x1="0" y1="0" x2="1200" y2="630" gradientUnits="userSpaceOnUse">
<stop stop-color="#0E1314"/>
<stop offset="0.55" stop-color="#142021"/>
<stop offset="1" stop-color="#0E1314"/>
</linearGradient>
<radialGradient id="glowGold" cx="0" cy="0" r="1" gradientUnits="userSpaceOnUse" gradientTransform="translate(300 80) rotate(120) scale(520 420)">
<stop stop-color="#E7B96B" stop-opacity="0.45"/>
<stop offset="1" stop-color="#E7B96B" stop-opacity="0"/>
</radialGradient>
<radialGradient id="glowTeal" cx="0" cy="0" r="1" gradientUnits="userSpaceOnUse" gradientTransform="translate(1040 140) rotate(140) scale(520 420)">
<stop stop-color="#6AD6C4" stop-opacity="0.35"/>
<stop offset="1" stop-color="#6AD6C4" stop-opacity="0"/>
</radialGradient>
<filter id="softBlur" x="-40%" y="-40%" width="180%" height="180%">
<feGaussianBlur stdDeviation="24"/>
</filter>
<filter id="cardShadow" x="-20%" y="-20%" width="140%" height="140%">
<feDropShadow dx="0" dy="18" stdDeviation="26" flood-color="#000000" flood-opacity="0.6"/>
</filter>
<linearGradient id="pill" x1="0" y1="0" x2="520" y2="0" gradientUnits="userSpaceOnUse">
<stop stop-color="#E7B96B" stop-opacity="0.26"/>
<stop offset="1" stop-color="#E7B96B" stop-opacity="0.12"/>
</linearGradient>
<linearGradient id="stroke" x1="0" y1="0" x2="0" y2="1">
<stop stop-color="#FFFFFF" stop-opacity="0.18"/>
<stop offset="1" stop-color="#FFFFFF" stop-opacity="0.08"/>
</linearGradient>
<clipPath id="cardClip">
<rect x="${cardX}" y="${cardY}" width="${cardW}" height="${cardH}" rx="${cardR}"/>
</clipPath>
</defs>
<rect width="1200" height="630" fill="url(#bg)"/>
<circle cx="300" cy="80" r="520" fill="url(#glowGold)" filter="url(#softBlur)"/>
<circle cx="1040" cy="140" r="520" fill="url(#glowTeal)" filter="url(#softBlur)"/>
<g opacity="0.12">
<path d="M0 90 C180 130 360 50 540 96 C720 142 840 220 1200 170" stroke="#FFFFFF" stroke-opacity="0.12" stroke-width="2"/>
<path d="M0 190 C240 250 400 170 600 214 C800 258 960 330 1200 300" stroke="#FFFFFF" stroke-opacity="0.1" stroke-width="2"/>
<path d="M0 450 C240 390 460 520 660 470 C860 420 1000 500 1200 460" stroke="#FFFFFF" stroke-opacity="0.08" stroke-width="2"/>
</g>
<g opacity="0.24" filter="url(#softBlur)">
<image href="${params.markDataUrl}" x="740" y="70" width="560" height="560" preserveAspectRatio="xMidYMid meet"/>
</g>
<g filter="url(#cardShadow)">
<rect x="${cardX}" y="${cardY}" width="${cardW}" height="${cardH}" rx="${cardR}" fill="#1B201F" fill-opacity="0.92" stroke="url(#stroke)"/>
</g>
<g clip-path="url(#cardClip)">
<image href="${params.markDataUrl}" x="108" y="134" width="46" height="46" preserveAspectRatio="xMidYMid meet"/>
<g>
<rect x="166" y="136" width="520" height="42" rx="21" fill="url(#pill)" stroke="#E7B96B" stroke-opacity="0.3"/>
<text x="186" y="163"
fill="#F7F1E8"
font-size="18"
font-weight="600"
font-family="${FONT_SANS}, sans-serif"
opacity="0.92">${escapeXml(pillText)}</text>
</g>
<text x="114" y="${titleY}"
fill="#F7F1E8"
font-size="${titleFontSize}"
font-weight="800"
font-family="${FONT_SANS}, sans-serif">${titleTspans}</text>
<text x="114" y="${descY}"
fill="#C7BFB5"
font-size="26"
font-weight="500"
font-family="${FONT_SANS}, sans-serif">${descTspans}</text>
<text x="114" y="${footerY}"
fill="#B7B0A6"
font-size="18"
font-family="${FONT_MONO}, monospace">${escapeXml(params.footer)}</text>
</g>
</svg>`
}

View File

@ -0,0 +1,112 @@
import { initWasm, Resvg } from '@resvg/resvg-wasm'
import { defineEventHandler, getQuery, getRequestHost, setHeader } from 'h3'
import type { SoulOgMeta } from '../../og/fetchSoulOgMeta'
import { fetchSoulOgMeta } from '../../og/fetchSoulOgMeta'
import {
FONT_MONO,
FONT_SANS,
getFontBuffers,
getMarkDataUrl,
getResvgWasm,
} from '../../og/ogAssets'
import { buildSoulOgSvg } from '../../og/soulOgSvg'
type OgQuery = {
slug?: string
owner?: string
version?: string
title?: string
description?: string
v?: string
}
let wasmInitPromise: Promise<void> | null = null
function cleanString(value: unknown) {
if (typeof value !== 'string') return ''
return value.trim()
}
function getApiBase(eventHost: string | null) {
const direct = process.env.VITE_CONVEX_SITE_URL?.trim()
if (direct) return direct
const site = process.env.SITE_URL?.trim() || process.env.VITE_SITE_URL?.trim()
if (site) return site
if (eventHost) return `https://${eventHost}`
return 'https://onlycrabs.ai'
}
async function ensureWasm() {
if (!wasmInitPromise) {
wasmInitPromise = getResvgWasm().then((wasm) => initWasm(wasm))
}
await wasmInitPromise
}
function buildFooter(host: string | null, slug: string, owner: string | null) {
const base = host ? host : 'onlycrabs.ai'
if (owner) return `${base}/@${owner}/${slug}`
return `${base}/souls/${slug}`
}
export default defineEventHandler(async (event) => {
const query = getQuery(event) as OgQuery
const slug = cleanString(query.slug)
if (!slug) {
setHeader(event, 'Content-Type', 'text/plain; charset=utf-8')
return 'Missing `slug` query param.'
}
const ownerFromQuery = cleanString(query.owner)
const versionFromQuery = cleanString(query.version)
const titleFromQuery = cleanString(query.title)
const descriptionFromQuery = cleanString(query.description)
const needFetch = !titleFromQuery || !descriptionFromQuery || !ownerFromQuery || !versionFromQuery
const meta: SoulOgMeta | null = needFetch
? await fetchSoulOgMeta(slug, getApiBase(getRequestHost(event)))
: null
const owner = ownerFromQuery || meta?.owner || ''
const version = versionFromQuery || meta?.version || ''
const title = titleFromQuery || meta?.displayName || slug
const description = descriptionFromQuery || meta?.summary || ''
const ownerLabel = owner ? `@${owner}` : 'onlycrabs.ai'
const versionLabel = version ? `v${version}` : 'latest'
const footer = buildFooter(getRequestHost(event), slug, owner || null)
const cacheKey = version ? 'public, max-age=31536000, immutable' : 'public, max-age=3600'
setHeader(event, 'Cache-Control', cacheKey)
setHeader(event, 'Content-Type', 'image/png')
const [markDataUrl, fontBuffers] = await Promise.all([
getMarkDataUrl(),
ensureWasm().then(() => getFontBuffers()),
])
const svg = buildSoulOgSvg({
markDataUrl,
title,
description,
ownerLabel,
versionLabel,
footer,
})
const resvg = new Resvg(svg, {
fitTo: { mode: 'width', value: 1200 },
font: {
fontBuffers,
defaultFontFamily: FONT_SANS,
sansSerifFamily: FONT_SANS,
monospaceFamily: FONT_MONO,
},
})
const png = resvg.render().asPng()
resvg.free()
return png
})

View File

@ -50,15 +50,11 @@ describe('Upload route', () => {
vi.unstubAllGlobals()
})
it('hides validation issues until submit', async () => {
it('shows validation issues before submit', async () => {
render(<Upload />)
const publishButton = screen.getByRole('button', { name: /publish/i })
expect(publishButton).toBeTruthy()
expect(screen.queryByText(/Slug is required/i)).toBeNull()
fireEvent.click(publishButton)
await waitFor(() => {
expect(screen.getByText(/Slug is required/i)).toBeTruthy()
})
expect(publishButton.getAttribute('disabled')).not.toBeNull()
expect(screen.getByText(/Slug is required/i)).toBeTruthy()
expect(screen.getByText(/Display name is required/i)).toBeTruthy()
})
@ -70,19 +66,19 @@ describe('Upload route', () => {
})
})
it('enables publish when fields and files are valid, and allows removing files', async () => {
it('enables publish when fields and files are valid', async () => {
generateUploadUrl.mockResolvedValue('https://upload.local')
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('my-skill-pack'), {
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'cool-skill' },
})
fireEvent.change(screen.getByPlaceholderText('My Skill Pack'), {
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'Cool Skill' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.2.3' },
})
fireEvent.change(screen.getByPlaceholderText('latest, beta'), {
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
const file = new File(['hello'], 'SKILL.md', { type: 'text/markdown' })
@ -90,26 +86,22 @@ describe('Upload route', () => {
fireEvent.change(input, { target: { files: [file] } })
const publishButton = screen.getByRole('button', { name: /publish/i }) as HTMLButtonElement
expect(await screen.findByText(/Ready to publish/i)).toBeTruthy()
fireEvent.click(screen.getByRole('button', { name: /remove/i }))
expect(screen.queryByText(/Add at least one file/i)).toBeNull()
fireEvent.click(publishButton)
expect(await screen.findByText(/Add at least one file/i)).toBeTruthy()
expect(await screen.findByText(/All checks passed/i)).toBeTruthy()
expect(publishButton.getAttribute('disabled')).toBeNull()
})
it('extracts zip uploads and unwraps top-level folders', async () => {
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('my-skill-pack'), {
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'cool-skill' },
})
fireEvent.change(screen.getByPlaceholderText('My Skill Pack'), {
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'Cool Skill' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.2.3' },
})
fireEvent.change(screen.getByPlaceholderText('latest, beta'), {
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
@ -125,23 +117,23 @@ describe('Upload route', () => {
expect(await screen.findByText('notes.txt', {}, { timeout: 3000 })).toBeTruthy()
expect(screen.getByText('SKILL.md')).toBeTruthy()
expect(await screen.findByText(/Ready to publish/i, {}, { timeout: 3000 })).toBeTruthy()
expect(await screen.findByText(/All checks passed/i, {}, { timeout: 3000 })).toBeTruthy()
})
it('unwraps folder uploads so SKILL.md can be at the top-level', async () => {
generateUploadUrl.mockResolvedValue('https://upload.local')
publishVersion.mockResolvedValue(undefined)
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('my-skill-pack'), {
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'ynab' },
})
fireEvent.change(screen.getByPlaceholderText('My Skill Pack'), {
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'YNAB' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.0.0' },
})
fireEvent.change(screen.getByPlaceholderText('latest, beta'), {
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
@ -152,7 +144,7 @@ describe('Upload route', () => {
fireEvent.change(input, { target: { files: [file] } })
expect(await screen.findByText('SKILL.md')).toBeTruthy()
expect(await screen.findByText(/Ready to publish/i)).toBeTruthy()
expect(await screen.findByText(/All checks passed/i)).toBeTruthy()
fireEvent.click(screen.getByRole('button', { name: /publish/i }))
await waitFor(() => {
@ -170,16 +162,16 @@ describe('Upload route', () => {
it('blocks non-text folder uploads (png)', async () => {
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('my-skill-pack'), {
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'cool-skill' },
})
fireEvent.change(screen.getByPlaceholderText('My Skill Pack'), {
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'Cool Skill' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.2.3' },
})
fireEvent.change(screen.getByPlaceholderText('latest, beta'), {
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
@ -200,26 +192,26 @@ describe('Upload route', () => {
publishVersion.mockRejectedValueOnce(new Error('Changelog is required'))
generateUploadUrl.mockResolvedValue('https://upload.local')
render(<Upload />)
fireEvent.change(screen.getByPlaceholderText('my-skill-pack'), {
fireEvent.change(screen.getByPlaceholderText('skill-name'), {
target: { value: 'cool-skill' },
})
fireEvent.change(screen.getByPlaceholderText('My Skill Pack'), {
fireEvent.change(screen.getByPlaceholderText('My skill'), {
target: { value: 'Cool Skill' },
})
fireEvent.change(screen.getByPlaceholderText('1.0.0'), {
target: { value: '1.2.3' },
})
fireEvent.change(screen.getByPlaceholderText('latest, beta'), {
fireEvent.change(screen.getByPlaceholderText('latest, stable'), {
target: { value: 'latest' },
})
fireEvent.change(screen.getByPlaceholderText('What changed in this version?'), {
fireEvent.change(screen.getByPlaceholderText('Describe what changed in this skill...'), {
target: { value: 'Initial drop.' },
})
const file = new File(['hello'], 'SKILL.md', { type: 'text/markdown' })
const input = screen.getByTestId('upload-input') as HTMLInputElement
fireEvent.change(input, { target: { files: [file] } })
const publishButton = screen.getByRole('button', { name: /publish/i }) as HTMLButtonElement
await screen.findByText(/Ready to publish/i)
await screen.findByText(/All checks passed/i)
fireEvent.click(publishButton)
expect(await screen.findByText(/Changelog is required/i)).toBeTruthy()
})

View File

@ -1,11 +1,14 @@
import { getSiteName } from '../lib/site'
export function Footer() {
const siteName = getSiteName()
return (
<footer className="site-footer">
<div className="site-footer-inner">
<div className="site-footer-divider" aria-hidden="true" />
<div className="site-footer-row">
<div className="site-footer-copy">
A{' '}
{siteName} · A{' '}
<a href="https://clawdbot.com" target="_blank" rel="noreferrer">
ClawdBot
</a>{' '}

View File

@ -2,9 +2,10 @@ import { useAuthActions } from '@convex-dev/auth/react'
import { Link } from '@tanstack/react-router'
import { useConvexAuth, useQuery } from 'convex/react'
import { Menu, Monitor, Moon, Sun } from 'lucide-react'
import { useRef } from 'react'
import { useMemo, useRef } from 'react'
import { api } from '../../convex/_generated/api'
import { gravatarUrl } from '../lib/gravatar'
import { getClawdHubSiteUrl, getOnlyCrabsSiteUrl, getSiteMode, getSiteName } from '../lib/site'
import { applyTheme, useThemeMode } from '../lib/theme'
import { startThemeTransition } from '../lib/theme-transition'
import {
@ -22,6 +23,11 @@ export default function Header() {
const me = useQuery(api.users.me)
const { mode, setMode } = useThemeMode()
const toggleRef = useRef<HTMLDivElement | null>(null)
const siteMode = getSiteMode()
const siteName = useMemo(() => getSiteName(siteMode), [siteMode])
const isSoulMode = siteMode === 'souls'
const onlyCrabsUrl = getOnlyCrabsSiteUrl()
const clawdHubUrl = getClawdHubSiteUrl()
const avatar = me?.image ?? (me?.email ? gravatarUrl(me.email) : undefined)
const handle = me?.handle ?? me?.displayName ?? 'user'
@ -48,25 +54,34 @@ export default function Header() {
<span className="brand-mark">
<img src="/clawd-logo.png" alt="" aria-hidden="true" />
</span>
<span className="brand-name">ClawdHub</span>
<span className="brand-name">{siteName}</span>
</Link>
<nav className="nav-links">
{isSoulMode ? (
<a href={clawdHubUrl}>ClawdHub</a>
) : (
<a href={onlyCrabsUrl}>onlycrabs.ai</a>
)}
<Link
to="/skills"
search={{
q: undefined,
sort: undefined,
dir: undefined,
highlighted: undefined,
view: undefined,
}}
to={isSoulMode ? '/souls' : '/skills'}
search={
isSoulMode
? undefined
: {
q: undefined,
sort: undefined,
dir: undefined,
highlighted: undefined,
view: undefined,
}
}
>
Skills
{isSoulMode ? 'Souls' : 'Skills'}
</Link>
<Link to="/upload" search={{ updateSlug: undefined }}>
Upload
</Link>
<Link to="/import">Import</Link>
{isSoulMode ? null : <Link to="/import">Import</Link>}
<Link to="/search" search={{ q: undefined, highlighted: undefined }}>
Search
</Link>
@ -82,18 +97,29 @@ export default function Header() {
</button>
</DropdownMenuTrigger>
<DropdownMenuContent align="end">
<DropdownMenuItem asChild>
{isSoulMode ? (
<a href={clawdHubUrl}>ClawdHub</a>
) : (
<a href={onlyCrabsUrl}>onlycrabs.ai</a>
)}
</DropdownMenuItem>
<DropdownMenuItem asChild>
<Link
to="/skills"
search={{
q: undefined,
sort: undefined,
dir: undefined,
highlighted: undefined,
view: undefined,
}}
to={isSoulMode ? '/souls' : '/skills'}
search={
isSoulMode
? undefined
: {
q: undefined,
sort: undefined,
dir: undefined,
highlighted: undefined,
view: undefined,
}
}
>
Skills
{isSoulMode ? 'Souls' : 'Skills'}
</Link>
</DropdownMenuItem>
<DropdownMenuItem asChild>
@ -101,9 +127,11 @@ export default function Header() {
Upload
</Link>
</DropdownMenuItem>
<DropdownMenuItem asChild>
<Link to="/import">Import</Link>
</DropdownMenuItem>
{isSoulMode ? null : (
<DropdownMenuItem asChild>
<Link to="/import">Import</Link>
</DropdownMenuItem>
)}
<DropdownMenuItem asChild>
<Link to="/search" search={{ q: undefined, highlighted: undefined }}>
Search

View File

@ -30,6 +30,7 @@ export function SkillDetailPage({
const setBatch = useMutation(api.skills.setBatch)
const getReadme = useAction(api.skills.getReadme)
const [readme, setReadme] = useState<string | null>(null)
const [readmeError, setReadmeError] = useState<string | null>(null)
const [comment, setComment] = useState('')
const [tagName, setTagName] = useState('latest')
const [tagVersionId, setTagVersionId] = useState<Id<'skillVersions'> | ''>('')
@ -107,11 +108,18 @@ export function SkillDetailPage({
useEffect(() => {
if (!latestVersion) return
setReadme(null)
setReadmeError(null)
let cancelled = false
void getReadme({ versionId: latestVersion._id }).then((data) => {
if (cancelled) return
setReadme(data.text)
})
void getReadme({ versionId: latestVersion._id })
.then((data) => {
if (cancelled) return
setReadme(data.text)
})
.catch((error) => {
if (cancelled) return
setReadmeError(error instanceof Error ? error.message : 'Failed to load README')
setReadme(null)
})
return () => {
cancelled = true
}
@ -383,9 +391,13 @@ export function SkillDetailPage({
SKILL.md
</h2>
<div className="markdown">
<ReactMarkdown remarkPlugins={[remarkGfm]}>
{readmeContent ?? 'Loading…'}
</ReactMarkdown>
{readmeContent ? (
<ReactMarkdown remarkPlugins={[remarkGfm]}>{readmeContent}</ReactMarkdown>
) : readmeError ? (
<div className="stat">Failed to load SKILL.md: {readmeError}</div>
) : (
<div>Loading</div>
)}
</div>
</div>
<div className="file-list">

View File

@ -0,0 +1,19 @@
import { Link } from '@tanstack/react-router'
import type { ReactNode } from 'react'
import type { Doc } from '../../convex/_generated/dataModel'
type SoulCardProps = {
soul: Doc<'souls'>
summaryFallback: string
meta: ReactNode
}
export function SoulCard({ soul, summaryFallback, meta }: SoulCardProps) {
return (
<Link to="/souls/$slug" params={{ slug: soul.slug }} className="card skill-card">
<h3 className="skill-card-title">{soul.displayName}</h3>
<p className="skill-card-summary">{soul.summary ?? summaryFallback}</p>
<div className="skill-card-footer">{meta}</div>
</Link>
)
}

View File

@ -0,0 +1,259 @@
import { useAction, useConvexAuth, useMutation, useQuery } from 'convex/react'
import { useEffect, useMemo, useRef, useState } from 'react'
import ReactMarkdown from 'react-markdown'
import remarkGfm from 'remark-gfm'
import { api } from '../../convex/_generated/api'
import type { Doc } from '../../convex/_generated/dataModel'
type SoulDetailPageProps = {
slug: string
}
export function SoulDetailPage({ slug }: SoulDetailPageProps) {
const { isAuthenticated } = useConvexAuth()
const me = useQuery(api.users.me)
const result = useQuery(api.souls.getBySlug, { slug })
const toggleStar = useMutation(api.soulStars.toggle)
const addComment = useMutation(api.soulComments.add)
const removeComment = useMutation(api.soulComments.remove)
const getReadme = useAction(api.souls.getReadme)
const ensureSoulSeeds = useAction(api.seed.ensureSoulSeeds)
const seedEnsuredRef = useRef(false)
const [readme, setReadme] = useState<string | null>(null)
const [readmeError, setReadmeError] = useState<string | null>(null)
const [comment, setComment] = useState('')
const isLoadingSoul = result === undefined
const soul = result?.soul
const owner = result?.owner
const latestVersion = result?.latestVersion
const versions = useQuery(
api.souls.listVersions,
soul ? { soulId: soul._id, limit: 50 } : 'skip',
) as Doc<'soulVersions'>[] | undefined
const isStarred = useQuery(
api.soulStars.isStarred,
isAuthenticated && soul ? { soulId: soul._id } : 'skip',
)
const comments = useQuery(
api.soulComments.listBySoul,
soul ? { soulId: soul._id, limit: 50 } : 'skip',
) as Array<{ comment: Doc<'soulComments'>; user: Doc<'users'> | null }> | undefined
const readmeContent = useMemo(() => {
if (!readme) return null
return stripFrontmatter(readme)
}, [readme])
useEffect(() => {
if (seedEnsuredRef.current) return
seedEnsuredRef.current = true
void ensureSoulSeeds({})
}, [ensureSoulSeeds])
useEffect(() => {
if (!latestVersion) return
setReadme(null)
setReadmeError(null)
let cancelled = false
void getReadme({ versionId: latestVersion._id })
.then((data) => {
if (cancelled) return
setReadme(data.text)
})
.catch((error) => {
if (cancelled) return
setReadmeError(error instanceof Error ? error.message : 'Failed to load SOUL.md')
setReadme(null)
})
return () => {
cancelled = true
}
}, [latestVersion, getReadme])
if (isLoadingSoul) {
return (
<main className="section">
<div className="card">
<div className="loading-indicator">Loading soul</div>
</div>
</main>
)
}
if (result === null || !soul) {
return (
<main className="section">
<div className="card">Soul not found.</div>
</main>
)
}
const ownerHandle = owner?.handle ?? owner?.name ?? null
const downloadBase = `${import.meta.env.VITE_CONVEX_SITE_URL}/api/v1/souls/${soul.slug}/file`
return (
<main className="section">
<div className="skill-detail-stack">
<div className="card skill-hero">
<div className="skill-hero-header">
<div className="skill-hero-title">
<h1 className="section-title" style={{ margin: 0 }}>
{soul.displayName}
</h1>
<p className="section-subtitle">{soul.summary ?? 'No summary provided.'}</p>
<div className="stat">
{soul.stats.stars} · {soul.stats.downloads} · {soul.stats.versions} versions
</div>
{ownerHandle ? (
<div className="stat">
by <a href={`/u/${ownerHandle}`}>@{ownerHandle}</a>
</div>
) : null}
<div className="skill-actions">
{isAuthenticated ? (
<button
className={`star-toggle${isStarred ? ' is-active' : ''}`}
type="button"
onClick={() => void toggleStar({ soulId: soul._id })}
aria-label={isStarred ? 'Unstar soul' : 'Star soul'}
>
<span aria-hidden="true"></span>
</button>
) : null}
</div>
</div>
<div className="skill-hero-cta">
<div className="skill-version-pill">
<span className="skill-version-label">Current version</span>
<strong>v{latestVersion?.version ?? '—'}</strong>
</div>
<a
className="btn btn-primary"
href={`${downloadBase}?path=SOUL.md`}
aria-label="Download SOUL.md"
>
Download SOUL.md
</a>
</div>
</div>
</div>
<div className="card">
<div className="skill-readme markdown">
{readmeContent ? (
<ReactMarkdown remarkPlugins={[remarkGfm]}>{readmeContent}</ReactMarkdown>
) : readmeError ? (
<div className="stat">Failed to load SOUL.md: {readmeError}</div>
) : (
<div className="loading-indicator">Loading SOUL.md</div>
)}
</div>
</div>
<div className="card">
<h2 className="section-title" style={{ fontSize: '1.2rem', marginBottom: 8 }}>
Versions
</h2>
<div className="version-scroll">
<div className="version-list">
{(versions ?? []).map((version) => (
<div key={version._id} className="version-row">
<div className="version-info">
<div>
v{version.version} · {new Date(version.createdAt).toLocaleDateString()}
{version.changelogSource === 'auto' ? (
<span style={{ color: 'var(--ink-soft)' }}> · auto</span>
) : null}
</div>
<div style={{ color: '#5c554e', whiteSpace: 'pre-wrap' }}>
{version.changelog}
</div>
</div>
<div className="version-actions">
<a
className="btn version-zip"
href={`${downloadBase}?path=SOUL.md&version=${encodeURIComponent(
version.version,
)}`}
>
SOUL.md
</a>
</div>
</div>
))}
</div>
</div>
</div>
<div className="card">
<h2 className="section-title" style={{ fontSize: '1.2rem', margin: 0 }}>
Comments
</h2>
{isAuthenticated ? (
<form
onSubmit={(event) => {
event.preventDefault()
if (!comment.trim()) return
void addComment({ soulId: soul._id, body: comment.trim() }).then(() =>
setComment(''),
)
}}
className="comment-form"
>
<textarea
className="comment-input"
rows={4}
value={comment}
onChange={(event) => setComment(event.target.value)}
placeholder="Leave a note…"
/>
<button className="btn comment-submit" type="submit">
Post comment
</button>
</form>
) : (
<p className="section-subtitle">Sign in to comment.</p>
)}
<div style={{ display: 'grid', gap: 12, marginTop: 16 }}>
{(comments ?? []).length === 0 ? (
<div className="stat">No comments yet.</div>
) : (
(comments ?? []).map((entry) => (
<div key={entry.comment._id} className="stat" style={{ alignItems: 'flex-start' }}>
<div>
<strong>@{entry.user?.handle ?? entry.user?.name ?? 'user'}</strong>
<div style={{ color: '#5c554e' }}>{entry.comment.body}</div>
</div>
{isAuthenticated &&
me &&
(me._id === entry.comment.userId ||
me.role === 'admin' ||
me.role === 'moderator') ? (
<button
className="btn"
type="button"
onClick={() => void removeComment({ commentId: entry.comment._id })}
>
Delete
</button>
) : null}
</div>
))
)}
</div>
</div>
</div>
</main>
)
}
function stripFrontmatter(content: string) {
const normalized = content.replace(/\r\n/g, '\n').replace(/\r/g, '\n')
if (!normalized.startsWith('---')) return content
const endIndex = normalized.indexOf('\n---', 3)
if (endIndex === -1) return content
return normalized.slice(endIndex + 4).replace(/^\n+/, '')
}

View File

@ -1,5 +1,5 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import { buildSkillMeta, fetchSkillMeta } from './og'
import { buildSkillMeta, buildSoulMeta, fetchSkillMeta, fetchSoulMeta } from './og'
describe('og helpers', () => {
afterEach(() => {
@ -27,6 +27,25 @@ describe('og helpers', () => {
expect(meta.image).not.toContain('description=')
})
it('builds soul metadata with summary', () => {
const meta = buildSoulMeta({
slug: 'north-star',
owner: 'someone',
displayName: 'North Star',
summary: 'Personal north star notes.',
version: '0.1.0',
})
expect(meta.title).toBe('North Star — onlycrabs.ai')
expect(meta.description).toBe('Personal north star notes.')
expect(meta.url).toContain('/souls/north-star')
expect(meta.owner).toBe('someone')
expect(meta.image).toContain('/og/soul.png?')
expect(meta.image).toContain('v=1')
expect(meta.image).toContain('slug=north-star')
expect(meta.image).toContain('owner=someone')
expect(meta.image).toContain('version=0.1.0')
})
it('uses defaults when owner and summary are missing', () => {
const meta = buildSkillMeta({ slug: 'parser' })
expect(meta.title).toBe('parser — ClawdHub')
@ -36,6 +55,15 @@ describe('og helpers', () => {
expect(meta.image).toContain('slug=parser')
})
it('uses soul defaults when owner and summary are missing', () => {
const meta = buildSoulMeta({ slug: 'signal' })
expect(meta.title).toBe('signal — onlycrabs.ai')
expect(meta.description).toMatch(/onlycrabs\.ai — the home for SOUL.md/i)
expect(meta.url).toContain('/souls/signal')
expect(meta.owner).toBeNull()
expect(meta.image).toContain('slug=signal')
})
it('truncates long descriptions', () => {
const longSummary = 'a'.repeat(240)
const meta = buildSkillMeta({ slug: 'long', summary: longSummary })
@ -63,6 +91,26 @@ describe('og helpers', () => {
})
})
it('fetches soul metadata when response is ok', async () => {
const fetchMock = vi.fn(async () => ({
ok: true,
json: async () => ({
soul: { displayName: 'North Star', summary: 'Signal' },
owner: { handle: 'steipete' },
latestVersion: { version: '0.1.0' },
}),
}))
vi.stubGlobal('fetch', fetchMock)
const meta = await fetchSoulMeta('north-star')
expect(meta).toEqual({
displayName: 'North Star',
summary: 'Signal',
owner: 'steipete',
version: '0.1.0',
})
})
it('returns null when response is not ok', async () => {
const fetchMock = vi.fn(async () => ({ ok: false }))
vi.stubGlobal('fetch', fetchMock)
@ -80,4 +128,14 @@ describe('og helpers', () => {
const meta = await fetchSkillMeta('weather')
expect(meta).toBeNull()
})
it('returns null when soul fetch throws', async () => {
const fetchMock = vi.fn(async () => {
throw new Error('network')
})
vi.stubGlobal('fetch', fetchMock)
const meta = await fetchSoulMeta('north-star')
expect(meta).toBeNull()
})
})

View File

@ -1,3 +1,5 @@
import { getClawdHubSiteUrl, getOnlyCrabsSiteUrl } from './site'
type SkillMetaSource = {
slug: string
owner?: string | null
@ -14,16 +16,39 @@ type SkillMeta = {
owner: string | null
}
const DEFAULT_SITE = 'https://clawdhub.com'
type SoulMetaSource = {
slug: string
owner?: string | null
displayName?: string | null
summary?: string | null
version?: string | null
}
type SoulMeta = {
title: string
description: string
image: string
url: string
owner: string | null
}
const DEFAULT_DESCRIPTION = 'ClawdHub — a fast skill registry for agents, with vector search.'
const DEFAULT_SOUL_DESCRIPTION =
'onlycrabs.ai — the home for SOUL.md bundles and personal system lore.'
const OG_SKILL_IMAGE_LAYOUT_VERSION = '5'
const OG_SOUL_IMAGE_LAYOUT_VERSION = '1'
export function getSiteUrl() {
return import.meta.env.VITE_SITE_URL ?? DEFAULT_SITE
return getClawdHubSiteUrl()
}
export function getSoulSiteUrl() {
return getOnlyCrabsSiteUrl()
}
export function getApiBase() {
return import.meta.env.VITE_CONVEX_SITE_URL ?? getSiteUrl()
const explicit = import.meta.env.VITE_CONVEX_SITE_URL?.trim()
return explicit || getSiteUrl()
}
export async function fetchSkillMeta(slug: string) {
@ -48,6 +73,28 @@ export async function fetchSkillMeta(slug: string) {
}
}
export async function fetchSoulMeta(slug: string) {
try {
const apiBase = getApiBase()
const url = new URL(`/api/v1/souls/${encodeURIComponent(slug)}`, apiBase)
const response = await fetch(url.toString(), { headers: { Accept: 'application/json' } })
if (!response.ok) return null
const payload = (await response.json()) as {
soul?: { displayName?: string; summary?: string | null } | null
owner?: { handle?: string | null } | null
latestVersion?: { version?: string | null } | null
}
return {
displayName: payload.soul?.displayName ?? null,
summary: payload.soul?.summary ?? null,
owner: payload.owner?.handle ?? null,
version: payload.latestVersion?.version ?? null,
}
} catch {
return null
}
}
export function buildSkillMeta(source: SkillMetaSource): SkillMeta {
const siteUrl = getSiteUrl()
const owner = clean(source.owner)
@ -72,6 +119,30 @@ export function buildSkillMeta(source: SkillMetaSource): SkillMeta {
}
}
export function buildSoulMeta(source: SoulMetaSource): SoulMeta {
const siteUrl = getSoulSiteUrl()
const owner = clean(source.owner)
const displayName = clean(source.displayName) || clean(source.slug)
const summary = clean(source.summary)
const version = clean(source.version)
const title = `${displayName} — onlycrabs.ai`
const description =
summary || (owner ? `Soul by @${owner} on onlycrabs.ai.` : DEFAULT_SOUL_DESCRIPTION)
const url = `${siteUrl}/souls/${source.slug}`
const imageParams = new URLSearchParams()
imageParams.set('v', OG_SOUL_IMAGE_LAYOUT_VERSION)
imageParams.set('slug', source.slug)
if (owner) imageParams.set('owner', owner)
if (version) imageParams.set('version', version)
return {
title,
description: truncate(description, 200),
image: `${siteUrl}/og/soul.png?${imageParams.toString()}`,
url,
owner: owner || null,
}
}
function clean(value?: string | null) {
return value?.trim() ?? ''
}

84
src/lib/site.ts Normal file
View File

@ -0,0 +1,84 @@
export type SiteMode = 'skills' | 'souls'
const DEFAULT_CLAWDHUB_SITE_URL = 'https://clawdhub.com'
const DEFAULT_ONLYCRABS_SITE_URL = 'https://onlycrabs.ai'
const DEFAULT_ONLYCRABS_HOST = 'onlycrabs.ai'
export function getClawdHubSiteUrl() {
return import.meta.env.VITE_SITE_URL ?? DEFAULT_CLAWDHUB_SITE_URL
}
export function getOnlyCrabsSiteUrl() {
const explicit = import.meta.env.VITE_SOULHUB_SITE_URL
if (explicit) return explicit
const siteUrl = import.meta.env.VITE_SITE_URL
if (siteUrl) {
try {
const url = new URL(siteUrl)
if (
url.hostname === 'localhost' ||
url.hostname === '127.0.0.1' ||
url.hostname === '0.0.0.0'
) {
return url.origin
}
} catch {
// ignore invalid URLs, fall through to default
}
}
return DEFAULT_ONLYCRABS_SITE_URL
}
export function getOnlyCrabsHost() {
return import.meta.env.VITE_SOULHUB_HOST ?? DEFAULT_ONLYCRABS_HOST
}
export function detectSiteMode(host?: string | null): SiteMode {
if (!host) return 'skills'
const onlyCrabsHost = getOnlyCrabsHost().toLowerCase()
const lower = host.toLowerCase()
if (lower === onlyCrabsHost || lower.endsWith(`.${onlyCrabsHost}`)) return 'souls'
return 'skills'
}
export function detectSiteModeFromUrl(value?: string | null): SiteMode {
if (!value) return 'skills'
try {
const host = new URL(value).hostname
return detectSiteMode(host)
} catch {
return detectSiteMode(value)
}
}
export function getSiteMode(): SiteMode {
if (typeof window !== 'undefined') {
return detectSiteMode(window.location.hostname)
}
const forced = import.meta.env.VITE_SITE_MODE
if (forced === 'souls' || forced === 'skills') return forced
const onlyCrabsSite = import.meta.env.VITE_SOULHUB_SITE_URL
if (onlyCrabsSite) return detectSiteModeFromUrl(onlyCrabsSite)
const siteUrl = import.meta.env.VITE_SITE_URL ?? process.env.SITE_URL
if (siteUrl) return detectSiteModeFromUrl(siteUrl)
return 'skills'
}
export function getSiteName(mode: SiteMode = getSiteMode()) {
return mode === 'souls' ? 'onlycrabs.ai' : 'ClawdHub'
}
export function getSiteDescription(mode: SiteMode = getSiteMode()) {
return mode === 'souls'
? 'onlycrabs.ai — the home for SOUL.md bundles and personal system lore.'
: 'ClawdHub — a fast skill registry for agents, with vector search.'
}
export function getSiteUrlForMode(mode: SiteMode = getSiteMode()) {
return mode === 'souls' ? getOnlyCrabsSiteUrl() : getClawdHubSiteUrl()
}

View File

@ -17,8 +17,10 @@ import { Route as ImportRouteImport } from './routes/import'
import { Route as DashboardRouteImport } from './routes/dashboard'
import { Route as AdminRouteImport } from './routes/admin'
import { Route as IndexRouteImport } from './routes/index'
import { Route as SoulsIndexRouteImport } from './routes/souls/index'
import { Route as SkillsIndexRouteImport } from './routes/skills/index'
import { Route as UHandleRouteImport } from './routes/u/$handle'
import { Route as SoulsSlugRouteImport } from './routes/souls/$slug'
import { Route as SkillsSlugRouteImport } from './routes/skills/$slug'
import { Route as CliAuthRouteImport } from './routes/cli/auth'
import { Route as OwnerSlugRouteImport } from './routes/$owner/$slug'
@ -63,6 +65,11 @@ const IndexRoute = IndexRouteImport.update({
path: '/',
getParentRoute: () => rootRouteImport,
} as any)
const SoulsIndexRoute = SoulsIndexRouteImport.update({
id: '/souls/',
path: '/souls/',
getParentRoute: () => rootRouteImport,
} as any)
const SkillsIndexRoute = SkillsIndexRouteImport.update({
id: '/skills/',
path: '/skills/',
@ -73,6 +80,11 @@ const UHandleRoute = UHandleRouteImport.update({
path: '/u/$handle',
getParentRoute: () => rootRouteImport,
} as any)
const SoulsSlugRoute = SoulsSlugRouteImport.update({
id: '/souls/$slug',
path: '/souls/$slug',
getParentRoute: () => rootRouteImport,
} as any)
const SkillsSlugRoute = SkillsSlugRouteImport.update({
id: '/skills/$slug',
path: '/skills/$slug',
@ -93,47 +105,53 @@ export interface FileRoutesByFullPath {
'/': typeof IndexRoute
'/admin': typeof AdminRoute
'/dashboard': typeof DashboardRoute
'/search': typeof SearchRoute
'/import': typeof ImportRoute
'/search': typeof SearchRoute
'/settings': typeof SettingsRoute
'/stars': typeof StarsRoute
'/upload': typeof UploadRoute
'/$owner/$slug': typeof OwnerSlugRoute
'/cli/auth': typeof CliAuthRoute
'/skills/$slug': typeof SkillsSlugRoute
'/souls/$slug': typeof SoulsSlugRoute
'/u/$handle': typeof UHandleRoute
'/skills': typeof SkillsIndexRoute
'/souls': typeof SoulsIndexRoute
}
export interface FileRoutesByTo {
'/': typeof IndexRoute
'/admin': typeof AdminRoute
'/dashboard': typeof DashboardRoute
'/search': typeof SearchRoute
'/import': typeof ImportRoute
'/search': typeof SearchRoute
'/settings': typeof SettingsRoute
'/stars': typeof StarsRoute
'/upload': typeof UploadRoute
'/$owner/$slug': typeof OwnerSlugRoute
'/cli/auth': typeof CliAuthRoute
'/skills/$slug': typeof SkillsSlugRoute
'/souls/$slug': typeof SoulsSlugRoute
'/u/$handle': typeof UHandleRoute
'/skills': typeof SkillsIndexRoute
'/souls': typeof SoulsIndexRoute
}
export interface FileRoutesById {
__root__: typeof rootRouteImport
'/': typeof IndexRoute
'/admin': typeof AdminRoute
'/dashboard': typeof DashboardRoute
'/search': typeof SearchRoute
'/import': typeof ImportRoute
'/search': typeof SearchRoute
'/settings': typeof SettingsRoute
'/stars': typeof StarsRoute
'/upload': typeof UploadRoute
'/$owner/$slug': typeof OwnerSlugRoute
'/cli/auth': typeof CliAuthRoute
'/skills/$slug': typeof SkillsSlugRoute
'/souls/$slug': typeof SoulsSlugRoute
'/u/$handle': typeof UHandleRoute
'/skills/': typeof SkillsIndexRoute
'/souls/': typeof SoulsIndexRoute
}
export interface FileRouteTypes {
fileRoutesByFullPath: FileRoutesByFullPath
@ -141,62 +159,70 @@ export interface FileRouteTypes {
| '/'
| '/admin'
| '/dashboard'
| '/search'
| '/import'
| '/search'
| '/settings'
| '/stars'
| '/upload'
| '/$owner/$slug'
| '/cli/auth'
| '/skills/$slug'
| '/souls/$slug'
| '/u/$handle'
| '/skills'
| '/souls'
fileRoutesByTo: FileRoutesByTo
to:
| '/'
| '/admin'
| '/dashboard'
| '/search'
| '/import'
| '/search'
| '/settings'
| '/stars'
| '/upload'
| '/$owner/$slug'
| '/cli/auth'
| '/skills/$slug'
| '/souls/$slug'
| '/u/$handle'
| '/skills'
| '/souls'
id:
| '__root__'
| '/'
| '/admin'
| '/dashboard'
| '/search'
| '/import'
| '/search'
| '/settings'
| '/stars'
| '/upload'
| '/$owner/$slug'
| '/cli/auth'
| '/skills/$slug'
| '/souls/$slug'
| '/u/$handle'
| '/skills/'
| '/souls/'
fileRoutesById: FileRoutesById
}
export interface RootRouteChildren {
IndexRoute: typeof IndexRoute
AdminRoute: typeof AdminRoute
DashboardRoute: typeof DashboardRoute
SearchRoute: typeof SearchRoute
ImportRoute: typeof ImportRoute
SearchRoute: typeof SearchRoute
SettingsRoute: typeof SettingsRoute
StarsRoute: typeof StarsRoute
UploadRoute: typeof UploadRoute
OwnerSlugRoute: typeof OwnerSlugRoute
CliAuthRoute: typeof CliAuthRoute
SkillsSlugRoute: typeof SkillsSlugRoute
SoulsSlugRoute: typeof SoulsSlugRoute
UHandleRoute: typeof UHandleRoute
SkillsIndexRoute: typeof SkillsIndexRoute
SoulsIndexRoute: typeof SoulsIndexRoute
}
declare module '@tanstack/react-router' {
@ -257,6 +283,13 @@ declare module '@tanstack/react-router' {
preLoaderRoute: typeof IndexRouteImport
parentRoute: typeof rootRouteImport
}
'/souls/': {
id: '/souls/'
path: '/souls'
fullPath: '/souls'
preLoaderRoute: typeof SoulsIndexRouteImport
parentRoute: typeof rootRouteImport
}
'/skills/': {
id: '/skills/'
path: '/skills'
@ -271,6 +304,13 @@ declare module '@tanstack/react-router' {
preLoaderRoute: typeof UHandleRouteImport
parentRoute: typeof rootRouteImport
}
'/souls/$slug': {
id: '/souls/$slug'
path: '/souls/$slug'
fullPath: '/souls/$slug'
preLoaderRoute: typeof SoulsSlugRouteImport
parentRoute: typeof rootRouteImport
}
'/skills/$slug': {
id: '/skills/$slug'
path: '/skills/$slug'
@ -299,26 +339,29 @@ const rootRouteChildren: RootRouteChildren = {
IndexRoute: IndexRoute,
AdminRoute: AdminRoute,
DashboardRoute: DashboardRoute,
SearchRoute: SearchRoute,
ImportRoute: ImportRoute,
SearchRoute: SearchRoute,
SettingsRoute: SettingsRoute,
StarsRoute: StarsRoute,
UploadRoute: UploadRoute,
OwnerSlugRoute: OwnerSlugRoute,
CliAuthRoute: CliAuthRoute,
SkillsSlugRoute: SkillsSlugRoute,
SoulsSlugRoute: SoulsSlugRoute,
UHandleRoute: UHandleRoute,
SkillsIndexRoute: SkillsIndexRoute,
SoulsIndexRoute: SoulsIndexRoute,
}
export const routeTree = rootRouteImport
._addFileChildren(rootRouteChildren)
._addFileTypes<FileRouteTypes>()
import type { getRouter } from './router.tsx'
import type { createStart } from '@tanstack/react-start'
import type { startInstance } from './start.ts'
declare module '@tanstack/react-start' {
interface Register {
ssr: true
router: Awaited<ReturnType<typeof getRouter>>
config: Awaited<ReturnType<typeof startInstance.getOptions>>
}
}

View File

@ -6,13 +6,16 @@ import { AppProviders } from '../components/AppProviders'
import { ClientOnly } from '../components/ClientOnly'
import { Footer } from '../components/Footer'
import Header from '../components/Header'
import { getSiteUrl } from '../lib/og'
import { getSiteDescription, getSiteMode, getSiteName, getSiteUrlForMode } from '../lib/site'
import appCss from '../styles.css?url'
export const Route = createRootRoute({
head: () => {
const siteUrl = getSiteUrl()
const mode = getSiteMode()
const siteName = getSiteName(mode)
const siteDescription = getSiteDescription(mode)
const siteUrl = getSiteUrlForMode(mode)
const ogImage = `${siteUrl}/og.png`
return {
@ -25,15 +28,15 @@ export const Route = createRootRoute({
content: 'width=device-width, initial-scale=1',
},
{
title: 'ClawdHub',
title: siteName,
},
{
name: 'description',
content: 'ClawdHub — a fast skill registry for agents, with vector search.',
content: siteDescription,
},
{
property: 'og:site_name',
content: 'ClawdHub',
content: siteName,
},
{
property: 'og:type',
@ -41,11 +44,11 @@ export const Route = createRootRoute({
},
{
property: 'og:title',
content: 'ClawdHub',
content: siteName,
},
{
property: 'og:description',
content: 'ClawdHub — a fast skill registry for agents, with vector search.',
content: siteDescription,
},
{
property: 'og:image',
@ -61,7 +64,7 @@ export const Route = createRootRoute({
},
{
property: 'og:image:alt',
content: 'ClawdHub — a fast skill registry for agents, with vector search.',
content: `${siteName}${siteDescription}`,
},
{
name: 'twitter:card',
@ -69,11 +72,11 @@ export const Route = createRootRoute({
},
{
name: 'twitter:title',
content: 'ClawdHub',
content: siteName,
},
{
name: 'twitter:description',
content: 'ClawdHub — a fast skill registry for agents, with vector search.',
content: siteDescription,
},
{
name: 'twitter:image',
@ -81,7 +84,7 @@ export const Route = createRootRoute({
},
{
name: 'twitter:image:alt',
content: 'ClawdHub — a fast skill registry for agents, with vector search.',
content: `${siteName}${siteDescription}`,
},
],
links: [

View File

@ -5,6 +5,8 @@ import { api } from '../../convex/_generated/api'
import type { Doc } from '../../convex/_generated/dataModel'
import { InstallSwitcher } from '../components/InstallSwitcher'
import { SkillCard } from '../components/SkillCard'
import { SoulCard } from '../components/SoulCard'
import { getSiteMode } from '../lib/site'
export const Route = createFileRoute('/')({
validateSearch: (search) => ({
@ -15,6 +17,11 @@ export const Route = createFileRoute('/')({
})
function Home() {
const mode = getSiteMode()
return mode === 'souls' ? <OnlyCrabsHome /> : <SkillsHome />
}
function SkillsHome() {
const navigate = Route.useNavigate()
const search = Route.useSearch()
const searchSkills = useAction(api.search.searchSkills)
@ -251,3 +258,200 @@ function Home() {
</main>
)
}
function OnlyCrabsHome() {
const navigate = Route.useNavigate()
const search = Route.useSearch()
const searchSouls = useAction(api.search.searchSouls)
const ensureSoulSeeds = useAction(api.seed.ensureSoulSeeds)
const latest = (useQuery(api.souls.list, { limit: 12 }) as Doc<'souls'>[]) ?? []
const [query, setQuery] = useState(search.q ?? '')
const [results, setResults] = useState<
Array<{ soul: Doc<'souls'>; version: Doc<'soulVersions'> | null; score: number }>
>([])
const [isSearching, setIsSearching] = useState(false)
const [searchMode, setSearchMode] = useState(Boolean(search.q))
const searchRequest = useRef(0)
const seedEnsuredRef = useRef(false)
const inputRef = useRef<HTMLInputElement | null>(null)
const trimmedQuery = useMemo(() => query.trim(), [query])
const hasQuery = trimmedQuery.length > 0
useEffect(() => {
setQuery(search.q ?? '')
if (search.q) {
setSearchMode(true)
}
}, [search.q])
useEffect(() => {
if (seedEnsuredRef.current) return
seedEnsuredRef.current = true
void ensureSoulSeeds({})
}, [ensureSoulSeeds])
useEffect(() => {
void navigate({
search: () => ({
q: trimmedQuery || undefined,
highlighted: undefined,
}),
replace: true,
})
}, [navigate, trimmedQuery])
useEffect(() => {
if (!trimmedQuery) {
setResults([])
setIsSearching(false)
return
}
searchRequest.current += 1
const requestId = searchRequest.current
setIsSearching(true)
const handle = window.setTimeout(() => {
void (async () => {
try {
const data = (await searchSouls({ query: trimmedQuery })) as Array<{
soul: Doc<'souls'>
version: Doc<'soulVersions'> | null
score: number
}>
if (requestId === searchRequest.current) {
setResults(data)
}
} finally {
if (requestId === searchRequest.current) {
setIsSearching(false)
}
}
})()
}, 220)
return () => window.clearTimeout(handle)
}, [searchSouls, trimmedQuery])
return (
<main>
<section className={`hero${searchMode ? ' search-mode' : ''}`}>
<div className="hero-inner">
<div className="hero-copy fade-up" data-delay="1">
<span className="hero-badge">SOUL.md, shared.</span>
<h1 className="hero-title">onlycrabs.ai, where system lore lives.</h1>
<p className="hero-subtitle">
Share SOUL.md bundles, version them like docs, and keep personal system lore in one
public place.
</p>
<div style={{ display: 'flex', gap: 12, marginTop: 20 }}>
<Link to="/upload" search={{ updateSlug: undefined }} className="btn btn-primary">
Publish a soul
</Link>
<Link
to="/souls"
search={{ q: undefined, sort: undefined, dir: undefined, view: undefined }}
className="btn"
>
Browse souls
</Link>
</div>
</div>
<div className="hero-card hero-search-card fade-up" data-delay="2">
<form
className="search-bar"
onSubmit={(event) => {
event.preventDefault()
if (!searchMode) setSearchMode(true)
inputRef.current?.focus()
}}
>
<span className="mono">/</span>
<input
ref={inputRef}
className="search-input"
placeholder="Search souls, prompts, or lore"
value={query}
onChange={(event) => setQuery(event.target.value)}
onFocus={() => setSearchMode(true)}
onKeyDown={(event) => {
if (event.key === 'Escape' && !trimmedQuery) {
setSearchMode(false)
inputRef.current?.blur()
}
}}
/>
</form>
{!searchMode ? (
<div className="hero-install" style={{ marginTop: 18 }}>
<div className="stat">Search souls. Versioned, readable, easy to remix.</div>
</div>
) : null}
</div>
</div>
</section>
{searchMode ? (
<section className="section">
<h2 className="section-title">Search results</h2>
<p className="section-subtitle">
{isSearching ? 'Searching now.' : 'Instant results as you type.'}
</p>
<div className="grid">
{!hasQuery ? (
<div className="card">Start typing to search.</div>
) : results.length === 0 ? (
<div className="card">No results yet. Try a different prompt.</div>
) : (
results.map((result) => (
<Link
key={result.soul._id}
to="/souls/$slug"
params={{ slug: result.soul.slug }}
className="card"
>
<div className="tag">Score {(result.score ?? 0).toFixed(2)}</div>
<h3 className="section-title" style={{ fontSize: '1.2rem', margin: 0 }}>
{result.soul.displayName}
</h3>
<p className="section-subtitle" style={{ margin: 0 }}>
{result.soul.summary ?? 'SOUL.md bundle'}
</p>
</Link>
))
)}
</div>
</section>
) : (
<section className="section">
<h2 className="section-title">Latest souls</h2>
<p className="section-subtitle">Newest SOUL.md bundles across the hub.</p>
<div className="grid">
{latest.length === 0 ? (
<div className="card">No souls yet. Be the first.</div>
) : (
latest.map((soul) => (
<SoulCard
key={soul._id}
soul={soul}
summaryFallback="A SOUL.md bundle."
meta={
<div className="stat">
{soul.stats.stars} · {soul.stats.downloads} · {soul.stats.versions} v
</div>
}
/>
))
)}
</div>
<div className="section-cta">
<Link
to="/souls"
search={{ q: undefined, sort: undefined, dir: undefined, view: undefined }}
className="btn"
>
See all souls
</Link>
</div>
</section>
)}
</main>
)
}

View File

@ -76,7 +76,10 @@ function SkillsIndex() {
case 'updated':
return (a.updatedAt - b.updatedAt) * multiplier
case 'name':
return a.displayName.localeCompare(b.displayName) || a.slug.localeCompare(b.slug)
return (
(a.displayName.localeCompare(b.displayName) || a.slug.localeCompare(b.slug)) *
multiplier
)
default:
return (a.createdAt - b.createdAt) * multiplier
}

View File

@ -0,0 +1,55 @@
import { createFileRoute } from '@tanstack/react-router'
import { SoulDetailPage } from '../../components/SoulDetailPage'
import { buildSoulMeta, fetchSoulMeta } from '../../lib/og'
export const Route = createFileRoute('/souls/$slug')({
loader: async ({ params }) => {
const data = await fetchSoulMeta(params.slug)
return {
owner: data?.owner ?? null,
displayName: data?.displayName ?? null,
summary: data?.summary ?? null,
version: data?.version ?? null,
}
},
head: ({ params, loaderData }) => {
const meta = buildSoulMeta({
slug: params.slug,
owner: loaderData?.owner ?? null,
displayName: loaderData?.displayName,
summary: loaderData?.summary,
version: loaderData?.version ?? null,
})
return {
links: [
{
rel: 'canonical',
href: meta.url,
},
],
meta: [
{ title: meta.title },
{ name: 'description', content: meta.description },
{ property: 'og:title', content: meta.title },
{ property: 'og:description', content: meta.description },
{ property: 'og:type', content: 'website' },
{ property: 'og:url', content: meta.url },
{ property: 'og:image', content: meta.image },
{ property: 'og:image:width', content: '1200' },
{ property: 'og:image:height', content: '630' },
{ property: 'og:image:alt', content: meta.title },
{ name: 'twitter:card', content: 'summary_large_image' },
{ name: 'twitter:title', content: meta.title },
{ name: 'twitter:description', content: meta.description },
{ name: 'twitter:image', content: meta.image },
{ name: 'twitter:image:alt', content: meta.title },
],
}
},
component: SoulDetail,
})
function SoulDetail() {
const { slug } = Route.useParams()
return <SoulDetailPage slug={slug} />
}

231
src/routes/souls/index.tsx Normal file
View File

@ -0,0 +1,231 @@
import { createFileRoute, Link } from '@tanstack/react-router'
import { useAction, useQuery } from 'convex/react'
import { useEffect, useMemo, useRef, useState } from 'react'
import { api } from '../../../convex/_generated/api'
import type { Doc } from '../../../convex/_generated/dataModel'
import { SoulCard } from '../../components/SoulCard'
const sortKeys = ['newest', 'downloads', 'stars', 'name', 'updated'] as const
type SortKey = (typeof sortKeys)[number]
type SortDir = 'asc' | 'desc'
function parseSort(value: unknown): SortKey {
if (typeof value !== 'string') return 'newest'
if ((sortKeys as readonly string[]).includes(value)) return value as SortKey
return 'newest'
}
function parseDir(value: unknown, sort: SortKey): SortDir {
if (value === 'asc' || value === 'desc') return value
return sort === 'name' ? 'asc' : 'desc'
}
export const Route = createFileRoute('/souls/')({
validateSearch: (search) => {
return {
q: typeof search.q === 'string' && search.q.trim() ? search.q : undefined,
sort: typeof search.sort === 'string' ? parseSort(search.sort) : undefined,
dir: search.dir === 'asc' || search.dir === 'desc' ? search.dir : undefined,
view: search.view === 'cards' || search.view === 'list' ? search.view : undefined,
}
},
component: SoulsIndex,
})
function SoulsIndex() {
const navigate = Route.useNavigate()
const search = Route.useSearch()
const sort = search.sort ?? 'newest'
const dir = parseDir(search.dir, sort)
const view = search.view ?? 'list'
const [query, setQuery] = useState(search.q ?? '')
const souls = useQuery(api.souls.list, { limit: 500 }) as Doc<'souls'>[] | undefined
const ensureSoulSeeds = useAction(api.seed.ensureSoulSeeds)
const seedEnsuredRef = useRef(false)
const isLoadingSouls = souls === undefined
useEffect(() => {
setQuery(search.q ?? '')
}, [search.q])
useEffect(() => {
if (seedEnsuredRef.current) return
seedEnsuredRef.current = true
void ensureSoulSeeds({})
}, [ensureSoulSeeds])
const filtered = useMemo(() => {
const value = query.trim().toLowerCase()
const all = souls ?? []
if (!value) return all
return all.filter((soul) => {
if (soul.slug.toLowerCase().includes(value)) return true
if (soul.displayName.toLowerCase().includes(value)) return true
return (soul.summary ?? '').toLowerCase().includes(value)
})
}, [query, souls])
const sorted = useMemo(() => {
const multiplier = dir === 'asc' ? 1 : -1
const results = [...filtered]
results.sort((a, b) => {
switch (sort) {
case 'downloads':
return (a.stats.downloads - b.stats.downloads) * multiplier
case 'stars':
return (a.stats.stars - b.stats.stars) * multiplier
case 'updated':
return (a.updatedAt - b.updatedAt) * multiplier
case 'name':
return (
(a.displayName.localeCompare(b.displayName) || a.slug.localeCompare(b.slug)) *
multiplier
)
default:
return (a.createdAt - b.createdAt) * multiplier
}
})
return results
}, [dir, filtered, sort])
const showing = sorted.length
const total = souls?.length
return (
<main className="section">
<header className="skills-header">
<div>
<h1 className="section-title" style={{ marginBottom: 8 }}>
Souls
</h1>
<p className="section-subtitle" style={{ marginBottom: 0 }}>
{isLoadingSouls
? 'Loading souls…'
: `${showing}${typeof total === 'number' ? ` of ${total}` : ''} souls.`}
</p>
</div>
<div className="skills-toolbar">
<div className="skills-search">
<input
className="skills-search-input"
value={query}
onChange={(event) => {
const next = event.target.value
const trimmed = next.trim()
setQuery(next)
void navigate({
search: (prev) => ({ ...prev, q: trimmed ? next : undefined }),
replace: true,
})
}}
placeholder="Filter by name, slug, or summary…"
/>
</div>
<div className="skills-toolbar-row">
<select
className="skills-sort"
value={sort}
onChange={(event) => {
const sort = parseSort(event.target.value)
void navigate({
search: (prev) => ({
...prev,
sort,
dir: parseDir(prev.dir, sort),
}),
replace: true,
})
}}
aria-label="Sort souls"
>
<option value="newest">Newest</option>
<option value="updated">Recently updated</option>
<option value="downloads">Downloads</option>
<option value="stars">Stars</option>
<option value="name">Name</option>
</select>
<button
className="skills-dir"
type="button"
aria-label={`Sort direction ${dir}`}
onClick={() => {
void navigate({
search: (prev) => ({
...prev,
dir: parseDir(prev.dir, sort) === 'asc' ? 'desc' : 'asc',
}),
replace: true,
})
}}
>
{dir === 'asc' ? '↑' : '↓'}
</button>
<button
className={`skills-view${view === 'cards' ? ' is-active' : ''}`}
type="button"
onClick={() => {
void navigate({
search: (prev) => ({
...prev,
view: prev.view === 'cards' ? undefined : 'cards',
}),
replace: true,
})
}}
>
{view === 'cards' ? 'List' : 'Cards'}
</button>
</div>
</div>
</header>
{isLoadingSouls ? (
<div className="card">
<div className="loading-indicator">Loading souls</div>
</div>
) : showing === 0 ? (
<div className="card">No souls match that filter.</div>
) : view === 'cards' ? (
<div className="grid">
{sorted.map((soul) => (
<SoulCard
key={soul._id}
soul={soul}
summaryFallback="A SOUL.md bundle."
meta={
<div className="stat">
{soul.stats.stars} · {soul.stats.downloads} · {soul.stats.versions} v
</div>
}
/>
))}
</div>
) : (
<div className="skills-list">
{sorted.map((soul) => (
<Link
key={soul._id}
className="skills-row"
to="/souls/$slug"
params={{ slug: soul.slug }}
>
<div className="skills-row-main">
<div className="skills-row-title">
<span>{soul.displayName}</span>
<span className="skills-row-slug">/{soul.slug}</span>
</div>
<div className="skills-row-summary">{soul.summary ?? 'SOUL.md bundle.'}</div>
</div>
<div className="skills-row-metrics">
<span> {soul.stats.downloads}</span>
<span> {soul.stats.stars}</span>
<span>{soul.stats.versions} v</span>
</div>
</Link>
))}
</div>
)}
</main>
)
}

View File

@ -3,6 +3,7 @@ import { useAction, useConvexAuth, useMutation, useQuery } from 'convex/react'
import { useEffect, useMemo, useRef, useState } from 'react'
import semver from 'semver'
import { api } from '../../convex/_generated/api'
import { getSiteMode } from '../lib/site'
import { expandFiles } from '../lib/uploadFiles'
import {
formatBytes,
@ -25,10 +26,35 @@ export const Route = createFileRoute('/upload')({
export function Upload() {
const { isAuthenticated } = useConvexAuth()
const { updateSlug } = useSearch({ from: '/upload' })
const siteMode = getSiteMode()
const isSoulMode = siteMode === 'souls'
const requiredFileLabel = isSoulMode ? 'SOUL.md' : 'SKILL.md'
const contentLabel = isSoulMode ? 'soul' : 'skill'
const generateUploadUrl = useMutation(api.uploads.generateUploadUrl)
const publishVersion = useAction(api.skills.publishVersion)
const generateChangelogPreview = useAction(api.skills.generateChangelogPreview)
const existingSkill = useQuery(api.skills.getBySlug, updateSlug ? { slug: updateSlug } : 'skip')
const publishVersion = useAction(
isSoulMode ? api.souls.publishVersion : api.skills.publishVersion,
)
const generateChangelogPreview = useAction(
isSoulMode ? api.souls.generateChangelogPreview : api.skills.generateChangelogPreview,
)
const existingSkill = useQuery(
api.skills.getBySlug,
!isSoulMode && updateSlug ? { slug: updateSlug } : 'skip',
)
const existingSoul = useQuery(
api.souls.getBySlug,
isSoulMode && updateSlug ? { slug: updateSlug } : 'skip',
)
const existing = (isSoulMode ? existingSoul : existingSkill) as
| {
skill?: { slug: string; displayName: string }
soul?: { slug: string; displayName: string }
latestVersion?: { version: string }
}
| null
| undefined
const [hasAttempted, setHasAttempted] = useState(false)
const [files, setFiles] = useState<File[]>([])
const [slug, setSlug] = useState(updateSlug ?? '')
@ -44,6 +70,7 @@ export function Upload() {
const changelogRequestRef = useRef(0)
const changelogKeyRef = useRef<string | null>(null)
const [status, setStatus] = useState<string | null>(null)
const isSubmitting = status !== null
const [error, setError] = useState<string | null>(null)
const [isDragging, setIsDragging] = useState(false)
const fileInputRef = useRef<HTMLInputElement | null>(null)
@ -71,13 +98,13 @@ export function Upload() {
}),
[files, stripRoot],
)
const hasSkillFile = useMemo(
const hasRequiredFile = useMemo(
() =>
normalizedPaths.some((path) => {
const lower = path.trim().toLowerCase()
return lower === 'skill.md' || lower === 'skills.md'
return isSoulMode ? lower === 'soul.md' : lower === 'skill.md' || lower === 'skills.md'
}),
[normalizedPaths],
[isSoulMode, normalizedPaths],
)
const sizeLabel = totalBytes ? formatBytes(totalBytes) : '0 B'
const trimmedSlug = slug.trim()
@ -85,38 +112,40 @@ export function Upload() {
const trimmedChangelog = changelog.trim()
useEffect(() => {
if (!existingSkill?.skill || !existingSkill?.latestVersion) return
setSlug(existingSkill.skill.slug)
setDisplayName(existingSkill.skill.displayName)
const nextVersion = semver.inc(existingSkill.latestVersion.version, 'patch')
if (!existing?.latestVersion || (!existing?.skill && !existing?.soul)) return
const name = existing.skill?.displayName ?? existing.soul?.displayName
const nextSlug = existing.skill?.slug ?? existing.soul?.slug
if (nextSlug) setSlug(nextSlug)
if (name) setDisplayName(name)
const nextVersion = semver.inc(existing.latestVersion.version, 'patch')
if (nextVersion) setVersion(nextVersion)
}, [existingSkill])
}, [existing])
useEffect(() => {
if (changelogTouchedRef.current) return
if (trimmedChangelog) return
if (!trimmedSlug || !SLUG_PATTERN.test(trimmedSlug)) return
if (!semver.valid(version)) return
if (!hasSkillFile) return
if (!hasRequiredFile) return
if (files.length === 0) return
const skillIndex = normalizedPaths.findIndex((path) => {
const requiredIndex = normalizedPaths.findIndex((path) => {
const lower = path.trim().toLowerCase()
return lower === 'skill.md' || lower === 'skills.md'
return isSoulMode ? lower === 'soul.md' : lower === 'skill.md' || lower === 'skills.md'
})
if (skillIndex < 0) return
if (requiredIndex < 0) return
const skillFile = files[skillIndex]
if (!skillFile) return
const requiredFile = files[requiredIndex]
if (!requiredFile) return
const key = `${trimmedSlug}:${version}:${skillFile.size}:${skillFile.lastModified}:${normalizedPaths.length}`
const key = `${trimmedSlug}:${version}:${requiredFile.size}:${requiredFile.lastModified}:${normalizedPaths.length}`
if (changelogKeyRef.current === key) return
changelogKeyRef.current = key
const requestId = ++changelogRequestRef.current
setChangelogStatus('loading')
void readText(skillFile)
void readText(requiredFile)
.then((text) => {
if (changelogRequestRef.current !== requestId) return null
return generateChangelogPreview({
@ -140,7 +169,8 @@ export function Upload() {
}, [
files,
generateChangelogPreview,
hasSkillFile,
hasRequiredFile,
isSoulMode,
normalizedPaths,
trimmedChangelog,
trimmedSlug,
@ -173,8 +203,8 @@ export function Upload() {
if (files.length === 0) {
issues.push('Add at least one file.')
}
if (!hasSkillFile) {
issues.push('SKILL.md is required.')
if (!hasRequiredFile) {
issues.push(`${requiredFileLabel} is required.`)
}
const invalidFiles = files.filter((file) => !isTextFile(file))
if (invalidFiles.length > 0) {
@ -192,7 +222,16 @@ export function Upload() {
issues,
ready: issues.length === 0,
}
}, [trimmedSlug, trimmedName, version, parsedTags.length, files, hasSkillFile, totalBytes])
}, [
trimmedSlug,
trimmedName,
version,
parsedTags.length,
files,
hasRequiredFile,
totalBytes,
requiredFileLabel,
])
useEffect(() => {
if (!fileInputRef.current) return
@ -203,7 +242,7 @@ export function Upload() {
if (!isAuthenticated) {
return (
<main className="section">
<div className="card">Sign in to upload a skill.</div>
<div className="card">Sign in to upload a {contentLabel}.</div>
</main>
)
}
@ -222,8 +261,8 @@ export function Upload() {
setError('Total size exceeds 50MB per version.')
return
}
if (!hasSkillFile) {
setError('SKILL.md is required.')
if (!hasRequiredFile) {
setError(`${requiredFileLabel} is required.`)
return
}
setStatus('Uploading files…')
@ -238,13 +277,13 @@ export function Upload() {
for (const file of files) {
const uploadUrl = await generateUploadUrl()
const storageId = await uploadFile(uploadUrl, file)
const sha256 = await hashFile(file)
const rawPath = (file.webkitRelativePath || file.name).replace(/^\.\//, '')
const path =
stripRoot && rawPath.startsWith(`${stripRoot}/`)
? rawPath.slice(stripRoot.length + 1)
: rawPath
const sha256 = await hashFile(file)
const storageId = await uploadFile(uploadUrl, file)
uploaded.push({
path,
size: file.size,
@ -254,9 +293,9 @@ export function Upload() {
})
}
setStatus('Publishing version…')
setStatus('Publishing…')
try {
await publishVersion({
const result = await publishVersion({
slug: trimmedSlug,
displayName: trimmedName,
version,
@ -264,240 +303,184 @@ export function Upload() {
tags: parsedTags,
files: uploaded,
})
setStatus('Published.')
void navigate({ to: '/skills/$slug', params: { slug: trimmedSlug } })
} catch (publishError) {
const message = formatPublishError(publishError)
setError(message)
setStatus(null)
if (validationRef.current && 'scrollIntoView' in validationRef.current) {
validationRef.current.scrollIntoView({ behavior: 'smooth', block: 'center' })
setError(null)
setHasAttempted(false)
setChangelogSource('user')
if (result) {
void navigate({
to: isSoulMode ? '/souls/$slug' : '/skills/$slug',
params: { slug: trimmedSlug },
})
}
}
}
async function handleFilesSelected(selected: File[]) {
if (selected.length === 0) return
setError(null)
setStatus('Preparing files…')
let expanded: File[] = []
try {
expanded = await expandFiles(selected)
} catch (error) {
setStatus(null)
} catch (expandError) {
const message =
expandError instanceof Error ? expandError.message : 'Could not extract files.'
setError(message)
setStatus(null)
return
setError(formatPublishError(error))
}
const next = new Map<string, File>()
for (const file of files) {
const key = `${file.webkitRelativePath || file.name}:${file.size}`
next.set(key, file)
}
for (const file of expanded) {
const key = `${file.webkitRelativePath || file.name}:${file.size}`
next.set(key, file)
}
setFiles(Array.from(next.values()))
}
function handleRemoveFile(target: File) {
setFiles((current) =>
current.filter(
(file) =>
`${file.webkitRelativePath || file.name}:${file.size}` !==
`${target.webkitRelativePath || target.name}:${target.size}`,
),
)
}
function handleDrop(event: React.DragEvent<HTMLButtonElement>) {
event.preventDefault()
setIsDragging(false)
void handleFilesSelected(Array.from(event.dataTransfer.files ?? []))
}
function handleDragOver(event: React.DragEvent<HTMLButtonElement>) {
event.preventDefault()
setIsDragging(true)
}
function handleDragLeave() {
setIsDragging(false)
}
return (
<main className="section upload-shell">
<header className="upload-header">
<div>
<span className="upload-kicker">Publish</span>
<h1 className="upload-title">Publish a skill</h1>
<p className="upload-subtitle">
Bundle SKILL.md + text files. Tag it, version it, ship it.
</p>
<main className="section">
<h1 className="section-title">Publish a {contentLabel}</h1>
<p className="section-subtitle">
Drop a folder with {requiredFileLabel} and text files. We will handle the rest.
</p>
<form onSubmit={handleSubmit} className="upload-grid">
<div className="card">
<label className="form-label" htmlFor="slug">
Slug
</label>
<input
className="form-input"
id="slug"
value={slug}
onChange={(event) => setSlug(event.target.value)}
placeholder={`${contentLabel}-name`}
/>
<label className="form-label" htmlFor="displayName">
Display name
</label>
<input
className="form-input"
id="displayName"
value={displayName}
onChange={(event) => setDisplayName(event.target.value)}
placeholder={`My ${contentLabel}`}
/>
<label className="form-label" htmlFor="version">
Version
</label>
<input
className="form-input"
id="version"
value={version}
onChange={(event) => setVersion(event.target.value)}
placeholder="1.0.0"
/>
<label className="form-label" htmlFor="tags">
Tags
</label>
<input
className="form-input"
id="tags"
value={tags}
onChange={(event) => setTags(event.target.value)}
placeholder="latest, stable"
/>
</div>
</header>
<form className="upload-card" onSubmit={handleSubmit}>
<div className="upload-grid">
<div className="upload-fields">
<label className="upload-field">
<span>Slug</span>
<input
className="search-input upload-input"
value={slug}
onChange={(event) => setSlug(event.target.value)}
placeholder="my-skill-pack"
/>
</label>
<label className="upload-field">
<span>Display name</span>
<input
className="search-input upload-input"
value={displayName}
onChange={(event) => setDisplayName(event.target.value)}
placeholder="My Skill Pack"
/>
</label>
<div className="upload-row">
<label className="upload-field">
<span>Version</span>
<input
className="search-input upload-input"
value={version}
onChange={(event) => setVersion(event.target.value)}
placeholder="1.0.0"
/>
</label>
<label className="upload-field">
<span>Tags</span>
<input
className="search-input upload-input"
value={tags}
onChange={(event) => setTags(event.target.value)}
placeholder="latest, beta"
/>
</label>
</div>
<label className="upload-field">
<div className="upload-field-header">
<span>Changelog</span>
{changelogSource === 'auto' ? (
<span className="upload-field-hint">
{changelogStatus === 'loading' ? 'Auto-generating…' : 'Auto-generated'}
</span>
) : changelogStatus === 'error' ? (
<span className="upload-field-hint">Auto-generation failed</span>
) : changelogStatus === 'loading' ? (
<span className="upload-field-hint">Auto-generating</span>
) : null}
</div>
<textarea
className="search-input upload-input"
rows={4}
value={changelog}
onChange={(event) => {
changelogTouchedRef.current = true
changelogRequestRef.current += 1
setChangelogSource('user')
setChangelogStatus('idle')
setChangelog(event.target.value)
}}
placeholder="What changed in this version?"
/>
</label>
</div>
<div className="upload-side">
<div className={`dropzone${isDragging ? ' is-dragging' : ''}`}>
<button
className="dropzone-button"
type="button"
onDrop={handleDrop}
onDragOver={handleDragOver}
onDragLeave={handleDragLeave}
onClick={() => fileInputRef.current?.click()}
>
<div className="dropzone-icon"></div>
<div>
<strong>Drop a folder, files, or zip</strong>
<p>Click to choose a folder. Archives auto-extract.</p>
</div>
<div className="card">
<label
className={`upload-dropzone${isDragging ? ' is-dragging' : ''}`}
onDragOver={(event) => {
event.preventDefault()
setIsDragging(true)
}}
onDragLeave={() => setIsDragging(false)}
onDrop={(event) => {
event.preventDefault()
setIsDragging(false)
const dropped = Array.from(event.dataTransfer.files)
void expandFiles(dropped).then((next) => setFiles(next))
}}
>
<input
ref={fileInputRef}
className="upload-input"
id="upload-files"
data-testid="upload-input"
type="file"
multiple
onChange={(event) => {
const picked = Array.from(event.target.files ?? [])
void expandFiles(picked).then((next) => setFiles(next))
}}
/>
<div className="upload-dropzone-copy">
<strong>Drop a folder</strong>
<span>
{files.length} files · {sizeLabel}
</span>
<button className="btn" type="button" onClick={() => fileInputRef.current?.click()}>
Choose folder
</button>
<input
ref={fileInputRef}
className="dropzone-input"
type="file"
multiple
data-testid="upload-input"
onChange={(event) => void handleFilesSelected(Array.from(event.target.files ?? []))}
/>
</div>
<div className="upload-summary">
<div>
<strong>{files.length}</strong> files · <span>{sizeLabel}</span>
</div>
<div className={`upload-requirement${hasSkillFile ? ' ok' : ''}`}>
SKILL.md {hasSkillFile ? 'found' : 'required'}
</div>
{files.length ? (
<div className="upload-filelist">
{files.map((file, index) => (
<div
key={`${file.webkitRelativePath || file.name}:${file.size}`}
className="upload-file"
>
<span>
{normalizedPaths[index] ?? (file.webkitRelativePath || file.name)}
</span>
<span>{formatBytes(file.size)}</span>
<button
className="upload-remove"
type="button"
onClick={() => handleRemoveFile(file)}
>
Remove
</button>
</div>
))}
</label>
<div className="upload-file-list">
{files.length === 0 ? (
<div className="stat">No files selected.</div>
) : (
normalizedPaths.map((path) => (
<div key={path} className="upload-file-row">
<span>{path}</span>
</div>
) : (
<p className="upload-muted">No files selected yet.</p>
)}
{files.length ? (
<button className="btn" type="button" onClick={() => setFiles([])}>
Clear selection
</button>
) : null}
</div>
<div className="upload-notes">
<strong>Checks</strong>
<ul>
<li>Include SKILL.md</li>
<li>50 MB max per version</li>
<li>Changelog optional</li>
<li>Valid semver version</li>
</ul>
</div>
))
)}
</div>
</div>
<div className="upload-footer" ref={validationRef}>
<button className="btn btn-primary" type="submit" disabled={Boolean(status)}>
Publish
<div className="card" ref={validationRef}>
<h2 className="section-title" style={{ fontSize: '1.2rem', margin: 0 }}>
Validation
</h2>
{validation.issues.length === 0 ? (
<div className="stat">All checks passed.</div>
) : (
<ul className="validation-list">
{validation.issues.map((issue) => (
<li key={issue}>{issue}</li>
))}
</ul>
)}
</div>
<div className="card">
<label className="form-label" htmlFor="changelog">
Changelog
</label>
<textarea
className="form-input"
id="changelog"
rows={6}
value={changelog}
onChange={(event) => {
changelogTouchedRef.current = true
setChangelogSource('user')
setChangelog(event.target.value)
}}
placeholder={`Describe what changed in this ${contentLabel}...`}
/>
{changelogStatus === 'loading' ? <div className="stat">Generating changelog</div> : null}
{changelogStatus === 'error' ? (
<div className="stat">Could not auto-generate changelog.</div>
) : null}
{changelogSource === 'auto' && changelog ? (
<div className="stat">Auto-generated changelog (edit as needed).</div>
) : null}
</div>
<div className="card">
{error ? (
<div className="error" role="alert">
{error}
</div>
) : null}
{status ? <div className="stat">{status}</div> : null}
<button
className="btn btn-primary"
type="submit"
disabled={!validation.ready || isSubmitting}
>
Publish {contentLabel}
</button>
{hasAttempted && !validation.ready ? (
<div className="upload-validation">
{validation.issues.map((issue) => (
<div key={issue} className="upload-validation-item">
{issue}
</div>
))}
</div>
) : error ? null : validation.ready ? (
<div className="upload-ready">Ready to publish.</div>
<div className="stat">Fix validation issues to continue.</div>
) : null}
{error ? <div className="stat upload-error">{error}</div> : null}
{status ? <div className="stat">{status}</div> : null}
</div>
</form>
</main>