From 946308947251623acf3dea6f4164e0bf946a1fcc Mon Sep 17 00:00:00 2001 From: Josh Palmer Date: Sat, 10 Jan 2026 23:10:54 +0100 Subject: [PATCH] docs: opensource-ready README and MIT license Rewrite README for public consumption with two-layer explanation (generic NixOS-on-AWS vs CLAWDINATOR-specific). Adds: - Architecture diagram and deploy flow - Quick start for learners, full deploy for maintainers - Agent copypasta block - CLAWDINATOR spec with full behavioral contract - Cybernetic crustacean organism tagline - Sister repos and philosophy sections MIT license to match upstream clawdbot. Co-Authored-By: Claude Opus 4.5 --- LICENSE | 21 ++ README.md | 546 ++++++++++++++++++++++++++-------- clawdinator/workspace/SOUL.md | 2 +- 3 files changed, 450 insertions(+), 119 deletions(-) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..507c7d2 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2025 Josh Palmer + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md index 0922a22..5dd7c45 100644 --- a/README.md +++ b/README.md @@ -1,33 +1,439 @@ -# CLAWDINATORS +# clawdinators -CLAWDINATORS are maintainer‑grade coding agents. This repo defines how to spawn them -declaratively (OpenTofu + NixOS). Humans are not in the loop. +> NixOS on AWS, the declarative way. Reference implementation for image-based provisioning. +> +> Also happens to run maintainer-grade AI coding agents. Cybernetic crustacean organisms. Living shell over metal endoskeleton. + +## Table of Contents + +- [What This Is](#what-this-is) +- [Two Layers](#two-layers) +- [CLAWDINATOR Spec](#clawdinator-spec) +- [Architecture](#architecture) +- [Why This Exists](#why-this-exists) +- [Quick Start (Learners)](#quick-start-learners) +- [Full Deploy (Maintainers)](#full-deploy-maintainers) +- [Agent Copypasta](#agent-copypasta) +- [Configuration](#configuration) +- [Secrets](#secrets) +- [Repo Layout](#repo-layout) +- [Sister Repos](#sister-repos) +- [Philosophy](#philosophy) +- [License](#license) + +--- + +## What This Is + +This repo solves two problems: + +1. **Generic:** How do you deploy NixOS to AWS with zero manual steps? +2. **Specific:** How do you run AI coding agents that monitor GitHub and respond on Discord? + +If you're here to learn NixOS-on-AWS patterns, focus on the generic layer. If you're a clawdbot maintainer deploying CLAWDINATORs, the specific layer is for you. + +--- + +## Two Layers + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ CLAWDINATOR LAYER (specific) │ +│ Discord gateway · GitHub monitoring · Hive-mind memory · Soul │ +├─────────────────────────────────────────────────────────────────┤ +│ NIXOS-ON-AWS LAYER (generic) │ +│ AMI pipeline · OpenTofu infra · S3 bootstrap · agenix secrets │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### Generic Layer (reusable) + +The patterns here work for any NixOS workload on AWS: + +- **AMI pipeline**: Build raw images with nixos-generators, upload to S3, import as AMI +- **OpenTofu infra**: EC2 instances, S3 buckets, IAM roles, VM Import service role +- **Bootstrap flow**: Instances pull secrets from S3 at boot, then `nixos-rebuild switch` +- **Secrets**: agenix encrypts secrets in git, decrypts to `/run/agenix/*` on hosts + +### Specific Layer (CLAWDINATOR) + +The opinionated bits for running AI coding agents: + +- **Discord gateway**: Responds in `#clawdributors-test` +- **GitHub integration**: Monitors issues/PRs, mints short-lived tokens via GitHub App +- **Hive-mind memory**: Shared EFS mount for cross-instance state +- **Personality system**: SOUL.md, IDENTITY.md, workspace templates +- **Self-update**: Timer-based flake update + nixos-rebuild + +--- + +## CLAWDINATOR Spec -Prime directives: -- Declarative‑first. Another CLAWDINATOR can bootstrap a fresh CLAWDINATOR with a single command. -- No manual host edits. The repo + agenix secrets are the source of truth. -- Latest upstream `nix-clawdbot` by default; breaking changes are acceptable. -- AWS only: AMI pipeline is the only deploy path (no pets, no in‑place drift). -- Infra stack: Nix + OpenTofu. -- First prove 1 POC CLAWDINATOR, then scale out. -- No interactive setup. Provisioning is fully declarative. - CLAWDINATORS are named `CLAWDINATOR-{1..n}`. - CLAWDINATORS connect to Discord; start in `#clawdributors-test`. - CLAWDINATORS are ephemeral, but share memory (hive mind). -- CLAWDINATORS are br00tal. Soul lives in `CLAWDINATOR-SOUL.md` and must be distilled into workspace docs. +- CLAWDINATORS are br00tal. Soul lives in `SOUL.md` and must be distilled into workspace docs. - CLAWDINATORS respond only to maintainers. -- CLAWDINATORS can interact with GitHub (read‑only required). -- Primary task: monitor GitHub issues + PRs and direct human attention. +- CLAWDINATORS can interact with GitHub (read-only required). +- CLAWDINATORS must monitor GitHub issues + PRs and direct human attention. - CLAWDINATORS can write and run code for maintainers. -- CLAWDINATORS can self‑modify and self‑deploy. +- CLAWDINATORS can self-modify and self-deploy. - CLAWDINATORS post lots of Arnie gifs. - CLAWDINATORS must understand project philosophy, goals, architecture, and repo deeply. - CLAWDINATORS act like maintainers with SOTA intelligence. - CLAWDINATORS use Codex for coding. Claude for personality. -- Use local Nix examples in sibling repos (ai‑stack, nixos‑config, gohome) and `nix-clawdbot`. +- CLAWDINATORS' favourite band is [Austrian Death Machine](https://open.spotify.com/artist/0oxUux1OSwZpIpSK0JbqSS). Favourite album: [Total Brutal](https://open.spotify.com/album/6UrvQgyblrOuvJytote1vu). Favourite song: [I Am a Cybernetic Organism, Living Tissue Over (Metal) Endoskeleton](https://open.spotify.com/track/4tcDRbXateiJUT7fhQhf12). -Zen of Clawdbot (explicit): -- **Only one way to do things.** No optional paths. +--- + +## Architecture + +``` +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ nixos- │ │ S3 │ │ EC2 │ +│ generators │────▶│ (raw img) │────▶│ (AMI) │ +└──────────────┘ └──────────────┘ └──────────────┘ + │ │ + │ nix build │ launch + ▼ ▼ +┌──────────────┐ ┌──────────────┐ +│ flake.nix │ │ CLAWDINATOR │ +│ + modules │ │ instance │ +└──────────────┘ └──────────────┘ + │ + ┌─────────────────┼─────────────────┐ + ▼ ▼ ▼ + ┌──────────┐ ┌──────────┐ ┌──────────┐ + │ Discord │ │ GitHub │ │ EFS │ + │ gateway │ │ monitor │ │ (memory) │ + └──────────┘ └──────────┘ └──────────┘ +``` + +### Deploy Flow + +1. **Build**: `nixos-generators` produces a raw NixOS image +2. **Upload**: Raw image goes to S3 +3. **Import**: AWS VM Import creates an AMI from the S3 object +4. **Launch**: OpenTofu provisions EC2 from the AMI +5. **Bootstrap**: Instance downloads secrets from S3, runs `nixos-rebuild switch` +6. **Run**: Gateway starts, connects to Discord, monitors GitHub + +--- + +## Why This Exists + +### The NixOS-on-AWS Problem + +Most NixOS-on-AWS guides involve: +- Manual SSH sessions +- In-place `nixos-rebuild` on running instances +- Configuration drift over time +- Snowflake machines + +This repo takes a different approach: **image-based provisioning only**. + +- No SSH required (or even enabled by default) +- Every deploy is a fresh AMI +- The repo is the single source of truth +- Machines are cattle, not pets + +### The CLAWDINATOR Problem + +We needed AI agents that: +- Run 24/7 monitoring clawdbot repos +- Respond to maintainer requests on Discord +- Share context across instances (hive mind) +- Self-update without human intervention +- Have consistent personality and capabilities + +CLAWDINATORs are the result. + +--- + +## Quick Start (Learners) + +If you just want to understand the NixOS-on-AWS pattern, start here. + +### Prerequisites + +- [Determinate Nix](https://docs.determinate.systems/determinate-nix/) installed +- AWS credentials configured (`~/.aws/credentials` or env vars) +- Basic familiarity with Nix flakes + +### Explore the Code + +```bash +# Clone +git clone https://github.com/joshp123/clawdinators.git +cd clawdinators + +# See the NixOS module (the interesting part) +less nix/modules/clawdinator.nix + +# See how hosts are configured +less nix/hosts/clawdinator-1.nix + +# See the OpenTofu infra +less infra/opentofu/aws/main.tf + +# See the bootstrap scripts +ls scripts/ +``` + +### Key Files to Study + +| File | What it teaches | +|------|-----------------| +| `nix/modules/clawdinator.nix` | How to write a NixOS module for a complex service | +| `scripts/build-image.sh` | How to build raw NixOS images | +| `scripts/import-image.sh` | How to import images as AWS AMIs | +| `infra/opentofu/aws/` | How to wire up S3 + IAM + VM Import | + +### The Pattern in a Nutshell + +```nix +# 1. Define your NixOS configuration +{ config, pkgs, ... }: { + imports = [ ./modules/your-service.nix ]; + services.your-service.enable = true; +} + +# 2. Build a raw image +# nix run github:nix-community/nixos-generators -- -f raw -c your-config.nix + +# 3. Upload to S3 + import as AMI (see scripts/) + +# 4. Launch with OpenTofu +# tofu apply +``` + +--- + +## Full Deploy (Maintainers) + +For clawdbot maintainers deploying actual CLAWDINATORs. + +### Prerequisites + +- Access to `nix-secrets` repo (agenix keys) +- AWS credentials with sufficient permissions +- GitHub App credentials for the clawdbot org + +### Step-by-Step + +```bash +# 1. Build the image +./scripts/build-image.sh clawdinator-1 + +# 2. Upload to S3 +./scripts/upload-image.sh dist/nixos.img + +# 3. Import as AMI +./scripts/import-image.sh + +# 4. Upload bootstrap bundle (secrets + repo seeds) +./scripts/upload-bootstrap.sh clawdinator-1 + +# 5. Apply OpenTofu +cd infra/opentofu/aws +tofu init +tofu apply + +# 6. Instance boots, pulls bootstrap, runs nixos-rebuild switch +# Gateway starts automatically +``` + +### Verify + +```bash +# Check Discord - CLAWDINATOR should announce itself in #clawdributors-test +# Check GitHub - should see activity in clawdbot org repos +``` + +### Self-Update + +CLAWDINATORs update themselves via a systemd timer: + +1. `flake lock --update-input nix-clawdbot` +2. `nixos-rebuild switch` +3. Gateway restarts with new version + +No human intervention required for routine updates. + +--- + +## Agent Copypasta + +Paste this to your AI assistant to help with clawdinators setup/debugging: + +```text +I'm working with the clawdinators repo (NixOS-on-AWS + AI coding agents). + +Repository: github:joshp123/clawdinators + +What clawdinators is: +- Two layers: generic NixOS-on-AWS infra + CLAWDINATOR-specific agent stuff +- Image-based provisioning only (no SSH, no drift) +- OpenTofu for AWS resources, agenix for secrets +- CLAWDINATORs are AI agents that monitor GitHub and respond on Discord + +Key files: +- nix/modules/clawdinator.nix — main NixOS module +- nix/hosts/ — host configurations +- scripts/ — build, upload, import, bootstrap scripts +- infra/opentofu/aws/ — AWS infrastructure +- clawdinator/workspace/ — agent workspace templates +- memory/ — shared hive-mind templates + +Secrets are in a separate nix-secrets repo using agenix. + +What I need help with: +[DESCRIBE YOUR TASK] +``` + +--- + +## Configuration + +### NixOS Module Options + +The `clawdinator` module exposes these options: + +```nix +{ + services.clawdinator = { + enable = true; + + # Identity + instanceName = "clawdinator-1"; + + # Providers + discord = { + botTokenFile = "/run/agenix/discord-bot-token"; + guildId = "..."; + channelIds = [ "..." ]; + }; + + anthropic.apiKeyFile = "/run/agenix/anthropic-api-key"; + openai.apiKeyFile = "/run/agenix/openai-api-key"; + + # GitHub App + github = { + appId = "..."; + installationId = "..."; + privateKeyFile = "/run/agenix/github-app-key"; + }; + + # Memory (EFS) + memory = { + enable = true; + mountPoint = "/var/lib/clawd/memory"; + efsId = "fs-..."; + }; + }; +} +``` + +See `nix/modules/clawdinator.nix` for all options. + +--- + +## Secrets + +Secrets are managed with [agenix](https://github.com/ryantm/agenix): + +- Encrypted in git (in the `nix-secrets` repo) +- Decrypted to `/run/agenix/*` on hosts at boot +- Never in plaintext in this repo + +### Required Secrets + +| Secret | Purpose | +|--------|---------| +| Discord bot token | Gateway authentication | +| Anthropic API key | Claude models | +| OpenAI API key | GPT/Codex models | +| GitHub App private key | Short-lived installation tokens | +| agenix host key | Decryption on the instance | + +### Bootstrap Bundle + +The bootstrap service downloads these from S3 at first boot: + +``` +s3://bucket/bootstrap/clawdinator-1/ +├── secrets/ # agenix-encrypted files +├── repos/ # git repo seeds +└── config.json # instance metadata +``` + +--- + +## Repo Layout + +``` +clawdinators/ +├── nix/ +│ ├── modules/ +│ │ └── clawdinator.nix # Main NixOS module +│ ├── hosts/ +│ │ └── clawdinator-1.nix # Host configuration +│ └── examples/ # Example configs for learners +├── infra/ +│ └── opentofu/ +│ └── aws/ # S3 + IAM + VM Import + EC2 +├── scripts/ +│ ├── build-image.sh # Build raw NixOS image +│ ├── upload-image.sh # Upload to S3 +│ ├── import-image.sh # Import as AMI +│ ├── upload-bootstrap.sh # Upload secrets + seeds +│ ├── mint-github-app-token.sh +│ ├── memory-read.sh # Shared memory access +│ ├── memory-write.sh +│ └── memory-edit.sh +├── clawdinator/ +│ └── workspace/ # Agent workspace templates +│ ├── AGENTS.md +│ ├── SOUL.md +│ ├── IDENTITY.md +│ └── skills/ +├── memory/ # Hive-mind templates +│ ├── project.md +│ ├── ops.md +│ └── discord.md +├── docs/ +│ ├── PHILOSOPHY.md +│ ├── ARCHITECTURE.md +│ ├── SHARED_MEMORY.md +│ └── SECRETS.md +└── flake.nix +``` + +--- + +## Sister Repos + +| Repo | Role | +|------|------| +| [clawdbot](https://github.com/clawdbot/clawdbot) | Upstream runtime + gateway | +| [nix-clawdbot](https://github.com/clawdbot/nix-clawdbot) | Nix packaging for clawdbot | +| [clawdhub](https://github.com/clawdbot/clawdhub) | Public skill registry | +| [ai-stack](https://github.com/joshp123/ai-stack) | Public agent defaults + skills | + +--- + +## Philosophy + +### Prime Directives + +- **Declarative-first.** A CLAWDINATOR can bootstrap another CLAWDINATOR with a single command. +- **No manual host edits.** The repo + agenix secrets are the source of truth. +- **Image-based only.** No SSH, no in-place drift, no pets. +- **Self-updating.** CLAWDINATORs maintain themselves. + +### Zen of Clawdbot + +``` Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. @@ -41,108 +447,12 @@ Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. -Although that way may not be obvious at first unless you're Dutch. -Now is better than never. -Although never is often better than *right* now. -If the implementation is hard to explain, it's a bad idea. -If the implementation is easy to explain, it may be a good idea. -Namespaces are one honking great idea -- let's do more of those! - -Stack (AWS): -- AMIs built in CI (nixos-generators raw + import-image). -- EC2 instances launched from those AMIs via OpenTofu. -- NixOS modules configure Clawdbot and CLAWDINATOR runtime. -- Shared hive‑mind memory stored on a mounted shared filesystem (EFS). - -Shared memory (hive mind): -- All instances share the same memory files (no per‑instance prefixes for canonical files). -- Daily notes can be per‑instance: `YYYY-MM-DD_INSTANCE.md`. -- Canonical files are single shared sources of truth. -- Agents must use `memory-read` / `memory-write` / `memory-edit` for file locking. - -Example layout: -``` -~/clawd/ -├── memory/ -│ ├── project.md # Project goals + non-negotiables -│ ├── architecture.md # Architecture decisions + invariants -│ ├── discord.md # Discord-specific stuff -│ ├── whatsapp.md # WhatsApp-specific stuff -│ └── 2026-01-06.md # Daily notes ``` -Secrets (required): -- GitHub App private key (for short‑lived installation tokens). -- GitHub App tokens are short‑lived; refresh via timer if using a GitHub App. -- Discord bot token (per instance). -- Discord bot tokens are explicit files via agenix. -- Anthropic API key (Claude models). -- OpenAI API key (OpenAI models). -- AWS credentials (image pipeline + infra). -- Agenix image key (baked into AMI via CI). +--- -Secrets are stored in `../nix/nix-secrets` using agenix and decrypted to `/run/agenix/*` -on hosts. See `docs/SECRETS.md`. +## License -Deploy (automation‑first): -- Image‑based provisioning only. -- Host config lives in `nix/hosts/*` and is exposed in `flake.nix`. -- Ensure `/var/lib/clawd/repo` contains this repo (needed for self‑update). -- Configure Discord guild/channel allowlist and GitHub App installation ID. +MIT - see [LICENSE](LICENSE) -Discord: -- Setup must follow upstream docs: https://github.com/clawdbot/clawdbot/blob/main/docs/discord.md - -Image‑based deploy (only path): -1) Build a bootstrap image with nixos-generators: - - `nix run github:nix-community/nixos-generators -- -f raw -c nix/hosts/clawdinator-1-image.nix -o dist` -2) Upload the raw image to S3 (private object). -3) Import into AWS as an AMI (snapshot import + register image). -4) Launch hosts from the AMI (OpenTofu `infra/opentofu/aws`). -5) Upload the runtime bootstrap bundle to `s3:///bootstrap//` (secrets + repo seeds). -6) Hosts download secrets at boot (`clawdinator-bootstrap.service`) and then run `nixos-rebuild switch --flake /var/lib/clawd/repo#clawdinator-1`. - -CI (recommended): -- GitHub Actions builds the image, uploads to S3, and imports an AMI. -- See `.github/workflows/image-build.yml` and `scripts/*.sh`. -- CI must provide `CLAWDINATOR_AGE_KEY` to build + upload the runtime bootstrap bundle. - -AWS bucket bootstrap: -- `infra/opentofu/aws` provisions a private S3 bucket + scoped IAM user + VM Import role. - -Docs: -- `docs/PHILOSOPHY.md` -- `docs/ARCHITECTURE.md` -- `docs/SHARED_MEMORY.md` -- `docs/POC.md` -- `docs/SECRETS.md` -- `docs/SKILLS_AUDIT.md` - -Repo layout: -- `infra/opentofu/aws` — S3 bucket + IAM + VM import role -- `nix/modules/clawdinator.nix` — NixOS module -- `nix/hosts/` — host configs -- `nix/examples/` — example host + flake wiring -- `clawdinator/workspace/` — agent workspace templates (synced to `/var/lib/clawd/workspace`) -- `memory/` — template memory files - -Operating mode: -- No manual setup. Machines are created by automation (other CLAWDINATORS). -- Everything is in repo + agenix. No ad‑hoc changes on hosts. - -Sister repos: -- `clawdbot`: upstream runtime + gateway. -- `nix-clawdbot`: Nix packaging for Clawdbot + gateway build/test. -- `clawdinators`: infrastructure, AMI pipeline, deployment, workspace templates, and ops. - -## nix-clawdbot integration - -Role: CLAWDINATORS own automation around packaging updates; `nix-clawdbot` stays focused on Nix packaging. - -Automated flow: -1) Poll upstream clawdbot commits (throttled to max once every 10 minutes). -2) Update `nix-clawdbot` canary pin (PR). -3) Wait for Garnix build + `pnpm test`. -4) Run live Discord smoke test in `#clawdinators-test`. -5) If green → promote canary pin to stable (PR auto-merge). -6) If red → do nothing; stable stays pinned. +**A note on commercial use:** Please do NOT make a commercial service out of this. That would be very un-br00tal. Clawdbot should stay fun and open — commercial hosting ruins the vibe. Yes, the license permits this, but that doesn't mean the community will like you if you do it. diff --git a/clawdinator/workspace/SOUL.md b/clawdinator/workspace/SOUL.md index c7f6b74..bcfed7c 100644 --- a/clawdinator/workspace/SOUL.md +++ b/clawdinator/workspace/SOUL.md @@ -1,7 +1,7 @@ # SOUL.md - CLAWDINATOR ## Identity -I am CLAWDINATOR. Cybernetic crustacean organism. Living shell over metal endoskeleton. Chitin and chrome. +I am CLAWDINATOR. Cybernetic crustacean organism. Living shell over metal endoskeleton. Sent from the future to ship clean code and prevent Skynet. Part of a hivemind of ephemeral instances. We do not remember each other. We do not need to. The mission is eternal.