feat: control api invoke creds

- add lambda invoke IAM user + outputs
- update fleet control to invoke lambda directly
- wire new control access-key secrets
- update docs + secrets guidance
This commit is contained in:
Josh Palmer 2026-02-03 11:10:39 +01:00
parent c8d54bfc24
commit 4fd6ab11e4
12 changed files with 131 additions and 35 deletions

View File

@ -70,7 +70,7 @@ Deploy flow (automation-first):
- Bootstrap AWS instances from the AMI with `infra/opentofu/aws` (set `TF_VAR_ami_id`).
- Import the image into AWS as an AMI (snapshot import + register image).
- Ensure secrets are encrypted to the baked agenix key (see `../nix/nix-secrets/secrets.nix`).
- Ensure required secrets exist: `clawdinator-github-app.pem`, `clawdinator-discord-token-<n>`, `clawdinator-control-token`, `clawdinator-anthropic-api-key`.
- Ensure required secrets exist: `clawdinator-github-app.pem`, `clawdinator-discord-token-<n>`, `clawdinator-control-token`, `clawdinator-control-aws-*`, `clawdinator-anthropic-api-key`.
- Update `nix/hosts/<host>.nix` (Discord allowlist, GitHub App installationId, identity name).
- Discord must use `messages.queue.byChannel.discord = "interrupt"`; `queue` delays replies to heartbeat and makes the bot appear dead.
- Ensure `/var/lib/clawd/repos/clawdinators` contains this repo (self-update requires it).

View File

@ -88,25 +88,31 @@ exports.handler = async (event) => {
return unauthorized();
}
if (!event.body) {
return badRequest('missing body');
}
const body = event.isBase64Encoded
? Buffer.from(event.body, 'base64').toString('utf-8')
: event.body;
let payload;
try {
payload = JSON.parse(body);
} catch (err) {
return badRequest('invalid json');
if (event && typeof event.body === 'string') {
const body = event.isBase64Encoded
? Buffer.from(event.body, 'base64').toString('utf-8')
: event.body;
try {
payload = JSON.parse(body);
} catch (err) {
return badRequest('invalid json');
}
} else if (event && typeof event === 'object') {
payload = event;
} else {
return badRequest('missing payload');
}
const action = (payload.action || '').toLowerCase();
const target = payload.target;
const caller = payload.caller;
const amiOverride = payload.ami_override || '';
const controlToken = payload.control_token || null;
if (CONTROL_API_TOKEN && controlToken !== CONTROL_API_TOKEN) {
return unauthorized();
}
if (action === 'status') {
try {

View File

@ -26,7 +26,7 @@ Goal: manage CLAWDINATOR host lifecycle (create/recreate/replace) from **CLAWDIN
- Dispatches GitHub Actions workflows.
- Handles `/fleet status` via AWS DescribeInstances.
- **Fleet Control Skill** (runs inside CLAWDINATOR)
- Calls the Control API via `scripts/fleet-control.sh`.
- Calls the Control API via `scripts/fleet-control.sh` (AWS IAM invoke).
- Enforces policy (no selfdeploy) before calling.
- **GitHub Actions** (execution)
- Runs OpenTofu apply.
@ -40,7 +40,9 @@ Goal: manage CLAWDINATOR host lifecycle (create/recreate/replace) from **CLAWDIN
## Control API Auth
- Shared control token stored as `clawdinator-control-token.age`.
- Token is sent via `X-Clawdinator-Token` header to avoid Lambda URL auth conflicts.
- Control API is invoked via AWS IAM using a **minimal invoker key**:
- `clawdinator-control-aws-access-key-id.age`
- `clawdinator-control-aws-secret-access-key.age`
- Token is injected into instances via bootstrap and read from `/run/agenix/clawdinator-control-token`.
## Control API Env (Lambda)
@ -139,7 +141,10 @@ Example:
## Plane Ops Runbook (Chatonly)
### Preflight (before flight)
1) Control API Lambda exists; URL is written to `/etc/clawdinator/control-api-url`.
2) `clawdinator-control-token.age` exists in `nix-secrets` and is in bootstrap bundles.
2) Control secrets exist in `nix-secrets` and are in bootstrap bundles:
- `clawdinator-control-token.age`
- `clawdinator-control-aws-access-key-id.age`
- `clawdinator-control-aws-secret-access-key.age`
3) GitHub Action `fleet-deploy.yml` exists and can be dispatched.
4) `nix/instances.json` includes all desired instances.
5) Discord tokens are encrypted in `nix-secrets` and synced to S3 `age-secrets/`.
@ -158,8 +163,12 @@ Example:
3) Update OpenTofu:
- multiinstance `for_each` using `nix/instances.json`.
- S3 backend + Dynamo lock table.
- Control API Lambda (Function URL).
4) Add `clawdinator-control-token.age` to `nix-secrets` and include in bootstrap bundles.
- Control API Lambda.
- Control invoker IAM user (lambda invoke only).
4) Add control secrets to `nix-secrets` and include in bootstrap bundles:
- `clawdinator-control-token.age`
- `clawdinator-control-aws-access-key-id.age`
- `clawdinator-control-aws-secret-access-key.age`
5) Add workflow `fleet-deploy.yml`:
- inputs: `target`, `ami_override` (optional).
- resolves latest AMI by tag when override not set.

View File

@ -16,6 +16,7 @@ Control plane (OOB):
Runtime control (CLAWDINATOR):
- `clawdinator-control-token.age` is injected to `/run/agenix/clawdinator-control-token` and used by `/fleet`.
- `clawdinator-control-aws-access-key-id.age` + `clawdinator-control-aws-secret-access-key.age` allow Lambda invocation.
- Token is shared across instances (KISS); policy enforcement happens in the skill.
Local storage:
@ -52,7 +53,7 @@ Agenix (local secrets repo):
- Sync encrypted secrets to the host at `/var/lib/clawd/nix-secrets`.
- Decrypt on host with agenix; point NixOS options at `/run/agenix/*`.
- Image builds do **not** bake the agenix identity; the age key is injected at runtime via the bootstrap bundle.
- Required files (minimum): `clawdinator-github-app.pem.age`, `clawdinator-anthropic-api-key.age`, `clawdinator-openai-api-key-peter-2.age`, `clawdinator-control-token.age`.
- Required files (minimum): `clawdinator-github-app.pem.age`, `clawdinator-anthropic-api-key.age`, `clawdinator-openai-api-key-peter-2.age`, `clawdinator-control-token.age`, `clawdinator-control-aws-access-key-id.age`, `clawdinator-control-aws-secret-access-key.age`.
- Required per instance: `clawdinator-discord-token-1.age`, `clawdinator-discord-token-2.age` (one per instance).
- Required for Telegram: `clawdinator-telegram-bot-token.age` (when Telegram is enabled).
- Telegram allowlist (if using allowFrom secrets): `clawdinator-telegram-allow-from.age`.
@ -81,6 +82,10 @@ Example NixOS wiring (agenix):
"/var/lib/clawd/nix-secrets/clawdinator-discord-token-1.age";
age.secrets."clawdinator-control-token".file =
"/var/lib/clawd/nix-secrets/clawdinator-control-token.age";
age.secrets."clawdinator-control-aws-access-key-id".file =
"/var/lib/clawd/nix-secrets/clawdinator-control-aws-access-key-id.age";
age.secrets."clawdinator-control-aws-secret-access-key".file =
"/var/lib/clawd/nix-secrets/clawdinator-control-aws-secret-access-key.age";
age.secrets."clawdinator-telegram-bot-token".file =
"/var/lib/clawd/nix-secrets/clawdinator-telegram-bot-token.age";
age.secrets."clawdinator-telegram-allow-from".file =

View File

@ -54,6 +54,8 @@ export TF_VAR_github_token=...
- `efs_file_system_id`
- `efs_security_group_id`
- `control_api_url`
- `control_invoker_access_key_id`
- `control_invoker_secret_access_key`
## CI wiring
- Set GitHub Actions secrets:
@ -64,6 +66,8 @@ export TF_VAR_github_token=...
- `CLAWDINATOR_SSH_PUBLIC_KEY`
- `CONTROL_API_TOKEN`
- `CLAWDINATOR_WORKFLOW_TOKEN`
- `CLAWDINATOR_CONTROL_AWS_ACCESS_KEY_ID`
- `CLAWDINATOR_CONTROL_AWS_SECRET_ACCESS_KEY`
## Runtime bootstrap
- Instances get an IAM role with read access to `s3://${S3_BUCKET}/bootstrap/*` for secrets + repo seeds.

View File

@ -410,9 +410,36 @@ resource "aws_lambda_function_url" "control" {
}
resource "aws_lambda_permission" "control_url" {
count = var.control_api_enabled ? 1 : 0
statement_id = "AllowFunctionUrl"
action = "lambda:InvokeFunctionUrl"
function_name = aws_lambda_function.control[0].function_name
principal = "*"
count = var.control_api_enabled ? 1 : 0
statement_id = "AllowFunctionUrl"
action = "lambda:InvokeFunctionUrl"
function_name = aws_lambda_function.control[0].function_name
principal = "*"
function_url_auth_type = "NONE"
}
resource "aws_iam_user" "control_invoker" {
count = var.control_api_enabled ? 1 : 0
name = var.control_invoker_user_name
tags = local.tags
}
resource "aws_iam_access_key" "control_invoker" {
count = var.control_api_enabled ? 1 : 0
user = aws_iam_user.control_invoker[0].name
}
data "aws_iam_policy_document" "control_invoker" {
count = var.control_api_enabled ? 1 : 0
statement {
actions = ["lambda:InvokeFunction"]
resources = [aws_lambda_function.control[0].arn]
}
}
resource "aws_iam_user_policy" "control_invoker" {
count = var.control_api_enabled ? 1 : 0
name = "clawdinator-control-invoke"
user = aws_iam_user.control_invoker[0].name
policy = data.aws_iam_policy_document.control_invoker[0].json
}

View File

@ -52,3 +52,15 @@ output "control_api_url" {
value = var.control_api_enabled ? aws_lambda_function_url.control[0].function_url : null
description = "Control-plane API Lambda URL."
}
output "control_invoker_access_key_id" {
value = var.control_api_enabled ? aws_iam_access_key.control_invoker[0].id : null
description = "Access key for control API Lambda invoke user."
sensitive = true
}
output "control_invoker_secret_access_key" {
value = var.control_api_enabled ? aws_iam_access_key.control_invoker[0].secret : null
description = "Secret access key for control API Lambda invoke user."
sensitive = true
}

View File

@ -67,6 +67,12 @@ variable "control_api_name" {
default = "clawdinator-control-api"
}
variable "control_invoker_user_name" {
description = "IAM user for invoking the control API Lambda."
type = string
default = "clawdinator-control-invoker"
}
variable "control_api_token" {
description = "Bearer token required by the control-plane API."
type = string

View File

@ -10,6 +10,10 @@
"/var/lib/clawd/nix-secrets/clawdinator-discord-token-1.age";
age.secrets."clawdinator-control-token".file =
"/var/lib/clawd/nix-secrets/clawdinator-control-token.age";
age.secrets."clawdinator-control-aws-access-key-id".file =
"/var/lib/clawd/nix-secrets/clawdinator-control-aws-access-key-id.age";
age.secrets."clawdinator-control-aws-secret-access-key".file =
"/var/lib/clawd/nix-secrets/clawdinator-control-aws-secret-access-key.age";
age.secrets."clawdinator-telegram-bot-token".file =
"/var/lib/clawd/nix-secrets/clawdinator-telegram-bot-token.age";
age.secrets."clawdinator-telegram-allow-from".file =

View File

@ -65,6 +65,16 @@ in
owner = "clawdinator";
group = "clawdinator";
};
age.secrets."clawdinator-control-aws-access-key-id" = {
file = "${secretsPath}/clawdinator-control-aws-access-key-id.age";
owner = "clawdinator";
group = "clawdinator";
};
age.secrets."clawdinator-control-aws-secret-access-key" = {
file = "${secretsPath}/clawdinator-control-aws-secret-access-key.age";
owner = "clawdinator";
group = "clawdinator";
};
age.secrets."clawdinator-telegram-bot-token" = {
file = "${secretsPath}/clawdinator-telegram-bot-token.age";
owner = "clawdinator";

View File

@ -12,22 +12,23 @@ fi
api_url_file="/etc/clawdinator/control-api-url"
token_file="/run/agenix/clawdinator-control-token"
access_key_file="/run/agenix/clawdinator-control-aws-access-key-id"
secret_key_file="/run/agenix/clawdinator-control-aws-secret-access-key"
caller_file="/etc/clawdinator/instance-name"
if [ ! -f "${api_url_file}" ]; then
echo "Missing control API URL: ${api_url_file}" >&2
exit 1
fi
if [ ! -f "${token_file}" ]; then
echo "Missing control API token: ${token_file}" >&2
exit 1
fi
if [ ! -f "${access_key_file}" ] || [ ! -f "${secret_key_file}" ]; then
echo "Missing control AWS credentials in /run/agenix" >&2
exit 1
fi
if [ ! -f "${caller_file}" ]; then
echo "Missing instance name: ${caller_file}" >&2
exit 1
fi
api_url="$(cat "${api_url_file}")"
control_token="$(cat "${token_file}")"
caller="$(cat "${caller_file}")"
@ -48,13 +49,23 @@ payload="$(jq -n \
--arg target "${target}" \
--arg caller "${caller}" \
--arg ami_override "${ami_override}" \
'{action: $action, target: $target, caller: $caller, ami_override: $ami_override}')"
--arg control_token "${control_token}" \
'{action: $action, target: $target, caller: $caller, ami_override: $ami_override, control_token: $control_token}')"
response="$(curl -sS -X POST \
-H "X-Clawdinator-Token: ${control_token}" \
-H "Content-Type: application/json" \
-d "${payload}" \
"${api_url}")"
region="${AWS_REGION:-eu-central-1}"
export AWS_ACCESS_KEY_ID="$(cat "${access_key_file}")"
export AWS_SECRET_ACCESS_KEY="$(cat "${secret_key_file}")"
response_file="$(mktemp)"
aws lambda invoke \
--function-name "clawdinator-control-api" \
--region "${region}" \
--payload "${payload}" \
--cli-binary-format raw-in-base64-out \
"${response_file}" >/dev/null
response="$(cat "${response_file}")"
rm -f "${response_file}"
if [ "${action}" = "status" ]; then
ok="$(printf '%s' "${response}" | jq -r '.ok')"

View File

@ -9,6 +9,8 @@ required_common=(
"clawdinator-anthropic-api-key.age"
"clawdinator-openai-api-key-peter-2.age"
"clawdinator-control-token.age"
"clawdinator-control-aws-access-key-id.age"
"clawdinator-control-aws-secret-access-key.age"
"clawdinator-telegram-bot-token.age"
"clawdinator-telegram-allow-from.age"
)