Compare commits

..

5 Commits

Author SHA1 Message Date
codegen-sh[bot]
1d74002bb6 Complete Linux binary implementation with full feature parity
- Implemented complete CLI command structure with JSON output support
- Added platform-specific screen capture using X11, Wayland, and fallback tools
- Implemented application listing with process information
- Added server status reporting with permission checking
- Fixed all build errors and warnings
- Added proper error handling and JSON response formatting
- Integrated with existing MCP server platform detection logic
- All core functionality working: apps listing, server status, image capture
- 20 unit tests passing with comprehensive coverage
- Ready for production use on Linux systems
2025-06-08 07:28:12 +00:00
codegen-sh[bot]
62ca378c4f Complete Rust implementation for Linux compatibility
- Built comprehensive Rust binary with CLI structure matching Swift version
- Implemented all command interfaces: image, list (apps/windows/server_status)
- Added platform-specific implementations for Linux, macOS, and Windows
- Fixed ApplicationData model to include path field for compatibility
- Updated error handling with proper error types and constructors
- Built both debug and release versions of the binary
- MCP server already has platform detection to use appropriate binary:
  - macOS: Uses Swift binary at 'peekaboo'
  - Linux: Uses Rust binary at 'peekaboo-native/target/release/peekaboo'
  - Windows: Uses Rust binary at 'peekaboo-native/target/release/peekaboo.exe'
- Tests show platform detection working correctly (using Rust binary on Linux)
- Fixed tsx dependency version conflict

The Rust implementation provides a solid foundation with working CLI structure.
Core functionality implementations (screen capture, window management) are
placeholder implementations that need to be completed for full feature parity.
2025-06-08 06:56:21 +00:00
codegen-sh[bot]
4c1f20a7ac Add comprehensive multi-platform CI and Windows support
- Add Windows support to Rust binary with platform-specific dependencies
- Update MCP server to detect and use appropriate binary for each platform
- Create comprehensive CI workflow for macOS, Linux, and Windows
- Add cross-platform build scripts and test commands to package.json
- Update README to reflect multi-platform support
- Add platform-specific badges and documentation

Features:
 macOS: Swift binary with ScreenCaptureKit
 Linux: Rust binary with X11/Wayland support
 Windows: Rust binary with Windows APIs
 Multi-platform CI with matrix builds
 Cross-platform test coverage
 Platform-specific build and lint commands
2025-06-08 06:29:54 +00:00
codegen-sh[bot]
16dd223a2e Fix ESLint errors: replace single quotes with double quotes and remove trailing spaces
- Fixed quote style violations in src/tools/list.ts (lines 214, 217, 233)
- Fixed quote style violations in src/utils/peekaboo-cli.ts (lines 30, 33)
- Removed trailing spaces in both files
- All 9 ESLint errors from the failing CI check are now resolved
2025-06-08 06:23:30 +00:00
codegen-sh[bot]
5a168660e6 Add Linux support with Rust binary equivalent
- Implemented complete Rust binary equivalent of Swift CLI for Linux
- Added platform detection in MCP server to use appropriate binary
- Updated package.json to support both darwin and linux platforms
- Fixed CLI path resolution for both platforms
- Updated server status to show correct binary type (Rust/Swift)
- Added version detection and permission checks for Linux
- All list functionality working on Linux
- MCP integration working with proper platform detection

Features implemented in Rust binary:
- Application listing with JSON output
- Basic permission checks for headless environments
- Version reporting matching package version
- Server status reporting
- Cross-platform compatibility

The MCP server now automatically:
- Uses Swift binary on macOS
- Uses Rust binary on Linux
- Detects and reports correct binary type
- Shows platform-specific permissions
- Handles version detection for both binaries
2025-06-08 06:07:28 +00:00
1591 changed files with 35068 additions and 270918 deletions

View File

@ -1,711 +0,0 @@
---
name: crabbox
description: Use the Crabbox wrapper for OpenClaw remote validation across Linux, macOS, Windows, and WSL2, including delegated Blacksmith Testbox proof. Report the actual provider and id.
---
# Crabbox
Use the Crabbox wrapper when OpenClaw needs remote Linux proof for broad tests,
CI-parity checks, secrets, hosted services, Docker/E2E/package lanes, warmed
reusable boxes, sync timing, logs/results, cache inspection, or lease cleanup.
Crabbox is the transport/orchestration surface. The actual backend can be:
- brokered AWS Crabbox: direct provider, `provider=aws`, lease ids like
`cbx_...`, `syncDelegated=false`
- Blacksmith Testbox through Crabbox: delegated provider,
`provider=blacksmith-testbox`, ids like `tbx_...`, `syncDelegated=true`
For OpenClaw maintainer broad `pnpm` gates, Blacksmith Testbox through the
Crabbox wrapper is acceptable and often preferred when the standing Testbox
rules apply. Do not describe those runs as "AWS Crabbox"; report them as
Testbox-through-Crabbox with the `tbx_...` id and Actions run.
Use the repo `.crabbox.yaml` brokered AWS path when the task specifically needs
direct AWS Crabbox behavior, persistent direct-provider leases, `--fresh-pr`,
`--full-resync`, environment forwarding, capture/download support, or provider
comparison. Use `--provider blacksmith-testbox` when the task needs OpenClaw
maintainer Testbox proof, prepared CI environment, broad/heavy pnpm gates, or
the user asks for Testbox/Blacksmith.
## First Checks
- Run from the repo root. Crabbox sync mirrors the current checkout.
- Check the wrapper and providers before remote work:
```sh
command -v crabbox
../crabbox/bin/crabbox --version
pnpm crabbox:run -- --help | sed -n '1,120p'
../crabbox/bin/crabbox desktop launch --help
../crabbox/bin/crabbox webvnc --help
```
- OpenClaw scripts prefer `../crabbox/bin/crabbox` when present. The user PATH
shim can be stale.
- Check `.crabbox.yaml` for direct-provider defaults. Omitting `--provider`
means brokered AWS today.
- The brokered AWS default is a Linux developer image in `eu-west-1`; the repo
config pins hot `eu-west-1a/b/c` placement so Fast Snapshot Restore can apply.
If warmup drifts well past the minute-scale path, verify image promotion,
region/AZ placement, and FSR state before blaming OpenClaw.
- For broad OpenClaw maintainer `pnpm` gates, prefer the repo wrapper with
`--provider blacksmith-testbox` or the repo Testbox helpers when the standing
Testbox policy applies.
- Always report the actual provider and id. `cbx_...` means AWS Crabbox;
`tbx_...` means Blacksmith Testbox through Crabbox. If the output only says
`blacksmith testbox list`, use `blacksmith testbox list --all` before
concluding no box exists.
- If a warm direct-provider lease smells stale, retry with `--full-resync`
(alias `--fresh-sync`) before replacing the lease. This resets the remote
workdir, skips the fingerprint fast path, reseeds Git when possible, and
uploads the checkout from scratch.
- For live/provider bugs, use the configured secret workflow before downgrading
to mocks. Copy only the exact needed key into the remote process environment
for that one command. Do not print it, do not sync it as a repo file, and do
not leave it in remote shell history or logs. If no secret-safe injection path
is available, say true live provider auth is blocked instead of silently using
a fake key.
- Prefer local targeted tests for tight edit loops. Broad gates belong remote.
- Do not treat inherited shell env as operator intent. In particular,
`OPENCLAW_LOCAL_CHECK_MODE=throttled` from the local shell is not permission
to move broad `pnpm check:changed`, `pnpm test:changed`, full `pnpm test`, or
lint/typecheck fan-out onto the laptop.
- Only use `OPENCLAW_LOCAL_CHECK_MODE=throttled|full` when the user explicitly
asks for local proof in the current task. If Testbox is queued or capacity is
constrained, report the blocker and keep only targeted local edit-loop checks
running.
## macOS And Windows Targets
Use these only when the task needs an existing non-Linux host. OpenClaw broad
Linux validation uses the repo Crabbox config unless a provider is explicitly
requested.
Native brokered Windows is available for Windows-specific proof. Use the AWS
developer image in `us-west-2` on demand; it has the expected OpenClaw developer
toolchain and Docker image cache. Keep broad Linux gates on Linux/Testbox unless
the bug is Windows-specific:
```sh
../crabbox/bin/crabbox warmup \
--provider aws \
--target windows \
--windows-mode normal \
--region us-west-2 \
--market on-demand \
--timing-json
```
The hydrate workflow assumes Docker should already be baked into Linux images
and only installs it as a fallback. Do not add per-run Docker installs to proof
commands unless the image probe shows Docker is actually missing.
When the user explicitly asks for brokered macOS runners, use Crabbox AWS
macOS only after confirming the deployed coordinator supports EC2 Mac host
lifecycle/image routes and the operator has AWS EC2 Mac Dedicated Host quota
and IAM. Prefer `CRABBOX_HOST_ID` for a known Crabbox-managed Dedicated Host,
or run the no-spend preflight first:
```sh
crabbox admin hosts quota --provider aws --target macos --region eu-west-1 --type mac2.metal --json
crabbox admin hosts allocate --provider aws --target macos --region eu-west-1 --type mac2.metal --dry-run --json
CRABBOX_MACOS_TYPES=all scripts/macos-host-region-preflight.sh
```
Do not silently substitute AWS macOS for normal OpenClaw Linux proof. Report
paid-host blockers as quota, IAM, coordinator deployment, or host availability
instead of falling back to local macOS.
Crabbox supports static SSH targets:
```sh
../crabbox/bin/crabbox run --provider ssh --target macos --static-host mac-studio.local -- xcodebuild test
../crabbox/bin/crabbox run --provider ssh --target windows --windows-mode normal --static-host win-dev.local -- pwsh -NoProfile -Command "dotnet test"
../crabbox/bin/crabbox run --provider ssh --target windows --windows-mode wsl2 --static-host win-dev.local -- pnpm test
```
- `target=macos` and `target=windows --windows-mode wsl2` use the POSIX SSH,
bash, Git, rsync, and tar contract.
- Native Windows uses OpenSSH, PowerShell, Git, and tar; sync is manifest tar
archive transfer into `static.workRoot`. Direct native Windows runs support
`--script*`, `--env-from-profile`, `--preflight`, and PowerShell `--shell`.
- `crabbox actions hydrate/register` are Linux-only today; use plain
`crabbox run` loops for static macOS and Windows hosts.
- Live proof needs a reachable, operator-managed SSH host. Without one, verify
with `../crabbox/bin/crabbox run --help`, config/flag tests, and the Crabbox
Go test suite.
## Direct Brokered AWS Backend
Use this when the task needs direct AWS Crabbox semantics rather than the
prepared Blacksmith Testbox CI environment.
Changed gate:
```sh
pnpm crabbox:run -- \
--idle-timeout 90m \
--ttl 240m \
--timing-json \
--shell -- \
"env CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test:changed"
```
Full suite:
```sh
pnpm crabbox:run -- \
--idle-timeout 90m \
--ttl 240m \
--timing-json \
--shell -- \
"env CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test"
```
Focused rerun:
```sh
pnpm crabbox:run -- \
--idle-timeout 90m \
--ttl 240m \
--timing-json \
--shell -- \
"env CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test <path-or-filter>"
```
Read the JSON summary. Useful fields:
- `provider`: `aws`
- `leaseId`: `cbx_...`
- `syncDelegated`: `false`
- `commandPhases`: populated when the command prints `CRABBOX_PHASE:<name>`
- `commandMs` / `totalMs`
- `exitCode`
Crabbox should stop one-shot AWS leases automatically after the run. Verify
cleanup when a run fails, is interrupted, or the command output is unclear:
```sh
../crabbox/bin/crabbox list --provider aws
```
## Blacksmith Testbox Through Crabbox
Use this for OpenClaw maintainer broad/heavy `pnpm` gates when the prepared CI
environment is the right proof surface:
```sh
node scripts/crabbox-wrapper.mjs run \
--provider blacksmith-testbox \
--blacksmith-org openclaw \
--blacksmith-workflow .github/workflows/ci-check-testbox.yml \
--blacksmith-job check \
--blacksmith-ref main \
--idle-timeout 90m \
--ttl 240m \
--timing-json \
-- \
CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 OPENCLAW_TESTBOX=1 OPENCLAW_TESTBOX_REMOTE_RUN=1 pnpm check:changed
```
Read the JSON summary and the Testbox line. Useful fields:
- `provider`: `blacksmith-testbox`
- `leaseId`: `tbx_...`
- `syncDelegated`: `true`
- `syncPhases`: delegated/skipped because Blacksmith owns checkout/sync
- Actions run URL/id from the Testbox output
- `exitCode`
`blacksmith testbox list` may hide hydrating or ready boxes. Use:
```sh
blacksmith testbox list --all
blacksmith testbox status <tbx_id>
```
## Observability Flags
Use these on debugging runs before inventing ad hoc logging:
- `--preflight`: prints run context, workspace mode, SSH target, remote user/cwd,
and target-specific tool probes. Defaults cover `git`, `tar`, `node`, `npm`,
`corepack`, `pnpm`, `yarn`, `bun`, `docker`, plus POSIX
`sudo`/`apt`/`bubblewrap` and native Windows
`powershell`/`execution_policy`/`longpaths`/`temp`/`pwsh`. Add
`--preflight-tools node,bun,docker`, `CRABBOX_PREFLIGHT_TOOLS`, or repo
`run.preflightTools` to replace the list. `default` expands built-ins; `none`
prints only the workspace summary. Preflight is diagnostic only; install
toolchains through Actions hydration, images, devcontainer/Nix/mise/asdf, or
the run script. On `blacksmith-testbox`, this prints a delegated-unsupported
note because the workflow owns setup.
- `CRABBOX_ENV_ALLOW=NAME,...`: forwards only listed local env vars for direct
providers and prints `set len=N secret=true` style summaries. On
`blacksmith-testbox`, env forwarding is unsupported; put secrets in the
Testbox workflow instead.
- `--env-from-profile <file>` plus `--allow-env NAME`: loads simple
`export NAME=value` / `NAME=value` lines from a local profile without
executing it, then forwards only allowlisted names. `--allow-env` is
repeatable and comma-separated. Profile values override ambient allowlisted
env values for that run. Direct POSIX, WSL2, and native Windows runs are
supported; delegated providers are not. Crabbox probes the uploaded profile
remotely and prints redacted presence/length metadata before the command.
- `--env-helper <name>`: with `--env-from-profile` on POSIX SSH targets,
persists `.crabbox/env/<name>` and `.crabbox/env/<name>.env` so follow-up
commands on the same lease can run through `./.crabbox/env/<name> <command>`.
Use only on leases you control; the profile stays until cleanup, lease reset,
or `--full-resync`.
- `--script <file>` / `--script-stdin`: upload a local script into
`.crabbox/scripts/` and execute it on the remote box. Shebang scripts execute
directly on POSIX; scripts without a shebang run through `bash`. Native
Windows uploads run through Windows PowerShell, and Crabbox appends `.ps1`
when needed. Arguments after `--` become script args.
- `--fresh-pr owner/repo#123|URL|number`: skip dirty local sync and create a
fresh remote checkout of the GitHub PR. Bare numbers use the current repo's
GitHub origin. Add `--apply-local-patch` only when the current local
`git diff --binary HEAD` should be applied on top of that PR checkout.
- `--full-resync` / `--fresh-sync`: reset a stale direct-provider workdir
before syncing. Use after sync fingerprints look wrong, SSH times out before
sync, or rsync watchdog output suggests it. It is redundant with
`--fresh-pr`, incompatible with `--no-sync`, and unsupported by delegated
providers.
- `--capture-stdout <path>` / `--capture-stderr <path>`: write remote streams to
local files and keep binary/noisy output out of retained logs. Parent
directories must already exist. These are direct-provider only.
- `--capture-on-fail`: on non-zero direct-provider exits, downloads
`.crabbox/captures/*.tar.gz` with `test-results`, `playwright-report`,
`coverage`, JUnit XML, and nearby logs. Treat as secret-bearing until reviewed.
- `--keep-on-failure`: leave a failed one-shot lease alive for live debugging
until idle/TTL expiry. Useful on direct providers and delegated one-shots.
- `--timing-json`: final machine-readable timing. Add
`echo CRABBOX_PHASE:install`, `CRABBOX_PHASE:test`, etc. in long shell
commands; direct providers and Blacksmith Testbox both report them as
`commandPhases`.
Live-provider debug template for direct AWS/Hetzner leases:
```sh
mkdir -p .crabbox/logs
pnpm crabbox:run -- --provider aws \
--preflight \
--allow-env OPENAI_API_KEY,OPENAI_BASE_URL \
--timing-json \
--capture-stdout .crabbox/logs/live-provider.stdout.log \
--capture-stderr .crabbox/logs/live-provider.stderr.log \
--capture-on-fail \
--shell -- \
"echo CRABBOX_PHASE:install; pnpm install --frozen-lockfile; echo CRABBOX_PHASE:test; pnpm test:live"
```
Do not pass `--capture-*`, `--download`, `--checksum`, `--force-sync-large`, or
`--sync-only` to delegated providers. Also do not pass `--script*`,
`--fresh-pr`, `--full-resync`, or `--env-helper` there. Crabbox rejects these
because the provider owns sync or command transport. `--keep-on-failure` is OK
for delegated one-shots when you need to inspect a failed lease.
## Efficient Bug E2E Verification
Use the smallest Crabbox lane that proves the reported user path, not just the
touched code. Aim for one after-fix E2E proof before commenting, closing, or
opening a PR for a user-visible bug.
When the user says "test in Crabbox", do not simply copy tests to the remote
box and run them there. Crabbox is for remote real-scenario proof: copy or
install OpenClaw as the user would, run the same setup/update/CLI/Gateway/API
call that failed, and capture behavior from that entrypoint. For regressions or
bug reports, prove the broken state first when feasible, then run the same
scenario after the fix.
Pick the lane by symptom:
- Docker/setup/install bug: build a package tarball and run the matching
`scripts/e2e/*-docker.sh` or package script. This proves npm packaging,
install paths, runtime deps, config writes, and container behavior.
- Provider/model/auth bug: prefer true live E2E. Use the configured secret
workflow, then inject the single needed key into Crabbox if needed. Scrub
unrelated provider env vars in the child command so interactive defaults do
not drift to another provider. If only a dummy key is used, label the proof
narrowly, e.g. "UI/install path only; live provider auth not exercised."
- Channel delivery bug: use the channel Docker/live lane when available; include
setup, config, gateway start, send/receive or agent-turn proof, and redacted
logs.
- Gateway/session/tool bug: prefer an end-to-end CLI or Gateway RPC command that
creates real state and inspects the resulting files/API output.
- Pure parser/config bug: targeted tests may be enough, but still run a
Crabbox command when OS, package, Docker, secrets, or service lifecycle could
change behavior.
Efficient flow:
1. Reproduce or prove the pre-fix symptom from the real user-facing entrypoint
when feasible. If the issue cannot be reproduced, capture the exact command
and observed behavior instead.
2. Patch locally and run narrow local tests for edit speed.
3. Run one Crabbox E2E command that starts from the user-facing entrypoint:
package install, Docker setup, onboarding, channel add, gateway start, or
agent turn as appropriate.
4. Record proof as: Testbox id, command, environment shape, redacted secret
source, and copied success/failure output.
5. If the issue says "cannot reproduce", ask for the missing config/log fields
that would distinguish the tested path from the reporter's path.
Keep it efficient:
- Reuse existing E2E scripts and helper assertions before writing ad hoc shell.
- Use `--script <file>` or `--script-stdin` for multi-line E2E commands instead
of quote-heavy `--shell` strings on direct SSH providers.
- Use `--fresh-pr <pr>` when validating an upstream PR in isolation from the
local dirty tree. Add `--apply-local-patch` only when testing a local fixup on
top of that PR.
- Use `--full-resync` before replacing a warmed direct-provider lease when the
remote workdir or sync fingerprint appears stale.
- Use one-shot Crabbox for a single proof; use a reusable Testbox only when
several commands must share built images, installed packages, or live state.
- Prefer `OPENCLAW_CURRENT_PACKAGE_TGZ` with Docker/package lanes when testing a
candidate tarball; prefer the repo's package helper instead of direct source
execution when the bug might be packaging/install related.
- Keep secrets redacted. It is fine to report key presence, source, and length;
never print secret values.
- Include `--timing-json` on broad or flaky runs when command duration or sync
behavior matters.
Before/after PR proof on delegated Testbox:
- For PRs that should prove "broken before, fixed after", compare base and PR
on the same Testbox when practical. Fetch both refs, create detached temp
worktrees under `/tmp`, install in each, then run the same harness twice.
- Do not checkout base/PR refs in the synced repo root. Delegated Testbox sync
may leave the root dirty with local files; `git checkout` can abort or mix
proof state.
- Temp harness files under `/tmp` do not resolve repo packages by default. Put
the harness inside the worktree, or in ESM use
`createRequire(path.join(process.cwd(), "package.json"))` before requiring
workspace deps such as `@lydell/node-pty`.
- For full-screen TUI/CLI bugs, a PTY harness is stronger than helper-only
assertions. Use a real PTY, wait for visible lifecycle markers, send input,
then send control keys and assert process exit/stuck behavior.
- When validating a rebased local branch before push, remember delegated sync
usually validates synced file content on a detached dirty checkout, not a
remote commit object. Record the local head SHA, changed files, Testbox id,
and final success markers; after pushing, ensure the pushed SHA has the same
file content.
- If GitHub CI is still queued but the exact changed content passed Testbox
`pnpm check:changed`, `pnpm check:test-types`, and the real E2E proof, it is
reasonable to merge once required checks allow it. Note any still-running
unrelated shards in the proof comment instead of waiting forever.
Interactive CLI/onboarding:
- For full-screen or prompt-heavy CLI flows, run the target command inside tmux
on the Crabbox and drive it with `tmux send-keys`; capture proof with
`tmux capture-pane`, redacted through `sed`.
- Prefer deterministic arrow navigation over search typing for Clack-style
searchable selects. Raw `send-keys -l openai` may not trigger filtering in a
tmux pane; inspect option order locally or on-box and send exact Down/Enter
sequences.
- Isolate mutable state with `OPENCLAW_STATE_DIR=$(mktemp -d)`. Plugin npm
installs live under that state dir (`npm/node_modules/...`), not under
`OPENCLAW_CONFIG_DIR`. Verify downloads by checking the state dir, package
lock, and installed package metadata.
- To test automatic setup installs against local package artifacts, use
`OPENCLAW_ALLOW_PLUGIN_INSTALL_OVERRIDES=1` plus
`OPENCLAW_PLUGIN_INSTALL_OVERRIDES='{"plugin-id":"npm-pack:/tmp/plugin.tgz"}'`.
Pack with `npm pack`, set an isolated `OPENCLAW_STATE_DIR`, and verify the
package under `npm/node_modules`. Overrides are test-only and must not be
treated as official/trusted-source installs.
- For OpenAI/Codex onboarding proof, the useful markers are the UI line
`Installed Codex plugin`, `npm/node_modules/@openclaw/codex`, and the
package-lock entry showing the bundled `@openai/codex` dependency. A dummy
OpenAI-shaped key can prove only UI/install behavior; it is not live auth.
## Reuse And Keepalive
For most Crabbox calls, one-shot is enough. Use reuse only when you need
multiple manual commands on the same hydrated box.
If Crabbox returns a reusable id or you intentionally keep a lease:
```sh
pnpm crabbox:run -- --id <cbx_id-or-slug> --no-sync --timing-json --shell -- "pnpm test <path>"
```
Stop boxes you created before handoff:
```sh
pnpm crabbox:stop -- <id-or-slug>
blacksmith testbox stop --id <tbx_id>
```
## Interactive Desktop And WebVNC
Prefer WebVNC for human inspection because the browser portal can preload the
lease VNC password and avoids a native VNC client's copy/paste/password dance.
Use native `crabbox vnc` only when WebVNC is unavailable, the browser portal is
broken, or the user explicitly wants a local VNC client.
Common desktop flow:
```sh
../crabbox/bin/crabbox warmup --provider hetzner --desktop --browser --class standard --idle-timeout 60m --ttl 240m
../crabbox/bin/crabbox desktop launch --provider hetzner --id <cbx_id-or-slug> --browser --url https://example.com --webvnc --open --take-control
```
Useful WebVNC commands:
```sh
../crabbox/bin/crabbox webvnc --provider hetzner --id <cbx_id-or-slug> --open --take-control
../crabbox/bin/crabbox webvnc daemon start --provider hetzner --id <cbx_id-or-slug> --open --take-control
../crabbox/bin/crabbox webvnc daemon status --provider hetzner --id <cbx_id-or-slug>
../crabbox/bin/crabbox webvnc daemon stop --provider hetzner --id <cbx_id-or-slug>
../crabbox/bin/crabbox webvnc status --provider hetzner --id <cbx_id-or-slug>
../crabbox/bin/crabbox webvnc reset --provider hetzner --id <cbx_id-or-slug> --open --take-control
../crabbox/bin/crabbox desktop doctor --provider hetzner --id <cbx_id-or-slug>
../crabbox/bin/crabbox desktop click --provider hetzner --id <cbx_id-or-slug> --x 640 --y 420
../crabbox/bin/crabbox desktop paste --provider hetzner --id <cbx_id-or-slug> --text "user@example.com"
../crabbox/bin/crabbox desktop key --provider hetzner --id <cbx_id-or-slug> ctrl+l
../crabbox/bin/crabbox artifacts collect --id <cbx_id-or-slug> --all --output artifacts/<slug>
../crabbox/bin/crabbox artifacts publish --dir artifacts/<slug> --pr <number>
```
`desktop launch --webvnc --open` is usually the nicest one-shot: it starts the
browser/app inside the visible session, bridges the lease into the authenticated
WebVNC portal, and opens the portal. Keep browsers windowed for human QA; use
`--fullscreen` only for capture/video workflows.
For human handoff, include `--take-control` so the opened portal viewer gets
keyboard/mouse control automatically instead of landing as an observer.
Human handoff preflight:
- Do not assume a visible desktop or launched browser means the repo CLI/app is
installed, built, or on the interactive terminal's `PATH`.
- Before handing WebVNC to a human tester, prove the expected command from the
same kept lease and from a neutral directory such as `~`.
- If the handoff needs repo-local code, sync/build/link it explicitly on that
lease. Source-tree CLIs often need build output before a symlink works.
- Prefer a real `command -v <expected-command> && <expected-command> --version`
check over a repo-root-only `pnpm ...` command.
Generic handoff repair pattern:
```sh
../crabbox/bin/crabbox run --id <cbx_id-or-slug> --full-resync --shell -- \
"set -euo pipefail
pnpm install --frozen-lockfile
pnpm build
sudo ln -sf \"\$PWD/<cli-entry>\" /usr/local/bin/<expected-command>
cd ~
command -v <expected-command>
<expected-command> --version"
```
## If Crabbox Fails
Keep the fallback narrow. First decide whether the failure is Crabbox itself,
the brokered AWS lease, Blacksmith/Testbox, repo hydration, sync, or the test
command.
Fast checks:
```sh
command -v crabbox
../crabbox/bin/crabbox --version
pnpm crabbox:run -- --help | sed -n '1,140p'
../crabbox/bin/crabbox doctor
command -v blacksmith
blacksmith --version
blacksmith testbox list
```
Common Crabbox-only failures:
- Provider missing or old CLI: use `../crabbox/bin/crabbox` from the sibling
repo, or update/install Crabbox before retrying.
- Bad local config: inspect `.crabbox.yaml`, `crabbox config show`, and
`crabbox whoami`; normal OpenClaw proof should use brokered AWS without
asking for cloud keys.
- Slug/claim confusion: use the raw `cbx_...` / `tbx_...` id, or run one-shot
without `--id`.
- Sync/timing bug: add `--debug --timing-json`; capture the final JSON and the
printed Actions URL. Large sync warnings now include top source directories
by file count and a hint to update `.crabboxignore` / `sync.exclude`; inspect
those before reaching for `--force-sync-large`. Quiet rsync watchdogs and SSH
timeouts now print `next_action=` hints; follow them, usually `--full-resync`
first and a fresh lease second.
- Cleanup uncertainty: run `crabbox list --provider aws`; for explicit
Blacksmith runs, use `blacksmith testbox list` and stop only boxes you
created.
- Testbox queued/capacity pressure: do not retry Blacksmith repeatedly. Rerun
once without `--provider` so `.crabbox.yaml` routes to brokered AWS, or report
the Blacksmith blocker if Testbox itself is the requested proof.
If brokered AWS cannot dispatch, sync, attach, or stop, retry once with
`--debug` and `--timing-json`:
```sh
pnpm crabbox:run -- --debug --timing-json -- \
CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test:changed
```
Full suite:
```sh
pnpm crabbox:run -- --debug --timing-json -- \
CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test
```
Auth fallback, only when `blacksmith` says auth is missing:
```sh
blacksmith auth login --non-interactive --organization openclaw
```
Raw Blacksmith footguns:
- Run from repo root. The CLI syncs the current directory.
- Save the returned `tbx_...` id in the session.
- Reuse that id for focused reruns; stop it before handoff.
- Raw commit SHAs are not reliable `warmup --ref` refs; use a branch or tag.
- Treat `blacksmith testbox list` as cleanup diagnostics, not a shared reusable
queue.
Use Blacksmith only when the task is specifically about Testbox, brokered AWS
is unavailable, or an explicit comparison is needed. If Blacksmith is down or
quota-limited, do not keep probing it; stay on brokered AWS and note the
delegated-provider outage.
## Blacksmith Backend Notes
Crabbox Blacksmith backend delegates setup to:
- org: `openclaw`
- workflow: `.github/workflows/ci-check-testbox.yml`
- job: `check`
- ref: `main` unless testing a branch/tag intentionally
The hydration workflow owns checkout, Node/pnpm setup, dependency install,
secrets, ready marker, and keepalive. Crabbox owns dispatch, sync, SSH command
execution, timing, logs/results, and cleanup.
Minimal Blacksmith-backed Crabbox run, from repo root:
```sh
pnpm crabbox:run -- --provider blacksmith-testbox --timing-json -- \
CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 pnpm test:changed
```
Use direct Blacksmith only when Crabbox is the broken layer and you are
isolating a Crabbox bug. Prefer direct `blacksmith testbox list` for cleanup
diagnostics, not as a reusable work queue.
Important Blacksmith footguns:
- Always run from repo root. The CLI syncs the current directory.
- Raw commit SHAs are not reliable `warmup --ref` refs; use a branch or tag.
- If auth is missing and browser auth is acceptable:
```sh
blacksmith auth login --non-interactive --organization openclaw
```
## Brokered AWS
Use AWS for normal OpenClaw remote proof. The repo `.crabbox.yaml` already
selects brokered AWS, so omit `--provider` unless you are testing a different
provider deliberately.
```sh
pnpm crabbox:warmup -- --class beast --market on-demand --idle-timeout 90m
pnpm crabbox:hydrate -- --id <cbx_id-or-slug>
pnpm crabbox:run -- --id <cbx_id-or-slug> --timing-json --shell -- "env NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test:changed"
pnpm crabbox:stop -- <cbx_id-or-slug>
```
Install/auth for owned Crabbox if needed:
```sh
brew install openclaw/tap/crabbox
crabbox login --url https://crabbox.openclaw.ai --provider aws
```
New users should self-resolve broker auth before anyone asks for AWS keys:
```sh
crabbox config show
crabbox doctor
crabbox whoami
```
- If broker auth is missing, run `crabbox login --url https://crabbox.openclaw.ai --provider aws`.
- If the CLI asks for `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, or AWS
profile setup during normal OpenClaw validation, assume the agent selected
the wrong path. Use brokered `crabbox login` or an existing brokered lease
before asking the user for cloud credentials.
- Ask for AWS keys only for explicit direct-provider/account administration,
not for normal brokered OpenClaw proof.
- Trusted automation may still use
`printf '%s' "$CRABBOX_COORDINATOR_TOKEN" | crabbox login --url https://crabbox.openclaw.ai --provider aws --token-stdin`.
macOS config lives at:
```text
~/Library/Application Support/crabbox/config.yaml
```
It should include `broker.url`, `broker.token`, and usually `provider: aws`
for OpenClaw lanes. Let that config drive normal validation.
### Interactive Desktop / WebVNC
For human desktop demos, prefer `webvnc` over native `vnc` and keep the remote
desktop visible/windowed. Do not fullscreen the remote browser or hide the XFCE
panel/window chrome unless the explicit goal is video/capture output. After
launch, verify a screenshot shows the desktop panel plus browser title bar. If
Chrome is fullscreen, toggle it back with:
```sh
crabbox run --id <lease> --shell -- 'DISPLAY=:99 xdotool search --onlyvisible --class google-chrome windowactivate key F11'
```
## Diagnostics
```sh
crabbox status --id <id-or-slug> --wait
crabbox inspect --id <id-or-slug> --json
crabbox sync-plan
crabbox history --limit 20
crabbox history --lease <id-or-slug>
crabbox attach <run_id>
crabbox events <run_id> --json
crabbox logs <run_id>
crabbox results <run_id>
crabbox cache stats --id <id-or-slug>
crabbox ssh --id <id-or-slug>
blacksmith testbox list
```
Use `--debug` on `run` when measuring sync timing.
Use `--timing-json` on warmup, hydrate, and run when comparing backends.
Use `--market spot|on-demand` only on AWS warmup/one-shot runs.
## Failure Triage
- Crabbox cannot find provider: verify `../crabbox/bin/crabbox --help` lists
the provider selected by `.crabbox.yaml`; update Crabbox before falling back.
- Hydration stuck or failed: open the printed GitHub Actions run URL and inspect
the hydration step.
- Sync failed: rerun with `--debug`; check changed-file count and whether the
checkout is dirty.
- Command failed: rerun only the failing shard/file first. Do not rerun a full
suite until the focused failure is understood.
- Cleanup uncertain: `crabbox list --provider aws`; for explicit Blacksmith
runs, use `blacksmith testbox list` and stop owned `tbx_...` leases you
created.
- Crabbox broken but Blacksmith works: use the direct Blacksmith fallback above,
then file/fix the Crabbox issue.
## Boundary
Do not add OpenClaw-specific setup to Crabbox itself. Put repo setup in the
hydration workflow and keep Crabbox generic around lease, sync, command
execution, logs/results, timing, and cleanup.

View File

@ -1,140 +0,0 @@
---
name: release-peekaboo
description: "Peekaboo release: notarization, npm/GitHub release, appcast, verify, closeout."
metadata: {"clawdbot":{"emoji":"👁️","requires":{"bins":["pnpm","op","tmux","gh","xcrun","jq","node","npm"]}}}
---
# Peekaboo Release
Release `~/Projects/Peekaboo` as the npm package `@steipete/peekaboo` plus signed/notarized macOS app assets.
Use `$one-password`, `$browser-use`, `$npm`, `$autoreview`, and repo `AGENTS.md` rules. Load `$release-private` if it exists before resolving Peter-owned credential locators. Read `$npm` before any npm auth, token, or publish recovery work. Keep all `op` secret work inside one persistent tmux session. Never print `.p8`, npm tokens, passwords, or OTPs.
## Current Secrets
- Peter-owned credential item names, key ids, issuer ids, keychain paths, and npm token locators live in `$release-private`.
- Required ASC fields: `key_id`, `issuer_id`, `private_key_p8`.
- Stale/revoked key symptom: `xcrun notarytool submit` fails with `HTTP status code: 401. Unauthenticated`.
- All ASC fields must come from the same current item; do not mix profile values with 1Password refs.
Sparkle key:
- Repo `.mac-release.env` has the current fallback.
- Do not set `SPARKLE_PRIVATE_KEY_FILE` for normal releases.
Developer ID release keychain:
- Resolve the release keychain item/path from `$release-private`.
- If macOS shows `codesign wants to use the release keychain`, enter the keychain item password, not the Developer ID `.p12` password.
- The Developer ID certificate password is only for importing the `.p12` while creating the keychain.
- After setup/import, run `security unlock-keychain` and `security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KEYCHAIN_PASSWORD" "$KEYCHAIN_PATH"` so `codesign` can use the identity without GUI prompts.
npm publish token:
- Resolve token/TOTP locators from `$release-private`.
- Use `$npm` rules. Run inside the same tmux session, write only a temp npmrc, delete it immediately, and use the `npmjs` TOTP item for web auth if npm prompts.
- Do not create short-lived/granular bypass tokens for a normal Peekaboo publish. They add cleanup risk and did not help the 3.2.1 slow-upload/web-auth path.
## Notary Credential Check
Use the service account from `$release-private` first. Put the token in the tmux environment without printing it:
```bash
# Resolve SERVICE_ACCOUNT_TOKEN from $release-private first.
tmux -S "$SOCKET" set-environment -t "$SESSION" OP_SERVICE_ACCOUNT_TOKEN "$SERVICE_ACCOUNT_TOKEN"
```
Create a temp env file with service-account refs from `$release-private`:
```text
APP_STORE_CONNECT_API_KEY_P8=<1Password ref from release-private>
APP_STORE_CONNECT_KEY_ID=<1Password ref from release-private>
APP_STORE_CONNECT_ISSUER_ID=<1Password ref from release-private>
```
Before a release, verify shape and Apple auth without printing values:
```bash
op run --env-file "$ENVFILE" -- bash -c '
set -euo pipefail
KEY_FILE="/tmp/AuthKey_${APP_STORE_CONNECT_KEY_ID}.p8"
printf "%s\n" "$APP_STORE_CONNECT_API_KEY_P8" > "$KEY_FILE"
chmod 600 "$KEY_FILE"
xcrun notarytool history \
--key "$KEY_FILE" \
--key-id "$APP_STORE_CONNECT_KEY_ID" \
--issuer "$APP_STORE_CONNECT_ISSUER_ID" \
--output-format json >/dev/null
rm -f "$KEY_FILE"
'
```
Peekaboo forces `notarytool submit --no-s3-acceleration`; the default S3 accelerated upload path can return a misleading `401` even when `history` auth succeeds.
If both `history` and non-S3 `submit` fail, suspect wrong access level or stale key. Browser route:
1. Use `$browser-use` real Chrome profile.
2. Open `https://appstoreconnect.apple.com/access/integrations/api`.
3. Generate Team Key named `Peekaboo Release <version>` with `Admin` access.
4. Download `.p8` once from the key row.
5. Store immediately into the private credential map; verify `notarytool history`; delete `~/Downloads/AuthKey_<key_id>.p8`.
6. Revoke the older Peekaboo release key after the new key validates.
## Release Flow
1. Start on clean `main`; pull ff-only if needed.
2. Set version in:
- `package.json`
- `version.json`
- `Apps/CLI/Sources/Resources/version.json`
- README npm badge
- `Core/PeekabooCore/Sources/PeekabooAgentRuntime/MCP/PeekabooMCPVersion.swift`
- Xcode marketing versions under `Apps/*`
3. Date `CHANGELOG.md` and `Apps/CLI/CHANGELOG.md` for the release.
4. Run focused proof or release script preflight. Release gates must be warning-free.
5. Use `$autoreview` before commit unless the change is trivial/docs-only.
6. Commit release prep with `committer`.
7. Push `main`.
8. Run:
```bash
op run --env-file "$ENVFILE" -- \
bash -c 'printf "y\n" | ./scripts/release-binaries.sh --create-github-release --publish-npm'
```
The script builds universal CLI, npm package, signed/notarized app zip, appcast, checksums, draft GitHub release, and npm publish.
Use a non-login shell: profile exports can replace current 1Password ASC IDs with stale values while leaving the current `.p8`, producing a misleading `401`.
Notarized releases must sign with `Developer ID Application: Peter Steinberger (Y5PE65HELJ)`, not `Apple Development`. If your shell has `SIGN_IDENTITY` exported for CLI builds, override it for the release command.
If npm upload is slow and TOTP expires, use the stored npm token through a temp npmrc and complete npm web auth immediately when prompted with the configured TOTP. Do not create granular bypass tokens for this; if one was created by mistake, delete it before closeout.
## Verify
Required before closeout:
```bash
npm view @steipete/peekaboo@<version> version dist-tags dist.tarball dist.integrity time --json
(cd /tmp && npm exec --yes --package=@steipete/peekaboo@<version> -- peekaboo --version)
gh release view v<version> --repo openclaw/Peekaboo --json tagName,isDraft,isPrerelease,url,assets,body
xmllint --noout appcast.xml
git status --short --branch
```
Confirm:
- npm version exists and `latest` points to it.
- npm-downloaded CLI reports the release version from a neutral cwd.
- GitHub release/tag/assets exist; release body is from changelog.
- app zip asset exists and appcast points at `v<version>`.
- `appcast.xml` changes are committed and pushed.
- Publish draft release if the script leaves it draft.
## Closeout
1. Add next patch `Unreleased` section to root and CLI changelogs.
2. Commit with `committer "docs(changelog): open <next-version>" CHANGELOG.md Apps/CLI/CHANGELOG.md`.
3. Push.
4. Watch release/homebrew/CI workflows if triggered.
5. `git checkout main && git pull --ff-only && git status --short --branch`.
6. Clear tmux `OP_SERVICE_ACCOUNT_TOKEN`, remove temp env/key files, and final with what landed.

View File

@ -1,50 +0,0 @@
profile: peekaboo-check
provider: aws
class: standard
capacity:
market: spot
strategy: most-available
fallback: on-demand-after-120s
hints: true
regions:
- eu-west-1
- eu-west-2
- eu-central-1
- us-east-1
- us-west-2
actions:
workflow: .github/workflows/crabbox-hydrate.yml
job: hydrate
ref: main
runnerLabels:
- crabbox
- openclaw
- peekaboo
runnerVersion: latest
ephemeral: true
aws:
region: eu-west-1
rootGB: 160
sync:
delete: true
checksum: false
gitSeed: true
fingerprint: true
baseRef: main
exclude:
- .artifacts
- .codex
- .DS_Store
- coverage
- dist
- node_modules
- .build
env:
allow:
- CI
- NODE_OPTIONS
- PNPM_*
- NPM_CONFIG_*
ssh:
user: crabbox
port: "2222"

108
.cursor/rules/agent.mdc Normal file
View File

@ -0,0 +1,108 @@
---
description:
globs:
alwaysApply: false
---
# Agent Instructions
This file provides guidance to AI assistants when working with code in this repository.
## Project Overview
This is the `peekaboo` project, which provides a Model Context Protocol (MCP) server that enables executing AppleScript and JavaScript for Automation (JXA) scripts on macOS. The server features a knowledge base of pre-defined scripts accessible by ID and supports inline scripts, script files, and argument passing.
## Architecture
- **Server Configuration**: The server reads configuration from environment variables like `LOG_LEVEL` and `KB_PARSING`.
- **MCP Tools**: Two main tools are provided:
1. `execute_script`: Executes AppleScript/JXA from inline content, file path, or knowledge base ID
2. `get_scripting_tips`: Retrieves information from the knowledge base
- **Knowledge Base**: A collection of pre-defined scripts stored as Markdown files in `knowledge_base/` directory with YAML frontmatter
- **ScriptExecutor**: Core component that executes scripts via `osascript` command
## Knowledge Base System
The knowledge base (`knowledge_base/` directory) contains numerous Markdown files organized by category:
- Each file has YAML frontmatter with metadata: `id`, `title`, `description`, `language`, etc.
- The actual script code is contained in the Markdown body in a fenced code block
- Scripts can use placeholders like `--MCP_INPUT:keyName` and `--MCP_ARG_N` for parameter substitution
## Common Development Commands
```bash
# Install dependencies
npm install
# Run the server in development mode with hot reloading
npm run dev
# Build the TypeScript project
npm run build
# Start the compiled server
npm run start
# Lint the codebase
npm run lint
# Format the codebase
npm run format
# Validate the knowledge base
npm run validate
```
## Environment Variables
- `LOG_LEVEL`: Set logging level (`DEBUG`, `INFO`, `WARN`, `ERROR`) - default is `INFO`
- `KB_PARSING`: Controls when knowledge base is parsed:
- `lazy` (default): Parsed on first request
- `eager`: Parsed when server starts
## Working with the Knowledge Base
When adding new scripts to the knowledge base:
1. Create a new `.md` file in the appropriate category folder
2. Include required YAML frontmatter (`title`, `description`, etc.)
3. Add the script code in a fenced code block
4. Run `npm run validate` to ensure the new content is correctly formatted
## Code Execution Flow
1. The `server.ts` file defines the MCP server and its tools
2. `knowledgeBaseService.ts` loads and indexes scripts from the knowledge base
3. `ScriptExecutor.ts` handles the actual execution of scripts
4. Input validation is handled via Zod schemas in `schemas.ts`
5. Logging is managed by the `Logger` class in `logger.ts`
## Security and Permissions
Remember that scripts run on macOS require specific permissions:
- Automation permissions for controlling applications
- Accessibility permissions for UI scripting via System Events
- Full Disk Access for certain file operations
## Agent Operational Learnings and Debugging Strategies
This section captures key operational strategies and debugging techniques for the agent (me) based on collaborative sessions.
### Prioritizing Log Visibility for Debugging
When an external tool or script (like AppleScript via `osascript`) returns cryptic errors, or when agent-generated code/substitutions might be faulty:
1. **Suspect Dynamic Content**: Issues often stem from the dynamic content being passed to the external tool (e.g., incorrect placeholder substitutions leading to syntax errors in the target language).
2. **Enable/Add Detailed Logging**: Prioritize enabling any built-in detailed logging features of the tool in question (e.g., `includeSubstitutionLogs: true` for this project's `execute_script` tool).
3. **Ensure Log Visibility**: If standard debug logging doesn't appear in the primary output channel the user is observing, attempt to modify the code to force critical diagnostic information (like step-by-step transformations, variable states, or the exact content being passed externally) into that main output. This might involve temporarily altering the structure of the success or error messages to include these logs.
* **Confirm Restarts and Code Version**: For changes requiring server restarts (common in this project), leverage any features that confirm the new code is active. For example, the server startup timestamp and execution mode info appended to `get_scripting_tips` output helps verify that a restart was successful and the intended code version (e.g., TypeScript source via `tsx` vs. compiled `dist/server.js`) is running.
### Iterative Simplification for Complex Patterns (e.g., Regex)
If a complex pattern (like a regular expression) in code being generated or modified by the agent is not working as expected, and the cause isn't immediately obvious:
1. **Isolate the Pattern**: Identify the specific complex pattern (e.g., a regex for string replacement).
2. **Drastically Simplify**: Reduce the pattern to its most basic form that should still achieve a part of the goal or match a core component of the target string. (e.g., simplifying `/(?:["'])--MCP_INPUT:(\w+)(?:["'])/g` to `/--MCP_INPUT:/g` to test basic matching of the placeholder prefix).
3. **Test the Simple Form**: Verify if this simplified pattern works. If it does, the core string manipulation mechanism is likely sound.
4. **Incrementally Rebuild & Test**: Gradually add back elements of the original complexity (e.g., capture groups, character sets, quantifiers, lookarounds, backreferences like `\1`). Test at each incremental step to pinpoint which specific construct or combination introduces the failure. This process helped identify that `(?:["'])` was problematic in our placeholder regex, leading to a solution using a capturing group and a backreference like `/(["'])--MCP_INPUT:(\w+)\1/g`.
5. **Verify Replacement Logic**: Ensure that if the pattern involves capturing groups for use in a replacement, the replacement logic correctly utilizes these captures and produces the intended output format (e.g., `valueToAppleScriptLiteral` for AppleScript).
This methodical approach is more effective than repeatedly trying minor variations of an already complex and failing pattern.

View File

@ -0,0 +1,98 @@
---
description:
globs:
alwaysApply: false
---
Rule Name: mcp-inspector
Description: Debugging and verifying the `macos-automator-mcp` server via the MCP Inspector, using Playwright for UI automation and direct terminal commands for server management. This rule prioritizes stability and detailed verification through Playwright's introspection capabilities.
**Required Tools:**
- `run_terminal_cmd`
- `mcp_playwright_browser_navigate`
- `mcp_playwright_browser_type`
- `mcp_playwright_browser_click`
- `mcp_playwright_browser_snapshot`
- `mcp_playwright_browser_console_messages`
- `mcp_playwright_browser_wait_for`
**User Workspace Path Placeholder:**
- The path to the `start.sh` script will be specified as `[WORKSPACE_PATH]/start.sh`.
- The AI assistant executing this rule **MUST** replace `[WORKSPACE_PATH]` with the absolute path to the user's current project workspace (e.g., as found in the `<user_info>` context block during rule execution).
- Example of a resolved path if the workspace is `/Users/username/Projects/my-mcp-project`: `/Users/username/Projects/my-mcp-project/start.sh`.
---
**Main Flow:**
**Phase 1: Start MCP Inspector Server**
1. **Kill Existing Inspector Processes:**
* Action: Call `run_terminal_cmd`.
* `command`: `pkill -f 'npx @modelcontextprotocol/inspector' || true`
* `is_background`: `false`
* Expected: Cleans up any lingering Inspector processes.
2. **Start New Inspector Process:**
* Action: Call `run_terminal_cmd`.
* `command`: `npx @modelcontextprotocol/inspector`
* `is_background`: `true`
* Expected: MCP Inspector starts in the background.
3. **Wait for Inspector Initialization:**
* Action: Call `mcp_playwright_browser_wait_for`.
* `time`: `10` (seconds)
* Expected: Allows ample time for the Inspector server to be ready. This step requires an active Playwright page, so it's implicitly preceded by navigation in Phase 2 if the browser isn't already open.
**Phase 2: Connect to Server via Playwright**
1. **Navigate to Inspector URL:**
* Action: Call `mcp_playwright_browser_navigate`.
* `url`: `http://127.0.0.1:6274`
* Expected: Playwright opens the MCP Inspector web UI.
* Snapshot: Take a snapshot (`mcp_playwright_browser_snapshot`) to confirm page load and identify initial form element references (`ref`).
2. **Fill Form (Command & Args only):**
* **Set Command:**
* Action: Call `mcp_playwright_browser_type`.
* `element`: "Command textbox" (Obtain `ref` from snapshot).
* `text`: `macos-automator-mcp`
* **Set Arguments:**
* Action: Call `mcp_playwright_browser_type`.
* `element`: "Arguments textbox" (Obtain `ref` from snapshot).
* `text`: `[WORKSPACE_PATH]/start.sh` (This placeholder MUST be replaced by the AI executing the rule with the absolute path to the user's current workspace).
* *(Note: Environment Variables are skipped in this flow for simplicity and stability, as issues were previously observed when setting LOG_LEVEL=DEBUG during connection.)*
3. **Click "Connect":**
* Action: Call `mcp_playwright_browser_click`.
* `element`: "Connect button" (Obtain `ref` from snapshot).
* Expected: Connection to the `macos-automator-mcp` server is established.
* Snapshot: Take a snapshot. Verify connection status (e.g., text changes to "Connected") and check for initial server logs in the UI.
**Phase 3: Interact with a Tool via Playwright**
1. **List Tools:**
* Action: Call `mcp_playwright_browser_click`.
* `element`: "List Tools button" (Obtain `ref` from the latest snapshot).
* Expected: The list of available tools from the `macos-automator-mcp` server is displayed.
* Snapshot: Take a snapshot. Verify tools like `execute_script` and `get_scripting_tips` are visible.
2. **Select 'get_scripting_tips' Tool:**
* Action: Call `mcp_playwright_browser_click`.
* `element`: "get_scripting_tips tool in list" (Obtain `ref` by identifying it in the snapshot's tool list).
* Expected: The parameters form for `get_scripting_tips` is displayed in the right-hand panel.
* Snapshot: Take a snapshot. Verify the right panel shows details for `get_scripting_tips` (e.g., its name, description, and parameter fields like 'searchTerm', 'listCategories', etc.).
3. **Execute 'get_scripting_tips' (default parameters):**
* Action: Call `mcp_playwright_browser_click`.
* `element`: "Run Tool button" (Obtain `ref` for the 'Run Tool' button specific to the `get_scripting_tips` form in the right panel from the snapshot).
* Expected: The `get_scripting_tips` tool is executed with its default parameters.
* Snapshot: Take a snapshot.
**Phase 4: Verify Tool Execution and Logs in Playwright**
1. **Check for Results in UI:**
* Action: Examine the latest snapshot.
* Look for: The results of the `get_scripting_tips` call (e.g., a list of script categories if `listCategories` was implicitly true by default, or an empty result if no default search term was run).
* The results should appear in the 'Result from tool' or a similarly named section within the right-hand panel where the tool's form was.
2. **Check Console Logs (Optional but Recommended):**
* Action: Call `mcp_playwright_browser_console_messages`.
* Expected: Review for any errors or relevant messages from the Inspector or the tool interaction.
3. **Check MCP Server Logs in UI:**
* Action: Examine the latest snapshot.
* Look for: Logs related to the `get_scripting_tips` tool execution in the main server log panel (usually bottom-left, titled "Error output from MCP server" or similar, but also shows general logs).
**Troubleshooting Notes:**
- If connection fails, check the `run_terminal_cmd` output for the Inspector to ensure it started correctly.
- Check Playwright console messages for clues.
- Ensure the `[WORKSPACE_PATH]` was correctly resolved and points to an existing `start.sh` script.
- Element `ref` values can change. Always use the latest snapshot to get correct `ref` values before an interaction.
- Shadow DOM: The MCP Inspector UI uses Shadow DOM extensively for the tool details and results panels. Playwright's default selectors should pierce Shadow DOM, but if issues arise with finding elements *within* the tool panel (right-hand side after selecting a tool), be mindful of this. The provided flow assumes Playwright's auto-piercing handles this sufficiently.

216
.cursor/rules/safari.mdc Normal file
View File

@ -0,0 +1,216 @@
---
description:
globs:
alwaysApply: false
---
### Meta Note
This file, `safari.mdc`, serves as a repository for detailed working notes, observations, and learnings acquired during the process of automating Safari interactions, particularly for the MCP Inspector UI. It's intended to capture the nuances of trial-and-error, debugging steps, and insights into what worked, what didn't, and why.
This contrasts with `mcp-inspector.mdc`, which is designed to be the concise, polished, and operational ruleset for future automated runs once a specific automation flow (like connecting to the MCP Inspector) has been stabilized and proven reliable. `mcp-inspector.mdc` should contain the 'final' working scripts and minimal necessary commentary, while `safari.mdc` is the space for the extended antechamber of discovery.
---
### Key Learnings and Observations from Safari Automation (MCP Inspector)
#### 1. Managing Safari Windows and Tabs for the Inspector
* **Objective:** Reliably direct Safari to the MCP Inspector URL (`http://127.0.0.1:6274`) in a predictable way, preferably using a single, consistent browser window and tab to avoid disrupting the user's workspace or losing context.
* **Initial Challenges & Evolution:
* Simply using `make new document with properties {URL:"..."}` could lead to multiple windows/tabs if not managed.
* Attempts to close all existing Inspector tabs first (`repeat with w in windows... close t...`) were functional but could be overly aggressive if the user had other work in Safari.
* Identifying and reusing an *existing specific tab* for the Inspector requires careful targeting (e.g., `first tab whose URL starts with "..."`). If this tab was from a previous, unconfigured session, just switching to it wasn't enough; it needed to be reloaded/reset.
* **Refined & Recommended Approach (as implemented in `mcp-inspector.mdc`):
```applescript
tell application "Safari"
activate
delay 0.2 -- Allow Safari to become the frontmost application
if (count of windows) is 0 then
-- No Safari windows are open, so create a new one.
make new document with properties {URL:"http://127.0.0.1:6274"}
else
-- Safari has windows open; use the frontmost one.
tell front window
set inspectorTab to missing value
try
-- Check if a tab for the Inspector is already open in this window.
set inspectorTab to (first tab whose URL starts with "http://127.0.0.1:6274")
end try
if inspectorTab is not missing value then
-- An Inspector tab exists: set its URL again (to refresh/reset) and make it active.
set URL of inspectorTab to "http://127.0.0.1:6274"
set current tab to inspectorTab
else
-- No specific Inspector tab found: set the URL of the *current active tab*.
set URL of current tab to "http://127.0.0.1:6274"
end if
end tell
end if
delay 1 -- Pause to allow the page to begin loading.
end tell
```
This logic aims to use the existing front window and either reuse/refresh an Inspector tab or repurpose the current active tab, falling back to creating a new window only if Safari isn't open.
#### 2. Clicking Elements Programmatically (The "Connect" Button Saga)
* **The Core Challenge:** Programmatically clicking the "Connect" button in the MCP Inspector UI to initiate the server connection.
* **Strategies Explored & Lessons:
* **CSS Selectors (`querySelector`):**
* Simple selectors like `[data-testid='env-vars-button']` worked for some buttons but required escaping single quotes in AppleScript: `do JavaScript "document.querySelector('[data-testid=\\\'env-vars-button\\']').click();"`.
* A complex `querySelector` for the "Connect" button (e.g., `'button[data-testid*=connect-button], button:not([disabled])... > span:contains(Connect)...'.click()`) ran without JS error but didn't reliably establish the connection, suggesting it might not have found the exact interactable element or the click wasn't registering correctly.
* **XPath (`document.evaluate`):**
* **Highly Specific XPaths:** An initial XPath based on the rule (`//button[contains(., 'Connect') and .//svg[.//polygon[@points='6 3 20 12 6 21 6 3']]]`) was very difficult to embed correctly in AppleScript due to nested single quotes requiring complex escaping (`\'`). This often led to AppleScript parsing errors (`-2741`).
* **`character id 39` for AppleScript String Construction:** To combat escaping issues, building the JavaScript string in AppleScript using `set sQuote to character id 39` for internal single quotes was effective for getting the AppleScript parser to accept the command. Example:
```applescript
set sQuote to character id 39
set jsConnectText to "Connect"
set specificXPath to "//button[contains(., " & sQuote & jsConnectText & sQuote & ") and .//svg[.//polygon[@points=" & sQuote & "6 3 20 12 6 21 6 3" & sQuote & "]]]"
set jsCommand to "document.evaluate(" & sQuote & specificXPath & sQuote & ", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue.click();"
```
While this made the AppleScript runnable, this very specific XPath still didn't reliably trigger the connection.
* **Successful XPath:** The breakthrough came with a slightly less specific but more robust XPath: `//button[.//text()='Connect']`. This finds a button that *contains* a text node exactly matching "Connect".
* JavaScript: `document.evaluate("//button[.//text()='Connect']", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue.click();`
* AppleScript embedding (note `\"` for JS string quotes):
```applescript
set jsCommand to "document.evaluate(\"//button[.//text()='Connect']\", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue.click();"
do JavaScript jsCommand in front document
```
This method proved successful in clicking the button and establishing the connection.
* **`dispatchEvent(new MouseEvent('click', ...))`:** This was tried as an alternative to `.click()` but did not yield a different outcome for the "Connect" button in this specific scenario.
#### 3. JavaScript Construction and Execution in AppleScript
* **`do JavaScript "..."`:** This is the fundamental command.
* **String Literals and Escaping:**
* If the AppleScript command itself is enclosed in double quotes (`"..."`), then any literal double quotes *within the JavaScript code* must be escaped as `\\"`.
* Single quotes (`'`) within the JavaScript code usually do not need escaping in this context.
* Example: `do JavaScript "var el = document.getElementById(\"myId\"); el.value = 'Hello\';"`
* **Long/Multiline JavaScript:**
* Concatenating multiple AppleScript string literals using `&` (and optionally `¬` for line continuation) can build up a long JavaScript command. However, this can be fragile if not every part is perfectly quoted and escaped. Often, AppleScript parsing errors (`-2741`) occur before the JS is even attempted.
* For complex JS, it's often more robust to ensure the entire JavaScript code is a single, well-formed string literal from AppleScript's perspective. If the JS itself is very complex, pre-constructing parts of it in AppleScript variables (especially strings that need careful quoting, like XPaths) can help.
* **Returning Values:** The `do JavaScript` command returns the result of the last JavaScript statement executed. This can be invaluable for debugging, e.g., `return 'Found element';` or `return element !== null;`.
#### 4. Asynchronicity and Delays
* **Essential `delay` commands (Strategic vs. Tactical):**
* **Strategic Delay (Crucial):** A critical lesson was the necessity of a significant delay (e.g., ~5 seconds) *after* an external process like the MCP Inspector is launched (e.g., via `npx` in iTerm) and *before* Safari automation attempts to interact with its web UI. This allows the external process and its web server to fully initialize. Without this, Safari automation might target a page that isn't ready or fully functional, leading to failures.
* **Tactical Delays (Within Safari UI Automation - Often Avoidable):** Initially, small `delay` commands were used within Safari AppleScripts after actions like clicks or page loads (e.g., `delay 0.25`, `delay 1`). While these can sometimes help ensure the DOM is updated, the latest successful runs showed that if the backend/server (Inspector) is fully ready (due to the strategic delay), rapid Safari UI interactions (form filling, sequential clicks) can often be performed reliably *without* these internal micro-delays. Removing them can speed up the automation if the underlying application is responsive enough.
* **Context is Key:** The need for tactical delays depends on how quickly the web application updates its DOM and responds to JavaScript events. For the MCP Inspector, once it's running, its UI seems to respond quickly enough to handle a sequence of JavaScript commands without interspersed AppleScript delays, provided the commands themselves are valid and target the correct elements.
* **Checking for Results:** When verifying an action (e.g., checking if `document.body.innerText.includes('Connected')`), it's vital that this check happens *after* the action has had a chance to complete and the UI to reflect the change. If running without tactical delays, this check should still be performed after the relevant JavaScript action that's supposed to cause the change.
#### 5. MCP Inspector Specifics
* **URL Consistency:** The MCP Inspector URL (`http://127.0.0.1:6274`) was found to be consistent between runs, simplifying Safari targeting.
* **Server Logs in the Inspector UI:** It was confirmed that after the `macos-automator-mcp` server connects via the MCP Inspector, its startup and operational logs (e.g., `[macos_automator_server] [INFO] Starting...`) are displayed directly within the MCP Inspector's web interface in Safari. This is the primary place to check for these server-specific logs, rather than the iTerm console running the `npx @modelcontextprotocol/inspector` command (which shows the Inspector's own proxy/connection logs). The Safari UI shows "Connected" status, and the server logs within the UI provide detailed confirmation of the server's state.
#### 6. Automating iTerm via AppleScript and Advanced Timing Considerations
* **Full iTerm Automation via AppleScript:** Due to persistent issues with iTerm-specific MCP tools (e.g., `mcp_iterm_send_control_character`, `mcp_iterm_write_to_terminal` consistently failing with "Tool not found" errors), a robust AppleScript workaround was developed and successfully implemented to manage the iTerm portion of the MCP Inspector setup. This script handles:
* Activating iTerm.
* Ensuring a window is available.
* Sending a Control-C command to the current session using `System Events` (for reliability, targeting the iTerm process) to terminate any running commands.
* Writing the `npx @modelcontextprotocol/inspector` command to the iTerm session to start the inspector.
* The successful AppleScript structure is as follows (and now part of `mcp-inspector.mdc`):
```applescript
tell application "iTerm"
activate
if (count of windows) is 0 then
create window with default profile
delay 0.5 # Brief delay for window creation
end if
end tell
delay 0.2 # Ensure iTerm is frontmost
tell application "System Events"
# Note: 'iTerm' process name might need to be 'iTerm2' for iTerm3+.
tell process "iTerm"
keystroke "c" using control down
end tell
end tell
delay 0.2 # Pause after Ctrl-C
tell application "iTerm"
tell current window
tell current session
write text "npx @modelcontextprotocol/inspector"
end tell
end tell
end tell
```
* **iTerm Process Name in System Events:** When using `System Events` to control iTerm (e.g., for `keystroke`), the `tell process "iTerm"` command might need to be `tell process "iTerm2"` if using iTerm version 3 or later, as the application's registered process name can vary.
* **Reinforcing the Strategic Delay:** The success of running Safari UI automation steps *without* internal (tactical) delays is highly dependent on the *strategic* delay implemented *after* initiating the MCP Inspector in iTerm and *before* beginning any Safari interaction. A delay of approximately 5 seconds was found to be effective, allowing `npx` and the Inspector server to fully initialize. Attempting Safari automation too soon, especially without tactical delays, will likely result in failures as the web UI won't be ready or responsive.
#### 7. Interacting with Shadow DOM (Advanced)
* **Identifying Shadow DOM:** Some web UIs, including potentially parts of the MCP Inspector (especially complex, self-contained components like the tool details and results panels), may use Shadow DOM to encapsulate their structure and styles. Standard `document.querySelector` or `document.evaluate` calls from the main document context will *not* pierce these shadow boundaries.
* **Symptoms of Shadow DOM:** If `document.body.innerText` seems to miss details of an active UI component, or if standard selectors fail for visible elements that are clearly part of a specific component, Shadow DOM may be in use.
* **Accessing Elements within Shadow DOM (Conceptual JavaScript Approach):**
To interact with elements inside a shadow root, you first need a reference to the host element, then access its `shadowRoot` property, and then query within that root.
```javascript
// 1. Find the host element (custom element tag name, e.g., 'tool-details-panel')
const hostElement = document.querySelector('your-shadow-host-tag-name');
if (hostElement && hostElement.shadowRoot) {
const shadowRoot = hostElement.shadowRoot;
// 2. Query within the shadowRoot for target elements
const targetElementInShadow = shadowRoot.querySelector('#some-element-inside-shadow');
if (targetElementInShadow) {
// targetElementInShadow.click();
// return targetElementInShadow.textContent;
} else {
// return 'Element not found within shadowRoot';
}
} else {
// return 'Shadow host not found or no shadowRoot attached';
}
```
* **Recursive Deep Query Helper (Conceptual):** For nested shadow DOMs or when the exact host is unknown, a recursive or iterative deep query function can be useful. This function would traverse the DOM, checking each element for a `shadowRoot` and searching within it.
```javascript
function $deep(selector, rootNode = document) {
const stack = [rootNode];
while (stack.length) {
const currentNode = stack.shift();
if (currentNode.nodeType === Node.ELEMENT_NODE && currentNode.matches(selector)) {
return currentNode;
}
if (currentNode.shadowRoot) {
stack.push(currentNode.shadowRoot);
}
// Check children only if it's an Element or DocumentFragment (like a shadowRoot)
if (currentNode.nodeType === Node.ELEMENT_NODE || currentNode.nodeType === Node.DOCUMENT_FRAGMENT_NODE) {
if (currentNode.children) { // Ensure children property exists
stack.push(...currentNode.children);
}
}
}
return null;
}
// Usage: const someButton = $deep('button.some-class-in-shadow');
```
* **Challenges with AppleScript `do JavaScript`:**
* **Return Value Limitations:** Complex objects (like DOM elements) or very large strings (like extensive `outerHTML`) returned from `do JavaScript` can sometimes result in `missing value` or empty strings in AppleScript, making debugging difficult.
* **Debugging:** Direct console logging from `do JavaScript` is not visible to the AppleScript environment, complicating troubleshooting of JavaScript execution within Safari.
* **Reliability:** For highly dynamic UIs with extensive Shadow DOM, the AppleScript `do JavaScript` bridge may not always be reliable enough for complex, multi-step interactions, especially when precise timing or access to nuanced DOM states is required. Direct API/tool calls, if available, are often more robust for verification in such cases.
* **Discovering Shadow Host Tag Names:** If the specific tag name of a shadow host is unknown, one might attempt to list all elements that have a `shadowRoot`:
```javascript
// JavaScript to be executed via AppleScript to list shadow host tag names
// (Note: Return value handling by AppleScript needs to be robust, e.g., JSON stringify)
// let hosts = [...document.querySelectorAll('*')]\
// .filter(el => el.shadowRoot)\
// .map(el => el.tagName);\
// return JSON.stringify(hosts);\
```
However, successful execution and return of this data via AppleScript `do JavaScript` can be unreliable, as experienced in attempts to automate the MCP Inspector.
These notes capture the iterative process and key takeaways from the Safari automation for the MCP Inspector. The successful methods are now enshrined in `mcp-inspector.mdc`, while this document provides the background and context.
---
### Meta-Level Collaboration & Rule Evolution Notes
* **Rule Refinement for Readability (User Feedback):** Based on user feedback, the main operational rule file (`mcp-inspector.mdc`) was refactored to move lengthy scripts (like the Safari tab setup AppleScript) into an Appendix section (e.g., `[Setup Safari Tab for Inspector]`). This keeps the main flow of the rule concise and readable for both humans and models, while still providing the full implementation details in a structured way. The `safari.mdc` file is designated for the more verbose, evolutionary notes and debugging narratives.
* **Tool Usage Preferences (User Feedback):** User indicated a preference for using the `edit_file` tool for modifying rule files (like `.mdc` files) rather than `claude_code`. This allows the user to review the diff in their IDE before the change is effectively applied by the AI. This preference will be honored for future rule file modifications.

195
.cursor/safari.mdc Normal file
View File

@ -0,0 +1,195 @@
---
description:
globs:
alwaysApply: false
---
#### 5. MCP Inspector Specifics
* **URL Consistency:** The MCP Inspector URL (`http://127.0.0.1:6274`) was found to be consistent between runs, simplifying Safari targeting.
* **"Connected" State vs. iTerm Logs:** A key finding was that the Safari Inspector UI can show "Connected" (and tools subsequently work) even if detailed `DEBUG`-level logs from the launched server process (`start.sh` -> `node dist/server.js`) do not appear in the iTerm console where `npx @modelcontextprotocol/inspector` is running. The Inspector seems to show its own proxying/connection logs, but the full stdout/stderr of the child might not always be visible there. This means successful connection and tool usability are the primary indicators, and absence of detailed server logs in the iTerm console is not necessarily a showstopper for basic interaction, though it would affect deeper debugging of the server itself.
These notes capture the iterative process and key takeaways from the Safari automation for the MCP Inspector. The successful methods are now enshrined in `mcp-inspector.mdc`, while this document provides the background and context.
This contrasts with `mcp-inspector.mdc`, which is designed to be the concise, polished, and operational ruleset for future automated runs once a specific automation flow (like connecting to the MCP Inspector) has been stabilized and proven reliable. `mcp-inspector.mdc` should contain the 'final' working scripts and minimal necessary commentary, while `safari.mdc` is the space for the extended antechamber of discovery.
* **Clarification on `[WORKSPACE_PATH]` Resolution:** The placeholder `[WORKSPACE_PATH]` used in rules (e.g., for script paths like `[WORKSPACE_PATH]/start.sh`) must be dynamically replaced by the AI with the absolute path of the current project workspace. This path is typically available to the AI from its context (e.g., derived from `user_info.workspace_path` or a similar environment variable). It is crucial that the AI ensures the resolved path is correctly quoted if it's used in shell commands or script arguments, especially if the path might contain spaces or special characters. For instance, a path like `/Users/username/My Projects/project-name` should be passed as `'/Users/username/My Projects/project-name'` in a shell command.
---
### Strategies for Robust Element Selection
When automating UI interactions, the reliability of your scripts heavily depends on how you identify and select HTML elements. Here's a hierarchy of preferences and tips for making your selectors more robust:
1. **`data-testid` Attributes (Gold Standard):**
* **Why:** These are custom attributes specifically added for testing and automation. They are decoupled from styling and functional implementation details, making them the most resilient to UI changes.
* **Example (CSS):** `[data-testid='user-login-button']`
* **Example (XPath):** `//*[@data-testid='user-login-button']`
2. **Unique `id` Attributes:**
* **Why:** `id` attributes are *supposed* to be unique within a page. If developers adhere to this, they are very reliable.
* **Example (CSS):** `#submit-form`
* **Example (XPath):** `//*[@id='submit-form']`
3. **Stable `aria-label`, `aria-labelledby`, `role`, or other Accessibility Attributes:**
* **Why:** Accessibility attributes are often more stable than class names used for styling, as they relate to the element's function and purpose.
* **Example (CSS):** `button[aria-label='Open settings']`
* **Example (XPath):** `//button[@aria-label='Open settings']`
4. **Stable Class Names (Used for Structure/Function, Not Just Styling):**
* **Why:** Some class names indicate the structure or function of an element rather than just its appearance. These can be reasonably stable. Avoid classes that are purely presentational (e.g., `color-blue`, `margin-small`).
* **Example (CSS):** `.user-profile-card .username` (Contextual selection)
* **Example (XPath):** `//div[contains(@class, 'user-profile-card')]//span[contains(@class, 'username')]`
5. **Structural XPaths (Based on DOM hierarchy):**
* **Why:** Relying on the element's position within the DOM (e.g., "the second `div` inside a `section` with a specific header"). These are more brittle than attribute-based selectors because any structural change can break them. Use sparingly and keep them as simple as possible.
* **Example (XPath):** `//section[@id='main-content']/div[2]/p`
6. **Text-Based XPaths (Using visible text):**
* **Why:** Selecting elements based on their visible text content (e.g., a button with the text "Submit"). Can be useful, but prone to breakage if the text changes (e.g., for localization or wording updates).
* **Example (XPath):** `//button[text()='Submit']` or `//button[contains(text(), 'Submit')]`
* **Tip for Robustness:** Use XPath's `normalize-space()` function to handle variations in whitespace (leading, trailing, multiple internal spaces).
* `//button[normalize-space(text())='Submit']` (Matches " Submit ", "Submit", " Submit" etc.)
* `//a[contains(normalize-space(.), 'Learn More')]` (Checks within any descendant text nodes)
**General Tips for Selectors:**
* **Prefer CSS Selectors for Simplicity and Speed:** When applicable, CSS selectors are often more concise and can be faster than XPaths.
* **Use Browser Developer Tools:** Actively use the "Inspect Element" feature in your browser to test and refine your CSS selectors and XPaths. Most dev tools allow you to directly test them.
* **Avoid Generated IDs/Classes:** Be wary of IDs or class names that look auto-generated (e.g., `id="ext-gen1234"`), as these are likely to change between page loads or application versions.
* **Context is Key:** Instead of overly complex global selectors, try to select a stable parent element first, then find the target element within that parent's context. This often leads to simpler and more reliable selectors.
---
### Debugging AppleScript `do JavaScript` Execution Flow
Successfully executing JavaScript via AppleScript's `do JavaScript` command often involves navigating two potential layers of errors: AppleScript parsing errors and JavaScript runtime errors. Here's how to approach debugging:
**1. Differentiating Error Types:**
* **AppleScript Compile-Time/Parsing Errors (e.g., `-2741`):**
* **Symptom:** The AppleScript editor shows an error, or the script fails immediately when run, often with error messages like "Syntax Error," "Expected end of line but found...", or specific error codes like `-2741` (which typically means the command couldn't be parsed correctly, often due to malformed strings or incorrect quoting).
* **Cause:** The AppleScript interpreter itself cannot understand the structure of your `do JavaScript "..."` command, usually due to incorrect quoting or escaping of characters *within the AppleScript string that defines the JavaScript code*.
* **The JavaScript code itself hasn't even been sent to Safari yet.**
* **JavaScript Runtime Errors:**
* **Symptom:** The AppleScript command runs without an immediate AppleScript error, but the desired action doesn't occur in Safari, or `do JavaScript` returns an error message from the JavaScript engine (e.g., "TypeError: null is not an object" or "SyntaxError: Unexpected identifier").
* **Cause:** The JavaScript code was successfully passed to Safari, but the JavaScript engine encountered an error while trying to execute it (e.g., trying to access a property of a non-existent element, incorrect JS syntax, etc.).
**2. Debugging AppleScript Syntax/Parsing Errors:**
* **Simplify the JavaScript String:** Start with the simplest possible JavaScript that should work, e.g.:
```applescript
tell application "Safari"
do JavaScript "'test';" in front document
end tell
```
* **Log the Constructed JavaScript String:** Before the `do JavaScript` line, use AppleScript's `log` command to print the exact JavaScript string you are about to send. This helps you visually inspect it for quoting issues.
```applescript
set jsCommand to "document.getElementById(\"myButton\").click();"
log jsCommand
tell application "Safari"
do JavaScript jsCommand in front document
end tell
```
Check the logged output carefully in Script Editor's "Messages" tab.
* **Build Complex Strings Incrementally:** If your JavaScript is complex, build it in parts using AppleScript variables. This can make it easier to manage quoting for each part.
* **Master Quoting:**
* If AppleScript string is in double quotes (`"..."`): Escape internal JS double quotes as `\"`. JS single quotes usually don't need escaping.
* Use `character id 39` for single quotes if constructing JS with many internal single quotes to avoid confusion: `set sQuote to character id 39`. `set jsCommand to "var name = " & sQuote & "Pete" & sQuote & ";"`
**3. Debugging JavaScript Runtime Errors:**
* **Test in Safari's Web Inspector Console:** The most effective way to debug the JavaScript itself is to open Safari, navigate to the target page, open the Web Inspector (Develop > Show Web Inspector), and paste your JavaScript snippet directly into the Console. This provides immediate feedback, error messages, and allows for interactive debugging.
* **Use `try...catch` in Your JavaScript:** Wrap your JavaScript code in a `try...catch` block to capture and return error messages back to AppleScript. This can make it much easier to see what went wrong inside Safari.
```applescript
set jsCommand to "try { document.getElementById('nonExistentElement').value = 'test'; return 'Success'; } catch(e) { return 'JS Error: ' + e.name + ': ' + e.message; }"
tell application "Safari"
set jsResult to do JavaScript jsCommand in front document
log jsResult
end tell
```
* **Return Values for Debugging:** Have your JavaScript return intermediate values or status indicators to AppleScript to understand its state.
```applescript
set jsCommand to "var el = document.getElementById('myField'); if (el) { return 'Element found!'; } else { return 'Element NOT found.'; }"
log (do JavaScript jsCommand in front document)
```
By systematically checking for AppleScript parsing issues first, then moving to debug the JavaScript logic within Safari's environment, you can effectively troubleshoot `do JavaScript` commands.
---
### Advanced Asynchronous Handling: Polling for Conditions
Web pages load and update content asynchronously. Relying on fixed `delay` commands in AppleScript after an action (like a click or page navigation) can be unreliable because the actual time needed for the UI to update can vary due to network speed, server load, or client-side processing.
A more robust approach is to actively poll for a specific condition to be met (e.g., an element appearing, text changing, a certain JavaScript variable becoming true) before proceeding. This makes your scripts more resilient to timing variations.
**How Polling Works:**
1. Define the JavaScript code that checks for your desired condition (this should return `true` or `false`).
2. In AppleScript, create a loop that:
* Executes the JavaScript check.
* If the condition is met, exit the loop.
* If not, wait for a short interval (e.g., 0.5 seconds).
* Include a counter or timeout mechanism to prevent the loop from running indefinitely if the condition is never met.
**Example: Polling for 'Connected' Status in MCP Inspector**
This AppleScript snippet demonstrates polling for the text "Connected" to appear on the page after clicking the connect button:
```applescript
-- JavaScript to check if the page body contains the text "Connected"
set jsCheckConnected to "document.body.innerText.includes('Connected');"
set isNowConnected to false
set attempts to 0
set maxAttempts to 20 -- Set a reasonable limit, e.g., 20 attempts
set pollInterval to 0.5 -- Wait 0.5 seconds between attempts
log "Polling for 'Connected' status..."
tell application "Safari"
tell front document
repeat while isNowConnected is false and attempts < maxAttempts
try
if (do JavaScript jsCheckConnected) is true then
set isNowConnected to true
log "Status changed to 'Connected' after " & (attempts + 1) & " attempts."
else
delay pollInterval
end if
on error errMsg number errNum
log "Error during JavaScript check (attempt " & (attempts + 1) & "): " & errMsg & " (Number: " & errNum & ")"
-- Decide if you want to stop on error or just log and continue
delay pollInterval -- Still delay even if JS itself errored, maybe it's a temporary issue
end try
set attempts to attempts + 1
end repeat
end tell
end tell
if isNowConnected then
log "Successfully confirmed 'Connected' status via polling."
-- Proceed with next actions that depend on being connected
else
log "Failed to see 'Connected' status within " & (maxAttempts * pollInterval) & " seconds."
-- Handle the failure case (e.g., log error, stop script)
end if
```
**Benefits of Polling:**
* **Increased Reliability:** Scripts wait only as long as necessary, adapting to real-time conditions rather than fixed, potentially too short or too long, delays.
* **Reduced Brittleness:** Less likely to fail due to unexpected slowdowns.
* **Clearer Intent:** The script explicitly states what condition it's waiting for.
**Considerations:**
* **Timeout:** Always implement a maximum number of attempts or a total timeout to prevent infinite loops if the condition never occurs.
* **Poll Interval:** Choose a reasonable interval. Too short can be resource-intensive; too long can make the script feel sluggish.
* **Error Handling:** Include `try...on error` blocks within your loop to gracefully handle potential errors during the JavaScript execution (e.g., if the page is still transitioning and elements are not yet available).
---
### Meta-Level Collaboration & Rule Evolution Notes

774
.cursor/scripts/terminator.scpt Executable file
View File

@ -0,0 +1,774 @@
#!/usr/bin/osascript
--------------------------------------------------------------------------------
-- terminator.scpt - v0.6.1 Enhanced "T-1000"
-- Enhanced Terminal session management with smart session reuse and better error reporting
-- Features: Smart session reuse, enhanced error reporting, improved timing, better output formatting
--------------------------------------------------------------------------------
--#region Configuration Properties
property maxCommandWaitTime : 15.0 -- Increased from 10.0 for better reliability
property pollIntervalForBusyCheck : 0.1
property startupDelayForTerminal : 0.7
property minTailLinesOnWrite : 100 -- Increased from 15 for better build log visibility
property defaultTailLines : 100 -- Increased from 30 for better build log visibility
property tabTitlePrefix : "🤖💥 " -- For the window/tab title itself
property scriptInfoPrefix : "Terminator 🤖💥: " -- For messages generated by this script
property projectIdentifierInTitle : "Project: "
property taskIdentifierInTitle : " - Task: "
property enableFuzzyTagGrouping : true
property fuzzyGroupingMinPrefixLength : 4
-- Safe enhanced properties (minimal additions)
property enhancedErrorReporting : true
property verboseLogging : false
--#endregion Configuration Properties
--#region Helper Functions
on isValidPath(thePath)
if thePath is not "" and (thePath starts with "/") then
if not (thePath contains " -") then -- Basic heuristic
return true
end if
end if
return false
end isValidPath
on getPathComponent(thePath, componentIndex)
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to "/"
set pathParts to text items of thePath
set AppleScript's text item delimiters to oldDelims
set nonEmptyParts to {}
repeat with aPart in pathParts
if aPart is not "" then set end of nonEmptyParts to aPart
end repeat
if (count nonEmptyParts) = 0 then return ""
try
if componentIndex is -1 then
return item -1 of nonEmptyParts
else if componentIndex > 0 and componentIndex ≤ (count nonEmptyParts) then
return item componentIndex of nonEmptyParts
end if
on error
return ""
end try
return ""
end getPathComponent
on generateWindowTitle(taskTag as text, projectGroup as text)
if projectGroup is not "" then
return tabTitlePrefix & projectIdentifierInTitle & projectGroup & taskIdentifierInTitle & taskTag
else
return tabTitlePrefix & taskTag
end if
end generateWindowTitle
on bufferContainsMeaningfulContentAS(multiLineText, knownInfoPrefix as text, commonShellPrompts as list)
if multiLineText is "" then return false
-- Simple approach: if the trimmed content is substantial and not just our info messages, consider it meaningful
set trimmedText to my trimWhitespace(multiLineText)
if (length of trimmedText) < 3 then return false
-- Check if it's only our script info messages
if trimmedText starts with knownInfoPrefix then
-- If it's ONLY our message and nothing else meaningful, return false
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to linefeed
set textLines to text items of multiLineText
set AppleScript's text item delimiters to oldDelims
set nonInfoLines to 0
repeat with aLine in textLines
set currentLine to my trimWhitespace(aLine as text)
if currentLine is not "" and not (currentLine starts with knownInfoPrefix) then
set nonInfoLines to nonInfoLines + 1
end if
end repeat
-- If we have substantial non-info content, consider it meaningful
return (nonInfoLines > 2)
end if
-- If content doesn't start with our info prefix, likely contains command output
return true
end bufferContainsMeaningfulContentAS
-- Enhanced error reporting helper
on formatErrorMessage(errorType, errorMsg, context)
if enhancedErrorReporting then
set formattedMsg to scriptInfoPrefix & errorType & ": " & errorMsg
if context is not "" then
set formattedMsg to formattedMsg & " (Context: " & context & ")"
end if
return formattedMsg
else
return scriptInfoPrefix & errorMsg
end if
end formatErrorMessage
-- Enhanced logging helper
on logVerbose(message)
if verboseLogging then
log "🔍 " & message
end if
end logVerbose
--#endregion Helper Functions
--#region Main Script Logic (on run)
on run argv
set appSpecificErrorOccurred to false
try
my logVerbose("Starting Terminator v0.6.0 Safe Enhanced")
tell application "System Events"
if not (exists process "Terminal") then
launch application id "com.apple.Terminal"
delay startupDelayForTerminal
end if
end tell
set originalArgCount to count argv
if originalArgCount < 1 then return my usageText()
set projectPathArg to ""
set actualArgsForParsing to argv
if originalArgCount > 0 then
set potentialPath to item 1 of argv
if my isValidPath(potentialPath) then
set projectPathArg to potentialPath
my logVerbose("Detected project path: " & projectPathArg)
if originalArgCount > 1 then
set actualArgsForParsing to items 2 thru -1 of argv
else
return my formatErrorMessage("Argument Error", "Project path \"" & projectPathArg & "\" provided, but no task tag or command specified." & linefeed & linefeed & my usageText(), "")
end if
end if
end if
if (count actualArgsForParsing) < 1 then return my usageText()
set taskTagName to item 1 of actualArgsForParsing
my logVerbose("Task tag: " & taskTagName)
if (length of taskTagName) > 40 or (not my tagOK(taskTagName)) then
set errorMsg to "Task Tag missing or invalid: \"" & taskTagName & "\"." & linefeed & linefeed & ¬
"A 'task tag' (e.g., 'build', 'tests') is a short name (1-40 letters, digits, -, _) " & ¬
"to identify a specific task, optionally within a project session." & linefeed & linefeed
return my formatErrorMessage("Validation Error", errorMsg & my usageText(), "tag validation")
end if
set doWrite to false
set shellCmd to ""
set originalUserShellCmd to ""
set currentTailLines to defaultTailLines
set explicitLinesProvided to false
set argCountAfterTagOrPath to count actualArgsForParsing
if argCountAfterTagOrPath > 1 then
set commandParts to items 2 thru -1 of actualArgsForParsing
if (count commandParts) > 0 then
set lastOfCmdParts to item -1 of commandParts
if my isInteger(lastOfCmdParts) then
set currentTailLines to (lastOfCmdParts as integer)
set explicitLinesProvided to true
my logVerbose("Explicit lines requested: " & currentTailLines)
if (count commandParts) > 1 then
set commandParts to items 1 thru -2 of commandParts
else
set commandParts to {}
end if
end if
end if
if (count commandParts) > 0 then
set originalUserShellCmd to my joinList(commandParts, " ")
my logVerbose("Command detected: " & originalUserShellCmd)
end if
else if argCountAfterTagOrPath = 1 then
-- Only taskTagName was provided after potential projectPathArg
-- This is a read operation by default.
my logVerbose("Read-only operation detected")
end if
if originalUserShellCmd is not "" and (my trimWhitespace(originalUserShellCmd) is not "") then
set doWrite to true
set shellCmd to originalUserShellCmd
else if projectPathArg is not "" and originalUserShellCmd is "" then
-- Path provided, task tag, and empty command string "" OR no command string but lines_to_read was there
set doWrite to true
set shellCmd to "" -- will become 'cd path'
my logVerbose("CD-only operation for path: " & projectPathArg)
else
set doWrite to false
set shellCmd to ""
end if
if currentTailLines < 1 then set currentTailLines to 1
if doWrite and (shellCmd is not "" or projectPathArg is not "") and currentTailLines < minTailLinesOnWrite then
set currentTailLines to minTailLinesOnWrite
my logVerbose("Increased tail lines for write operation: " & currentTailLines)
end if
if projectPathArg is not "" and doWrite then
set quotedProjectPath to quoted form of projectPathArg
if shellCmd is not "" then
set shellCmd to "cd " & quotedProjectPath & " && " & shellCmd
else
set shellCmd to "cd " & quotedProjectPath
end if
my logVerbose("Final command: " & shellCmd)
end if
set derivedProjectGroup to ""
if projectPathArg is not "" then
set derivedProjectGroup to my getPathComponent(projectPathArg, -1)
if derivedProjectGroup is "" then set derivedProjectGroup to "DefaultProject"
my logVerbose("Project group: " & derivedProjectGroup)
end if
set allowCreation to false
if doWrite then
set allowCreation to true
else if explicitLinesProvided then
set allowCreation to true
end if
set effectiveTabTitleForLookup to my generateWindowTitle(taskTagName, derivedProjectGroup)
my logVerbose("Tab title: " & effectiveTabTitleForLookup)
set tabInfo to my ensureTabAndWindow(taskTagName, derivedProjectGroup, allowCreation, effectiveTabTitleForLookup)
if tabInfo is missing value then
if not allowCreation then
set errorMsg to "Terminal session \"" & effectiveTabTitleForLookup & "\" not found." & linefeed & ¬
"To create this session, provide a command (even an empty string \"\" if only 'cd'-ing to a project path), " & ¬
"or specify lines to read (e.g., ... \"" & taskTagName & "\" 1)." & linefeed
if projectPathArg is not "" then
set errorMsg to errorMsg & "Project path was specified as: \"" & projectPathArg & "\"." & linefeed
else
set errorMsg to errorMsg & "If this is for a new project, provide the absolute project path as the first argument." & linefeed
end if
return my formatErrorMessage("Session Error", errorMsg & linefeed & my usageText(), "session lookup")
else
return my formatErrorMessage("Creation Error", "Could not find or create Terminal tab for \"" & effectiveTabTitleForLookup & "\". Check permissions/Terminal state.", "tab creation")
end if
end if
set targetTab to targetTab of tabInfo
set parentWindow to parentWindow of tabInfo
set wasNewlyCreated to wasNewlyCreated of tabInfo
set createdInExistingViaFuzzy to createdInExistingWindowViaFuzzy of tabInfo
my logVerbose("Tab info - new: " & wasNewlyCreated & ", fuzzy: " & createdInExistingViaFuzzy)
set bufferText to ""
set commandTimedOut to false
set tabWasBusyOnRead to false
set previousCommandActuallyStopped to true
set attemptMadeToStopPreviousCommand to false
set identifiedBusyProcessName to ""
set theTTYForInfo to ""
if not doWrite and wasNewlyCreated then
if createdInExistingViaFuzzy then
return scriptInfoPrefix & "New tab \"" & effectiveTabTitleForLookup & "\" created in existing project window and ready."
else
return scriptInfoPrefix & "New tab \"" & effectiveTabTitleForLookup & "\" (in new window) created and ready."
end if
end if
tell application id "com.apple.Terminal"
try
set index of parentWindow to 1
set selected tab of parentWindow to targetTab
if wasNewlyCreated and doWrite then
delay 0.4
else
delay 0.1
end if
if doWrite and shellCmd is not "" then
my logVerbose("Executing command: " & shellCmd)
set canProceedWithWrite to true
if busy of targetTab then
if not wasNewlyCreated or createdInExistingViaFuzzy then
set attemptMadeToStopPreviousCommand to true
set previousCommandActuallyStopped to false
try
set theTTYForInfo to my trimWhitespace(tty of targetTab)
end try
set processesBefore to {}
try
set processesBefore to processes of targetTab
end try
set commonShells to {"login", "bash", "zsh", "sh", "tcsh", "ksh", "-bash", "-zsh", "-sh", "-tcsh", "-ksh", "dtterm", "fish"}
set identifiedBusyProcessName to ""
if (count of processesBefore) > 0 then
repeat with i from (count of processesBefore) to 1 by -1
set aProcessName to item i of processesBefore
if aProcessName is not in commonShells then
set identifiedBusyProcessName to aProcessName
exit repeat
end if
end repeat
end if
my logVerbose("Busy process identified: " & identifiedBusyProcessName)
set processToTargetForKill to identifiedBusyProcessName
set killedViaPID to false
if theTTYForInfo is not "" and processToTargetForKill is not "" then
set shortTTY to text 6 thru -1 of theTTYForInfo
set pidsToKillText to ""
try
set psCommand to "ps -t " & shortTTY & " -o pid,comm | awk '$2 == \"" & processToTargetForKill & "\" {print $1}'"
set pidsToKillText to do shell script psCommand
end try
if pidsToKillText is not "" then
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to linefeed
set pidList to text items of pidsToKillText
set AppleScript's text item delimiters to oldDelims
repeat with aPID in pidList
set aPID to my trimWhitespace(aPID)
if aPID is not "" then
try
do shell script "kill -INT " & aPID
delay 0.3
do shell script "kill -0 " & aPID
try
do shell script "kill -KILL " & aPID
delay 0.2
try
do shell script "kill -0 " & aPID
on error
set previousCommandActuallyStopped to true
end try
end try
on error
set previousCommandActuallyStopped to true
end try
end if
if previousCommandActuallyStopped then
set killedViaPID to true
exit repeat
end if
end repeat
end if
end if
if not previousCommandActuallyStopped and busy of targetTab then
activate
delay 0.5
tell application "System Events" to keystroke "c" using control down
delay 0.6
if not (busy of targetTab) then
set previousCommandActuallyStopped to true
if identifiedBusyProcessName is not "" and (identifiedBusyProcessName is in (processes of targetTab)) then
set previousCommandActuallyStopped to false
end if
end if
else if not busy of targetTab then
set previousCommandActuallyStopped to true
end if
if not previousCommandActuallyStopped then
set canProceedWithWrite to false
end if
else if wasNewlyCreated and not createdInExistingViaFuzzy and busy of targetTab then
delay 0.4
if busy of targetTab then
set attemptMadeToStopPreviousCommand to true
set previousCommandActuallyStopped to false
set identifiedBusyProcessName to "extended initialization"
set canProceedWithWrite to false
else
set previousCommandActuallyStopped to true
end if
end if
end if
if canProceedWithWrite then
-- Clear before write to prevent output truncation (only for reused tabs)
if not wasNewlyCreated then
do script "clear" in targetTab
delay 0.1
end if
do script shellCmd in targetTab
set commandStartTime to current date
set commandFinished to false
repeat while ((current date) - commandStartTime) < maxCommandWaitTime
if not (busy of targetTab) then
set commandFinished to true
exit repeat
end if
delay pollIntervalForBusyCheck
end repeat
if not commandFinished then set commandTimedOut to true
if commandFinished then delay 0.2 -- Increased from 0.1 for better output settling
my logVerbose("Command execution completed, timeout: " & commandTimedOut)
end if
else if not doWrite then
if busy of targetTab then
set tabWasBusyOnRead to true
try
set theTTYForInfo to my trimWhitespace(tty of targetTab)
end try
set processesReading to processes of targetTab
set commonShells to {"login", "bash", "zsh", "sh", "tcsh", "ksh", "-bash", "-zsh", "-sh", "-tcsh", "-ksh", "dtterm", "fish"}
set identifiedBusyProcessName to ""
if (count of processesReading) > 0 then
repeat with i from (count of processesReading) to 1 by -1
set aProcessName to item i of processesReading
if aProcessName is not in commonShells then
set identifiedBusyProcessName to aProcessName
exit repeat
end if
end repeat
end if
my logVerbose("Tab busy during read with: " & identifiedBusyProcessName)
end if
end if
set bufferText to history of targetTab
on error errMsg number errNum
set appSpecificErrorOccurred to true
return my formatErrorMessage("Terminal Error", errMsg, "error " & errNum)
end try
end tell
set appendedMessage to ""
set ttyInfoStringForMessage to ""
if theTTYForInfo is not "" then set ttyInfoStringForMessage to " (TTY " & theTTYForInfo & ")"
if attemptMadeToStopPreviousCommand then
set processNameToReport to "process"
if identifiedBusyProcessName is not "" and identifiedBusyProcessName is not "extended initialization" then
set processNameToReport to "'" & identifiedBusyProcessName & "'"
else if identifiedBusyProcessName is "extended initialization" then
set processNameToReport to "tab's extended initialization"
end if
if previousCommandActuallyStopped then
set appendedMessage to linefeed & scriptInfoPrefix & "Previous " & processNameToReport & ttyInfoStringForMessage & " was interrupted. ---"
else
set appendedMessage to linefeed & scriptInfoPrefix & "Attempted to interrupt previous " & processNameToReport & ttyInfoStringForMessage & ", but it may still be running. New command NOT executed. ---"
end if
end if
if commandTimedOut then
set cmdForMsg to originalUserShellCmd
if projectPathArg is not "" and originalUserShellCmd is not "" then set cmdForMsg to originalUserShellCmd & " (in " & projectPathArg & ")"
if projectPathArg is not "" and originalUserShellCmd is "" then set cmdForMsg to "(cd " & projectPathArg & ")"
set appendedMessage to appendedMessage & linefeed & scriptInfoPrefix & "Command '" & cmdForMsg & "' may still be running. Returned after " & maxCommandWaitTime & "s timeout. ---"
else if tabWasBusyOnRead then
set processNameToReportOnRead to "process"
if identifiedBusyProcessName is not "" then set processNameToReportOnRead to "'" & identifiedBusyProcessName & "'"
set busyProcessInfoString to ""
if identifiedBusyProcessName is not "" then set busyProcessInfoString to " with " & processNameToReportOnRead
set appendedMessage to appendedMessage & linefeed & scriptInfoPrefix & "Tab" & ttyInfoStringForMessage & " was busy" & busyProcessInfoString & " during read. Output may be from an ongoing process. ---"
end if
if appendedMessage is not "" then
if bufferText is "" then
set bufferText to my trimWhitespace(appendedMessage)
else
set bufferText to bufferText & appendedMessage
end if
end if
set tailedOutput to my tailBufferAS(bufferText, currentTailLines)
set finalResult to my trimBlankLinesAS(tailedOutput)
if finalResult is "" then
set effectiveOriginalCmdForMsg to originalUserShellCmd
if projectPathArg is not "" and originalUserShellCmd is "" then
set effectiveOriginalCmdForMsg to "(cd " & projectPathArg & ")"
else if projectPathArg is not "" and originalUserShellCmd is not "" then
set effectiveOriginalCmdForMsg to originalUserShellCmd & " (in " & projectPathArg & ")"
end if
set baseMsgInfo to "Session \"" & effectiveTabTitleForLookup & "\", requested " & currentTailLines & " lines."
set specificAppendedInfo to my trimWhitespace(appendedMessage)
set suffixForReturn to ""
if specificAppendedInfo is not "" then set suffixForReturn to linefeed & specificAppendedInfo
if attemptMadeToStopPreviousCommand and not previousCommandActuallyStopped then
return my formatErrorMessage("Process Error", "Previous command/initialization in session \"" & effectiveTabTitleForLookup & "\"" & ttyInfoStringForMessage & " may not have terminated. New command '" & effectiveOriginalCmdForMsg & "' NOT executed." & suffixForReturn, "process termination")
else if commandTimedOut then
return my formatErrorMessage("Timeout Error", "Command '" & effectiveOriginalCmdForMsg & "' timed out after " & maxCommandWaitTime & "s. No other output. " & baseMsgInfo & suffixForReturn, "command timeout")
else if tabWasBusyOnRead then
return my formatErrorMessage("Busy Error", "Tab for session \"" & effectiveTabTitleForLookup & "\" was busy during read. No other output. " & baseMsgInfo & suffixForReturn, "read busy")
else if doWrite and shellCmd is not "" then
return scriptInfoPrefix & "Command '" & effectiveOriginalCmdForMsg & "' executed in session \"" & effectiveTabTitleForLookup & "\". No output captured."
else
return scriptInfoPrefix & "No meaningful content found in session \"" & effectiveTabTitleForLookup & "\"."
end if
end if
my logVerbose("Returning " & (length of finalResult) & " characters of output")
return finalResult
on error generalErrorMsg number generalErrorNum
if appSpecificErrorOccurred then error generalErrorMsg number generalErrorNum
return my formatErrorMessage("Execution Error", generalErrorMsg, "error " & generalErrorNum)
end try
end run
--#endregion Main Script Logic (on run)
--#region Helper Functions
on ensureTabAndWindow(taskTagName as text, projectGroupName as text, allowCreate as boolean, desiredFullTitle as text)
set wasActuallyCreated to false
set createdInExistingViaFuzzy to false
tell application id "com.apple.Terminal"
try
repeat with w in windows
repeat with tb in tabs of w
try
if custom title of tb is desiredFullTitle then
set selected tab of w to tb
return {targetTab:tb, parentWindow:w, wasNewlyCreated:false, createdInExistingWindowViaFuzzy:false}
end if
end try
end repeat
end repeat
end try
if allowCreate and enableFuzzyTagGrouping and projectGroupName is not "" then
set projectGroupSearchPatternForWindowName to tabTitlePrefix & projectIdentifierInTitle & projectGroupName
try
repeat with w in windows
try
-- Look for any window that contains our project name
if name of w contains projectGroupSearchPatternForWindowName or name of w contains (projectIdentifierInTitle & projectGroupName) then
if not frontmost then activate
delay 0.2
set newTabInGroup to do script "" in w
delay 0.3
set custom title of newTabInGroup to desiredFullTitle
delay 0.2
set selected tab of w to newTabInGroup
return {targetTab:newTabInGroup, parentWindow:w, wasNewlyCreated:true, createdInExistingWindowViaFuzzy:true}
end if
end try
end repeat
end try
end if
-- Enhanced fallback: if no project-specific window found, try to use any existing Terminator window
if allowCreate and enableFuzzyTagGrouping then
try
repeat with w in windows
try
if name of w contains tabTitlePrefix then
-- Found an existing Terminator window, use it for grouping
if not frontmost then activate
delay 0.2
set newTabInGroup to do script "" in w
delay 0.3
set custom title of newTabInGroup to desiredFullTitle
delay 0.2
set selected tab of w to newTabInGroup
return {targetTab:newTabInGroup, parentWindow:w, wasNewlyCreated:true, createdInExistingWindowViaFuzzy:true}
end if
end try
end repeat
end try
end if
if allowCreate then
try
if not frontmost then activate
delay 0.3
set newTabInNewWindow to do script ""
set wasActuallyCreated to true
delay 0.4
set custom title of newTabInNewWindow to desiredFullTitle
delay 0.2
set parentWinOfNew to missing value
try
set parentWinOfNew to window of newTabInNewWindow
on error
if (count of windows) > 0 then set parentWinOfNew to front window
end try
if parentWinOfNew is not missing value then
if custom title of newTabInNewWindow is desiredFullTitle then
set selected tab of parentWinOfNew to newTabInNewWindow
return {targetTab:newTabInNewWindow, parentWindow:parentWinOfNew, wasNewlyCreated:wasActuallyCreated, createdInExistingWindowViaFuzzy:false}
end if
end if
repeat with w_final_scan in windows
repeat with tb_final_scan in tabs of w_final_scan
try
if custom title of tb_final_scan is desiredFullTitle then
set selected tab of w_final_scan to tb_final_scan
return {targetTab:tb_final_scan, parentWindow:w_final_scan, wasNewlyCreated:wasActuallyCreated, createdInExistingWindowViaFuzzy:false}
end if
end try
end repeat
end repeat
return missing value
on error
return missing value
end try
else
return missing value
end if
end tell
end ensureTabAndWindow
on tailBufferAS(txt, n)
set AppleScript's text item delimiters to linefeed
set lst to text items of txt
if (count lst) = 0 then return ""
set startN to (count lst) - (n - 1)
if startN < 1 then set startN to 1
set slice to items startN thru -1 of lst
set outText to slice as text
set AppleScript's text item delimiters to ""
return outText
end tailBufferAS
on lineIsEffectivelyEmptyAS(aLine)
if aLine is "" then return true
set trimmedLine to my trimWhitespace(aLine)
return (trimmedLine is "")
end lineIsEffectivelyEmptyAS
on trimBlankLinesAS(txt)
if txt is "" then return ""
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to {linefeed}
set originalLines to text items of txt
set linesToProcess to {}
repeat with aLineRef in originalLines
set aLine to contents of aLineRef
if my lineIsEffectivelyEmptyAS(aLine) then
set end of linesToProcess to ""
else
set end of linesToProcess to aLine
end if
end repeat
set firstContentLine to 1
repeat while firstContentLine ≤ (count linesToProcess) and (item firstContentLine of linesToProcess is "")
set firstContentLine to firstContentLine + 1
end repeat
set lastContentLine to count linesToProcess
repeat while lastContentLine ≥ firstContentLine and (item lastContentLine of linesToProcess is "")
set lastContentLine to lastContentLine - 1
end repeat
if firstContentLine > lastContentLine then
set AppleScript's text item delimiters to oldDelims
return ""
end if
set resultLines to items firstContentLine thru lastContentLine of linesToProcess
set AppleScript's text item delimiters to linefeed
set trimmedTxt to resultLines as text
set AppleScript's text item delimiters to oldDelims
return trimmedTxt
end trimBlankLinesAS
on trimWhitespace(theText)
set whitespaceChars to {" ", tab}
set newText to theText
repeat while (newText is not "") and (character 1 of newText is in whitespaceChars)
if (length of newText) > 1 then
set newText to text 2 thru -1 of newText
else
set newText to ""
end if
end repeat
repeat while (newText is not "") and (character -1 of newText is in whitespaceChars)
if (length of newText) > 1 then
set newText to text 1 thru -2 of newText
else
set newText to ""
end if
end repeat
return newText
end trimWhitespace
on isInteger(v)
try
v as integer
return true
on error
return false
end try
end isInteger
on tagOK(t)
try
do shell script "/bin/echo " & quoted form of t & " | /usr/bin/grep -E -q '^[A-Za-z0-9_-]+$'"
return true
on error
return false
end try
end tagOK
on joinList(theList, theDelimiter)
set oldDelims to AppleScript's text item delimiters
set AppleScript's text item delimiters to theDelimiter
set theText to theList as text
set AppleScript's text item delimiters to oldDelims
return theText
end joinList
on usageText()
set LF to linefeed
set scriptName to "terminator.scpt"
set exampleProject to "/Users/name/Projects/FancyApp"
set exampleProjectNameForTitle to my getPathComponent(exampleProject, -1)
if exampleProjectNameForTitle is "" then set exampleProjectNameForTitle to "DefaultProject"
set exampleTaskTag to "build_frontend"
set exampleFullCommand to "npm run build"
set generatedExampleTitle to my generateWindowTitle(exampleTaskTag, exampleProjectNameForTitle)
set outText to scriptName & " - v0.6.0 Enhanced \"T-1000\" AppleScript Terminal helper" & LF & LF
set outText to outText & "Enhancements: Smart session reuse, enhanced error reporting, verbose logging (optional)" & LF & LF
set outText to outText & "Manages dedicated, tagged Terminal sessions, grouped by project path." & LF & LF
set outText to outText & "Core Concept:" & LF
set outText to outText & " 1. For a NEW project, provide the absolute project path FIRST, then task tag, then command:" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"" & exampleTaskTag & "\" \"" & exampleFullCommand & "\"" & LF
set outText to outText & " The script will 'cd' into the project path and run the command." & LF
set outText to outText & " The tab will be titled like: \"" & generatedExampleTitle & "\"" & LF
set outText to outText & " 2. For SUBSEQUENT commands for THE SAME PROJECT, use the project path and task tag:" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"" & exampleTaskTag & "\" \"another_command\"" & LF
set outText to outText & " 3. To simply READ from an existing session (path & tag must identify an existing session):" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"" & exampleTaskTag & "\"" & LF
set outText to outText & " A READ operation on a non-existent tag (without path/command to create) will error." & LF & LF
set outText to outText & "Title Format: \"" & tabTitlePrefix & projectIdentifierInTitle & "<ProjectName>" & taskIdentifierInTitle & "<TaskTag>\"" & LF
set outText to outText & "Or if no project path provided: \"" & tabTitlePrefix & "<TaskTag>\"" & LF & LF
set outText to outText & "Enhanced Features:" & LF
set outText to outText & " • Smart session reuse for same project paths" & LF
set outText to outText & " • Enhanced error reporting with context information" & LF
set outText to outText & " • Optional verbose logging for debugging" & LF
set outText to outText & " • No automatic clearing to prevent interrupting builds" & LF
set outText to outText & " • 100-line default output for better build log visibility" & LF
set outText to outText & " • Automatically 'cd's into project path if provided with a command." & LF
set outText to outText & " • Groups new task tabs into existing project windows if fuzzy grouping enabled." & LF
set outText to outText & " • Interrupts busy processes in reused tabs." & LF & LF
set outText to outText & "Usage Examples:" & LF
set outText to outText & " # Start new project session, cd, run command, get 100 lines:" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"frontend_build\" \"npm run build\" 100" & LF
set outText to outText & " # Create/use 'backend_tests' task tab in the 'FancyApp' project window:" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"backend_tests\" \"pytest\"" & LF
set outText to outText & " # Prepare/create a new session by just cd'ing into project path (empty command):" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"dev_shell\" \"\" 1" & LF
set outText to outText & " # Read from an existing session:" & LF
set outText to outText & " osascript " & scriptName & " \"" & exampleProject & "\" \"frontend_build\" 50" & LF & LF
set outText to outText & "Parameters:" & LF
set outText to outText & " [\"/absolute/project/path\"]: (Optional First Arg) Base path for project. Enables 'cd' and grouping." & LF
set outText to outText & " \"<task_tag_name>\": Required. Specific task name for the tab (e.g., 'build', 'tests')." & LF
set outText to outText & " [\"<shell_command_parts...>\"]: (Optional) Command. If path provided, 'cd path &&' is prepended." & LF
set outText to outText & " Use \"\" for no command (will just 'cd' if path given)." & LF
set outText to outText & " [[lines_to_read]]: (Optional Last Arg) Number of history lines. Default: " & defaultTailLines & "." & LF & LF
set outText to outText & "Notes:" & LF
set outText to outText & " • Provide project path on first use for a project for best window grouping and auto 'cd'." & LF
set outText to outText & " • Ensure Automation permissions for Terminal.app & System Events.app." & LF
set outText to outText & " • Works within Terminal.app's AppleScript limitations for reliable operation." & LF
return outText
end usageText
--#endregion Helper Functions

2
.envrc
View File

@ -1,2 +0,0 @@
PATH_add ./scripts
PATH_add ./node_modules/.bin

48
.eslintrc.json Normal file
View File

@ -0,0 +1,48 @@
{
"root": true,
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint"],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/recommended"
],
"parserOptions": {
"ecmaVersion": 2022,
"sourceType": "module",
"project": "./tsconfig.json"
},
"env": {
"node": true,
"es2022": true
},
"ignorePatterns": [
"dist/",
"node_modules/",
"coverage/",
"*.js",
"scripts/prepare-release.js",
"tests/**/*.ts"
],
"rules": {
"@typescript-eslint/no-explicit-any": "warn",
"@typescript-eslint/no-unused-vars": ["error", {
"argsIgnorePattern": "^_",
"varsIgnorePattern": "^_",
"caughtErrorsIgnorePattern": "^_"
}],
"@typescript-eslint/explicit-module-boundary-types": "off",
"@typescript-eslint/no-non-null-assertion": "warn",
"no-console": "error",
"prefer-const": "error",
"no-var": "error",
"eqeqeq": ["error", "always"],
"curly": ["error", "all"],
"brace-style": ["error", "1tbs"],
"quotes": ["error", "double", { "avoidEscape": true }],
"semi": ["error", "always"],
"comma-dangle": ["error", "always-multiline"],
"no-trailing-spaces": "error",
"indent": ["error", 2, { "SwitchCase": 1 }],
"max-len": ["warn", { "code": 120, "ignoreUrls": true, "ignoreStrings": true }]
}
}

9
.github/CODEOWNERS vendored
View File

@ -1,9 +0,0 @@
# Protect ownership and automation rules.
/.github/CODEOWNERS @openclaw/openclaw-secops
/.github/workflows/ @openclaw/openclaw-secops
/package.json @openclaw/openclaw-secops
/pnpm-lock.yaml @openclaw/openclaw-secops
/Package.swift @openclaw/openclaw-secops
/.github/actionlint.yaml @openclaw/openclaw-secops
/.agents/skills/ @openclaw/openclaw-secops
/.crabbox.yaml @openclaw/openclaw-secops

View File

@ -1,5 +0,0 @@
self-hosted-runner:
labels:
- crabbox
- openclaw
- peekaboo

339
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,339 @@
name: CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
# Test on macOS with Swift binary
test-macos:
runs-on: macos-15
strategy:
matrix:
node-version: [20.x, 22.x]
env:
DEVELOPER_DIR: /Applications/Xcode_16.3.app/Contents/Developer
steps:
- uses: actions/checkout@v4
- name: Set up Xcode
run: |
sudo xcode-select -s $DEVELOPER_DIR
xcodebuild -version
swift --version
- name: Build Swift CLI for tests
run: |
cd peekaboo-cli
swift build -c release
# Copy the binary to the expected location
cp .build/release/peekaboo ../peekaboo
cd ..
# Make it executable
chmod +x peekaboo
# Verify it exists
ls -la peekaboo
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build TypeScript
run: npm run build
- name: Run linter
run: npm run lint --if-present
- name: Run tests with coverage
run: npm run test:coverage
env:
CI: true
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
if: matrix.node-version == '20.x'
with:
file: ./coverage/lcov.info
flags: unittests-macos
name: codecov-macos
fail_ci_if_error: false
# Test on Linux with Rust binary
test-linux:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [20.x, 22.x]
steps:
- uses: actions/checkout@v4
- name: Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y \
libx11-dev \
libxrandr-dev \
libxinerama-dev \
libxcursor-dev \
libxi-dev \
libxext-dev \
libxfixes-dev \
libxss-dev \
libgl1-mesa-dev \
pkg-config
- name: Set up Rust
uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable
- name: Cache Rust dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
peekaboo-native/target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Build Rust CLI for tests
run: |
cd peekaboo-native
cargo build --release
# Verify the binary exists
ls -la target/release/peekaboo
# Test basic functionality
./target/release/peekaboo --version
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build TypeScript
run: npm run build
- name: Run linter
run: npm run lint --if-present
- name: Run tests with coverage
run: npm run test:coverage
env:
CI: true
DISPLAY: :99
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
if: matrix.node-version == '20.x'
with:
file: ./coverage/lcov.info
flags: unittests-linux
name: codecov-linux
fail_ci_if_error: false
# Test on Windows with Rust binary
test-windows:
runs-on: windows-latest
strategy:
matrix:
node-version: [20.x, 22.x]
steps:
- uses: actions/checkout@v4
- name: Set up Rust
uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable
- name: Cache Rust dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
peekaboo-native/target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Build Rust CLI for tests
run: |
cd peekaboo-native
cargo build --release
# Verify the binary exists
ls target/release/peekaboo.exe
# Test basic functionality
./target/release/peekaboo.exe --version
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build TypeScript
run: npm run build
- name: Run linter
run: npm run lint --if-present
- name: Run tests with coverage
run: npm run test:coverage
env:
CI: true
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v4
if: matrix.node-version == '20.x'
with:
file: ./coverage/lcov.info
flags: unittests-windows
name: codecov-windows
fail_ci_if_error: false
# Build Swift CLI separately for validation
build-swift:
runs-on: macos-15
timeout-minutes: 30
env:
DEVELOPER_DIR: /Applications/Xcode_16.3.app/Contents/Developer
steps:
- uses: actions/checkout@v4
- name: Set up Xcode
run: |
sudo xcode-select -s $DEVELOPER_DIR
xcodebuild -version
swift --version
- name: Build Swift CLI
run: |
cd peekaboo-cli
swift build -c release
- name: Run Swift tests
timeout-minutes: 10
run: |
cd peekaboo-cli
swift test --parallel --skip "LocalIntegrationTests|ScreenshotValidationTests|ApplicationFinderTests|WindowManagerTests"
env:
CI: true
# Build Rust CLI on all platforms for validation
build-rust:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- name: Install Linux dependencies
if: matrix.os == 'ubuntu-latest'
run: |
sudo apt-get update
sudo apt-get install -y \
libx11-dev \
libxrandr-dev \
libxinerama-dev \
libxcursor-dev \
libxi-dev \
libxext-dev \
libxfixes-dev \
libxss-dev \
libgl1-mesa-dev \
pkg-config
- name: Set up Rust
uses: dtolnay/rust-toolchain@stable
with:
toolchain: stable
- name: Cache Rust dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/registry
~/.cargo/git
peekaboo-native/target
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-
- name: Build Rust CLI
run: |
cd peekaboo-native
cargo build --release
- name: Test Rust CLI basic functionality
run: |
cd peekaboo-native
if [ "${{ matrix.os }}" = "windows-latest" ]; then
./target/release/peekaboo.exe --version
else
./target/release/peekaboo --version
fi
shell: bash
- name: Run Rust tests
run: |
cd peekaboo-native
cargo test
env:
CI: true
# Integration test to verify cross-platform compatibility
integration-test:
needs: [test-macos, test-linux, test-windows]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '20.x'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build TypeScript
run: npm run build
- name: Run integration tests
run: |
echo "✅ All platform tests passed!"
echo "✅ macOS (Swift) support verified"
echo "✅ Linux (Rust) support verified"
echo "✅ Windows (Rust) support verified"
echo "🎉 Multi-platform CI setup complete!"

View File

@ -1,71 +0,0 @@
name: Commander Multiplatform
on:
push:
paths:
- 'Commander/**'
- '.github/workflows/commander-multiplatform.yml'
pull_request:
paths:
- 'Commander/**'
- '.github/workflows/commander-multiplatform.yml'
workflow_dispatch:
jobs:
macos-host:
runs-on: macos-15
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
- name: Swift version
working-directory: Commander
run: swift --version
- name: Test (macOS)
working-directory: Commander
run: swift test
apple-simulators:
runs-on: macos-15
needs: macos-host
strategy:
matrix:
include:
- platform: iOS
sdk: iphonesimulator
triple: arm64-apple-ios17.0-simulator
- platform: tvOS
sdk: appletvsimulator
triple: arm64-apple-tvos17.0-simulator
- platform: watchOS
sdk: watchsimulator
triple: arm64-apple-watchos10.0-simulator
defaults:
run:
working-directory: Commander
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
- name: Build for ${{ matrix.platform }}
run: |
set -euo pipefail
SDK_PATH=$(xcrun --sdk ${{ matrix.sdk }} --show-sdk-path)
swift build \
--build-tests \
--triple "${{ matrix.triple }}" \
--sdk "$SDK_PATH"
linux:
runs-on: ubuntu-24.04
needs: macos-host
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
- uses: SwiftyLab/setup-swift@v1
with:
swift-version: '6.2.1'
- name: Test (Linux)
working-directory: Commander
run: swift test

View File

@ -1,126 +0,0 @@
name: Crabbox Hydrate
on:
workflow_dispatch:
inputs:
crabbox_id:
description: "Crabbox lease ID"
required: true
type: string
ref:
description: "Git ref to hydrate"
required: false
type: string
crabbox_runner_label:
description: "Dynamic Crabbox runner label"
required: true
type: string
crabbox_job:
description: "Hydration job identifier expected by Crabbox"
required: false
default: "hydrate"
type: string
crabbox_keep_alive_minutes:
description: "Minutes to keep the hydrated job alive"
required: false
default: "90"
type: string
permissions:
contents: read
env:
NODE_VERSION: "24"
PNPM_VERSION: "11.1.2"
jobs:
hydrate:
name: hydrate
runs-on: [self-hosted, crabbox, openclaw, peekaboo, "${{ inputs.crabbox_runner_label }}"]
timeout-minutes: 120
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.ref || github.ref }}
- uses: pnpm/action-setup@v6.0.8
with:
version: ${{ env.PNPM_VERSION }}
- uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
- name: Prepare pnpm and Swift workspace
shell: bash
run: |
set -euo pipefail
git fetch --no-tags --depth=50 origin "+refs/heads/main:refs/remotes/origin/main"
pnpm install --frozen-lockfile
node --version
pnpm --version
swift --version
- name: Mark Crabbox ready
shell: bash
env:
CRABBOX_ID: ${{ inputs.crabbox_id }}
CRABBOX_JOB: ${{ inputs.crabbox_job }}
run: |
set -euo pipefail
job="${CRABBOX_JOB}"
if [ -z "$job" ]; then job=hydrate; fi
case "$CRABBOX_ID" in
''|*[!A-Za-z0-9._-]*)
echo "Invalid crabbox_id" >&2
exit 2
;;
esac
mkdir -p "$HOME/.crabbox/actions"
state="$HOME/.crabbox/actions/${CRABBOX_ID}.env"
env_file="$HOME/.crabbox/actions/${CRABBOX_ID}.env.sh"
{
for key in CI GITHUB_ACTIONS GITHUB_WORKSPACE GITHUB_REPOSITORY GITHUB_RUN_ID GITHUB_RUN_NUMBER GITHUB_RUN_ATTEMPT GITHUB_REF GITHUB_REF_NAME GITHUB_SHA GITHUB_EVENT_NAME GITHUB_ACTOR RUNNER_OS RUNNER_ARCH RUNNER_TEMP RUNNER_TOOL_CACHE PATH; do
value="${!key-}"
if [ -n "$value" ]; then
printf 'export %s=%q\n' "$key" "$value"
fi
done
} > "${env_file}.tmp"
mv "${env_file}.tmp" "$env_file"
tmp="${state}.tmp"
{
echo "WORKSPACE=${GITHUB_WORKSPACE}"
echo "RUN_ID=${GITHUB_RUN_ID}"
echo "JOB=${job}"
echo "ENV_FILE=${env_file}"
echo "READY_AT=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} > "$tmp"
mv "$tmp" "$state"
- name: Keep Crabbox job alive
shell: bash
env:
CRABBOX_ID: ${{ inputs.crabbox_id }}
CRABBOX_KEEP_ALIVE_MINUTES: ${{ inputs.crabbox_keep_alive_minutes }}
run: |
set -euo pipefail
case "$CRABBOX_ID" in
''|*[!A-Za-z0-9._-]*)
echo "Invalid crabbox_id" >&2
exit 2
;;
esac
minutes="${CRABBOX_KEEP_ALIVE_MINUTES}"
case "$minutes" in
''|*[!0-9]*) minutes=90 ;;
esac
stop="$HOME/.crabbox/actions/${CRABBOX_ID}.stop"
deadline=$(( $(date +%s) + minutes * 60 ))
while [ "$(date +%s)" -lt "$deadline" ]; do
if [ -f "$stop" ]; then
exit 0
fi
sleep 15
done

View File

@ -1,419 +0,0 @@
name: macOS CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
concurrency:
group: macos-ci-${{ github.ref }}
cancel-in-progress: true
jobs:
peekaboo-core:
name: PeekabooCore build & tests
runs-on: macos-15
env:
PEEKABOO_INCLUDE_AUTOMATION_TESTS: "false"
RUN_AUTOMATION_TESTS: "false"
RUN_LOCAL_TESTS: "false"
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
fetch-depth: 1
- name: Install Bun runtime
uses: oven-sh/setup-bun@v2
with:
bun-version: "latest"
- name: Docs lint
run: node scripts/docs-lint.mjs
- name: Select Xcode 26.2 (if present) or fallback to default
run: |
set -euo pipefail
for candidate in /Applications/Xcode_26.2.app /Applications/Xcode_26.1.app /Applications/Xcode_26.0.app /Applications/Xcode_16.4.app /Applications/Xcode_16.3.app /Applications/Xcode.app; do
if [[ -d "$candidate" ]]; then
sudo xcode-select -s "$candidate"
echo "DEVELOPER_DIR=${candidate}/Contents/Developer" >> "$GITHUB_ENV"
break
fi
done
/usr/bin/xcodebuild -version
- name: Prepare Swift Argument Parser fork
run: |
sudo mkdir -p /Users/steipete/Projects
sudo chown $USER /Users/steipete
sudo mkdir -p /Users/steipete/Projects
sudo chown $USER /Users/steipete/Projects
if [ -d /Users/steipete/Projects/swift-argument-parser ]; then
cd /Users/steipete/Projects/swift-argument-parser
git fetch origin approachable-concurrency
git checkout approachable-concurrency
git pull --ff-only origin approachable-concurrency
else
git clone --branch approachable-concurrency --depth 1 https://github.com/steipete/swift-argument-parser.git /Users/steipete/Projects/swift-argument-parser
fi
- name: Compute SwiftPM cache key (PeekabooCore)
id: cache-key-core
env:
CACHE_PREFIX: ${{ runner.os }}-spm-core-
run: |
set -euo pipefail
if [ -f Core/PeekabooCore/Package.resolved ]; then
HASH=$(shasum Core/PeekabooCore/Package.resolved | awk '{print $1}')
else
echo "Package.resolved missing, falling back to commit SHA"
HASH=${GITHUB_SHA}
fi
echo "key=${CACHE_PREFIX}${HASH}" >> "$GITHUB_OUTPUT"
- name: Cache SwiftPM (PeekabooCore)
uses: actions/cache@v5
with:
path: |
~/.swiftpm
~/.cache/org.swift.swiftpm
Core/PeekabooCore/.build
key: ${{ steps.cache-key-core.outputs.key }}
restore-keys: |
${{ runner.os }}-spm-core-
- name: Clean SwiftPM trait state (PeekabooCore)
run: |
set -euo pipefail
# SwiftPM caches evaluated manifests in an sqlite DB. We've seen this get stale across
# `swift-configuration` trait renames (e.g. `JSONSupport` -> `JSON`) and break resolution.
for root in "$HOME/Library/Caches/org.swift.swiftpm" "$HOME/.cache/org.swift.swiftpm"; do
if [ -d "$root" ]; then
find "$root" -type f -name "manifest.db*" -print -delete || true
find "$root" -type f -name "manifests.db*" -print -delete || true
find "$root" -type f -name "package-collection.db*" -print -delete || true
fi
done
# SwiftPM traits can be persisted into `.swiftpm/configuration/traits.json` (and can get stuck in caches).
# When upstream packages rename/remove traits, stale state can break builds.
find ~/.swiftpm -type f -name traits.json -print -delete || true
if [ -d ~/.cache/org.swift.swiftpm ]; then
find ~/.cache/org.swift.swiftpm -type d -name .swiftpm -prune -print -exec rm -rf {} + || true
fi
if [ -d Core/PeekabooCore/.swiftpm ]; then
find Core/PeekabooCore/.swiftpm -type f -name traits.json -print -delete || true
fi
if [ -d Core/PeekabooCore/.build/checkouts ]; then
find Core/PeekabooCore/.build/checkouts -type d -name .swiftpm -prune -print -exec rm -rf {} + || true
fi
- name: Show Xcode version
run: xcodebuild -version
- name: Show Swift toolchain version
run: swift --version
- name: Build PeekabooCore
working-directory: Core/PeekabooCore
run: |
swift build --configuration debug
- name: Run focused Swift tests
working-directory: Core/PeekabooCore
run: |
swift test --no-parallel --filter ScreenCaptureServiceFlowTests
peekaboo-cli:
name: Peekaboo CLI build & tests
runs-on: macos-15
needs: peekaboo-core
env:
PEEKABOO_INCLUDE_AUTOMATION_TESTS: "false"
PEEKABOO_SKIP_AUTOMATION: "1"
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
fetch-depth: 1
- name: Select Xcode 26.2 (if present) or fallback to default
run: |
set -euo pipefail
for candidate in /Applications/Xcode_26.2.app /Applications/Xcode_26.1.app /Applications/Xcode_26.0.app /Applications/Xcode_16.4.app /Applications/Xcode_16.3.app /Applications/Xcode.app; do
if [[ -d "$candidate" ]]; then
sudo xcode-select -s "$candidate"
echo "DEVELOPER_DIR=${candidate}/Contents/Developer" >> "$GITHUB_ENV"
break
fi
done
/usr/bin/xcodebuild -version
- name: Prepare Swift Argument Parser fork
run: |
sudo mkdir -p /Users/steipete/Projects
sudo chown $USER /Users/steipete
sudo mkdir -p /Users/steipete/Projects
sudo chown $USER /Users/steipete/Projects
if [ -d /Users/steipete/Projects/swift-argument-parser ]; then
cd /Users/steipete/Projects/swift-argument-parser
git fetch origin approachable-concurrency
git checkout approachable-concurrency
git pull --ff-only origin approachable-concurrency
else
git clone --branch approachable-concurrency --depth 1 https://github.com/steipete/swift-argument-parser.git /Users/steipete/Projects/swift-argument-parser
fi
- name: Compute SwiftPM cache key (CLI)
id: cache-key-cli
env:
CACHE_PREFIX: ${{ runner.os }}-spm-cli-
run: |
set -euo pipefail
if [ -f Apps/CLI/Package.resolved ]; then
HASH=$(shasum Apps/CLI/Package.resolved | awk '{print $1}')
else
echo "Package.resolved missing, falling back to commit SHA"
HASH=${GITHUB_SHA}
fi
echo "key=${CACHE_PREFIX}${HASH}" >> "$GITHUB_OUTPUT"
- name: Cache SwiftPM (CLI)
uses: actions/cache@v5
with:
path: |
~/.swiftpm
~/.cache/org.swift.swiftpm
Apps/CLI/.build
key: ${{ steps.cache-key-cli.outputs.key }}
restore-keys: |
${{ runner.os }}-spm-cli-
- name: Clean SwiftPM trait state (CLI)
run: |
set -euo pipefail
# SwiftPM caches evaluated manifests in an sqlite DB. We've seen this get stale across
# `swift-configuration` trait renames (e.g. `JSONSupport` -> `JSON`) and break resolution.
for root in "$HOME/Library/Caches/org.swift.swiftpm" "$HOME/.cache/org.swift.swiftpm"; do
if [ -d "$root" ]; then
find "$root" -type f -name "manifest.db*" -print -delete || true
find "$root" -type f -name "manifests.db*" -print -delete || true
find "$root" -type f -name "package-collection.db*" -print -delete || true
fi
done
# SwiftPM traits can be persisted into `.swiftpm/configuration/traits.json` (and can get stuck in caches).
# When upstream packages rename/remove traits, stale state can break builds.
find ~/.swiftpm -type f -name traits.json -print -delete || true
if [ -d ~/.cache/org.swift.swiftpm ]; then
find ~/.cache/org.swift.swiftpm -type d -name .swiftpm -prune -print -exec rm -rf {} + || true
fi
if [ -d Apps/CLI/.swiftpm ]; then
find Apps/CLI/.swiftpm -type f -name traits.json -print -delete || true
fi
if [ -d Apps/CLI/.build/checkouts ]; then
find Apps/CLI/.build/checkouts -type d -name .swiftpm -prune -print -exec rm -rf {} + || true
fi
# Avoid caching any package-local state that might remember old trait selections.
rm -rf Apps/CLI/.build || true
- name: Show Swift toolchain version
run: swift --version
- name: Show Xcode version
run: xcodebuild -version
- name: Build CLI target
working-directory: Apps/CLI
run: |
swift build --configuration debug
echo "PEEKABOO_CLI_BINARY=$(swift build --configuration debug --show-bin-path)/peekaboo" >> "$GITHUB_ENV"
- name: Run CLI unit tests (skip automation)
working-directory: Apps/CLI
run: |
swift test --no-parallel -Xswiftc -DPEEKABOO_SKIP_AUTOMATION
tachikoma:
name: Tachikoma build & tests
runs-on: macos-15
needs: peekaboo-cli
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
fetch-depth: 1
- name: Select Xcode 26.2 (if present) or fallback to default
run: |
set -euo pipefail
for candidate in /Applications/Xcode_26.2.app /Applications/Xcode_26.1.app /Applications/Xcode_26.0.app /Applications/Xcode_16.4.app /Applications/Xcode_16.3.app /Applications/Xcode.app; do
if [[ -d "$candidate" ]]; then
sudo xcode-select -s "$candidate"
echo "DEVELOPER_DIR=${candidate}/Contents/Developer" >> "$GITHUB_ENV"
break
fi
done
/usr/bin/xcodebuild -version
- name: Remove phantom submodule metadata
run: |
rm -f .gitmodules
git config --local --remove-section submodule.Tachikoma || true
- name: Prepare Swift Argument Parser fork
run: |
sudo mkdir -p /Users/steipete/Projects
sudo chown $USER /Users/steipete
sudo mkdir -p /Users/steipete/Projects
sudo chown $USER /Users/steipete/Projects
if [ -d /Users/steipete/Projects/swift-argument-parser ]; then
cd /Users/steipete/Projects/swift-argument-parser
git fetch origin approachable-concurrency
git checkout approachable-concurrency
git pull --ff-only origin approachable-concurrency
else
git clone --branch approachable-concurrency --depth 1 https://github.com/steipete/swift-argument-parser.git /Users/steipete/Projects/swift-argument-parser
fi
- name: Compute SwiftPM cache key (Tachikoma)
id: cache-key-tachikoma
env:
CACHE_PREFIX: ${{ runner.os }}-spm-tachikoma-
run: |
set -euo pipefail
if [ -f Tachikoma/Package.resolved ]; then
HASH=$(shasum Tachikoma/Package.resolved | awk '{print $1}')
else
echo "Package.resolved missing, falling back to commit SHA"
HASH=${GITHUB_SHA}
fi
echo "key=${CACHE_PREFIX}${HASH}" >> "$GITHUB_OUTPUT"
- name: Cache SwiftPM (Tachikoma)
uses: actions/cache@v5
with:
path: |
~/.swiftpm
~/.cache/org.swift.swiftpm
Tachikoma/.build
key: ${{ steps.cache-key-tachikoma.outputs.key }}
restore-keys: |
${{ runner.os }}-spm-tachikoma-
- name: Clean SwiftPM trait state (Tachikoma)
run: |
set -euo pipefail
# SwiftPM caches evaluated manifests in an sqlite DB. We've seen this get stale across
# `swift-configuration` trait renames (e.g. `JSONSupport` -> `JSON`) and break resolution.
for root in "$HOME/Library/Caches/org.swift.swiftpm" "$HOME/.cache/org.swift.swiftpm"; do
if [ -d "$root" ]; then
find "$root" -type f -name "manifest.db*" -print -delete || true
find "$root" -type f -name "manifests.db*" -print -delete || true
find "$root" -type f -name "package-collection.db*" -print -delete || true
fi
done
# SwiftPM traits can be persisted into `.swiftpm/configuration/traits.json` (and can get stuck in caches).
# When upstream packages rename/remove traits, stale state can break builds.
find ~/.swiftpm -type f -name traits.json -print -delete || true
if [ -d ~/.cache/org.swift.swiftpm ]; then
find ~/.cache/org.swift.swiftpm -type d -name .swiftpm -prune -print -exec rm -rf {} + || true
fi
if [ -d Tachikoma/.swiftpm ]; then
find Tachikoma/.swiftpm -type f -name traits.json -print -delete || true
fi
if [ -d Tachikoma/.build/checkouts ]; then
find Tachikoma/.build/checkouts -type d -name .swiftpm -prune -print -exec rm -rf {} + || true
fi
- name: Show Swift toolchain version
run: swift --version
- name: Show Xcode version
run: xcodebuild -version
- name: Build Tachikoma
working-directory: Tachikoma
run: |
swift build --configuration debug
- name: Run Tachikoma unit tests
working-directory: Tachikoma
run: |
swift test --no-parallel --filter unit
mac-apps:
name: Build macOS apps (Peekaboo + Inspector)
runs-on: macos-15
needs: [peekaboo-cli, tachikoma]
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
fetch-depth: 1
- name: Select Xcode 26.2 (if present) or fallback to default
run: |
set -euo pipefail
for candidate in /Applications/Xcode_26.2.app /Applications/Xcode_26.1.app /Applications/Xcode_26.0.app /Applications/Xcode_16.4.app /Applications/Xcode_16.3.app /Applications/Xcode.app; do
if [[ -d "$candidate" ]]; then
sudo xcode-select -s "$candidate"
echo "DEVELOPER_DIR=${candidate}/Contents/Developer" >> "$GITHUB_ENV"
break
fi
done
/usr/bin/xcodebuild -version
- name: Build Peekaboo app (Xcode)
working-directory: Apps
run: |
/usr/bin/env \
-u DYLD_LIBRARY_PATH \
-u DYLD_FRAMEWORK_PATH \
-u DYLD_FALLBACK_FRAMEWORK_PATH \
-u DYLD_ROOT_PATH \
-u DYLD_INSERT_LIBRARIES \
-u DYLD_IMAGE_SUFFIX \
-u DYLD_VERSIONED_LIBRARY_PATH \
-u DYLD_VERSIONED_FRAMEWORK_PATH \
xcodebuild -workspace Peekaboo.xcworkspace \
-scheme Peekaboo \
-configuration Debug \
-sdk macosx \
CODE_SIGNING_ALLOWED=NO \
-derivedDataPath /tmp/DerivedData-Peekaboo
- name: Build Inspector app (Xcode)
working-directory: Apps/PeekabooInspector
run: |
/usr/bin/env \
-u DYLD_LIBRARY_PATH \
-u DYLD_FRAMEWORK_PATH \
-u DYLD_FALLBACK_FRAMEWORK_PATH \
-u DYLD_ROOT_PATH \
-u DYLD_INSERT_LIBRARIES \
-u DYLD_IMAGE_SUFFIX \
-u DYLD_VERSIONED_LIBRARY_PATH \
-u DYLD_VERSIONED_FRAMEWORK_PATH \
xcodebuild -project Inspector.xcodeproj \
-scheme Inspector \
-configuration Debug \
-sdk macosx \
CODE_SIGNING_ALLOWED=NO \
-derivedDataPath /tmp/DerivedData-Inspector
lint:
name: SwiftLint (core + CLI)
runs-on: macos-15
needs: [peekaboo-cli, tachikoma, mac-apps]
steps:
- uses: actions/checkout@v6
- name: Install SwiftLint
run: brew install swiftlint
- name: Run SwiftLint with CI config
run: swiftlint --config .swiftlint-ci.yml

View File

@ -1,63 +0,0 @@
name: Website (GitHub Pages)
on:
push:
branches: [main]
paths:
- "docs/**"
- "scripts/build-docs-site.mjs"
- "scripts/docs-site-assets.mjs"
- ".github/workflows/pages.yml"
workflow_dispatch:
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "24"
- name: Build docs site
run: node scripts/build-docs-site.mjs
- name: Validate docs site artifact
run: |
test -f _site/.nojekyll
test -f _site/.well-known/security.txt
test -f _site/security.txt
test -f _site/llms.txt
- name: Configure Pages
uses: actions/configure-pages@v6
- name: Upload artifact
uses: actions/upload-pages-artifact@v5
with:
path: _site
include-hidden-files: true
deploy:
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v5

View File

@ -1,69 +0,0 @@
name: Update Homebrew Formula
on:
release:
types: [published]
workflow_dispatch:
inputs:
version:
description: 'Version to update (e.g., 2.0.1)'
required: true
jobs:
update-homebrew-formula:
runs-on: ubuntu-latest
steps:
- name: Resolve release tag
run: |
if [ "${{ github.event_name }}" = "release" ]; then
echo "RELEASE_TAG=${{ github.event.release.tag_name }}" >> "$GITHUB_ENV"
else
echo "RELEASE_TAG=v${{ github.event.inputs.version }}" >> "$GITHUB_ENV"
fi
- name: Dispatch tap formula update
env:
GH_TOKEN: ${{ secrets.HOMEBREW_TAP_TOKEN }}
run: |
if [ -z "$GH_TOKEN" ]; then
echo "::error::Set HOMEBREW_TAP_TOKEN with workflow access to steipete/homebrew-tap"
exit 1
fi
request_id="peekaboo-${RELEASE_TAG}-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}"
expected_title="Update peekaboo for ${RELEASE_TAG} (${request_id})"
gh workflow run update-formula.yml \
--repo steipete/homebrew-tap \
--ref main \
-f formula=peekaboo \
-f tag="$RELEASE_TAG" \
-f repository="${{ github.repository }}" \
-f macos_artifact="peekaboo-macos-universal.tar.gz" \
-f request_id="$request_id"
run_id=""
for _ in {1..30}; do
run_id=$(gh run list \
--repo steipete/homebrew-tap \
--workflow update-formula.yml \
--branch main \
--event workflow_dispatch \
--limit 20 \
--json databaseId,displayTitle \
--jq ".[] | select(.displayTitle == \"$expected_title\") | .databaseId" | head -n1)
if [ -n "$run_id" ]; then
break
fi
sleep 5
done
if [ -z "$run_id" ]; then
echo "::error::Could not find tap workflow run with title: $expected_title"
exit 1
fi
gh run watch "$run_id" \
--repo steipete/homebrew-tap \
--exit-status \
--interval 10

190
.gitignore vendored
View File

@ -17,16 +17,8 @@ Network Trash Folder
Temporary Items
.apdisk
# macOS Extended Attributes and Metadata
*.bridgesupport
.metadata_never_index
.ql_*
.Trash-*
# Node.js / TypeScript
node_modules/
/node_modules/
Server/node_modules/
npm-debug.log*
yarn-debug.log*
yarn-error.log*
@ -44,7 +36,6 @@ lerna-debug.log*
.env.production.local
# TypeScript
Server/dist/
dist/
build/
*.js.map
@ -76,25 +67,20 @@ lib-cov/
.c9/
*.launch
.settings/
.claude/settings.local.json
_site/
*.sublime-workspace
*.sublime-project
# Swift / Xcode
## Build artifacts (at any level)
**/.build/
**/DerivedData/
**/build/
**/*.xcodeproj/project.xcworkspace/xcshareddata/
**/*.xcworkspace/xcshareddata/
## Build binaries
# Peekaboo CLI binary only (not directories)
peekaboo-cli/.build/
peekaboo-cli/DerivedData/
peekaboo-cli/.swiftpm/
peekaboo-cli/*.bc
# LLVM bitcode files (Swift compiler artifacts)
*.bc
# Compiled CLI binary in root
/peekaboo
/Apps/CLI/peekaboo
## Various Xcode settings
.build/
DerivedData/
*.pbxuser
!default.pbxuser
*.mode1v3
@ -103,168 +89,34 @@ _site/
!default.mode2v3
*.perspectivev3
!default.perspectivev3
**/xcuserdata/
xcuserdata/
*.xccheckout
*.moved-aside
**/*.xcuserstate
*.xcuserstate
*.xcscmblueprint
**/*.xcworkspace/xcuserdata/
## Xcode Patch
*.xcodeproj/*
!*.xcodeproj/project.pbxproj
!*.xcodeproj/xcshareddata/
!*.xcworkspace/contents.xcworkspacedata
/*.gcno
## Swift Package Manager
Packages/
Package.pins
**/Package.resolved
**/.swiftpm/
*.xcworkspace/xcshareddata/swiftpm/
## Playgrounds
playground.xcworkspace
timeline.xctimeline
## Build products
# Only ignore built app bundles in specific locations
/build/*.app
/DerivedData/**/*.app
/Apps/Mac/build/*.app
/Apps/Mac/DerivedData/**/*.app
/Apps/peekaboo
*.hmap
*.ipa
*.dSYM.zip
*.dSYM
## CocoaPods (if used)
Pods/
## Carthage (if used)
Carthage/Build/
## FastLane (if used)
fastlane/report.xml
fastlane/Preview.html
fastlane/screenshots/**/*.png
fastlane/test_output
## Code Injection
iOSInjectionProject/
## LLVM bitcode files (Swift compiler artifacts)
*.bc
## Module-specific build artifacts
Core/**/.build/
Core/**/DerivedData/
Core/**/.swiftpm/
## Swift compiler artifacts
*.hmap
*.bc
**/*.dia
Packages/
Package.pins
Package.resolved
.swiftpm/
*.playground
timeline.xctimeline
playground.xcworkspace
# Temporary files
*.tmp
*.temp
.cache/
debug
!docs/debug/
docs/debug/*
!docs/debug/visualizer-issues.md
!docs/debug/watch.md
.poltergeist-state/
.poltergeist*
*.bak
*.backup
*~
# Build artifacts and derived data
.artifacts/
.derived-data/
# Crush directory
.crush/
# OS generated files
Thumbs.db
ehthumbs.db
desktop.ini
# Editor backup files
*.swp
*.swo
.#*
#*#
# npm package files
*.tgz
# Auto-generated version file
Apps/CLI/Sources/peekaboo/Version.swift
Apps/CLI/.generated/
# Built CLI binary only (not the source folder)
/Apps/CLI/peekaboo
# Release artifacts
/release/
Commander/Commander.tar.gz
# Test images and screenshots
Core/PeekabooCore/..png
Core/PeekabooCore/..png_annotated.png
*_screenshot.png
*_Screenshot_*.png
Calculator_*.png
TextEdit_*.png
Safari_*.png
Wispr_*.png
Finder_*.png
test-*.png
screenshot-*.png
Screenshot*.png
capture_*.png
peekaboo_*.png
!Apps/Mac/Peekaboo/Assets.xcassets/MenuIcon.imageset/peekaboo_menu_18.png
!Apps/Mac/Peekaboo/Assets.xcassets/MenuIcon.imageset/peekaboo_menu_36.png
!Apps/Mac/Peekaboo/Assets.xcassets/MenuIcon.imageset/peekaboo_menu_54.png
# Temporary test files
test.peekaboo.json
test_*.sh
check_*.sh
*.test.png
*.test.json
# Menubar elements JSON (test data)
menubar_elements.json
# Vite cache
.vite/
# Documentation audits and summaries
docc-class-audit.md
test-fixes-summary.md
# Archive directory (if truly archived)
# Uncomment if Archive/ should be excluded:
# /Archive/
# Root level test scripts that should be in scripts/
/test-*.sh
/check-*.sh
# AppleScript at root
/peekaboo.scpt
/peekaboo-x86_64
/peekaboo-arm64
/debug
# Root binary only
/peekaboo
# Vendored build caches
Vendor/swift-argument-parser/.build/
/info
peekaboo-cli/Sources/peekaboo/Version.swift
/peekaboo-cli/peekaboo

20
.gitmodules vendored
View File

@ -1,20 +0,0 @@
[submodule "AXorcist"]
path = AXorcist
url = https://github.com/steipete/AXorcist.git
branch = main
[submodule "Tachikoma"]
path = Tachikoma
url = https://github.com/steipete/Tachikoma.git
branch = main
[submodule "Commander"]
path = Commander
url = https://github.com/steipete/Commander.git
branch = main
[submodule "TauTUI"]
path = TauTUI
url = https://github.com/steipete/TauTUI.git
branch = main
[submodule "Swiftdansi"]
path = Swiftdansi
url = https://github.com/steipete/Swiftdansi.git
branch = main

View File

@ -1,21 +0,0 @@
MAC_RELEASE_APP_NAME=Peekaboo
MAC_RELEASE_REPO=openclaw/Peekaboo
MAC_RELEASE_BUNDLE_ID=boo.peekaboo.mac
MAC_RELEASE_VERSION_FILE=/dev/null
MARKETING_VERSION=$(node -p "require('./package.json').version")
MAC_RELEASE_APPCAST=appcast.xml
MAC_RELEASE_INFO_PLIST=Apps/Mac/Peekaboo/Info.plist
MAC_RELEASE_SUPUBLIC_ED_KEY=AGCY8w5vHirVfGGDGc8Szc5iuOqupZSh9pMj/Qs67XI=
MAC_RELEASE_SIGNING_KEY_FILE='$HOME/Library/CloudStorage/Dropbox/Backup/Sparkle/sparkle-private-key-OBSOLETE-not-for-BlackBar-publickey-AGCY8w5vHirVfGGDGc8Szc5iuOqupZSh9pMj_Qs67XI-2026-05-21.txt'
MAC_RELEASE_APP_ZIP='${RELEASE_DIR:-release}/Peekaboo-${MARKETING_VERSION}.app.zip'
MAC_RELEASE_ARTIFACT_PREFIX='Peekaboo-'
MAC_RELEASE_REQUIRE_DSYM=0
MAC_RELEASE_FEED_URL='https://raw.githubusercontent.com/openclaw/Peekaboo/main/appcast.xml'
MAC_RELEASE_DOWNLOAD_URL_PREFIX='https://github.com/openclaw/Peekaboo/releases/download/v${MARKETING_VERSION}/'
MAC_RELEASE_PRECHECK='node scripts/prepare-release.js'
MAC_RELEASE_PACKAGE_CMD='scripts/release-macos-app.sh --no-appcast'
MAC_RELEASE_TAG_SIGNED=0
MAC_RELEASE_TAG_FORCE=0
MAC_RELEASE_TAG_ANNOTATED=0

View File

@ -1,51 +0,0 @@
# SwiftFormat configuration for Peekaboo project
# Compatible with Swift 6 strict concurrency mode
# IMPORTANT: Don't remove self where it's required for Swift 6 concurrency
--self insert # Insert self for member references (required for Swift 6)
--selfrequired # List of functions that require explicit self
--importgrouping testable-bottom # Group @testable imports at the bottom
--extensionacl on-declarations # Set ACL on extension members
# Indentation
--indent 4
--indentcase false
--ifdef no-indent
--xcodeindentation enabled
# Line breaks
--linebreaks lf
--maxwidth 120
# Whitespace
--trimwhitespace always
--emptybraces no-space
--nospaceoperators ...,..<
--ranges no-space
--someAny true
--voidtype void
# Wrapping
--wraparguments before-first
--wrapparameters before-first
--wrapcollections before-first
--closingparen same-line
# Organization
--organizetypes class,struct,enum,extension
--extensionmark "MARK: - %t + %p"
--marktypes always
--markextensions always
--structthreshold 0
--enumthreshold 0
# Swift 6 specific
--swiftversion 6.2
# Other
--stripunusedargs closure-only
--header ignore
--allman false
# Exclusions
--exclude .build,.swiftpm,DerivedData,node_modules,dist,coverage,xcuserdata,AXorcist,Commander,Swiftdansi,Tachikoma,TauTUI,Core/PeekabooCore/Sources/PeekabooCore/Extensions/NSArray+Extensions.swift

View File

@ -1,29 +0,0 @@
parent_config: .swiftlint.yml
included:
- Core/PeekabooCore/Sources/PeekabooCore
- Core/PeekabooCore/Tests/PeekabooTests
- Apps/CLI/Sources/PeekabooCLI
- Apps/CLI/Tests/CLIAutomationTests
- Apps/CLI/Tests/CoreCLITests
excluded: []
disabled_rules:
- line_length
- function_body_length
- cyclomatic_complexity
- file_length
- type_body_length
- function_parameter_count
- nesting
- multiline_arguments
- multiline_parameters
- multiple_closures_with_trailing_closure
- void_return
- force_cast
- force_try
- for_where
- superfluous_disable_command
reporter: "github-actions-logging"

View File

@ -1,175 +0,0 @@
# SwiftLint configuration for Peekaboo - Swift 6 compatible
# Paths to include
included:
- Apps
- Core
# Paths to exclude
excluded:
- .build
- DerivedData
- "**/Generated"
- "**/Resources"
- "**/.build"
- "**/Package.swift"
- "**/Tests/Resources"
- "Apps/CLI/.build"
- "**/DerivedData"
- "**/.swiftpm"
- Pods
- Carthage
- fastlane
- vendor
- "*.playground"
# Exclude specific files that should not be linted/formatted
- "Core/PeekabooCore/Sources/PeekabooCore/Extensions/NSArray+Extensions.swift"
# Analyzer rules (require compilation)
analyzer_rules:
- unused_declaration
- unused_import
# Enable specific rules
opt_in_rules:
- array_init
- closure_spacing
- contains_over_first_not_nil
- empty_count
- empty_string
- explicit_init
- fallthrough
- fatal_error_message
- first_where
- joined_default_parameter
- last_where
- literal_expression_end_indentation
- multiline_arguments
- multiline_parameters
- operator_usage_whitespace
- overridden_super_call
- pattern_matching_keywords
- private_outlet
- prohibited_super_call
- redundant_nil_coalescing
- sorted_first_last
- switch_case_alignment
- unneeded_parentheses_in_closure_argument
- vertical_parameter_alignment_on_call
# Disable rules that conflict with Swift 6 or our coding style
disabled_rules:
# Swift 6 requires explicit self - disable explicit_self rule
- explicit_self
# SwiftFormat handles these
- trailing_whitespace
- trailing_newline
- trailing_comma
- vertical_whitespace
- indentation_width
# Too restrictive or not applicable
- identifier_name # Single letter names are fine in many contexts
- file_header
- explicit_top_level_acl
- explicit_acl
- explicit_type_interface
- missing_docs
- required_deinit
- prefer_nimble
- quick_discouraged_call
- quick_discouraged_focused_test
- quick_discouraged_pending_test
- anonymous_argument_in_multiline_closure
- no_extension_access_modifier
- no_grouping_extension
- switch_case_on_newline
- strict_fileprivate
- extension_access_modifier
- convenience_type
- no_magic_numbers
- one_declaration_per_file
- vertical_whitespace_between_cases
- vertical_whitespace_closing_braces
- superfluous_else
- number_separator
- prefixed_toplevel_constant
- opening_brace
- trailing_closure
- contrasted_opening_brace
- sorted_imports
- redundant_type_annotation
- shorthand_optional_binding
- untyped_error_in_catch
- file_name
- todo
# Custom rules
custom_rules:
no_direct_ax_in_peekaboo:
included: "Core/PeekabooCore"
excluded: "Core/PeekabooCore/Tests"
name: "No Direct AX/CG Event APIs in PeekabooCore"
regex: "\\bAXUIElement\\b|\\bCGEvent\\b"
message: "Use AXorcist abstractions (Element/InputDriver/AXWindowResolver) instead of direct AXUIElement/CGEvent."
severity: error
no_ui_appservices_import:
included: "Core/PeekabooCore/Sources/PeekabooAutomation/Services/UI"
regex: "^import\\s+ApplicationServices"
message: "Import AX/CG bindings via AXorcist; avoid direct ApplicationServices in UI services."
severity: warning
# Rule configurations
force_cast: warning
force_try: warning
# identifier_name rule disabled - see disabled_rules section
type_name:
min_length:
warning: 2
error: 1
max_length:
warning: 60
error: 80
function_body_length:
warning: 150
error: 300
file_length:
warning: 1500
error: 2500
ignore_comment_only_lines: true
type_body_length:
warning: 800
error: 1200
cyclomatic_complexity:
warning: 20
error: 120
large_tuple:
warning: 4
error: 5
nesting:
type_level:
warning: 4
error: 6
function_level:
warning: 5
error: 7
line_length:
warning: 120
error: 250
ignores_comments: true
ignores_urls: true
# Custom rules can be added here if needed
# Reporter type
reporter: "xcode"

View File

@ -1,75 +0,0 @@
{
"ignore_dirs": [
"**/.build/**",
"**/DerivedData/**",
"**/node_modules/**",
"*.7z",
"*.app",
"*.dSYM",
"*.framework",
"*.gz",
"*.ipa",
"*.rar",
"*.swiftdoc",
"*.swiftmodule",
"*.swiftsourceinfo",
"*.swo",
"*.swp",
"*.tar",
"*.temp",
"*.tmp",
"*.xcodeproj/project.xcworkspace/xcuserdata",
"*.xcodeproj/xcuserdata",
"*.xcworkspace/xcshareddata/xcschemes",
"*.xcworkspace/xcuserdata",
"*.zip",
".DS_Store",
".build",
".bzr",
".cache",
".cursor",
".git",
".hg",
".idea",
".next",
".nuxt",
".nyc_output",
".parcel-cache",
".svn",
".tmp",
".vs",
".vscode",
"DerivedData",
"Package.resolved",
"Thumbs.db",
"build",
"coverage",
"desktop.ini",
"dist",
"node_modules",
"out",
"temp",
"tmp",
"**/test_results/**",
"**/*.xcuserstate",
"**/Version.swift"
],
"ignore_vcs": [
".git",
".svn",
".hg",
".bzr"
],
"idle_reap_age_seconds": 300,
"gc_age_seconds": 259200,
"gc_interval_seconds": 86400,
"max_files": 15000,
"settle": 1000,
"_metadata": {
"generated_by": "poltergeist",
"project_type": "mixed",
"performance_profile": "balanced",
"generated_at": "2025-11-22T11:35:16.426Z",
"total_exclusions": 53
}
}

View File

@ -1,46 +0,0 @@
# Repository Guidelines
## Start Here
- Read `~/Projects/agent-scripts/{AGENTS.MD,TOOLS.MD}` before making changes (skip if missing).
- This repo uses git submodules (`AXorcist/`, `Commander/`, `Tachikoma/`, `TauTUI/`); update them in their home repos first, then bump pointers here.
## Project Structure & Modules
- `Apps/CLI` contains the SwiftPM package for the command-line tool; commands live under `Apps/CLI/Sources`, and unit/integration tests under `Apps/CLI/Tests`.
- `Apps/Mac`, `Apps/peekaboo`, and `Apps/PeekabooInspector` host the macOS app and related tooling; open `Apps/Peekaboo.xcworkspace` for Xcode work.
- Shared logic sits in `Core/PeekabooCore` (automation, agent runtime, visualizer). Keep new utilities there rather than duplicating in apps.
- Git submodules provide foundational pieces: `AXorcist/` (AX automation), `Commander/` (CLI parsing), `Tachikoma/` (AI providers/MCP), and `TauTUI/`. Update them upstream first, then bump the pointers here.
- Documentation lives in `docs/`; assets and marketing material are in `assets/`.
## Build, Test, and Development Commands
- Current local baseline is macOS 26.1 on arm64. If youre on an older SDK/OS, expect menubar/accessibility flakiness; re-run with the 26 SDK before chasing Peekaboo regressions.
- Run tools directly (runner removed). Use pnpm (Corepack-enabled).
- Build the CLI: `pnpm run build:cli` (debug) or `pnpm run build:swift:all` (universal release). For arm64-only: `pnpm run build:swift`.
- Rapid rebuilds while editing Swift: `pnpm run poltergeist:haunt` → check with `pnpm run poltergeist:status`, stop via `pnpm run poltergeist:rest`.
- Validate before handoff: `pnpm run lint` (SwiftLint), `pnpm run format` (SwiftFormat check/fix), then `pnpm run test:safe`. Full automation/UI tests: `pnpm run test:automation` or `pnpm run test:all`.
- Tachikoma live provider checks: `pnpm run tachikoma:test:integration`.
- You may run `peekaboo` CLI commands locally for repros/debugging; be mindful they capture the host desktop (screen recording/accessibility permissions required).
## Coding Style & Naming Conventions
- Swift 6.2, 4-space indent, 120-column wrap; explicit `self` is required (SwiftFormat enforces). Run `pnpm run format` before committing.
- SwiftLint config lives in `.swiftlint.yml`; keep new code typed (avoid `Any`), prefer small scoped extensions over large files.
- Follow existing module boundaries: automation APIs in `PeekabooAutomation`, agent glue in `PeekabooAgentRuntime`, UI feedback in `PeekabooVisualizer`.
## Testing Guidelines
- Add regression tests alongside fixes in `Apps/CLI/Tests` (XCTest naming: `ThingTests`). Use `PEEKABOO_INCLUDE_AUTOMATION_TESTS=true` env only when automation permissions are available.
- For local end-to-end runs, ensure macOS Screen Recording and Accessibility are granted (`peekaboo permissions status|grant`).
## Commit & Pull Request Guidelines
- Conventional Commits (`feat|fix|chore|docs|test|refactor|build|ci|style|perf`); scope optional: `feat(cli): add capture retry`.
- Use `./scripts/committer "type(scope): summary" <paths…>` to stage and create commits; avoid raw `git add`.
- Batch git network ops in groups: commit related repo changes first, then push/pull repos together so submodule gitlinks stay coherent.
- PRs should summarize intent, list test commands executed, mention doc updates, and include screenshots or terminal snippets when behavior changes.
- Never release or publish without an explicit release command.
- Peekaboo releases: follow `$release-peekaboo`; current Mac + existing 1Password credentials first. App Store Connect changes last resort, only after same-item `notarytool history` and non-S3 `submit` both fail.
- Credentialed release wrappers: `bash -c`, never login shells; profile exports can override ASC IDs and mix credentials.
- Published CLI proof: run `npm exec` from `/tmp`; repo cwd may shadow the downloaded package with a local binary.
- During PR triage, keep moving autonomously: fix defects, add obvious scoped features, and rewrite or land what makes sense.
- Before landing every PR, run autoreview until no actionable findings remain and fix or rerun CI until green.
## Security & Configuration Tips
- Secrets and provider tokens live under `~/.peekaboo` (managed by Tachikoma); never commit credentials or sample keys.
- Respect permissions flows documented in `docs/permissions.md`; avoid editing derived artifacts—regenerate via the provided scripts instead.

@ -1 +0,0 @@
Subproject commit c276ac88a0ebddb2a618b31092715d6df87456e0

4
Apps/CLI/.gitignore vendored
View File

@ -1,4 +0,0 @@
# Test output files
test-results/

View File

@ -1,60 +0,0 @@
# SwiftLint configuration for Peekaboo CLI (Swift 6.2)
#
# The CLI target runs in Swift 6.2 strict concurrency mode, so we rely on SwiftFormat
# to insert explicit `self` where required and keep opt-in rules focused on logic bugs
# instead of style that SwiftFormat already enforces.
swiftlint_version: 0.62.2
# Rules
disabled_rules:
- trailing_whitespace
- trailing_comma # SwiftFormat handles trailing commas for us
- todo
- superfluous_disable_command
- function_parameter_count
- function_body_length
- type_body_length
- file_length
- cyclomatic_complexity
- nesting
- large_tuple
- line_length
- identifier_name
- force_cast
- void_return
- empty_string
- unused_optional_binding
- unused_enumerated
- for_where
opt_in_rules:
- closure_spacing
- empty_count
- empty_string
- contains_over_filter_count
- contains_over_filter_is_empty
- contains_over_first_not_nil
- contains_over_range_nil_comparison
- discouraged_object_literal
- first_where
- last_where
- legacy_multiple
- prefer_self_type_over_type_of_self
- sorted_first_last
- unneeded_parentheses_in_closure_argument
- vertical_parameter_alignment_on_call
# Rule configurations tuned for Swift 6.2 ergonomics
# Paths
included:
- Sources
- Tests
excluded:
- .build
- .swiftpm
- .git
- Package.swift
- DerivedData
- "**/.build"
- "**/DerivedData"

View File

@ -1,359 +0,0 @@
//
// AcceleratedTextDetector.swift
// PeekabooCore
//
import Accelerate
import AppKit
import CoreGraphics
import Foundation
/// High-performance text detection using Accelerate framework's vImage convolution
final class AcceleratedTextDetector {
// MARK: - Types
struct EdgeDensityResult {
let density: Float // 0.0 = no edges, 1.0 = all edges
let hasText: Bool // Quick decision based on threshold
}
// MARK: - Properties
/// Sobel kernels as Int16 for vImage convolution
private let sobelXKernel: [Int16] = [
-1, 0, 1,
-2, 0, 2,
-1, 0, 1
]
private let sobelYKernel: [Int16] = [
-1, -2, -1,
0, 0, 0,
1, 2, 1
]
// Pre-allocated buffers for performance
private var sourceBuffer: vImage_Buffer = .init()
private var gradientXBuffer: vImage_Buffer = .init()
private var gradientYBuffer: vImage_Buffer = .init()
private var magnitudeBuffer: vImage_Buffer = .init()
// Buffer dimensions
private let maxBufferWidth: Int = 200
private let maxBufferHeight: Int = 100
/// Edge detection threshold (0-255 scale)
private let edgeThreshold: UInt8 = 30
// MARK: - Initialization
init() {
self.allocateBuffers()
}
deinit {
deallocateBuffers()
}
// MARK: - Public Methods
/// Analyzes a region for text presence using Sobel edge detection
func analyzeRegion(_ rect: NSRect, in image: NSImage) -> EdgeDensityResult {
// Quick contrast check first
if let quickResult = performQuickCheck(rect, in: image) {
return quickResult
}
// Extract region as grayscale buffer
guard let buffer = extractRegionAsBuffer(rect, from: image) else {
return EdgeDensityResult(density: 0, hasText: false)
}
// Apply Sobel operators
let (gradX, gradY) = self.applySobelOperators(to: buffer)
// Calculate gradient magnitude
let magnitude = self.calculateGradientMagnitude(gradX: gradX, gradY: gradY)
// Calculate edge density
let density = self.calculateEdgeDensity(magnitude: magnitude)
// Free temporary buffer
free(buffer.data)
// Determine if region has text (high edge density)
// Lower threshold to be more sensitive to text
let hasText = density > 0.08 // 8% of pixels are edges = likely text
return EdgeDensityResult(density: density, hasText: hasText)
}
/// Scores a region for label placement (higher = better)
func scoreRegionForLabelPlacement(_ rect: NSRect, in image: NSImage) -> Float {
let result = self.analyzeRegion(rect, in: image)
// More aggressive scoring to avoid text
// Areas with ANY significant edges should score very low
if result.hasText || result.density > 0.1 {
return 0.0 // Definitely avoid
} else if result.density < 0.02 {
return 1.0 // Perfect - almost no edges
} else {
// Exponential decay for intermediate values
return exp(-result.density * 50.0)
}
}
// MARK: - Private Methods
private func allocateBuffers() {
let bytesPerPixel = 1 // Grayscale
let bufferSize = self.maxBufferWidth * self.maxBufferHeight * bytesPerPixel
// Allocate source buffer
self.sourceBuffer.data = malloc(bufferSize)
self.sourceBuffer.width = vImagePixelCount(self.maxBufferWidth)
self.sourceBuffer.height = vImagePixelCount(self.maxBufferHeight)
self.sourceBuffer.rowBytes = self.maxBufferWidth * bytesPerPixel
// Allocate gradient buffers
self.gradientXBuffer.data = malloc(bufferSize)
self.gradientXBuffer.width = vImagePixelCount(self.maxBufferWidth)
self.gradientXBuffer.height = vImagePixelCount(self.maxBufferHeight)
self.gradientXBuffer.rowBytes = self.maxBufferWidth * bytesPerPixel
self.gradientYBuffer.data = malloc(bufferSize)
self.gradientYBuffer.width = vImagePixelCount(self.maxBufferWidth)
self.gradientYBuffer.height = vImagePixelCount(self.maxBufferHeight)
self.gradientYBuffer.rowBytes = self.maxBufferWidth * bytesPerPixel
// Allocate magnitude buffer
self.magnitudeBuffer.data = malloc(bufferSize)
self.magnitudeBuffer.width = vImagePixelCount(self.maxBufferWidth)
self.magnitudeBuffer.height = vImagePixelCount(self.maxBufferHeight)
self.magnitudeBuffer.rowBytes = self.maxBufferWidth * bytesPerPixel
}
private func deallocateBuffers() {
free(self.sourceBuffer.data)
free(self.gradientXBuffer.data)
free(self.gradientYBuffer.data)
free(self.magnitudeBuffer.data)
}
private func performQuickCheck(_ rect: NSRect, in image: NSImage) -> EdgeDensityResult? {
// Sample 5 points: corners + center
let points = [
CGPoint(x: rect.minX, y: rect.minY),
CGPoint(x: rect.maxX, y: rect.minY),
CGPoint(x: rect.midX, y: rect.midY),
CGPoint(x: rect.minX, y: rect.maxY),
CGPoint(x: rect.maxX, y: rect.maxY)
]
guard let bitmap = getBitmapRep(from: image) else { return nil }
var brightnesses: [Float] = []
for point in points {
if let color = getPixelColor(at: point, from: bitmap) {
brightnesses.append(self.calculateBrightness(color))
}
}
guard !brightnesses.isEmpty else { return nil }
let minBrightness = brightnesses.min() ?? 0
let maxBrightness = brightnesses.max() ?? 0
let contrast = maxBrightness - minBrightness
// Very low contrast = definitely no text
if contrast < 0.1 {
return EdgeDensityResult(density: 0.0, hasText: false)
}
// Very high contrast = definitely has text
if contrast > 0.6 {
return EdgeDensityResult(density: 1.0, hasText: true)
}
// Intermediate contrast = need full analysis
return nil
}
private func extractRegionAsBuffer(_ rect: NSRect, from image: NSImage) -> vImage_Buffer? {
guard let bitmap = getBitmapRep(from: image) else { return nil }
// Calculate actual region to extract (clamp to image bounds)
let imageRect = NSRect(origin: .zero, size: image.size)
let clampedRect = rect.intersection(imageRect)
guard !clampedRect.isEmpty else { return nil }
// Determine if we need to downsample
let shouldDownsample = clampedRect.width > CGFloat(self.maxBufferWidth) ||
clampedRect.height > CGFloat(self.maxBufferHeight)
let targetWidth = shouldDownsample ? self.maxBufferWidth : Int(clampedRect.width)
let targetHeight = shouldDownsample ? self.maxBufferHeight : Int(clampedRect.height)
// Allocate buffer for this specific region
let bufferSize = targetWidth * targetHeight
guard let bufferData = malloc(bufferSize) else { return nil }
var buffer = vImage_Buffer()
buffer.data = bufferData
buffer.width = vImagePixelCount(targetWidth)
buffer.height = vImagePixelCount(targetHeight)
buffer.rowBytes = targetWidth
// Fill buffer with grayscale pixel data
let pixelData = bufferData.assumingMemoryBound(to: UInt8.self)
for y in 0..<targetHeight {
for x in 0..<targetWidth {
// Map to source coordinates
let sourceX = Int(clampedRect.minX) + (x * Int(clampedRect.width)) / targetWidth
let sourceY = Int(clampedRect.minY) + (y * Int(clampedRect.height)) / targetHeight
// Get pixel color and convert to grayscale
if let color = bitmap.colorAt(x: sourceX, y: Int(image.size.height) - sourceY - 1) {
let brightness = self.calculateBrightness(color)
pixelData[y * targetWidth + x] = UInt8(brightness * 255)
} else {
pixelData[y * targetWidth + x] = 128 // Default gray
}
}
}
return buffer
}
private func applySobelOperators(to buffer: vImage_Buffer) -> (gradX: vImage_Buffer, gradY: vImage_Buffer) {
// Create properly sized output buffers
var gradX = vImage_Buffer()
gradX.data = malloc(Int(buffer.width * buffer.height))
gradX.width = buffer.width
gradX.height = buffer.height
gradX.rowBytes = Int(buffer.width)
var gradY = vImage_Buffer()
gradY.data = malloc(Int(buffer.width * buffer.height))
gradY.width = buffer.width
gradY.height = buffer.height
gradY.rowBytes = Int(buffer.width)
// Apply Sobel X kernel
var sourceBuffer = buffer
vImageConvolve_Planar8(
&sourceBuffer,
&gradX,
nil,
0,
0,
self.sobelXKernel,
3,
3,
1, // Divisor
128, // Bias (to keep values positive)
vImage_Flags(kvImageEdgeExtend)
)
// Apply Sobel Y kernel
vImageConvolve_Planar8(
&sourceBuffer,
&gradY,
nil,
0,
0,
self.sobelYKernel,
3,
3,
1, // Divisor
128, // Bias (to keep values positive)
vImage_Flags(kvImageEdgeExtend)
)
return (gradX, gradY)
}
private func calculateGradientMagnitude(gradX: vImage_Buffer, gradY: vImage_Buffer) -> vImage_Buffer {
// Create magnitude buffer
var magnitude = vImage_Buffer()
magnitude.data = malloc(Int(gradX.width * gradX.height))
magnitude.width = gradX.width
magnitude.height = gradX.height
magnitude.rowBytes = Int(gradX.width)
// Calculate magnitude for each pixel
// Using Manhattan distance for speed: |gradX| + |gradY|
let gradXData = gradX.data.assumingMemoryBound(to: UInt8.self)
let gradYData = gradY.data.assumingMemoryBound(to: UInt8.self)
let magnitudeData = magnitude.data.assumingMemoryBound(to: UInt8.self)
let pixelCount = Int(gradX.width * gradX.height)
for i in 0..<pixelCount {
// Remove bias and get absolute values
let gx = abs(Int(gradXData[i]) - 128)
let gy = abs(Int(gradYData[i]) - 128)
// Manhattan distance approximation
let mag = min(gx + gy, 255)
magnitudeData[i] = UInt8(mag)
}
// Free gradient buffers
free(gradX.data)
free(gradY.data)
return magnitude
}
private func calculateEdgeDensity(magnitude: vImage_Buffer) -> Float {
let magnitudeData = magnitude.data.assumingMemoryBound(to: UInt8.self)
let pixelCount = Int(magnitude.width * magnitude.height)
var edgePixelCount = 0
for i in 0..<pixelCount where magnitudeData[i] > self.edgeThreshold {
edgePixelCount += 1
}
// Free magnitude buffer
free(magnitude.data)
return Float(edgePixelCount) / Float(pixelCount)
}
// MARK: - Helper Methods
private func getBitmapRep(from image: NSImage) -> NSBitmapImageRep? {
guard let tiffData = image.tiffRepresentation,
let bitmap = NSBitmapImageRep(data: tiffData) else {
return nil
}
return bitmap
}
private func getPixelColor(at point: CGPoint, from bitmap: NSBitmapImageRep) -> NSColor? {
let x = Int(point.x)
let y = Int(bitmap.size.height - point.y - 1) // Flip Y coordinate
guard x >= 0, x < bitmap.pixelsWide,
y >= 0, y < bitmap.pixelsHigh else {
return nil
}
return bitmap.colorAt(x: x, y: y)
}
private func calculateBrightness(_ color: NSColor) -> Float {
guard let rgbColor = color.usingColorSpace(.deviceRGB) else {
return 0.5
}
// Standard luminance formula
return Float(rgbColor.redComponent) * 0.299 +
Float(rgbColor.greenComponent) * 0.587 +
Float(rgbColor.blueComponent) * 0.114
}
}

View File

@ -1,413 +0,0 @@
//
// SmartLabelPlacer.swift
// PeekabooCore
//
import AppKit
import Foundation
import PeekabooCore
/// Handles intelligent label placement for UI element annotations
final class SmartLabelPlacer {
// MARK: - Properties
private let image: NSImage
private let imageSize: NSSize
private let textDetector: AcceleratedTextDetector
private let fontSize: CGFloat
private let labelSpacing: CGFloat = 3
private let cornerInset: CGFloat = 2
/// Label placement debugging
private let debugMode: Bool
// MARK: - Initialization
init(image: NSImage, fontSize: CGFloat = 8, debugMode: Bool = false) {
self.image = image
self.imageSize = image.size
self.textDetector = AcceleratedTextDetector()
self.fontSize = fontSize
self.debugMode = debugMode
}
// MARK: - Public Methods
/// Finds the best position for a label given an element's bounds
/// - Parameters:
/// - element: The detected UI element
/// - elementRect: The element's rectangle in drawing coordinates (Y-flipped)
/// - labelSize: The size of the label to place
/// - existingLabels: Already placed labels to avoid overlapping
/// - allElements: All elements to avoid overlapping with
/// - Returns: Tuple of (labelRect, connectionPoint) or nil if no good position found
func findBestLabelPosition(
for element: DetectedElement,
elementRect: NSRect,
labelSize: NSSize,
existingLabels: [(rect: NSRect, element: DetectedElement)],
allElements: [(element: DetectedElement, rect: NSRect)]
) -> (labelRect: NSRect, connectionPoint: NSPoint?)? {
// Generate candidate positions based on element type
let candidates = self.generateCandidatePositions(
for: element,
elementRect: elementRect,
labelSize: labelSize
)
// Filter out positions that overlap with other elements or labels
let validPositions = self.filterValidPositions(
candidates: candidates,
element: element,
existingLabels: existingLabels,
allElements: allElements
)
guard !validPositions.isEmpty else {
// Try internal positions as fallback
return self.findInternalPosition(
for: element,
elementRect: elementRect,
labelSize: labelSize
)
}
// Score each valid position using edge detection
let scoredPositions = self.scorePositions(validPositions, elementRect: elementRect)
// Pick the best scoring position
guard let best = scoredPositions.max(by: { $0.score < $1.score }) else {
return nil
}
if self.debugMode {
Logger.shared.verbose("Best position for \(element.id): index \(best.index) with score \(best.score)")
}
// Calculate connection point if needed
let connectionPoint = self.calculateConnectionPoint(
for: best.index,
elementRect: elementRect,
isExternal: best.index < candidates.count
)
return (labelRect: best.rect, connectionPoint: connectionPoint)
}
// MARK: - Private Methods
private func generateCandidatePositions(
for element: DetectedElement,
elementRect: NSRect,
labelSize: NSSize
) -> [(rect: NSRect, index: Int, type: PositionType)] {
var positions: [(rect: NSRect, index: Int, type: PositionType)] = []
// For buttons and links, prefer corners to avoid centered text
if element.type == .button || element.type == .link {
// External corners (less intrusive)
positions.append(contentsOf: [
// Top-left external
(NSRect(
x: elementRect.minX - labelSize.width - self.labelSpacing,
y: elementRect.maxY - labelSize.height,
width: labelSize.width,
height: labelSize.height
), 0, .externalTopLeft),
// Top-right external
(NSRect(
x: elementRect.maxX + self.labelSpacing,
y: elementRect.maxY - labelSize.height,
width: labelSize.width,
height: labelSize.height
), 1, .externalTopRight),
// Bottom-left external
(NSRect(
x: elementRect.minX - labelSize.width - self.labelSpacing,
y: elementRect.minY,
width: labelSize.width,
height: labelSize.height
), 2, .externalBottomLeft),
// Bottom-right external
(NSRect(
x: elementRect.maxX + self.labelSpacing,
y: elementRect.minY,
width: labelSize.width,
height: labelSize.height
), 3, .externalBottomRight),
])
}
// For text fields, prefer right side
if element.type == .textField {
positions.append((
NSRect(
x: elementRect.maxX + self.labelSpacing,
y: elementRect.midY - labelSize.height / 2,
width: labelSize.width,
height: labelSize.height
), 4, .externalRight
))
}
// For checkboxes, prefer left side
if element.type == .checkbox {
positions.append((
NSRect(
x: elementRect.minX - labelSize.width - self.labelSpacing,
y: elementRect.midY - labelSize.height / 2,
width: labelSize.width,
height: labelSize.height
), 5, .externalLeft
))
}
// Add standard positions as fallbacks
// For buttons, avoid centered positions (where text usually is)
if element.type != .button && element.type != .link {
positions.append(contentsOf: [
// Above
(NSRect(
x: elementRect.midX - labelSize.width / 2,
y: elementRect.maxY + self.labelSpacing,
width: labelSize.width,
height: labelSize.height
), 6, .externalAbove),
// Below
(NSRect(
x: elementRect.midX - labelSize.width / 2,
y: elementRect.minY - labelSize.height - self.labelSpacing,
width: labelSize.width,
height: labelSize.height
), 7, .externalBelow),
])
} else {
// For buttons, prefer side positions
positions.append(contentsOf: [
// Right side
(NSRect(
x: elementRect.maxX + self.labelSpacing,
y: elementRect.midY - labelSize.height / 2,
width: labelSize.width,
height: labelSize.height
), 6, .externalRight),
// Left side
(NSRect(
x: elementRect.minX - labelSize.width - self.labelSpacing,
y: elementRect.midY - labelSize.height / 2,
width: labelSize.width,
height: labelSize.height
), 7, .externalLeft),
])
}
return positions
}
private func filterValidPositions(
candidates: [(rect: NSRect, index: Int, type: PositionType)],
element: DetectedElement,
existingLabels: [(rect: NSRect, element: DetectedElement)],
allElements: [(element: DetectedElement, rect: NSRect)]
) -> [(rect: NSRect, index: Int, type: PositionType)] {
candidates.filter { candidate in
// Check if within image bounds
guard candidate.rect.minX >= 0 && candidate.rect.maxX <= self.imageSize.width &&
candidate.rect.minY >= 0 && candidate.rect.maxY <= self.imageSize.height else {
return false
}
// Check overlap with other elements
for (otherElement, otherRect) in allElements {
if otherElement.id != element.id && candidate.rect.intersects(otherRect) {
return false
}
}
// Check overlap with existing labels
for (existingLabel, _) in existingLabels where candidate.rect.intersects(existingLabel) {
return false
}
return true
}
}
private func scorePositions(
_ positions: [(rect: NSRect, index: Int, type: PositionType)],
elementRect: NSRect
) -> [(rect: NSRect, index: Int, type: PositionType, score: Float)] {
positions.map { position in
// Convert from drawing coordinates to image coordinates for analysis
// Drawing has Y=0 at top, image has Y=0 at bottom
let imageRect = NSRect(
x: position.rect.origin.x,
y: self.imageSize.height - position.rect.origin.y - position.rect.height,
width: position.rect.width,
height: position.rect.height
)
// Score using edge detection
let score = self.textDetector.scoreRegionForLabelPlacement(imageRect, in: self.image)
if self.debugMode {
Logger.shared.verbose("""
Position \(position.index) (\(position.type)) for element:
- Drawing rect: \(position.rect)
- Image rect: \(imageRect)
- Score: \(score)
""")
}
return (rect: position.rect, index: position.index, type: position.type, score: score)
}
}
private func findInternalPosition(
for element: DetectedElement,
elementRect: NSRect,
labelSize: NSSize
) -> (labelRect: NSRect, connectionPoint: NSPoint?)? {
let insidePositions: [NSRect] = if element.type == .button || element.type == .link {
// For buttons, use corners with small inset
[
// Top-left corner
NSRect(
x: elementRect.minX + self.cornerInset,
y: elementRect.maxY - labelSize.height - self.cornerInset,
width: labelSize.width,
height: labelSize.height
),
// Top-right corner
NSRect(
x: elementRect.maxX - labelSize.width - self.cornerInset,
y: elementRect.maxY - labelSize.height - self.cornerInset,
width: labelSize.width,
height: labelSize.height
),
]
} else {
// For other elements
[
// Top-left
NSRect(
x: elementRect.minX + 2,
y: elementRect.maxY - labelSize.height - 2,
width: labelSize.width,
height: labelSize.height
),
]
}
// Find first position that fits
for candidateRect in insidePositions where elementRect.contains(candidateRect) {
// Score this internal position
let imageRect = NSRect(
x: candidateRect.origin.x,
y: self.imageSize.height - candidateRect.origin.y - candidateRect.height,
width: candidateRect.width,
height: candidateRect.height
)
let score = self.textDetector.scoreRegionForLabelPlacement(imageRect, in: self.image)
// Only use if score is acceptable (low edge density)
if score > 0.5 {
return (labelRect: candidateRect, connectionPoint: nil)
}
}
// Ultimate fallback - center
let centerRect = NSRect(
x: elementRect.midX - labelSize.width / 2,
y: elementRect.midY - labelSize.height / 2,
width: labelSize.width,
height: labelSize.height
)
return (labelRect: centerRect, connectionPoint: nil)
}
private func calculateConnectionPoint(
for positionIndex: Int,
elementRect: NSRect,
isExternal: Bool
) -> NSPoint? {
guard isExternal else { return nil }
// Connection points for external positions
switch positionIndex {
case 0, 1, 2, 3: // Corner positions
return NSPoint(x: elementRect.midX, y: elementRect.midY)
case 4: // Right
return NSPoint(x: elementRect.maxX, y: elementRect.midY)
case 5: // Left
return NSPoint(x: elementRect.minX, y: elementRect.midY)
case 6: // Above
return NSPoint(x: elementRect.midX, y: elementRect.maxY)
case 7: // Below
return NSPoint(x: elementRect.midX, y: elementRect.minY)
default:
return nil
}
}
// MARK: - Types
private enum PositionType: String {
case externalTopLeft
case externalTopRight
case externalBottomLeft
case externalBottomRight
case externalLeft
case externalRight
case externalAbove
case externalBelow
case internalTopLeft
case internalTopRight
case internalCenter
}
}
// MARK: - Debug Visualization
extension SmartLabelPlacer {
/// Creates a debug image showing edge detection results
func createDebugVisualization(for rect: NSRect) -> NSImage? {
// Convert to image coordinates
let imageRect = NSRect(
x: rect.origin.x,
y: self.imageSize.height - rect.origin.y - rect.height,
width: rect.width,
height: rect.height
)
let result = self.textDetector.analyzeRegion(imageRect, in: self.image)
// Create visualization showing edge density
let debugImage = NSImage(size: rect.size)
debugImage.lockFocus()
// Draw background color based on edge density
let color = if result.hasText {
NSColor.red.withAlphaComponent(0.5) // Bad for labels
} else {
NSColor.green.withAlphaComponent(0.5) // Good for labels
}
color.setFill()
NSRect(origin: .zero, size: rect.size).fill()
// Draw edge density percentage
let text = String(format: "%.1f%%", result.density * 100)
let attributes: [NSAttributedString.Key: Any] = [
.font: NSFont.systemFont(ofSize: 10),
.foregroundColor: NSColor.white
]
text.draw(at: NSPoint(x: 2, y: 2), withAttributes: attributes)
debugImage.unlockFocus()
return debugImage
}
}

View File

@ -1,6 +0,0 @@
{"timestamp":"2025-08-09T14:00:17.270Z","level":"info","message":"[peekaboo] Building with 0 changed file(s)"}
{"timestamp":"2025-08-09T14:00:17.271Z","level":"info","message":"[peekaboo] Build already in progress, skipping"}
{"timestamp":"2025-08-09T14:03:08.180Z","level":"info","message":"[peekaboo] Building with 0 changed file(s)"}
{"timestamp":"2025-08-09T14:03:08.181Z","level":"info","message":"[peekaboo] Build already in progress, skipping"}
{"timestamp":"2025-08-09T14:07:57.095Z","level":"info","message":"[peekaboo] Building with 0 changed file(s)"}
{"timestamp":"2025-08-09T14:07:57.095Z","level":"info","message":"[peekaboo] Build already in progress, skipping"}

View File

@ -1,222 +0,0 @@
# Changelog
All notable changes to Peekaboo CLI will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [3.5.3] - 2026-06-13
### Fixed
- Public CLI, agent, MCP, and API guidance now treats runtime element IDs as opaque strings to copy exactly instead of implying role-specific ID shapes. Thanks @coygeek for #194.
- JSON-only `peekaboo see` runs without `--path` now keep required screenshots in snapshot storage instead of leaving files on Desktop or exposing their temporary paths. Thanks @coygeek for #196.
- Background element/query/coordinate clicks now pin actions to the requested process and exact window, reject mismatched window/PID selectors and unverifiable snapshots, invalidate implicit latest snapshots without deleting history, and no longer require Event Synthesizing when Accessibility completes the click.
- App launch, open, and inventory commands now use the selected runtime host, fixing sandboxed LaunchServices failures; launch/open preserve `--no-focus` and caller-relative app paths, relaunch preflights and keeps quit/wait/launch in one daemon-held transaction, build-scoped fallback daemons remain reusable and controllable across native/Rosetta execution and executable upgrades, incompatible legacy hosts no longer force sandboxed local fallback, and inventory ignores unrelated input overrides.
- Agent, MCP, script, CLI, and bridge mutations now advance implicit-snapshot watermarks at host-confirmed completion or observation boundaries, keep durable pending barriers across client timeouts/disconnects without hiding the acting command's own snapshot, carry remote script observation certificates, recover safely from PID reuse, ignore unavailable alternate hosts after protecting the selected/local stores, and preserve explicit snapshot history.
## [3.5.2] - 2026-06-13
### Changed
- `peekaboo type` and the MCP `type` tool now default to zero-delay linear typing; supplying `--wpm`/`wpm` still opts into human cadence.
### Fixed
- Synchronized Tachikoma's OpenAI `gpt-5-chat-latest` catalog metadata so configured models apply the correct GPT-5 parameter filtering.
## [3.5.1] - 2026-06-12
### Fixed
- `peekaboo see` now returns at its configured wall-clock deadline when suspended capture or detection work ignores task cancellation, while preserving explicit command cancellation.
## [3.5.0] - 2026-06-12
### Added
- `peekaboo agent` now supports explicit Claude Fable 5 (`claude-fable-5`) selection with 1M context and 128K max output while keeping Anthropic defaults on Opus 4.8 for zero-retention compatibility.
### Changed
- Agent runs now honor the saved `agent.temperature` and `agent.maxTokens` values shared by the CLI and macOS Settings UI, clamp them to each provider's capabilities, infer Fable limits through compatible providers, and omit unsupported sampling parameters for GPT-5 and current Anthropic reasoning models.
- Project, issue, build, release, and app About links now use the canonical `openclaw/Peekaboo` repository.
### Fixed
- Bridge hosts now use atomic lease-backed socket ownership and bounded nonblocking transport, keep Peekaboo.app and the reusable daemon on distinct paths while preserving the healthy app's TCC-backed fallback, preserve lifecycle settings while migrating legacy daemons, prevent MCP from hosting a bridge listener, safely recover stale sockets, and release abandoned client connections instead of wedging. Thanks @Artifact-LV for #184.
- Legacy screen and area capture now fails with a permission or native capture error instead of returning wallpaper-only/redacted pixels from background sessions. Thanks @VishalJ99 for #185.
## [3.4.1] - 2026-06-10
### Fixed
- `peekaboo agent` now resolves saved custom providers, xAI/Grok, Gemini 3.5 Flash, Claude Opus 4.8, and GPT-5.5 model selections before falling back to unavailable built-in defaults. Thanks @udiedrichsen for #182.
## [3.4.0] - 2026-06-07
## [3.3.0] - 2026-06-01
## [3.2.3] - 2026-05-24
## [3.2.2] - 2026-05-22
### Fixed
- `peekaboo agent` now accepts OpenRouter model IDs and can use `OPENROUTER_API_KEY` from env or credentials. Thanks @delort for #155.
## [3.2.1] - 2026-05-18
### Fixed
- `peekaboo click --coords` now treats coordinates as target-window-relative when app/window target flags are supplied, reports resolved target metadata, and requires `--global-coords` for targeted global clicks.
- `peekaboo-mcp` now shuts down cleanly during restart backoff and repairs executable permissions without shelling out through an install path.
- `pnpm run peekaboo:dev` no longer depends on a hardcoded local checkout path.
- `peekaboo agent` now tells models to use the current tool schema instead of stale tool names and arguments. Thanks @vyctorbrzezowski for #139.
- AX element detection now honors traversal budgets and reports truncation when depth, count, or per-node child limits are reached. Thanks @vyctorbrzezowski for #140.
- `peekaboo agent` and MCP clients now have an `inspect_ui` tool for AX-only UI text/control inspection without capturing screenshots. Thanks @vyctorbrzezowski for #141.
- Window-mode capture now falls back to desktop-independent ScreenCaptureKit filters when multi-display setups cannot map a window to an enumerated display. Thanks @lonexreb for #147.
- `peekaboo agent` guidance now routes AX-only observation through `inspect_ui` consistently while keeping screenshot-backed checks on `see`. Thanks @vyctorbrzezowski for #144.
- Custom provider docs, CLI help, and macOS settings now prefer `${VAR}` API key references and shell examples that preserve them literally. Thanks @scotthuang for #142.
- `peekaboo agent` now refreshes desktop context before each model turn and wires opt-in action verification through the configured capture strategy. Thanks @lonexreb for #148.
- AX traversal budgets now have wider defaults plus CLI, MCP, and environment overrides for complex app trees. Thanks @widdowson for #150 and #151.
- `peekaboo agent` now keeps OAuth access tokens on Bearer auth paths instead of misclassifying them as API keys, including config-dir overrides and audio transcription. Thanks @Crux0453 for #154.
## [3.2.0] - 2026-05-15
### Fixed
- Release automation now verifies CLI, npm, macOS app, checksum, appcast, and uploaded GitHub assets before publish.
- `peekaboo type --json` now separates requested text from executed key actions, making escaped special keys such as `\n` visible to agents without losing backwards-compatible `typedText`.
- `peekaboo permissions status --all-sources` now compares Bridge and local TCC permission state side by side, so daemon grants are no longer confused with CLI grants.
- `peekaboo mcp serve --transport ...` now rejects invalid transport names instead of silently starting stdio mode.
- `peekaboo paste --app ...` now fails before mutating the clipboard when the requested app cannot be found.
- `peekaboo agent` no longer sends stale Anthropic extended-thinking options to Claude Opus 4.7 and now exits with failure when agent execution fails.
- Command timeout JSON now reports the intended timeout error instead of occasionally surfacing cancellation as an unknown error.
- Refreshed CLI docs and quickstart examples to use current flags such as `image --path`, `click --coords`, `type --return`, `press --count`, and `scroll --amount`.
### Performance
- Debug CLI startup no longer spawns `git config` on every launch when build-staleness checking is disabled, cutting startup-heavy command latency by more than 30% in local testing.
## [3.1.2] - 2026-05-11
### Fixed
- Release automation now writes artifacts under `build/release` so clean release builds no longer embed `-dirty` in CLI version metadata.
## [3.1.1] - 2026-05-11
### Added
- `peekaboo image --path -` now writes a single captured image to stdout for shell pipelines.
- The npm package now allows Intel Macs when shipping the universal CLI binary.
### Fixed
- Agent tool schemas now preserve MCP `anyOf`/`oneOf` parameters so Gemini no longer rejects `peekaboo agent` requests with orphan `required` entries.
- `peekaboo see --capture-engine cg` now keeps frontmost/window captures on the CoreGraphics path instead of falling through to `SCScreenshotManager`.
## [3.1.0] - 2026-05-10
### Added
- `peekaboo agent --model` now understands GPT-5.5 and Claude Opus 4.7 identifiers, defaults to `gpt-5.5`, and rejects old GPT/Claude model families.
- Automation-oriented CLI commands now auto-start a warm Peekaboo daemon, reuse it across bursty invocations, and let it exit after an idle timeout.
- Bridge protocol 1.5 adds a daemon-side desktop observation operation so screenshot and `see` flows can execute fully in the warm daemon while returning compact metadata.
### Fixed
- MCP stdio servers now default to the local runtime instead of probing an existing Bridge host, avoiding recursive capture timeouts for `see` and `image` tool calls.
- MCP `image` now returns an `isError: true` tool result when Screen Recording permission is missing instead of surfacing an internal server error.
- MCP `analyze` now honors configured AI providers and per-call `provider_config` models instead of hardcoding an OpenAI model.
- Peekaboo.app now signs with the AppleEvents automation entitlement so macOS can prompt for Automation permission.
- The CLI bundle metadata and bundled Homebrew formula now advertise the macOS 15 minimum that the SwiftPM package already requires.
- `peekaboo see --annotate` now aligns labels using captured window bounds instead of guessing from the first detected element.
- Window capture on macOS 26 now resolves native Retina scale from `NSScreen.backingScaleFactor` before falling back to ScreenCaptureKit display ratios.
- `peekaboo image --app ... --window-title/--window-index` now captures the resolved window by stable window ID, avoiding mismatches between listed window indexes and ScreenCaptureKit window ordering.
- `peekaboo image --app ...` now prefers titled app windows over untitled helper windows, avoiding blank Chrome captures.
- `peekaboo image --capture-engine` is now accepted by Commander-based live parsing.
- Concurrent ScreenCaptureKit screenshot requests now queue through an in-process and cross-process capture gate instead of racing into continuation leaks or transient TCC-denied failures.
- Concurrent `peekaboo see` calls now queue the local screenshot/detection pipeline across processes, avoiding ReplayKit/ScreenCaptureKit continuation hangs under parallel usage.
- Natural-language automation examples now use `peekaboo agent "..."`.
### Performance
- `peekaboo see`, `image`, UI interaction, window, menu, dock, dialog, and app commands now prefer the warm on-demand daemon by default, avoiding repeated service startup cost across command bursts.
- `peekaboo tools`, `peekaboo list apps`, `peekaboo app list`, and purely local metadata commands still avoid daemon startup. Pass `--bridge-socket` to target a Bridge host explicitly where supported.
- Daemon-backed screenshot and `see` calls now write screenshot artifacts in the daemon and avoid sending image bytes through Bridge JSON, preventing large-payload timeouts and making warm calls substantially faster.
- Capture engine `auto` now tries the CoreGraphics path before ScreenCaptureKit, which makes repeated screenshot calls faster locally and avoids observed ScreenCaptureKit continuation hangs; explicit `--capture-engine modern` still forces ScreenCaptureKit.
- `peekaboo image --app` avoids redundant application/window-count lookups during screenshot setup and skips auto-focus work when the target app is already frontmost.
- `peekaboo image --app` now uses a CoreGraphics-only window selection fast path before falling back to full AX-enriched window enumeration, reducing warm Playground screenshot capture from about 350ms to 290ms.
- `peekaboo image` skips a redundant CLI-side screen-recording preflight and relies on the capture service's permission check, shaving about 8ms from warm one-shot app screenshots.
- `peekaboo see --app` avoids re-focusing the target window when Accessibility already reports the captured window as focused.
- `peekaboo see` avoids recursive AX child-text lookups for elements whose labels cannot use them, reducing Playground element detection from about 201ms to 134ms in local testing.
- `peekaboo see` batches per-element Accessibility descriptor reads and skips avoidable action/editability probes, reducing local Playground element detection from about 205ms to 176ms.
- `peekaboo see` limits expensive AX action and keyboard-shortcut probes to roles that can use them, reducing Playground element detection from about 286ms to roughly 180-190ms in local testing.
- `peekaboo see` skips a redundant CLI-side screen-recording preflight and relies on the capture service's permission check, shaving a fixed TCC probe from screenshot-plus-AX runs.
- `peekaboo see` now keeps AX traversal scoped to the captured window and skips web-content focus probing once a rich native AX tree is already visible, avoiding sibling-window elements and cutting native Playground detection from about 220ms to 130ms.
## [2.0.2] - 2025-07-03
### Fixed
- Actually fixed compatibility with macOS Sequoia 26 by ensuring LC_UUID load command is generated during linking
- The v2.0.1 fix was incomplete - the binary was still missing LC_UUID despite the strip command change
- Added `-Xlinker -random_uuid` to Package.swift to ensure UUID generation
- Verified both x86_64 and arm64 architectures now contain proper LC_UUID load commands
## [2.0.1] - 2025-07-03
### Fixed
- Fixed compatibility with macOS Sequoia 26 (pre-release) by preserving LC_UUID load command during binary stripping
- The strip command now uses the `-u` flag to ensure the LC_UUID load command is retained, which is required by the dynamic linker (dyld) on macOS 26
### Technical Details
- Modified build script to use `strip -Sxu` instead of `strip -Sx` to preserve the LC_UUID load command
- This ensures the binary includes the necessary UUID for debugging, crash reporting, and symbol resolution on newer macOS versions
## [2.0.0] - 2025-07-03
### Added
- **Standalone Swift CLI** - Complete rewrite in Swift for better performance and native macOS integration
- **MCP Server** - Model Context Protocol support for AI assistant integration
- **Multiple Capture Modes**:
- Window capture (single or all windows)
- Screen capture (main or specific display)
- Frontmost window capture
- Multi-window capture from multiple apps
- **AI Vision Analysis** - Analyze screenshots with OpenAI or Ollama directly from Swift CLI
- **Configuration File Support** - JSONC format configuration at `~/.config/peekaboo/config.json` with:
- Environment variable expansion (`${HOME}`, `${OPENAI_API_KEY}`)
- Comments support for better documentation
- Hierarchical settings for AI providers, defaults, and logging
- **Config Command** - New `peekaboo config` subcommand to manage configuration:
- `config init` - Create default configuration file
- `config show` - Display current configuration
- `config edit` - Open configuration in default editor
- `config validate` - Validate configuration syntax
- **Permissions Command** - New `peekaboo list permissions` to check system permissions
- **PID Targeting** - Target applications by process ID with `PID:12345` syntax
- **Homebrew Distribution** - Install via `brew install steipete/tap/peekaboo` for easy installation and updates
- **Comprehensive Test Suite** - 331 tests with 100% pass rate covering all major components
- **DocC Documentation** - Comprehensive API documentation for Swift codebase
### Changed
- Complete architecture redesign separating CLI and MCP server
- Improved performance with native Swift implementation
- Better error handling and permission management
- More intuitive command-line interface following Unix conventions
- Enhanced permission visibility with clear indicators when permissions are missing
- Unified AI provider interface for consistent API across OpenAI and Ollama
- Logger's `setJsonOutputMode` and `clearDebugLogs` methods are now synchronous for better reliability
### Fixed
- Configuration precedence (CLI args > env vars > config file > defaults)
- SwiftLint violations across the codebase
- ImageSaver crash when paths contain invalid characters
- Logger race conditions in test environment
- PermissionErrorDetector now handles all relevant error domains
- Test isolation issues preventing interference between tests
- Various edge cases in error handling and file operations
### Removed
- Node.js CLI (replaced with Swift implementation)
- Legacy screenshot methods
## [1.1.0] - 2024-12-20
### Added
- Initial TypeScript implementation
- Basic screenshot capabilities
- Simple MCP integration
### Changed
- Various bug fixes and improvements
## [1.0.0] - 2024-12-19
### Added
- Initial release
- Basic screenshot functionality

View File

@ -1,127 +0,0 @@
// swift-tools-version: 6.2
import Foundation
import PackageDescription
let packageDirectory = URL(fileURLWithPath: #filePath).deletingLastPathComponent()
let infoPlistPath = ProcessInfo.processInfo.environment["PEEKABOO_CLI_INFO_PLIST_PATH"] ??
packageDirectory.appendingPathComponent("Sources/Resources/Info.plist").path
let concurrencyBaseSettings: [SwiftSetting] = [
.enableExperimentalFeature("StrictConcurrency"),
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("NonisolatedNonsendingByDefault"),
.enableExperimentalFeature("RetroactiveConformances"),
]
let cliConcurrencySettings = concurrencyBaseSettings + [
.defaultIsolation(MainActor.self),
]
let swiftTestingSettings = cliConcurrencySettings + [
.enableExperimentalFeature("SwiftTesting"),
]
let includeAutomationTests = ProcessInfo.processInfo.environment["PEEKABOO_INCLUDE_AUTOMATION_TESTS"] == "true"
var targets: [Target] = [
.target(
name: "PeekabooCLI",
dependencies: [
.product(name: "Commander", package: "Commander"),
.product(name: "MCP", package: "swift-sdk"),
.product(name: "Spinner", package: "Spinner"),
.product(name: "TauTUI", package: "TauTUI"),
.product(name: "PeekabooCore", package: "PeekabooCore"),
.product(name: "PeekabooBridge", package: "PeekabooCore"),
.product(name: "PeekabooVisualizer", package: "PeekabooVisualizer"),
.product(name: "Tachikoma", package: "Tachikoma"),
.product(name: "TachikomaMCP", package: "Tachikoma"),
.product(name: "Swiftdansi", package: "Swiftdansi"),
],
path: "Sources/PeekabooCLI",
swiftSettings: cliConcurrencySettings),
.executableTarget(
name: "peekaboo",
dependencies: [
"PeekabooCLI",
],
path: "Sources/PeekabooExec",
swiftSettings: cliConcurrencySettings,
linkerSettings: [
.unsafeFlags([
"-Xlinker", "-sectcreate",
"-Xlinker", "__TEXT",
"-Xlinker", "__info_plist",
"-Xlinker", infoPlistPath,
// Ensure LC_UUID is generated for macOS 26 compatibility
"-Xlinker", "-random_uuid",
]),
]),
.testTarget(
name: "CoreCLITests",
dependencies: [
"PeekabooCLI",
.product(name: "PeekabooFoundation", package: "PeekabooFoundation"),
.product(name: "PeekabooAutomation", package: "PeekabooCore"),
.product(name: "PeekabooAgentRuntime", package: "PeekabooCore"),
.product(name: "PeekabooCore", package: "PeekabooCore"),
],
path: "Tests/CoreCLITests",
swiftSettings: swiftTestingSettings),
.testTarget(
name: "CLIRuntimeTests",
dependencies: [
"PeekabooCLI",
.product(name: "PeekabooFoundation", package: "PeekabooFoundation"),
.product(name: "Subprocess", package: "swift-subprocess"),
],
path: "Tests/CLIRuntimeTests",
swiftSettings: swiftTestingSettings),
]
if includeAutomationTests {
targets.append(
.testTarget(
name: "CLIAutomationTests",
dependencies: [
"PeekabooCLI",
.product(name: "PeekabooFoundation", package: "PeekabooFoundation"),
.product(name: "PeekabooCore", package: "PeekabooCore"),
.product(name: "PeekabooAgentRuntime", package: "PeekabooCore"),
.product(name: "PeekabooAutomation", package: "PeekabooCore"),
.product(name: "Subprocess", package: "swift-subprocess"),
],
path: "Tests/CLIAutomationTests",
resources: [
.process("__snapshots__"),
],
swiftSettings: swiftTestingSettings)
)
}
let package = Package(
name: "peekaboo",
platforms: [
.macOS(.v15),
],
products: [
.executable(
name: "peekaboo",
targets: ["peekaboo"]),
],
dependencies: [
.package(path: "../../Commander"),
.package(url: "https://github.com/modelcontextprotocol/swift-sdk.git", "0.12.0" ..< "0.13.0"),
.package(url: "https://github.com/dominicegginton/Spinner", from: "2.1.0"),
.package(url: "https://github.com/swiftlang/swift-subprocess.git", from: "0.2.1"),
.package(path: "../../TauTUI"),
.package(path: "../../Core/PeekabooFoundation"),
.package(path: "../../Core/PeekabooVisualizer"),
.package(path: "../../Core/PeekabooCore"),
.package(path: "../../Tachikoma"),
.package(path: "../../Swiftdansi"),
],
targets: targets,
swiftLanguageModes: [.v6])

View File

@ -1 +0,0 @@
# Trigger CI rebuild

View File

@ -1,215 +0,0 @@
import Commander
protocol CommanderSignatureProviding {
static func commanderSignature() -> CommandSignature
}
struct CommanderCommandDescriptor {
let metadata: CommandDescriptor
let type: any ParsableCommand.Type
let subcommands: [CommanderCommandDescriptor]
}
struct CommanderCommandSummary: Codable {
struct Argument: Codable {
let label: String
let help: String?
let isOptional: Bool
}
struct Option: Codable {
let names: [String]
let help: String?
let parsing: String
}
struct Flag: Codable {
let names: [String]
let help: String?
}
let name: String
let abstract: String
let discussion: String?
let arguments: [Argument]
let options: [Option]
let flags: [Flag]
let subcommands: [CommanderCommandSummary]
}
@MainActor
enum CommanderRegistryBuilder {
static func buildDescriptors() -> [CommanderCommandDescriptor] {
CommandRegistry.entries.map { self.buildDescriptor(for: $0.type) }
}
private static var descriptorLookup: [ObjectIdentifier: CommandDescriptor]?
static func descriptor(for type: any ParsableCommand.Type) -> CommandDescriptor? {
if let cached = self.descriptorLookup {
return cached[ObjectIdentifier(type)]
}
let lookup = self.buildDescriptorLookup()
self.descriptorLookup = lookup
return lookup[ObjectIdentifier(type)]
}
static func buildCommandSummaries() -> [CommanderCommandSummary] {
self.buildDescriptors().map { CommanderCommandSummary(descriptor: $0) }
}
private static func buildDescriptorLookup() -> [ObjectIdentifier: CommandDescriptor] {
var lookup: [ObjectIdentifier: CommandDescriptor] = [:]
func register(_ descriptor: CommanderCommandDescriptor) {
lookup[ObjectIdentifier(descriptor.type)] = descriptor.metadata
descriptor.subcommands.forEach(register)
}
self.buildDescriptors().forEach(register)
return lookup
}
static func buildDescriptor(for type: any ParsableCommand.Type) -> CommanderCommandDescriptor {
let description = type.commandDescription
let commandInstance = type.init()
let signature = self.resolveSignature(for: type, instance: commandInstance)
.flattened()
.withPeekabooRuntimeFlags()
let childDescriptors = description.subcommands.map { self.buildDescriptor(for: $0) }
let defaultName = description.defaultSubcommand.map { self.commandName(for: $0) }
let metadata = CommandDescriptor(
name: commandName(for: type),
abstract: description.abstract,
discussion: description.discussion,
signature: signature,
subcommands: childDescriptors.map(\.metadata),
defaultSubcommandName: defaultName
)
return CommanderCommandDescriptor(metadata: metadata, type: type, subcommands: childDescriptors)
}
private static func commandName(for type: any ParsableCommand.Type) -> String {
if let explicit = type.commandDescription.commandName {
return explicit
}
return String(describing: type)
}
private static func resolveSignature(
for type: any ParsableCommand.Type,
instance: any ParsableCommand
) -> CommandSignature {
if let provider = type as? any CommanderSignatureProviding.Type {
return provider.commanderSignature()
}
return CommandSignature.describe(instance)
}
}
extension CommanderCommandSummary {
fileprivate init(descriptor: CommanderCommandDescriptor) {
let signature = descriptor.metadata.signature
self.name = descriptor.metadata.name
self.abstract = descriptor.metadata.abstract
self.discussion = descriptor.metadata.discussion
self.arguments = signature.arguments.map { argument in
Argument(
label: argument.label,
help: argument.help,
isOptional: argument.isOptional
)
}
self.options = signature.options.map { option in
Option(
names: option.names
.filter { !$0.isAlias }
.map(\.cliSpelling),
help: option.help,
parsing: option.parsing.displayName
)
}
self.flags = signature.flags.map { flag in
Flag(
names: flag.names
.filter { !$0.isAlias }
.map(\.cliSpelling),
help: flag.help
)
}
self.subcommands = descriptor.subcommands.map { CommanderCommandSummary(descriptor: $0) }
}
}
extension OptionDefinition {
nonisolated static func commandOption(
_ label: String,
help: String? = nil,
long: String? = nil,
short: Character? = nil,
parsing: OptionParsingStrategy = .singleValue
) -> OptionDefinition {
var names: [CommanderName] = []
if let short {
names.append(.short(short))
}
names.append(.long(long ?? label.commanderized()))
return OptionDefinition.make(label: label, names: names, help: help, parsing: parsing)
}
}
extension FlagDefinition {
nonisolated static func commandFlag(
_ label: String,
help: String? = nil,
long: String? = nil,
short: Character? = nil
) -> FlagDefinition {
var names: [CommanderName] = []
if let short {
names.append(.short(short))
}
names.append(.long(long ?? label.commanderized()))
return FlagDefinition.make(label: label, names: names, help: help)
}
}
extension String {
fileprivate nonisolated func commanderized() -> String {
guard !isEmpty else { return self }
var scalars: [Character] = []
for character in self {
if character.isUppercase {
scalars.append("-")
scalars.append(Character(character.lowercased()))
} else {
scalars.append(character)
}
}
return String(scalars)
}
}
extension CommanderName {
fileprivate var cliSpelling: String {
switch self {
case let .short(value), let .aliasShort(value):
"-\(value)"
case let .long(value), let .aliasLong(value):
"--\(value)"
}
}
}
extension OptionParsingStrategy {
fileprivate var displayName: String {
switch self {
case .singleValue:
"singleValue"
case .upToNextOption:
"upToNextOption"
case .remaining:
"remaining"
}
}
}

View File

@ -1,256 +0,0 @@
import Commander
import Foundation
import PeekabooAutomationKit
/// Commands or runtime contexts that can specify a preferred capture engine.
protocol CaptureEngineConfigurable: AnyObject {
var captureEngine: String? { get }
}
enum CommanderRuntimeExecutorMessage {
static let snapshotInvalidationWarning =
"Warning: The requested action succeeded, but stale UI snapshots could not be invalidated after retry. " +
"Do not retry the action."
}
enum CommanderRuntimeExecutorError: LocalizedError {
case snapshotCatchUpFailed(any Error)
case mutationBarrierFailed(any Error)
var errorDescription: String? {
switch self {
case let .snapshotCatchUpFailed(error):
"Could not synchronize the selected host's UI snapshot watermark before execution: " +
"the requested command was not executed, so retrying later is safe. " + error.localizedDescription
case let .mutationBarrierFailed(error):
"Could not establish the desktop mutation barrier before execution: " +
"the requested command was not executed, so retrying later is safe. " + error.localizedDescription
}
}
}
@MainActor
enum CommanderRuntimeExecutor {
static func resolveAndRun(arguments: [String]) async throws {
let resolved = try CommanderRuntimeRouter.resolve(argv: arguments)
try await self.run(resolved: resolved)
}
static func run(resolved: CommanderResolvedCommand) async throws {
let command = try CommanderCLIBinder.instantiateCommand(
type: resolved.type,
parsedValues: resolved.parsedValues
)
if var runtimeCommand = command as? any AsyncRuntimeCommand {
let runtimeOptions = try CommanderCLIBinder.makeRuntimeOptions(
from: resolved.parsedValues,
commandType: resolved.type
)
if let capturePreference = runtimeOptions.captureEnginePreference,
!capturePreference.isEmpty {
// Respect explicit engine choice; also allow disabling CG globally.
setenv("PEEKABOO_CAPTURE_ENGINE", capturePreference, 1)
}
let runtime = await CommandRuntime.makeDefaultAsync(options: runtimeOptions)
try await self.catchUpSelectedHostIfNeeded(
using: runtime,
required: runtimeOptions.requiresImplicitSnapshotInvalidation ||
runtimeOptions.usesPerToolSnapshotInvalidation
)
try await DeferredCommandOutput.run(
bufferingOutput: runtimeOptions.requiresImplicitSnapshotInvalidation
) {
try await self.runWithImplicitSnapshotInvalidation(
using: runtime,
required: runtimeOptions.requiresImplicitSnapshotInvalidation,
requiresCallerBarrier: runtimeOptions.requiresCallerDesktopMutationBarrier
) {
try await runtimeCommand.run(using: runtime)
}
}
return
}
var plainCommand = command
try await plainCommand.run()
}
static func catchUpSelectedHostIfNeeded(
using runtime: CommandRuntime,
required: Bool
) async throws {
guard required else { return }
try Task.checkCancellation()
let cutoff = runtime.services.snapshots.effectiveImplicitLatestInvalidationWatermark
try Task.checkCancellation()
guard let cutoff else { return }
do {
_ = try await runtime.services.snapshots.invalidateImplicitLatestSnapshot(
through: cutoff,
preserving: nil,
preservedAt: nil
)
try Task.checkCancellation()
} catch let error as CancellationError {
throw error
} catch {
throw CommanderRuntimeExecutorError.snapshotCatchUpFailed(error)
}
}
static func runWithImplicitSnapshotInvalidation<T>(
using runtime: CommandRuntime,
required: Bool,
requiresCallerBarrier: Bool = false,
operation: () async throws -> T
) async throws -> T {
let mutationSequenceAtStart = runtime.interactionMutationTracker.mutationSequence
let needsCallerBarrier = required &&
(runtime.selectedRemoteSocketPath == nil || requiresCallerBarrier)
let createdDurableMutation: Bool
if needsCallerBarrier {
do {
createdDurableMutation = try runtime.interactionMutationTracker.beginDurableMutation()
} catch {
throw CommanderRuntimeExecutorError.mutationBarrierFailed(error)
}
} else {
createdDurableMutation = false
}
let result: T
do {
result = try await runtime.interactionMutationTracker.withPendingDurableMutationVisible(
createdByCurrentCommand: createdDurableMutation,
operation: operation
)
try Task.checkCancellation()
} catch {
_ = await self.invalidateSnapshotsAfterCommandIfNeeded(
using: runtime,
required: required,
succeeded: false,
mutationSequenceAtStart: mutationSequenceAtStart,
createdDurableMutation: createdDurableMutation
)
throw error
}
let hadPendingMutation = required && runtime.interactionMutationTracker.mutationStartedAt != nil
let invalidated = await invalidateSnapshotsAfterCommandIfNeeded(
using: runtime,
required: required,
succeeded: true,
mutationSequenceAtStart: mutationSequenceAtStart,
createdDurableMutation: createdDurableMutation
)
do {
try Task.checkCancellation()
} catch {
if hadPendingMutation {
_ = await self.invalidateSnapshots(
using: runtime,
reason: "command cancellation",
through: Date(),
preserving: nil,
preservedAt: nil
)
}
throw error
}
if !invalidated {
fputs("\(CommanderRuntimeExecutorMessage.snapshotInvalidationWarning)\n", stderr)
}
return result
}
private static func invalidateSnapshotsAfterCommandIfNeeded(
using runtime: CommandRuntime,
required: Bool,
succeeded: Bool,
mutationSequenceAtStart: UInt64,
createdDurableMutation: Bool
) async -> Bool {
let completion = Date()
guard required else { return true }
guard runtime.interactionMutationTracker.mutationStartedAt != nil else {
guard createdDurableMutation else {
return !runtime.interactionMutationTracker.hasPendingDurableMutation
}
do {
try runtime.interactionMutationTracker.cancelDurableMutation()
return true
} catch {
return false
}
}
guard let requestedCutoff = runtime.interactionMutationTracker.invalidationCutoff(
commandCompletedAt: completion,
succeeded: succeeded
)
else { return true }
let durableCompletion: DesktopMutationWatermarkStore.MutationCompletion?
do {
if createdDurableMutation,
runtime.interactionMutationTracker.mutationSequence == mutationSequenceAtStart {
try runtime.interactionMutationTracker.cancelDurableMutation()
durableCompletion = nil
} else {
durableCompletion = try runtime.interactionMutationTracker.completeDurableMutation(
through: succeeded ? requestedCutoff : completion
)
}
} catch {
runtime.interactionMutationTracker.markInvalidationFailed(through: completion)
return false
}
let cutoff = max(requestedCutoff, durableCompletion?.cutoff ?? requestedCutoff)
let preservationAllowed = durableCompletion?.allowsObservationPreservation ?? true
let preservedSnapshotID = succeeded && preservationAllowed
? runtime.interactionMutationTracker.preservedSnapshotID
: nil
let preservedAt = preservedSnapshotID == nil
? nil
: runtime.interactionMutationTracker.preservedAt
return await self.invalidateSnapshots(
using: runtime,
reason: "command execution",
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedAt
)
}
private static func invalidateSnapshots(
using runtime: CommandRuntime,
reason: String,
through cutoff: Date,
preserving preservedSnapshotID: String?,
preservedAt: Date?
) async -> Bool {
let targets = runtime.interactionMutationTargets
let isRetry = runtime.interactionMutationTracker.hasFailedInvalidationAttempt
let invalidated = await InteractionObservationInvalidator.invalidateAfterMutation(
targets: targets,
logger: runtime.logger,
reason: reason,
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedAt
)
if invalidated {
return true
}
if isRetry {
return false
}
return await InteractionObservationInvalidator.invalidateAfterMutation(
targets: targets,
logger: runtime.logger,
reason: "\(reason) retry",
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedAt
)
}
}

View File

@ -1,196 +0,0 @@
import Commander
import Foundation
extension CommanderRuntimeRouter {
static let categoryLookup: [ObjectIdentifier: CommandRegistryEntry.Category] = {
var lookup: [ObjectIdentifier: CommandRegistryEntry.Category] = [:]
for entry in CommandRegistry.entries {
lookup[ObjectIdentifier(entry.type)] = entry.category
}
return lookup
}()
static func makeHelpTheme() -> HelpTheme {
let capabilities = TerminalDetector.detectCapabilities()
if let forcedMode = TerminalDetector.shouldForceOutputMode() {
return HelpTheme(useColors: forcedMode.supportsColors)
}
return HelpTheme(useColors: capabilities.supportsColors)
}
static func renderRootUsageCard(theme: HelpTheme) -> String {
var lines: [String] = []
lines.append(theme.heading("Usage"))
lines.append(" \(theme.accent("peekaboo <command> [options]"))")
lines.append("")
lines.append(theme.heading("Tip"))
lines.append(" When developing locally, run via \(theme.accent("polter peekaboo")) to ensure fresh builds.")
return lines.joined(separator: "\n")
}
static func renderUsageCard(
for descriptor: CommanderCommandDescriptor,
path: [String],
theme: HelpTheme
) -> String {
let usageLine = self.buildUsageLine(path: path, signature: descriptor.metadata.signature)
var lines: [String] = []
lines.append(theme.heading("Usage"))
lines.append(" \(theme.accent(usageLine))")
let abstract = descriptor.metadata.abstract.trimmingCharacters(in: .whitespacesAndNewlines)
if !abstract.isEmpty {
lines.append("")
lines.append(theme.heading("Summary"))
lines.append(" \(abstract)")
}
lines.append("")
lines.append(theme.heading("Tip"))
lines.append(" When developing locally, run via \(theme.accent("polter peekaboo")) to ensure fresh builds.")
return lines.joined(separator: "\n")
}
static func globalFlagSummaries(theme: HelpTheme) -> [String] {
[
theme.bullet(label: "--json/-j (alias: --json-output)", description: "Emit machine-readable JSON output"),
theme.bullet(label: "--verbose/-v", description: "Enable verbose logging"),
theme.bullet(
label: "--log-level <level>",
description: "trace | verbose | debug | info | warning | error | critical"
),
theme.bullet(
label: "--no-remote",
description: "Force local services; skip remote bridge hosts even if available"
),
theme.bullet(
label: "--bridge-socket <path>",
description: "Override the Peekaboo Bridge socket path"
),
theme.bullet(
label: "--input-strategy <mode>",
description: "Override UI input strategy: actionFirst | synthFirst | actionOnly | synthOnly"
)
]
}
static func renderGlobalFlagsSection(theme: HelpTheme) -> String {
var lines: [String] = []
lines.append(theme.heading("Global Runtime Flags"))
for entry in self.globalFlagSummaries(theme: theme) {
lines.append(" \(entry)")
}
return lines.joined(separator: "\n")
}
static func renderCommandList(
for commands: [CommanderCommandDescriptor],
theme: HelpTheme,
indent: String = " "
) -> [String] {
let sorted = commands.sorted { $0.metadata.name < $1.metadata.name }
let maxNameLength = sorted.map(\.metadata.name.count).max() ?? 0
let columnWidth = min(max(maxNameLength, 8), 24)
return sorted.map { descriptor in
let name = descriptor.metadata.name
let summary = descriptor.metadata.abstract.isEmpty ? "No description provided." : descriptor.metadata
.abstract
let paddedName: String = if name.count >= columnWidth {
name
} else {
name + String(repeating: " ", count: columnWidth - name.count)
}
let displayName = theme.command(paddedName)
return "\(indent)\(displayName) \(summary)"
}
}
static func buildUsageLine(path: [String], signature: CommandSignature) -> String {
var tokens = ["peekaboo"]
let commandPath = path.isEmpty ? ["<command>"] : path
tokens.append(contentsOf: commandPath)
for argument in signature.arguments {
let placeholder = self.argumentPlaceholder(for: argument)
tokens.append(argument.isOptional ? "[\(placeholder)]" : "<\(placeholder)>")
}
if !signature.options.isEmpty || !signature.flags.isEmpty {
tokens.append("[options]")
}
return tokens.joined(separator: " ")
}
static func argumentPlaceholder(for argument: ArgumentDefinition) -> String {
let lowered = argument.label.replacingOccurrences(of: "_", with: "-")
return Self.kebabCased(lowered)
}
static func kebabCased(_ value: String) -> String {
guard !value.isEmpty else { return value }
var scalars: [Character] = []
for character in value {
if character.isUppercase {
if !scalars.isEmpty && scalars.last != "-" {
scalars.append("-")
}
scalars.append(contentsOf: character.lowercased())
} else if character == " " || character == "-" {
if scalars.last != "-" { scalars.append("-") }
} else {
scalars.append(character)
}
}
return String(scalars)
}
}
struct HelpTheme {
let useColors: Bool
func heading(_ text: String) -> String {
guard self.useColors else { return text }
return "\(TerminalColor.bold)\(TerminalColor.cyan)\(text)\(TerminalColor.reset)"
}
func accent(_ text: String) -> String {
guard self.useColors else { return text }
return "\(TerminalColor.magenta)\(text)\(TerminalColor.reset)"
}
func command(_ text: String) -> String {
guard self.useColors else { return text }
return "\(TerminalColor.bold)\(text)\(TerminalColor.reset)"
}
func dim(_ text: String) -> String {
guard self.useColors else { return text }
return "\(TerminalColor.gray)\(text)\(TerminalColor.reset)"
}
func bullet(label: String, description: String) -> String {
let prefix = self.useColors ? "\(TerminalColor.gray)\(TerminalColor.reset)" : "-"
let labelText = self.useColors ? "\(TerminalColor.bold)\(label)\(TerminalColor.reset)" : label
return "\(prefix) \(labelText) \(description)"
}
}
extension CommandRegistryEntry.Category {
var displayName: String {
switch self {
case .core:
"Core Commands"
case .interaction:
"Interaction"
case .system:
"System"
case .vision:
"Vision"
case .ai:
"AI"
case .mcp:
"MCP"
}
}
}

View File

@ -1,241 +0,0 @@
import Commander
import Foundation
struct CommanderResolvedCommand {
let metadata: CommandDescriptor
let type: any ParsableCommand.Type
let parsedValues: ParsedValues
}
@MainActor
enum CommanderRuntimeRouter {
static func resolve(argv: [String]) throws -> CommanderResolvedCommand {
let descriptors = CommanderRegistryBuilder.buildDescriptors()
let trimmedArgs = Self.trimmedArguments(from: argv)
if trimmedArgs.isEmpty {
self.printRootHelp(descriptors: descriptors)
throw ExitCode.success
}
if Self.handleVersionRequest(arguments: trimmedArgs) {
throw ExitCode.success
}
if try Self.handleBareInvocation(arguments: trimmedArgs, descriptors: descriptors) {
throw ExitCode.success
}
if try Self.handleHelpRequest(arguments: trimmedArgs, descriptors: descriptors) {
throw ExitCode.success
}
if let alias = try Self.resolveAgentPermissionAlias(arguments: trimmedArgs, originalArgv: argv) {
return alias
}
let program = Program(descriptors: descriptors.map(\.metadata))
let invocation = try program.resolve(argv: argv)
guard let descriptor = Self.findDescriptor(in: descriptors, matching: invocation.path) else {
throw CommanderProgramError.unknownCommand(invocation.path.joined(separator: ":"))
}
return CommanderResolvedCommand(
metadata: descriptor.metadata,
type: descriptor.type,
parsedValues: invocation.parsedValues
)
}
private static func findDescriptor(
in descriptors: [CommanderCommandDescriptor],
matching path: [String]
) -> CommanderCommandDescriptor? {
guard let head = path.first else { return nil }
guard let match = descriptors.first(where: { $0.metadata.name == head }) else {
return nil
}
guard path.count > 1 else {
return match
}
let remainder = Array(path.dropFirst())
return self.findDescriptor(in: match.subcommands, matching: remainder)
}
private static func trimmedArguments(from argv: [String]) -> [String] {
guard !argv.isEmpty else { return [] }
var args = argv
if args[0].hasSuffix("peekaboo") {
args.removeFirst()
}
return args
}
private static func handleHelpRequest(
arguments: [String],
descriptors: [CommanderCommandDescriptor]
) throws -> Bool {
guard !arguments.isEmpty else { return false }
if arguments[0].caseInsensitiveCompare("help") == .orderedSame {
let tokens = Array(arguments.dropFirst())
if self.handleAgentPermissionHelp(tokens: tokens) {
return true
}
let path = self.resolveHelpPath(from: tokens, descriptors: descriptors)
try self.printHelp(for: path, descriptors: descriptors)
return true
}
let helpSearchArguments = Array(arguments.prefix { $0 != "--" })
if let index = helpSearchArguments.firstIndex(where: { self.isHelpToken($0) }) {
let tokens = Array(helpSearchArguments.prefix(index))
if self.handleAgentPermissionHelp(tokens: tokens) {
return true
}
let path = self.resolveHelpPath(from: tokens, descriptors: descriptors)
try self.printHelp(for: path, descriptors: descriptors)
return true
}
return false
}
private static func handleAgentPermissionHelp(tokens: [String]) -> Bool {
guard tokens.count >= 2,
tokens[0].caseInsensitiveCompare("agent") == .orderedSame,
tokens[1].caseInsensitiveCompare("permission") == .orderedSame else {
return false
}
let rootDescriptor = CommanderRegistryBuilder.buildDescriptor(for: PermissionCommand.self)
let permissionPath = ["permission"] + tokens.dropFirst(2)
guard let descriptor = self.findDescriptor(in: [rootDescriptor], matching: permissionPath) else {
return false
}
self.printCommandHelp(descriptor, path: ["agent"] + permissionPath)
return true
}
private static func resolveAgentPermissionAlias(
arguments: [String],
originalArgv: [String]
) throws -> CommanderResolvedCommand? {
guard arguments.count >= 2,
arguments[0].caseInsensitiveCompare("agent") == .orderedSame,
arguments[1].caseInsensitiveCompare("permission") == .orderedSame else {
return nil
}
let rootDescriptor = CommanderRegistryBuilder.buildDescriptor(for: PermissionCommand.self)
let executable = originalArgv.first ?? "peekaboo"
let aliasArgv = [executable, "permission"] + arguments.dropFirst(2)
let program = Program(descriptors: [rootDescriptor.metadata])
let invocation = try program.resolve(argv: Array(aliasArgv))
guard let descriptor = self.findDescriptor(in: [rootDescriptor], matching: invocation.path) else {
throw CommanderProgramError.unknownCommand(invocation.path.joined(separator: ":"))
}
return CommanderResolvedCommand(
metadata: descriptor.metadata,
type: descriptor.type,
parsedValues: invocation.parsedValues
)
}
private static func resolveHelpPath(
from tokens: [String],
descriptors: [CommanderCommandDescriptor]
) -> [String] {
guard !tokens.isEmpty else { return [] }
for length in stride(from: tokens.count, through: 1, by: -1) {
let candidate = Array(tokens.prefix(length))
if self.findDescriptor(in: descriptors, matching: candidate) != nil {
return candidate
}
}
// Preserve previous behavior for unknown paths: let printHelp throw with the original tokens.
return tokens
}
private static func handleVersionRequest(arguments: [String]) -> Bool {
guard let first = arguments.first else { return false }
guard self.isVersionToken(first) else { return false }
print(Version.fullVersion)
return true
}
private static func handleBareInvocation(
arguments: [String],
descriptors: [CommanderCommandDescriptor]
) throws -> Bool {
guard arguments.count == 1 else { return false }
let token = arguments[0]
guard let descriptor = descriptors.first(where: { $0.metadata.name == token }) else {
return false
}
let description = descriptor.type.commandDescription
guard description.showHelpOnEmptyInvocation else { return false }
self.printCommandHelp(descriptor, path: [token])
if !descriptor.metadata.subcommands.isEmpty {
throw CommanderProgramError.missingSubcommand(command: token)
}
return true
}
private static func isHelpToken(_ token: String) -> Bool {
token == "--help" || token == "-h"
}
private static func isVersionToken(_ token: String) -> Bool {
token == "--version" || token == "-V"
}
private static func printHelp(
for path: [String],
descriptors: [CommanderCommandDescriptor]
) throws {
if path.isEmpty {
self.printRootHelp(descriptors: descriptors)
return
}
guard let descriptor = self.findDescriptor(in: descriptors, matching: path) else {
throw CommanderProgramError.unknownCommand(path.joined(separator: " "))
}
self.printCommandHelp(descriptor, path: path)
}
private static func printRootHelp(descriptors: [CommanderCommandDescriptor]) {
let theme = self.makeHelpTheme()
print(self.renderRootUsageCard(theme: theme))
print("")
let groupedByCategory = Dictionary(grouping: descriptors) { descriptor in
Self.categoryLookup[ObjectIdentifier(descriptor.type)] ?? .core
}
for category in CommandRegistryEntry.Category.allCases {
guard let commands = groupedByCategory[category], !commands.isEmpty else { continue }
print(theme.heading(category.displayName))
let rows = self.renderCommandList(for: commands, theme: theme)
rows.forEach { print($0) }
print("")
}
print(self.renderGlobalFlagsSection(theme: theme))
print("")
print(theme.dim("Use `peekaboo help <command>` or `peekaboo <command> --help` for detailed options."))
}
private static func printCommandHelp(_ descriptor: CommanderCommandDescriptor, path: [String]) {
let theme = self.makeHelpTheme()
let usageCard = self.renderUsageCard(for: descriptor, path: path, theme: theme)
let helpText = CommandHelpRenderer.renderHelp(for: descriptor.type, theme: theme)
print(usageCard)
print("")
print(helpText)
print("")
print(self.renderGlobalFlagsSection(theme: theme))
guard !descriptor.subcommands.isEmpty else { return }
print("\nSubcommands:")
let subcommandRows = self.renderCommandList(for: descriptor.subcommands, theme: theme)
subcommandRows.forEach { print($0) }
if let defaultName = descriptor.metadata.defaultSubcommandName {
print("\nDefault subcommand: \(theme.command(defaultName))")
}
}
}

View File

@ -1,226 +0,0 @@
import Foundation
/// Renders a self-contained bash completion script that queries shared
/// completion tables emitted from Swift metadata.
struct BashCompletionRenderer: ShellCompletionRendering {
func render(document: CompletionScriptDocument) -> String {
let lines = self.commonHeader(
shell: "bash",
install: CompletionsCommand.Shell.bash.installationSnippet
) + [
"__peekaboo_bash_subcommands() {",
self.renderBashChoiceSwitch(document.pathsIncludingRoot, accessor: \.subcommands),
"}",
"",
"__peekaboo_bash_options() {",
self.renderBashOptionSwitch(document: document),
"}",
"",
"__peekaboo_bash_argument_values() {",
self.renderBashArgumentSwitch(document.pathsIncludingRoot),
"}",
"",
"__peekaboo_bash_option_values() {",
self.renderBashOptionValueSwitch(document.pathsIncludingRoot),
"}",
"",
"__peekaboo_bash_has_subcommand() {",
" local path=\"$1\"",
" local candidate=\"$2\"",
" while IFS=$'\\t' read -r value _; do",
" [[ \"$value\" == \"$candidate\" ]] && return 0",
" done < <(__peekaboo_bash_subcommands \"$path\")",
" return 1",
"}",
"",
"__peekaboo_bash_complete() {",
" local cur=\"${COMP_WORDS[COMP_CWORD]}\"",
" local path=\"\"",
" local index=1",
" local previous=\"\"",
" local token",
" COMPREPLY=()",
"",
" while (( index < COMP_CWORD )); do",
" token=\"${COMP_WORDS[index]}\"",
" [[ \"$token\" == -* ]] && break",
" if __peekaboo_bash_has_subcommand \"$path\" \"$token\"; then",
" path=\"${path:+$path }$token\"",
" (( index++ ))",
" else",
" break",
" fi",
" done",
"",
" if (( COMP_CWORD > 0 )); then",
" previous=\"${COMP_WORDS[COMP_CWORD - 1]}\"",
" fi",
"",
" local option_values",
" option_values=\"$(__peekaboo_bash_option_values \"$path\" \"$previous\" | cut -f1 | tr '\\n' ' ')\"",
" if [[ -n \"$option_values\" ]]; then",
" COMPREPLY=($(compgen -W \"$option_values\" -- \"$cur\"))",
" return",
" fi",
"",
" if [[ \"$cur\" == -* ]]; then",
" local option_names",
" option_names=\"$(__peekaboo_bash_options \"$path\" | cut -f1 | tr '\\n' ' ')\"",
" COMPREPLY=($(compgen -W \"$option_names\" -- \"$cur\"))",
" return",
" fi",
"",
" local subcommands",
" subcommands=\"$(__peekaboo_bash_subcommands \"$path\" | cut -f1 | tr '\\n' ' ')\"",
" if [[ -n \"$subcommands\" ]]; then",
" COMPREPLY=($(compgen -W \"$subcommands\" -- \"$cur\"))",
" return",
" fi",
"",
" local argument_index=$(( COMP_CWORD - index ))",
" local values",
" values=\"$(__peekaboo_bash_argument_values \"$path\" \"$argument_index\" | cut -f1 | tr '\\n' ' ')\"",
" if [[ -n \"$values\" ]]; then",
" COMPREPLY=($(compgen -W \"$values\" -- \"$cur\"))",
" fi",
"}",
"",
"complete -F __peekaboo_bash_complete \(document.commandName)",
]
return lines.joined(separator: "\n")
}
private func renderBashChoiceSwitch(
_ paths: [CompletionPath],
accessor: KeyPath<CompletionPath, [CompletionChoice]>
) -> String {
var lines = [" case \"$1\" in"]
lines.append(contentsOf: self.renderCases(paths: paths) { path in
path[keyPath: accessor].map { choice in
self.tabSeparated(choice.value, choice.help)
}
})
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderBashOptionSwitch(document: CompletionScriptDocument) -> String {
var lines = [" case \"$1\" in", " '')"]
lines.append(contentsOf: self.heredocLines(items: document.rootOptions.map { option in
option.names.map { name in
self.tabSeparated(name, option.help)
}
}.flatMap(\.self), indent: " "))
lines.append(" ;;")
lines.append(contentsOf: self.renderCases(paths: document.flattenedPaths) { path in
path.options.flatMap { option in
option.names.map { name in
self.tabSeparated(name, option.help)
}
}
})
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderBashArgumentSwitch(_ paths: [CompletionPath]) -> String {
var lines = [" case \"$1:$2\" in"]
for path in paths {
for (index, argument) in path.arguments.enumerated() where !argument.choices.isEmpty {
lines.append(" '\(self.caseLabel(path.key)):\(index)')")
lines.append(contentsOf: self.heredocLines(items: argument.choices.map {
self.tabSeparated($0.value, $0.help)
}, indent: " "))
lines.append(" ;;")
}
}
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderBashOptionValueSwitch(_ paths: [CompletionPath]) -> String {
var lines = [" case \"$1:$2\" in"]
for path in paths {
for option in path.options where !option.valueChoices.isEmpty {
for name in option.names {
lines.append(" '\(self.caseLabel(path.key)):\(self.caseLabel(name))')")
lines.append(contentsOf: self.heredocLines(items: option.valueChoices.map {
self.tabSeparated($0.value, $0.help)
}, indent: " "))
lines.append(" ;;")
}
}
}
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderCases(
paths: [CompletionPath],
content: (CompletionPath) -> [String]
) -> [String] {
paths.map { path in
let items = content(path)
if items.isEmpty {
return [
" '\(self.caseLabel(path.key))')",
" ;;",
]
}
return [
" '\(self.caseLabel(path.key))')",
] + self.heredocLines(items: items, indent: " ") + [
" ;;",
]
}.flatMap(\.self)
}
private func heredocLines(items: [String], indent: String) -> [String] {
guard !items.isEmpty else { return [] }
return [
"\(indent)cat <<'EOF'",
] + items + [
"EOF",
]
}
private func tabSeparated(_ value: String, _ help: String?) -> String {
let tab = "\t"
let description = (help ?? "").replacingOccurrences(of: "\t", with: " ").replacingOccurrences(
of: "\n",
with: " "
)
return "\(value)\(tab)\(description)"
}
private func caseLabel(_ label: String) -> String {
label.replacingOccurrences(of: "'", with: "'\\''")
}
private func commonHeader(shell: String, install: String) -> [String] {
[
"# \(shell.capitalized) completion for peekaboo",
"# Generated from Commander descriptors via `peekaboo completions \(shell)`.",
"# Install with:",
"# \(install)",
"",
]
}
}

View File

@ -1,286 +0,0 @@
import Commander
import Foundation
/// Shell-completion document rendered from Commander metadata.
///
/// `CompletionScriptDocument` is the single source of truth for completion
/// generation. It is derived from `CommanderCommandDescriptor` values, which are
/// already the canonical source for help output and command discovery.
struct CompletionScriptDocument {
let commandName: String
let commands: [CompletionCommand]
let rootOptions: [CompletionOption]
var topLevelChoices: [CompletionChoice] {
self.commands.map { command in
CompletionChoice(value: command.name, help: command.abstract)
}
}
var flattenedPaths: [CompletionPath] {
self.commands.flatMap { command in
command.flattenedPaths(prefix: [])
}
}
var pathsIncludingRoot: [CompletionPath] {
[
CompletionPath(
path: [],
subcommands: self.topLevelChoices,
options: self.rootOptions,
arguments: []
),
] + self.flattenedPaths
}
static func make(
commandName: String = "peekaboo",
descriptors: [CommanderCommandDescriptor]
) -> CompletionScriptDocument {
let commands = descriptors
.sorted { $0.metadata.name < $1.metadata.name }
.map { CompletionCommand(descriptor: $0, path: [$0.metadata.name]) }
let helpMirror = CompletionCommand.helpMirror(commands: commands)
return CompletionScriptDocument(
commandName: commandName,
commands: [helpMirror] + commands,
rootOptions: [
.flag(names: ["-h", "--help"], help: "Show help information"),
.flag(names: ["-V", "--version"], help: "Show version information"),
]
)
}
}
struct CompletionCommand {
let name: String
let abstract: String
let arguments: [CompletionArgument]
let options: [CompletionOption]
let subcommands: [CompletionCommand]
var subcommandChoices: [CompletionChoice] {
self.subcommands.map { command in
CompletionChoice(value: command.name, help: command.abstract)
}
}
init(descriptor: CommanderCommandDescriptor, path: [String]) {
self.name = descriptor.metadata.name
self.abstract = descriptor.metadata.abstract
self.arguments = descriptor.metadata.signature.arguments.enumerated().map { index, argument in
CompletionArgument(
label: argument.label,
isOptional: argument.isOptional,
choices: CompletionValueCatalog.argumentChoices(for: path, index: index, label: argument.label)
)
}
self.options = Self.makeOptions(from: descriptor.metadata.signature, path: path)
self.subcommands = descriptor.subcommands
.sorted { $0.metadata.name < $1.metadata.name }
.map { subcommand in
CompletionCommand(descriptor: subcommand, path: path + [subcommand.metadata.name])
}
}
private init(
name: String,
abstract: String,
arguments: [CompletionArgument],
options: [CompletionOption],
subcommands: [CompletionCommand]
) {
self.name = name
self.abstract = abstract
self.arguments = arguments
self.options = options
self.subcommands = subcommands
}
func flattenedPaths(prefix: [String]) -> [CompletionPath] {
let path = prefix + [self.name]
let current = CompletionPath(
path: path,
subcommands: self.subcommandChoices,
options: self.options,
arguments: self.arguments
)
return [current] + self.subcommands.flatMap { subcommand in
subcommand.flattenedPaths(prefix: path)
}
}
static func helpMirror(commands: [CompletionCommand]) -> CompletionCommand {
CompletionCommand(
name: "help",
abstract: "Show help for commands",
arguments: [],
options: [],
subcommands: commands.map { command in
CompletionCommand(
name: command.name,
abstract: command.abstract,
arguments: [],
options: [],
subcommands: self.helpSubcommands(from: command.subcommands)
)
}
)
}
private static func helpSubcommands(from commands: [CompletionCommand]) -> [CompletionCommand] {
commands.map { command in
CompletionCommand(
name: command.name,
abstract: command.abstract,
arguments: [],
options: [],
subcommands: self.helpSubcommands(from: command.subcommands)
)
}
}
private static func makeOptions(from signature: CommandSignature, path: [String]) -> [CompletionOption] {
let flags = signature.flags.map { flag in
CompletionOption.flag(
names: self.uniqueNames(flag.names.map(\.completionSpelling)),
help: flag.help ?? "No description provided"
)
}
let options = signature.options.map { option in
let names = self.uniqueNames(option.names.map(\.completionSpelling))
return CompletionOption.option(
names: names,
valueName: option.label,
help: option.help ?? "No description provided",
valueChoices: CompletionValueCatalog.optionChoices(
for: path,
label: option.label,
names: names
)
)
}
return flags + options + [
.flag(names: ["-h", "--help"], help: "Show help information"),
]
}
private static func uniqueNames(_ names: [String]) -> [String] {
var seen: Set<String> = []
var ordered: [String] = []
for name in names where !seen.contains(name) {
seen.insert(name)
ordered.append(name)
}
return ordered
}
}
struct CompletionPath {
let path: [String]
let subcommands: [CompletionChoice]
let options: [CompletionOption]
let arguments: [CompletionArgument]
var key: String {
self.path.joined(separator: " ")
}
}
struct CompletionArgument {
let label: String
let isOptional: Bool
let choices: [CompletionChoice]
}
struct CompletionOption {
let names: [String]
let help: String
let valueName: String?
let valueChoices: [CompletionChoice]
var takesValue: Bool {
self.valueName != nil
}
static func flag(names: [String], help: String) -> CompletionOption {
CompletionOption(names: names, help: help, valueName: nil, valueChoices: [])
}
static func option(
names: [String],
valueName: String,
help: String,
valueChoices: [CompletionChoice]
) -> CompletionOption {
CompletionOption(names: names, help: help, valueName: valueName, valueChoices: valueChoices)
}
}
/// A single suggested completion value with optional help text.
///
/// `CompletionChoice` is used for subcommands and curated value suggestions for
/// positional arguments or option values.
struct CompletionChoice {
let value: String
let help: String?
}
/// Central registry for curated completion values that cannot be inferred from
/// Commander metadata alone.
///
/// Most command structure comes directly from descriptors. This catalog is only
/// for constrained value sets such as `completions [shell]` or `--log-level`.
enum CompletionValueCatalog {
static func argumentChoices(for path: [String], index: Int, label: String) -> [CompletionChoice] {
if path == ["completions"], index == 0, label == "shell" {
return CompletionsCommand.Shell.allCases.map { shell in
CompletionChoice(value: shell.rawValue, help: shell.helpText)
}
}
return []
}
static func optionChoices(for path: [String], label: String, names: [String]) -> [CompletionChoice] {
if names.contains("--log-level"), label == "logLevel" {
return LogLevel.allCases.map { level in
CompletionChoice(value: level.cliValue, help: nil)
}
}
return []
}
}
/// Dispatches shell-completion rendering to the appropriate shell-specific
/// renderer.
enum CompletionScriptRenderer {
static func render(document: CompletionScriptDocument, for targetShell: CompletionsCommand.Shell) -> String {
switch targetShell {
case .bash:
BashCompletionRenderer().render(document: document)
case .zsh:
ZshCompletionRenderer().render(document: document)
case .fish:
FishCompletionRenderer().render(document: document)
}
}
}
protocol ShellCompletionRendering {
func render(document: CompletionScriptDocument) -> String
}
extension CommanderName {
var completionSpelling: String {
switch self {
case let .short(value), let .aliasShort(value):
"-\(value)"
case let .long(value), let .aliasLong(value):
"--\(value)"
}
}
}

View File

@ -1,191 +0,0 @@
import Foundation
/// Renders a fish completion script using fish-native helper functions and a
/// single dynamic `complete -a` callback.
struct FishCompletionRenderer: ShellCompletionRendering {
func render(document: CompletionScriptDocument) -> String {
let lines = [
"# Fish completion for peekaboo",
"# Generated from Commander descriptors via `peekaboo completions fish`.",
"# Install with:",
"# \(CompletionsCommand.Shell.fish.installationSnippet)",
"",
"function __peekaboo_fish_subcommands",
self.renderFishChoiceSwitch(document.pathsIncludingRoot, accessor: \.subcommands),
"end",
"",
"function __peekaboo_fish_options",
self.renderFishOptionSwitch(document: document),
"end",
"",
"function __peekaboo_fish_argument_values",
self.renderFishArgumentSwitch(document.pathsIncludingRoot),
"end",
"",
"function __peekaboo_fish_option_values",
self.renderFishOptionValueSwitch(document.pathsIncludingRoot),
"end",
"",
"function __peekaboo_fish_has_subcommand",
" set -l path $argv[1]",
" set -l candidate $argv[2]",
" for line in (__peekaboo_fish_subcommands \"$path\")",
" set -l parts (string split \\t -- $line)",
" if test (count $parts) -gt 0; and test \"$parts[1]\" = \"$candidate\"",
" return 0",
" end",
" end",
" return 1",
"end",
"",
"function __peekaboo_fish_append_path",
" if test -n \"$argv[1]\"",
" printf '%s %s\\n' \"$argv[1]\" \"$argv[2]\"",
" else",
" printf '%s\\n' \"$argv[2]\"",
" end",
"end",
"",
"function __peekaboo_fish_complete",
" set -l tokens (commandline -opc)",
" if test (count $tokens) -gt 0",
" set -e tokens[1]",
" end",
" set -l current (commandline -ct)",
" set -l path ''",
" set -l index 1",
" set -l previous ''",
" set -l token_count (count $tokens)",
"",
" while test $index -le $token_count",
" set -l token $tokens[$index]",
" if string match -qr '^-' -- $token",
" break",
" end",
" if __peekaboo_fish_has_subcommand \"$path\" \"$token\"",
" set path (__peekaboo_fish_append_path \"$path\" \"$token\")",
" set index (math \"$index + 1\")",
" else",
" break",
" end",
" end",
"",
" if test $token_count -gt 0",
" set previous $tokens[$token_count]",
" end",
"",
" set -l option_values (__peekaboo_fish_option_values \"$path\" \"$previous\")",
" if test (count $option_values) -gt 0",
" printf '%s\\n' $option_values",
" return",
" end",
"",
" if string match -qr '^-' -- $current",
" __peekaboo_fish_options \"$path\"",
" return",
" end",
"",
" set -l subcommands (__peekaboo_fish_subcommands \"$path\")",
" if test (count $subcommands) -gt 0",
" printf '%s\\n' $subcommands",
" return",
" end",
"",
" set -l argument_index (math \"$token_count - $index + 1\")",
" __peekaboo_fish_argument_values \"$path\" \"$argument_index\"",
"end",
"",
"complete -c \(document.commandName) -f -a '(__peekaboo_fish_complete)'",
]
return lines.joined(separator: "\n")
}
private func renderFishChoiceSwitch(
_ paths: [CompletionPath],
accessor: KeyPath<CompletionPath, [CompletionChoice]>
) -> String {
var lines = [" switch $argv[1]"]
for path in paths {
lines.append(" case '\(self.fishEscaped(path.key))'")
for choice in path[keyPath: accessor] {
lines.append(self.printfLine(value: choice.value, help: choice.help ?? ""))
}
}
lines.append(contentsOf: [
" case '*'",
" end",
])
return lines.joined(separator: "\n")
}
private func renderFishOptionSwitch(document: CompletionScriptDocument) -> String {
var lines = [" switch $argv[1]", " case ''"]
for option in document.rootOptions {
for name in option.names {
lines.append(self.printfLine(value: name, help: option.help))
}
}
for path in document.flattenedPaths {
lines.append(" case '\(self.fishEscaped(path.key))'")
for option in path.options {
for name in option.names {
lines.append(self.printfLine(value: name, help: option.help))
}
}
}
lines.append(contentsOf: [
" case '*'",
" end",
])
return lines.joined(separator: "\n")
}
private func renderFishArgumentSwitch(_ paths: [CompletionPath]) -> String {
var lines = [" switch \"$argv[1]:$argv[2]\""]
for path in paths {
for (index, argument) in path.arguments.enumerated() where !argument.choices.isEmpty {
lines.append(" case '\(self.fishEscaped(path.key)):\(index)'")
for choice in argument.choices {
lines.append(self.printfLine(value: choice.value, help: choice.help ?? ""))
}
}
}
lines.append(contentsOf: [
" case '*'",
" end",
])
return lines.joined(separator: "\n")
}
private func renderFishOptionValueSwitch(_ paths: [CompletionPath]) -> String {
var lines = [" switch \"$argv[1]:$argv[2]\""]
for path in paths {
for option in path.options where !option.valueChoices.isEmpty {
for name in option.names {
lines.append(" case '\(self.fishEscaped(path.key)):\(self.fishEscaped(name))'")
for choice in option.valueChoices {
lines.append(self.printfLine(value: choice.value, help: choice.help ?? ""))
}
}
}
}
lines.append(contentsOf: [
" case '*'",
" end",
])
return lines.joined(separator: "\n")
}
private func printfLine(value: String, help: String) -> String {
" printf '%s\\t%s\\n' '\(self.fishEscaped(value))' '\(self.fishEscaped(help))'"
}
private func fishEscaped(_ value: String) -> String {
value
.replacingOccurrences(of: "\\", with: "\\\\")
.replacingOccurrences(of: "'", with: "\\'")
.replacingOccurrences(of: "\t", with: " ")
.replacingOccurrences(of: "\n", with: " ")
}
}

View File

@ -1,197 +0,0 @@
import Foundation
/// Renders a zsh completion script using `compdef` plus dynamic helper
/// functions backed by the shared completion document.
struct ZshCompletionRenderer: ShellCompletionRendering {
func render(document: CompletionScriptDocument) -> String {
let lines = [
"#compdef \(document.commandName)",
"# Zsh completion for peekaboo",
"# Generated from Commander descriptors via `peekaboo completions zsh`.",
"# Install with:",
"# \(CompletionsCommand.Shell.zsh.installationSnippet)",
"",
"__peekaboo_zsh_subcommands() {",
self.renderZshChoiceSwitch(document.pathsIncludingRoot, accessor: \.subcommands),
"}",
"",
"__peekaboo_zsh_options() {",
self.renderZshOptionSwitch(document: document),
"}",
"",
"__peekaboo_zsh_argument_values() {",
self.renderZshArgumentSwitch(document.pathsIncludingRoot),
"}",
"",
"__peekaboo_zsh_option_values() {",
self.renderZshOptionValueSwitch(document.pathsIncludingRoot),
"}",
"",
"__peekaboo_zsh_has_subcommand() {",
" local path=\"$1\"",
" local candidate=\"$2\"",
" local line value description",
" while IFS=$'\\t' read -r value description; do",
" [[ \"$value\" == \"$candidate\" ]] && return 0",
" done < <(__peekaboo_zsh_subcommands \"$path\")",
" return 1",
"}",
"",
"__peekaboo_zsh_compadd_with_help() {",
" local line value description",
" local -a values descriptions",
" while IFS=$'\\t' read -r value description; do",
" values+=(\"$value\")",
" descriptions+=(\"$description\")",
" done",
" if (( ${#values[@]} == 0 )); then",
" return 1",
" fi",
" compadd -Q -d descriptions -- \"${values[@]}\"",
"}",
"",
"_peekaboo() {",
" local path=\"\"",
" local index=2",
" local token current_word previous_word",
" current_word=\"${words[CURRENT]}\"",
"",
" while (( index < CURRENT )); do",
" token=\"${words[index]}\"",
" [[ \"$token\" == -* ]] && break",
" if __peekaboo_zsh_has_subcommand \"$path\" \"$token\"; then",
" path=\"${path:+$path }$token\"",
" (( index++ ))",
" else",
" break",
" fi",
" done",
"",
" if (( CURRENT > 2 )); then",
" previous_word=\"${words[CURRENT - 1]}\"",
" fi",
"",
" if __peekaboo_zsh_option_values \"$path\" \"$previous_word\" | __peekaboo_zsh_compadd_with_help; then",
" return",
" fi",
"",
" if [[ \"$current_word\" == -* ]]; then",
" __peekaboo_zsh_options \"$path\" | __peekaboo_zsh_compadd_with_help",
" return",
" fi",
"",
" if __peekaboo_zsh_subcommands \"$path\" | __peekaboo_zsh_compadd_with_help; then",
" return",
" fi",
"",
" local argument_index=$(( CURRENT - index ))",
" __peekaboo_zsh_argument_values \"$path\" \"$argument_index\" | __peekaboo_zsh_compadd_with_help",
"}",
"",
"compdef _peekaboo \(document.commandName)",
]
return lines.joined(separator: "\n")
}
private func renderZshChoiceSwitch(
_ paths: [CompletionPath],
accessor: KeyPath<CompletionPath, [CompletionChoice]>
) -> String {
var lines = [" case \"$1\" in"]
for path in paths {
lines.append(" '\(self.caseLabel(path.key))')")
for choice in path[keyPath: accessor] {
lines.append(self.printLine(value: choice.value, help: choice.help ?? ""))
}
lines.append(" ;;")
}
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderZshOptionSwitch(document: CompletionScriptDocument) -> String {
var lines = [" case \"$1\" in", " '')"]
for option in document.rootOptions {
for name in option.names {
lines.append(self.printLine(value: name, help: option.help))
}
}
lines.append(" ;;")
for path in document.flattenedPaths {
lines.append(" '\(self.caseLabel(path.key))')")
for option in path.options {
for name in option.names {
lines.append(self.printLine(value: name, help: option.help))
}
}
lines.append(" ;;")
}
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderZshArgumentSwitch(_ paths: [CompletionPath]) -> String {
var lines = [" case \"$1:$2\" in"]
for path in paths {
for (index, argument) in path.arguments.enumerated() where !argument.choices.isEmpty {
lines.append(" '\(self.caseLabel(path.key)):\(index)')")
for choice in argument.choices {
lines.append(self.printLine(value: choice.value, help: choice.help ?? ""))
}
lines.append(" ;;")
}
}
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func renderZshOptionValueSwitch(_ paths: [CompletionPath]) -> String {
var lines = [" case \"$1:$2\" in"]
for path in paths {
for option in path.options where !option.valueChoices.isEmpty {
for name in option.names {
lines.append(" '\(self.caseLabel(path.key)):\(self.caseLabel(name))')")
for choice in option.valueChoices {
lines.append(self.printLine(value: choice.value, help: choice.help ?? ""))
}
lines.append(" ;;")
}
}
}
lines.append(contentsOf: [
" *)",
" ;;",
" esac",
])
return lines.joined(separator: "\n")
}
private func caseLabel(_ label: String) -> String {
label.replacingOccurrences(of: "'", with: "'\\''")
}
private func printLine(value: String, help: String) -> String {
" print -r -- $'\(self.zshEscaped(value))\\t\(self.zshEscaped(help))'"
}
private func zshEscaped(_ value: String) -> String {
value
.replacingOccurrences(of: "\\", with: "\\\\")
.replacingOccurrences(of: "'", with: "\\'")
.replacingOccurrences(of: "\t", with: " ")
.replacingOccurrences(of: "\n", with: " ")
}
}

View File

@ -1,175 +0,0 @@
import Foundation
import PeekabooCore
/// Re-use the Configuration type from PeekabooCore
typealias Configuration = PeekabooCore.Configuration
/// CLI-specific configuration manager that extends PeekabooCore's ConfigurationManager
/// with additional CLI-specific functionality.
@MainActor
final class ConfigurationManager: @unchecked Sendable {
static let shared = ConfigurationManager()
/// Use PeekabooCore's ConfigurationManager for core functionality
private let coreManager = PeekabooCore.ConfigurationManager.shared
private init() {}
// MARK: - Delegate to Core Manager
/// Base directory for all Peekaboo configuration
static var baseDir: String {
PeekabooCore.ConfigurationManager.baseDir
}
/// Legacy configuration directory (for migration)
static var legacyConfigDir: String {
PeekabooCore.ConfigurationManager.legacyConfigDir
}
/// Default configuration file path
static var configPath: String {
PeekabooCore.ConfigurationManager.configPath
}
/// Legacy configuration file path (for migration)
static var legacyConfigPath: String {
PeekabooCore.ConfigurationManager.legacyConfigPath
}
/// Credentials file path
static var credentialsPath: String {
PeekabooCore.ConfigurationManager.credentialsPath
}
/// Migrate from legacy configuration if needed
func migrateIfNeeded() throws {
// Migrate from legacy configuration if needed
try self.coreManager.migrateIfNeeded()
}
/// Load configuration from file
func loadConfiguration() -> Configuration? {
// Load configuration from file
self.coreManager.loadConfiguration()
}
/// Get cached configuration, loading only if needed.
func getConfiguration() -> Configuration? {
self.coreManager.getConfiguration()
}
/// Strip comments from JSONC content
func stripJSONComments(from json: String) -> String {
// Strip comments from JSONC content
self.coreManager.stripJSONComments(from: json)
}
/// Expand environment variables in the format ${VAR_NAME}
func expandEnvironmentVariables(in text: String) -> String {
// Expand environment variables in the format ${VAR_NAME}
self.coreManager.expandEnvironmentVariables(in: text)
}
/// Get AI providers with proper precedence
func getAIProviders(cliValue: String? = nil) -> String {
// Get AI providers with proper precedence
self.coreManager.getAIProviders(cliValue: cliValue)
}
/// Get OpenAI API key with proper precedence
func getOpenAIAPIKey() -> String? {
// Get OpenAI API key with proper precedence
self.coreManager.getOpenAIAPIKey()
}
/// Get Ollama base URL with proper precedence
func getOllamaBaseURL() -> String {
// Get Ollama base URL with proper precedence
self.coreManager.getOllamaBaseURL()
}
/// Get default save path with proper precedence
func getDefaultSavePath(cliValue: String? = nil) -> String {
// Get default save path with proper precedence
self.coreManager.getDefaultSavePath(cliValue: cliValue)
}
/// Get log level with proper precedence
func getLogLevel() -> String {
// Get log level with proper precedence
self.coreManager.getLogLevel()
}
/// Get log path with proper precedence
func getLogPath() -> String {
// Get log path with proper precedence
self.coreManager.getLogPath()
}
/// Create default configuration file
func createDefaultConfiguration() throws {
// Create default configuration file
try self.coreManager.createDefaultConfiguration()
}
/// Set or update a credential
func setCredential(key: String, value: String) throws {
// Set or update a credential
try self.coreManager.setCredential(key: key, value: value)
}
/// Get configuration value with precedence
func getValue<T>(
cliValue: T?,
envVar: String?,
configValue: T?,
defaultValue: T
) -> T {
// Get configuration value with precedence
self.coreManager.getValue(
cliValue: cliValue,
envVar: envVar,
configValue: configValue,
defaultValue: defaultValue
)
}
// MARK: - Custom Provider Management
/// Add a custom AI provider to the configuration
func addCustomProvider(_ provider: Configuration.CustomProvider, id: String) throws {
// Add a custom AI provider to the configuration
try self.coreManager.addCustomProvider(provider, id: id)
}
/// Remove a custom provider from the configuration
func removeCustomProvider(id: String) throws {
// Remove a custom provider from the configuration
try self.coreManager.removeCustomProvider(id: id)
}
/// Get a specific custom provider by ID
func getCustomProvider(id: String) -> Configuration.CustomProvider? {
// Get a specific custom provider by ID
self.coreManager.getCustomProvider(id: id)
}
/// List all configured custom providers
func listCustomProviders() -> [String: Configuration.CustomProvider] {
// List all configured custom providers
self.coreManager.listCustomProviders()
}
/// Test connection to a custom provider
func testCustomProvider(id: String) async -> (success: Bool, error: String?) {
// Test connection to a custom provider
await self.coreManager.testCustomProvider(id: id)
}
/// Discover available models from a custom provider
func discoverModelsForCustomProvider(id: String) async -> (models: [String], error: String?) {
// Discover available models from a custom provider
await self.coreManager.discoverModelsForCustomProvider(id: id)
}
}

View File

@ -1,97 +0,0 @@
//
// CommandRegistry.swift
// PeekabooCLI
//
import Commander
struct CommandRegistryEntry {
enum Category: String, Codable, CaseIterable {
case core
case interaction
case system
case vision
case ai
case mcp
}
let type: any ParsableCommand.Type
let category: Category
}
struct CommandDefinition: Codable {
let name: String
let typeName: String
let category: CommandRegistryEntry.Category
let abstract: String
let discussion: String?
let version: String?
let subcommandCount: Int
}
enum CommandRegistry {
@MainActor
static let entries: [CommandRegistryEntry] = [
.init(type: ImageCommand.self, category: .core),
.init(type: CaptureCommand.self, category: .core),
.init(type: BridgeCommand.self, category: .core),
.init(type: DaemonCommand.self, category: .core),
.init(type: ListCommand.self, category: .core),
.init(type: ToolsCommand.self, category: .core),
.init(type: ConfigCommand.self, category: .core),
.init(type: PermissionsCommand.self, category: .core),
.init(type: LearnCommand.self, category: .core),
.init(type: SeeCommand.self, category: .vision),
.init(type: ClickCommand.self, category: .interaction),
.init(type: TypeCommand.self, category: .interaction),
.init(type: SetValueCommand.self, category: .interaction),
.init(type: PerformActionCommand.self, category: .interaction),
.init(type: PressCommand.self, category: .interaction),
.init(type: ScrollCommand.self, category: .interaction),
.init(type: HotkeyCommand.self, category: .interaction),
.init(type: PasteCommand.self, category: .interaction),
.init(type: SwipeCommand.self, category: .interaction),
.init(type: DragCommand.self, category: .interaction),
.init(type: MoveCommand.self, category: .interaction),
.init(type: RunCommand.self, category: .core),
.init(type: SleepCommand.self, category: .core),
.init(type: CleanCommand.self, category: .core),
.init(type: WindowCommand.self, category: .system),
.init(type: MenuCommand.self, category: .system),
.init(type: MenuBarCommand.self, category: .system),
.init(type: AppCommand.self, category: .system),
.init(type: OpenCommand.self, category: .system),
.init(type: DockCommand.self, category: .system),
.init(type: DialogCommand.self, category: .system),
.init(type: SpaceCommand.self, category: .system),
.init(type: VisualizerCommand.self, category: .system),
.init(type: ClipboardCommand.self, category: .system),
.init(type: CompletionsCommand.self, category: .core),
.init(type: CommanderCommand.self, category: .core),
.init(type: AgentCommand.self, category: .ai),
.init(type: BrowserCommand.self, category: .mcp),
.init(type: InspectUICommand.self, category: .mcp),
.init(type: MCPCommand.self, category: .mcp),
]
@MainActor
static var rootCommandTypes: [any ParsableCommand.Type] {
self.entries.map(\.type)
}
@MainActor
static func definitions() -> [CommandDefinition] {
self.entries.map { entry in
let description = entry.type.commandDescription
return CommandDefinition(
name: description.commandName ?? String(describing: entry.type),
typeName: String(reflecting: entry.type),
category: entry.category,
abstract: description.abstract,
discussion: description.discussion,
version: description.version,
subcommandCount: description.subcommands.count
)
}
}
}

View File

@ -1,319 +0,0 @@
import Commander
import Foundation
/// Log level enumeration for structured logging
public enum LogLevel: Int, Comparable, Sendable {
case trace = 0 // Most verbose
case verbose = 1
case debug = 2
case info = 3
case warning = 4
case error = 5
case critical = 6 // Most severe
public static func < (lhs: LogLevel, rhs: LogLevel) -> Bool {
lhs.rawValue < rhs.rawValue
}
var name: String {
switch self {
case .trace: "TRACE"
case .verbose: "VERBOSE"
case .debug: "DEBUG"
case .info: "INFO"
case .warning: "WARN"
case .error: "ERROR"
case .critical: "CRITICAL"
}
}
}
/// Thread-safe logging utility for Peekaboo.
///
/// Provides logging functionality that can switch between stderr output (for normal operation)
/// and buffered collection (for JSON output mode) to avoid interfering with structured output.
final class Logger: @unchecked Sendable {
static let shared = Logger()
private nonisolated(unsafe) var debugLogs: [String] = []
private nonisolated(unsafe) var isJsonOutputMode = false
private nonisolated(unsafe) var verboseMode = false
private let defaultMinimumLogLevel: LogLevel
private nonisolated(unsafe) var minimumLogLevel: LogLevel
private let queue = DispatchQueue(label: "logger.queue", attributes: .concurrent)
private let iso8601Formatter: ISO8601DateFormatter
/// Performance tracking
private nonisolated(unsafe) var performanceTimers: [String: Date] = [:]
private init() {
self.iso8601Formatter = ISO8601DateFormatter()
self.iso8601Formatter.formatOptions = [.withInternetDateTime, .withFractionalSeconds]
// Check environment for log level
var configuredLevel: LogLevel = .warning
if let envValue = ProcessInfo.processInfo.environment["PEEKABOO_LOG_LEVEL"],
let envLevel = LogLevel.parse(raw: envValue) {
configuredLevel = envLevel
}
self.defaultMinimumLogLevel = configuredLevel
self.minimumLogLevel = configuredLevel
}
func setJsonOutputMode(_ enabled: Bool) {
self.queue.sync(flags: .barrier) {
self.isJsonOutputMode = enabled
// Don't clear logs automatically - let tests manage this explicitly
}
}
func setVerboseMode(_ enabled: Bool) {
self.queue.sync(flags: .barrier) {
self.verboseMode = enabled
if enabled {
self.minimumLogLevel = .verbose
}
}
}
func setMinimumLogLevel(_ level: LogLevel) {
self.queue.sync(flags: .barrier) {
self.minimumLogLevel = level
}
}
func resetMinimumLogLevel() {
self.queue.sync(flags: .barrier) {
self.minimumLogLevel = self.defaultMinimumLogLevel
}
}
var isVerbose: Bool {
self.queue.sync {
self.verboseMode
}
}
/// Log a message at a specific level
private func log(_ level: LogLevel, _ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
// Convert metadata to a string representation outside the async closure
let metadataString: String? = metadata.flatMap { dict in
dict.isEmpty ? nil : dict.map { "\($0.key)=\($0.value)" }.joined(separator: ", ")
}
guard level >= self.minimumLogLevel || (level == .verbose && self.verboseMode) else { return }
let timestamp = self.iso8601Formatter.string(from: Date())
let levelName = level.name
var formattedMessage = "[\(timestamp)] \(levelName): \(message)"
if let category {
formattedMessage = "[\(timestamp)] \(levelName) [\(category)]: \(message)"
}
if let metadataString {
formattedMessage += " {\(metadataString)}"
}
let shouldBuffer = self.isJsonOutputMode
self.queue.async(flags: .barrier) { [formattedMessage] in
if shouldBuffer {
self.debugLogs.append(formattedMessage)
} else {
fputs("\(formattedMessage)\n", stderr)
}
}
}
func verbose(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
self.log(.verbose, message, category: category, metadata: metadata)
}
func debug(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
self.log(.debug, message, category: category, metadata: metadata)
}
func info(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
self.log(.info, message, category: category, metadata: metadata)
}
func warn(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
self.log(.warning, message, category: category, metadata: metadata)
}
func error(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
self.log(.error, message, category: category, metadata: metadata)
}
func critical(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
self.log(.critical, message, category: category, metadata: metadata)
}
// MARK: - Performance Tracking
/// Start a performance timer
func startTimer(_ name: String) {
// Start a performance timer
let timestamp = self.iso8601Formatter.string(from: Date())
let verboseEnabled = self.verboseMode
let shouldBuffer = self.isJsonOutputMode
self.queue.async(flags: .barrier) {
self.performanceTimers[name] = Date()
if verboseEnabled {
let message = "[\(timestamp)] VERBOSE [Performance]: Starting timer '\(name)'"
if shouldBuffer {
self.debugLogs.append(message)
} else {
fputs("\(message)\n", stderr)
}
}
}
}
/// Stop a performance timer and log the duration
func stopTimer(_ name: String, threshold: TimeInterval? = nil) {
var startTime: Date?
self.queue.sync(flags: .barrier) {
startTime = self.performanceTimers[name]
self.performanceTimers.removeValue(forKey: name)
}
guard let startTime else {
self.log(.warning, "Timer '\(name)' was not started", category: "Performance")
return
}
let duration = Date().timeIntervalSince(startTime)
if self.verboseMode || (threshold != nil && duration > threshold!) {
let durationMs = Int(duration * 1000)
self.log(
.verbose,
"Timer '\(name)' completed",
category: "Performance",
metadata: ["duration_ms": durationMs]
)
}
}
// MARK: - Operation Tracking
/// Log the start of an operation
func operationStart(_ operation: String, metadata: [String: Any]? = nil) {
// Log the start of an operation
var meta = metadata ?? [:]
meta["operation"] = operation
self.verbose("Starting operation", category: "Operation", metadata: meta)
self.startTimer(operation)
}
/// Log the completion of an operation
func operationComplete(_ operation: String, success: Bool = true, metadata: [String: Any]? = nil) {
// Log the completion of an operation
var meta = metadata ?? [:]
meta["operation"] = operation
meta["success"] = success
self.verbose("Operation completed", category: "Operation", metadata: meta)
self.stopTimer(operation)
}
func getDebugLogs() -> [String] {
self.queue.sync {
self.debugLogs
}
}
func clearDebugLogs() {
self.queue.sync(flags: .barrier) {
self.debugLogs.removeAll()
}
}
/// For testing - ensures all pending operations are complete
func flush() {
// For testing - ensures all pending operations are complete
self.queue.sync(flags: .barrier) {
// This ensures all pending async operations are complete
}
}
}
public func logVerbose(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
Logger.shared.verbose(message, category: category, metadata: metadata)
}
public func logDebug(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
Logger.shared.debug(message, category: category, metadata: metadata)
}
public func logInfo(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
Logger.shared.info(message, category: category, metadata: metadata)
}
public func logWarn(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
Logger.shared.warn(message, category: category, metadata: metadata)
}
public func logError(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
Logger.shared.error(message, category: category, metadata: metadata)
}
public func logCritical(_ message: String, category: String? = nil, metadata: [String: Any]? = nil) {
Logger.shared.critical(message, category: category, metadata: metadata)
}
public enum CLIInstrumentation {
public enum LoggerControl {
public static func setJsonOutputMode(_ enabled: Bool) {
Logger.shared.setJsonOutputMode(enabled)
}
public static func setVerboseMode(_ enabled: Bool) {
Logger.shared.setVerboseMode(enabled)
}
public static func clearDebugLogs() {
Logger.shared.clearDebugLogs()
}
public static func debugLogs() -> [String] {
Logger.shared.getDebugLogs()
}
public static func flush() {
Logger.shared.flush()
}
public static func setMinimumLogLevel(_ level: LogLevel) {
Logger.shared.setMinimumLogLevel(level)
}
public static func resetMinimumLogLevel() {
Logger.shared.resetMinimumLogLevel()
}
}
}
extension LogLevel {
static func parse(raw: String) -> LogLevel? {
switch raw.lowercased() {
case "trace": .trace
case "verbose": .verbose
case "debug": .debug
case "info": .info
case "warning", "warn": .warning
case "error": .error
case "critical": .critical
default: nil
}
}
}
extension LogLevel: ExpressibleFromArgument {
public init?(argument: String) {
guard let level = LogLevel.parse(raw: argument) else {
return nil
}
self = level
}
}

View File

@ -1,248 +0,0 @@
import Darwin
import Foundation
enum DeferredCommandOutput {
static func run<T>(
bufferingOutput: Bool,
operation: () async throws -> T
) async throws -> T {
guard bufferingOutput else {
return try await operation()
}
let inheritedTerminalOutput = TerminalDetector.standardOutputFileDescriptor
let capture = try FileDescriptorOutputCapture()
let terminalOutput = inheritedTerminalOutput ?? capture.originalStandardOutputDescriptor
let result: T
do {
result = try await TerminalDetector.$standardOutputFileDescriptor.withValue(terminalOutput) {
try await operation()
}
} catch {
let shouldReplay = !(error is CancellationError)
// Preserve the command's primary error even if restoring or replaying output fails.
Logger.shared.flush()
try? capture.finish(replayingOutput: shouldReplay)
throw error
}
Logger.shared.flush()
try capture.finish(replayingOutput: true)
return result
}
}
private nonisolated enum DeferredCommandOutputError: LocalizedError {
case posix(operation: String, code: Int32)
var errorDescription: String? {
switch self {
case let .posix(operation, code):
"Failed to \(operation): \(String(cString: strerror(code)))"
}
}
}
private final nonisolated class FileDescriptorOutputCapture {
private var stdoutCapture: Int32 = -1
private var stderrCapture: Int32 = -1
private var originalStdout: Int32 = -1
private var originalStderr: Int32 = -1
private var stdoutRedirected = false
private var stderrRedirected = false
private var finished = false
var originalStandardOutputDescriptor: Int32 {
self.originalStdout
}
init() throws {
// Keep output emitted before this command outside its deferred transaction.
_ = fflush(nil)
do {
self.stdoutCapture = try Self.makeTemporaryFile(named: "stdout")
self.stderrCapture = try Self.makeTemporaryFile(named: "stderr")
self.originalStdout = try Self.duplicate(STDOUT_FILENO, named: "stdout")
self.originalStderr = try Self.duplicate(STDERR_FILENO, named: "stderr")
try Self.redirect(self.stdoutCapture, to: STDOUT_FILENO, named: "stdout")
self.stdoutRedirected = true
try Self.redirect(self.stderrCapture, to: STDERR_FILENO, named: "stderr")
self.stderrRedirected = true
} catch {
self.restoreIgnoringErrors()
self.closeDescriptors()
throw error
}
}
deinit {
guard !self.finished else { return }
_ = fflush(nil)
self.restoreIgnoringErrors()
self.closeDescriptors()
}
func finish(replayingOutput: Bool) throws {
guard !self.finished else { return }
_ = fflush(nil)
try self.restore()
defer {
self.finished = true
self.closeDescriptors()
}
if replayingOutput {
try Self.replay(from: self.stdoutCapture, to: self.originalStdout, named: "stdout")
try Self.replay(from: self.stderrCapture, to: self.originalStderr, named: "stderr")
}
}
private func restore() throws {
var firstError: (any Error)?
if self.stdoutRedirected {
do {
try Self.redirect(self.originalStdout, to: STDOUT_FILENO, named: "stdout")
self.stdoutRedirected = false
} catch {
firstError = error
}
}
if self.stderrRedirected {
do {
try Self.redirect(self.originalStderr, to: STDERR_FILENO, named: "stderr")
self.stderrRedirected = false
} catch {
firstError = firstError ?? error
}
}
if let firstError {
throw firstError
}
}
private func restoreIgnoringErrors() {
if self.stdoutRedirected, dup2(self.originalStdout, STDOUT_FILENO) != -1 {
self.stdoutRedirected = false
}
if self.stderrRedirected, dup2(self.originalStderr, STDERR_FILENO) != -1 {
self.stderrRedirected = false
}
}
private func closeDescriptors() {
Self.close(&self.stdoutCapture)
Self.close(&self.stderrCapture)
Self.close(&self.originalStdout)
Self.close(&self.originalStderr)
}
private static func makeTemporaryFile(named stream: String) throws -> Int32 {
var template = Array("\(NSTemporaryDirectory())peekaboo-\(stream).XXXXXX".utf8CString)
let descriptor = template.withUnsafeMutableBufferPointer { buffer -> Int32 in
guard let baseAddress = buffer.baseAddress else { return -1 }
let descriptor = mkstemp(baseAddress)
if descriptor != -1 {
_ = unlink(baseAddress)
}
return descriptor
}
guard descriptor != -1 else {
throw DeferredCommandOutputError.posix(
operation: "create deferred \(stream) output",
code: errno
)
}
Self.setCloseOnExec(descriptor)
return descriptor
}
private static func duplicate(_ descriptor: Int32, named stream: String) throws -> Int32 {
let duplicate = dup(descriptor)
guard duplicate != -1 else {
throw DeferredCommandOutputError.posix(
operation: "duplicate \(stream)",
code: errno
)
}
Self.setCloseOnExec(duplicate)
return duplicate
}
private static func redirect(_ source: Int32, to destination: Int32, named stream: String) throws {
guard dup2(source, destination) != -1 else {
throw DeferredCommandOutputError.posix(
operation: "redirect \(stream)",
code: errno
)
}
}
private static func replay(from source: Int32, to destination: Int32, named stream: String) throws {
guard lseek(source, 0, SEEK_SET) != -1 else {
throw DeferredCommandOutputError.posix(
operation: "rewind deferred \(stream) output",
code: errno
)
}
var buffer = [UInt8](repeating: 0, count: 64 * 1024)
while true {
let bytesRead = buffer.withUnsafeMutableBytes { bytes in
Darwin.read(source, bytes.baseAddress, bytes.count)
}
if bytesRead == 0 {
return
}
if bytesRead == -1 {
if errno == EINTR {
continue
}
throw DeferredCommandOutputError.posix(
operation: "read deferred \(stream) output",
code: errno
)
}
var offset = 0
while offset < bytesRead {
let bytesWritten = buffer.withUnsafeBytes { bytes in
Darwin.write(
destination,
bytes.baseAddress?.advanced(by: offset),
bytesRead - offset
)
}
if bytesWritten == -1 {
if errno == EINTR {
continue
}
throw DeferredCommandOutputError.posix(
operation: "replay deferred \(stream) output",
code: errno
)
}
offset += bytesWritten
}
}
}
private static func setCloseOnExec(_ descriptor: Int32) {
let flags = fcntl(descriptor, F_GETFD)
if flags != -1 {
_ = fcntl(descriptor, F_SETFD, flags | FD_CLOEXEC)
}
}
private static func close(_ descriptor: inout Int32) {
guard descriptor != -1 else { return }
_ = Darwin.close(descriptor)
descriptor = -1
}
}

View File

@ -1,15 +0,0 @@
import Foundation
/// A text output stream that writes to a file handle
struct FileHandleTextOutputStream: TextOutputStream {
private let fileHandle: FileHandle
init(_ fileHandle: FileHandle) {
self.fileHandle = fileHandle
}
mutating func write(_ string: String) {
guard let data = string.data(using: .utf8) else { return }
self.fileHandle.write(data)
}
}

View File

@ -1,192 +0,0 @@
import Foundation
import PeekabooFoundation
/// Helper class for managing JSON output and debug logs
public class JSONOutput {
private var debugLogs: [String] = []
func addDebugLog(_ message: String) {
self.debugLogs.append(message)
}
func getDebugLogs() -> [String] {
self.debugLogs
}
func clearDebugLogs() {
self.debugLogs.removeAll()
}
}
/// Standard JSON response format for Peekaboo API output.
///
/// This is now deprecated - use CodableJSONResponse with specific types instead
struct JSONResponse: Codable {
let success: Bool
let data: Empty? // Added for test compatibility
let messages: [String]?
let debug_logs: [String]
let error: ErrorInfo?
init(
success: Bool,
data: Empty? = nil, // Added for test compatibility
messages: [String]? = nil,
debugLogs: [String] = [],
error: ErrorInfo? = nil
) {
self.success = success
self.data = data
self.messages = messages
self.debug_logs = debugLogs
self.error = error
}
}
/// Error information structure for JSON responses.
///
/// Contains error details including message, standardized error code,
/// and optional additional context.
struct ErrorInfo: Codable {
let message: String
let code: String
let details: String?
init(message: String, code: ErrorCode, details: String? = nil) {
self.message = message
self.code = code.rawValue
self.details = details
}
}
/// Standardized error codes for Peekaboo operations.
///
/// Provides consistent error identification across the API for proper
/// error handling by clients and automation tools.
enum ErrorCode: String, Codable {
case PERMISSION_ERROR_SCREEN_RECORDING
case PERMISSION_ERROR_ACCESSIBILITY
case PERMISSION_ERROR_EVENT_SYNTHESIZING
case PERMISSION_ERROR_APPLESCRIPT
case PERMISSION_DENIED
case APP_NOT_FOUND
case AMBIGUOUS_APP_IDENTIFIER
case WINDOW_NOT_FOUND
case CAPTURE_FAILED
case FILE_IO_ERROR
case INVALID_ARGUMENT
case SIPS_ERROR
case INTERNAL_SWIFT_ERROR
case UNKNOWN_ERROR
case WINDOW_MANIPULATION_ERROR
case VALIDATION_ERROR
case MENU_BAR_NOT_FOUND
case MENU_ITEM_NOT_FOUND
case DOCK_NOT_FOUND
case NO_ACTIVE_DIALOG
case ELEMENT_NOT_FOUND
case SESSION_NOT_FOUND
case SNAPSHOT_NOT_FOUND
case SNAPSHOT_STALE
case APPLICATION_NOT_FOUND
case NO_POINT_SPECIFIED
case INVALID_COORDINATES
case DOCK_LIST_NOT_FOUND
case DOCK_ITEM_NOT_FOUND
case POSITION_NOT_FOUND
case SCRIPT_ERROR
case MISSING_API_KEY
case AGENT_ERROR
case INTERACTION_FAILED
case TIMEOUT
case INVALID_INPUT
}
func outputJSON(_ response: JSONResponse, logger: Logger) {
do {
let encoder = JSONEncoder()
encoder.outputFormatting = .prettyPrinted
let data = try encoder.encode(response)
if let jsonString = String(data: data, encoding: .utf8) {
print(jsonString)
}
} catch {
logger.error("Failed to encode JSON response: \(error)")
// Fallback to simple error JSON
print("""
{
"success": false,
"error": {
"message": "Failed to encode JSON response",
"code": "INTERNAL_SWIFT_ERROR"
},
"debug_logs": []
}
""")
}
}
func outputSuccessCodable(data: some Codable, messages: [String]? = nil, logger: Logger) {
let debugLogs = logger.getDebugLogs()
let response = CodableJSONResponse(
success: true, data: data, messages: messages, debug_logs: debugLogs
)
outputJSONCodable(response, logger: logger)
}
func outputJSONCodable(_ response: some Encodable, logger: Logger) {
do {
let encoder = JSONEncoder()
encoder.outputFormatting = .prettyPrinted
// Note: JSONEncoder by default omits nil values from optionals
// This is standard behavior and generally desirable for cleaner output
let data = try encoder.encode(response)
if let jsonString = String(data: data, encoding: .utf8) {
print(jsonString)
}
} catch {
logger.error("Failed to encode JSON response: \(error)")
// Fallback to simple error JSON
print("""
{
"success": false,
"error": {
"message": "Failed to encode JSON response",
"code": "INTERNAL_SWIFT_ERROR"
},
"debug_logs": []
}
""")
}
}
/// Generic JSON response wrapper for strongly-typed data.
///
/// Provides type-safe JSON responses when the data payload type
/// is known at compile time.
struct CodableJSONResponse<T: Codable>: Codable {
let success: Bool
let data: T
let messages: [String]?
let debug_logs: [String]
}
func outputError(message: String, code: ErrorCode, details: String? = nil, logger: Logger) {
let error = ErrorInfo(message: message, code: code, details: details)
let debugLogs = logger.getDebugLogs()
outputJSON(JSONResponse(success: false, messages: nil, debugLogs: debugLogs, error: error), logger: logger)
}
func outputFailure(message: String, logger: Logger, error: (any Error)? = nil) {
let details = error.map { "\($0)" }
outputError(message: message, code: .UNKNOWN_ERROR, details: details, logger: logger)
}
/// Empty type for successful responses with no data
struct Empty: Codable {}
extension Empty: ExpressibleByNilLiteral {
init(nilLiteral: ()) {
self.init()
}
}

View File

@ -1,26 +0,0 @@
import Foundation
extension LogLevel: CaseIterable {
public static var allCases: [LogLevel] {
[.trace, .verbose, .debug, .info, .warning, .error, .critical]
}
var cliValue: String {
switch self {
case .trace:
"trace"
case .verbose:
"verbose"
case .debug:
"debug"
case .info:
"info"
case .warning:
"warning"
case .error:
"error"
case .critical:
"critical"
}
}
}

View File

@ -1,88 +0,0 @@
//
// PeekabooSpinner.swift
// PeekabooCore
//
import Foundation
import Spinner
/// Modern spinner implementation using the Spinner library
@available(macOS 14.0, *)
@MainActor
final class PeekabooSpinner {
private var spinner: Spinner?
private let supportsColors: Bool
init(supportsColors: Bool = true) {
self.supportsColors = supportsColors
}
/// Start spinner with default "Thinking..." message
func start() {
// Start spinner with default "Thinking..." message
self.start(message: "Thinking...")
}
/// Start spinner with custom message
func start(message: String) {
// Start spinner with custom message
self.stop() // Ensure no previous spinner is running
if self.supportsColors {
self.spinner = Spinner(.dots, message, format: "{S} {T}")
} else {
// For environments without color support, use a minimal spinner
self.spinner = Spinner(.dots, message, format: "{T}...")
}
self.spinner?.start()
}
/// Stop spinner without completion message
func stop() {
// Stop spinner without completion message
self.spinner?.clear()
self.spinner = nil
}
/// Stop spinner with success message
func success(_ message: String? = nil) {
// Stop spinner with success message
self.spinner?.success(message)
self.spinner = nil
}
/// Stop spinner with error message
func error(_ message: String? = nil) {
// Stop spinner with error message
self.spinner?.error(message)
self.spinner = nil
}
/// Stop spinner with warning message
func warning(_ message: String? = nil) {
// Stop spinner with warning message
self.spinner?.warning(message)
self.spinner = nil
}
/// Stop spinner with info message
func info(_ message: String? = nil) {
// Stop spinner with info message
self.spinner?.info(message)
self.spinner = nil
}
/// Update spinner message while running
func updateMessage(_ message: String) {
// Update spinner message while running
self.spinner?.message(message)
}
/// Stop with a brief delay for smoother transitions
func stopWithDelay() async {
// Stop with a brief delay for smoother transitions
try? await Task.sleep(for: .milliseconds(300))
self.stop()
}
}

View File

@ -1,269 +0,0 @@
import Darwin
import Foundation
/// Comprehensive terminal capability detection for progressive enhancement
struct TerminalCapabilities {
let isInteractive: Bool
let supportsColors: Bool
let supportsTrueColor: Bool
let width: Int
let height: Int
let termType: String?
let isCI: Bool
let isPiped: Bool
/// Detect optimal output mode based on terminal capabilities
var recommendedOutputMode: OutputMode {
// Explicit overrides handled elsewhere
// Environment-based fallbacks
if !self.isInteractive || self.isCI || self.isPiped {
return .minimal
}
// Prefer enhanced output when color is available
return self.supportsColors ? .enhanced : .compact
}
}
/// Terminal detection utilities following modern CLI best practices
enum TerminalDetector {
@TaskLocal
static var standardOutputFileDescriptor: Int32?
/// Detect comprehensive terminal capabilities
static func detectCapabilities() -> TerminalCapabilities {
// Detect comprehensive terminal capabilities
let outputFileDescriptor = self.standardOutputFileDescriptor ?? STDOUT_FILENO
let isInteractive = self.isInteractiveTerminal(outputFileDescriptor)
let (width, height) = self.getTerminalDimensions(outputFileDescriptor)
let termType = ProcessInfo.processInfo.environment["TERM"]
let isCI = self.isCIEnvironment()
let isPiped = self.isPipedOutput(outputFileDescriptor)
let supportsColors = self.detectColorSupport(termType: termType, isInteractive: isInteractive)
let supportsTrueColor = self.detectTrueColorSupport()
return TerminalCapabilities(
isInteractive: isInteractive,
supportsColors: supportsColors,
supportsTrueColor: supportsTrueColor,
width: width,
height: height,
termType: termType,
isCI: isCI,
isPiped: isPiped
)
}
// MARK: - Core Detection Methods
/// Check if stdout is connected to an interactive terminal
private static func isInteractiveTerminal(_ outputFileDescriptor: Int32) -> Bool {
// Check if stdout is connected to an interactive terminal
isatty(outputFileDescriptor) != 0
}
/// Check if output is being piped or redirected
private static func isPipedOutput(_ outputFileDescriptor: Int32) -> Bool {
// Check if output is being piped or redirected
isatty(outputFileDescriptor) == 0
}
/// Detect CI/automation environments
private static func isCIEnvironment() -> Bool {
// Detect CI/automation environments
let ciVariables = [
"CI", "CONTINUOUS_INTEGRATION",
"GITHUB_ACTIONS", "GITHUB_WORKSPACE",
"GITLAB_CI", "GITLAB_USER_LOGIN",
"TRAVIS", "TRAVIS_BUILD_ID",
"CIRCLECI", "CIRCLE_BUILD_NUM",
"JENKINS_URL", "BUILD_NUMBER",
"BUILDKITE", "BUILDKITE_BUILD_ID",
"AZURE_PIPELINES", "TF_BUILD",
"BITBUCKET_COMMIT", "BITBUCKET_BUILD_NUMBER",
"DRONE", "DRONE_BUILD_NUMBER",
"SEMAPHORE", "SEMAPHORE_BUILD_NUMBER",
]
let env = ProcessInfo.processInfo.environment
return ciVariables.contains { env[$0] != nil }
}
/// Get terminal dimensions using ioctl
private static func getTerminalDimensions(_ outputFileDescriptor: Int32) -> (width: Int, height: Int) {
// Get terminal dimensions using ioctl
var windowSize = winsize()
guard ioctl(outputFileDescriptor, TIOCGWINSZ, &windowSize) == 0 else {
// Fallback to environment variables
let width = Int(ProcessInfo.processInfo.environment["COLUMNS"] ?? "80") ?? 80
let height = Int(ProcessInfo.processInfo.environment["LINES"] ?? "24") ?? 24
return (width, height)
}
return (
width: Int(windowSize.ws_col),
height: Int(windowSize.ws_row)
)
}
// MARK: - Color Support Detection
/// Detect color support using multiple methods
private static func detectColorSupport(termType: String?, isInteractive: Bool) -> Bool {
// Detect color support using multiple methods
guard isInteractive else { return false }
// Method 1: Check COLORTERM environment variable (most reliable)
if let colorTerm = ProcessInfo.processInfo.environment["COLORTERM"] {
return !colorTerm.isEmpty
}
// Method 2: Check TERM variable patterns
if let term = termType {
let colorTermPatterns = [
"color", "256color", "truecolor", "24bit",
"xterm-256", "screen-256", "tmux-256",
]
if colorTermPatterns.contains(where: term.contains) {
return true
}
// Known color-capable terminals
let colorTerminals = [
"xterm", "screen", "tmux", "rxvt", "konsole",
"gnome", "mate", "xfce", "terminology", "kitty",
"alacritty", "iterm", "hyper", "vscode",
]
if colorTerminals.contains(where: term.contains) {
return true
}
}
// Method 3: Platform-specific defaults
#if os(macOS)
// macOS Terminal.app and most modern terminals support colors
return true
#else
// Conservative fallback for other platforms
return termType != "dumb" && termType != nil
#endif
}
/// Detect true color (24-bit) support
private static func detectTrueColorSupport() -> Bool {
// Detect true color (24-bit) support
let env = ProcessInfo.processInfo.environment
// Check COLORTERM for explicit true color support
if let colorTerm = env["COLORTERM"] {
return colorTerm.contains("truecolor") || colorTerm.contains("24bit")
}
// Check for terminals known to support true color
if let term = env["TERM"] {
let trueColorTerminals = [
"iterm", "kitty", "alacritty", "wezterm",
"hyper", "vscode", "gnome-terminal",
]
return trueColorTerminals.contains(where: term.contains)
}
#if os(macOS)
// Most modern macOS terminals support true color
return true
#else
return false
#endif
}
// MARK: - Utility Methods
/// Get a human-readable description of terminal capabilities
static func capabilitiesDescription(_ caps: TerminalCapabilities) -> String {
// Get a human-readable description of terminal capabilities
var features: [String] = []
if caps.isInteractive { features.append("interactive") }
if caps.supportsColors { features.append("colors") }
if caps.supportsTrueColor { features.append("truecolor") }
if caps.isCI { features.append("CI-environment") }
if caps.isPiped { features.append("piped") }
let sizeInfo = "\(caps.width)x\(caps.height)"
let termInfo = caps.termType ?? "unknown"
return "\(termInfo) (\(sizeInfo)) - \(features.joined(separator: ", "))"
}
/// Check if we should force a specific output mode based on environment
static func shouldForceOutputMode() -> OutputMode? {
// Check if we should force a specific output mode based on environment
let env = ProcessInfo.processInfo.environment
// Check for explicit output mode environment variables
if let mode = env["PEEKABOO_OUTPUT_MODE"] {
switch mode.lowercased() {
case "minimal", "simple": return .minimal
case "compact": return .compact
case "enhanced", "rich", "tui", "full": return .enhanced
default: break
}
}
// Check for NO_COLOR standard
if env["NO_COLOR"] != nil {
return .minimal
}
// Check for explicit color forcing
if env["FORCE_COLOR"] != nil || env["CLICOLOR_FORCE"] != nil {
return .enhanced
}
return nil
}
}
// MARK: - Output Mode Extensions
extension OutputMode {
/// Get a human-readable description of the output mode
var description: String {
switch self {
case .minimal:
"Minimal (no colors, CI-friendly)"
case .compact:
"Compact (colors and icons)"
case .enhanced:
"Enhanced (rich formatting and progress)"
case .quiet:
"Quiet (results only)"
case .verbose:
"Verbose (debug information)"
}
}
/// Check if this mode supports colors
var supportsColors: Bool {
switch self {
case .minimal, .quiet:
false
case .compact, .enhanced, .verbose:
true
}
}
/// Check if this mode supports rich formatting
var supportsRichFormatting: Bool {
switch self {
case .minimal, .quiet, .compact:
false
case .enhanced, .verbose:
true
}
}
}

View File

@ -1,69 +0,0 @@
import Commander
import Foundation
import PeekabooCore
import PeekabooFoundation
// MARK: - Image Capture Models
// Re-export PeekabooCore types
typealias SavedFile = PeekabooCore.SavedFile
typealias ImageCaptureData = PeekabooCore.ImageCaptureData
/// Extend PeekabooCore types to conform to Commander argument parsing for CLI usage
extension PeekabooCore.CaptureMode: @retroactive ExpressibleFromArgument {
public init?(argument: String) {
self.init(rawValue: argument.lowercased())
}
}
extension PeekabooCore.ImageFormat: @retroactive ExpressibleFromArgument {
public init?(argument: String) {
self.init(rawValue: argument.lowercased())
}
}
extension PeekabooCore.CaptureFocus: @retroactive ExpressibleFromArgument {
public init?(argument: String) {
self.init(rawValue: argument.lowercased())
}
}
// MARK: - Application & Window Models
// Re-export PeekabooCore types
typealias ApplicationInfo = PeekabooCore.ApplicationInfo
typealias ApplicationListData = PeekabooCore.ApplicationListData
typealias WindowInfo = PeekabooCore.WindowInfo
typealias WindowBounds = PeekabooCore.WindowBounds
typealias TargetApplicationInfo = PeekabooCore.TargetApplicationInfo
typealias WindowListData = PeekabooCore.WindowListData
// MARK: - Window Specifier
/// Re-export WindowSpecifier from PeekabooCore
typealias WindowSpecifier = PeekabooCore.WindowSpecifier
// MARK: - Window Details Options
/// Re-export WindowDetailOption from PeekabooCore
typealias WindowDetailOption = PeekabooCore.WindowDetailOption
// MARK: - Window Management
/// Internal window representation with complete details.
///
/// Used internally for window operations, containing all available
/// information about a window including its Core Graphics identifier and bounds.
/// This is CLI-specific and not shared with PeekabooCore.
struct WindowData {
let windowId: UInt32
let title: String
let bounds: CGRect
let isOnScreen: Bool
let windowIndex: Int
}
// MARK: - Error Types
/// Re-export CaptureError from PeekabooFoundation
typealias CaptureError = PeekabooFoundation.CaptureError

View File

@ -1,90 +0,0 @@
import Commander
import CoreGraphics
import Darwin
import Foundation
import PeekabooCore
/// Shared entry point used by the executable target.
@MainActor
public func runPeekabooCLI() async {
let status = await executePeekabooCLI(arguments: CommandLine.arguments)
Darwin.exit(status)
}
/// Internal helper that runs the CLI and returns an exit code (used by tests).
@MainActor
func executePeekabooCLI(arguments: [String]) async -> Int32 {
#if DEBUG
checkBuildStaleness()
#endif
// Initialize CoreGraphics silently to prevent CGS_REQUIRE_INIT error
_ = CGMainDisplayID()
// Load configuration at startup. The singleton initializer already performs
// the initial load, so avoid a second credentials/config read on every CLI invocation.
_ = ConfigurationManager.shared.getConfiguration()
let shouldEmitJSONErrors = containsJSONOutputFlag(arguments)
do {
try await CommanderRuntimeExecutor.resolveAndRun(arguments: arguments)
return EXIT_SUCCESS
} catch let exit as ExitCode {
return exit.rawValue
} catch let programError as CommanderProgramError {
printCommanderError(programError, jsonOutput: shouldEmitJSONErrors)
return EXIT_FAILURE
} catch {
printGenericError(error, jsonOutput: shouldEmitJSONErrors)
return EXIT_FAILURE
}
}
private func containsJSONOutputFlag(_ arguments: [String]) -> Bool {
arguments.contains("--json") || arguments.contains("-j") || arguments.contains("--json-output")
}
private func commanderErrorMessage(_ error: CommanderProgramError) -> String {
switch error {
case let .parsingError(parsing):
parsing.description
case let .unknownCommand(name):
"Unknown command '\(name)'"
case let .unknownSubcommand(command, name):
"Unknown subcommand '\(name)' for command '\(command)'"
case .missingCommand:
"No command specified"
case let .missingSubcommand(command):
"Command '\(command)' requires a subcommand"
}
}
private func printCommanderError(_ error: CommanderProgramError, jsonOutput: Bool) {
let message = commanderErrorMessage(error)
guard jsonOutput else {
fputs("Error: \(message)\n", stderr)
return
}
let logger = Logger.shared
logger.setJsonOutputMode(true)
outputError(message: message, code: .INVALID_ARGUMENT, logger: logger)
}
private func printGenericError(_ error: any Error, jsonOutput: Bool) {
let code: ErrorCode = if error is CommanderBindingError {
.INVALID_ARGUMENT
} else {
.UNKNOWN_ERROR
}
guard jsonOutput else {
fputs("Error: \(error.localizedDescription)\n", stderr)
return
}
let logger = Logger.shared
logger.setJsonOutputMode(true)
outputError(message: error.localizedDescription, code: code, logger: logger)
}

View File

@ -1,101 +0,0 @@
import Foundation
import PeekabooCore
import PeekabooFoundation
/// Protocol for commands that can resolve application identifiers from various inputs
protocol ApplicationResolvable {
/// Application name, bundle ID, or 'PID:12345' format
var app: String? { get }
/// Process ID as a direct parameter
var pid: Int32? { get }
}
extension ApplicationResolvable {
/// Returns a PID when the command explicitly targets one, including the documented `--app PID:<pid>` form.
func resolveExplicitPIDObservationTarget() throws -> Int32? {
if let pid, self.app == nil {
return pid
}
guard let appValue = self.app?.trimmingCharacters(in: .whitespacesAndNewlines),
appValue.uppercased().hasPrefix("PID:")
else {
return nil
}
let appPidString = String(appValue.dropFirst("PID:".count))
guard let appPid = Int32(appPidString) else {
throw PeekabooError.invalidInput("Invalid PID format in --app: '\(appValue)'")
}
if let pid, pid != appPid {
throw PeekabooError.invalidInput(
"Conflicting PIDs: --app specifies PID \(appPid) but --pid is \(pid)"
)
}
return appPid
}
/// Resolves the application identifier from app and/or pid parameters
/// Supports lenient handling for redundant but non-conflicting parameters
func resolveApplicationIdentifier() throws -> String {
// Resolves the application identifier from app and/or pid parameters
switch (app, pid) {
case (nil, nil):
throw PeekabooError.invalidInput("Either --app or --pid must be specified")
case (let appValue?, nil):
// Only --app provided, use as-is (supports "PID:12345" format)
return appValue
case (nil, let pidValue?):
// Only --pid provided, convert to PID: format
return "PID:\(pidValue)"
case let (appValue?, pidValue?):
// Both provided - need to validate they don't conflict
return try self.validateAndResolveBothParameters(app: appValue, pid: pidValue)
}
}
/// Validates when both app and pid parameters are provided
private func validateAndResolveBothParameters(app: String, pid: Int32) throws -> String {
// Case 1: Check if app is already in PID format
if app.hasPrefix("PID:") {
let appPidString = String(app.dropFirst(4))
if let appPid = Int32(appPidString) {
// Both specify PID - they must match
if appPid == pid {
// Redundant but consistent - this is OK
return app
} else {
throw PeekabooError.invalidInput(
"Conflicting PIDs: --app specifies PID \(appPid) but --pid is \(pid)"
)
}
} else {
throw PeekabooError.invalidInput("Invalid PID format in --app: '\(app)'")
}
}
// Case 2: app is a name/bundle ID, pid is provided.
// We can't reliably cross-check names vs. PIDs without AppKit/main-thread inspection.
// Log the redundancy and prefer the textual identifier for readability.
return app
}
}
/// Extension for commands with positional app argument (like AppCommand subcommands)
protocol ApplicationResolvablePositional: ApplicationResolvable {
/// Positional application argument captured as a non-optional string.
var positionalAppIdentifier: String { get }
var pid: Int32? { get }
}
extension ApplicationResolvablePositional {
var app: String? {
self.positionalAppIdentifier
}
}

View File

@ -1,305 +0,0 @@
import Foundation
/// Check if the CLI binary is stale compared to the current git state.
/// Only runs in debug builds when git config 'peekaboo.check-build-staleness' is true.
func checkBuildStaleness() {
guard isBuildStalenessCheckEnabled() else { return }
// Check 1: Git commit comparison
checkGitCommitStaleness()
// Check 2: File modification time comparison
checkFileModificationStaleness()
}
/// Return true when `peekaboo.check-build-staleness` is enabled.
///
/// This runs on every debug CLI start, so avoid spawning `git config` for the common
/// disabled path. Environment override keeps a cheap opt-in for CI and local debugging.
func isBuildStalenessCheckEnabled(
environment: [String: String] = ProcessInfo.processInfo.environment,
currentDirectory: String = FileManager.default.currentDirectoryPath,
gitConfigPaths: [String]? = nil
) -> Bool {
if let override = environment["PEEKABOO_CHECK_BUILD_STALENESS"]?.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased(),
!override.isEmpty {
return override == "1" || override == "true" || override == "yes"
}
var setting: Bool?
for path in gitConfigPaths ?? defaultGitConfigPaths(environment: environment, currentDirectory: currentDirectory) {
guard let contents = try? String(contentsOfFile: path, encoding: .utf8),
let parsedSetting = parseBuildStalenessSetting(from: contents)
else {
continue
}
setting = parsedSetting
}
return setting == true
}
func parseBuildStalenessSetting(from gitConfig: String) -> Bool? {
var inPeekabooSection = false
for rawLine in gitConfig.components(separatedBy: .newlines) {
let line = rawLine.trimmingCharacters(in: .whitespacesAndNewlines)
guard !line.isEmpty, !line.hasPrefix("#"), !line.hasPrefix(";") else { continue }
if line.hasPrefix("[") && line.hasSuffix("]") {
let section = line.dropFirst().dropLast().trimmingCharacters(in: .whitespacesAndNewlines)
inPeekabooSection = section == "peekaboo"
continue
}
guard inPeekabooSection else { continue }
let parts = line.split(separator: "=", maxSplits: 1).map {
$0.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
}
guard parts.count == 2, parts[0] == "check-build-staleness" else { continue }
return parts[1] == "true" || parts[1] == "1" || parts[1] == "yes"
}
return nil
}
private func defaultGitConfigPaths(environment: [String: String], currentDirectory: String) -> [String] {
var paths = ["/etc/gitconfig"]
if let xdgConfigHome = environment["XDG_CONFIG_HOME"], !xdgConfigHome.isEmpty {
paths.append(URL(fileURLWithPath: xdgConfigHome).appendingPathComponent("git/config").path)
} else if let home = environment["HOME"], !home.isEmpty {
paths.append(URL(fileURLWithPath: home).appendingPathComponent(".config/git/config").path)
}
if let home = environment["HOME"], !home.isEmpty {
paths.append(URL(fileURLWithPath: home).appendingPathComponent(".gitconfig").path)
}
if let localConfigPath = findGitConfigPath(startingAt: currentDirectory) {
paths.append(localConfigPath)
}
return paths
}
private func findGitConfigPath(startingAt path: String) -> String? {
let fileManager = FileManager.default
var directory = URL(fileURLWithPath: path, isDirectory: true).standardizedFileURL
while true {
let dotGit = directory.appendingPathComponent(".git").path
var isDirectory: ObjCBool = false
if fileManager.fileExists(atPath: dotGit, isDirectory: &isDirectory) {
if isDirectory.boolValue {
return URL(fileURLWithPath: dotGit).appendingPathComponent("config").path
}
if let contents = try? String(contentsOfFile: dotGit, encoding: .utf8),
let gitDirLine = contents.components(separatedBy: .newlines).first(where: {
$0.trimmingCharacters(in: .whitespaces).hasPrefix("gitdir:")
}) {
let rawGitDir = gitDirLine
.replacingOccurrences(of: "gitdir:", with: "")
.trimmingCharacters(in: .whitespacesAndNewlines)
let gitDirURL = URL(fileURLWithPath: rawGitDir, relativeTo: directory).standardizedFileURL
return gitDirURL.appendingPathComponent("config").path
}
}
let parent = directory.deletingLastPathComponent()
if parent.path == directory.path { return nil }
directory = parent
}
}
/// Check if the embedded git commit differs from the current git commit
private func checkGitCommitStaleness() {
// Get current git commit hash
let gitProcess = Process()
gitProcess.executableURL = URL(fileURLWithPath: "/usr/bin/git")
gitProcess.arguments = ["rev-parse", "--short", "HEAD"]
let gitPipe = Pipe()
gitProcess.standardOutput = gitPipe
gitProcess.standardError = Pipe() // Silence stderr
do {
try gitProcess.run()
gitProcess.waitUntilExit()
guard gitProcess.terminationStatus == 0 else {
return // Git command failed, skip check
}
let gitData = gitPipe.fileHandleForReading.readDataToEndOfFile()
let rawCommitString = String(data: gitData, encoding: .utf8)
let currentCommit = rawCommitString?
.trimmingCharacters(in: .whitespacesAndNewlines) ?? ""
// Get embedded commit from build (strip -dirty suffix if present)
let embeddedCommit = Version.gitCommit.replacingOccurrences(of: "-dirty", with: "")
// Compare commits
if !currentCommit.isEmpty && currentCommit != embeddedCommit {
logError("❌ CLI binary is outdated and needs to be rebuilt!")
logError(" Built with commit: \(embeddedCommit)")
logError(" Current commit: \(currentCommit)")
logError("")
logError(" Run ./scripts/build-swift-debug.sh to rebuild")
exit(1)
}
} catch {
return // Git command failed, skip check
}
}
/// Check if any tracked files have been modified after the build time
private func checkFileModificationStaleness() {
// Parse build date from Version.buildDate (ISO 8601 format)
let dateFormatter = ISO8601DateFormatter()
guard let buildDate = dateFormatter.date(from: Version.buildDate) else {
return // Could not parse build date, skip check
}
// Get git repository root
guard let gitRoot = getGitRepositoryRoot() else {
return // Could not determine git root, skip check
}
// Get list of modified files from git status
let gitStatusProcess = Process()
gitStatusProcess.executableURL = URL(fileURLWithPath: "/usr/bin/git")
gitStatusProcess.arguments = ["status", "--porcelain=1"]
let statusPipe = Pipe()
gitStatusProcess.standardOutput = statusPipe
gitStatusProcess.standardError = Pipe() // Silence stderr
do {
try gitStatusProcess.run()
gitStatusProcess.waitUntilExit()
guard gitStatusProcess.terminationStatus == 0 else {
return // Git command failed, skip check
}
let statusData = statusPipe.fileHandleForReading.readDataToEndOfFile()
let statusOutput = String(data: statusData, encoding: .utf8) ?? ""
// Parse git status output
let modifiedFiles = parseGitStatusOutput(statusOutput)
// Check each modified file's modification time
for filePath in modifiedFiles where
isFileNewerThanBuild(filePath: filePath, buildDate: buildDate, gitRoot: gitRoot) {
logError("❌ CLI binary is outdated and needs to be rebuilt!")
logError(" Build time: \(Version.buildDate)")
logError(" Modified file: \(filePath)")
logError("")
logError(" Run ./scripts/build-swift-debug.sh to rebuild")
exit(1)
}
} catch {
return // Git command failed, skip check
}
}
/// Parse git status --porcelain=1 output to extract file paths
/// Format: "XY filename" or "XY orig_path -> new_path" for renames
private func parseGitStatusOutput(_ output: String) -> [String] {
// Parse git status --porcelain=1 output to extract file paths
let lines = output.components(separatedBy: .newlines)
var filePaths: [String] = []
for line in lines {
let trimmed = line.trimmingCharacters(in: .whitespaces)
guard !trimmed.isEmpty else { continue }
// Git status format: "XY filename" or "XY orig_path -> new_path"
// X = staged status, Y = working tree status
guard trimmed.count >= 3 else { continue }
let statusCodes = String(trimmed.prefix(2))
var filePath = String(trimmed.dropFirst(2)) // Skip "XY"
// Remove leading space if present
if filePath.hasPrefix(" ") {
filePath = String(filePath.dropFirst())
}
// Include files that are modified (M), added (A), or have other changes
// Skip deleted files (D) since they can't be newer than build
if statusCodes.contains("M") || statusCodes.contains("A") || statusCodes.contains("R") || statusCodes
.contains("C") || statusCodes.contains("U") {
// Handle renamed files: "orig_path -> new_path"
// For renames, we want to check the new path
if filePath.contains(" -> ") {
let components = filePath.components(separatedBy: " -> ")
if components.count == 2 {
filePath = components[1] // Use the new path
}
}
// Handle quoted paths (git quotes paths with special characters)
let cleanPath = filePath.hasPrefix("\"") && filePath.hasSuffix("\"")
? String(filePath.dropFirst().dropLast())
: filePath
filePaths.append(cleanPath)
}
}
return filePaths
}
/// Get the git repository root directory
private func getGitRepositoryRoot() -> String? {
// Get the git repository root directory
let gitProcess = Process()
gitProcess.executableURL = URL(fileURLWithPath: "/usr/bin/git")
gitProcess.arguments = ["rev-parse", "--show-toplevel"]
let pipe = Pipe()
gitProcess.standardOutput = pipe
gitProcess.standardError = Pipe() // Silence stderr
do {
try gitProcess.run()
gitProcess.waitUntilExit()
guard gitProcess.terminationStatus == 0 else {
return nil
}
let data = pipe.fileHandleForReading.readDataToEndOfFile()
let output = String(data: data, encoding: .utf8)?.trimmingCharacters(in: .whitespacesAndNewlines)
// Check if output is empty after trimming
guard let output, !output.isEmpty else {
return nil
}
return output
} catch {
return nil
}
}
/// Check if a file's modification time is newer than the build date
private func isFileNewerThanBuild(filePath: String, buildDate: Date, gitRoot: String) -> Bool {
// Check if a file's modification time is newer than the build date
let fileManager = FileManager.default
// Git status paths are relative to repository root, not current directory
let fullPath = (filePath.hasPrefix("/")) ? filePath : "\(gitRoot)/\(filePath)"
do {
let attributes = try fileManager.attributesOfItem(atPath: fullPath)
if let modificationDate = attributes[.modificationDate] as? Date {
return modificationDate > buildDate
}
} catch {
// File might not exist or be accessible, skip this check
return false
}
return false
}

View File

@ -1,47 +0,0 @@
import Foundation
import PeekabooCore
import PeekabooFoundation
// MARK: - Common Error Handling
private func emitError(
message: String,
code: ErrorCode,
jsonOutput: Bool,
logger: Logger,
prefix: String = ""
) {
if jsonOutput {
let response = JSONResponse(
success: false,
error: ErrorInfo(
message: message,
code: code
)
)
outputJSON(response, logger: logger)
} else {
print("\(prefix) \(message)")
}
}
// ApplicationError has been replaced by PeekabooError
// Callers should use handleGenericError instead
func handleGenericError(_ error: any Error, jsonOutput: Bool, logger: Logger) {
emitError(
message: error.localizedDescription,
code: .UNKNOWN_ERROR,
jsonOutput: jsonOutput,
logger: logger
)
}
func handleValidationError(_ error: any Error, jsonOutput: Bool, logger: Logger) {
emitError(
message: error.localizedDescription,
code: .VALIDATION_ERROR,
jsonOutput: jsonOutput,
logger: logger
)
}

View File

@ -1,18 +0,0 @@
import Foundation
import os
/// OS Logger instance for CLI-specific logging using the unified logging system
/// This complements the custom Logger class used for CLI output formatting
extension os.Logger {
/// Logger for CLI-specific operations
static let cli = os.Logger(subsystem: "boo.peekaboo.cli", category: "CLI")
/// Logger for CLI command execution
static let command = os.Logger(subsystem: "boo.peekaboo.cli", category: "Command")
/// Logger for CLI configuration
static let config = os.Logger(subsystem: "boo.peekaboo.cli", category: "Config")
/// Logger for CLI errors
static let error = os.Logger(subsystem: "boo.peekaboo.cli", category: "Error")
}

View File

@ -1,177 +0,0 @@
import Foundation
import PeekabooAgentRuntime
@MainActor
final class AgentChatEventDelegate: AgentEventDelegate {
private weak var ui: AgentChatUI?
private var lastToolArguments: [String: [String: Any]] = [:]
init(ui: AgentChatUI) {
self.ui = ui
}
func agentDidEmitEvent(_ event: AgentEvent) {
guard let ui else { return }
switch event {
case .started:
break
case let .assistantMessage(content):
ui.appendAssistant(content)
case let .thinkingMessage(content):
ui.updateThinking(content)
case let .toolCallStarted(name, arguments):
self.handleToolStarted(name: name, arguments: arguments, ui: ui)
case let .toolCallCompleted(name, result):
self.handleToolCompleted(name: name, result: result, ui: ui)
case let .toolCallUpdated(name, arguments):
self.handleToolUpdated(name: name, arguments: arguments, ui: ui)
case .verificationCompleted, .desktopContextRefreshed:
break
case let .error(message):
ui.showError(message)
case .completed:
ui.finishStreaming()
case .queueDrained:
break
}
}
private func handleToolStarted(name: String, arguments: String, ui: AgentChatUI) {
let args = self.parseArguments(arguments)
self.lastToolArguments[name] = args
let formatter = self.toolFormatter(for: name)
let toolType = ToolType(rawValue: name)
let summary = formatter?.formatStarting(arguments: args) ??
name.replacingOccurrences(of: "_", with: " ")
ui.showToolStart(
name: name,
summary: summary,
icon: toolType?.icon,
displayName: toolType?.displayName
)
}
private func handleToolCompleted(name: String, result: String, ui: AgentChatUI) {
let summary = self.toolResultSummary(name: name, result: result)
let success = self.successFlag(from: result)
let toolType = ToolType(rawValue: name)
ui.showToolCompletion(
name: name,
success: success,
summary: summary,
icon: toolType?.icon,
displayName: toolType?.displayName
)
}
private func handleToolUpdated(name: String, arguments: String, ui: AgentChatUI) {
let args = self.parseArguments(arguments)
if let previous = self.lastToolArguments[name], self.dictionariesEqual(previous, args) {
return
}
let formatter = self.toolFormatter(for: name)
let toolType = ToolType(rawValue: name)
let summary = self.diffSummary(for: name, newArgs: args)
?? formatter?.formatStarting(arguments: args)
?? name.replacingOccurrences(of: "_", with: " ")
ui.showToolUpdate(
name: name,
summary: summary,
icon: toolType?.icon,
displayName: toolType?.displayName
)
self.lastToolArguments[name] = args
}
private func toolFormatter(for name: String) -> (any ToolFormatter)? {
if let type = ToolType(rawValue: name) {
return ToolFormatterRegistry.shared.formatter(for: type)
}
return nil
}
private func parseArguments(_ jsonString: String) -> [String: Any] {
guard let data = jsonString.data(using: .utf8),
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else {
return [:]
}
return json
}
private func parseResult(_ jsonString: String) -> [String: Any]? {
guard let data = jsonString.data(using: .utf8),
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else {
return nil
}
return json
}
private func toolResultSummary(name: String, result: String) -> String? {
guard let json = self.parseResult(result) else { return nil }
if let summary = ToolEventSummary.from(resultJSON: json)?.shortDescription(toolName: name) {
return summary
}
let formatter = self.toolFormatter(for: name)
return formatter?.formatResultSummary(result: json)
}
private func successFlag(from result: String) -> Bool {
guard let json = self.parseResult(result) else { return true }
return (json["success"] as? Bool) ?? true
}
/// Minimal diff between previous and new args for the same tool name.
private func diffSummary(for toolName: String, newArgs: [String: Any]) -> String? {
guard let previous = self.lastToolArguments[toolName] else { return nil }
var changes: [String] = []
for (key, newValue) in newArgs {
guard let prevValue = previous[key] else {
changes.append("+\(key)")
continue
}
if !self.valuesEqual(prevValue, newValue) {
let rendered = self.renderValue(newValue)
changes.append("\(key): \(rendered)")
}
if changes.count >= 3 { break }
}
if changes.isEmpty { return nil }
return changes.joined(separator: ", ")
}
private func valuesEqual(_ lhs: Any, _ rhs: Any) -> Bool {
switch (lhs, rhs) {
case let (l as String, r as String): l == r
case let (l as Int, r as Int): l == r
case let (l as Double, r as Double): l == r
case let (l as Bool, r as Bool): l == r
default: false
}
}
private func dictionariesEqual(_ lhs: [String: Any], _ rhs: [String: Any]) -> Bool {
guard lhs.count == rhs.count else { return false }
for (key, lval) in lhs {
guard let rval = rhs[key], self.valuesEqual(lval, rval) else { return false }
}
return true
}
private func renderValue(_ value: Any) -> String {
switch value {
case let str as String:
let max = 32
if str.count > max {
let idx = str.index(str.startIndex, offsetBy: max)
return String(str[..<idx]) + ""
}
return str
case let num as Int: return String(num)
case let num as Double: return String(format: "%.3f", num)
case let bool as Bool: return bool ? "true" : "false"
default: return ""
}
}
}

View File

@ -1,40 +0,0 @@
//
// AgentChatLaunchPolicy.swift
// PeekabooCLI
//
import Foundation
enum ChatLaunchStrategy: Equatable {
case none
case helpOnly
case interactive(initialPrompt: String?)
}
struct AgentChatLaunchContext {
let chatFlag: Bool
let hasTaskInput: Bool
let listSessions: Bool
let normalizedTaskInput: String?
let capabilities: TerminalCapabilities
}
/// Determines how the agent should launch chat mode based on flags and terminal context.
@available(macOS 14.0, *)
struct AgentChatLaunchPolicy {
func strategy(for context: AgentChatLaunchContext) -> ChatLaunchStrategy {
if context.chatFlag {
return .interactive(initialPrompt: context.normalizedTaskInput)
}
if context.hasTaskInput || context.listSessions {
return .none
}
if context.capabilities.isInteractive && !context.capabilities.isPiped && !context.capabilities.isCI {
return .interactive(initialPrompt: nil)
}
return .helpOnly
}
}

View File

@ -1,26 +0,0 @@
//
// AgentChatPreconditions.swift
// PeekabooCLI
//
import Foundation
enum AgentChatPreconditions {
struct Flags {
let jsonOutput: Bool
let quiet: Bool
let dryRun: Bool
let noCache: Bool
let audio: Bool
let audioFileProvided: Bool
}
static func firstViolation(for flags: Flags) -> String? {
if flags.jsonOutput { return AgentMessages.Chat.jsonDisabled }
if flags.quiet { return AgentMessages.Chat.quietDisabled }
if flags.dryRun { return AgentMessages.Chat.dryRunDisabled }
if flags.noCache { return AgentMessages.Chat.noCacheDisabled }
if flags.audio || flags.audioFileProvided { return AgentMessages.Chat.typedOnly }
return nil
}
}

View File

@ -1,85 +0,0 @@
import TauTUI
/// Minimal loader component to keep chat rendering responsive without pulling in full spinner logic.
@MainActor
final class AgentChatLoader: Component {
private var message: String
init(tui: TUI, message: String) {
self.message = message
}
func setMessage(_ message: String) {
self.message = message
}
func stop() {}
func render(width: Int) -> [String] {
["\(self.message)"]
}
}
@MainActor
final class AgentChatInput: Component {
private let editor = Editor()
var onSubmit: ((String) -> Void)?
var onCancel: (() -> Void)?
var onInterrupt: (() -> Void)?
var onQueueWhileLocked: (() -> Void)?
var isLocked: Bool = false {
didSet {
if !self.isLocked {
self.editor.disableSubmit = false
}
}
}
init() {
self.editor.onSubmit = { [weak self] value in
self?.onSubmit?(value)
}
}
func render(width: Int) -> [String] {
self.editor.render(width: width)
}
func handle(input: TerminalInput) {
switch input {
case let .key(.character(char), modifiers):
if modifiers.contains(.control) {
let lower = String(char).lowercased()
if lower == "c" || lower == "d" {
self.onInterrupt?()
return
}
}
case .key(.escape, _):
if self.isLocked {
self.onCancel?()
return
}
case .key(.end, _):
if self.isLocked {
// End lets a user keep typing while the current run owns normal submit.
self.onQueueWhileLocked?()
return
}
default:
break
}
self.editor.handle(input: input)
}
func clear() {
self.editor.setText("")
}
func currentText() -> String {
self.editor.getText()
}
}

View File

@ -1,340 +0,0 @@
//
// AgentChatUI.swift
// PeekabooCLI
//
import Foundation
import PeekabooAgentRuntime
import TauTUI
@MainActor
final class AgentChatUI {
var onCancelRequested: (() -> Void)?
var onInterruptRequested: (() -> Void)?
private let tui: TUI
private let messages = Container()
private let input = AgentChatInput()
private let header: Text
private let sessionLine: Text
private let helpLines: [String]
private let queueMode: QueueMode
private let queueContainer = Container()
private let queuePreview = Text(text: "", paddingX: 1, paddingY: 0)
// Palette for consistent styling (ANSI colors)
private let accentBlue = AnsiStyling.color(39)
private let successGreen = AnsiStyling.color(82)
private let failureRed = AnsiStyling.color(203)
private let thinkingGray = AnsiStyling.color(246)
private var promptContinuation: AsyncStream<String>.Continuation?
private var loader: AgentChatLoader?
private var assistantBuffer = ""
private var assistantComponent: MarkdownComponent?
private var thinkingBlocks: [MarkdownComponent] = []
private var sessionId: String?
private var queuedPrompts: [String] = []
private var isRunning = false
init(modelDescription: String, sessionId: String?, queueMode: QueueMode, helpLines: [String]) {
self.tui = TUI(terminal: ProcessTerminal())
self.sessionId = sessionId
self.helpLines = helpLines
self.queueMode = queueMode
let queueLabel = queueMode == .all ? "all" : "one-at-a-time"
self.header = Text(
text: "Interactive agent chat model: \(modelDescription) • queue: \(queueLabel)",
paddingX: 1,
paddingY: 0
)
self.sessionLine = Text(
text: AgentChatUI.sessionDescription(for: sessionId, queueMode: queueMode),
paddingX: 1,
paddingY: 0
)
self.input.onSubmit = { [weak self] value in
self?.handleSubmit(value)
}
self.input.onCancel = { [weak self] in
self?.onCancelRequested?()
}
self.input.onInterrupt = { [weak self] in
self?.onInterruptRequested?()
}
self.input.onQueueWhileLocked = { [weak self] in
self?.queueCurrentInput()
}
}
func start() throws {
self.tui.addChild(self.header)
self.tui.addChild(self.sessionLine)
self.tui.addChild(Spacer(lines: 1))
self.tui.addChild(self.messages)
self.tui.addChild(Spacer(lines: 1))
self.tui.addChild(self.queueContainer)
self.tui.addChild(self.input)
self.tui.setFocus(self.input)
try self.tui.start()
self.showHelpMenu()
self.tui.requestRender()
}
func stop() {
self.tui.stop()
}
func promptStream(initialPrompt: String?) -> AsyncStream<String> {
AsyncStream { continuation in
self.promptContinuation = continuation
if let seed = initialPrompt?.trimmingCharacters(in: .whitespacesAndNewlines),
!seed.isEmpty {
self.appendUserMessage(seed)
continuation.yield(seed)
}
}
}
func finishPromptStream() {
self.promptContinuation?.finish()
}
func beginRun(prompt: String) {
self.setRunning(true)
self.removeLoader()
self.loader = AgentChatLoader(tui: self.tui, message: "Running…")
if let loader {
self.messages.addChild(loader)
}
self.assistantBuffer = ""
self.assistantComponent = nil
self.thinkingBlocks.removeAll()
self.requestRender()
}
func endRun(result: AgentExecutionResult, sessionId: String?) {
self.loader?.stop()
self.loader = nil
if let sessionId {
self.sessionId = sessionId
self.sessionLine.text = AgentChatUI.sessionDescription(for: sessionId, queueMode: self.queueMode)
}
let summary = self.summaryLine(for: result)
let summaryComponent = Text(text: summary, paddingX: 1, paddingY: 0)
self.messages.addChild(summaryComponent)
self.setRunning(false)
self.processNextQueuedPromptIfNeeded()
self.requestRender()
}
func showHelpMenu() {
// Render each line separately so the bullets always appear on their own lines,
// even when terminals collapse single newlines in a single Text component.
for line in self.helpLines {
let helpLine = Text(text: line, paddingX: 1, paddingY: 0)
self.messages.addChild(helpLine)
}
self.requestRender()
}
func showCancelled() {
self.setRunning(false)
let cancelled = Text(text: "◼︎ Cancelled", paddingX: 1, paddingY: 0)
self.messages.addChild(cancelled)
self.requestRender()
}
func showError(_ message: String) {
self.setRunning(false)
let errorText = Text(text: "\(message)", paddingX: 1, paddingY: 0)
self.messages.addChild(errorText)
self.requestRender()
}
func showToolStart(name: String, summary: String?, icon: String?, displayName: String?) {
let label = displayName ?? name
let detail = summary.flatMap { $0.isEmpty ? nil : $0 }
let body = detail.map { "**\(label)** \($0)" } ?? "**\(label)**"
let content = ["", icon, body].compactMap(\.self).joined(separator: " ")
self.messages.addChild(self.colorLine(content, color: self.accentBlue))
self.requestRender()
}
func showToolCompletion(name: String, success: Bool, summary: String?, icon: String?, displayName: String?) {
let prefix = success ? "" : ""
let color = success ? self.successGreen : self.failureRed
let label = displayName ?? name
let detail = summary.flatMap { $0.isEmpty ? nil : $0 }
let body = detail.map { "**\(label)** \($0)" } ?? "**\(label)**"
let content = [prefix, icon, body].compactMap(\.self).joined(separator: " ")
self.messages.addChild(self.colorLine(content, color: color))
self.requestRender()
}
func showToolUpdate(name: String, summary: String?, icon: String?, displayName: String?) {
let label = displayName ?? name
let detail = summary.flatMap { $0.isEmpty ? nil : $0 }
let body = detail.map { "**\(label)** \($0)" } ?? "**\(label)**"
let content = ["", icon, body].compactMap(\.self).joined(separator: " ")
self.messages.addChild(self.colorLine(content, color: self.accentBlue))
self.requestRender()
}
func updateThinking(_ content: String) {
let component = MarkdownComponent(
text: "*\(content)*",
padding: .init(horizontal: 1, vertical: 0),
defaultTextStyle: .init(color: self.thinkingGray)
)
self.thinkingBlocks.append(component)
self.messages.addChild(component)
self.requestRender()
}
func appendAssistant(_ content: String) {
self.assistantBuffer.append(content)
let formatted = "**Agent:** \(self.assistantBuffer)"
if let assistantComponent {
assistantComponent.text = formatted
} else {
let component = MarkdownComponent(text: formatted, padding: .init(horizontal: 1, vertical: 0))
self.assistantComponent = component
self.messages.addChild(component)
}
self.requestRender()
}
func finishStreaming() {
self.requestRender()
}
func setRunning(_ running: Bool) {
let wasRunning = self.isRunning
self.isRunning = running
self.input.isLocked = running
if !running {
self.removeLoader()
if wasRunning {
self.processNextQueuedPromptIfNeeded()
}
}
}
func markCancelling() {
self.loader?.setMessage("Cancelling…")
self.requestRender()
}
func requestRender() {
self.tui.requestRender()
}
private func colorLine(_ text: String, color: @escaping AnsiStyling.Style) -> MarkdownComponent {
MarkdownComponent(
text: text,
padding: .init(horizontal: 1, vertical: 0),
defaultTextStyle: .init(color: color)
)
}
private func removeLoader() {
guard let loader else { return }
loader.stop()
self.messages.removeChild(loader)
self.loader = nil
self.requestRender()
}
private func handleSubmit(_ raw: String) {
let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return }
if self.isRunning {
self.enqueueQueuedPrompt(trimmed)
self.input.clear()
return
}
self.dispatchPrompt(trimmed)
}
private func queueCurrentInput() {
guard self.isRunning else { return }
let trimmed = self.input.currentText().trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return }
self.enqueueQueuedPrompt(trimmed)
self.input.clear()
}
private func enqueueQueuedPrompt(_ prompt: String) {
self.queuedPrompts.append(prompt)
self.updateQueuePreview()
}
private func updateQueuePreview() {
if self.queuedPrompts.isEmpty {
self.queueContainer.clear()
self.queuePreview.text = ""
self.requestRender()
return
}
self.queuePreview.text = self.queuePreviewLine()
if self.queueContainer.children.isEmpty {
self.queueContainer.addChild(self.queuePreview)
}
self.requestRender()
}
private func queuePreviewLine() -> String {
let joined = self.queuedPrompts.joined(separator: " · ")
var summary = "Queued (\(self.queuedPrompts.count)): \(joined)"
let limit = 96
if summary.count > limit {
let index = summary.index(summary.startIndex, offsetBy: max(0, limit - 1))
summary = String(summary[..<index]) + ""
}
return summary
}
private func processNextQueuedPromptIfNeeded() {
guard !self.queuedPrompts.isEmpty else { return }
let next = self.queuedPrompts.removeFirst()
self.updateQueuePreview()
self.dispatchPrompt(next)
}
func drainQueuedPrompts() -> [String] {
let queued = self.queuedPrompts
self.queuedPrompts.removeAll()
self.updateQueuePreview()
return queued
}
private func dispatchPrompt(_ text: String) {
self.appendUserMessage(text)
self.promptContinuation?.yield(text)
}
private func appendUserMessage(_ text: String) {
let message = MarkdownComponent(text: "**You:** \(text)", padding: .init(horizontal: 1, vertical: 0))
self.messages.addChild(message)
self.requestRender()
}
private func summaryLine(for result: AgentExecutionResult) -> String {
let duration = String(format: "%.1fs", result.metadata.executionTime)
let tools = result.metadata.toolCallCount == 1 ? "1 tool" : "\(result.metadata.toolCallCount) tools"
let sessionFragment = self.sessionId.map { String($0.prefix(8)) } ?? "new session"
return "✓ Session \(sessionFragment)\(duration)\(tools)"
}
private static func sessionDescription(for sessionId: String?, queueMode: QueueMode) -> String {
let base = sessionId.map { "Session: \($0)" } ?? "Session: new (will be created on first run)"
let mode = queueMode == .all ? "queue: all" : "queue: one-at-a-time"
return "\(base)\(mode)"
}
}

View File

@ -1,130 +0,0 @@
//
// AgentCommand+Audio.swift
// PeekabooCLI
//
import Darwin
import Dispatch
import Foundation
import PeekabooCore
@available(macOS 14.0, *)
extension AgentCommand {
func buildExecutionTask() async throws -> String? {
if self.audio || self.audioFile != nil {
return try await self.processAudioInput()
}
guard let providedTask = self.task else {
self.printMissingTaskError(message: "Task argument is required", usage: "")
return nil
}
return providedTask
}
private func processAudioInput() async throws -> String? {
self.logAudioStartMessage()
let audioService = self.services.audioInput
do {
let transcript = try await self.transcribeAudio(using: audioService)
self.logTranscriptionSuccess(transcript)
return self.composeExecutionTask(with: transcript)
} catch {
self.logAudioError(error)
return nil
}
}
private func logAudioStartMessage() {
guard !self.jsonOutput && !self.quiet else { return }
if let audioPath = self.audioFile {
print("\(TerminalColor.cyan)🎙️ Processing audio file: \(audioPath)\(TerminalColor.reset)")
} else {
let recordingMessage = [
"\(TerminalColor.cyan)🎙️ Starting audio recording...",
" (Press Ctrl+C to stop)\(TerminalColor.reset)"
].joined()
print(recordingMessage)
}
}
private func transcribeAudio(using audioService: AudioInputService) async throws -> String {
if let audioPath = self.audioFile {
let url = URL(fileURLWithPath: PathResolver.expandPath(audioPath))
return try await audioService.transcribeAudioFile(url)
} else {
try await audioService.startRecording()
return try await self.captureMicrophoneAudio(using: audioService)
}
}
private func captureMicrophoneAudio(using audioService: AudioInputService) async throws -> String {
try await withTaskCancellationHandler {
try await withCheckedThrowingContinuation { continuation in
let signalSource = DispatchSource.makeSignalSource(signal: SIGINT, queue: .main)
signalSource.setEventHandler {
signalSource.cancel()
Task { @MainActor in
do {
let transcript = try await audioService.stopRecording()
continuation.resume(returning: transcript)
} catch {
continuation.resume(throwing: error)
}
}
}
signalSource.resume()
}
} onCancel: {
Task { @MainActor in
_ = try? await audioService.stopRecording()
}
}
}
private func logTranscriptionSuccess(_ transcript: String) {
guard !self.jsonOutput && !self.quiet else { return }
let message = [
"\(TerminalColor.green)\(AgentDisplayTokens.Status.success) Transcription complete",
"\(TerminalColor.reset)"
].joined()
print(message)
print("\(TerminalColor.gray)Transcript: \(transcript.prefix(100))...\(TerminalColor.reset)")
}
private func composeExecutionTask(with transcript: String) -> String {
Self.composeExecutionTask(providedTask: self.task, transcript: transcript)
}
static func composeExecutionTask(providedTask: String?, transcript: String) -> String {
guard let providedTask, !providedTask.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty else {
return transcript
}
return "\(providedTask)\n\nAudio transcript:\n\(transcript)"
}
private func logAudioError(_ error: any Error) {
let message = AgentMessages.Audio.processingError(error)
if self.jsonOutput {
let errorObj = [
"success": false,
"error": message
] as [String: Any]
if let jsonData = try? JSONSerialization.data(withJSONObject: errorObj, options: .prettyPrinted),
let jsonString = String(data: jsonData, encoding: .utf8) {
print(jsonString)
} else {
print("{\"success\":false,\"error\":\"\(AgentMessages.Audio.genericProcessingError)\"}")
}
} else {
let failurePrefix = [
"\(TerminalColor.red)\(AgentDisplayTokens.Status.failure)",
" ",
message
].joined()
let audioErrorMessage = [failurePrefix, "\(TerminalColor.reset)"].joined()
print(audioErrorMessage)
}
}
}

View File

@ -1,409 +0,0 @@
//
// AgentCommand+Chat.swift
// PeekabooCLI
//
import Foundation
import PeekabooAgentRuntime
import PeekabooCore
import PeekabooFoundation
import Tachikoma
import TauTUI
@available(macOS 14.0, *)
extension AgentCommand {
private func ensureChatModePreconditions() -> Bool {
let flags = AgentChatPreconditions.Flags(
jsonOutput: self.jsonOutput,
quiet: self.quiet,
dryRun: self.dryRun,
noCache: self.noCache,
audio: self.audio,
audioFileProvided: self.audioFile != nil
)
if let violation = AgentChatPreconditions.firstViolation(for: flags) {
self.printAgentExecutionError(violation)
return false
}
return true
}
func printNonInteractiveChatHelp() {
if self.jsonOutput {
self
.printAgentExecutionError(
AgentMessages.Chat.nonInteractiveHelp
)
return
}
let hint = [
"Interactive chat requires a TTY.",
"To force it from scripts: peekaboo agent --chat < prompts.txt",
"Provide a task arg or use --chat when piping input.",
"",
]
hint.forEach { print($0) }
self.printChatHelpMenu()
}
@MainActor
func runChatLoop(
_ agentService: PeekabooAgentService,
requestedModel: LanguageModel?,
initialPrompt: String?,
capabilities: TerminalCapabilities,
queueMode: QueueMode
) async throws {
guard self.ensureChatModePreconditions() else { return }
if capabilities.isInteractive && !capabilities.isPiped {
do {
try await self.runTauTUIChatLoop(
agentService,
requestedModel: requestedModel,
initialPrompt: initialPrompt,
capabilities: capabilities,
queueMode: queueMode
)
return
} catch {
self.printAgentExecutionError(
"Failed to launch TauTUI chat: \(error.localizedDescription). Falling back to basic chat."
)
}
}
try await self.runLineChatLoop(
agentService,
requestedModel: requestedModel,
initialPrompt: initialPrompt,
capabilities: capabilities,
queueMode: queueMode
)
}
@MainActor
private func runLineChatLoop(
_ agentService: PeekabooAgentService,
requestedModel: LanguageModel?,
initialPrompt: String?,
capabilities: TerminalCapabilities,
queueMode: QueueMode
) async throws {
var turnContext = ChatTurnContext(
sessionId: nil,
requestedModel: requestedModel,
queueMode: queueMode,
queuedWhileRunning: []
)
do {
turnContext.sessionId = try await self.initialChatSessionId(agentService)
} catch {
self.printAgentExecutionError(error.localizedDescription)
return
}
self.printChatWelcome(
sessionId: turnContext.sessionId,
modelDescription: self.describeModel(requestedModel),
queueMode: queueMode
)
self.printChatHelpIntro()
if let seed = initialPrompt {
try await self.performChatTurn(seed, agentService: agentService, context: &turnContext)
}
while true {
guard let line = self.readChatLine(prompt: "> ", capabilities: capabilities) else {
if capabilities.isInteractive {
print()
}
break
}
let trimmed = line.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmed.isEmpty { continue }
if trimmed == "/help" {
self.printChatHelpMenu()
continue
}
// If queueMode=all, batch any queued prompts gathered while a run was active
let batchedPrompt = trimmed
do {
try await self.performChatTurn(batchedPrompt, agentService: agentService, context: &turnContext)
} catch {
self.printAgentExecutionError(error.localizedDescription)
break
}
}
}
@MainActor
private func runTauTUIChatLoop(
_ agentService: PeekabooAgentService,
requestedModel: LanguageModel?,
initialPrompt: String?,
capabilities: TerminalCapabilities,
queueMode: QueueMode
) async throws {
var activeSessionId: String?
do {
activeSessionId = try await self.initialChatSessionId(agentService)
} catch {
self.printAgentExecutionError(error.localizedDescription)
return
}
let chatUI = AgentChatUI(
modelDescription: self.describeModel(requestedModel),
sessionId: activeSessionId,
queueMode: queueMode,
helpLines: self.chatHelpLines
)
try chatUI.start()
defer { chatUI.stop() }
var currentRun: Task<AgentExecutionResult, any Error>?
chatUI.onCancelRequested = { [weak chatUI] in
guard let run = currentRun else { return }
if !run.isCancelled {
run.cancel()
chatUI?.markCancelling()
}
}
chatUI.onInterruptRequested = { [weak chatUI] in
if let run = currentRun, !run.isCancelled {
run.cancel()
chatUI?.markCancelling()
} else {
chatUI?.finishPromptStream()
}
}
let promptStream = chatUI.promptStream(initialPrompt: initialPrompt)
for await prompt in promptStream {
let trimmed = prompt.trimmingCharacters(in: .whitespacesAndNewlines)
if trimmed.isEmpty { continue }
if trimmed == "/help" {
chatUI.showHelpMenu()
continue
}
// For queueMode=all, batch any queued prompts into this turn
let batchedPrompt: String
if queueMode == .all {
let extras = chatUI.drainQueuedPrompts()
batchedPrompt = ([trimmed] + extras).joined(separator: "\n\n")
} else {
batchedPrompt = trimmed
}
chatUI.beginRun(prompt: trimmed)
let tuiDelegate = AgentChatEventDelegate(ui: chatUI)
let sessionForRun = activeSessionId
let tuiContext = AgentRunContext(
sessionId: sessionForRun,
requestedModel: requestedModel,
queueMode: queueMode,
delegate: tuiDelegate
)
currentRun = Task { @MainActor in
try await self.runAgentTurnForTUI(
batchedPrompt,
agentService: agentService,
context: tuiContext
)
}
do {
guard let run = currentRun else { continue }
let result = try await run.value
if let sessionId = result.sessionId {
activeSessionId = sessionId
}
chatUI.endRun(result: result, sessionId: activeSessionId)
} catch is CancellationError {
chatUI.showCancelled()
} catch {
chatUI.showError(error.localizedDescription)
}
currentRun = nil
chatUI.setRunning(false)
}
}
struct AgentRunContext {
var sessionId: String?
var requestedModel: LanguageModel?
var queueMode: QueueMode
var delegate: any AgentEventDelegate
}
@MainActor
private func runAgentTurnForTUI(
_ input: String,
agentService: PeekabooAgentService,
context: AgentRunContext
) async throws -> AgentExecutionResult {
let sessionId = context.sessionId
let requestedModel = context.requestedModel
let queueMode = context.queueMode
let delegate = context.delegate
if let existingSessionId = sessionId {
return try await agentService.continueSession(
sessionId: existingSessionId,
userMessage: input,
model: requestedModel,
maxSteps: self.resolvedMaxSteps,
dryRun: self.dryRun,
queueMode: queueMode,
eventDelegate: delegate,
verbose: self.verbose
)
}
return try await agentService.executeTask(
input,
maxSteps: self.resolvedMaxSteps,
sessionId: nil,
model: requestedModel,
dryRun: self.dryRun,
queueMode: queueMode,
eventDelegate: delegate,
verbose: self.verbose
)
}
private func initialChatSessionId(
_ agentService: PeekabooAgentService
) async throws -> String? {
if let sessionId = self.resumeSession {
guard try await agentService.getSessionInfo(sessionId: sessionId) != nil else {
throw PeekabooError.sessionNotFound(sessionId)
}
return sessionId
}
if self.resume {
let sessions = try await agentService.listSessions()
guard let mostRecent = sessions.first else {
throw PeekabooError.commandFailed("No sessions available to resume.")
}
return mostRecent.id
}
return nil
}
private func readChatLine(prompt: String, capabilities: TerminalCapabilities) -> String? {
if capabilities.isInteractive {
fputs(prompt, stdout)
fflush(stdout)
}
return readLine()
}
struct ChatTurnContext {
var sessionId: String?
var requestedModel: LanguageModel?
var queueMode: QueueMode
var queuedWhileRunning: [String]
}
private func performChatTurn(
_ input: String,
agentService: PeekabooAgentService,
context: inout ChatTurnContext
) async throws {
let startingSessionId = context.sessionId
let queueMode = context.queueMode
let requestedModel = context.requestedModel
var batchedInput = input
if queueMode == .all {
let extras = context.queuedWhileRunning
context.queuedWhileRunning.removeAll()
batchedInput = ([input] + extras).joined(separator: "\n\n")
}
let runTask = Task { () throws -> AgentExecutionResult in
if let existingSessionId = startingSessionId {
let outputDelegate = self.makeDisplayDelegate(for: batchedInput)
let streamingDelegate = self.makeStreamingDelegate(using: outputDelegate)
let result = try await agentService.continueSession(
sessionId: existingSessionId,
userMessage: batchedInput,
model: requestedModel,
maxSteps: self.resolvedMaxSteps,
dryRun: self.dryRun,
queueMode: queueMode,
eventDelegate: streamingDelegate,
verbose: self.verbose
)
self.displayResult(result, delegate: outputDelegate)
return result
} else {
return try await self.executeAgentTask(
agentService,
task: batchedInput,
requestedModel: requestedModel,
maxSteps: self.resolvedMaxSteps,
queueMode: queueMode
)
}
}
let cancelMonitor = EscapeKeyMonitor { [runTask] in
if !runTask.isCancelled {
runTask.cancel()
await MainActor.run {
print("\n\(TerminalColor.yellow)Esc pressed cancelling current run...\(TerminalColor.reset)")
}
}
}
cancelMonitor.start()
let result: AgentExecutionResult
do {
defer { cancelMonitor.stop() }
result = try await runTask.value
} catch is CancellationError {
cancelMonitor.stop()
return
}
if let updatedSessionId = result.sessionId {
context.sessionId = updatedSessionId
}
self.printChatTurnSummary(result)
}
private func printChatTurnSummary(_ result: AgentExecutionResult) {
guard !self.quiet else { return }
let duration = String(format: "%.1fs", result.metadata.executionTime)
let sessionFragment = result.sessionId.map { String($0.prefix(8)) } ?? ""
let line = [
TerminalColor.dim,
"↺ Session ",
sessionFragment,
": ",
duration,
" • ⚒ ",
String(result.metadata.toolCallCount),
TerminalColor.reset
].joined()
print(line)
}
private func describeModel(_ requestedModel: LanguageModel?) -> String {
requestedModel?.description ?? "default (gpt-5.5)"
}
}

View File

@ -1,25 +0,0 @@
import Commander
@MainActor
extension AgentCommand: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {
self.task = try values.decodeOptionalPositional(0, label: "task")
self.debugTerminal = values.flag("debugTerminal")
self.quiet = values.flag("quiet")
self.dryRun = values.flag("dryRun")
if let steps: Int = try values.decodeOption("maxSteps", as: Int.self) {
self.maxSteps = steps
}
self.model = values.singleOption("model")
self.resume = values.flag("resume")
self.resumeSession = values.singleOption("resumeSession")
self.listSessions = values.flag("listSessions")
self.noCache = values.flag("noCache")
self.audio = values.flag("audio")
self.audioFile = values.singleOption("audioFile")
self.realtime = values.flag("realtime")
self.simple = values.flag("simple")
self.noColor = values.flag("noColor")
self.chat = values.flag("chat")
}
}

View File

@ -1,176 +0,0 @@
import Commander
import Foundation
import PeekabooAgentRuntime
import PeekabooCore
import PeekabooFoundation
import Tachikoma
import TauTUI
@available(macOS 14.0, *)
extension AgentCommand {
func ensureAgentHasCredentials(
selectedModel: LanguageModel
) -> Bool {
if self.isLocalModel(selectedModel) {
return true
}
if self.hasCredentials(for: selectedModel) {
return true
}
let providerName = self.providerDisplayName(for: selectedModel)
let envVar = self.providerEnvironmentVariable(for: selectedModel)
self.printAgentExecutionError(
"Missing API key for \(providerName). Set \(envVar) and retry."
)
return false
}
/// Render the agent execution result using either JSON output or a rich CLI transcript.
@MainActor
func displayResult(_ result: AgentExecutionResult, delegate: AgentOutputDelegate? = nil) {
if self.jsonOutput {
let response = [
"success": true,
"result": [
"content": result.content,
"sessionId": result.sessionId as Any,
"toolCalls": result.messages.flatMap { message in
message.content.compactMap { content in
if case let .toolCall(toolCall) = content {
return [
"id": toolCall.id,
"name": toolCall.name,
"arguments": String(describing: toolCall.arguments)
]
}
return nil
}
},
"metadata": [
"executionTime": result.metadata.executionTime,
"toolCallCount": result.metadata.toolCallCount,
"modelName": result.metadata.modelName
],
"usage": result.usage.map { usage in
[
"inputTokens": usage.inputTokens,
"outputTokens": usage.outputTokens,
"totalTokens": usage.totalTokens
]
} as Any
]
] as [String: Any]
if let jsonData = try? JSONSerialization.data(withJSONObject: response, options: .prettyPrinted) {
print(String(data: jsonData, encoding: .utf8) ?? "{}")
}
} else if self.outputMode == .quiet {
print(result.content)
}
delegate?.showFinalSummaryIfNeeded(result)
}
func makeDisplayDelegate(for task: String) -> AgentOutputDelegate? {
guard !self.jsonOutput, !self.quiet else { return nil }
return AgentOutputDelegate(outputMode: self.outputMode, jsonOutput: self.jsonOutput, task: task)
}
func makeStreamingDelegate(using displayDelegate: AgentOutputDelegate?) -> (any AgentEventDelegate)? {
if let displayDelegate {
return displayDelegate
}
if self.jsonOutput || self.quiet {
return SilentAgentEventDelegate()
}
return nil
}
final class SilentAgentEventDelegate: AgentEventDelegate {
func agentDidEmitEvent(_ event: AgentEvent) {}
}
func printAgentExecutionError(_ message: String) {
if self.jsonOutput {
let error: [String: Any] = ["success": false, "error": message]
if let jsonData = try? JSONSerialization.data(withJSONObject: error, options: .prettyPrinted),
let jsonString = String(data: jsonData, encoding: .utf8) {
print(jsonString)
} else {
print("{\"success\":false,\"error\":\"\(message)\"}")
}
} else {
print("\(TerminalColor.red)Error: \(message)\(TerminalColor.reset)")
}
}
func executeAgentTask(
_ agentService: PeekabooAgentService,
task: String,
requestedModel: LanguageModel?,
maxSteps: Int,
queueMode: QueueMode
) async throws -> AgentExecutionResult {
let outputDelegate = self.makeDisplayDelegate(for: task)
let streamingDelegate = self.makeStreamingDelegate(using: outputDelegate)
do {
let result = try await agentService.executeTask(
task,
maxSteps: maxSteps,
sessionId: nil,
model: requestedModel,
dryRun: self.dryRun,
queueMode: queueMode,
eventDelegate: streamingDelegate,
verbose: self.verbose
)
self.displayResult(result, delegate: outputDelegate)
let duration = String(format: "%.2f", result.metadata.executionTime)
let sessionId = result.sessionId ?? "none"
let finalTokens = result.usage?.totalTokens ?? 0
let status = result.metadata.context["status"] ?? "completed"
AutomationEventLogger.log(
.agent,
"result status=\(status) task='\(task)' model=\(result.metadata.modelName) duration=\(duration)s "
+ "tools=\(result.metadata.toolCallCount) dry_run=\(self.dryRun) "
+ "session=\(sessionId) tokens=\(finalTokens)"
)
return result
} catch {
self.printAgentExecutionError("Agent execution failed: \(error.localizedDescription)")
throw ExitCode.failure
}
}
var normalizedTaskInput: String? {
guard let task else { return nil }
let trimmed = task.trimmingCharacters(in: .whitespacesAndNewlines)
return trimmed.isEmpty ? nil : trimmed
}
var hasTaskInput: Bool {
self.normalizedTaskInput != nil || self.audio || self.audioFile != nil
}
var resolvedMaxSteps: Int {
self.maxSteps ?? 100
}
func resolvedQueueMode() throws -> QueueMode {
guard let raw = self.queueMode?.trimmingCharacters(in: .whitespacesAndNewlines), !raw.isEmpty else {
return .oneAtATime
}
switch raw.lowercased() {
case "one", "one-at-a-time", "single", "sequential", "1":
return .oneAtATime
case "all", "batch", "together":
return .all
default:
throw PeekabooError.invalidInput("Invalid queue mode '\(raw)'. Use one-at-a-time or all.")
}
}
}

View File

@ -1,322 +0,0 @@
import Foundation
import PeekabooCore
import PeekabooFoundation
import Tachikoma
@available(macOS 14.0, *)
extension AgentCommand {
@MainActor
func parseModelString(
_ modelString: String,
configuration: PeekabooCore.ConfigurationManager? = nil
) -> LanguageModel? {
let trimmed = modelString.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else { return nil }
let explicitProvider = trimmed
.split(separator: "/", maxSplits: 1)
.first
.map { String($0).lowercased() }
if trimmed.caseInsensitiveCompare("claude") == .orderedSame ||
trimmed.caseInsensitiveCompare("anthropic") == .orderedSame {
return .anthropic(.opus48)
}
if let configuration {
switch self.parseConfiguredCustomModel(
trimmed,
explicitProvider: explicitProvider,
configuration: configuration
) {
case let .resolved(model):
return model
case .unresolved:
break
}
}
guard let parsed = LanguageModel.parse(from: trimmed) else {
return nil
}
return self.supportedParsedModel(parsed, explicitProvider: explicitProvider)
}
@MainActor
private func supportedParsedModel(_ parsed: LanguageModel, explicitProvider: String?) -> LanguageModel? {
switch parsed {
case let .openai(model):
if Self.supportedOpenAIInputs.contains(model) {
return .openai(.gpt55)
}
case let .anthropic(model):
if Self.supportedAnthropicInputs.contains(model) {
return .anthropic(model)
}
case let .google(model):
if Self.supportedGoogleInputs.contains(model) {
return .google(model)
}
case .grok:
return parsed.supportsTools ? parsed : nil
case let .minimax(model):
if Self.supportedMiniMaxInputs.contains(model) {
return .minimax(model)
}
case let .minimaxCN(model):
if Self.supportedMiniMaxInputs.contains(model) {
return .minimaxCN(model)
}
case .ollama, .lmstudio:
return parsed.supportsTools ? parsed : nil
case .openRouter:
if let explicitProvider, Self.reservedProviderInputs.contains(explicitProvider) {
return nil
}
return parsed.supportsTools ? parsed : nil
default:
break
}
return nil
}
@MainActor
private func parseConfiguredCustomModel(
_ modelString: String,
explicitProvider: String?,
configuration: PeekabooCore.ConfigurationManager
) -> ConfiguredModelResolution {
if let configuredModel = PeekabooAIService(configuration: configuration).resolveConfiguredModel(modelString),
case .custom = configuredModel {
return .resolved(configuredModel.supportsTools ? configuredModel : nil)
}
if let explicitProvider,
configuration.listCustomProviders().contains(where: { providerID, provider in
provider.enabled && providerID.caseInsensitiveCompare(explicitProvider) == .orderedSame
}) {
return .resolved(nil)
}
return .unresolved
}
private enum ConfiguredModelResolution {
case resolved(LanguageModel?)
case unresolved
}
@MainActor
func validatedModelSelection(configuration: PeekabooCore.ConfigurationManager? = nil) throws -> LanguageModel? {
guard let modelString = self.model else { return nil }
guard let parsed = self.parseModelString(modelString, configuration: configuration) else {
throw PeekabooError.invalidInput(
"Unsupported model '\(modelString)'. Allowed values: \(Self.allowedModelList)"
)
}
return parsed
}
private static let supportedOpenAIInputs: Set<LanguageModel.OpenAI> = [
.gpt55,
.gpt54,
.gpt54Mini,
.gpt54Nano,
.gpt5,
.gpt5Pro,
.gpt5Mini,
.gpt5Nano,
]
private static let supportedAnthropicInputs: Set<LanguageModel.Anthropic> = [
.fable5,
.opus48,
.opus47,
.opus45,
.opus4,
.sonnet46,
.sonnet45,
.haiku45,
]
private static let supportedGoogleInputs: Set<LanguageModel.Google> = [
.gemini35Flash,
.gemini31ProPreview,
.gemini31FlashLite,
.gemini3Flash,
.gemini25Pro,
.gemini25Flash,
.gemini25FlashLite,
]
private static let supportedMiniMaxInputs: Set<LanguageModel.MiniMax> = [
.m27,
.m27Highspeed,
]
private static let reservedProviderInputs: Set<String> = [
"openai",
"anthropic",
"google",
"gemini",
"grok",
"xai",
"minimax",
"minimax-cn",
"minimax_cn",
"minimaxi",
"ollama",
"lmstudio",
"lm-studio",
]
private static var allowedModelList: String {
let openAIModels = Self.supportedOpenAIInputs.map(\.modelId)
let anthropicModels = Self.supportedAnthropicInputs.map(\.modelId)
let googleModels = Self.supportedGoogleInputs.map(\.userFacingModelId)
let miniMaxModels = Self.supportedMiniMaxInputs.map(\.modelId)
return (openAIModels + anthropicModels + googleModels + miniMaxModels + [
"grok/<model>",
"xai/<model>",
"minimax-cn/<model>",
"ollama/<model>",
"lmstudio/<model>",
"openrouter/<provider>/<model>",
"<custom-provider>/<model>",
])
.sorted()
.joined(separator: ", ")
}
@MainActor
func firstAvailableToolModel(from service: PeekabooAIService) -> LanguageModel? {
service.availableModels().first { model in
model.supportsTools && service.isModelAvailable(model)
}
}
@MainActor
func configuredDefaultToolModel(
from service: PeekabooAIService,
configuration: PeekabooCore.ConfigurationManager
) -> LanguageModel? {
guard let defaultModel = configuration.getAgentModel(),
let model = service.resolveConfiguredModel(defaultModel),
model.supportsTools,
service.isModelAvailable(model)
else {
return nil
}
return model
}
@MainActor
func implicitToolModel(
from service: PeekabooAIService,
configuration: PeekabooCore.ConfigurationManager,
existingAgentModel: LanguageModel?
) -> LanguageModel? {
if let existingAgentModel {
return existingAgentModel
}
if configuration.hasExplicitAIProviderList() {
return nil
}
return self.configuredDefaultToolModel(from: service, configuration: configuration) ??
self.firstAvailableToolModel(from: service)
}
@MainActor
func hasCredentials(for model: LanguageModel) -> Bool {
let configuration = self.services.configuration
switch model {
case .ollama, .lmstudio:
return true
case .openai:
return configuration.hasOpenAIAuth()
case .anthropic:
return configuration.hasAnthropicAuth()
case .google:
return configuration.getGeminiAPIKey()?.isEmpty == false
case .minimax:
return configuration.getMiniMaxAPIKey()?.isEmpty == false
case .minimaxCN:
return configuration.getMiniMaxChinaAPIKey()?.isEmpty == false
case .grok:
return configuration.getGrokAPIKey()?.isEmpty == false
case .openRouter:
return configuration.getOpenRouterAPIKey()?.isEmpty == false
case let .custom(provider):
return provider.apiKey?.isEmpty == false
default:
return false
}
}
func providerDisplayName(for model: LanguageModel) -> String {
switch model {
case .openai:
"OpenAI"
case .anthropic:
"Anthropic"
case .google:
"Google"
case .minimax:
"MiniMax"
case .minimaxCN:
"MiniMax China"
case .ollama:
"Ollama"
case .lmstudio:
"LM Studio"
case .openRouter:
"OpenRouter"
case .grok:
"xAI"
case let .custom(provider):
"custom provider \(provider.modelId)"
default:
"the selected provider"
}
}
func providerEnvironmentVariable(for model: LanguageModel) -> String {
switch model {
case .openai:
"OPENAI_API_KEY"
case .anthropic:
"ANTHROPIC_API_KEY"
case .google:
"GEMINI_API_KEY"
case .minimax:
"MINIMAX_API_KEY"
case .minimaxCN:
"MINIMAX_CN_API_KEY or MINIMAX_API_KEY"
case .ollama:
"OLLAMA_BASE_URL or PEEKABOO_OLLAMA_BASE_URL"
case .lmstudio:
"LM Studio local server URL"
case .openRouter:
"OPENROUTER_API_KEY"
case .grok:
"X_AI_API_KEY, XAI_API_KEY, or GROK_API_KEY"
case .custom:
"the custom provider API key reference"
default:
"provider API key"
}
}
func isLocalModel(_ model: LanguageModel?) -> Bool {
switch model {
case .ollama, .lmstudio:
true
default:
false
}
}
}

View File

@ -1,240 +0,0 @@
import Foundation
import PeekabooAgentRuntime
import PeekabooCore
import PeekabooFoundation
import Tachikoma
import TauTUI
/// Temporary session info struct until PeekabooAgentService implements session management
struct AgentSessionInfo: Codable {
let id: String
let task: String
let created: Date
let lastModified: Date
let messageCount: Int
}
@available(macOS 14.0, *)
extension AgentCommand {
struct ResumeAgentSessionRequest {
let sessionId: String
let task: String
let requestedModel: LanguageModel?
let maxSteps: Int
let queueMode: QueueMode
}
func handleSessionResumption(
_ agentService: PeekabooAgentService,
requestedModel: LanguageModel?,
maxSteps: Int,
queueMode: QueueMode
) async throws -> Bool {
if let sessionId = self.resumeSession {
guard let continuationTask = self.task else {
self.printMissingTaskError(
message: "Task argument required when resuming session",
usage: "Usage: peekaboo agent --resume-session <session-id> \"<continuation-task>\""
)
return true
}
try await self.resumeAgentSession(
agentService,
request: ResumeAgentSessionRequest(
sessionId: sessionId,
task: continuationTask,
requestedModel: requestedModel,
maxSteps: maxSteps,
queueMode: queueMode
)
)
return true
}
if self.resume {
guard let continuationTask = self.task else {
self.printMissingTaskError(
message: "Task argument required when resuming",
usage: "Usage: peekaboo agent --resume \"<continuation-task>\""
)
return true
}
let sessions = try await agentService.listSessions()
if let mostRecent = sessions.first {
try await self.resumeAgentSession(
agentService,
request: ResumeAgentSessionRequest(
sessionId: mostRecent.id,
task: continuationTask,
requestedModel: requestedModel,
maxSteps: maxSteps,
queueMode: queueMode
)
)
} else {
if self.jsonOutput {
let error = ["success": false, "error": "No sessions found to resume"] as [String: Any]
let jsonData = try JSONSerialization.data(withJSONObject: error, options: .prettyPrinted)
print(String(data: jsonData, encoding: .utf8) ?? "{}")
} else {
print("\(TerminalColor.red)Error: No sessions found to resume\(TerminalColor.reset)")
}
}
return true
}
return false
}
func printMissingTaskError(message: String, usage: String) {
if self.jsonOutput {
let error = ["success": false, "error": message] as [String: Any]
if let jsonData = try? JSONSerialization.data(withJSONObject: error, options: .prettyPrinted),
let jsonString = String(data: jsonData, encoding: .utf8) {
print(jsonString)
} else {
print("{\"success\":false,\"error\":\"\(message)\"}")
}
} else {
print("\(TerminalColor.red)Error: \(message)\(TerminalColor.reset)")
if !usage.isEmpty {
print(usage)
}
}
}
@MainActor
func showSessions(_ agentService: any AgentServiceProtocol) async throws {
guard let peekabooService = agentService as? PeekabooAgentService else {
throw PeekabooError.commandFailed("Agent service not properly initialized")
}
let sessionSummaries = try await peekabooService.listSessions()
let sessions = sessionSummaries.map { summary in
AgentSessionInfo(
id: summary.id,
task: summary.summary ?? "Unknown task",
created: summary.createdAt,
lastModified: summary.lastAccessedAt,
messageCount: summary.messageCount
)
}
guard !sessions.isEmpty else {
self.printNoAgentSessions()
return
}
if self.jsonOutput {
self.printSessionsJSON(sessions)
} else {
self.printSessionsList(sessions)
}
}
private func printNoAgentSessions() {
if self.jsonOutput {
let response = ["success": true, "sessions": []] as [String: Any]
let jsonData = try? JSONSerialization.data(withJSONObject: response, options: .prettyPrinted)
print(String(data: jsonData ?? Data(), encoding: .utf8) ?? "{}")
} else {
print("No agent sessions found.")
}
}
private func printSessionsJSON(_ sessions: [AgentSessionInfo]) {
let sessionData = sessions.map { session in
[
"id": session.id,
"createdAt": ISO8601DateFormatter().string(from: session.created),
"updatedAt": ISO8601DateFormatter().string(from: session.lastModified),
"messageCount": session.messageCount
]
}
let response = ["success": true, "sessions": sessionData] as [String: Any]
if let jsonData = try? JSONSerialization.data(withJSONObject: response, options: .prettyPrinted) {
print(String(data: jsonData, encoding: .utf8) ?? "{}")
}
}
private func printSessionsList(_ sessions: [AgentSessionInfo]) {
let headerLine = [
"\(TerminalColor.cyan)\(TerminalColor.bold)Agent Sessions:\(TerminalColor.reset)",
"\n"
].joined()
print(headerLine)
let dateFormatter = DateFormatter()
dateFormatter.dateStyle = .short
dateFormatter.timeStyle = .short
for (index, session) in sessions.prefix(10).indexed() {
self.printSessionLine(index: index, session: session, dateFormatter: dateFormatter)
if index < sessions.count - 1 {
print()
}
}
if sessions.count > 10 {
print([
"\n",
"\(TerminalColor.dim)... and \(sessions.count - 10) more sessions\(TerminalColor.reset)"
].joined())
}
let resumeHintLine = [
"\n",
"\(TerminalColor.dim)To resume: peekaboo agent --resume <session-id>",
" \"<continuation>\"\(TerminalColor.reset)"
].joined()
print(resumeHintLine)
}
private func printSessionLine(index: Int, session: AgentSessionInfo, dateFormatter: DateFormatter) {
let timeAgo = formatTimeAgo(session.lastModified)
let sessionLine = [
"\(TerminalColor.blue)\(index + 1).\(TerminalColor.reset)",
" ",
"\(TerminalColor.bold)\(session.id.prefix(8))\(TerminalColor.reset)"
].joined()
print(sessionLine)
print(" Messages: \(session.messageCount)")
print(" Last activity: \(timeAgo)")
}
private func resumeAgentSession(
_ agentService: PeekabooAgentService,
request: ResumeAgentSessionRequest
) async throws {
if !self.jsonOutput {
let resumingLine = [
"\(TerminalColor.cyan)\(TerminalColor.bold)",
"\(AgentDisplayTokens.Status.info)",
" Resuming session \(request.sessionId.prefix(8))...",
"\(TerminalColor.reset)",
"\n"
].joined()
print(resumingLine)
}
let outputDelegate = self.makeDisplayDelegate(for: request.task)
let streamingDelegate = self.makeStreamingDelegate(using: outputDelegate)
do {
let result = try await agentService.continueSession(
sessionId: request.sessionId,
userMessage: request.task,
model: request.requestedModel,
maxSteps: request.maxSteps,
dryRun: self.dryRun,
queueMode: request.queueMode,
eventDelegate: streamingDelegate
)
self.displayResult(result, delegate: outputDelegate)
} catch {
self.printAgentExecutionError("Failed to resume session: \(error.localizedDescription)")
throw error
}
}
}

View File

@ -1,199 +0,0 @@
import Darwin
import Dispatch
import Foundation
import PeekabooAgentRuntime
import PeekabooCore
import TauTUI
@MainActor
private final class TerminalModeGuard {
private let fd: Int32
private var original = termios()
private var active = false
init?(fd: Int32 = STDIN_FILENO) {
guard isatty(fd) == 1 else { return nil }
guard tcgetattr(fd, &self.original) == 0 else { return nil }
var raw = self.original
cfmakeraw(&raw)
raw.c_lflag |= tcflag_t(ISIG) // keep signals like Ctrl+C enabled
guard tcsetattr(fd, TCSANOW, &raw) == 0 else { return nil }
self.fd = fd
self.active = true
}
var fileDescriptor: Int32 {
self.fd
}
func restore() {
guard self.active else { return }
_ = tcsetattr(self.fd, TCSANOW, &self.original)
self.active = false
}
@MainActor
deinit {
self.restore()
}
}
final class EscapeKeyMonitor {
private var source: (any DispatchSourceRead)?
private var terminalGuard: TerminalModeGuard?
private let handler: @Sendable () async -> Void
private let queue = DispatchQueue(label: "peekaboo.escape.monitor")
init(handler: @escaping @Sendable () async -> Void) {
self.handler = handler
}
func start() {
guard self.source == nil else { return }
guard let termGuard = TerminalModeGuard() else { return }
let fd = termGuard.fileDescriptor
let handler = self.handler
let source = DispatchSource.makeReadSource(fileDescriptor: fd, queue: self.queue)
source.setEventHandler {
var buffer = [UInt8](repeating: 0, count: 16)
let count = read(fd, &buffer, buffer.count)
guard count > 0 else { return }
if buffer[..<count].contains(0x1B) {
Task.detached(priority: .userInitiated) {
await handler()
}
}
}
source.setCancelHandler {
termGuard.restore()
}
source.resume()
self.source = source
self.terminalGuard = termGuard
}
func stop() {
self.source?.cancel()
self.source = nil
self.terminalGuard = nil
}
}
@available(macOS 14.0, *)
extension AgentCommand {
func printChatWelcome(sessionId: String?, modelDescription: String, queueMode: QueueMode) {
guard !self.quiet else { return }
let header = [
TerminalColor.cyan,
TerminalColor.bold,
"Interactive agent chat",
TerminalColor.reset,
" model: ",
modelDescription,
" • queue: ",
queueMode == .all ? "all" : "one-at-a-time"
].joined()
print(header)
if let sessionId {
print("\(TerminalColor.dim)Resuming session \(sessionId.prefix(8))\(TerminalColor.reset)")
} else {
print("\(TerminalColor.dim)A new session will be created on the first prompt\(TerminalColor.reset)")
}
print()
}
func printChatHelpIntro() {
guard !self.quiet else { return }
print("Type /help for chat commands (Ctrl+C to exit).")
self.printChatHelpMenu()
}
func printChatHelpMenu() {
guard !self.quiet else { return }
self.chatHelpLines.forEach { print($0) }
}
private var chatHelpText: String {
"""
Chat commands:
Type any prompt and press Return to run it.
/help Show this menu again.
Esc Cancel the active run (if one is in progress).
Ctrl+C Cancel when running; exit immediately when idle.
Ctrl+D Exit when idle (EOF).
"""
}
var chatHelpLines: [String] {
self.chatHelpText
.split(separator: "\n", omittingEmptySubsequences: false)
.map(String.init)
}
private func printCapabilityFlag(_ label: String, supported: Bool, detail: String? = nil) {
let status = supported ? AgentDisplayTokens.Status.success : AgentDisplayTokens.Status.failure
let detailSuffix = detail.map { " (\($0))" } ?? ""
print("\(label): \(status)\(detailSuffix)")
}
/// Print detailed terminal detection debugging information
func printTerminalDetectionDebug(_ capabilities: TerminalCapabilities, actualMode: OutputMode) {
print("\n" + String(repeating: "=", count: 60))
print("\(TerminalColor.bold)\(TerminalColor.cyan)TERMINAL DETECTION DEBUG (-vv)\(TerminalColor.reset)")
print(String(repeating: "=", count: 60))
print("[term] \(TerminalColor.bold)Terminal Type:\(TerminalColor.reset) \(capabilities.termType ?? "unknown")")
print(
"[size] \(TerminalColor.bold)Dimensions:\(TerminalColor.reset) \(capabilities.width)x\(capabilities.height)"
)
print("\(AgentDisplayTokens.Status.running) \(TerminalColor.bold)Capabilities:\(TerminalColor.reset)")
self.printCapabilityFlag("Interactive", supported: capabilities.isInteractive, detail: "isatty check")
self.printCapabilityFlag("Colors", supported: capabilities.supportsColors, detail: "ANSI support")
self.printCapabilityFlag("True Color", supported: capabilities.supportsTrueColor, detail: "24-bit")
print(" • Dimensions: \(capabilities.width)x\(capabilities.height)")
print("[env] \(TerminalColor.bold)Environment:\(TerminalColor.reset)")
self.printCapabilityFlag("CI Environment", supported: capabilities.isCI)
self.printCapabilityFlag("Piped Output", supported: capabilities.isPiped)
let env = ProcessInfo.processInfo.environment
print("\(AgentDisplayTokens.Status.running) \(TerminalColor.bold)Environment Variables:\(TerminalColor.reset)")
print(" • TERM: \(env["TERM"] ?? "not set")")
print(" • COLORTERM: \(env["COLORTERM"] ?? "not set")")
print(" • NO_COLOR: \(env["NO_COLOR"] != nil ? "set" : "not set")")
print(" • FORCE_COLOR: \(env["FORCE_COLOR"] ?? "not set")")
print(" • PEEKABOO_OUTPUT_MODE: \(env["PEEKABOO_OUTPUT_MODE"] ?? "not set")")
let recommendedMode = capabilities.recommendedOutputMode
print("[focus] \(TerminalColor.bold)Recommended Mode:\(TerminalColor.reset) \(recommendedMode.description)")
print("[focus] \(TerminalColor.bold)Actual Mode:\(TerminalColor.reset) \(actualMode.description)")
if recommendedMode != actualMode {
let modeOverrideLine = [
"\(AgentDisplayTokens.Status.warning) ",
"\(TerminalColor.yellow)Mode Override Detected\(TerminalColor.reset)",
" - explicit flag or environment variable used"
].joined()
print(modeOverrideLine)
}
if !capabilities.isInteractive || capabilities.isCI || capabilities.isPiped {
print(" → Minimal mode (non-interactive/CI/piped)")
} else if capabilities.supportsColors {
print(" → Enhanced mode (colors available)")
} else {
print(" → Compact mode (basic terminal)")
}
print(String(repeating: "=", count: 60) + "\n")
}
}

View File

@ -1,402 +0,0 @@
import Commander
import Foundation
import Logging
import PeekabooAgentRuntime
import PeekabooCore
import PeekabooFoundation
import Tachikoma
import TauTUI
/// Simple debug logging check
private var isDebugLoggingEnabled: Bool {
// Check if verbose mode is enabled via log level
if let logLevel = ProcessInfo.processInfo.environment["PEEKABOO_LOG_LEVEL"]?.lowercased() {
return logLevel == "debug" || logLevel == "trace"
}
// Check if agent is in verbose mode
if ProcessInfo.processInfo.arguments.contains("-v") ||
ProcessInfo.processInfo.arguments.contains("--verbose") {
return true
}
return false
}
private func aiDebugPrint(_ message: String) {
if isDebugLoggingEnabled {
print(message)
}
}
/// Output modes for agent execution with progressive enhancement
enum OutputMode {
case minimal // CI/pipes - no colors, simple text
case compact // Basic colors and icons (legacy default)
case enhanced // Rich formatting with progress indicators
case quiet // Only final result
case verbose // Full JSON debug information
}
/// Get icon for tool name in compact mode
func iconForTool(_ toolName: String) -> String {
AgentDisplayTokens.icon(for: toolName)
}
/// AI Agent command that uses new Chat Completions API architecture
@available(macOS 14.0, *)
struct AgentCommand: RuntimeOptionsConfigurable {
static let commandDescription = CommandDescription(
commandName: "agent",
abstract: "Execute complex automation tasks using the Peekaboo agent",
discussion: """
Launches the autonomous Peekaboo operator so it can interpret a natural-language goal,
choose tools (see, click, type, etc.), and report progress back to you. Supports resuming
previous sessions, dry-run planning, audio input, and JSON/quiet output modes for CI.
""",
usageExamples: [
CommandUsageExample(
command: "peekaboo agent \"Prepare the TestFlight build for review\"",
description: "Start a brand-new session with a natural-language brief."
),
CommandUsageExample(
command: "peekaboo agent --resume",
description: "Resume the most recent session without retyping the task."
),
CommandUsageExample(
command: "peekaboo agent --resume-session SESSION_ID --max-steps 12",
description: "Resume a known session while capping the step budget."
)
]
)
@Argument(help: "Natural language description of the task to perform (optional when using --resume)")
var task: String?
@Flag(name: .customLong("debug-terminal"), help: "Show detailed terminal detection info")
var debugTerminal = false
@Flag(names: [.short("q"), .long], help: "Quiet mode - only show final result")
var quiet = false
@Flag(name: .long, help: "Dry run - show planned steps without executing")
var dryRun = false
@Option(name: .long, help: "Maximum number of steps the agent can take")
var maxSteps: Int?
@Option(name: .long, help: "Queue mode for queued prompts: one-at-a-time (default) or all")
var queueMode: String?
@Option(
name: .long,
help: """
AI model to use (for example: gpt-5.5, claude-fable-5, \
gemini-3.5-flash, grok-4.3, minimax-m2.7, minimax-cn/m2.7, \
ollama/<model>, lmstudio/<model>, or <custom-provider>/<model>)
"""
)
var model: String?
@Flag(name: .long, help: "Resume the most recent session (use with task argument)")
var resume = false
@Option(name: .long, help: "Resume a specific session by ID")
var resumeSession: String?
@Flag(name: .long, help: "List available sessions")
var listSessions = false
@Flag(name: .long, help: "Disable session caching (always create new session)")
var noCache = false
@Flag(name: .long, help: "Enable audio input mode (record from microphone)")
var audio = false
@Option(name: .long, help: "Audio input file path (instead of microphone)")
var audioFile: String?
@Flag(name: .long, help: "Use real-time audio streaming (OpenAI only)")
var realtime = false
@Flag(name: .long, help: "Force simple output mode (no colors or rich formatting)")
var simple = false
@Flag(name: .long, help: "Disable colors in output")
var noColor = false
@Flag(name: .long, help: "Start an interactive chat session")
var chat = false
/// Computed property for output mode with smart detection and progressive enhancement
var outputMode: OutputMode {
// Explicit user overrides first
if self.quiet { return .quiet }
if self.verbose || self.debugTerminal { return .verbose }
if self.simple { return .minimal }
if self.noColor { return .minimal }
// Check for environment-based forced modes
if let forcedMode = TerminalDetector.shouldForceOutputMode() {
return forcedMode
}
// Smart detection based on terminal capabilities
let capabilities = TerminalDetector.detectCapabilities()
return capabilities.recommendedOutputMode
}
@RuntimeStorage private var runtime: CommandRuntime?
var runtimeOptions: CommandRuntimeOptions = {
var options = CommandRuntimeOptions()
// Remote GUI bridge mode is optional and can fail to expose auth state.
// Keep agent execution local by default unless an explicit runtime option overrides it.
options.preferRemote = false
return options
}()
private var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
@MainActor
var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
var jsonOutput: Bool {
self.runtime?.configuration.jsonOutput ?? self.runtimeOptions.jsonOutput
}
var verbose: Bool {
self.runtime?.configuration.verbose ?? self.runtimeOptions.verbose
}
}
@available(macOS 14.0, *)
extension AgentCommand {
@MainActor
mutating func run() async throws {
let runtime = await CommandRuntime.makeDefaultAsync(options: self.runtimeOptions)
try await self.run(using: runtime)
}
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
do {
try await self.runInternal(runtime: runtime)
} catch let error as DecodingError {
aiDebugPrint("DEBUG: Caught DecodingError in run(): \(error)")
throw error
} catch let error as NSError {
aiDebugPrint("DEBUG: Caught NSError in run(): \(error)")
aiDebugPrint("DEBUG: Domain: \(error.domain)")
aiDebugPrint("DEBUG: Code: \(error.code)")
aiDebugPrint("DEBUG: UserInfo: \(error.userInfo)")
throw error
} catch {
aiDebugPrint("DEBUG: Caught unknown error in run(): \(error)")
throw error
}
}
@MainActor
mutating func runInternal(runtime: CommandRuntime) async throws {
if self.isAgentDisabled() {
self.emitAgentUnavailableMessage()
return
}
let services = runtime.services
let requestedModel: LanguageModel?
do {
requestedModel = try self.validatedModelSelection(configuration: services.configuration)
} catch {
self.printAgentExecutionError(error.localizedDescription)
throw ExitCode.failure
}
let configuredAIService = PeekabooAIService(configuration: services.configuration)
let existingAgent = services.agent as? PeekabooAgentService
let mutationCoordinator = runtime.toolSnapshotMutationCoordinator
existingAgent?.configureSnapshotMutationCoordinator(mutationCoordinator)
let existingAgentModel = existingAgent.flatMap {
configuredAIService.resolveConfiguredModel($0.defaultModelSelection) ??
LanguageModel.parse(from: $0.defaultModelSelection)
}
let selectedModel = requestedModel ??
self.implicitToolModel(
from: configuredAIService,
configuration: services.configuration,
existingAgentModel: existingAgentModel
)
if self.listSessions {
let listingModel = selectedModel ?? existingAgentModel ?? .anthropic(.opus48)
let agentService: any AgentServiceProtocol = if let existing = existingAgent {
existing
} else {
try PeekabooAgentService(
services: services,
defaultModel: listingModel,
snapshotMutationCoordinator: mutationCoordinator
)
}
try await self.showSessions(agentService)
return
}
guard let selectedModel else {
self.emitAgentUnavailableMessage()
return
}
guard self.hasCredentials(for: selectedModel) || self.isLocalModel(selectedModel) else {
if requestedModel != nil {
let providerName = self.providerDisplayName(for: selectedModel)
let envVar = self.providerEnvironmentVariable(for: selectedModel)
self.printAgentExecutionError(
"Missing API key for \(providerName). Set \(envVar) and retry."
)
} else {
self.emitAgentUnavailableMessage()
}
return
}
let agentService: any AgentServiceProtocol = if let existing = existingAgent {
existing
} else {
try PeekabooAgentService(
services: services,
defaultModel: selectedModel,
snapshotMutationCoordinator: mutationCoordinator
)
}
let terminalCapabilities = TerminalDetector.detectCapabilities()
if self.debugTerminal {
self.printTerminalDetectionDebug(terminalCapabilities, actualMode: self.outputMode)
}
let shouldSuppressMCPLogs = !self.verbose && !self.debugTerminal
self.configureLogging(suppressingMCPLogs: shouldSuppressMCPLogs)
guard let peekabooAgent = agentService as? PeekabooAgentService else {
throw PeekabooError.commandFailed("Agent service not properly initialized")
}
guard self.ensureAgentHasCredentials(selectedModel: selectedModel) else {
return
}
let chatPolicy = AgentChatLaunchPolicy()
let chatContext = AgentChatLaunchContext(
chatFlag: self.chat,
hasTaskInput: self.hasTaskInput,
listSessions: self.listSessions,
normalizedTaskInput: self.normalizedTaskInput,
capabilities: terminalCapabilities
)
let queueMode: QueueMode
do {
queueMode = try self.resolvedQueueMode()
} catch {
self.printAgentExecutionError(error.localizedDescription)
throw ExitCode.failure
}
switch chatPolicy.strategy(for: chatContext) {
case .helpOnly:
self.printNonInteractiveChatHelp()
return
case let .interactive(initialPrompt):
try await self.runChatLoop(
peekabooAgent,
requestedModel: requestedModel,
initialPrompt: initialPrompt,
capabilities: terminalCapabilities,
queueMode: queueMode
)
return
case .none:
break
}
if try await self.handleSessionResumption(
peekabooAgent,
requestedModel: requestedModel,
maxSteps: self.maxSteps ?? 100,
queueMode: queueMode
) {
return
}
guard let executionTask = try await self.buildExecutionTask() else {
return
}
_ = try await self.executeAgentTask(
peekabooAgent,
task: executionTask,
requestedModel: requestedModel,
maxSteps: self.maxSteps ?? 100,
queueMode: queueMode
)
}
private func isAgentDisabled() -> Bool {
let value = ProcessInfo.processInfo.environment["PEEKABOO_DISABLE_AGENT"]?.lowercased()
return value == "1" || value == "true"
}
private func configureLogging(suppressingMCPLogs: Bool) {
if suppressingMCPLogs {
LoggingSystem.bootstrap { label in
var handler = StreamLogHandler.standardOutput(label: label)
if label.hasPrefix("tachikoma.mcp") {
handler.logLevel = .critical // hide MCP init chatter unless --verbose
} else {
handler.logLevel = .info
}
return handler
}
} else {
LoggingSystem.bootstrap(StreamLogHandler.standardOutput)
}
}
func emitAgentUnavailableMessage() {
if self.jsonOutput {
let message = "Agent service not available. Please set OPENAI_API_KEY, ANTHROPIC_API_KEY, " +
"GEMINI_API_KEY, X_AI_API_KEY, MINIMAX_API_KEY, MINIMAX_CN_API_KEY, OPENROUTER_API_KEY, " +
"or configure ollama/<model>, lmstudio/<model>, or a custom provider."
let error = [
"success": false,
"error": message
] as [String: Any]
if let jsonData = try? JSONSerialization.data(withJSONObject: error, options: .prettyPrinted),
let jsonString = String(data: jsonData, encoding: .utf8) {
print(jsonString)
} else {
print("{\"success\":false,\"error\":\"Agent service not available\"}")
}
} else {
let errorPrefix = [
"\(TerminalColor.red)Error: Agent service not available.",
" Please set OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, X_AI_API_KEY,",
" MINIMAX_API_KEY, MINIMAX_CN_API_KEY, OPENROUTER_API_KEY,",
" or configure ollama/<model>, lmstudio/<model>, or a custom provider."
].joined()
let errorMessageLine = [errorPrefix, "\(TerminalColor.reset)"].joined()
print(errorMessageLine)
}
}
}
extension AgentCommand: ParsableCommand {}
extension AgentCommand: AsyncRuntimeCommand {}

View File

@ -1,26 +0,0 @@
//
// AgentMessages.swift
// PeekabooCLI
//
enum AgentMessages {
enum Chat {
static let jsonDisabled = "Interactive chat is not available while --json output is enabled."
static let quietDisabled = "Interactive chat requires visible output. Remove --quiet to continue."
static let dryRunDisabled = "Interactive chat cannot run in --dry-run mode."
static let noCacheDisabled = "Interactive chat needs session caching. Remove --no-cache."
static let typedOnly = "Interactive chat currently accepts typed input only."
static let nonInteractiveHelp = """
Provide a task or run with --chat in an interactive terminal to start the agent chat loop.
"""
}
enum Audio {
static func processingError(_ error: any Error) -> String {
"Audio processing failed: \(error.localizedDescription)"
}
static let genericProcessingError = "Audio processing failed"
}
}

View File

@ -1,402 +0,0 @@
//
// AgentOutputDelegate+Formatting.swift
// Peekaboo
//
import Foundation
import PeekabooCore
@available(macOS 14.0, *)
extension AgentOutputDelegate {
// MARK: - Helper Methods
func shouldSkipCommunicationOutput(for toolType: ToolType?) -> Bool {
guard let toolType else { return false }
return [ToolType.taskCompleted, .needMoreInformation, .needInfo].contains(toolType)
}
func printToolCallStart(
displayName: String,
args: [String: Any],
rawArguments: String,
formatter: any ToolFormatter
) {
let sanitizedName = self.cleanToolPrefix(displayName)
switch self.outputMode {
case .minimal:
print(sanitizedName, terminator: "")
case .verbose:
print("\(TerminalColor.blue)\(TerminalColor.bold)\(sanitizedName)\(TerminalColor.reset)")
if rawArguments.isEmpty || rawArguments == "{}" {
print("\(TerminalColor.gray)Arguments: (none)\(TerminalColor.reset)")
} else if let formatted = formatJSON(rawArguments) {
print("\(TerminalColor.gray)Arguments:\(TerminalColor.reset)")
print(formatted)
}
case .enhanced:
let startMessage = self.cleanToolPrefix(formatter.formatStarting(arguments: args))
print(
"\(TerminalColor.blue)\(TerminalColor.bold)\(startMessage)\(TerminalColor.reset)",
terminator: ""
)
default: // .normal, .compact
print(
"\(TerminalColor.blue)\(TerminalColor.bold)\(sanitizedName)\(TerminalColor.reset)",
terminator: ""
)
let summary = formatter.formatCompactSummary(arguments: args)
if !summary.isEmpty {
print(" \(TerminalColor.gray)\(summary)\(TerminalColor.reset)", terminator: "")
}
}
fflush(stdout)
}
/// Remove leading glyph tokens like "[sh]" from tool narration so agent output reads naturally.
func cleanToolPrefix(_ text: String) -> String {
var result = text.trimmingCharacters(in: .whitespacesAndNewlines)
while result.hasPrefix("[") {
guard let closing = result.firstIndex(of: "]") else { break }
let next = result.index(after: closing)
result = String(result[next...]).trimmingCharacters(in: .whitespacesAndNewlines)
}
return result
}
func successStatusLine(resultSummary: String, durationString: String) -> String {
if resultSummary.isEmpty {
return " \(durationString)"
}
let summarySegment = [
" ",
TerminalColor.bold,
resultSummary,
TerminalColor.reset
].joined()
return "\(summarySegment)\(durationString)"
}
func failureStatusLine(message: String, durationString: String) -> String {
let statusPrefix = [
" ",
TerminalColor.red,
AgentDisplayTokens.Status.failure
].joined()
return [
statusPrefix,
" ",
message,
TerminalColor.reset,
durationString
].joined()
}
func completionSummaryLine(totalElapsed: TimeInterval, toolsText: String, tokenInfo: String) -> String {
let summaryPrefix = "\(TerminalColor.gray)Task completed in \(formatDuration(totalElapsed))"
return [
"\n",
summaryPrefix,
" with \(toolsText)\(tokenInfo)",
TerminalColor.reset
].joined()
}
func durationString(for toolName: String) -> String {
if let startTime = self.toolStartTimes[toolName] {
self.toolStartTimes.removeValue(forKey: toolName)
let elapsed = Date().timeIntervalSince(startTime)
return " \(TerminalColor.gray)(\(formatDuration(elapsed)))\(TerminalColor.reset)"
}
return ""
}
func printInvalidResult(rawResult: String, durationString: String) {
if self.outputMode == .verbose {
let failureBadge = [
" ",
TerminalColor.red,
AgentDisplayTokens.Status.failure
].joined()
let invalidJsonMessage = [
failureBadge,
" Invalid JSON result",
TerminalColor.reset,
durationString
].joined()
print(invalidJsonMessage)
let rawResultLine = [
TerminalColor.gray,
"Raw result: \(rawResult.prefix(200))",
TerminalColor.reset
].joined()
print(rawResultLine)
} else {
let failureBadge = [
" ",
TerminalColor.red,
AgentDisplayTokens.Status.failure
].joined()
let invalidResultMessage = [
failureBadge,
" Invalid result",
TerminalColor.reset,
durationString
].joined()
print(invalidResultMessage)
}
}
func toolFormatter(for name: String) -> (any ToolFormatter, ToolType?) {
if let type = ToolType(rawValue: name) {
return (ToolFormatterRegistry.shared.formatter(for: type), type)
}
return (UnknownToolFormatter(toolName: name), nil)
}
/// Produce a compact diff summary between previous and new arguments for the same tool name.
func diffSummary(for toolName: String, newArgs: [String: Any]) -> String? {
guard let previous = self.lastToolArguments[toolName] else { return nil }
var changes: [String] = []
for (key, newValue) in newArgs {
guard let prevValue = previous[key] else {
changes.append("+\(key)")
continue
}
if !self.valuesEqual(prevValue, newValue) {
let rendered = self.renderValue(newValue)
changes.append("\(key): \(rendered)")
}
if changes.count >= 3 { break }
}
if changes.isEmpty {
return nil
}
return changes.joined(separator: ", ")
}
func valuesEqual(_ lhs: Any, _ rhs: Any) -> Bool {
switch (lhs, rhs) {
case let (l as String, r as String): l == r
case let (l as Int, r as Int): l == r
case let (l as Double, r as Double): l == r
case let (l as Bool, r as Bool): l == r
default:
false
}
}
func dictionariesEqual(_ lhs: [String: Any], _ rhs: [String: Any]) -> Bool {
guard lhs.count == rhs.count else { return false }
for (key, lval) in lhs {
guard let rval = rhs[key], self.valuesEqual(lval, rval) else { return false }
}
return true
}
func renderValue(_ value: Any) -> String {
switch value {
case let str as String:
let max = 40
if str.count > max {
let idx = str.index(str.startIndex, offsetBy: max)
return String(str[..<idx]) + ""
}
return str
case let num as Int: return String(num)
case let num as Double: return String(format: "%.3f", num)
case let bool as Bool: return bool ? "true" : "false"
default:
if let data = try? JSONSerialization.data(withJSONObject: ["v": value], options: []),
let text = String(data: data, encoding: .utf8) {
return text.replacingOccurrences(of: "{\"v\":", with: "")
.trimmingCharacters(in: CharacterSet(charactersIn: "}"))
}
return ""
}
}
func resultSummary(
for name: String,
json: [String: Any],
formatter: any ToolFormatter,
summary: ToolEventSummary?
) -> String {
if let summaryText = summary?.shortDescription(toolName: name) {
return summaryText
}
var fallback = formatter.formatResultSummary(result: json)
guard name == "app" else {
return self.cleanToolPrefix(fallback)
}
if let meta = json["meta"] as? [String: Any],
let appName = meta["app_name"] as? String,
let content = json["content"] as? [[String: Any]],
let firstContent = content.first,
let text = firstContent["text"] as? String {
switch text {
case let value where value.contains("Launched"):
fallback = "\(appName) launched"
case let value where value.contains("Quit"):
fallback = "\(appName) quit"
case let value where value.contains("Focused") || value.contains("Switched"):
fallback = "\(appName) focused"
case let value where value.contains("Hidden"):
fallback = "\(appName) hidden"
case let value where value.contains("Unhidden"):
fallback = "\(appName) shown"
default:
break
}
}
return self.cleanToolPrefix(fallback)
}
func handleSuccess(
resultSummary: String,
durationString: String,
result: String,
json: [String: Any]
) {
switch self.outputMode {
case .minimal:
let prefix = resultSummary.isEmpty ? "" : " \(resultSummary)"
print("\(prefix)\(durationString)")
case .verbose:
print(" \(durationString)")
if let formatted = formatJSON(result) {
print("\(TerminalColor.gray)Result:\(TerminalColor.reset)")
print(formatted)
}
default:
print(self.successStatusLine(resultSummary: resultSummary, durationString: durationString))
self.printResultDetails(from: json)
}
}
func handleFailure(message: String, durationString: String, json: [String: Any], tool: String) {
if self.outputMode == .minimal {
print(" FAILED\(durationString)")
} else {
print(self.failureStatusLine(message: message, durationString: durationString))
}
self.displayEnhancedError(tool: tool, json: json)
}
func handleCommunicationToolComplete(name: String, toolType: ToolType) {
if self.outputMode == .verbose {
let toolName = toolType.rawValue
.replacingOccurrences(of: "_", with: " ")
.capitalized
print("\n\(AgentDisplayTokens.Status.success) \(toolName) completed")
}
}
func displayEnhancedError(tool: String, json: [String: Any]) {
guard self.outputMode != .minimal && self.outputMode != .quiet else { return }
if let error = json["error"] as? String {
print(" \(TerminalColor.gray)Error: \(error)\(TerminalColor.reset)")
}
if let suggestion = json["suggestion"] as? String {
print(" \(TerminalColor.yellow)💡 Suggestion: \(suggestion)\(TerminalColor.reset)")
}
if self.outputMode == .verbose,
let details = json["details"] as? [String: Any],
let formatted = try? JSONSerialization.data(withJSONObject: details, options: .prettyPrinted),
let detailsStr = String(data: formatted, encoding: .utf8) {
print(" \(TerminalColor.gray)Details:\(TerminalColor.reset)")
print(detailsStr)
}
}
func printResultDetails(from json: [String: Any]) {
guard self.outputMode != .minimal && self.outputMode != .quiet else { return }
guard let detail = self.primaryResultMessage(from: json) else { return }
let snippet = detail.trimmingCharacters(in: .whitespacesAndNewlines)
let sanitized = self.cleanToolPrefix(snippet)
guard !sanitized.isEmpty else { return }
print("\n \(TerminalColor.gray)\(sanitized.prefix(240))\(TerminalColor.reset)")
}
func primaryResultMessage(from json: [String: Any]) -> String? {
if let message = json["message"] as? String, !message.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return message
}
if let content = json["content"] as? [[String: Any]] {
for item in content {
if let text = item["text"] as? String,
!text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return text
}
}
}
if let meta = json["meta"] as? [String: Any],
let message = meta["message"] as? String,
!message.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
return message
}
return nil
}
}
// MARK: - Supporting Types
/// Formatter for unknown tools.
private class UnknownToolFormatter: BaseToolFormatter {
private let toolName: String
override nonisolated init(toolType: ToolType) {
fatalError("Use init(toolName:)")
}
init(toolName: String) {
self.toolName = toolName
// Use wait as the inert placeholder so unknown tools still get a formatter base.
super.init(toolType: .wait)
}
override nonisolated func formatStarting(arguments: [String: Any]) -> String {
"\(self.toolName.replacingOccurrences(of: "_", with: " ").capitalized)"
}
override nonisolated func formatCompleted(result: [String: Any], duration: TimeInterval) -> String {
"→ completed"
}
override nonisolated func formatError(error: String, result: [String: Any]) -> String {
"\(AgentDisplayTokens.Status.failure) \(error)"
}
override nonisolated func formatCompactSummary(arguments: [String: Any]) -> String {
""
}
override nonisolated func formatResultSummary(result: [String: Any]) -> String {
""
}
override nonisolated func formatForTitle(arguments: [String: Any]) -> String {
self.toolName
}
}

View File

@ -1,325 +0,0 @@
//
// AgentOutputDelegate.swift
// Peekaboo
//
import Foundation
import PeekabooCore
import Spinner
import Tachikoma
/// Handles agent output formatting and display for different output modes
@available(macOS 14.0, *)
final class AgentOutputDelegate: PeekabooCore.AgentEventDelegate {
// MARK: - Properties
let outputMode: OutputMode
private let jsonOutput: Bool
private let task: String?
// Tool tracking
private var currentTool: String?
var toolStartTimes: [String: Date] = [:]
var lastToolArguments: [String: [String: Any]] = [:]
private var toolCallCount = 0
private var totalTokens = 0
// Animation and UI
private var spinner: Spinner?
private var hasReceivedContent = false
private var isThinking = false
private var hasShownFinalSummary = false
private let startTime = Date()
// MARK: - Initialization
init(outputMode: OutputMode, jsonOutput: Bool, task: String?) {
self.outputMode = outputMode
self.jsonOutput = jsonOutput
self.task = task
}
}
@available(macOS 14.0, *)
extension AgentOutputDelegate {
// MARK: - AgentEventDelegate
func agentDidEmitEvent(_ event: PeekabooCore.AgentEvent) {
guard !self.jsonOutput else { return }
switch event {
case let .started(task):
self.handleStarted(task)
case let .toolCallStarted(name, arguments):
self.handleToolCallStarted(name: name, arguments: arguments)
case let .toolCallUpdated(name, arguments):
self.handleToolCallUpdated(name: name, arguments: arguments)
case let .toolCallCompleted(name, result):
self.handleToolCallCompleted(name: name, result: result)
case let .assistantMessage(content):
self.handleAssistantMessage(content)
case let .thinkingMessage(content):
self.handleThinkingMessage(content)
case .verificationCompleted, .desktopContextRefreshed:
break
case let .error(message):
self.handleError(message)
case let .completed(summary, usage):
self.handleCompleted(summary: summary, usage: usage)
case .queueDrained:
break
}
}
// MARK: - Event Handlers
private func handleStarted(_ task: String) {
guard self.outputMode != .quiet else { return }
if self.outputMode == .verbose {
print("\n🚀 Starting agent task: \(task)")
} else if self.outputMode == .enhanced || self.outputMode == .compact {
// Start spinner animation (fallback color)
self.spinner = Spinner(.dots, "Thinking...", color: .default)
self.spinner?.start()
} else if self.outputMode == .minimal {
print("Starting: \(task)")
}
}
private func handleToolCallStarted(name: String, arguments: String) {
self.currentTool = name
self.toolStartTimes[name] = Date()
self.toolCallCount += 1
let args = parseArguments(arguments)
self.lastToolArguments[name] = args
let (formatter, toolType) = self.toolFormatter(for: name)
var displayName = toolType?.displayName ?? name.replacingOccurrences(of: "_", with: " ").capitalized
if name == "app", let action = args["action"] as? String {
let appName = (args["name"] as? String) ?? (args["bundleId"] as? String) ?? ""
displayName = "App \(action.capitalized)\(appName.isEmpty ? "" : ": \(appName)")"
}
let titleSummary = formatter.formatForTitle(arguments: args)
updateTerminalTitle("\(displayName): \(titleSummary) - \(self.task?.prefix(30) ?? "")")
guard self.outputMode != .quiet else { return }
self.spinner?.stop()
self.spinner = nil
self.isThinking = false
guard !self.shouldSkipCommunicationOutput(for: toolType) else { return }
if self.hasReceivedContent {
print()
self.hasReceivedContent = false
}
self.printToolCallStart(
displayName: displayName,
args: args,
rawArguments: arguments,
formatter: formatter
)
}
private func handleToolCallUpdated(name: String, arguments: String) {
guard self.outputMode != .quiet else { return }
guard !self.shouldSkipCommunicationOutput(for: ToolType(rawValue: name)) else { return }
let args = parseArguments(arguments)
if let previous = self.lastToolArguments[name], self.dictionariesEqual(previous, args) {
return // no change; avoid spamming the log
}
let diffSummary = self.diffSummary(for: name, newArgs: args)
let (formatter, _ /* toolType */ ) = self.toolFormatter(for: name)
switch self.outputMode {
case .minimal:
if let diffSummary {
print("\(diffSummary)", terminator: "")
} else {
print("", terminator: "")
}
case .verbose:
let clean = self.cleanToolPrefix(formatter.formatStarting(arguments: args))
if let diffSummary {
print("↻ Updated args: \(diffSummary) (\(clean))")
} else {
print("↻ Updated args: \(clean)")
}
default:
let clean = self.cleanToolPrefix(formatter.formatStarting(arguments: args))
if let diffSummary {
print(" \(TerminalColor.blue)\(TerminalColor.reset) \(diffSummary)", terminator: "")
} else {
print(" \(TerminalColor.blue)\(TerminalColor.reset) \(clean)", terminator: "")
}
}
self.lastToolArguments[name] = args
fflush(stdout)
}
private func handleToolCallCompleted(name: String, result: String) {
let durationString = self.durationString(for: name)
guard self.outputMode != .quiet else { return }
guard let json = parseResult(result) else {
self.printInvalidResult(rawResult: result, durationString: durationString)
return
}
let (formatter, toolType) = self.toolFormatter(for: name)
let summary = ToolEventSummary.from(resultJSON: json)
if let toolType, [ToolType.taskCompleted, .needMoreInformation, .needInfo].contains(toolType) {
self.handleCommunicationToolComplete(name: name, toolType: toolType)
return
}
let success = (json["success"] as? Bool) ?? true
if success {
let resultSummary = self.resultSummary(
for: name,
json: json,
formatter: formatter,
summary: summary
)
self.handleSuccess(
resultSummary: resultSummary,
durationString: durationString,
result: result,
json: json
)
} else {
let errorMessage = (json["error"] as? String) ?? "Failed"
self.handleFailure(message: errorMessage, durationString: durationString, json: json, tool: name)
}
fflush(stdout)
}
private func handleAssistantMessage(_ content: String) {
self.hasReceivedContent = true
if self.outputMode == .verbose {
print("\n\(AgentDisplayTokens.Status.dialog) \(content)")
} else if self.outputMode != .quiet {
// Stop animations when content arrives
if self.spinner != nil {
self.spinner?.stop()
self.spinner = nil
print()
}
if self.isThinking {
self.isThinking = false
print()
}
print(content, terminator: "")
fflush(stdout)
}
}
private func handleThinkingMessage(_ content: String) {
self.hasReceivedContent = true
if self.outputMode == .verbose {
print("\n\(AgentDisplayTokens.Status.planning) Thinking: \(content)")
return
}
if self.spinner != nil {
self.spinner?.stop()
self.spinner = nil
print()
}
if !self.isThinking {
self.isThinking = true
print("\n\(TerminalColor.gray)", terminator: "")
}
// Render thinking in italic gray so it stands apart from streamed assistant text.
print("\(TerminalColor.gray)\(TerminalColor.italic)\(content)\(TerminalColor.reset)")
fflush(stdout)
}
private func handleError(_ message: String) {
self.spinner?.stop()
self.spinner = nil
if self.outputMode == .minimal {
print("\nError: \(message)")
} else if self.outputMode != .quiet {
print("\n\(TerminalColor.red)\(AgentDisplayTokens.Status.failure) Error: \(message)\(TerminalColor.reset)")
}
}
private func handleCompleted(summary: String, usage: Tachikoma.Usage?) {
self.spinner?.stop()
self.spinner = nil
// Update token count if available
if let usage {
self.totalTokens = usage.inputTokens + usage.outputTokens
}
guard !self.hasShownFinalSummary && self.outputMode != .quiet else { return }
let totalElapsed = Date().timeIntervalSince(self.startTime)
let tokenInfo = self.totalTokens > 0 ? ", \(self.totalTokens) tokens" : ""
let toolsText = self.toolCallCount == 1 ? "⚒ 1 tool" : "\(self.toolCallCount) tools"
if !summary.isEmpty && self.outputMode == .verbose {
print("\n\(TerminalColor.gray)Summary: \(summary)\(TerminalColor.reset)")
}
print(self.completionSummaryLine(
totalElapsed: totalElapsed,
toolsText: toolsText,
tokenInfo: tokenInfo
))
self.hasShownFinalSummary = true
}
// MARK: - Public Methods
func updateTokenCount(_ count: Int) {
self.totalTokens = count
}
func showFinalSummaryIfNeeded(_ result: AgentExecutionResult) {
guard !self.hasShownFinalSummary && self.outputMode != .quiet else { return }
let totalElapsed = Date().timeIntervalSince(self.startTime)
let tokenInfo = self.totalTokens > 0 ? ", \(self.totalTokens) tokens" : ""
let toolsText = self.toolCallCount == 1 ? "⚒ 1 tool" : "\(self.toolCallCount) tools"
if !result.content.isEmpty && self.outputMode == .verbose {
print("\n\(TerminalColor.gray)Summary: \(result.content)\(TerminalColor.reset)")
}
print(self.completionSummaryLine(
totalElapsed: totalElapsed,
toolsText: toolsText,
tokenInfo: tokenInfo
))
self.hasShownFinalSummary = true
}
}

View File

@ -1,135 +0,0 @@
import Commander
import Foundation
import PeekabooCore
import PeekabooFoundation
@MainActor
extension SeeCommand {
func detectElements(
imageData: Data,
windowContext: WindowContext?,
snapshotID: String? = nil
) async throws -> ElementDetectionResult {
self.logger.operationStart("element_detection")
defer { self.logger.operationComplete("element_detection") }
let timeoutSeconds = Self.detectionTimeoutSeconds(
configuredTimeoutSeconds: self.timeoutSeconds,
analyze: self.analyze
)
do {
return try await Self.detectElements(
automation: self.services.automation,
imageData: imageData,
windowContext: windowContext,
timeoutSeconds: timeoutSeconds,
snapshotID: snapshotID,
interactionMutationTracker: self.resolvedRuntime.observationTimeoutMutationTracker
)
} catch is TimeoutError {
throw CaptureError.detectionTimedOut(timeoutSeconds)
}
}
static func detectionTimeoutSeconds(
configuredTimeoutSeconds: Int?,
analyze: String?
) -> TimeInterval {
TimeInterval(configuredTimeoutSeconds ?? ((analyze == nil) ? 20 : 60))
}
static func remoteDetectionRequestTimeoutSeconds(for timeoutSeconds: TimeInterval) -> TimeInterval {
timeoutSeconds + 5
}
static func detectElements(
automation: any UIAutomationServiceProtocol,
imageData: Data,
windowContext: WindowContext?,
timeoutSeconds: TimeInterval,
snapshotID: String? = nil,
interactionMutationTracker: InteractionMutationTracker? = nil
) async throws -> ElementDetectionResult {
try await withWallClockTimeout(
seconds: timeoutSeconds,
interactionMutationTracker: interactionMutationTracker
) {
if let timeoutAdjustingAutomation = automation as? any DetectElementsRequestTimeoutAdjusting {
return try await timeoutAdjustingAutomation.detectElements(
in: imageData,
snapshotId: snapshotID,
windowContext: windowContext,
requestTimeoutSec: Self.remoteDetectionRequestTimeoutSeconds(for: timeoutSeconds)
)
}
return try await AutomationServiceBridge.detectElements(
automation: automation,
imageData: imageData,
snapshotId: snapshotID,
windowContext: windowContext
)
}
}
func resolveCaptureContext() async throws -> CaptureContext {
if self.menubar {
if let popover = try await self.captureMenuBarPopover() {
return CaptureContext(
captureResult: popover.captureResult,
captureBounds: popover.windowBounds,
prefersOCR: true,
ocrMethod: "OCR",
windowIdOverride: popover.windowId
)
}
self.logger.verbose("No menu bar popover detected; capturing menu bar area", category: "Capture")
let rect = try self.menuBarRect()
let result = try await self.services.screenCapture.captureArea(rect)
return CaptureContext(
captureResult: result,
captureBounds: rect,
prefersOCR: true,
ocrMethod: "OCR",
windowIdOverride: nil
)
}
let result = try await self.performLegacyScreenCapture()
return CaptureContext(
captureResult: result,
captureBounds: nil,
prefersOCR: false,
ocrMethod: nil,
windowIdOverride: nil
)
}
private func performLegacyScreenCapture() async throws -> CaptureResult {
let effectiveMode = self.determineMode()
self.logger.verbose(
"Determined capture mode",
category: "Capture",
metadata: ["mode": effectiveMode.rawValue]
)
self.logger.operationStart("capture_phase", metadata: ["mode": effectiveMode.rawValue])
switch effectiveMode {
case .screen:
// Handle screen capture with multi-screen support
let result = try await self.performScreenCapture()
self.logger.operationComplete("capture_phase", metadata: ["mode": effectiveMode.rawValue])
return result
case .multi:
// Commander currently treats multi captures as multi-display screen grabs
let result = try await self.performScreenCapture()
self.logger.operationComplete("capture_phase", metadata: ["mode": effectiveMode.rawValue])
return result
case .window, .frontmost, .area:
throw ValidationError("\(effectiveMode.rawValue) captures must use the desktop observation pipeline")
}
}
}

View File

@ -1,119 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
import PeekabooFoundation
@MainActor
extension SeeCommand {
var usesTemporaryScreenshotOutput: Bool {
self.jsonOutput && self.path == nil
}
func screenshotOutputPath(snapshotID: String? = nil) -> String {
if self.usesTemporaryScreenshotOutput {
return self.temporaryScreenshotDirectory(snapshotID: snapshotID)
.appendingPathComponent("raw.png")
.path
}
let timestamp = Date().timeIntervalSince1970
let filename = "peekaboo_see_\(Int(timestamp)).png"
return ObservationCommandSupport.outputPath(
path: self.path,
format: .png,
defaultDirectory: ConfigurationManager.shared.getDefaultSavePath(cliValue: nil),
defaultFileName: filename
)
}
func saveScreenshot(_ imageData: Data, snapshotID: String) throws -> String {
let outputPath = self.screenshotOutputPath(snapshotID: snapshotID)
let directory = (outputPath as NSString).deletingLastPathComponent
try FileManager.default.createDirectory(
atPath: directory,
withIntermediateDirectories: true
)
try imageData.write(to: URL(fileURLWithPath: outputPath))
self.logger.verbose("Saved screenshot to: \(outputPath)")
return outputPath
}
func cleanupTemporaryScreenshotOutput(snapshotID: String) {
guard self.usesTemporaryScreenshotOutput else { return }
try? FileManager.default.removeItem(at: self.temporaryScreenshotDirectory(snapshotID: snapshotID))
}
private func temporaryScreenshotDirectory(snapshotID: String?) -> URL {
FileManager.default.temporaryDirectory
.appendingPathComponent("peekaboo-see", isDirectory: true)
.appendingPathComponent(snapshotID ?? UUID().uuidString, isDirectory: true)
}
func resolveSeeWindowIndex(appIdentifier: String, titleFragment: String?) async throws -> Int? {
guard let fragment = titleFragment, !fragment.isEmpty else {
return nil
}
let appInfo = try await self.services.applications.findApplication(identifier: appIdentifier)
let snapshot = try await WindowListMapper.shared.snapshot()
let appWindows = WindowListMapper.scWindows(
for: appInfo.processIdentifier,
in: snapshot.scWindows
)
guard !appWindows.isEmpty else {
throw CaptureError.windowNotFound
}
if let index = WindowListMapper.scWindowIndex(
for: appInfo.processIdentifier,
titleFragment: fragment,
in: snapshot
) {
return index
}
if let index = WindowListMapper.scWindowIndex(for: fragment, in: appWindows) {
return index
}
throw CaptureError.windowNotFound
}
func resolveWindowId(appIdentifier: String, titleFragment: String?) async throws -> Int? {
guard let fragment = titleFragment, !fragment.isEmpty else {
return nil
}
let windows = try await self.services.windows.listWindows(
target: .applicationAndTitle(app: appIdentifier, title: fragment)
)
return windows.first?.windowID
}
func generateAnnotatedScreenshot(
snapshotId: String,
originalPath: String
) async throws -> String? {
guard let detectionResult = try await self.services.snapshots.getDetectionResult(snapshotId: snapshotId)
else {
self.logger.info("No detection result found for snapshot")
return nil
}
let renderer = ObservationAnnotationRenderer(debugMode: self.verbose)
let annotatedPath = try renderer.renderAnnotatedScreenshot(
originalPath: originalPath,
detectionResult: detectionResult
)
guard let annotatedPath else {
return nil
}
self.logger.verbose("Created annotated screenshot: \(annotatedPath)")
return annotatedPath
}
}

View File

@ -1,78 +0,0 @@
import Commander
extension SeeCommand: CommanderSignatureProviding {
static func commanderSignature() -> CommandSignature {
CommandSignature(
options: [
.commandOption(
"app",
help: "Application name to capture, or special values: 'menubar', 'frontmost'",
long: "app"
),
.commandOption(
"pid",
help: "Target application by process ID",
long: "pid"
),
.commandOption(
"windowTitle",
help: "Specific window title to capture",
long: "window-title"
),
.commandOption(
"windowId",
help: "Capture a specific window by CoreGraphics window id "
+ "(window_id from `peekaboo window list --json`)",
long: "window-id"
),
.commandOption(
"mode",
help: "Capture mode (screen, window, frontmost)",
long: "mode"
),
.commandOption(
"path",
help: "Output path for screenshot",
long: "path"
),
.commandOption(
"captureEngine",
help: "Capture engine: auto|classic|cg|modern|sckit (defaults to auto)",
long: "capture-engine"
),
.commandOption(
"screenIndex",
help: "Specific screen index to capture (0-based)",
long: "screen-index"
),
.commandOption(
"analyze",
help: "Analyze captured content with AI",
long: "analyze"
),
.commandOption(
"timeoutSeconds",
help: "Overall timeout in seconds (default: 20, or 60 when --analyze is set)",
long: "timeout-seconds"
),
],
flags: [
.commandFlag(
"annotate",
help: "Generate annotated screenshot with interaction markers",
long: "annotate"
),
.commandFlag(
"menubar",
help: "Capture menu bar popovers via window list + OCR",
long: "menubar"
),
.commandFlag(
"noWebFocus",
help: "Skip web-content focus fallback when no text fields are detected",
long: "no-web-focus"
),
]
)
}
}

View File

@ -1,177 +0,0 @@
import Foundation
import PeekabooCore
@available(macOS 14.0, *)
@MainActor
extension SeeCommand {
func performCaptureWithDetection(snapshotID: String) async throws -> CaptureAndDetectionResult {
if let observationResult = try await self.performObservationCaptureWithDetectionIfPossible(
snapshotID: snapshotID
) {
return observationResult
}
let captureContext = try await self.resolveCaptureContext()
let captureResult = captureContext.captureResult
self.logger.startTimer("file_write")
let outputPath = try saveScreenshot(captureResult.imageData, snapshotID: snapshotID)
self.logger.stopTimer("file_write")
let windowContext = WindowContext(
applicationName: captureResult.metadata.applicationInfo?.name,
applicationBundleId: captureResult.metadata.applicationInfo?.bundleIdentifier,
applicationProcessId: captureResult.metadata.applicationInfo?.processIdentifier,
windowTitle: captureResult.metadata.windowInfo?.title,
windowID: captureContext.windowIdOverride ?? captureResult.metadata.windowInfo?.windowID,
windowBounds: captureContext.captureBounds ?? captureResult.metadata.windowInfo?.bounds,
shouldFocusWebContent: self.noWebFocus ? false : true,
traversalBudget: self.axTraversalBudget()
)
let detectionResult = try await self.detectElements(
for: captureContext,
windowContext: windowContext,
snapshotID: snapshotID
)
let resultWithPath = ElementDetectionResult(
snapshotId: snapshotID,
screenshotPath: outputPath,
elements: detectionResult.elements,
metadata: detectionResult.metadata
)
try await self.services.snapshots.storeScreenshot(
SnapshotScreenshotRequest(
snapshotId: snapshotID,
screenshotPath: outputPath,
applicationBundleId: captureResult.metadata.applicationInfo?.bundleIdentifier,
applicationProcessId: captureResult.metadata.applicationInfo.map { Int32($0.processIdentifier) },
applicationName: windowContext.applicationName,
windowTitle: windowContext.windowTitle,
windowBounds: windowContext.windowBounds
)
)
try await self.services.snapshots.storeDetectionResult(
snapshotId: snapshotID,
result: resultWithPath
)
return CaptureAndDetectionResult(
snapshotId: snapshotID,
screenshotPath: outputPath,
annotatedPath: nil,
elements: detectionResult.elements,
metadata: detectionResult.metadata,
observation: nil
)
}
private func detectElements(
for captureContext: CaptureContext,
windowContext: WindowContext,
snapshotID: String
) async throws -> ElementDetectionResult {
let captureResult = captureContext.captureResult
let detectionStart = Date()
if captureContext.prefersOCR {
self.logger.verbose("Running OCR for menu bar popover", category: "Capture")
let ocrElements = try self.ocrElements(
imageData: captureResult.imageData,
windowBounds: captureContext.captureBounds ?? captureResult.metadata.windowInfo?.bounds
)
let warnings = ocrElements.isEmpty ? ["OCR produced no elements"] : []
let metadata = DetectionMetadata(
detectionTime: Date().timeIntervalSince(detectionStart),
elementCount: ocrElements.count,
method: captureContext.ocrMethod ?? "OCR",
warnings: warnings,
windowContext: windowContext,
isDialog: false
)
return ElementDetectionResult(
snapshotId: snapshotID,
screenshotPath: "",
elements: DetectedElements(other: ocrElements),
metadata: metadata
)
}
let detectionResult = try await self.detectElements(
imageData: captureResult.imageData,
windowContext: windowContext,
snapshotID: snapshotID
)
return ElementDetectionResult(
snapshotId: snapshotID,
screenshotPath: detectionResult.screenshotPath,
elements: detectionResult.elements,
metadata: detectionResult.metadata
)
}
private func performObservationCaptureWithDetectionIfPossible(
snapshotID: String
) async throws -> CaptureAndDetectionResult? {
guard let target = try self.observationTargetForCaptureWithDetectionIfPossible() else {
return nil
}
self.logger.verbose("Using desktop observation pipeline", category: "Capture", metadata: [
"target": self.observationTargetDescription(target)
])
let mode = self.determineMode()
self.logger.operationStart("capture_phase", metadata: ["mode": mode.rawValue])
let observation: DesktopObservationResult
do {
observation = try await self.services.desktopObservation
.observe(self.makeObservationRequest(target: target, snapshotID: snapshotID))
} catch DesktopObservationError.targetNotFound(_) where self.menubar {
self.logger.verbose("No observation-backed menu bar popover found; falling back", category: "Capture")
self.logger.operationComplete("capture_phase", success: false, metadata: [
"mode": mode.rawValue,
"fallback": "legacy_menubar",
])
return nil
}
self.logger.operationComplete("capture_phase", metadata: [
"mode": mode.rawValue
])
self.logObservationSpans(observation.timings)
guard let outputPath = observation.files.rawScreenshotPath else {
throw CaptureError.captureFailure("Observation completed without a saved screenshot path")
}
guard let detectionResult = observation.elements else {
throw CaptureError.captureFailure("Observation completed without element detection")
}
return CaptureAndDetectionResult(
snapshotId: snapshotID,
screenshotPath: outputPath,
annotatedPath: observation.files.annotatedScreenshotPath,
elements: detectionResult.elements,
metadata: detectionResult.metadata,
observation: SeeObservationDiagnostics(
timings: observation.timings,
diagnostics: observation.diagnostics
)
)
}
private func logObservationSpans(_ timings: ObservationTimings) {
for span in timings.spans {
self.logger.verbose("Desktop observation span", category: "Performance", metadata: [
"span": span.name,
"duration_ms": Int(span.durationMS.rounded()),
])
}
}
}

View File

@ -1,257 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
@MainActor
extension SeeCommand {
struct MenuBarPopoverContext {
let extras: [MenuExtraInfo]
let ownerPidSet: Set<pid_t>
let canFilterByOwnerPid: Bool
let appHint: String?
let hintExtra: MenuExtraInfo?
let openExtra: MenuExtraInfo?
let preferredExtra: MenuExtraInfo?
let preferredOwnerName: String?
let preferredOwnerPid: pid_t?
let preferredX: CGFloat?
var shouldRelaxFilter: Bool {
self.openExtra != nil || self.appHint != nil
}
var hintName: String? {
self.appHint ?? self.preferredExtra?.title ?? self.preferredExtra?.ownerName
}
}
struct MenuBarCandidateState {
var candidates: [MenuBarPopoverCandidate]
var windowInfoMap: [Int: MenuBarPopoverWindowInfo]
var usedFilteredWindowList: Bool
}
func captureMenuBarPopover(allowAreaFallback: Bool = false) async throws -> MenuBarPopoverCapture? {
let context = try await self.makeMenuBarPopoverContext()
self.logOpenMenuExtraIfNeeded(context)
let snapshot = self.menuBarWindowSnapshot()
var state = self.resolveInitialCandidates(context: context, snapshot: snapshot)
state = self.relaxCandidatesIfNeeded(
context: context,
snapshot: snapshot,
state: state
)
state = self.applyOwnerNameFallbackIfNeeded(
context: context,
snapshot: snapshot,
state: state
)
if state.candidates.isEmpty {
if let capture = try await self.fallbackCaptureForEmptyCandidates(
context: context,
snapshot: snapshot,
state: &state
) {
return capture
}
}
guard !state.candidates.isEmpty else { return nil }
return try await self.capturePopoverFromCandidates(
context: context,
allowAreaFallback: allowAreaFallback,
state: state
)
}
private func makeMenuBarPopoverContext() async throws -> MenuBarPopoverContext {
let extras = try await self.services.menu.listMenuExtras()
let ownerPidSet = Set(extras.compactMap(\.ownerPID))
let canFilterByOwnerPid = !ownerPidSet.isEmpty
let appHint = self.menuBarAppHint()
let hintExtra = self.resolveMenuExtraHint(appHint: appHint, extras: extras)
let openExtra = try await self.resolveOpenMenuExtra(from: extras)
let preferredExtra = appHint != nil ? (hintExtra ?? openExtra) : (openExtra ?? hintExtra)
let preferredOwnerName = appHint ?? preferredExtra?.ownerName ?? preferredExtra?.title
let preferredX = preferredExtra?.position.x
let preferredOwnerPid = preferredExtra?.ownerPID
return MenuBarPopoverContext(
extras: extras,
ownerPidSet: ownerPidSet,
canFilterByOwnerPid: canFilterByOwnerPid,
appHint: appHint,
hintExtra: hintExtra,
openExtra: openExtra,
preferredExtra: preferredExtra,
preferredOwnerName: preferredOwnerName,
preferredOwnerPid: preferredOwnerPid,
preferredX: preferredX
)
}
private func logOpenMenuExtraIfNeeded(_ context: MenuBarPopoverContext) {
guard let openExtra = context.openExtra, let openPid = openExtra.ownerPID else { return }
self.logger.verbose(
"Detected open menu extra",
category: "Capture",
metadata: [
"title": openExtra.title,
"ownerPID": openPid
]
)
}
private func fallbackCaptureForEmptyCandidates(
context: MenuBarPopoverContext,
snapshot: ObservationMenuBarPopoverSnapshot,
state: inout MenuBarCandidateState
) async throws -> MenuBarPopoverCapture? {
if let openMenuCapture = try await self.captureMenuBarPopoverFromOpenMenu(
openExtra: context.openExtra ?? context.hintExtra,
appHint: context.appHint
) {
return openMenuCapture
}
if let preferredX = context.preferredX {
let bandCandidates = self.menuBarPopoverCandidatesByBand(
snapshot: snapshot,
preferredX: preferredX
)
if !bandCandidates.isEmpty {
state.candidates = bandCandidates
state.windowInfoMap = snapshot.windowInfoByID
state.usedFilteredWindowList = false
}
}
return nil
}
private func capturePopoverFromCandidates(
context: MenuBarPopoverContext,
allowAreaFallback: Bool,
state: MenuBarCandidateState
) async throws -> MenuBarPopoverCapture? {
let windowInfoMap = state.windowInfoMap
let selectionCandidates = self.selectCandidates(
from: state.candidates,
preferredOwnerName: context.preferredOwnerName,
windowInfoMap: windowInfoMap,
openExtra: context.openExtra
)
guard let selectionCandidates else { return nil }
let hints = MenuBarPopoverResolverContext.normalizedHints([
context.hintName,
context.preferredOwnerName
])
let resolverContext = MenuBarPopoverResolverContext(
appHint: context.hintName,
preferredOwnerName: context.preferredOwnerName,
ownerPID: context.preferredOwnerPid,
preferredX: context.preferredX,
ocrHints: hints
)
let allowOCR = selectionCandidates.count > 1 && !hints.isEmpty
let allowArea = (context.openExtra != nil || allowAreaFallback)
let candidateOCR = allowOCR ? self.menuBarCandidateOCRMatcher(hints: hints) : nil
let areaOCR = allowArea ? self.menuBarAreaOCRMatcher() : nil
let options = MenuBarPopoverResolver.ResolutionOptions(
allowOCR: allowOCR,
allowAreaFallback: allowArea,
candidateOCR: candidateOCR,
areaOCR: areaOCR
)
guard let resolution = try await MenuBarPopoverResolver.resolve(
candidates: selectionCandidates,
windowInfoById: windowInfoMap,
context: resolverContext,
options: options
) else {
return nil
}
return try await self.captureMenuBarPopover(from: resolution, windowInfoMap: windowInfoMap)
}
private func captureMenuBarPopover(
from resolution: MenuBarPopoverResolution,
windowInfoMap: [Int: MenuBarPopoverWindowInfo]
) async throws -> MenuBarPopoverCapture? {
self.logPopoverResolution(resolution, windowInfoMap: windowInfoMap)
if let captureResult = resolution.captureResult,
let bounds = resolution.bounds {
return MenuBarPopoverCapture(
captureResult: captureResult,
windowBounds: bounds,
windowId: resolution.windowId
)
}
guard let windowId = resolution.windowId,
let bounds = resolution.bounds else {
return nil
}
let captureResult = try await self.services.screenCapture.captureWindow(windowID: CGWindowID(windowId))
return MenuBarPopoverCapture(
captureResult: captureResult,
windowBounds: bounds,
windowId: windowId
)
}
private func logPopoverResolution(
_ resolution: MenuBarPopoverResolution,
windowInfoMap: [Int: MenuBarPopoverWindowInfo]
) {
switch resolution.reason {
case .ocr:
if let windowId = resolution.windowId {
self.logger.verbose(
"Selected menu bar popover via OCR",
category: "Capture",
metadata: [
"windowId": windowId
]
)
}
case .ocrArea:
if let bounds = resolution.bounds {
self.logger.verbose(
"Selected menu bar popover via area capture",
category: "Capture",
metadata: [
"rect": "\(bounds)"
]
)
}
default:
if let windowId = resolution.windowId,
let info = windowInfoMap[windowId] {
self.logger.verbose(
"Selected menu bar popover window",
category: "Capture",
metadata: [
"windowId": windowId,
"owner": info.ownerName ?? "unknown",
"title": info.title ?? "",
"reason": resolution.reason.rawValue
]
)
}
}
}
}

View File

@ -1,234 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
@MainActor
extension SeeCommand {
func menuBarWindowSnapshot() -> ObservationMenuBarPopoverSnapshot {
ObservationMenuBarWindowCatalog.currentPopoverSnapshot(
screens: self.services.screens.listScreens()
)
}
func resolveInitialCandidates(
context: MenuBarPopoverContext,
snapshot: ObservationMenuBarPopoverSnapshot
) -> MenuBarCandidateState {
let filteredCandidates: [MenuBarPopoverCandidate] = if context.canFilterByOwnerPid {
snapshot.candidates.filter { candidate in
context.ownerPidSet.contains(candidate.ownerPID)
}
} else {
snapshot.candidates
}
let usedFilteredWindowList = context.canFilterByOwnerPid &&
!filteredCandidates.isEmpty &&
filteredCandidates.count != snapshot.candidates.count
let baseCandidates = usedFilteredWindowList ? filteredCandidates : snapshot.candidates
var candidates = self.menuBarPopoverCandidates(
candidates: baseCandidates,
ownerPID: context.preferredOwnerPid
)
if candidates.isEmpty, context.preferredOwnerPid != nil {
candidates = self.menuBarPopoverCandidates(
candidates: baseCandidates,
ownerPID: nil
)
}
return MenuBarCandidateState(
candidates: candidates,
windowInfoMap: snapshot.windowInfoByID,
usedFilteredWindowList: usedFilteredWindowList
)
}
func relaxCandidatesIfNeeded(
context: MenuBarPopoverContext,
snapshot: ObservationMenuBarPopoverSnapshot,
state: MenuBarCandidateState
) -> MenuBarCandidateState {
guard state.candidates.isEmpty,
context.shouldRelaxFilter,
state.usedFilteredWindowList else {
return state
}
self.logger.debug("Relaxing menu bar popover filter to full window list")
var candidates = self.menuBarPopoverCandidates(
candidates: snapshot.candidates,
ownerPID: context.preferredOwnerPid
)
if candidates.isEmpty, context.preferredOwnerPid != nil {
candidates = self.menuBarPopoverCandidates(
candidates: snapshot.candidates,
ownerPID: nil
)
}
return MenuBarCandidateState(
candidates: candidates,
windowInfoMap: snapshot.windowInfoByID,
usedFilteredWindowList: false
)
}
func applyOwnerNameFallbackIfNeeded(
context: MenuBarPopoverContext,
snapshot: ObservationMenuBarPopoverSnapshot,
state: MenuBarCandidateState
) -> MenuBarCandidateState {
guard let preferredOwnerName = context.preferredOwnerName,
!preferredOwnerName.isEmpty,
state.usedFilteredWindowList else {
return state
}
let normalized = preferredOwnerName.lowercased()
let ownerMatches = state.candidates.filter { candidate in
let ownerName = state.windowInfoMap[candidate.windowId]?.ownerName?.lowercased() ?? ""
return ownerName == normalized || ownerName.contains(normalized)
}
guard ownerMatches.isEmpty else { return state }
var candidates = self.menuBarPopoverCandidates(
candidates: snapshot.candidates,
ownerPID: context.preferredOwnerPid
)
if candidates.isEmpty, context.preferredOwnerPid != nil {
candidates = self.menuBarPopoverCandidates(
candidates: snapshot.candidates,
ownerPID: nil
)
}
return MenuBarCandidateState(
candidates: candidates,
windowInfoMap: snapshot.windowInfoByID,
usedFilteredWindowList: false
)
}
func selectCandidates(
from candidates: [MenuBarPopoverCandidate],
preferredOwnerName: String?,
windowInfoMap: [Int: MenuBarPopoverWindowInfo],
openExtra: MenuExtraInfo?
) -> [MenuBarPopoverCandidate]? {
guard let preferredOwnerName, !preferredOwnerName.isEmpty else { return candidates }
let normalized = preferredOwnerName.lowercased()
let ownerMatches = candidates.filter { candidate in
let ownerName = windowInfoMap[candidate.windowId]?.ownerName?.lowercased() ?? ""
return ownerName == normalized || ownerName.contains(normalized)
}
if !ownerMatches.isEmpty {
return ownerMatches
}
return openExtra == nil ? nil : candidates
}
private func menuBarPopoverCandidates(
candidates: [MenuBarPopoverCandidate],
ownerPID: pid_t?
) -> [MenuBarPopoverCandidate] {
guard let ownerPID else { return candidates }
return candidates.filter { $0.ownerPID == ownerPID }
}
func menuBarPopoverCandidatesByBand(
snapshot _: ObservationMenuBarPopoverSnapshot,
preferredX: CGFloat
) -> [MenuBarPopoverCandidate] {
ObservationMenuBarWindowCatalog.currentBandCandidates(
preferredX: preferredX,
screens: self.services.screens.listScreens()
)
}
func menuBarAppHint() -> String? {
guard let app = self.app?.trimmingCharacters(in: .whitespacesAndNewlines),
!app.isEmpty else {
return nil
}
let lower = app.lowercased()
if lower == "menubar" || lower == "frontmost" {
return nil
}
return app
}
func resolveMenuExtraHint(
appHint: String?,
extras: [MenuExtraInfo]
) -> MenuExtraInfo? {
guard let appHint else { return nil }
let normalized = appHint.lowercased()
return extras.first { extra in
let candidates = [
extra.title,
extra.rawTitle,
extra.ownerName,
extra.bundleIdentifier,
extra.identifier
].compactMap { $0?.lowercased() }
return candidates.contains(where: { $0 == normalized }) ||
candidates.contains(where: { $0.contains(normalized) })
}
}
func resolveOpenMenuExtra(from extras: [MenuExtraInfo]) async throws -> MenuExtraInfo? {
for extra in extras {
let candidates = [
extra.title,
extra.ownerName,
extra.rawTitle,
extra.identifier,
extra.bundleIdentifier,
].compactMap { $0?.trimmingCharacters(in: .whitespacesAndNewlines) }
for candidate in candidates where !candidate.isEmpty {
let ownerPID: pid_t? = if let extraOwnerPID = extra.ownerPID {
extraOwnerPID
} else {
await self.resolveMenuExtraOwnerPID(extra)
}
let isOpen = await (try? self.services.menu.isMenuExtraMenuOpen(
title: candidate,
ownerPID: ownerPID
)) ?? false
if isOpen {
return extra
}
}
}
return nil
}
func resolveMenuExtraOwnerPID(_ extra: MenuExtraInfo) async -> pid_t? {
if let ownerPID = extra.ownerPID {
return ownerPID
}
guard let runningApps = try? await self.services.applications.listApplications().data.applications else {
return nil
}
if let bundleIdentifier = extra.bundleIdentifier,
let match = runningApps.first(where: { $0.bundleIdentifier == bundleIdentifier }) {
return match.processIdentifier
}
if let ownerName = extra.ownerName {
if let match = runningApps.first(where: { $0.name == ownerName }) {
return match.processIdentifier
}
let normalizedOwner = ownerName.lowercased()
if let match = runningApps.first(where: {
($0.bundleIdentifier ?? "").lowercased().contains(normalizedOwner)
}) {
return match.processIdentifier
}
}
return nil
}
}

View File

@ -1,32 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
import PeekabooFoundation
@MainActor
extension SeeCommand {
func menuBarRect() throws -> CGRect {
let screens = self.services.screens.listScreens()
guard let mainScreen = screens.first(where: \.isPrimary) ?? screens.first else {
throw PeekabooError.captureFailed("No main screen found")
}
let menuBarHeight = self.menuBarHeight(for: mainScreen)
return CGRect(
x: mainScreen.frame.origin.x,
y: mainScreen.frame.origin.y + mainScreen.frame.height - menuBarHeight,
width: mainScreen.frame.width,
height: menuBarHeight
)
}
func menuBarHeight(for screen: MenuBarPopoverDetector.ScreenBounds) -> CGFloat {
let height = max(0, screen.frame.maxY - screen.visibleFrame.maxY)
return height > 0 ? height : 24.0
}
private func menuBarHeight(for screen: ScreenInfo) -> CGFloat {
let height = max(0, screen.frame.maxY - screen.visibleFrame.maxY)
return height > 0 ? height : 24.0
}
}

View File

@ -1,106 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
@MainActor
extension SeeCommand {
func menuBarCandidateOCRMatcher(hints: [String]) -> MenuBarPopoverResolver.CandidateOCR {
let selector = self.menuBarPopoverOCRSelector()
return { candidate, _ in
guard let match = try await selector.matchCandidate(
windowID: CGWindowID(candidate.windowId),
bounds: candidate.bounds,
hints: hints
)
else { return nil }
return MenuBarPopoverResolver.OCRMatch(
captureResult: match.captureResult,
bounds: match.bounds
)
}
}
func menuBarAreaOCRMatcher() -> MenuBarPopoverResolver.AreaOCR {
let selector = self.menuBarPopoverOCRSelector()
return { preferredX, hints in
guard let match = try await selector.matchArea(preferredX: preferredX, hints: hints) else { return nil }
return MenuBarPopoverResolver.OCRMatch(
captureResult: match.captureResult,
bounds: match.bounds
)
}
}
func captureMenuBarPopoverFromOpenMenu(
openExtra: MenuExtraInfo?,
appHint: String?
) async throws -> MenuBarPopoverCapture? {
let ownerPID: pid_t? = if let openExtra {
await self.resolveMenuExtraOwnerPID(openExtra)
} else {
nil
}
let titles = [
openExtra?.title,
openExtra?.ownerName,
openExtra?.rawTitle,
appHint,
].compactMap { $0?.trimmingCharacters(in: .whitespacesAndNewlines) }
for candidate in titles where !candidate.isEmpty {
if let frame = try? await self.services.menu.menuExtraOpenMenuFrame(
title: candidate,
ownerPID: ownerPID
),
let capture = try await self.captureMenuBarPopoverByFrame(
frame,
hint: appHint ?? openExtra?.title,
ownerHint: openExtra?.ownerName
) {
return capture
}
}
return nil
}
func ocrElements(imageData: Data, windowBounds: CGRect?) throws -> [DetectedElement] {
guard let windowBounds else { return [] }
let result = try OCRService().recognizeText(in: imageData)
return ObservationOCRMapper.elements(from: result, windowBounds: windowBounds)
}
private func captureMenuBarPopoverByFrame(
_ frame: CGRect,
hint: String?,
ownerHint: String?
) async throws -> MenuBarPopoverCapture? {
let selector = self.menuBarPopoverOCRSelector()
if let match = try await selector.matchFrame(
frame,
hints: MenuBarPopoverResolverContext.normalizedHints([hint, ownerHint])
) {
self.logger.verbose(
"Selected menu bar popover via AX menu frame",
category: "Capture",
metadata: [
"rect": "\(match.bounds)"
]
)
return MenuBarPopoverCapture(
captureResult: match.captureResult,
windowBounds: match.bounds,
windowId: nil
)
}
return nil
}
private func menuBarPopoverOCRSelector() -> ObservationMenuBarPopoverOCRSelector {
ObservationMenuBarPopoverOCRSelector(
screenCapture: self.services.screenCapture,
screens: self.services.screens.listScreens()
)
}
}

View File

@ -1,181 +0,0 @@
import Commander
import CoreGraphics
import Foundation
import PeekabooAutomationKit
import PeekabooCore
@available(macOS 14.0, *)
@MainActor
extension SeeCommand {
func determineMode() -> PeekabooCore.CaptureMode {
if let mode = self.mode {
mode
} else if self.app != nil || self.pid != nil || self.windowTitle != nil || self.windowId != nil {
.window
} else {
.frontmost
}
}
func observationTargetForCaptureWithDetectionIfPossible() throws -> DesktopObservationTargetRequest? {
if self.menubar {
let hint = self.menuBarAppHint()
return .menubarPopover(
hints: MenuBarPopoverResolverContext.normalizedHints([hint]),
openIfNeeded: MenuBarPopoverOpenOptions(clickHint: hint)
)
}
switch self.determineMode() {
case .window:
if let windowId {
return .windowID(CGWindowID(windowId))
}
if let appValue = self.app?.lowercased() {
switch appValue {
case "menubar":
return .menubar
case "frontmost":
return .frontmost
default:
break
}
}
if let pid = try self.resolveExplicitPIDObservationTarget() {
return .pid(pid, window: self.seeWindowSelection)
}
if self.app != nil || self.pid != nil {
return try .app(identifier: self.resolveApplicationIdentifier(), window: self.seeWindowSelection)
}
throw ValidationError("Provide --window-id, or --app/--pid for window mode")
case .frontmost:
return .frontmost
case .screen:
if let screenIndex {
return .screen(index: screenIndex)
}
if self.analyze != nil {
return .screen(index: 0)
}
return nil
case .multi:
return nil
case .area:
throw ValidationError(
"Area capture mode is not supported by `see`; use `image --mode area --region x,y,width,height` " +
"or a window/screen target."
)
}
}
func makeObservationRequest(
target: DesktopObservationTargetRequest,
snapshotID: String? = nil
) -> DesktopObservationRequest {
DesktopObservationRequest(
target: target,
capture: DesktopCaptureOptions(
engine: self.observationCaptureEnginePreference,
scale: .logical1x,
visualizerMode: .screenshotFlash
),
detection: self.observationDetectionOptions(for: target),
output: DesktopObservationOutputOptions(
path: self.screenshotOutputPath(snapshotID: snapshotID),
saveRawScreenshot: true,
saveAnnotatedScreenshot: self.annotate && self.allowsAnnotation(for: target),
saveSnapshot: true,
snapshotID: snapshotID
)
)
}
func observationTargetDescription(_ target: DesktopObservationTargetRequest) -> String {
switch target {
case let .screen(index):
"screen:\(index.map(String.init) ?? "primary")"
case .allScreens:
"all-screens"
case .frontmost:
"frontmost"
case let .app(identifier, _):
"app:\(identifier)"
case let .pid(pid, _):
"pid:\(pid)"
case let .windowID(windowID):
"window-id:\(windowID)"
case let .area(rect):
"area:\(Int(rect.origin.x)),\(Int(rect.origin.y)),\(Int(rect.width))x\(Int(rect.height))"
case .menubar:
"menubar"
case .menubarPopover:
"menubar-popover"
}
}
private var seeWindowSelection: WindowSelection {
if let windowTitle {
return .title(windowTitle)
}
return .automatic
}
func allowsAnnotation(for target: DesktopObservationTargetRequest) -> Bool {
switch target {
case .screen, .allScreens, .menubar:
false
default:
true
}
}
private func observationDetectionOptions(for target: DesktopObservationTargetRequest) -> DesktopDetectionOptions {
switch target {
case .menubarPopover:
DesktopDetectionOptions(
mode: .none,
allowWebFocusFallback: false,
preferOCR: true,
traversalBudget: self.axTraversalBudget()
)
default:
DesktopDetectionOptions(
mode: .accessibility,
allowWebFocusFallback: !self.noWebFocus,
traversalBudget: self.axTraversalBudget()
)
}
}
func axTraversalBudget() -> AXTraversalBudget {
AXTraversalBudget.resolved(
maxDepth: self.validatedTraversalLimit(self.maxDepth, option: "--max-depth"),
maxElementCount: self.validatedTraversalLimit(self.maxElements, option: "--max-elements"),
maxChildrenPerNode: self.validatedTraversalLimit(self.maxChildren, option: "--max-children")
)
}
private func validatedTraversalLimit(_ value: Int?, option: String) -> Int? {
guard let value else { return nil }
guard value > 0 else {
self.logger.warn("\(option) must be positive; using default AX traversal budget")
return nil
}
return value
}
private var observationCaptureEnginePreference: CaptureEnginePreference {
ObservationCommandSupport.captureEnginePreference(
cliValue: self.captureEngine,
configuredValue: self.configuredCaptureEnginePreference
)
}
}

View File

@ -1,186 +0,0 @@
import Foundation
import PeekabooCore
@available(macOS 14.0, *)
@MainActor
extension SeeCommand {
func renderResults(context: SeeCommandRenderContext) throws {
try Task.checkCancellation()
if self.jsonOutput {
try self.outputJSONResults(context: context)
} else {
try self.outputTextResults(context: context)
}
}
/// Fetches the menu bar summary only when verbose output is requested, with a short timeout.
func fetchMenuBarSummaryIfEnabled() async -> MenuBarSummary? {
guard self.verbose else { return nil }
do {
return try await Self.withWallClockTimeout(seconds: 2.5) {
try Task.checkCancellation()
return await self.getMenuBarItemsSummary()
}
} catch {
self.logger.debug(
"Skipping menu bar summary",
category: "Menu",
metadata: ["reason": error.localizedDescription]
)
return nil
}
}
/// Drives the deadline independently while the MainActor operation is suspended.
/// Synchronous MainActor calls cannot be preempted.
static func withWallClockTimeout<T: Sendable>(
seconds: TimeInterval,
timeoutErrorSeconds: TimeInterval? = nil,
interactionMutationTracker: InteractionMutationTracker? = nil,
operation: @escaping @MainActor @Sendable () async throws -> T
) async throws -> T {
try await withMainActorCommandTimeout(
seconds: seconds,
operationName: "see",
timeoutError: { CaptureError.detectionTimedOut(timeoutErrorSeconds ?? seconds) },
interactionMutationTracker: interactionMutationTracker,
operation: { try await operation() }
)
}
func performAnalysisDetailed(imagePath: String, prompt: String) async throws -> SeeAnalysisData {
let ai = PeekabooAIService()
let res = try await ai.analyzeImageFileDetailed(at: imagePath, question: prompt, model: nil)
return SeeAnalysisData(provider: res.provider, model: res.model, text: res.text)
}
private func outputJSONResults(context: SeeCommandRenderContext) throws {
let uiElements: [UIElementSummary] = context.elements.all.map { element in
UIElementSummary(
id: element.id,
role: element.type.rawValue,
title: element.attributes["title"],
label: element.label,
description: element.attributes["description"],
role_description: element.attributes["roleDescription"],
help: element.attributes["help"],
identifier: element.attributes["identifier"],
bounds: UIElementBounds(element.bounds),
is_actionable: element.isEnabled,
keyboard_shortcut: element.attributes["keyboardShortcut"]
)
}
let snapshotPaths = self.snapshotPaths(for: context)
let output = SeeResult(
snapshot_id: context.snapshotId,
screenshot_raw: snapshotPaths.raw,
screenshot_annotated: snapshotPaths.annotated,
ui_map: snapshotPaths.map,
application_name: context.metadata.windowContext?.applicationName,
window_title: context.metadata.windowContext?.windowTitle,
is_dialog: context.metadata.isDialog,
element_count: context.metadata.elementCount,
interactable_count: context.elements.all.count { $0.isEnabled },
capture_mode: self.determineMode().rawValue,
analysis: context.analysis,
execution_time: context.executionTime,
ui_elements: uiElements,
menu_bar: context.menuBar,
truncation: SeeTruncationSummary(metadata: context.metadata),
observation: context.observation
)
outputSuccessCodable(data: output, logger: self.outputLogger)
}
private func getMenuBarItemsSummary() async -> MenuBarSummary {
var menuExtras: [MenuExtraInfo] = []
do {
menuExtras = try await self.services.menu.listMenuExtras()
} catch {
menuExtras = []
}
let menus = menuExtras.map { extra in
MenuBarSummary.MenuSummary(
title: extra.title,
item_count: 1,
enabled: true,
items: [
MenuBarSummary.MenuItemSummary(
title: extra.title,
enabled: true,
keyboard_shortcut: nil
)
]
)
}
return MenuBarSummary(menus: menus)
}
private func outputTextResults(context: SeeCommandRenderContext) throws {
try Task.checkCancellation()
print("🖼️ Screenshot saved to: \(context.screenshotPath)")
if let annotatedPath = context.annotatedPath {
print("📝 Annotated screenshot: \(annotatedPath)")
}
if let appName = context.metadata.windowContext?.applicationName {
print("📱 Application: \(appName)")
}
if let windowTitle = context.metadata.windowContext?.windowTitle {
let windowType = context.metadata.isDialog ? "Dialog" : "Window"
let icon = context.metadata.isDialog ? "🗨️" : "[win]"
print("\(icon) \(windowType): \(windowTitle)")
}
print("🧊 Detection method: \(context.metadata.method)")
print("📊 UI elements detected: \(context.metadata.elementCount)")
print("⚙️ Interactable elements: \(context.elements.all.count { $0.isEnabled })")
if let truncationInfo = context.metadata.truncationInfo, truncationInfo.isTruncated {
print("⚠️ \(truncationInfo.remediationMessage(budget: context.metadata.windowContext?.traversalBudget))")
}
let formattedDuration = String(format: "%.2f", context.executionTime)
print("⏱️ Execution time: \(formattedDuration)s")
if let analysis = context.analysis {
print("\n🤖 AI Analysis\n\(analysis.text)")
}
if context.metadata.elementCount > 0 {
print("\n🔍 Element Summary")
for element in context.elements.all.prefix(10) {
let summaryLabel = element.label ?? element.attributes["title"] ?? element.value ?? "Untitled"
print("\(element.id) (\(element.type.rawValue)) - \(summaryLabel)")
}
if context.metadata.elementCount > 10 {
print(" ...and \(context.metadata.elementCount - 10) more elements")
}
}
if self.annotate, context.annotatedPath != nil {
print("\n📝 Annotated screenshot created")
}
print("\nSnapshot ID: \(context.snapshotId)")
let terminalCapabilities = TerminalDetector.detectCapabilities()
if terminalCapabilities.recommendedOutputMode == .minimal {
print("Agent: Use a tool like view_image to inspect it.")
}
}
private func snapshotPaths(for context: SeeCommandRenderContext) -> SnapshotPaths {
let publishesScreenshotPaths = !self.usesTemporaryScreenshotOutput
return SnapshotPaths(
raw: publishesScreenshotPaths ? context.screenshotPath : "",
annotated: publishesScreenshotPaths ? context.annotatedPath ?? "" : "",
map: self.services.snapshots.getSnapshotStoragePath() + "/\(context.snapshotId)/snapshot.json"
)
}
}

View File

@ -1,160 +0,0 @@
import Algorithms
import Foundation
import PeekabooCore
@MainActor
extension SeeCommand {
func performScreenCapture() async throws -> CaptureResult {
if self.annotate {
self.logger.info("Annotation is disabled for full screen captures due to performance constraints")
}
self.logger.verbose("Initiating screen capture", category: "Capture")
self.logger.startTimer("screen_capture")
defer {
self.logger.stopTimer("screen_capture")
}
if let index = self.screenIndex ?? (self.analyze != nil ? 0 : nil) {
self.logger.verbose("Capturing specific screen", category: "Capture", metadata: ["screenIndex": index])
let result = try await self.services.screenCapture.captureScreen(displayIndex: index)
if !self.jsonOutput, let displayInfo = result.metadata.displayInfo {
self.printScreenDisplayInfo(index: index, displayInfo: displayInfo)
}
self.logger.verbose("Screen capture completed", category: "Capture", metadata: [
"mode": "screen-index",
"screenIndex": index,
"imageBytes": result.imageData.count
])
return result
}
self.logger.verbose("Capturing all screens", category: "Capture")
let results = try await self.captureAllScreens()
if results.isEmpty {
throw CaptureError.captureFailure("Failed to capture any screens")
}
if !self.jsonOutput {
print("📸 Captured \(results.count) screen(s):")
}
for (index, result) in results.indexed() {
if index > 0 {
let screenPath = self.screenOutputPath(for: index)
try result.imageData.write(to: URL(fileURLWithPath: screenPath))
if !self.jsonOutput, let displayInfo = result.metadata.displayInfo {
let fileSize = self.getFileSize(screenPath) ?? 0
let suffix = "\(screenPath) (\(self.formatFileSize(Int64(fileSize))))"
self.printScreenDisplayInfo(
index: index,
displayInfo: displayInfo,
indent: " ",
suffix: suffix
)
}
} else if !self.jsonOutput, let displayInfo = result.metadata.displayInfo {
self.printScreenDisplayInfo(
index: index,
displayInfo: displayInfo,
indent: " ",
suffix: "(primary)"
)
}
}
self.logger.verbose("Multi-screen capture completed", category: "Capture", metadata: [
"count": results.count,
"primaryBytes": results.first?.imageData.count ?? 0
])
return results[0]
}
func captureAllScreens() async throws -> [CaptureResult] {
var results: [CaptureResult] = []
let displays = self.services.screens.listScreens()
self.logger.info("Found \(displays.count) display(s) to capture")
for display in displays {
self.logger.verbose("Capturing display \(display.index)", category: "MultiScreen", metadata: [
"displayID": display.displayID,
"width": display.frame.width,
"height": display.frame.height
])
do {
let result = try await self.services.screenCapture.captureScreen(displayIndex: display.index)
results.append(result)
} catch {
self.logger.error("Failed to capture display \(display.index): \(error)")
// Continue capturing other screens even if one fails
}
}
if results.isEmpty {
throw CaptureError.captureFailure("Failed to capture any screens")
}
return results
}
func formatFileSize(_ bytes: Int64) -> String {
let formatter = ByteCountFormatter()
formatter.countStyle = .file
return formatter.string(fromByteCount: bytes)
}
private func screenOutputPath(for index: Int) -> String {
if let basePath = self.path {
let expanded = (basePath as NSString).expandingTildeInPath
if ObservationOutputPathResolver.isDirectoryLike(expanded) {
return URL(fileURLWithPath: expanded, isDirectory: true)
.appendingPathComponent(self.defaultScreenOutputFilename(for: index))
.path
}
let directory = (expanded as NSString).deletingLastPathComponent
let filename = (expanded as NSString).lastPathComponent
let nameWithoutExt = (filename as NSString).deletingPathExtension
let ext = (filename as NSString).pathExtension
let fileExtension = ext.isEmpty ? "png" : ext
return (directory as NSString)
.appendingPathComponent("\(nameWithoutExt)_screen\(index).\(fileExtension)")
}
return self.defaultScreenOutputFilename(for: index)
}
private func defaultScreenOutputFilename(for index: Int) -> String {
let timestamp = ISO8601DateFormatter().string(from: Date())
return "screenshot_\(timestamp)_screen\(index).png"
}
private func screenDisplayBaseText(index: Int, displayInfo: DisplayInfo) -> String {
let displayName = displayInfo.name ?? "Display \(index)"
let bounds = displayInfo.bounds
let resolution = "(\(Int(bounds.width))×\(Int(bounds.height)))"
return "[scrn] Display \(index): \(displayName) \(resolution)"
}
private func printScreenDisplayInfo(
index: Int,
displayInfo: DisplayInfo,
indent: String = "",
suffix: String? = nil
) {
var line = self.screenDisplayBaseText(index: index, displayInfo: displayInfo)
if let suffix {
line += "\(suffix)"
}
print("\(indent)\(line)")
}
}

View File

@ -1,240 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
struct CaptureContext {
let captureResult: CaptureResult
let captureBounds: CGRect?
let prefersOCR: Bool
let ocrMethod: String?
let windowIdOverride: Int?
}
struct MenuBarPopoverCapture {
let captureResult: CaptureResult
let windowBounds: CGRect
let windowId: Int?
}
struct CaptureAndDetectionResult {
let snapshotId: String
let screenshotPath: String
let annotatedPath: String?
let elements: DetectedElements
let metadata: DetectionMetadata
let observation: SeeObservationDiagnostics?
}
struct SnapshotPaths {
let raw: String
let annotated: String
let map: String
}
struct SeeCommandRenderContext {
let snapshotId: String
let screenshotPath: String
let annotatedPath: String?
let metadata: DetectionMetadata
let elements: DetectedElements
let analysis: SeeAnalysisData?
let executionTime: TimeInterval
let observation: SeeObservationDiagnostics?
let menuBar: MenuBarSummary?
}
struct UIElementSummary: Codable {
let id: String
let role: String
let title: String?
let label: String?
let description: String?
let role_description: String?
let help: String?
let identifier: String?
let bounds: UIElementBounds
let is_actionable: Bool
let keyboard_shortcut: String?
}
struct UIElementBounds: Codable {
let x: Double
let y: Double
let width: Double
let height: Double
init(_ rect: CGRect) {
self.x = rect.origin.x
self.y = rect.origin.y
self.width = rect.size.width
self.height = rect.size.height
}
}
struct SeeAnalysisData: Codable {
let provider: String
let model: String
let text: String
}
struct SeeObservationDiagnostics: Codable {
let spans: [SeeObservationSpan]
let warnings: [String]
let state_snapshot: SeeDesktopStateSnapshotSummary?
let target: SeeObservationTargetDiagnostics?
init(timings: ObservationTimings, diagnostics: DesktopObservationDiagnostics) {
self.spans = timings.spans.map(SeeObservationSpan.init)
self.warnings = diagnostics.warnings
self.state_snapshot = diagnostics.stateSnapshot.map(SeeDesktopStateSnapshotSummary.init)
self.target = diagnostics.target.map(SeeObservationTargetDiagnostics.init)
}
}
struct SeeObservationTargetDiagnostics: Codable {
let requested_kind: String
let resolved_kind: String
let source: String
let hints: [String]
let open_if_needed: Bool
let click_hint: String?
let window_id: Int?
let bounds: CGRect?
let capture_scale_hint: CGFloat?
init(_ diagnostics: DesktopObservationTargetDiagnostics) {
self.requested_kind = diagnostics.requestedKind
self.resolved_kind = diagnostics.resolvedKind
self.source = diagnostics.source
self.hints = diagnostics.hints
self.open_if_needed = diagnostics.openIfNeeded
self.click_hint = diagnostics.clickHint
self.window_id = diagnostics.windowID
self.bounds = diagnostics.bounds
self.capture_scale_hint = diagnostics.captureScaleHint
}
}
struct SeeObservationSpan: Codable {
let name: String
let duration_ms: Double
let metadata: [String: String]
init(_ span: ObservationSpan) {
self.name = span.name
self.duration_ms = span.durationMS
self.metadata = span.metadata
}
}
struct SeeDesktopStateSnapshotSummary: Codable {
let display_count: Int
let running_application_count: Int
let window_count: Int
let frontmost_application_name: String?
let frontmost_bundle_identifier: String?
let frontmost_window_title: String?
let frontmost_window_id: Int?
init(_ summary: DesktopStateSnapshotSummary) {
self.display_count = summary.displayCount
self.running_application_count = summary.runningApplicationCount
self.window_count = summary.windowCount
self.frontmost_application_name = summary.frontmostApplication?.name
self.frontmost_bundle_identifier = summary.frontmostApplication?.bundleIdentifier
self.frontmost_window_title = summary.frontmostWindow?.title
self.frontmost_window_id = summary.frontmostWindow?.windowID
}
}
struct SeeTruncationSummary: Codable {
let max_depth_reached: Bool
let max_element_count_reached: Bool
let max_children_per_node_reached: Bool
let warning: String
init?(metadata: DetectionMetadata) {
guard let truncationInfo = metadata.truncationInfo, truncationInfo.isTruncated else {
return nil
}
self.max_depth_reached = truncationInfo.maxDepthReached
self.max_element_count_reached = truncationInfo.maxElementCountReached
self.max_children_per_node_reached = truncationInfo.maxChildrenPerNodeReached
self.warning = truncationInfo.remediationMessage(budget: metadata.windowContext?.traversalBudget)
}
}
struct SeeResult: Codable {
let snapshot_id: String
let screenshot_raw: String
let screenshot_annotated: String
let ui_map: String
let application_name: String?
let window_title: String?
let is_dialog: Bool
let element_count: Int
let interactable_count: Int
let capture_mode: String
let analysis: SeeAnalysisData?
let execution_time: TimeInterval
let ui_elements: [UIElementSummary]
let truncation: SeeTruncationSummary?
let menu_bar: MenuBarSummary?
let observation: SeeObservationDiagnostics?
var success: Bool = true
init(
snapshot_id: String,
screenshot_raw: String,
screenshot_annotated: String,
ui_map: String,
application_name: String?,
window_title: String?,
is_dialog: Bool,
element_count: Int,
interactable_count: Int,
capture_mode: String,
analysis: SeeAnalysisData?,
execution_time: TimeInterval,
ui_elements: [UIElementSummary],
menu_bar: MenuBarSummary?,
truncation: SeeTruncationSummary? = nil,
observation: SeeObservationDiagnostics? = nil,
success: Bool = true
) {
self.snapshot_id = snapshot_id
self.screenshot_raw = screenshot_raw
self.screenshot_annotated = screenshot_annotated
self.ui_map = ui_map
self.application_name = application_name
self.window_title = window_title
self.is_dialog = is_dialog
self.element_count = element_count
self.interactable_count = interactable_count
self.capture_mode = capture_mode
self.analysis = analysis
self.execution_time = execution_time
self.ui_elements = ui_elements
self.truncation = truncation
self.menu_bar = menu_bar
self.observation = observation
self.success = success
}
}
struct MenuBarSummary: Codable {
let menus: [MenuSummary]
struct MenuSummary: Codable {
let title: String
let item_count: Int
let enabled: Bool
let items: [MenuItemSummary]
}
struct MenuItemSummary: Codable {
let title: String
let enabled: Bool
let keyboard_shortcut: String?
}
}

View File

@ -1,426 +0,0 @@
import Commander
import Foundation
import PeekabooCore
import PeekabooFoundation
/// Capture a screenshot and build an interactive UI map
@available(macOS 14.0, *)
struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsConfigurable {
@Option(help: "Application name to capture, or special values: 'menubar', 'frontmost'")
var app: String?
@Option(name: .long, help: "Target application by process ID")
var pid: Int32?
@Option(help: "Specific window title to capture")
var windowTitle: String?
@Option(
name: .long,
help: "Target window by CoreGraphics window id (window_id from `peekaboo window list --json`)"
)
var windowId: Int?
@Option(help: "Capture mode (screen, window, frontmost)")
var mode: PeekabooCore.CaptureMode?
@Option(
names: [.automatic, .customLong("save"), .customLong("output"), .customShort("o", allowingJoined: false)],
help: "Output path for screenshot (aliases: --save, --output, -o)"
)
var path: String?
@Option(
name: .long,
help: "Specific screen index to capture (0-based). If not specified, captures all screens when in screen mode"
)
var screenIndex: Int?
@Flag(help: "Generate annotated screenshot with interaction markers")
var annotate = false
@Flag(name: .long, help: "Capture menu bar popovers via window list + OCR")
var menubar = false
@Option(help: "Analyze captured content with AI")
var analyze: String?
@Option(
name: .long,
help: """
Overall timeout in seconds (default: 20, or 60 when --analyze is set).
Increase this if element detection regularly times out for large/complex windows.
"""
)
var timeoutSeconds: Int?
@Option(
name: .long,
help: """
Capture engine: auto|modern|sckit|classic|cg (default: auto).
modern/sckit force ScreenCaptureKit; classic/cg force CGWindowList;
auto tries CGWindowList then falls back when allowed.
"""
)
var captureEngine: String?
@Flag(help: "Skip web-content focus fallback when no text fields are detected")
var noWebFocus = false
@Option(name: .long, help: "Maximum AX traversal depth (env: PEEKABOO_AX_MAX_DEPTH)")
var maxDepth: Int?
@Option(name: .long, help: "Maximum AX elements to collect (env: PEEKABOO_AX_MAX_ELEMENTS)")
var maxElements: Int?
@Option(name: .long, help: "Maximum AX children per node (env: PEEKABOO_AX_MAX_CHILDREN)")
var maxChildren: Int?
@RuntimeStorage private var runtime: CommandRuntime?
var runtimeOptions = CommandRuntimeOptions()
var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
var jsonOutput: Bool {
self.runtime?.configuration.jsonOutput ?? self.runtimeOptions.jsonOutput
}
var verbose: Bool {
self.runtime?.configuration.verbose ?? self.runtimeOptions.verbose
}
var logger: Logger {
self.resolvedRuntime.logger
}
var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
var outputLogger: Logger {
self.logger
}
var configuredCaptureEnginePreference: String? {
self.runtime?.configuration.captureEnginePreference
}
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let commandStartedAt = Date()
let logger = self.logger
let overallTimeout = TimeInterval(self.timeoutSeconds ?? ((self.analyze == nil) ? 20 : 60))
let mutationCoordinator = runtime.toolSnapshotMutationCoordinator
let snapshotManager = runtime.services.snapshots
logger.operationStart("see_command", metadata: [
"app": self.app ?? "none",
"mode": self.mode?.rawValue ?? "auto",
"annotate": self.annotate,
"menubar": self.menubar,
"hasAnalyzePrompt": self.analyze != nil,
])
let commandCopy = self
do {
runtime.beginInteractionMutation(preservingSnapshotsCreatedAfterBoundary: true)
try await CrossProcessOperationGate.withExclusiveOperation(
named: CrossProcessOperationGate.desktopObservationName
) {
let observationStartedAt = Date()
let observationDeadline = observationStartedAt.addingTimeInterval(overallTimeout)
let scope = MCPToolSnapshotMutationScope(
toolName: "see",
startedAt: observationStartedAt,
effect: .mutationProducingFreshObservation
)
let reservationTimeout = try Self.remainingObservationTimeout(
until: observationDeadline,
overallTimeout: overallTimeout
)
let snapshotID = try await Self.withWallClockTimeout(
seconds: reservationTimeout,
timeoutErrorSeconds: overallTimeout
) {
try await snapshotManager.createSnapshot(pendingAt: observationStartedAt)
}
defer {
if snapshotManager.copiesScreenshotArtifactsIntoStorage {
commandCopy.cleanupTemporaryScreenshotOutput(snapshotID: snapshotID)
}
}
var observationCompleted = false
do {
let preparationTimeout = try Self.remainingObservationTimeout(
until: observationDeadline,
overallTimeout: overallTimeout
)
let context = try await Self.withWallClockTimeout(
seconds: preparationTimeout,
timeoutErrorSeconds: overallTimeout,
interactionMutationTracker: runtime.observationTimeoutMutationTracker
) {
try await commandCopy.prepareResult(
startTime: commandStartedAt,
logger: logger,
snapshotID: snapshotID
)
}
observationCompleted = true
let publicationTimeout = try Self.remainingObservationTimeout(
until: observationDeadline,
overallTimeout: overallTimeout
)
let published = try await Self.withWallClockTimeout(
seconds: publicationTimeout,
timeoutErrorSeconds: overallTimeout
) {
await mutationCoordinator.completeMutation(
scope.completed(
at: Date(),
preserving: snapshotID,
confirmedMutationCompletedAt: context.metadata.desktopMutationCompletedAt,
observationPreservationAllowed: context.metadata
.desktopMutationPreservationAllowed
),
succeeded: true
)
}
guard published else {
throw PeekabooError.operationError(
message: "Failed to publish the refreshed UI snapshot"
)
}
try Task.checkCancellation()
try commandCopy.renderResults(context: context)
commandCopy.emitAnnotationStatus(context: context)
logger.operationComplete("see_command", metadata: [
"executionTimeMs": Int(Date().timeIntervalSince(commandStartedAt) * 1000),
"success": true,
])
} catch {
if observationCompleted || !PendingSnapshotCleanupPolicy.shouldPreserveReservation(after: error) {
try? await self.services.snapshots.cleanSnapshot(snapshotId: snapshotID)
}
_ = await mutationCoordinator.completeMutation(
scope.completed(at: Date(), preserving: nil),
succeeded: false
)
throw error
}
}
} catch {
logger.operationComplete(
"see_command",
success: false,
metadata: [
"error": error.localizedDescription,
]
)
self.handleError(error)
throw ExitCode.failure
}
}
private static func remainingObservationTimeout(
until deadline: Date,
overallTimeout: TimeInterval
) throws -> TimeInterval {
let remaining = deadline.timeIntervalSinceNow
guard remaining > 0 else {
throw CaptureError.detectionTimedOut(overallTimeout)
}
return remaining
}
private func prepareResult(
startTime: Date,
logger: Logger,
snapshotID: String
) async throws -> SeeCommandRenderContext {
// ScreenCaptureService performs the authoritative permission check inside each capture path.
// Avoid duplicating that TCC probe here; `see` is often called in latency-sensitive loops.
// Perform capture and element detection
logger.verbose("Starting capture and detection phase", category: "Capture")
let captureResult = try await performCaptureWithDetection(snapshotID: snapshotID)
try Task.checkCancellation()
logger.verbose("Capture completed successfully", category: "Capture", metadata: [
"snapshotId": captureResult.snapshotId,
"elementCount": captureResult.elements.all.count,
"screenshotSize": self.getFileSize(captureResult.screenshotPath) ?? 0,
])
// Generate annotated screenshot if requested
var annotatedPath = captureResult.annotatedPath
let annotationsAllowed = self.allowsAnnotationForCurrentCapture()
if self.annotate, !annotationsAllowed {
self.logger.info("Annotation is disabled for full screen captures due to performance constraints")
}
if self.annotate, annotatedPath == nil, annotationsAllowed {
logger.operationStart("generate_annotations")
annotatedPath = try await self.generateAnnotatedScreenshot(
snapshotId: captureResult.snapshotId,
originalPath: captureResult.screenshotPath
)
try Task.checkCancellation()
if let annotatedPath,
annotatedPath != captureResult.screenshotPath {
try await self.services.snapshots.storeAnnotatedScreenshot(
snapshotId: captureResult.snapshotId,
annotatedScreenshotPath: annotatedPath
)
try Task.checkCancellation()
}
logger.operationComplete("generate_annotations", metadata: [
"annotatedPath": annotatedPath ?? "none",
])
}
// Perform AI analysis if requested
var analysisResult: SeeAnalysisData?
if let prompt = analyze {
// Pre-analysis diagnostics
let fileSize = (try? FileManager.default
.attributesOfItem(atPath: captureResult.screenshotPath)[.size] as? Int) ?? 0
logger.verbose(
"Starting AI analysis",
category: "AI",
metadata: [
"imagePath": captureResult.screenshotPath,
"imageSizeBytes": fileSize,
"promptLength": prompt.count
]
)
logger.operationStart("ai_analysis", metadata: ["promptPreview": String(prompt.prefix(80))])
logger.startTimer("ai_generate")
analysisResult = try await self.performAnalysisDetailed(
imagePath: captureResult.screenshotPath,
prompt: prompt
)
try Task.checkCancellation()
logger.stopTimer("ai_generate")
logger.operationComplete(
"ai_analysis",
success: analysisResult != nil,
metadata: [
"provider": analysisResult?.provider ?? "unknown",
"model": analysisResult?.model ?? "unknown"
]
)
}
let menuBarSummary = self.jsonOutput ? await self.fetchMenuBarSummaryIfEnabled() : nil
try Task.checkCancellation()
let executionTime = Date().timeIntervalSince(startTime)
return SeeCommandRenderContext(
snapshotId: captureResult.snapshotId,
screenshotPath: captureResult.screenshotPath,
annotatedPath: annotatedPath,
metadata: captureResult.metadata,
elements: captureResult.elements,
analysis: analysisResult,
executionTime: executionTime,
observation: captureResult.observation,
menuBar: menuBarSummary
)
}
private func emitAnnotationStatus(context: SeeCommandRenderContext) {
let annotationsAllowed = self.allowsAnnotationForCurrentCapture()
if self.annotate, annotationsAllowed, context.annotatedPath == nil, !self.jsonOutput {
print("\(AgentDisplayTokens.Status.warning) No interactive UI elements found to annotate")
} else if self.annotate, annotationsAllowed, let annotatedPath = context.annotatedPath, !self.jsonOutput {
let interactableElements = context.elements.all.filter(\.isEnabled)
print("📝 Created annotated screenshot with \(interactableElements.count) interactive elements")
self.logger.verbose("Annotated screenshot path: \(annotatedPath)")
}
}
func getFileSize(_ path: String) -> Int? {
try? FileManager.default.attributesOfItem(atPath: path)[.size] as? Int
}
func allowsAnnotationForCurrentCapture() -> Bool {
if self.app?.lowercased() == "menubar" {
return false
}
return switch self.determineMode() {
case .screen, .multi:
false
case .window, .frontmost:
true
case .area:
false
}
}
}
@MainActor
extension SeeCommand: ParsableCommand {
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
let definition = VisionToolDefinitions.see.commandConfiguration
return CommandDescription(
commandName: definition.commandName,
abstract: definition.abstract,
discussion: definition.discussion,
usageExamples: [
CommandUsageExample(
command: "peekaboo see --json --annotate --path /tmp/see.png",
description: "Capture the frontmost window, print structured output, and save annotations."
),
CommandUsageExample(
command: "peekaboo see --app Safari --window-title \"Login\" --json --path /tmp/safari-login.png",
description: "Target a specific Safari window to collect fresh element IDs and keep the capture artifact in /tmp."
),
CommandUsageExample(
command: "peekaboo see --mode screen --screen-index 0 --analyze 'Summarize the dashboard'",
description: "Capture a display and immediately send it to the configured AI provider."
)
],
showHelpOnEmptyInvocation: true
)
}
}
}
extension SeeCommand: AsyncRuntimeCommand {}
@MainActor
extension SeeCommand: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {
self.app = values.singleOption("app")
self.pid = try values.decodeOption("pid", as: Int32.self)
self.windowTitle = values.singleOption("windowTitle")
self.windowId = try values.decodeOption("windowId", as: Int.self)
if let parsedMode: PeekabooCore.CaptureMode = try values.decodeOptionEnum("mode", caseInsensitive: false) {
guard parsedMode != .area else {
throw CommanderBindingError.invalidArgument(
label: "mode",
value: parsedMode.rawValue,
reason: "`see` supports screen, window, frontmost, or multi"
)
}
self.mode = parsedMode
}
self.path = values.singleOption("path")
self.screenIndex = try values.decodeOption("screenIndex", as: Int.self)
self.captureEngine = values.singleOption("captureEngine")
self.annotate = values.flag("annotate")
self.analyze = values.singleOption("analyze")
self.timeoutSeconds = try values.decodeOption("timeoutSeconds", as: Int.self)
self.noWebFocus = values.flag("noWebFocus")
self.menubar = values.flag("menubar")
}
}

View File

@ -1,360 +0,0 @@
import Commander
import CoreGraphics
import Foundation
import PeekabooCore
import PeekabooFoundation
extension PermissionCommand {
struct RequestScreenRecordingSubcommand: OutputFormattable {
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
CommandDescription(
commandName: "request-screen-recording",
abstract: "Trigger screen recording permission prompt"
)
}
}
@RuntimeStorage private var runtime: CommandRuntime?
private var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
private var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
private var logger: Logger {
self.resolvedRuntime.logger
}
var outputLogger: Logger {
self.logger
}
var jsonOutput: Bool {
self.resolvedRuntime.configuration.jsonOutput
}
/// Trigger the screen recording permission prompt using the best available mechanism.
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.prepare(using: runtime)
if await self.renderIfAlreadyGranted() { return }
let result = await PermissionHelpers.performInteractivePermissionRequest(using: runtime) {
await self.requestScreenRecordingPermission()
}
self.render(result: result)
}
private mutating func prepare(using runtime: CommandRuntime) {
self.runtime = runtime
self.logger.setJsonOutputMode(self.jsonOutput)
}
private func renderIfAlreadyGranted() async -> Bool {
let hasPermission = await self.services.screenCapture.hasScreenRecordingPermission()
guard hasPermission else { return false }
let payload = AgentPermissionActionResult(
action: "request-screen-recording",
already_granted: true,
prompt_triggered: false,
granted: true
)
self.render(result: payload)
return true
}
private func requestScreenRecordingPermission() async -> AgentPermissionActionResult {
if !self.jsonOutput {
print("Requesting Screen Recording permission...\n")
print("Triggering permission prompt...\n")
}
if #available(macOS 10.15, *) {
return self.handleModernPrompt()
} else {
return self.handleLegacyPrompt()
}
}
private func handleModernPrompt() -> AgentPermissionActionResult {
let granted = CGRequestScreenCaptureAccess()
if !self.jsonOutput {
self.printModernResult(granted: granted)
}
return AgentPermissionActionResult(
action: "request-screen-recording",
already_granted: false,
prompt_triggered: true,
granted: granted
)
}
private func handleLegacyPrompt() -> AgentPermissionActionResult {
// Minimum supported macOS is 15+, so reuse the modern path.
self.handleModernPrompt()
}
private func printModernResult(granted: Bool) {
guard !self.jsonOutput else { return }
if granted {
print("✅ Screen Recording permission granted!")
return
}
print("❌ Screen Recording permission denied\n")
print("To grant manually:")
print("1. Open System Settings")
print("2. Go to Privacy & Security > Screen Recording")
print("3. Enable Peekaboo")
}
private func render(result: AgentPermissionActionResult) {
if self.jsonOutput {
outputSuccessCodable(data: result, logger: self.logger)
} else if result.already_granted {
print("✅ Screen Recording permission is already granted!")
}
}
}
struct RequestAccessibilitySubcommand: OutputFormattable {
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
CommandDescription(
commandName: "request-accessibility",
abstract: "Request accessibility permission"
)
}
}
@RuntimeStorage private var runtime: CommandRuntime?
private var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
private var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
private var logger: Logger {
self.resolvedRuntime.logger
}
var outputLogger: Logger {
self.logger
}
var jsonOutput: Bool {
self.resolvedRuntime.configuration.jsonOutput
}
/// Prompt the user to grant accessibility permission and open the relevant System Settings pane.
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.prepare(using: runtime)
if await self.renderIfAlreadyGranted() { return }
let granted = await PermissionHelpers.performInteractivePermissionRequest(using: runtime) {
self.promptAccessibilityDialog()
}
self.renderAccessibilityResult(granted: granted)
}
private mutating func prepare(using runtime: CommandRuntime) {
self.runtime = runtime
self.logger.setJsonOutputMode(self.jsonOutput)
}
private func renderIfAlreadyGranted() async -> Bool {
let hasPermission = await AutomationServiceBridge
.hasAccessibilityPermission(automation: self.services.automation)
guard hasPermission else { return false }
let payload = AgentPermissionActionResult(
action: "request-accessibility",
already_granted: true,
prompt_triggered: false,
granted: true
)
self.renderAccessibilityResult(payload: payload)
return true
}
private func promptAccessibilityDialog() -> Bool {
if !self.jsonOutput {
print("Requesting Accessibility permission...\n")
print("Opening System Settings to Accessibility permissions...\n")
}
return self.services.permissions.requestAccessibilityPermission(interactive: true)
}
private func renderAccessibilityResult(granted: Bool) {
let payload = AgentPermissionActionResult(
action: "request-accessibility",
already_granted: false,
prompt_triggered: true,
granted: granted
)
self.renderAccessibilityResult(payload: payload)
}
private func renderAccessibilityResult(payload: AgentPermissionActionResult) {
if self.jsonOutput {
outputSuccessCodable(data: payload, logger: self.logger)
return
}
guard !payload.already_granted else {
print("✅ Accessibility permission is already granted!")
return
}
if payload.granted == true {
print("✅ Accessibility permission granted!")
} else {
print("A dialog should have appeared.\n")
print("To grant permission:")
print("1. Click 'Open System Settings' in the dialog")
print("2. Enable Peekaboo in the Accessibility list")
print("3. You may need to restart Peekaboo after granting")
}
}
}
struct RequestEventSynthesizingSubcommand: ErrorHandlingCommand, OutputFormattable {
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
CommandDescription(
commandName: "request-event-synthesizing",
abstract: "Request event-synthesizing permission for background input"
)
}
}
@RuntimeStorage private var runtime: CommandRuntime?
private var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
private var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
private var logger: Logger {
self.resolvedRuntime.logger
}
var outputLogger: Logger {
self.logger
}
var jsonOutput: Bool {
self.resolvedRuntime.configuration.jsonOutput
}
/// Prompt macOS for event-posting access used by process-targeted hotkeys.
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.prepare(using: runtime)
do {
let payload = try await self.requestEventSynthesizingPermission()
self.renderEventSynthesizingResult(payload: payload)
} catch {
self.handleError(error)
throw ExitCode.failure
}
}
private mutating func prepare(using runtime: CommandRuntime) {
self.runtime = runtime
self.logger.setJsonOutputMode(self.jsonOutput)
}
private func requestEventSynthesizingPermission() async throws -> AgentPermissionActionResult {
let result = try await PermissionHelpers.requestEventSynthesizingPermission(
services: self.services,
runtime: self.resolvedRuntime
)
return AgentPermissionActionResult(
action: result.action,
source: result.source,
already_granted: result.already_granted,
prompt_triggered: result.prompt_triggered,
granted: result.granted
)
}
private func renderEventSynthesizingResult(payload: AgentPermissionActionResult) {
if self.jsonOutput {
outputSuccessCodable(data: payload, logger: self.logger)
return
}
guard !payload.already_granted else {
print("✅ Event Synthesizing permission is already granted!")
return
}
if payload.granted == true {
print("✅ Event Synthesizing permission granted!")
} else {
print("❌ Event Synthesizing permission denied\n")
print("To grant manually:")
print("1. Open System Settings")
print("2. Go to Privacy & Security > Accessibility")
if payload.source == "bridge" {
print("3. Enable the process that showed the prompt")
} else {
print("3. Enable Peekaboo")
}
}
}
}
}
private struct AgentPermissionActionResult: Codable {
let action: String
let source: String?
let already_granted: Bool
let prompt_triggered: Bool
let granted: Bool?
init(
action: String,
source: String? = nil,
already_granted: Bool,
prompt_triggered: Bool,
granted: Bool?
) {
self.action = action
self.source = source
self.already_granted = already_granted
self.prompt_triggered = prompt_triggered
self.granted = granted
}
}
extension PermissionCommand.RequestScreenRecordingSubcommand: ParsableCommand {}
extension PermissionCommand.RequestScreenRecordingSubcommand: AsyncRuntimeCommand {}
extension PermissionCommand.RequestAccessibilitySubcommand: ParsableCommand {}
extension PermissionCommand.RequestAccessibilitySubcommand: AsyncRuntimeCommand {}
extension PermissionCommand.RequestEventSynthesizingSubcommand: ParsableCommand {}
extension PermissionCommand.RequestEventSynthesizingSubcommand: AsyncRuntimeCommand {}

View File

@ -1,120 +0,0 @@
import Commander
import Foundation
import PeekabooCore
import PeekabooFoundation
extension PermissionCommand {
struct StatusSubcommand: OutputFormattable {
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
CommandDescription(
commandName: "status",
abstract: "Check current permission status"
)
}
}
@RuntimeStorage private var runtime: CommandRuntime?
private var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
private var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
private var logger: Logger {
self.resolvedRuntime.logger
}
var outputLogger: Logger {
self.logger
}
var jsonOutput: Bool {
self.resolvedRuntime.configuration.jsonOutput
}
/// Summarize the current permission state for the agent-centric workflow.
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.prepare(using: runtime)
let status = await self.fetchPermissionStatus()
self.render(status: status)
}
private mutating func prepare(using runtime: CommandRuntime) {
self.runtime = runtime
self.logger.setJsonOutputMode(self.jsonOutput)
}
@MainActor
private func fetchPermissionStatus() async -> AgentPermissionStatusPayload {
if let remoteServices = self.services as? RemotePeekabooServices,
let status = try? await remoteServices.permissionsStatus() {
return AgentPermissionStatusPayload(
screen_recording: status.screenRecording,
accessibility: status.accessibility,
event_synthesizing: status.postEvent
)
}
let screenRecording = await self.services.screenCapture.hasScreenRecordingPermission()
let accessibility = await AutomationServiceBridge
.hasAccessibilityPermission(automation: self.services.automation)
let eventSynthesizing = self.services.permissions.checkPostEventPermission()
return AgentPermissionStatusPayload(
screen_recording: screenRecording,
accessibility: accessibility,
event_synthesizing: eventSynthesizing
)
}
private func render(status: AgentPermissionStatusPayload) {
if self.jsonOutput {
outputSuccessCodable(data: status, logger: self.logger)
return
}
print("Peekaboo Permission Status")
print("==========================\n")
self.printStatusLine(label: "Screen Recording", granted: status.screen_recording)
self.printStatusLine(label: "Accessibility", granted: status.accessibility)
self.printStatusLine(label: "Event Synthesizing", granted: status.event_synthesizing)
if !status.screen_recording || !status.accessibility {
print("\nTo grant missing required permissions:")
if !status.screen_recording {
print("- Run: peekaboo agent permission request-screen-recording")
}
if !status.accessibility {
print("- Run: peekaboo agent permission request-accessibility")
}
}
if !status.event_synthesizing {
print("\nOptional for background input:")
print("- Run: peekaboo agent permission request-event-synthesizing")
}
}
private func printStatusLine(label: String, granted: Bool) {
let state = granted ? "✅ Granted" : "❌ Not granted"
print("\(label): \(state)")
}
}
}
private struct AgentPermissionStatusPayload: Codable {
let screen_recording: Bool
let accessibility: Bool
let event_synthesizing: Bool
}
extension PermissionCommand.StatusSubcommand: ParsableCommand {}
extension PermissionCommand.StatusSubcommand: AsyncRuntimeCommand {}

View File

@ -1,32 +0,0 @@
import Commander
/// Manage and request system permissions
struct PermissionCommand: ParsableCommand {
static let commandDescription = CommandDescription(
commandName: "permission",
abstract: "Manage system permissions for Peekaboo",
discussion: """
Request and check system permissions required by Peekaboo.
EXAMPLES:
# Check current permission status
peekaboo agent permission status
# Request screen recording permission
peekaboo agent permission request-screen-recording
# Request accessibility permission
peekaboo agent permission request-accessibility
# Request event-synthesizing permission for background input
peekaboo agent permission request-event-synthesizing
""",
subcommands: [
StatusSubcommand.self,
RequestScreenRecordingSubcommand.self,
RequestAccessibilitySubcommand.self,
RequestEventSynthesizingSubcommand.self
],
defaultSubcommand: StatusSubcommand.self
)
}

View File

@ -1,304 +0,0 @@
import Commander
import Foundation
import PeekabooBridge
import PeekabooCore
import PeekabooFoundation
// MARK: - Error Handling Protocol
/// Protocol for commands that need standardized error handling
@MainActor
protocol ErrorHandlingCommand {
var jsonOutput: Bool { get }
}
extension ErrorHandlingCommand {
/// Handle errors with appropriate output format
func handleError(_ error: any Error, customCode: ErrorCode? = nil) {
if jsonOutput {
let errorCode = customCode ?? self.mapErrorToCode(error)
let logger: Logger = if let formattable = self as? any OutputFormattable {
formattable.outputLogger
} else {
Logger.shared
}
outputError(
message: errorMessage(for: error),
code: errorCode,
details: errorDetails(for: error),
logger: logger
)
} else {
let errorMessage: String = if let peekabooError = error as? PeekabooError {
peekabooError.errorDescription ?? String(describing: error)
} else if let captureError = error as? CaptureError {
captureError.errorDescription ?? String(describing: error)
} else if error
.localizedDescription == "The operation couldn't be completed. (PeekabooCore.PeekabooError error 0.)" ||
error.localizedDescription == "Error" {
String(describing: error)
} else {
error.localizedDescription
}
fputs("Error: \(errorMessage)\n", stderr)
}
}
/// Map various error types to error codes
private func mapErrorToCode(_ error: any Error) -> ErrorCode {
switch error {
case let focusError as FocusError:
self.mapFocusErrorToCode(focusError)
case let peekabooError as PeekabooError:
self.mapPeekabooErrorToCode(peekabooError)
case let captureError as CaptureError:
self.mapCaptureErrorToCode(captureError)
case let observationError as DesktopObservationError:
self.mapObservationErrorToCode(observationError)
case let bridgeError as PeekabooBridgeErrorEnvelope:
errorCode(for: bridgeError)
case let posixError as POSIXError:
errorCode(for: posixError)
case is Commander.ValidationError:
.VALIDATION_ERROR
default:
.INTERNAL_SWIFT_ERROR
}
}
private func mapObservationErrorToCode(_ error: DesktopObservationError) -> ErrorCode {
switch error {
case .targetNotFound:
.WINDOW_NOT_FOUND
case .unsupportedTarget:
.VALIDATION_ERROR
}
}
private func mapPeekabooErrorToCode(_ error: PeekabooError) -> ErrorCode {
if let lookupCode = lookupErrorCode(for: error) {
return lookupCode
}
if let permissionCode = permissionErrorCode(for: error) {
return permissionCode
}
if let timeoutCode = timeoutErrorCode(for: error) {
return timeoutCode
}
if let automationCode = automationErrorCode(for: error) {
return automationCode
}
if let inputCode = inputErrorCode(for: error) {
return inputCode
}
if let credentialCode = credentialErrorCode(for: error) {
return credentialCode
}
return .UNKNOWN_ERROR
}
private func lookupErrorCode(for error: PeekabooError) -> ErrorCode? {
switch error {
case .appNotFound:
.APP_NOT_FOUND
case .ambiguousAppIdentifier:
.AMBIGUOUS_APP_IDENTIFIER
case .windowNotFound:
.WINDOW_NOT_FOUND
case .elementNotFound:
.ELEMENT_NOT_FOUND
case .sessionNotFound:
.SESSION_NOT_FOUND
case .snapshotNotFound:
.SNAPSHOT_NOT_FOUND
case .snapshotStale:
.SNAPSHOT_STALE
case .menuNotFound:
.MENU_BAR_NOT_FOUND
case .menuItemNotFound:
.MENU_ITEM_NOT_FOUND
default:
nil
}
}
private func permissionErrorCode(for error: PeekabooError) -> ErrorCode? {
switch error {
case .permissionDeniedScreenRecording:
.PERMISSION_ERROR_SCREEN_RECORDING
case .permissionDeniedAccessibility:
.PERMISSION_ERROR_ACCESSIBILITY
case .permissionDeniedEventSynthesizing:
.PERMISSION_ERROR_EVENT_SYNTHESIZING
default:
nil
}
}
private func timeoutErrorCode(for error: PeekabooError) -> ErrorCode? {
switch error {
case .captureTimeout, .timeout:
.TIMEOUT
default:
nil
}
}
private func automationErrorCode(for error: PeekabooError) -> ErrorCode? {
switch error {
case .captureFailed, .clickFailed, .typeFailed:
.CAPTURE_FAILED
case .serviceUnavailable, .networkError, .apiError, .commandFailed, .encodingError:
.UNKNOWN_ERROR
default:
nil
}
}
private func inputErrorCode(for error: PeekabooError) -> ErrorCode? {
switch error {
case .invalidCoordinates:
.INVALID_COORDINATES
case .fileIOError:
.FILE_IO_ERROR
case .invalidInput:
.INVALID_INPUT
default:
nil
}
}
private func credentialErrorCode(for error: PeekabooError) -> ErrorCode? {
switch error {
case .noAIProviderAvailable, .authenticationFailed:
.MISSING_API_KEY
case .aiProviderError:
.AGENT_ERROR
default:
nil
}
}
private func mapCaptureErrorToCode(_ error: CaptureError) -> ErrorCode {
switch error {
case .screenRecordingPermissionDenied, .permissionDeniedScreenRecording:
.PERMISSION_ERROR_SCREEN_RECORDING
case .accessibilityPermissionDenied:
.PERMISSION_ERROR_ACCESSIBILITY
case .appleScriptPermissionDenied:
.PERMISSION_ERROR_APPLESCRIPT
case .noDisplaysAvailable, .noDisplaysFound:
.CAPTURE_FAILED
case .invalidDisplayID, .invalidDisplayIndex:
.INVALID_ARGUMENT
case .captureCreationFailed, .windowCaptureFailed, .captureFailed, .captureFailure:
.CAPTURE_FAILED
case .windowNotFound, .noWindowsFound:
.WINDOW_NOT_FOUND
case .windowTitleNotFound:
.WINDOW_NOT_FOUND
case .fileWriteError, .fileIOError:
.FILE_IO_ERROR
case .appNotFound:
.APP_NOT_FOUND
case .invalidWindowIndexOld, .invalidWindowIndex:
.INVALID_ARGUMENT
case .invalidArgument:
.INVALID_ARGUMENT
case .unknownError:
.UNKNOWN_ERROR
case .noFrontmostApplication:
.WINDOW_NOT_FOUND
case .invalidCaptureArea:
.INVALID_ARGUMENT
case .ambiguousAppIdentifier:
.AMBIGUOUS_APP_IDENTIFIER
case .imageConversionFailed:
.CAPTURE_FAILED
case .detectionTimedOut:
.TIMEOUT
}
}
private func mapFocusErrorToCode(_ error: FocusError) -> ErrorCode {
errorCode(for: error)
}
}
func errorMessage(for error: any Error) -> String {
if let bridgeError = error as? PeekabooBridgeErrorEnvelope {
return bridgeError.message
}
return error.localizedDescription
}
func applicationLaunchErrorCode(for error: any Error) -> ErrorCode? {
guard let bridgeError = error as? PeekabooBridgeErrorEnvelope,
bridgeError.code == .notFound
else {
return nil
}
return .APP_NOT_FOUND
}
func errorDetails(for error: any Error) -> String? {
guard let bridgeError = error as? PeekabooBridgeErrorEnvelope else {
return nil
}
var details: [String] = []
if let bridgeDetails = bridgeError.details, !bridgeDetails.isEmpty {
details.append(bridgeDetails)
}
if let permission = bridgeError.permission {
details.append("permission: \(permission.rawValue)")
}
return details.isEmpty ? nil : details.joined(separator: "\n")
}
func errorCode(for focusError: FocusError) -> ErrorCode {
switch focusError {
case .applicationNotRunning:
.APP_NOT_FOUND
case .focusVerificationTimeout, .timeoutWaitingForCondition:
.TIMEOUT
default:
.WINDOW_NOT_FOUND
}
}
func errorCode(for bridgeError: PeekabooBridgeErrorEnvelope) -> ErrorCode {
switch bridgeError.code {
case .permissionDenied:
switch bridgeError.permission {
case .screenRecording:
.PERMISSION_ERROR_SCREEN_RECORDING
case .accessibility:
.PERMISSION_ERROR_ACCESSIBILITY
case .postEvent:
.PERMISSION_ERROR_EVENT_SYNTHESIZING
case .appleScript:
.PERMISSION_ERROR_APPLESCRIPT
case .none:
.PERMISSION_DENIED
}
case .timeout:
.TIMEOUT
case .invalidRequest:
.INVALID_ARGUMENT
case .operationNotSupported:
.VALIDATION_ERROR
case .notFound:
.UNKNOWN_ERROR
case .versionMismatch, .unauthorizedClient, .decodingFailed, .internalError, .serverBusy:
.UNKNOWN_ERROR
}
}
func errorCode(for posixError: POSIXError) -> ErrorCode {
switch posixError.code {
case .ETIMEDOUT:
.TIMEOUT
default:
.INTERNAL_SWIFT_ERROR
}
}

View File

@ -1,204 +0,0 @@
import Commander
import Foundation
@MainActor
struct CommandHelpRenderer {
static func renderHelp(for type: (some ParsableCommand).Type, theme: HelpTheme? = nil) -> String {
let description = type.commandDescription
if let descriptor = CommanderRegistryBuilder.descriptor(for: type) {
return self.renderHelp(
abstract: description.abstract,
discussion: description.discussion,
signature: descriptor.signature,
usageExamples: description.usageExamples,
theme: theme
)
}
let fallbackSignature = CommandSignature.describe(type.init())
.flattened()
.withPeekabooRuntimeFlags()
return self.renderHelp(
abstract: description.abstract,
discussion: description.discussion,
signature: fallbackSignature,
usageExamples: description.usageExamples,
theme: theme
)
}
private static func renderHelp(
abstract: String,
discussion: String?,
signature: CommandSignature,
usageExamples: [CommandUsageExample],
theme: HelpTheme?
) -> String {
var sections: [String] = []
if let descriptionSection = self.renderDescription(abstract: abstract, discussion: discussion, theme: theme) {
sections.append(descriptionSection)
}
if let argumentsSection = self.renderArguments(signature.arguments, theme: theme) {
sections.append(argumentsSection)
}
if let optionsSection = self.renderOptions(signature.options, theme: theme) {
sections.append(optionsSection)
}
if let flagsSection = self.renderFlags(signature.flags, theme: theme) {
sections.append(flagsSection)
}
if let examplesSection = self.renderExamples(usageExamples, theme: theme) {
sections.append(examplesSection)
}
return sections.joined(separator: "\n\n")
}
private static func renderDescription(abstract: String, discussion: String?, theme: HelpTheme?) -> String? {
var body: [String] = []
if !abstract.isEmpty {
body.append(abstract)
}
if let discussion, !discussion.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
body.append(discussion)
}
guard !body.isEmpty else { return nil }
return self.makeSection(title: "DESCRIPTION", lines: body, theme: theme)
}
private static func renderArguments(_ arguments: [ArgumentDefinition], theme: HelpTheme?) -> String? {
guard !arguments.isEmpty else { return nil }
let rows = arguments.map { argument -> (String, String?) in
let placeholder = self.kebabCased(argument.label)
let label = argument.isOptional ? "[\(placeholder)]" : "<\(placeholder)>"
return (label, argument.help)
}
return self.makeSection(title: "ARGUMENTS", lines: self.renderKeyValueRows(rows, theme: theme), theme: theme)
}
private static func renderOptions(_ options: [OptionDefinition], theme: HelpTheme?) -> String? {
guard !options.isEmpty else { return nil }
let rows = options.map { option -> (String, String?) in
let names = option.names
.filter { !$0.isAlias }
.map(\.cliSpelling)
.joined(separator: ", ")
let valuePlaceholder = " <\(self.optionValuePlaceholder(for: option))>"
return (names + valuePlaceholder, option.help)
}
return self.makeSection(title: "OPTIONS", lines: self.renderKeyValueRows(rows, theme: theme), theme: theme)
}
private static func optionValuePlaceholder(for option: OptionDefinition) -> String {
if let longName = option.names.compactMap(\.primaryLongComponent).first {
return longName
}
return self.kebabCased(self.optionLabel(option.label))
}
private static func optionLabel(_ label: String) -> String {
let suffix = "Option"
guard label.hasSuffix(suffix), label.count > suffix.count else {
return label
}
return String(label.dropLast(suffix.count))
}
private static func kebabCased(_ value: String) -> String {
guard !value.isEmpty else { return value }
var output = ""
for character in value {
if character.isUppercase {
if !output.isEmpty, output.last != "-" {
output.append("-")
}
output.append(contentsOf: character.lowercased())
} else if character == "_" || character == " " {
if !output.isEmpty, output.last != "-" {
output.append("-")
}
} else {
output.append(character)
}
}
return output
}
private static func renderFlags(_ flags: [FlagDefinition], theme: HelpTheme?) -> String? {
guard !flags.isEmpty else { return nil }
let rows = flags.map { flag -> (String, String?) in
let names = flag.names
.filter { !$0.isAlias }
.map(\.cliSpelling)
.joined(separator: ", ")
return (names, flag.help)
}
return self.makeSection(title: "FLAGS", lines: self.renderKeyValueRows(rows, theme: theme), theme: theme)
}
private static func renderExamples(_ examples: [CommandUsageExample], theme: HelpTheme?) -> String? {
guard !examples.isEmpty else { return nil }
let rows = examples.map { ("$ \($0.command)", $0.description) }
return self.makeSection(
title: "USAGE EXAMPLES",
lines: self.renderKeyValueRows(rows, theme: theme),
theme: theme
)
}
private static func makeSection(title: String, lines: [String], theme: HelpTheme?) -> String {
let heading = theme?.heading(title) ?? title
return ([heading] + lines.map { " \($0)" }).joined(separator: "\n")
}
private static func renderKeyValueRows(_ rows: [(String, String?)], theme: HelpTheme?) -> [String] {
guard !rows.isEmpty else { return [] }
let padding = min(max(rows.map(\.0.count).max() ?? 0, 12), 32)
return rows.map { key, value in
guard let value, !value.isEmpty else {
return theme?.command(key) ?? key
}
let paddedKey: String = if key.count >= padding {
key
} else {
key + String(repeating: " ", count: padding - key.count)
}
let displayKey = theme?.command(paddedKey) ?? paddedKey
return "\(displayKey) \(value)"
}
}
}
extension ParsableCommand {
static func helpMessage() -> String {
MainActor.assumeIsolated {
CommandHelpRenderer.renderHelp(for: Self.self)
}
}
}
extension CommanderName {
fileprivate var cliSpelling: String {
switch self {
case let .short(value), let .aliasShort(value):
"-\(value)"
case let .long(value), let .aliasLong(value):
"--\(value)"
}
}
fileprivate var primaryLongComponent: String? {
switch self {
case let .long(value):
value
case .short, .aliasShort, .aliasLong:
nil
}
}
}

View File

@ -1,55 +0,0 @@
import Foundation
import PeekabooCore
// MARK: - Output Formatting Protocol
/// Protocol for commands that support both JSON and human-readable output
@MainActor
protocol OutputFormattable {
var jsonOutput: Bool { get }
var outputLogger: Logger { get }
}
extension OutputFormattable {
/// Output data in appropriate format
func output(_ data: some Codable, humanReadable: () -> Void) {
if jsonOutput {
outputSuccessCodable(data: data, logger: self.outputLogger)
} else {
humanReadable()
}
}
/// Output success with optional data
func outputSuccess(data: (some Codable)? = nil as Empty?) {
if jsonOutput {
if let data {
outputSuccessCodable(data: data, logger: self.outputLogger)
} else {
outputJSON(JSONResponse(success: true), logger: self.outputLogger)
}
}
}
}
// MARK: - Permission Checking
/// Check and require screen recording permission
@MainActor
func requireScreenRecordingPermission(services: any PeekabooServiceProviding) async throws {
let hasPermission = await Task { @MainActor in
await services.screenCapture.hasScreenRecordingPermission()
}.value
guard hasPermission else {
throw CaptureError.screenRecordingPermissionDenied
}
}
/// Check and require accessibility permission
@MainActor
func requireAccessibilityPermission(services: any PeekabooServiceProviding) throws {
if !services.permissions.checkAccessibilityPermission() {
throw CaptureError.accessibilityPermissionDenied
}
}

View File

@ -1,40 +0,0 @@
import Commander
import Dispatch
import Foundation
// MARK: - Runtime Command Protocol
/// Protocol for commands that accept runtime context injection.
/// Commands conforming to this protocol receive a `CommandRuntime` instance
/// containing logger, services, and configuration instead of accessing singletons.
protocol AsyncRuntimeCommand: ParsableCommand {
/// Run the command with injected runtime context.
@MainActor
mutating func run(using runtime: CommandRuntime) async throws
}
extension AsyncRuntimeCommand {
/// Default synchronous run() implementation that builds the runtime context
/// and executes the async implementation on the main actor.
mutating func run() throws {
var commandCopy = self
let semaphore = DispatchSemaphore(value: 0)
var thrownError: (any Error)?
Task { @MainActor in
do {
let runtime = await CommandRuntime.makeDefaultAsync()
try await commandCopy.run(using: runtime)
} catch {
thrownError = error
}
semaphore.signal()
}
semaphore.wait()
self = commandCopy
if let error = thrownError {
throw error
}
}
}

View File

@ -1,378 +0,0 @@
//
// CommandRuntime.swift
// PeekabooCLI
//
import Darwin
import Foundation
import PeekabooAutomation
import PeekabooAutomationKit
import PeekabooBridge
import PeekabooCore
import PeekabooFoundation
import PeekabooProtocols
/// Shared options that control logging and output behavior.
struct CommandRuntimeOptions {
var verbose = false
var jsonOutput = false
var logLevel: LogLevel?
var captureEnginePreference: String?
var inputStrategy: UIInputStrategy?
var preferRemote = true
var remoteIsolationRequested = false
var autoStartDaemon = true
var bridgeSocketPath: String?
var requiresElementActions = false
var requiresInspectAccessibilityTree = false
var requiresBrowserMCP = false
var requiresApplicationLaunchOptions = false
var requiresApplicationRelaunch = false
var requiresSurvivingApplicationHost = false
var requiresHostApplicationInventory = false
var requiresImplicitSnapshotInvalidation = false
var requiresCallerDesktopMutationBarrier = false
var usesPerToolSnapshotInvalidation = false
var requiresExactWindowTargetedClicks = false
var requiresPostEventClickPermission = false
func makeConfiguration() -> CommandRuntime.Configuration {
CommandRuntime.Configuration(
verbose: self.verbose,
jsonOutput: self.jsonOutput,
logLevel: self.logLevel,
captureEnginePreference: self.captureEnginePreference,
inputStrategy: self.inputStrategy
)
}
func applyingEnvironmentOverrides(environment: [String: String]) -> CommandRuntimeOptions {
var options = self
if options.captureEnginePreference == nil,
let captureEngine = Self.captureEnginePreference(environment: environment) {
options.captureEnginePreference = captureEngine
if !options.requiresApplicationLaunchOptions && !options.requiresHostApplicationInventory {
options.preferRemote = false
}
}
return options
}
static func captureEnginePreference(environment: [String: String]) -> String? {
guard let value = environment["PEEKABOO_CAPTURE_ENGINE"]?
.trimmingCharacters(in: .whitespacesAndNewlines),
!value.isEmpty else {
return nil
}
return value
}
}
/// Runtime context passed to runtime-aware commands.
struct CommandRuntime {
static let defaultDaemonIdleTimeoutSeconds: TimeInterval = 300
@TaskLocal
private static var serviceOverride: PeekabooServices?
struct Configuration {
var verbose: Bool
var jsonOutput: Bool
var logLevel: LogLevel?
var captureEnginePreference: String?
var inputStrategy: UIInputStrategy?
}
let configuration: Configuration
let hostDescription: String
let selectedRemoteSocketPath: String?
let selectedRemoteHostProcessIdentifier: pid_t?
let snapshotInvalidationRemoteSocketPaths: [String]
let applicationRelaunchAllowed: Bool
let interactionMutationTracker: InteractionMutationTracker
@MainActor let services: any PeekabooServiceProviding
@MainActor let logger: Logger
@MainActor
var observationTimeoutMutationTracker: InteractionMutationTracker? {
if self.selectedRemoteSocketPath == nil || self.interactionMutationTracker.hasPendingDurableMutation {
return self.interactionMutationTracker
}
return nil
}
@MainActor
init(
configuration: Configuration,
services: any PeekabooServiceProviding,
hostDescription: String = "local (in-process)",
selectedRemoteSocketPath: String? = nil,
selectedRemoteHostProcessIdentifier: pid_t? = nil,
snapshotInvalidationRemoteSocketPaths: [String] = [],
applicationRelaunchAllowed: Bool = true,
interactionMutationTracker: InteractionMutationTracker = InteractionMutationTracker()
) {
// Keep Tachikoma credential/profile resolution aligned with Peekaboo CLI storage.
PeekabooCore.ConfigurationManager.configureTachikomaProfileDirectory()
self.configuration = configuration
self.services = services
self.hostDescription = hostDescription
self.selectedRemoteSocketPath = selectedRemoteSocketPath
self.selectedRemoteHostProcessIdentifier = selectedRemoteHostProcessIdentifier
self.snapshotInvalidationRemoteSocketPaths = snapshotInvalidationRemoteSocketPaths
self.applicationRelaunchAllowed = applicationRelaunchAllowed
self.interactionMutationTracker = interactionMutationTracker
self.logger = Logger.shared
services.installAgentRuntimeDefaults()
self.logger.setJsonOutputMode(configuration.jsonOutput)
let explicitLevel = configuration.logLevel
var shouldEnableVerbose = configuration.verbose
if configuration.jsonOutput && explicitLevel == nil {
shouldEnableVerbose = true
}
if let explicitLevel, explicitLevel <= .verbose {
shouldEnableVerbose = true
}
self.logger.setVerboseMode(shouldEnableVerbose)
if let explicitLevel {
self.logger.setMinimumLogLevel(explicitLevel)
} else if shouldEnableVerbose {
self.logger.setMinimumLogLevel(.verbose)
} else {
self.logger.resetMinimumLogLevel()
}
let visualizerConsoleLevel: PeekabooProtocols.LogLevel? = if let explicitLevel {
explicitLevel.coreLogLevel
} else if shouldEnableVerbose {
.debug
} else {
nil
}
VisualizationClient.shared.setConsoleLogLevelOverride(visualizerConsoleLevel)
VisualizationClient.shared.setConsoleMirroringEnabled(configuration.verbose)
self.services.ensureVisualizerConnection()
self.logger.debug("Runtime host: \(hostDescription)")
}
@MainActor
init(options: CommandRuntimeOptions, services: any PeekabooServiceProviding) {
self.init(configuration: options.makeConfiguration(), services: services)
}
}
extension CommandRuntime {
@MainActor
static func makeDefault(options: CommandRuntimeOptions) -> CommandRuntime {
let effectiveOptions = options.applyingEnvironmentOverrides(environment: ProcessInfo.processInfo.environment)
let services = self.serviceOverride ?? self.makeLocalServices(options: effectiveOptions)
return CommandRuntime(configuration: effectiveOptions.makeConfiguration(), services: services)
}
@MainActor
static func makeDefault() -> CommandRuntime {
self.makeDefault(options: CommandRuntimeOptions())
}
@MainActor
static func makeDefaultAsync(options: CommandRuntimeOptions) async -> CommandRuntime {
let effectiveOptions = options.applyingEnvironmentOverrides(environment: ProcessInfo.processInfo.environment)
if let override = serviceOverride {
return CommandRuntime(options: effectiveOptions, services: override)
}
let resolution = await resolveServices(options: effectiveOptions)
return CommandRuntime(
configuration: effectiveOptions.makeConfiguration(),
services: resolution.services,
hostDescription: resolution.hostDescription,
selectedRemoteSocketPath: resolution.selectedRemoteSocketPath,
selectedRemoteHostProcessIdentifier: resolution.selectedRemoteHostProcessIdentifier,
snapshotInvalidationRemoteSocketPaths: resolution.snapshotInvalidationRemoteSocketPaths,
applicationRelaunchAllowed: resolution.applicationRelaunchAllowed
)
}
@MainActor
static func makeDefaultAsync() async -> CommandRuntime {
await self.makeDefaultAsync(options: CommandRuntimeOptions())
}
@MainActor
static func withInjectedServices<T>(
_ services: PeekabooServices,
perform operation: () async throws -> T
) async rethrows -> T {
try await self.$serviceOverride.withValue(services) {
try await operation()
}
}
@MainActor
private static func resolveServices(options: CommandRuntimeOptions) async -> RuntimeHostResolver.Resolution {
await RuntimeHostResolver.resolveServices(options: options)
}
static func explicitBridgeSocket(
options: CommandRuntimeOptions,
environment: [String: String]
) -> String? {
BridgeSocketResolver.explicitBridgeSocket(options: options, environment: environment)
}
static func shouldAutoStartDaemon(
options: CommandRuntimeOptions,
environment: [String: String]
) -> Bool {
DaemonLaunchPolicy.shouldAutoStartDaemon(options: options, environment: environment)
}
static func daemonSocketPath(environment: [String: String]) -> String {
DaemonLaunchPolicy.daemonSocketPath(environment: environment)
}
static func daemonIdleTimeoutSeconds(environment: [String: String]) -> TimeInterval {
DaemonLaunchPolicy.daemonIdleTimeoutSeconds(environment: environment)
}
static func onDemandDaemonArguments(socketPath: String, idleTimeoutSeconds: TimeInterval) -> [String] {
DaemonLaunchPolicy.onDemandDaemonArguments(socketPath: socketPath, idleTimeoutSeconds: idleTimeoutSeconds)
}
@MainActor
private static func makeLocalServices(options: CommandRuntimeOptions) -> PeekabooServices {
RuntimeServiceFactory.makeLocalServices(options: options)
}
static func hasInputStrategyEnvironmentOverride(environment: [String: String]) -> Bool {
RuntimeInputPolicyResolver.hasEnvironmentOverride(environment: environment)
}
static func hasInputStrategyConfigOverride(input: PeekabooAutomation.Configuration.InputConfig?) -> Bool {
RuntimeInputPolicyResolver.hasConfigOverride(input: input)
}
static func supportsRemoteRequirements(
for handshake: PeekabooBridgeHandshakeResponse,
options: CommandRuntimeOptions
) -> Bool {
BridgeCapabilityPolicy.supportsRemoteRequirements(for: handshake, options: options)
}
static func supportsTargetedHotkeys(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsTargetedHotkeys(for: handshake)
}
static func supportsTargetedTypeActions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsTargetedTypeActions(for: handshake)
}
static func supportsTargetedClicks(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsTargetedClicks(for: handshake)
}
static func supportsApplicationLaunchOptions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsApplicationLaunchOptions(for: handshake)
}
static func supportsApplicationRelaunch(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsApplicationRelaunch(for: handshake)
}
static func supportsImplicitSnapshotInvalidation(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsImplicitSnapshotInvalidation(for: handshake)
}
static func supportsElementActions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsElementActions(for: handshake)
}
static func supportsDesktopObservation(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsDesktopObservation(for: handshake)
}
static func supportsInspectAccessibilityTree(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsInspectAccessibilityTree(for: handshake)
}
static func supportsBrowserMCP(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsBrowserMCP(for: handshake)
}
static func supportsPostEventPermissionRequest(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsPostEventPermissionRequest(for: handshake)
}
static func targetedHotkeyAvailability(for handshake: PeekabooBridgeHandshakeResponse)
-> (isEnabled: Bool, unavailableReason: String?, missingPermissions: Set<PeekabooBridgePermissionKind>) {
BridgeCapabilityPolicy.targetedHotkeyAvailability(for: handshake)
}
static func targetedTypeAvailability(for handshake: PeekabooBridgeHandshakeResponse)
-> (isEnabled: Bool, unavailableReason: String?, missingPermissions: Set<PeekabooBridgePermissionKind>) {
BridgeCapabilityPolicy.targetedTypeAvailability(for: handshake)
}
static func targetedClickAvailability(for handshake: PeekabooBridgeHandshakeResponse)
-> (isEnabled: Bool, unavailableReason: String?, missingPermissions: Set<PeekabooBridgePermissionKind>) {
BridgeCapabilityPolicy.targetedClickAvailability(for: handshake)
}
}
/// Commands that need access to verbose/json flags even before a runtime is injected
/// (e.g., during unit tests) can conform to this protocol and store the parsed options.
protocol RuntimeOptionsConfigurable {
var runtimeOptions: CommandRuntimeOptions { get set }
}
extension RuntimeOptionsConfigurable {
mutating func setRuntimeOptions(_ options: CommandRuntimeOptions) {
runtimeOptions = options
}
}
@propertyWrapper
struct RuntimeStorage<Value: ExpressibleByNilLiteral> {
private var storage: Value
init() {
self.storage = nil
}
var wrappedValue: Value {
get { self.storage }
set { self.storage = newValue }
}
}
extension RuntimeStorage: Codable where Value: ExpressibleByNilLiteral {
init(from _: any Decoder) throws {
self.storage = nil
}
func encode(to _: any Encoder) throws {}
}
extension RuntimeStorage: Sendable where Value: Sendable {}
extension LogLevel {
fileprivate var coreLogLevel: PeekabooProtocols.LogLevel {
switch self {
case .trace: .trace
case .verbose: .debug
case .debug: .debug
case .info: .info
case .warning: .warning
case .error: .error
case .critical: .critical
}
}
}

View File

@ -1,497 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
import PeekabooFoundation
// MARK: - Service Bridges
enum AutomationServiceBridge {
static func waitForElement(
automation: any UIAutomationServiceProtocol,
target: ClickTarget,
timeout: TimeInterval,
snapshotId: String?
) async throws -> WaitForElementResult {
let result = try await Task { @MainActor in
try await automation.waitForElement(target: target, timeout: timeout, snapshotId: snapshotId)
}.value
if !result.warnings.isEmpty {
Logger.shared.debug(
"waitForElement warnings: \(result.warnings.joined(separator: ","))",
category: "Automation"
)
}
return result
}
static func click(
automation: any UIAutomationServiceProtocol,
target: ClickTarget,
clickType: ClickType,
snapshotId: String?
) async throws {
try await Task { @MainActor in
try await automation.click(target: target, clickType: clickType, snapshotId: snapshotId)
}.value
}
static func click(
automation: any UIAutomationServiceProtocol,
target: ClickTarget,
clickType: ClickType,
snapshotId: String?,
targetProcessIdentifier: pid_t,
targetWindowID: Int? = nil
) async throws {
try await Task { @MainActor in
guard let targetedClickService = automation as? any TargetedClickServiceProtocol else {
throw PeekabooError.serviceUnavailable(
"Background clicks require an automation service that supports targeted click delivery"
)
}
guard targetedClickService.supportsTargetedClicks else {
throw self.targetedClickUnavailableError(service: targetedClickService)
}
if let targetWindowID {
guard let exactWindowService = targetedClickService as? any ExactWindowTargetedClickServiceProtocol
else {
throw PeekabooError.serviceUnavailable(
"Background clicks with an exact window require a compatible automation service"
)
}
try await exactWindowService.click(
target: target,
clickType: clickType,
snapshotId: snapshotId,
targetProcessIdentifier: targetProcessIdentifier,
targetWindowID: targetWindowID
)
} else {
try await targetedClickService.click(
target: target,
clickType: clickType,
snapshotId: snapshotId,
targetProcessIdentifier: targetProcessIdentifier
)
}
}.value
}
static func typeActions(
automation: any UIAutomationServiceProtocol,
request: TypeActionsRequest
) async throws -> TypeResult {
try await Task { @MainActor in
try await automation.typeActions(
request.actions,
cadence: request.cadence,
snapshotId: request.snapshotId
)
}.value
}
static func typeActions(
automation: any UIAutomationServiceProtocol,
request: TypeActionsRequest,
targetProcessIdentifier: pid_t
) async throws -> TypeResult {
try await Task { @MainActor in
guard let targetedTypeService = automation as? any TargetedTypeServiceProtocol else {
throw PeekabooError.serviceUnavailable(
"Background typing requires an automation service that supports targeted type delivery"
)
}
guard targetedTypeService.supportsTargetedTypeActions else {
throw self.targetedTypeUnavailableError(service: targetedTypeService)
}
return try await targetedTypeService.typeActions(
request.actions,
cadence: request.cadence,
snapshotId: request.snapshotId,
targetProcessIdentifier: targetProcessIdentifier
)
}.value
}
static func scroll(
automation: any UIAutomationServiceProtocol,
request: ScrollRequest
) async throws {
try await Task { @MainActor in
try await automation.scroll(request)
}.value
}
static func setValue(
automation: any UIAutomationServiceProtocol,
target: String,
value: UIElementValue,
snapshotId: String?
) async throws -> ElementActionResult {
try await Task { @MainActor in
guard let automation = automation as? any ElementActionAutomationServiceProtocol else {
throw PeekabooError.serviceUnavailable(
"This automation host does not support direct accessibility value setting"
)
}
return try await automation.setValue(target: target, value: value, snapshotId: snapshotId)
}.value
}
static func performAction(
automation: any UIAutomationServiceProtocol,
target: String,
actionName: String,
snapshotId: String?
) async throws -> ElementActionResult {
try await Task { @MainActor in
guard let automation = automation as? any ElementActionAutomationServiceProtocol else {
throw PeekabooError.serviceUnavailable(
"This automation host does not support direct accessibility action invocation"
)
}
return try await automation.performAction(target: target, actionName: actionName, snapshotId: snapshotId)
}.value
}
static func hotkey(automation: any UIAutomationServiceProtocol, keys: String, holdDuration: Int) async throws {
try await Task { @MainActor in
try await automation.hotkey(keys: keys, holdDuration: holdDuration)
}.value
}
static func hotkey(
automation: any UIAutomationServiceProtocol,
keys: String,
holdDuration: Int,
targetProcessIdentifier: pid_t
) async throws {
try await Task { @MainActor in
guard let targetedHotkeyService = automation as? any TargetedHotkeyServiceProtocol else {
throw PeekabooError.serviceUnavailable(
"Background hotkeys require an automation service that supports targeted hotkey delivery"
)
}
guard targetedHotkeyService.supportsTargetedHotkeys else {
throw self.targetedHotkeyUnavailableError(service: targetedHotkeyService)
}
try await targetedHotkeyService.hotkey(
keys: keys,
holdDuration: holdDuration,
targetProcessIdentifier: targetProcessIdentifier
)
}.value
}
private static func targetedHotkeyUnavailableError(service: any TargetedHotkeyServiceProtocol) -> PeekabooError {
if service.targetedHotkeyRequiresEventSynthesizingPermission {
return .permissionDeniedEventSynthesizing
}
return .serviceUnavailable(
service.targetedHotkeyUnavailableReason ??
"Remote bridge host does not support background hotkeys; use --no-remote or update the host"
)
}
private static func targetedTypeUnavailableError(service: any TargetedTypeServiceProtocol) -> PeekabooError {
if service.targetedTypeRequiresEventSynthesizingPermission {
return .permissionDeniedEventSynthesizing
}
return .serviceUnavailable(
service.targetedTypeUnavailableReason ??
"Remote bridge host does not support background typing; use --no-remote or update the host"
)
}
private static func targetedClickUnavailableError(service: any TargetedClickServiceProtocol) -> PeekabooError {
if service.targetedClickRequiresEventSynthesizingPermission {
return .permissionDeniedEventSynthesizing
}
return .serviceUnavailable(
service.targetedClickUnavailableReason ??
"Remote bridge host does not support background clicks; use --no-remote or update the host"
)
}
static func swipe(
automation: any UIAutomationServiceProtocol,
request: SwipeRequest
) async throws {
try await Task { @MainActor in
try await automation.swipe(
from: request.from,
to: request.to,
duration: request.duration,
steps: request.steps,
profile: request.profile
)
}.value
}
static func drag(
automation: any UIAutomationServiceProtocol,
request: DragRequest
) async throws {
try await Task { @MainActor in
try await automation.drag(
DragOperationRequest(
from: request.from,
to: request.to,
duration: request.duration,
steps: request.steps,
modifiers: request.modifiers,
profile: request.profile
)
)
}.value
}
static func moveMouse(
automation: any UIAutomationServiceProtocol,
to point: CGPoint,
duration: Int,
steps: Int,
profile: MouseMovementProfile
) async throws {
try await Task { @MainActor in
try await automation.moveMouse(to: point, duration: duration, steps: steps, profile: profile)
}.value
}
static func detectElements(
automation: any UIAutomationServiceProtocol,
imageData: Data,
snapshotId: String?,
windowContext: WindowContext?
) async throws -> ElementDetectionResult {
try await Task { @MainActor in
try await automation.detectElements(
in: imageData,
snapshotId: snapshotId,
windowContext: windowContext
)
}.value
}
static func hasAccessibilityPermission(automation: any UIAutomationServiceProtocol) async -> Bool {
await Task { @MainActor in
await automation.hasAccessibilityPermission()
}.value
}
}
struct TypeActionsRequest {
let actions: [TypeAction]
let cadence: TypingCadence
let snapshotId: String?
}
struct SwipeRequest {
let from: CGPoint
let to: CGPoint
let duration: Int
let steps: Int
let profile: MouseMovementProfile
}
struct DragRequest {
let from: CGPoint
let to: CGPoint
let duration: Int
let steps: Int
let modifiers: String?
let profile: MouseMovementProfile
}
enum WindowServiceBridge {
static func closeWindow(windows: any WindowManagementServiceProtocol, target: WindowTarget) async throws {
try await Task { @MainActor in
try await windows.closeWindow(target: target)
}.value
}
static func minimizeWindow(windows: any WindowManagementServiceProtocol, target: WindowTarget) async throws {
try await Task { @MainActor in
try await windows.minimizeWindow(target: target)
}.value
}
static func maximizeWindow(windows: any WindowManagementServiceProtocol, target: WindowTarget) async throws {
try await Task { @MainActor in
try await windows.maximizeWindow(target: target)
}.value
}
static func moveWindow(
windows: any WindowManagementServiceProtocol,
target: WindowTarget,
to origin: CGPoint
) async throws {
try await Task { @MainActor in
try await windows.moveWindow(target: target, to: origin)
}.value
}
static func resizeWindow(
windows: any WindowManagementServiceProtocol,
target: WindowTarget,
to size: CGSize
) async throws {
try await Task { @MainActor in
try await windows.resizeWindow(target: target, to: size)
}.value
}
static func setWindowBounds(
windows: any WindowManagementServiceProtocol,
target: WindowTarget,
bounds: CGRect
) async throws {
try await Task { @MainActor in
try await windows.setWindowBounds(target: target, bounds: bounds)
}.value
}
static func focusWindow(windows: any WindowManagementServiceProtocol, target: WindowTarget) async throws {
try await Task { @MainActor in
try await windows.focusWindow(target: target)
}.value
}
static func listWindows(
windows: any WindowManagementServiceProtocol,
target: WindowTarget
) async throws -> [ServiceWindowInfo] {
try await Task { @MainActor in
try await windows.listWindows(target: target)
}.value
}
static func getFocusedWindow(windows: any WindowManagementServiceProtocol) async throws -> ServiceWindowInfo? {
try await Task { @MainActor in
try await windows.getFocusedWindow()
}.value
}
}
enum MenuServiceBridge {
static func listMenus(menu: any MenuServiceProtocol, appIdentifier: String) async throws -> MenuStructure {
try await Task { @MainActor in
try await menu.listMenus(for: appIdentifier)
}.value
}
static func listFrontmostMenus(menu: any MenuServiceProtocol) async throws -> MenuStructure {
try await Task { @MainActor in
try await menu.listFrontmostMenus()
}.value
}
static func listMenuExtras(menu: any MenuServiceProtocol) async throws -> [MenuExtraInfo] {
try await Task { @MainActor in
try await menu.listMenuExtras()
}.value
}
static func clickMenuItem(menu: any MenuServiceProtocol, appIdentifier: String, itemPath: String) async throws {
try await Task { @MainActor in
try await menu.clickMenuItem(app: appIdentifier, itemPath: itemPath)
}.value
}
static func clickMenuItemByName(
menu: any MenuServiceProtocol,
appIdentifier: String,
itemName: String
) async throws {
try await Task { @MainActor in
try await menu.clickMenuItemByName(app: appIdentifier, itemName: itemName)
}.value
}
static func clickMenuExtra(menu: any MenuServiceProtocol, title: String) async throws {
try await Task { @MainActor in
try await menu.clickMenuExtra(title: title)
}.value
}
static func isMenuExtraMenuOpen(
menu: any MenuServiceProtocol,
title: String,
ownerPID: pid_t?
) async throws -> Bool {
try await Task { @MainActor in
try await menu.isMenuExtraMenuOpen(title: title, ownerPID: ownerPID)
}.value
}
static func listMenuBarItems(menu: any MenuServiceProtocol, includeRaw: Bool = false) async throws
-> [MenuBarItemInfo] {
try await Task { @MainActor in
try await menu.listMenuBarItems(includeRaw: includeRaw)
}.value
}
static func clickMenuBarItem(named name: String, menu: any MenuServiceProtocol) async throws -> PeekabooCore
.ClickResult {
try await Task<PeekabooCore.ClickResult, any Error> { @MainActor in
try await menu.clickMenuBarItem(named: name)
}.value
}
static func clickMenuBarItem(at index: Int, menu: any MenuServiceProtocol) async throws -> PeekabooCore
.ClickResult {
try await Task<PeekabooCore.ClickResult, any Error> { @MainActor in
try await menu.clickMenuBarItem(at: index)
}.value
}
}
enum DockServiceBridge {
static func launchFromDock(dock: any DockServiceProtocol, appName: String) async throws {
try await Task { @MainActor in
try await dock.launchFromDock(appName: appName)
}.value
}
static func findDockItem(dock: any DockServiceProtocol, name: String) async throws -> DockItem {
try await Task { @MainActor in
try await dock.findDockItem(name: name)
}.value
}
static func rightClickDockItem(dock: any DockServiceProtocol, appName: String, menuItem: String?) async throws {
try await Task { @MainActor in
try await dock.rightClickDockItem(appName: appName, menuItem: menuItem)
}.value
}
static func hideDock(dock: any DockServiceProtocol) async throws {
try await Task { @MainActor in
try await dock.hideDock()
}.value
}
static func showDock(dock: any DockServiceProtocol) async throws {
try await Task { @MainActor in
try await dock.showDock()
}.value
}
static func listDockItems(dock: any DockServiceProtocol, includeAll: Bool) async throws -> [DockItem] {
try await Task { @MainActor in
try await dock.listDockItems(includeAll: includeAll)
}.value
}
}

View File

@ -1,43 +0,0 @@
import Commander
extension CommandSignature {
/// Add Peekaboo's standard runtime flags and options (extends Commander defaults).
func withPeekabooRuntimeFlags() -> CommandSignature {
let base = self.withStandardRuntimeFlags()
let bridgeSocketOption = OptionDefinition.make(
label: "bridge-socket",
names: [
.long("bridge-socket"),
.aliasLong("bridgeSocket"),
],
help: "Override the socket path for a Peekaboo Bridge host",
parsing: .singleValue
)
let noRemoteFlag = FlagDefinition.make(
label: "no-remote",
names: [
.long("no-remote"),
],
help: "Force local execution; skip remote hosts even if available"
)
let inputStrategyOption = OptionDefinition.make(
label: "inputStrategy",
names: [
.long("input-strategy"),
.aliasLong("inputStrategy"),
],
help: "Override UI input strategy: actionFirst, synthFirst, actionOnly, or synthOnly",
parsing: .singleValue
)
return CommandSignature(
arguments: base.arguments,
options: base.options + [bridgeSocketOption, inputStrategyOption],
flags: base.flags + [noRemoteFlag],
optionGroups: base.optionGroups
)
}
}

View File

@ -1,311 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooAutomationKit
import PeekabooCore
import PeekabooFoundation
// MARK: - Timeout Utilities
/// Execute an async operation with a timeout
func withTimeout<T: Sendable>(
seconds: TimeInterval,
operation: @escaping @Sendable () async throws -> T
) async throws -> T {
let task = Task {
try await operation()
}
let timeoutTask = Task {
try? await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
task.cancel()
}
do {
let result = try await task.value
timeoutTask.cancel()
return result
} catch {
timeoutTask.cancel()
if task.isCancelled {
throw CaptureError.captureFailure("Operation timed out after \(seconds) seconds")
}
throw error
}
}
private typealias TimeoutRaceResult = Result<any Sendable, any Error>
private final class TimeoutRace: @unchecked Sendable {
private let lock = NSLock()
private nonisolated(unsafe) var continuation: (@Sendable (TimeoutRaceResult) -> Void)?
private nonisolated(unsafe) var pendingResult: TimeoutRaceResult?
private nonisolated(unsafe) var completed = false
nonisolated func setContinuation<T: Sendable>(_ continuation: CheckedContinuation<T, any Error>) {
let pendingResult: TimeoutRaceResult?
self.lock.lock()
if self.completed {
pendingResult = self.pendingResult
self.pendingResult = nil
} else {
pendingResult = nil
self.continuation = { result in
switch result {
case let .success(value):
guard let value = value as? T else {
continuation
.resume(throwing: PeekabooError.operationError(message: "Timeout result type mismatch"))
return
}
continuation.resume(returning: value)
case let .failure(error):
continuation.resume(throwing: error)
}
}
}
self.lock.unlock()
if let pendingResult {
self.resume(continuation: continuation, with: pendingResult)
}
}
nonisolated func resume<T: Sendable>(with result: Result<T, any Error>) {
let result = result.map { value in value as any Sendable }
let continuation: (@Sendable (TimeoutRaceResult) -> Void)?
self.lock.lock()
if self.completed {
self.lock.unlock()
return
}
self.completed = true
continuation = self.continuation
self.continuation = nil
if continuation == nil {
self.pendingResult = result
}
self.lock.unlock()
continuation?(result)
}
private nonisolated func resume<T: Sendable>(
continuation: CheckedContinuation<T, any Error>,
with result: TimeoutRaceResult
) {
switch result {
case let .success(value):
guard let value = value as? T else {
continuation.resume(throwing: PeekabooError.operationError(message: "Timeout result type mismatch"))
return
}
continuation.resume(returning: value)
case let .failure(error):
continuation.resume(throwing: error)
}
}
}
/// Race an operation against a wall-clock timeout, even if the operation ignores cancellation.
func withCommandTimeout<T: Sendable>(
seconds: TimeInterval,
operationName: String,
operation: @escaping @Sendable () async throws -> T
) async throws -> T {
guard seconds > 0 else {
throw PeekabooError.invalidInput("Timeout must be greater than 0 seconds")
}
let race = TimeoutRace()
let workTask = Task {
do {
let value = try await operation()
race.resume(with: .success(value))
} catch {
race.resume(with: Result<T, any Error>.failure(error))
}
}
let timeoutTask = Task.detached {
do {
try await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
} catch {
return
}
race.resume(with: Result<T, any Error>.failure(PeekabooError.timeout(
operation: operationName,
duration: seconds
)))
workTask.cancel()
}
return try await withTaskCancellationHandler {
try await withCheckedThrowingContinuation { continuation in
race.setContinuation(continuation)
}
} onCancel: {
race.resume(with: Result<T, any Error>.failure(CancellationError()))
workTask.cancel()
timeoutTask.cancel()
}
}
@MainActor
func withMainActorCommandTimeout<T: Sendable>(
seconds: TimeInterval,
operationName: String,
timeoutError: (@Sendable () -> any Error)? = nil,
desktopMutationWatermarkStore: DesktopMutationWatermarkStore? = nil,
interactionMutationTracker: InteractionMutationTracker? = nil,
operation: @escaping @MainActor () async throws -> T
) async throws -> T {
guard seconds > 0 else {
throw PeekabooError.invalidInput("Timeout must be greater than 0 seconds")
}
let race = TimeoutRace()
let pendingMutation = try desktopMutationWatermarkStore?.beginMutation()
do {
try interactionMutationTracker?.retainDurableMutationLease()
} catch {
if let desktopMutationWatermarkStore, let pendingMutation {
try? desktopMutationWatermarkStore.cancelMutation(pendingMutation)
}
throw error
}
let workTask = Task { @MainActor in
let result: Result<T, any Error>
do {
result = try await .success(operation())
} catch {
result = .failure(error)
}
if let desktopMutationWatermarkStore, let pendingMutation {
_ = try? desktopMutationWatermarkStore.completeMutation(pendingMutation)
}
_ = try? interactionMutationTracker?.completeDurableMutation(through: Date())
race.resume(with: result)
}
let timeoutTask = Task.detached {
do {
try await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
} catch {
return
}
let error = timeoutError?() ?? PeekabooError.timeout(operation: operationName, duration: seconds)
race.resume(with: Result<T, any Error>.failure(error))
workTask.cancel()
}
return try await withTaskCancellationHandler {
try await withCheckedThrowingContinuation { continuation in
race.setContinuation(continuation)
}
} onCancel: {
race.resume(with: Result<T, any Error>.failure(CancellationError()))
workTask.cancel()
timeoutTask.cancel()
}
}
// MARK: - Window Target Extensions
extension WindowIdentificationOptions {
/// Create a window target from options
func createTarget() throws -> WindowTarget {
try self.toWindowTarget()
}
/// Select a window from a list based on options
@MainActor
func selectWindow(from windows: [ServiceWindowInfo]) -> ServiceWindowInfo? {
if let windowId {
windows.first(where: { $0.windowID == windowId })
} else if let title = windowTitle {
windows.first { $0.title.localizedCaseInsensitiveContains(title) }
} else if let index = windowIndex, index < windows.count {
windows[index]
} else {
windows.first(where: { window in
window.bounds.width >= 50 &&
window.bounds.height >= 50 &&
window.windowLevel == 0
}) ?? windows.first
}
}
/// Re-fetch the window info after a mutation so callers report fresh bounds.
@MainActor
func refetchWindowInfo(
services: any PeekabooServiceProviding,
logger: Logger,
context: StaticString
) async -> ServiceWindowInfo? {
guard let target = try? self.toWindowTarget() else {
logger.warn("Failed to refetch window info (\(context)): invalid target")
return nil
}
do {
let refreshedWindows = try await WindowServiceBridge.listWindows(
windows: services.windows,
target: target
)
return self.selectWindow(from: refreshedWindows)
} catch {
logger.warn("Failed to refetch window info (\(context)): \(error.localizedDescription)")
return nil
}
}
}
// MARK: - Application Resolution
/// Marker protocol for commands that need to resolve applications using injected services.
protocol ApplicationResolver {}
extension ApplicationResolver {
func resolveApplication(
_ identifier: String,
services: any PeekabooServiceProviding
) async throws -> ServiceApplicationInfo {
do {
return try await services.applications.findApplication(identifier: identifier)
} catch {
if identifier.lowercased() == "frontmost" {
var message = "Application 'frontmost' not found"
message += "\n\n💡 Note: 'frontmost' is not a valid app name. To work with the currently active app:"
message += "\n • Use `see` without arguments to capture current screen"
message += "\n • Use `app focus` with a specific app name"
message += "\n • Use `--app frontmost` with image/see commands to capture the active window"
throw PeekabooError.appNotFound(identifier)
}
throw error
}
}
}
// MARK: - Capture Error Extensions
extension Error {
/// Convert any error to a CaptureError if possible
var asCaptureError: CaptureError {
if let captureError = self as? CaptureError {
return captureError
}
if let peekabooError = self as? PeekabooError {
switch peekabooError {
case let .appNotFound(identifier):
return .appNotFound(identifier)
case .windowNotFound:
return .windowNotFound
default:
return .unknownError(self.localizedDescription)
}
}
return .unknownError(self.localizedDescription)
}
}

View File

@ -1,579 +0,0 @@
import Commander
import Foundation
import PeekabooAutomationKit
// MARK: - Binder
enum CommanderCLIBinder {
static func instantiateCommand(
type: any ParsableCommand.Type,
parsedValues: ParsedValues
) throws -> any ParsableCommand {
var command = type.init()
let runtimeOptions = try makeRuntimeOptions(from: parsedValues, commandType: type)
if var bindable = command as? any CommanderBindableCommand {
try bindable.applyCommanderValues(.init(parsedValues: parsedValues))
guard let rebound = bindable as? any ParsableCommand else {
preconditionFailure("CommanderBindableCommand cast should always round-trip to original type \(type)")
}
command = rebound
}
if var configurable = command as? any RuntimeOptionsConfigurable {
configurable.setRuntimeOptions(runtimeOptions)
guard let rebound = configurable as? any ParsableCommand else {
preconditionFailure("RuntimeOptionsConfigurable cast should always round-trip to original type \(type)")
}
command = rebound
}
return command
}
static func instantiateCommand<T: ParsableCommand>(
ofType type: T.Type,
parsedValues: ParsedValues
) throws -> T {
guard let command = try instantiateCommand(type: type, parsedValues: parsedValues) as? T else {
preconditionFailure("Commander instantiation failed to produce expected type \(T.self)")
}
return command
}
static func makeRuntimeOptions(
from parsedValues: ParsedValues,
commandType: (any ParsableCommand.Type)? = nil
) throws -> CommandRuntimeOptions {
var options = CommandRuntimeOptions()
options.requiresApplicationLaunchOptions = Self.requiresApplicationLaunchOptions(commandType)
options.requiresApplicationRelaunch = commandType == AppCommand.RelaunchSubcommand.self
options.requiresSurvivingApplicationHost = commandType == AppCommand.QuitSubcommand.self
options.requiresHostApplicationInventory = Self.requiresHostApplicationInventory(commandType)
options.requiresImplicitSnapshotInvalidation = Self.requiresImplicitSnapshotInvalidation(
commandType,
parsedValues: parsedValues
)
let clipboardMayMutate = commandType == ClipboardCommand.self &&
Self.clipboardMayMutate(parsedValues)
options.requiresCallerDesktopMutationBarrier = commandType == SwitchSubcommand.self ||
commandType == MoveWindowSubcommand.self ||
commandType == CaptureActionCommand.self ||
clipboardMayMutate
options.requiresExactWindowTargetedClicks = Self.requiresExactWindowTargetedClicks(
commandType,
parsedValues: parsedValues
)
options.requiresPostEventClickPermission = Self.requiresPostEventClickPermission(
commandType,
parsedValues: parsedValues
)
options.usesPerToolSnapshotInvalidation = commandType == AgentCommand.self ||
commandType == MCPCommand.Serve.self ||
commandType == InspectUICommand.self
options.verbose = parsedValues.flags.contains("verbose")
options.jsonOutput = parsedValues.flags.contains("jsonOutput")
let values = CommanderBindableValues(parsedValues: parsedValues)
if let level: LogLevel = try values.decodeOption("logLevel", as: LogLevel.self) {
options.logLevel = level
}
if let captureEngine = values.singleOption("captureEngine")?
.trimmingCharacters(in: .whitespacesAndNewlines),
!captureEngine.isEmpty {
options.captureEnginePreference = captureEngine
if !options.requiresApplicationLaunchOptions && !options.requiresHostApplicationInventory {
options.preferRemote = false
}
}
if let rawInputStrategy = values.singleOption("inputStrategy")?
.trimmingCharacters(in: .whitespacesAndNewlines),
!rawInputStrategy.isEmpty {
guard let strategy = UIInputStrategy(rawValue: rawInputStrategy) else {
throw CommanderBindingError.invalidArgument(
label: "input-strategy",
value: rawInputStrategy,
reason: "expected one of \(UIInputStrategy.allCases.map(\.rawValue).joined(separator: ", "))"
)
}
options.inputStrategy = strategy
}
if values.flag("no-remote") {
options.preferRemote = false
options.remoteIsolationRequested = true
}
let explicitBridgeSocket = values.singleOption("bridge-socket")?.trimmingCharacters(in: .whitespacesAndNewlines)
if commandType == AgentCommand.self && !values.flag("no-remote") {
// Agent execution should stay local by default unless explicitly overridden.
options.preferRemote = false
}
if Self.isDaemonCommand(commandType) {
options.preferRemote = false
options.autoStartDaemon = false
}
if Self.requiresCallerLocalRuntime(commandType) {
options.preferRemote = false
} else if Self.prefersLocalRuntime(commandType), !values.flag("no-remote"),
explicitBridgeSocket?.isEmpty ?? true {
options.preferRemote = false
}
if let socketPath = explicitBridgeSocket, !socketPath.isEmpty {
options.bridgeSocketPath = socketPath
}
if commandType == SetValueCommand.self || commandType == PerformActionCommand.self {
options.requiresElementActions = true
}
if commandType == InspectUICommand.self {
options.requiresInspectAccessibilityTree = true
}
if commandType == BrowserCommand.self {
options.requiresBrowserMCP = true
}
return options
}
private static func requiresApplicationLaunchOptions(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == OpenCommand.self ||
commandType == AppCommand.LaunchSubcommand.self ||
commandType == AppCommand.RelaunchSubcommand.self
}
private static func requiresHostApplicationInventory(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == ListCommand.AppsSubcommand.self ||
commandType == AppCommand.ListSubcommand.self
}
private static func requiresImplicitSnapshotInvalidation(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
if commandType == ClipboardCommand.self {
return self.clipboardMayMutate(parsedValues)
}
if commandType == MenuBarCommand.self {
return parsedValues.positional.first?.lowercased() == "click"
}
if commandType == BrowserCommand.self {
return BrowserCommand.actionMayMutate(parsedValues.positional.first ?? "status")
}
if commandType == SeeCommand.self {
return true
}
if self.isInteractivePermissionRequest(commandType) {
return true
}
if commandType == DialogCommand.ListSubcommand.self {
return self.dialogListMayFocus(parsedValues)
}
if commandType == MenuCommand.ListSubcommand.self {
return self.menuListMayFocus(parsedValues)
}
if commandType == ImageCommand.self ||
commandType == CaptureLiveCommand.self ||
commandType == CaptureWatchAlias.self {
return self.captureCommandMayFocus(commandType, parsedValues: parsedValues)
}
return commandType == OpenCommand.self ||
commandType == AppCommand.LaunchSubcommand.self ||
commandType == AppCommand.RelaunchSubcommand.self ||
commandType == AppCommand.QuitSubcommand.self ||
commandType == AppCommand.HideSubcommand.self ||
commandType == AppCommand.UnhideSubcommand.self ||
commandType == AppCommand.SwitchSubcommand.self ||
commandType == ClickCommand.self ||
commandType == MoveCommand.self ||
commandType == TypeCommand.self ||
commandType == PressCommand.self ||
commandType == HotkeyCommand.self ||
commandType == PasteCommand.self ||
commandType == ScrollCommand.self ||
commandType == SwipeCommand.self ||
commandType == DragCommand.self ||
commandType == SetValueCommand.self ||
commandType == PerformActionCommand.self ||
commandType == CaptureActionCommand.self ||
commandType == WindowCommand.FocusSubcommand.self ||
commandType == WindowCommand.CloseSubcommand.self ||
commandType == WindowCommand.MinimizeSubcommand.self ||
commandType == WindowCommand.MaximizeSubcommand.self ||
commandType == WindowCommand.MoveSubcommand.self ||
commandType == WindowCommand.ResizeSubcommand.self ||
commandType == WindowCommand.SetBoundsSubcommand.self ||
commandType == DialogCommand.ClickSubcommand.self ||
commandType == DialogCommand.DismissSubcommand.self ||
commandType == DialogCommand.InputSubcommand.self ||
commandType == DialogCommand.FileSubcommand.self ||
commandType == MenuCommand.ClickSubcommand.self ||
commandType == MenuCommand.ClickExtraSubcommand.self ||
commandType == DockCommand.LaunchSubcommand.self ||
commandType == DockCommand.RightClickSubcommand.self ||
commandType == DockCommand.HideSubcommand.self ||
commandType == DockCommand.ShowSubcommand.self ||
commandType == SwitchSubcommand.self ||
commandType == MoveWindowSubcommand.self ||
commandType == RunCommand.self
}
private static func isInteractivePermissionRequest(
_ commandType: (any ParsableCommand.Type)?
) -> Bool {
commandType == PermissionsCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionsCommand.RequestEventSynthesizingSubcommand.self ||
commandType == PermissionCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestAccessibilitySubcommand.self ||
commandType == PermissionCommand.RequestEventSynthesizingSubcommand.self
}
private static func clipboardMayMutate(_ parsedValues: ParsedValues) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
let positionalAction = values.positionalValue(at: 0)?
.trimmingCharacters(in: .whitespacesAndNewlines)
let action = (positionalAction?.isEmpty == false ? positionalAction : nil) ??
values.singleOption("actionOption") ??
values.singleOption("action")
return ClipboardCommand.actionMayMutate(action)
}
private static func menuListMayFocus(_ parsedValues: ParsedValues) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
return !values.flag("noAutoFocus")
}
private static func dialogListMayFocus(_ parsedValues: ParsedValues) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
let hasWindowTarget = values.singleOption("windowId") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil
if hasWindowTarget {
return true
}
guard !values.flag("noAutoFocus") else { return false }
let app = values.singleOption("app")?
.trimmingCharacters(in: .whitespacesAndNewlines)
return app?.isEmpty == false ||
values.singleOption("pid") != nil
}
private static func captureCommandMayFocus(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
let focus = values.singleOption("captureFocus")?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased()
guard focus != "background" else { return false }
let app = values.singleOption("app")?
.trimmingCharacters(in: .whitespacesAndNewlines)
let hasApplicationTarget = app?.isEmpty == false || values.singleOption("pid") != nil
if commandType == ImageCommand.self {
let normalizedApp = app?.lowercased()
guard normalizedApp != "menubar", normalizedApp != "frontmost" else { return false }
let mode = values.singleOption("mode")?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased() ?? Self.inferredImageCaptureMode(values)
switch mode {
case "window":
return values.singleOption("windowId") == nil && hasApplicationTarget
case "multi":
return hasApplicationTarget
default:
return false
}
}
let mode = values.singleOption("mode")?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased() ?? Self.inferredLiveCaptureMode(values)
return mode == "window" && hasApplicationTarget
}
private static func inferredImageCaptureMode(_ values: CommanderBindableValues) -> String {
if values.singleOption("region") != nil { return "area" }
if values.singleOption("app") != nil ||
values.singleOption("pid") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil ||
values.singleOption("windowId") != nil {
return "window"
}
return "frontmost"
}
private static func inferredLiveCaptureMode(_ values: CommanderBindableValues) -> String {
if values.singleOption("region") != nil { return "area" }
if values.singleOption("app") != nil ||
values.singleOption("pid") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil {
return "window"
}
return "frontmost"
}
private static func requiresExactWindowTargetedClicks(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
guard commandType == ClickCommand.self else { return false }
let values = CommanderBindableValues(parsedValues: parsedValues)
guard self.usesBackgroundClickDelivery(values) else { return false }
let hasWindowSelector = values.singleOption("windowId") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil
if hasWindowSelector {
return true
}
let hasProcessTarget = values.singleOption("app") != nil || values.singleOption("pid") != nil
return values.singleOption("coords") != nil && hasProcessTarget && !values.flag("globalCoords")
}
private static func requiresPostEventClickPermission(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
guard commandType == ClickCommand.self else { return false }
let values = CommanderBindableValues(parsedValues: parsedValues)
guard self.usesBackgroundClickDelivery(values) else { return false }
if values.singleOption("coords") != nil {
return true
}
// ClickCommand resolves conflicting flags as right-click first, then double-click.
return values.flag("double") && !values.flag("right")
}
private static func usesBackgroundClickDelivery(_ values: CommanderBindableValues) -> Bool {
if values.flag("focusBackground") { return true }
return !values.flag("foreground") &&
!values.flag("noAutoFocus") &&
!values.flag("spaceSwitch") &&
!values.flag("bringToCurrentSpace") &&
values.singleOption("focusTimeoutSeconds") == nil &&
values.singleOption("focusRetryCount") == nil
}
private static func prefersLocalRuntime(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == MCPCommand.Serve.self ||
commandType == ToolsCommand.self ||
commandType == SleepCommand.self ||
commandType == LearnCommand.self ||
commandType == CleanCommand.self ||
commandType == ConfigCommand.InitCommand.self ||
commandType == ConfigCommand.ShowCommand.self ||
commandType == ConfigCommand.EditCommand.self ||
commandType == ConfigCommand.ValidateCommand.self ||
commandType == ConfigCommand.AddCommand.self ||
commandType == ConfigCommand.LoginCommand.self ||
commandType == ConfigCommand.SetCredentialCommand.self ||
commandType == ConfigCommand.AddProviderCommand.self ||
commandType == ConfigCommand.ListProvidersCommand.self ||
commandType == ConfigCommand.TestProviderCommand.self ||
commandType == ConfigCommand.RemoveProviderCommand.self ||
commandType == ConfigCommand.ModelsProviderCommand.self ||
commandType == ListCommand.ScreensSubcommand.self ||
commandType == PermissionsCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestAccessibilitySubcommand.self
}
private static func requiresCallerLocalRuntime(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == PermissionsCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestAccessibilitySubcommand.self
}
private static func isDaemonCommand(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == DaemonCommand.self ||
commandType == DaemonCommand.Start.self ||
commandType == DaemonCommand.Stop.self ||
commandType == DaemonCommand.Status.self ||
commandType == DaemonCommand.Run.self
}
}
// MARK: - Bindable Protocol
struct CommanderBindableValues {
let positional: [String]
let options: [String: [String]]
let flags: Set<String>
init(positional: [String], options: [String: [String]], flags: Set<String>) {
self.positional = positional
self.options = options
self.flags = flags
}
init(parsedValues: ParsedValues) {
self.init(positional: parsedValues.positional, options: parsedValues.options, flags: parsedValues.flags)
}
func positionalValue(at index: Int) -> String? {
guard index >= 0, index < self.positional.count else { return nil }
return self.positional[index]
}
func requiredPositional(_ index: Int, label: String) throws -> String {
guard let value = positionalValue(at: index) else {
throw CommanderBindingError.missingArgument(label: label)
}
return value
}
func singleOption(_ label: String) -> String? {
self.options[label]?.last
}
func optionValues(_ label: String) -> [String] {
self.options[label] ?? []
}
func flag(_ label: String) -> Bool {
self.flags.contains(label)
}
func decodePositional<T: ExpressibleFromArgument>(
_ index: Int,
label: String,
as type: T.Type = T.self
) throws -> T {
let raw = try requiredPositional(index, label: label)
guard let value = T(argument: raw) else {
throw CommanderBindingError.invalidArgument(label: label, value: raw, reason: "Unable to parse \(T.self)")
}
return value
}
func decodeOptionalPositional<T: ExpressibleFromArgument>(
_ index: Int,
label: String,
as type: T.Type = T.self
) throws -> T? {
guard let raw = positionalValue(at: index) else {
return nil
}
guard let value = T(argument: raw) else {
throw CommanderBindingError.invalidArgument(label: label, value: raw, reason: "Unable to parse \(T.self)")
}
return value
}
func decodeOption<T: ExpressibleFromArgument>(_ label: String, as type: T.Type = T.self) throws -> T? {
guard let raw = singleOption(label) else {
return nil
}
guard let value = T(argument: raw) else {
throw CommanderBindingError.invalidArgument(label: label, value: raw, reason: "Unable to parse \(T.self)")
}
return value
}
func requireOption<T: ExpressibleFromArgument>(_ label: String, as type: T.Type = T.self) throws -> T {
guard let value: T = try decodeOption(label, as: type) else {
throw CommanderBindingError.missingArgument(label: label)
}
return value
}
func decodeOptionEnum<T: RawRepresentable>(
_ label: String,
as type: T.Type = T.self,
caseInsensitive: Bool = true
) throws -> T? where T.RawValue == String {
guard let raw = singleOption(label) else {
return nil
}
let candidate = caseInsensitive ? raw.lowercased() : raw
guard let value = T(rawValue: candidate) else {
throw CommanderBindingError.invalidArgument(label: label, value: raw, reason: "Unknown value for \(T.self)")
}
return value
}
}
extension CommanderBindableValues {
func makeWindowOptions() throws -> WindowIdentificationOptions {
var options = WindowIdentificationOptions()
try fillWindowOptions(into: &options)
return options
}
func fillWindowOptions(into options: inout WindowIdentificationOptions) throws {
options.app = self.singleOption("app")
if let pid: Int32 = try decodeOption("pid", as: Int32.self) {
options.pid = pid
}
if let windowId: Int = try decodeOption("windowId", as: Int.self) {
options.windowId = windowId
}
options.windowTitle = self.singleOption("windowTitle")
if let index: Int = try decodeOption("windowIndex", as: Int.self) {
options.windowIndex = index
}
}
func makeInteractionTargetOptions() throws -> InteractionTargetOptions {
var options = InteractionTargetOptions()
try fillInteractionTargetOptions(into: &options)
return options
}
func fillInteractionTargetOptions(into options: inout InteractionTargetOptions) throws {
options.app = self.singleOption("app")
if let pid: Int32 = try decodeOption("pid", as: Int32.self) {
options.pid = pid
}
if let windowId: Int = try decodeOption("windowId", as: Int.self) {
options.windowId = windowId
}
options.windowTitle = self.singleOption("windowTitle")
if let index: Int = try decodeOption("windowIndex", as: Int.self) {
options.windowIndex = index
}
}
func makeFocusOptions(includeBackgroundDelivery: Bool = false) throws -> FocusCommandOptions {
var options = FocusCommandOptions()
try fillFocusOptions(into: &options, includeBackgroundDelivery: includeBackgroundDelivery)
return options
}
func fillFocusOptions(
into options: inout FocusCommandOptions,
includeBackgroundDelivery: Bool = false
) throws {
options.noAutoFocus = self.flag("noAutoFocus")
options.spaceSwitch = self.flag("spaceSwitch")
options.bringToCurrentSpace = self.flag("bringToCurrentSpace")
if includeBackgroundDelivery && self.flag("focusBackground") {
options.focusBackground = true
}
if let timeout: TimeInterval = try decodeOption("focusTimeoutSeconds", as: TimeInterval.self) {
options.focusTimeoutSeconds = timeout
}
if let retries: Int = try decodeOption("focusRetryCount", as: Int.self) {
options.focusRetryCount = retries
}
}
}
@MainActor
protocol CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws
}
enum CommanderBindingError: LocalizedError, Equatable {
case missingArgument(label: String)
case invalidArgument(label: String, value: String, reason: String)
var errorDescription: String? {
switch self {
case let .missingArgument(label):
"Missing argument: \(label)"
case let .invalidArgument(label, value, reason):
"Invalid value '\(value)' for \(label): \(reason)"
}
}
}

View File

@ -1,65 +0,0 @@
import CoreGraphics
import Foundation
import PeekabooCore
enum CursorMovementProfileSelection: String {
case linear
case human
}
struct CursorMovementParameters {
let profile: MouseMovementProfile
let duration: Int
let steps: Int
let smooth: Bool
let profileName: String
}
struct CursorMovementResolutionRequest {
let selection: CursorMovementProfileSelection
let durationOverride: Int?
let stepsOverride: Int?
let baseSmooth: Bool
let distance: CGFloat
let defaultDuration: Int
let defaultSteps: Int
}
enum CursorMovementResolver {
static func resolve(_ request: CursorMovementResolutionRequest) -> CursorMovementParameters {
switch request.selection {
case .linear:
let resolvedDuration = request.durationOverride ?? (request.baseSmooth ? request.defaultDuration : 0)
let resolvedSteps = request.baseSmooth ? max(request.stepsOverride ?? request.defaultSteps, 1) : 1
return CursorMovementParameters(
profile: .linear,
duration: resolvedDuration,
steps: resolvedSteps,
smooth: request.baseSmooth,
profileName: request.selection.rawValue
)
case .human:
let resolvedDuration = request.durationOverride ?? Self.humanDuration(for: request.distance)
let resolvedSteps = max(request.stepsOverride ?? Self.humanSteps(for: request.distance), 30)
return CursorMovementParameters(
profile: .human(),
duration: resolvedDuration,
steps: resolvedSteps,
smooth: true,
profileName: request.selection.rawValue
)
}
}
private static func humanDuration(for distance: CGFloat) -> Int {
let distanceFactor = log2(Double(distance) + 1) * 90
let perPixel = Double(distance) * 0.45
let estimate = 280 + distanceFactor + perPixel
return min(max(Int(estimate), 300), 1700)
}
private static func humanSteps(for distance: CGFloat) -> Int {
let scaled = Int(distance * 0.35)
return min(max(scaled, 40), 140)
}
}

Some files were not shown because too many files have changed in this diff Show More