Compare commits

...

35 Commits
v3.4.1 ... main

Author SHA1 Message Date
Peter Steinberger
dda07c245f
fix: document element IDs as opaque (#202)
Some checks failed
macOS CI / PeekabooCore build & tests (push) Has been cancelled
Website (GitHub Pages) / build (push) Has been cancelled
macOS CI / Peekaboo CLI build & tests (push) Has been cancelled
macOS CI / Tachikoma build & tests (push) Has been cancelled
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Has been cancelled
macOS CI / SwiftLint (core + CLI) (push) Has been cancelled
Website (GitHub Pages) / deploy (push) Has been cancelled
2026-06-24 09:11:52 +01:00
Brandon Charleson
efde5b18ca
fix(release): remove appcast entry for unpublished 3.5.3 release (#199)
* fix(release): remove appcast entry for unpublished 3.5.3 release

The appcast advertised v3.5.3 but no GitHub release asset exists,
causing Sparkle updates to fail with a 404 when downloading the zip.

Co-authored-by: Cursor <cursoragent@cursor.com>

* docs: credit appcast rollback contributor

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-24 08:42:00 +01:00
Peter Steinberger
cde3b04991
test(cli): stabilize timeout watermark checks (#201) 2026-06-24 08:18:58 +01:00
Peter Steinberger
4085f18ddc
fix(cli): keep implicit see screenshots private (#200)
Some checks are pending
macOS CI / PeekabooCore build & tests (push) Waiting to run
macOS CI / Peekaboo CLI build & tests (push) Blocked by required conditions
macOS CI / Tachikoma build & tests (push) Blocked by required conditions
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Blocked by required conditions
macOS CI / SwiftLint (core + CLI) (push) Blocked by required conditions
Website (GitHub Pages) / build (push) Waiting to run
Website (GitHub Pages) / deploy (push) Blocked by required conditions
2026-06-24 00:58:16 +01:00
Sebastien Tardif
db5192bb37
fix(capture): honor stop during watch transient backoff (#193)
* fix(capture): honor stop during watch transient backoff

* test(capture): tighten transient stop proof and add proof script

* test(capture): tighten transient stop regression

* chore: complete main merge

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-24 00:09:05 +01:00
Coy Geek
1771d7db34
docs: refresh Peekaboo agent skill guidance (#197)
* docs(skill): refresh Peekaboo agent skill

* docs(skill): polish Peekaboo guidance

* docs: note refreshed agent skill guidance

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-23 15:41:58 -07:00
Peter Steinberger
ab0d96e6d5
chore(deps): update TauTUI and Tachikoma 2026-06-23 22:10:01 +01:00
Peter Steinberger
9b9c5de43b
chore(release): update appcast for 3.5.3
Some checks failed
macOS CI / PeekabooCore build & tests (push) Has been cancelled
Website (GitHub Pages) / build (push) Has been cancelled
macOS CI / Peekaboo CLI build & tests (push) Has been cancelled
macOS CI / Tachikoma build & tests (push) Has been cancelled
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Has been cancelled
macOS CI / SwiftLint (core + CLI) (push) Has been cancelled
Website (GitHub Pages) / deploy (push) Has been cancelled
2026-06-13 23:09:35 -07:00
Peter Steinberger
f22e46cc1a
chore(release): prepare 3.5.3 2026-06-13 22:30:12 -07:00
Peter Steinberger
4f2b0e9cb4
fix: harden background computer automation 2026-06-13 21:44:58 -07:00
Peter Steinberger
5fba9b79de
docs: refresh project banner (#190)
Some checks are pending
macOS CI / PeekabooCore build & tests (push) Waiting to run
macOS CI / Peekaboo CLI build & tests (push) Blocked by required conditions
macOS CI / Tachikoma build & tests (push) Blocked by required conditions
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Blocked by required conditions
macOS CI / SwiftLint (core + CLI) (push) Blocked by required conditions
2026-06-13 14:59:28 -07:00
Peter Steinberger
3aaa96bfa4
fix: filter background apps from app list (#189) 2026-06-13 14:28:47 -07:00
Peter Steinberger
8cf0796692
Fix clicks on hidden menu extras (#188)
Some checks failed
macOS CI / PeekabooCore build & tests (push) Waiting to run
macOS CI / Peekaboo CLI build & tests (push) Blocked by required conditions
macOS CI / Tachikoma build & tests (push) Blocked by required conditions
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Blocked by required conditions
macOS CI / SwiftLint (core + CLI) (push) Blocked by required conditions
Website (GitHub Pages) / build (push) Has been cancelled
Website (GitHub Pages) / deploy (push) Has been cancelled
* fix: reject hidden menu extras

* docs: note hidden menu extra safety
2026-06-13 02:59:25 -07:00
Peter Steinberger
26c7291292
docs(changelog): open 3.5.3 2026-06-13 02:06:20 -07:00
Peter Steinberger
84100e4cc1
chore(release): update appcast for 3.5.2 2026-06-13 02:04:38 -07:00
Peter Steinberger
1fa8eead7e
chore(release): prepare 3.5.2 2026-06-13 01:15:58 -07:00
Peter Steinberger
b0f5086ad4 chore(deps): synchronize Tachikoma 2026-06-13 02:43:13 -04:00
Peter Steinberger
aabea1550e fix(type): remove default keystroke delay 2026-06-13 02:20:30 -04:00
Peter Steinberger
1c6273c017 fix(type): default to fast linear typing 2026-06-13 02:17:10 -04:00
Peter Steinberger
131be20a69
docs(release): harden release workflow 2026-06-12 22:10:28 -07:00
Peter Steinberger
8656242865
docs(changelog): open 3.5.2 2026-06-12 21:30:34 -07:00
Peter Steinberger
e123fa5bc1
chore(release): update appcast for 3.5.1 2026-06-12 21:30:18 -07:00
Peter Steinberger
6a932d0004
chore(release): prepare 3.5.1 2026-06-12 20:51:04 -07:00
Peter Steinberger
ee3f90c404
fix(cli): enforce suspended observation deadlines 2026-06-12 15:23:45 -07:00
Peter Steinberger
a5bbd1ebdc
docs(changelog): open 3.5.1
Some checks are pending
macOS CI / PeekabooCore build & tests (push) Waiting to run
macOS CI / Peekaboo CLI build & tests (push) Blocked by required conditions
macOS CI / Tachikoma build & tests (push) Blocked by required conditions
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Blocked by required conditions
macOS CI / SwiftLint (core + CLI) (push) Blocked by required conditions
Website (GitHub Pages) / build (push) Waiting to run
Website (GitHub Pages) / deploy (push) Blocked by required conditions
2026-06-12 11:52:29 -07:00
Peter Steinberger
371bed775b
chore(release): update appcast for 3.5.0 2026-06-12 11:52:04 -07:00
Peter Steinberger
231fa48370
fix(release): publish verified npm tarballs from dirty trees 2026-06-12 01:36:34 -07:00
Peter Steinberger
0f66ff5c24
fix(release): build app before Developer ID signing 2026-06-12 01:12:50 -07:00
Peter Steinberger
e183cd15fb
chore(release): prepare 3.5.0 2026-06-12 00:45:49 -07:00
Peter Steinberger
64a4bd6184
docs: refresh runtime and provider guidance 2026-06-12 00:45:45 -07:00
Peter Steinberger
d50472e5a3
fix(bridge): enforce exclusive socket ownership (#187)
Give Bridge listeners exclusive lease-backed socket ownership, bound transport and shutdown drains, isolate reusable daemon hosting, preserve healthy Peekaboo.app fallback, and safely migrate legacy daemons.

Fixes #184.
2026-06-12 00:17:34 -07:00
Vishal Jain
b873daf790
fix(capture): avoid false-success screen captures (#185)
Some checks are pending
macOS CI / PeekabooCore build & tests (push) Waiting to run
macOS CI / Peekaboo CLI build & tests (push) Blocked by required conditions
macOS CI / Tachikoma build & tests (push) Blocked by required conditions
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Blocked by required conditions
macOS CI / SwiftLint (core + CLI) (push) Blocked by required conditions
Website (GitHub Pages) / build (push) Waiting to run
Website (GitHub Pages) / deploy (push) Blocked by required conditions
* fix(capture): avoid false-success screen captures

* fix(capture): fail closed when screen capture fallback is unsafe

* fix(capture): translate screencapture display regions

* fix(capture): harden legacy screen capture

* fix(cli): keep screen permission requests local

---------

Co-authored-by: Peter Steinberger <steipete@gmail.com>
2026-06-11 19:24:39 -07:00
Peter Steinberger
e4cd616e19
chore(release): prepare 3.4.2 2026-06-11 18:01:03 -07:00
Peter Steinberger
e44486ff16
feat: add Claude Fable 5 support (#186) 2026-06-11 17:59:43 -07:00
Peter Steinberger
7c3862b032
docs(changelog): open 3.4.2
Some checks failed
macOS CI / PeekabooCore build & tests (push) Has been cancelled
macOS CI / Peekaboo CLI build & tests (push) Has been cancelled
macOS CI / Tachikoma build & tests (push) Has been cancelled
macOS CI / Build macOS apps (Peekaboo + Inspector) (push) Has been cancelled
macOS CI / SwiftLint (core + CLI) (push) Has been cancelled
2026-06-10 05:56:30 +01:00
362 changed files with 28880 additions and 3489 deletions

View File

@ -99,10 +99,11 @@ If both `history` and non-S3 `submit` fail, suspect wrong access level or stale
```bash
op run --env-file "$ENVFILE" -- \
bash -lc 'printf "y\n" | ./scripts/release-binaries.sh --create-github-release --publish-npm'
bash -c 'printf "y\n" | ./scripts/release-binaries.sh --create-github-release --publish-npm'
```
The script builds universal CLI, npm package, signed/notarized app zip, appcast, checksums, draft GitHub release, and npm publish.
Use a non-login shell: profile exports can replace current 1Password ASC IDs with stale values while leaving the current `.p8`, producing a misleading `401`.
Notarized releases must sign with `Developer ID Application: Peter Steinberger (Y5PE65HELJ)`, not `Apple Development`. If your shell has `SIGN_IDENTITY` exported for CLI builds, override it for the release command.
@ -114,6 +115,7 @@ Required before closeout:
```bash
npm view @steipete/peekaboo@<version> version dist-tags dist.tarball dist.integrity time --json
(cd /tmp && npm exec --yes --package=@steipete/peekaboo@<version> -- peekaboo --version)
gh release view v<version> --repo openclaw/Peekaboo --json tagName,isDraft,isPrerelease,url,assets,body
xmllint --noout appcast.xml
git status --short --branch
@ -122,6 +124,7 @@ git status --short --branch
Confirm:
- npm version exists and `latest` points to it.
- npm-downloaded CLI reports the release version from a neutral cwd.
- GitHub release/tag/assets exist; release body is from changelog.
- app zip asset exists and appcast points at `v<version>`.
- `appcast.xml` changes are committed and pushed.

View File

@ -35,6 +35,9 @@
- Batch git network ops in groups: commit related repo changes first, then push/pull repos together so submodule gitlinks stay coherent.
- PRs should summarize intent, list test commands executed, mention doc updates, and include screenshots or terminal snippets when behavior changes.
- Never release or publish without an explicit release command.
- Peekaboo releases: follow `$release-peekaboo`; current Mac + existing 1Password credentials first. App Store Connect changes last resort, only after same-item `notarytool history` and non-S3 `submit` both fail.
- Credentialed release wrappers: `bash -c`, never login shells; profile exports can override ASC IDs and mix credentials.
- Published CLI proof: run `npm exec` from `/tmp`; repo cwd may shadow the downloaded package with a local binary.
- During PR triage, keep moving autonomously: fix defects, add obvious scoped features, and rewrite or land what makes sense.
- Before landing every PR, run autoreview until no actionable findings remain and fix or rerun CI until green.

View File

@ -5,6 +5,41 @@ All notable changes to Peekaboo CLI will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [3.5.3] - 2026-06-13
### Fixed
- Public CLI, agent, MCP, and API guidance now treats runtime element IDs as opaque strings to copy exactly instead of implying role-specific ID shapes. Thanks @coygeek for #194.
- JSON-only `peekaboo see` runs without `--path` now keep required screenshots in snapshot storage instead of leaving files on Desktop or exposing their temporary paths. Thanks @coygeek for #196.
- Background element/query/coordinate clicks now pin actions to the requested process and exact window, reject mismatched window/PID selectors and unverifiable snapshots, invalidate implicit latest snapshots without deleting history, and no longer require Event Synthesizing when Accessibility completes the click.
- App launch, open, and inventory commands now use the selected runtime host, fixing sandboxed LaunchServices failures; launch/open preserve `--no-focus` and caller-relative app paths, relaunch preflights and keeps quit/wait/launch in one daemon-held transaction, build-scoped fallback daemons remain reusable and controllable across native/Rosetta execution and executable upgrades, incompatible legacy hosts no longer force sandboxed local fallback, and inventory ignores unrelated input overrides.
- Agent, MCP, script, CLI, and bridge mutations now advance implicit-snapshot watermarks at host-confirmed completion or observation boundaries, keep durable pending barriers across client timeouts/disconnects without hiding the acting command's own snapshot, carry remote script observation certificates, recover safely from PID reuse, ignore unavailable alternate hosts after protecting the selected/local stores, and preserve explicit snapshot history.
## [3.5.2] - 2026-06-13
### Changed
- `peekaboo type` and the MCP `type` tool now default to zero-delay linear typing; supplying `--wpm`/`wpm` still opts into human cadence.
### Fixed
- Synchronized Tachikoma's OpenAI `gpt-5-chat-latest` catalog metadata so configured models apply the correct GPT-5 parameter filtering.
## [3.5.1] - 2026-06-12
### Fixed
- `peekaboo see` now returns at its configured wall-clock deadline when suspended capture or detection work ignores task cancellation, while preserving explicit command cancellation.
## [3.5.0] - 2026-06-12
### Added
- `peekaboo agent` now supports explicit Claude Fable 5 (`claude-fable-5`) selection with 1M context and 128K max output while keeping Anthropic defaults on Opus 4.8 for zero-retention compatibility.
### Changed
- Agent runs now honor the saved `agent.temperature` and `agent.maxTokens` values shared by the CLI and macOS Settings UI, clamp them to each provider's capabilities, infer Fable limits through compatible providers, and omit unsupported sampling parameters for GPT-5 and current Anthropic reasoning models.
- Project, issue, build, release, and app About links now use the canonical `openclaw/Peekaboo` repository.
### Fixed
- Bridge hosts now use atomic lease-backed socket ownership and bounded nonblocking transport, keep Peekaboo.app and the reusable daemon on distinct paths while preserving the healthy app's TCC-backed fallback, preserve lifecycle settings while migrating legacy daemons, prevent MCP from hosting a bridge listener, safely recover stale sockets, and release abandoned client connections instead of wedging. Thanks @Artifact-LV for #184.
- Legacy screen and area capture now fails with a permission or native capture error instead of returning wallpaper-only/redacted pixels from background sessions. Thanks @VishalJ99 for #185.
## [3.4.1] - 2026-06-10
### Fixed

View File

@ -1,11 +1,34 @@
import Commander
import Foundation
import PeekabooAutomationKit
/// Commands or runtime contexts that can specify a preferred capture engine.
protocol CaptureEngineConfigurable: AnyObject {
var captureEngine: String? { get }
}
enum CommanderRuntimeExecutorMessage {
static let snapshotInvalidationWarning =
"Warning: The requested action succeeded, but stale UI snapshots could not be invalidated after retry. " +
"Do not retry the action."
}
enum CommanderRuntimeExecutorError: LocalizedError {
case snapshotCatchUpFailed(any Error)
case mutationBarrierFailed(any Error)
var errorDescription: String? {
switch self {
case let .snapshotCatchUpFailed(error):
"Could not synchronize the selected host's UI snapshot watermark before execution: " +
"the requested command was not executed, so retrying later is safe. " + error.localizedDescription
case let .mutationBarrierFailed(error):
"Could not establish the desktop mutation barrier before execution: " +
"the requested command was not executed, so retrying later is safe. " + error.localizedDescription
}
}
}
@MainActor
enum CommanderRuntimeExecutor {
static func resolveAndRun(arguments: [String]) async throws {
@ -30,11 +53,204 @@ enum CommanderRuntimeExecutor {
setenv("PEEKABOO_CAPTURE_ENGINE", capturePreference, 1)
}
let runtime = await CommandRuntime.makeDefaultAsync(options: runtimeOptions)
try await runtimeCommand.run(using: runtime)
try await self.catchUpSelectedHostIfNeeded(
using: runtime,
required: runtimeOptions.requiresImplicitSnapshotInvalidation ||
runtimeOptions.usesPerToolSnapshotInvalidation
)
try await DeferredCommandOutput.run(
bufferingOutput: runtimeOptions.requiresImplicitSnapshotInvalidation
) {
try await self.runWithImplicitSnapshotInvalidation(
using: runtime,
required: runtimeOptions.requiresImplicitSnapshotInvalidation,
requiresCallerBarrier: runtimeOptions.requiresCallerDesktopMutationBarrier
) {
try await runtimeCommand.run(using: runtime)
}
}
return
}
var plainCommand = command
try await plainCommand.run()
}
static func catchUpSelectedHostIfNeeded(
using runtime: CommandRuntime,
required: Bool
) async throws {
guard required else { return }
try Task.checkCancellation()
let cutoff = runtime.services.snapshots.effectiveImplicitLatestInvalidationWatermark
try Task.checkCancellation()
guard let cutoff else { return }
do {
_ = try await runtime.services.snapshots.invalidateImplicitLatestSnapshot(
through: cutoff,
preserving: nil,
preservedAt: nil
)
try Task.checkCancellation()
} catch let error as CancellationError {
throw error
} catch {
throw CommanderRuntimeExecutorError.snapshotCatchUpFailed(error)
}
}
static func runWithImplicitSnapshotInvalidation<T>(
using runtime: CommandRuntime,
required: Bool,
requiresCallerBarrier: Bool = false,
operation: () async throws -> T
) async throws -> T {
let mutationSequenceAtStart = runtime.interactionMutationTracker.mutationSequence
let needsCallerBarrier = required &&
(runtime.selectedRemoteSocketPath == nil || requiresCallerBarrier)
let createdDurableMutation: Bool
if needsCallerBarrier {
do {
createdDurableMutation = try runtime.interactionMutationTracker.beginDurableMutation()
} catch {
throw CommanderRuntimeExecutorError.mutationBarrierFailed(error)
}
} else {
createdDurableMutation = false
}
let result: T
do {
result = try await runtime.interactionMutationTracker.withPendingDurableMutationVisible(
createdByCurrentCommand: createdDurableMutation,
operation: operation
)
try Task.checkCancellation()
} catch {
_ = await self.invalidateSnapshotsAfterCommandIfNeeded(
using: runtime,
required: required,
succeeded: false,
mutationSequenceAtStart: mutationSequenceAtStart,
createdDurableMutation: createdDurableMutation
)
throw error
}
let hadPendingMutation = required && runtime.interactionMutationTracker.mutationStartedAt != nil
let invalidated = await invalidateSnapshotsAfterCommandIfNeeded(
using: runtime,
required: required,
succeeded: true,
mutationSequenceAtStart: mutationSequenceAtStart,
createdDurableMutation: createdDurableMutation
)
do {
try Task.checkCancellation()
} catch {
if hadPendingMutation {
_ = await self.invalidateSnapshots(
using: runtime,
reason: "command cancellation",
through: Date(),
preserving: nil,
preservedAt: nil
)
}
throw error
}
if !invalidated {
fputs("\(CommanderRuntimeExecutorMessage.snapshotInvalidationWarning)\n", stderr)
}
return result
}
private static func invalidateSnapshotsAfterCommandIfNeeded(
using runtime: CommandRuntime,
required: Bool,
succeeded: Bool,
mutationSequenceAtStart: UInt64,
createdDurableMutation: Bool
) async -> Bool {
let completion = Date()
guard required else { return true }
guard runtime.interactionMutationTracker.mutationStartedAt != nil else {
guard createdDurableMutation else {
return !runtime.interactionMutationTracker.hasPendingDurableMutation
}
do {
try runtime.interactionMutationTracker.cancelDurableMutation()
return true
} catch {
return false
}
}
guard let requestedCutoff = runtime.interactionMutationTracker.invalidationCutoff(
commandCompletedAt: completion,
succeeded: succeeded
)
else { return true }
let durableCompletion: DesktopMutationWatermarkStore.MutationCompletion?
do {
if createdDurableMutation,
runtime.interactionMutationTracker.mutationSequence == mutationSequenceAtStart {
try runtime.interactionMutationTracker.cancelDurableMutation()
durableCompletion = nil
} else {
durableCompletion = try runtime.interactionMutationTracker.completeDurableMutation(
through: succeeded ? requestedCutoff : completion
)
}
} catch {
runtime.interactionMutationTracker.markInvalidationFailed(through: completion)
return false
}
let cutoff = max(requestedCutoff, durableCompletion?.cutoff ?? requestedCutoff)
let preservationAllowed = durableCompletion?.allowsObservationPreservation ?? true
let preservedSnapshotID = succeeded && preservationAllowed
? runtime.interactionMutationTracker.preservedSnapshotID
: nil
let preservedAt = preservedSnapshotID == nil
? nil
: runtime.interactionMutationTracker.preservedAt
return await self.invalidateSnapshots(
using: runtime,
reason: "command execution",
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedAt
)
}
private static func invalidateSnapshots(
using runtime: CommandRuntime,
reason: String,
through cutoff: Date,
preserving preservedSnapshotID: String?,
preservedAt: Date?
) async -> Bool {
let targets = runtime.interactionMutationTargets
let isRetry = runtime.interactionMutationTracker.hasFailedInvalidationAttempt
let invalidated = await InteractionObservationInvalidator.invalidateAfterMutation(
targets: targets,
logger: runtime.logger,
reason: reason,
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedAt
)
if invalidated {
return true
}
if isRetry {
return false
}
return await InteractionObservationInvalidator.invalidateAfterMutation(
targets: targets,
logger: runtime.logger,
reason: "\(reason) retry",
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedAt
)
}
}

View File

@ -0,0 +1,248 @@
import Darwin
import Foundation
enum DeferredCommandOutput {
static func run<T>(
bufferingOutput: Bool,
operation: () async throws -> T
) async throws -> T {
guard bufferingOutput else {
return try await operation()
}
let inheritedTerminalOutput = TerminalDetector.standardOutputFileDescriptor
let capture = try FileDescriptorOutputCapture()
let terminalOutput = inheritedTerminalOutput ?? capture.originalStandardOutputDescriptor
let result: T
do {
result = try await TerminalDetector.$standardOutputFileDescriptor.withValue(terminalOutput) {
try await operation()
}
} catch {
let shouldReplay = !(error is CancellationError)
// Preserve the command's primary error even if restoring or replaying output fails.
Logger.shared.flush()
try? capture.finish(replayingOutput: shouldReplay)
throw error
}
Logger.shared.flush()
try capture.finish(replayingOutput: true)
return result
}
}
private nonisolated enum DeferredCommandOutputError: LocalizedError {
case posix(operation: String, code: Int32)
var errorDescription: String? {
switch self {
case let .posix(operation, code):
"Failed to \(operation): \(String(cString: strerror(code)))"
}
}
}
private final nonisolated class FileDescriptorOutputCapture {
private var stdoutCapture: Int32 = -1
private var stderrCapture: Int32 = -1
private var originalStdout: Int32 = -1
private var originalStderr: Int32 = -1
private var stdoutRedirected = false
private var stderrRedirected = false
private var finished = false
var originalStandardOutputDescriptor: Int32 {
self.originalStdout
}
init() throws {
// Keep output emitted before this command outside its deferred transaction.
_ = fflush(nil)
do {
self.stdoutCapture = try Self.makeTemporaryFile(named: "stdout")
self.stderrCapture = try Self.makeTemporaryFile(named: "stderr")
self.originalStdout = try Self.duplicate(STDOUT_FILENO, named: "stdout")
self.originalStderr = try Self.duplicate(STDERR_FILENO, named: "stderr")
try Self.redirect(self.stdoutCapture, to: STDOUT_FILENO, named: "stdout")
self.stdoutRedirected = true
try Self.redirect(self.stderrCapture, to: STDERR_FILENO, named: "stderr")
self.stderrRedirected = true
} catch {
self.restoreIgnoringErrors()
self.closeDescriptors()
throw error
}
}
deinit {
guard !self.finished else { return }
_ = fflush(nil)
self.restoreIgnoringErrors()
self.closeDescriptors()
}
func finish(replayingOutput: Bool) throws {
guard !self.finished else { return }
_ = fflush(nil)
try self.restore()
defer {
self.finished = true
self.closeDescriptors()
}
if replayingOutput {
try Self.replay(from: self.stdoutCapture, to: self.originalStdout, named: "stdout")
try Self.replay(from: self.stderrCapture, to: self.originalStderr, named: "stderr")
}
}
private func restore() throws {
var firstError: (any Error)?
if self.stdoutRedirected {
do {
try Self.redirect(self.originalStdout, to: STDOUT_FILENO, named: "stdout")
self.stdoutRedirected = false
} catch {
firstError = error
}
}
if self.stderrRedirected {
do {
try Self.redirect(self.originalStderr, to: STDERR_FILENO, named: "stderr")
self.stderrRedirected = false
} catch {
firstError = firstError ?? error
}
}
if let firstError {
throw firstError
}
}
private func restoreIgnoringErrors() {
if self.stdoutRedirected, dup2(self.originalStdout, STDOUT_FILENO) != -1 {
self.stdoutRedirected = false
}
if self.stderrRedirected, dup2(self.originalStderr, STDERR_FILENO) != -1 {
self.stderrRedirected = false
}
}
private func closeDescriptors() {
Self.close(&self.stdoutCapture)
Self.close(&self.stderrCapture)
Self.close(&self.originalStdout)
Self.close(&self.originalStderr)
}
private static func makeTemporaryFile(named stream: String) throws -> Int32 {
var template = Array("\(NSTemporaryDirectory())peekaboo-\(stream).XXXXXX".utf8CString)
let descriptor = template.withUnsafeMutableBufferPointer { buffer -> Int32 in
guard let baseAddress = buffer.baseAddress else { return -1 }
let descriptor = mkstemp(baseAddress)
if descriptor != -1 {
_ = unlink(baseAddress)
}
return descriptor
}
guard descriptor != -1 else {
throw DeferredCommandOutputError.posix(
operation: "create deferred \(stream) output",
code: errno
)
}
Self.setCloseOnExec(descriptor)
return descriptor
}
private static func duplicate(_ descriptor: Int32, named stream: String) throws -> Int32 {
let duplicate = dup(descriptor)
guard duplicate != -1 else {
throw DeferredCommandOutputError.posix(
operation: "duplicate \(stream)",
code: errno
)
}
Self.setCloseOnExec(duplicate)
return duplicate
}
private static func redirect(_ source: Int32, to destination: Int32, named stream: String) throws {
guard dup2(source, destination) != -1 else {
throw DeferredCommandOutputError.posix(
operation: "redirect \(stream)",
code: errno
)
}
}
private static func replay(from source: Int32, to destination: Int32, named stream: String) throws {
guard lseek(source, 0, SEEK_SET) != -1 else {
throw DeferredCommandOutputError.posix(
operation: "rewind deferred \(stream) output",
code: errno
)
}
var buffer = [UInt8](repeating: 0, count: 64 * 1024)
while true {
let bytesRead = buffer.withUnsafeMutableBytes { bytes in
Darwin.read(source, bytes.baseAddress, bytes.count)
}
if bytesRead == 0 {
return
}
if bytesRead == -1 {
if errno == EINTR {
continue
}
throw DeferredCommandOutputError.posix(
operation: "read deferred \(stream) output",
code: errno
)
}
var offset = 0
while offset < bytesRead {
let bytesWritten = buffer.withUnsafeBytes { bytes in
Darwin.write(
destination,
bytes.baseAddress?.advanced(by: offset),
bytesRead - offset
)
}
if bytesWritten == -1 {
if errno == EINTR {
continue
}
throw DeferredCommandOutputError.posix(
operation: "replay deferred \(stream) output",
code: errno
)
}
offset += bytesWritten
}
}
}
private static func setCloseOnExec(_ descriptor: Int32) {
let flags = fcntl(descriptor, F_GETFD)
if flags != -1 {
_ = fcntl(descriptor, F_SETFD, flags | FD_CLOEXEC)
}
}
private static func close(_ descriptor: inout Int32) {
guard descriptor != -1 else { return }
_ = Darwin.close(descriptor)
descriptor = -1
}
}

View File

@ -28,14 +28,18 @@ struct TerminalCapabilities {
/// Terminal detection utilities following modern CLI best practices
enum TerminalDetector {
@TaskLocal
static var standardOutputFileDescriptor: Int32?
/// Detect comprehensive terminal capabilities
static func detectCapabilities() -> TerminalCapabilities {
// Detect comprehensive terminal capabilities
let isInteractive = self.isInteractiveTerminal()
let (width, height) = self.getTerminalDimensions()
let outputFileDescriptor = self.standardOutputFileDescriptor ?? STDOUT_FILENO
let isInteractive = self.isInteractiveTerminal(outputFileDescriptor)
let (width, height) = self.getTerminalDimensions(outputFileDescriptor)
let termType = ProcessInfo.processInfo.environment["TERM"]
let isCI = self.isCIEnvironment()
let isPiped = self.isPipedOutput()
let isPiped = self.isPipedOutput(outputFileDescriptor)
let supportsColors = self.detectColorSupport(termType: termType, isInteractive: isInteractive)
let supportsTrueColor = self.detectTrueColorSupport()
@ -54,15 +58,15 @@ enum TerminalDetector {
// MARK: - Core Detection Methods
/// Check if stdout is connected to an interactive terminal
private static func isInteractiveTerminal() -> Bool {
private static func isInteractiveTerminal(_ outputFileDescriptor: Int32) -> Bool {
// Check if stdout is connected to an interactive terminal
isatty(STDOUT_FILENO) != 0
isatty(outputFileDescriptor) != 0
}
/// Check if output is being piped or redirected
private static func isPipedOutput() -> Bool {
private static func isPipedOutput(_ outputFileDescriptor: Int32) -> Bool {
// Check if output is being piped or redirected
isatty(STDOUT_FILENO) == 0
isatty(outputFileDescriptor) == 0
}
/// Detect CI/automation environments
@ -79,7 +83,7 @@ enum TerminalDetector {
"AZURE_PIPELINES", "TF_BUILD",
"BITBUCKET_COMMIT", "BITBUCKET_BUILD_NUMBER",
"DRONE", "DRONE_BUILD_NUMBER",
"SEMAPHORE", "SEMAPHORE_BUILD_NUMBER"
"SEMAPHORE", "SEMAPHORE_BUILD_NUMBER",
]
let env = ProcessInfo.processInfo.environment
@ -87,11 +91,11 @@ enum TerminalDetector {
}
/// Get terminal dimensions using ioctl
private static func getTerminalDimensions() -> (width: Int, height: Int) {
private static func getTerminalDimensions(_ outputFileDescriptor: Int32) -> (width: Int, height: Int) {
// Get terminal dimensions using ioctl
var windowSize = winsize()
guard ioctl(STDOUT_FILENO, TIOCGWINSZ, &windowSize) == 0 else {
guard ioctl(outputFileDescriptor, TIOCGWINSZ, &windowSize) == 0 else {
// Fallback to environment variables
let width = Int(ProcessInfo.processInfo.environment["COLUMNS"] ?? "80") ?? 80
let height = Int(ProcessInfo.processInfo.environment["LINES"] ?? "24") ?? 24
@ -120,7 +124,7 @@ enum TerminalDetector {
if let term = termType {
let colorTermPatterns = [
"color", "256color", "truecolor", "24bit",
"xterm-256", "screen-256", "tmux-256"
"xterm-256", "screen-256", "tmux-256",
]
if colorTermPatterns.contains(where: term.contains) {
@ -131,7 +135,7 @@ enum TerminalDetector {
let colorTerminals = [
"xterm", "screen", "tmux", "rxvt", "konsole",
"gnome", "mate", "xfce", "terminology", "kitty",
"alacritty", "iterm", "hyper", "vscode"
"alacritty", "iterm", "hyper", "vscode",
]
if colorTerminals.contains(where: term.contains) {
@ -163,7 +167,7 @@ enum TerminalDetector {
if let term = env["TERM"] {
let trueColorTerminals = [
"iterm", "kitty", "alacritty", "wezterm",
"hyper", "vscode", "gnome-terminal"
"hyper", "vscode", "gnome-terminal",
]
return trueColorTerminals.contains(where: term.contains)
}

View File

@ -18,17 +18,21 @@ extension AgentCommand {
.first
.map { String($0).lowercased() }
if let configuration {
if let configuredModel = PeekabooAIService(configuration: configuration).resolveConfiguredModel(trimmed),
case .custom = configuredModel {
return configuredModel.supportsTools ? configuredModel : nil
}
if trimmed.caseInsensitiveCompare("claude") == .orderedSame ||
trimmed.caseInsensitiveCompare("anthropic") == .orderedSame {
return .anthropic(.opus48)
}
if let explicitProvider,
configuration.listCustomProviders().contains(where: { providerID, provider in
provider.enabled && providerID.caseInsensitiveCompare(explicitProvider) == .orderedSame
}) {
return nil
if let configuration {
switch self.parseConfiguredCustomModel(
trimmed,
explicitProvider: explicitProvider,
configuration: configuration
) {
case let .resolved(model):
return model
case .unresolved:
break
}
}
@ -36,6 +40,11 @@ extension AgentCommand {
return nil
}
return self.supportedParsedModel(parsed, explicitProvider: explicitProvider)
}
@MainActor
private func supportedParsedModel(_ parsed: LanguageModel, explicitProvider: String?) -> LanguageModel? {
switch parsed {
case let .openai(model):
if Self.supportedOpenAIInputs.contains(model) {
@ -43,7 +52,7 @@ extension AgentCommand {
}
case let .anthropic(model):
if Self.supportedAnthropicInputs.contains(model) {
return .anthropic(.opus48)
return .anthropic(model)
}
case let .google(model):
if Self.supportedGoogleInputs.contains(model) {
@ -73,6 +82,32 @@ extension AgentCommand {
return nil
}
@MainActor
private func parseConfiguredCustomModel(
_ modelString: String,
explicitProvider: String?,
configuration: PeekabooCore.ConfigurationManager
) -> ConfiguredModelResolution {
if let configuredModel = PeekabooAIService(configuration: configuration).resolveConfiguredModel(modelString),
case .custom = configuredModel {
return .resolved(configuredModel.supportsTools ? configuredModel : nil)
}
if let explicitProvider,
configuration.listCustomProviders().contains(where: { providerID, provider in
provider.enabled && providerID.caseInsensitiveCompare(explicitProvider) == .orderedSame
}) {
return .resolved(nil)
}
return .unresolved
}
private enum ConfiguredModelResolution {
case resolved(LanguageModel?)
case unresolved
}
@MainActor
func validatedModelSelection(configuration: PeekabooCore.ConfigurationManager? = nil) throws -> LanguageModel? {
guard let modelString = self.model else { return nil }
@ -96,6 +131,7 @@ extension AgentCommand {
]
private static let supportedAnthropicInputs: Set<LanguageModel.Anthropic> = [
.fable5,
.opus48,
.opus47,
.opus45,

View File

@ -89,7 +89,7 @@ struct AgentCommand: RuntimeOptionsConfigurable {
@Option(
name: .long,
help: """
AI model to use (for example: gpt-5.5, claude-opus-4-8, \
AI model to use (for example: gpt-5.5, claude-fable-5, \
gemini-3.5-flash, grok-4.3, minimax-m2.7, minimax-cn/m2.7, \
ollama/<model>, lmstudio/<model>, or <custom-provider>/<model>)
"""
@ -221,6 +221,8 @@ extension AgentCommand {
let configuredAIService = PeekabooAIService(configuration: services.configuration)
let existingAgent = services.agent as? PeekabooAgentService
let mutationCoordinator = runtime.toolSnapshotMutationCoordinator
existingAgent?.configureSnapshotMutationCoordinator(mutationCoordinator)
let existingAgentModel = existingAgent.flatMap {
configuredAIService.resolveConfiguredModel($0.defaultModelSelection) ??
LanguageModel.parse(from: $0.defaultModelSelection)
@ -236,7 +238,11 @@ extension AgentCommand {
let agentService: any AgentServiceProtocol = if let existing = existingAgent {
existing
} else {
try PeekabooAgentService(services: services, defaultModel: listingModel)
try PeekabooAgentService(
services: services,
defaultModel: listingModel,
snapshotMutationCoordinator: mutationCoordinator
)
}
try await self.showSessions(agentService)
return
@ -263,7 +269,11 @@ extension AgentCommand {
let agentService: any AgentServiceProtocol = if let existing = existingAgent {
existing
} else {
try PeekabooAgentService(services: services, defaultModel: selectedModel)
try PeekabooAgentService(
services: services,
defaultModel: selectedModel,
snapshotMutationCoordinator: mutationCoordinator
)
}
let terminalCapabilities = TerminalDetector.detectCapabilities()

View File

@ -7,7 +7,8 @@ import PeekabooFoundation
extension SeeCommand {
func detectElements(
imageData: Data,
windowContext: WindowContext?
windowContext: WindowContext?,
snapshotID: String? = nil
) async throws -> ElementDetectionResult {
self.logger.operationStart("element_detection")
defer { self.logger.operationComplete("element_detection") }
@ -22,7 +23,9 @@ extension SeeCommand {
automation: self.services.automation,
imageData: imageData,
windowContext: windowContext,
timeoutSeconds: timeoutSeconds
timeoutSeconds: timeoutSeconds,
snapshotID: snapshotID,
interactionMutationTracker: self.resolvedRuntime.observationTimeoutMutationTracker
)
} catch is TimeoutError {
throw CaptureError.detectionTimedOut(timeoutSeconds)
@ -44,13 +47,18 @@ extension SeeCommand {
automation: any UIAutomationServiceProtocol,
imageData: Data,
windowContext: WindowContext?,
timeoutSeconds: TimeInterval
timeoutSeconds: TimeInterval,
snapshotID: String? = nil,
interactionMutationTracker: InteractionMutationTracker? = nil
) async throws -> ElementDetectionResult {
try await withWallClockTimeout(seconds: timeoutSeconds) {
try await withWallClockTimeout(
seconds: timeoutSeconds,
interactionMutationTracker: interactionMutationTracker
) {
if let timeoutAdjustingAutomation = automation as? any DetectElementsRequestTimeoutAdjusting {
return try await timeoutAdjustingAutomation.detectElements(
in: imageData,
snapshotId: nil,
snapshotId: snapshotID,
windowContext: windowContext,
requestTimeoutSec: Self.remoteDetectionRequestTimeoutSeconds(for: timeoutSeconds)
)
@ -58,7 +66,7 @@ extension SeeCommand {
return try await AutomationServiceBridge.detectElements(
automation: automation,
imageData: imageData,
snapshotId: nil,
snapshotId: snapshotID,
windowContext: windowContext
)
}

View File

@ -5,7 +5,17 @@ import PeekabooFoundation
@MainActor
extension SeeCommand {
func screenshotOutputPath() -> String {
var usesTemporaryScreenshotOutput: Bool {
self.jsonOutput && self.path == nil
}
func screenshotOutputPath(snapshotID: String? = nil) -> String {
if self.usesTemporaryScreenshotOutput {
return self.temporaryScreenshotDirectory(snapshotID: snapshotID)
.appendingPathComponent("raw.png")
.path
}
let timestamp = Date().timeIntervalSince1970
let filename = "peekaboo_see_\(Int(timestamp)).png"
return ObservationCommandSupport.outputPath(
@ -16,8 +26,8 @@ extension SeeCommand {
)
}
func saveScreenshot(_ imageData: Data) throws -> String {
let outputPath = self.screenshotOutputPath()
func saveScreenshot(_ imageData: Data, snapshotID: String) throws -> String {
let outputPath = self.screenshotOutputPath(snapshotID: snapshotID)
let directory = (outputPath as NSString).deletingLastPathComponent
try FileManager.default.createDirectory(
@ -31,6 +41,17 @@ extension SeeCommand {
return outputPath
}
func cleanupTemporaryScreenshotOutput(snapshotID: String) {
guard self.usesTemporaryScreenshotOutput else { return }
try? FileManager.default.removeItem(at: self.temporaryScreenshotDirectory(snapshotID: snapshotID))
}
private func temporaryScreenshotDirectory(snapshotID: String?) -> URL {
FileManager.default.temporaryDirectory
.appendingPathComponent("peekaboo-see", isDirectory: true)
.appendingPathComponent(snapshotID ?? UUID().uuidString, isDirectory: true)
}
func resolveSeeWindowIndex(appIdentifier: String, titleFragment: String?) async throws -> Int? {
guard let fragment = titleFragment, !fragment.isEmpty else {
return nil

View File

@ -4,8 +4,10 @@ import PeekabooCore
@available(macOS 14.0, *)
@MainActor
extension SeeCommand {
func performCaptureWithDetection() async throws -> CaptureAndDetectionResult {
if let observationResult = try await self.performObservationCaptureWithDetectionIfPossible() {
func performCaptureWithDetection(snapshotID: String) async throws -> CaptureAndDetectionResult {
if let observationResult = try await self.performObservationCaptureWithDetectionIfPossible(
snapshotID: snapshotID
) {
return observationResult
}
@ -13,7 +15,7 @@ extension SeeCommand {
let captureResult = captureContext.captureResult
self.logger.startTimer("file_write")
let outputPath = try saveScreenshot(captureResult.imageData)
let outputPath = try saveScreenshot(captureResult.imageData, snapshotID: snapshotID)
self.logger.stopTimer("file_write")
let windowContext = WindowContext(
@ -27,10 +29,14 @@ extension SeeCommand {
traversalBudget: self.axTraversalBudget()
)
let detectionResult = try await self.detectElements(for: captureContext, windowContext: windowContext)
let detectionResult = try await self.detectElements(
for: captureContext,
windowContext: windowContext,
snapshotID: snapshotID
)
let resultWithPath = ElementDetectionResult(
snapshotId: detectionResult.snapshotId,
snapshotId: snapshotID,
screenshotPath: outputPath,
elements: detectionResult.elements,
metadata: detectionResult.metadata
@ -38,7 +44,7 @@ extension SeeCommand {
try await self.services.snapshots.storeScreenshot(
SnapshotScreenshotRequest(
snapshotId: detectionResult.snapshotId,
snapshotId: snapshotID,
screenshotPath: outputPath,
applicationBundleId: captureResult.metadata.applicationInfo?.bundleIdentifier,
applicationProcessId: captureResult.metadata.applicationInfo.map { Int32($0.processIdentifier) },
@ -49,12 +55,12 @@ extension SeeCommand {
)
try await self.services.snapshots.storeDetectionResult(
snapshotId: detectionResult.snapshotId,
snapshotId: snapshotID,
result: resultWithPath
)
return CaptureAndDetectionResult(
snapshotId: detectionResult.snapshotId,
snapshotId: snapshotID,
screenshotPath: outputPath,
annotatedPath: nil,
elements: detectionResult.elements,
@ -65,7 +71,8 @@ extension SeeCommand {
private func detectElements(
for captureContext: CaptureContext,
windowContext: WindowContext
windowContext: WindowContext,
snapshotID: String
) async throws -> ElementDetectionResult {
let captureResult = captureContext.captureResult
let detectionStart = Date()
@ -87,20 +94,29 @@ extension SeeCommand {
isDialog: false
)
return ElementDetectionResult(
snapshotId: UUID().uuidString,
snapshotId: snapshotID,
screenshotPath: "",
elements: DetectedElements(other: ocrElements),
metadata: metadata
)
}
return try await self.detectElements(
let detectionResult = try await self.detectElements(
imageData: captureResult.imageData,
windowContext: windowContext
windowContext: windowContext,
snapshotID: snapshotID
)
return ElementDetectionResult(
snapshotId: snapshotID,
screenshotPath: detectionResult.screenshotPath,
elements: detectionResult.elements,
metadata: detectionResult.metadata
)
}
private func performObservationCaptureWithDetectionIfPossible() async throws -> CaptureAndDetectionResult? {
private func performObservationCaptureWithDetectionIfPossible(
snapshotID: String
) async throws -> CaptureAndDetectionResult? {
guard let target = try self.observationTargetForCaptureWithDetectionIfPossible() else {
return nil
}
@ -114,7 +130,7 @@ extension SeeCommand {
let observation: DesktopObservationResult
do {
observation = try await self.services.desktopObservation
.observe(self.makeObservationRequest(target: target))
.observe(self.makeObservationRequest(target: target, snapshotID: snapshotID))
} catch DesktopObservationError.targetNotFound(_) where self.menubar {
self.logger.verbose("No observation-backed menu bar popover found; falling back", category: "Capture")
self.logger.operationComplete("capture_phase", success: false, metadata: [
@ -138,7 +154,7 @@ extension SeeCommand {
}
return CaptureAndDetectionResult(
snapshotId: detectionResult.snapshotId,
snapshotId: snapshotID,
screenshotPath: outputPath,
annotatedPath: observation.files.annotatedScreenshotPath,
elements: detectionResult.elements,

View File

@ -76,7 +76,10 @@ extension SeeCommand {
}
}
func makeObservationRequest(target: DesktopObservationTargetRequest) -> DesktopObservationRequest {
func makeObservationRequest(
target: DesktopObservationTargetRequest,
snapshotID: String? = nil
) -> DesktopObservationRequest {
DesktopObservationRequest(
target: target,
capture: DesktopCaptureOptions(
@ -86,10 +89,11 @@ extension SeeCommand {
),
detection: self.observationDetectionOptions(for: target),
output: DesktopObservationOutputOptions(
path: self.screenshotOutputPath(),
path: self.screenshotOutputPath(snapshotID: snapshotID),
saveRawScreenshot: true,
saveAnnotatedScreenshot: self.annotate && self.allowsAnnotation(for: target),
saveSnapshot: true
saveSnapshot: true,
snapshotID: snapshotID
)
)
}

View File

@ -4,16 +4,17 @@ import PeekabooCore
@available(macOS 14.0, *)
@MainActor
extension SeeCommand {
func renderResults(context: SeeCommandRenderContext) async {
func renderResults(context: SeeCommandRenderContext) throws {
try Task.checkCancellation()
if self.jsonOutput {
await self.outputJSONResults(context: context)
try self.outputJSONResults(context: context)
} else {
await self.outputTextResults(context: context)
try self.outputTextResults(context: context)
}
}
/// Fetches the menu bar summary only when verbose output is requested, with a short timeout.
private func fetchMenuBarSummaryIfEnabled() async -> MenuBarSummary? {
func fetchMenuBarSummaryIfEnabled() async -> MenuBarSummary? {
guard self.verbose else { return nil }
do {
@ -31,27 +32,21 @@ extension SeeCommand {
}
}
/// Timeout helper that is not MainActor-bound, so it can still fire if the main actor is blocked.
/// Drives the deadline independently while the MainActor operation is suspended.
/// Synchronous MainActor calls cannot be preempted.
static func withWallClockTimeout<T: Sendable>(
seconds: TimeInterval,
operation: @escaping @Sendable () async throws -> T
timeoutErrorSeconds: TimeInterval? = nil,
interactionMutationTracker: InteractionMutationTracker? = nil,
operation: @escaping @MainActor @Sendable () async throws -> T
) async throws -> T {
try await withThrowingTaskGroup(of: T.self) { group in
group.addTask {
try await operation()
}
group.addTask {
try await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
throw CaptureError.detectionTimedOut(seconds)
}
guard let result = try await group.next() else {
throw CaptureError.detectionTimedOut(seconds)
}
group.cancelAll()
return result
}
try await withMainActorCommandTimeout(
seconds: seconds,
operationName: "see",
timeoutError: { CaptureError.detectionTimedOut(timeoutErrorSeconds ?? seconds) },
interactionMutationTracker: interactionMutationTracker,
operation: { try await operation() }
)
}
func performAnalysisDetailed(imagePath: String, prompt: String) async throws -> SeeAnalysisData {
@ -60,12 +55,7 @@ extension SeeCommand {
return SeeAnalysisData(provider: res.provider, model: res.model, text: res.text)
}
private func buildMenuSummaryIfNeeded() async -> MenuBarSummary? {
// Placeholder for future UI summary generation; currently unused.
nil
}
private func outputJSONResults(context: SeeCommandRenderContext) async {
private func outputJSONResults(context: SeeCommandRenderContext) throws {
let uiElements: [UIElementSummary] = context.elements.all.map { element in
UIElementSummary(
id: element.id,
@ -84,10 +74,6 @@ extension SeeCommand {
let snapshotPaths = self.snapshotPaths(for: context)
// Menu bar enumeration can be slow or hang on some setups. Only attempt it in verbose
// mode and bound it with a short timeout so JSON output is responsive by default.
let menuSummary = await self.fetchMenuBarSummaryIfEnabled()
let output = SeeResult(
snapshot_id: context.snapshotId,
screenshot_raw: snapshotPaths.raw,
@ -102,7 +88,7 @@ extension SeeCommand {
analysis: context.analysis,
execution_time: context.executionTime,
ui_elements: uiElements,
menu_bar: menuSummary,
menu_bar: context.menuBar,
truncation: SeeTruncationSummary(metadata: context.metadata),
observation: context.observation
)
@ -137,7 +123,8 @@ extension SeeCommand {
return MenuBarSummary(menus: menus)
}
private func outputTextResults(context: SeeCommandRenderContext) async {
private func outputTextResults(context: SeeCommandRenderContext) throws {
try Task.checkCancellation()
print("🖼️ Screenshot saved to: \(context.screenshotPath)")
if let annotatedPath = context.annotatedPath {
print("📝 Annotated screenshot: \(annotatedPath)")
@ -180,17 +167,6 @@ extension SeeCommand {
print("\n📝 Annotated screenshot created")
}
if let menuSummary = await self.buildMenuSummaryIfNeeded() {
print("\n🧭 Menu Bar Summary")
for menu in menuSummary.menus {
print("- \(menu.title) (\(menu.enabled ? "Enabled" : "Disabled"))")
for item in menu.items.prefix(5) {
let shortcut = item.keyboard_shortcut.map { " [\($0)]" } ?? ""
print("\(item.title)\(shortcut)")
}
}
}
print("\nSnapshot ID: \(context.snapshotId)")
let terminalCapabilities = TerminalDetector.detectCapabilities()
@ -200,9 +176,10 @@ extension SeeCommand {
}
private func snapshotPaths(for context: SeeCommandRenderContext) -> SnapshotPaths {
SnapshotPaths(
raw: context.screenshotPath,
annotated: context.annotatedPath ?? "",
let publishesScreenshotPaths = !self.usesTemporaryScreenshotOutput
return SnapshotPaths(
raw: publishesScreenshotPaths ? context.screenshotPath : "",
annotated: publishesScreenshotPaths ? context.annotatedPath ?? "" : "",
map: self.services.snapshots.getSnapshotStoragePath() + "/\(context.snapshotId)/snapshot.json"
)
}

View File

@ -40,6 +40,7 @@ struct SeeCommandRenderContext {
let analysis: SeeAnalysisData?
let executionTime: TimeInterval
let observation: SeeObservationDiagnostics?
let menuBar: MenuBarSummary?
}
struct UIElementSummary: Codable {

View File

@ -79,7 +79,7 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
@RuntimeStorage private var runtime: CommandRuntime?
var runtimeOptions = CommandRuntimeOptions()
private var resolvedRuntime: CommandRuntime {
var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
@ -113,9 +113,11 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let startTime = Date()
let commandStartedAt = Date()
let logger = self.logger
let overallTimeout = TimeInterval(self.timeoutSeconds ?? ((self.analyze == nil) ? 20 : 60))
let mutationCoordinator = runtime.toolSnapshotMutationCoordinator
let snapshotManager = runtime.services.snapshots
logger.operationStart("see_command", metadata: [
"app": self.app ?? "none",
@ -128,25 +130,92 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
let commandCopy = self
do {
runtime.beginInteractionMutation(preservingSnapshotsCreatedAfterBoundary: true)
try await CrossProcessOperationGate.withExclusiveOperation(
named: CrossProcessOperationGate.desktopObservationName
) {
try await withThrowingTaskGroup(of: Void.self) { group in
group.addTask {
try await commandCopy.runImpl(startTime: startTime, logger: logger)
let observationStartedAt = Date()
let observationDeadline = observationStartedAt.addingTimeInterval(overallTimeout)
let scope = MCPToolSnapshotMutationScope(
toolName: "see",
startedAt: observationStartedAt,
effect: .mutationProducingFreshObservation
)
let reservationTimeout = try Self.remainingObservationTimeout(
until: observationDeadline,
overallTimeout: overallTimeout
)
let snapshotID = try await Self.withWallClockTimeout(
seconds: reservationTimeout,
timeoutErrorSeconds: overallTimeout
) {
try await snapshotManager.createSnapshot(pendingAt: observationStartedAt)
}
defer {
if snapshotManager.copiesScreenshotArtifactsIntoStorage {
commandCopy.cleanupTemporaryScreenshotOutput(snapshotID: snapshotID)
}
group.addTask {
try await Task.sleep(nanoseconds: UInt64(overallTimeout * 1_000_000_000))
throw CaptureError.detectionTimedOut(overallTimeout)
}
var observationCompleted = false
do {
let preparationTimeout = try Self.remainingObservationTimeout(
until: observationDeadline,
overallTimeout: overallTimeout
)
let context = try await Self.withWallClockTimeout(
seconds: preparationTimeout,
timeoutErrorSeconds: overallTimeout,
interactionMutationTracker: runtime.observationTimeoutMutationTracker
) {
try await commandCopy.prepareResult(
startTime: commandStartedAt,
logger: logger,
snapshotID: snapshotID
)
}
observationCompleted = true
let publicationTimeout = try Self.remainingObservationTimeout(
until: observationDeadline,
overallTimeout: overallTimeout
)
let published = try await Self.withWallClockTimeout(
seconds: publicationTimeout,
timeoutErrorSeconds: overallTimeout
) {
await mutationCoordinator.completeMutation(
scope.completed(
at: Date(),
preserving: snapshotID,
confirmedMutationCompletedAt: context.metadata.desktopMutationCompletedAt,
observationPreservationAllowed: context.metadata
.desktopMutationPreservationAllowed
),
succeeded: true
)
}
guard published else {
throw PeekabooError.operationError(
message: "Failed to publish the refreshed UI snapshot"
)
}
do {
_ = try await group.next()
group.cancelAll()
} catch {
group.cancelAll()
throw error
try Task.checkCancellation()
try commandCopy.renderResults(context: context)
commandCopy.emitAnnotationStatus(context: context)
logger.operationComplete("see_command", metadata: [
"executionTimeMs": Int(Date().timeIntervalSince(commandStartedAt) * 1000),
"success": true,
])
} catch {
if observationCompleted || !PendingSnapshotCleanupPolicy.shouldPreserveReservation(after: error) {
try? await self.services.snapshots.cleanSnapshot(snapshotId: snapshotID)
}
_ = await mutationCoordinator.completeMutation(
scope.completed(at: Date(), preserving: nil),
succeeded: false
)
throw error
}
}
} catch {
@ -162,13 +231,29 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
}
}
private func runImpl(startTime: Date, logger: Logger) async throws {
private static func remainingObservationTimeout(
until deadline: Date,
overallTimeout: TimeInterval
) throws -> TimeInterval {
let remaining = deadline.timeIntervalSinceNow
guard remaining > 0 else {
throw CaptureError.detectionTimedOut(overallTimeout)
}
return remaining
}
private func prepareResult(
startTime: Date,
logger: Logger,
snapshotID: String
) async throws -> SeeCommandRenderContext {
// ScreenCaptureService performs the authoritative permission check inside each capture path.
// Avoid duplicating that TCC probe here; `see` is often called in latency-sensitive loops.
// Perform capture and element detection
logger.verbose("Starting capture and detection phase", category: "Capture")
let captureResult = try await performCaptureWithDetection()
let captureResult = try await performCaptureWithDetection(snapshotID: snapshotID)
try Task.checkCancellation()
logger.verbose("Capture completed successfully", category: "Capture", metadata: [
"snapshotId": captureResult.snapshotId,
"elementCount": captureResult.elements.all.count,
@ -187,25 +272,19 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
snapshotId: captureResult.snapshotId,
originalPath: captureResult.screenshotPath
)
try Task.checkCancellation()
if let annotatedPath,
annotatedPath != captureResult.screenshotPath {
try await self.services.snapshots.storeAnnotatedScreenshot(
snapshotId: captureResult.snapshotId,
annotatedScreenshotPath: annotatedPath
)
try Task.checkCancellation()
}
logger.operationComplete("generate_annotations", metadata: [
"annotatedPath": annotatedPath ?? "none",
])
}
if self.annotate, annotationsAllowed, annotatedPath == nil, !self.jsonOutput {
print("\(AgentDisplayTokens.Status.warning) No interactive UI elements found to annotate")
} else if self.annotate, annotationsAllowed, let annotatedPath, !self.jsonOutput {
let interactableElements = captureResult.elements.all.filter(\.isEnabled)
print("📝 Created annotated screenshot with \(interactableElements.count) interactive elements")
self.logger.verbose("Annotated screenshot path: \(annotatedPath)")
}
// Perform AI analysis if requested
var analysisResult: SeeAnalysisData?
if let prompt = analyze {
@ -227,6 +306,7 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
imagePath: captureResult.screenshotPath,
prompt: prompt
)
try Task.checkCancellation()
logger.stopTimer("ai_generate")
logger.operationComplete(
"ai_analysis",
@ -238,14 +318,11 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
)
}
// Output results
let executionTime = Date().timeIntervalSince(startTime)
logger.operationComplete("see_command", metadata: [
"executionTimeMs": Int(executionTime * 1000),
"success": true,
])
let menuBarSummary = self.jsonOutput ? await self.fetchMenuBarSummaryIfEnabled() : nil
try Task.checkCancellation()
let context = SeeCommandRenderContext(
let executionTime = Date().timeIntervalSince(startTime)
return SeeCommandRenderContext(
snapshotId: captureResult.snapshotId,
screenshotPath: captureResult.screenshotPath,
annotatedPath: annotatedPath,
@ -253,9 +330,20 @@ struct SeeCommand: ApplicationResolvable, ErrorHandlingCommand, RuntimeOptionsCo
elements: captureResult.elements,
analysis: analysisResult,
executionTime: executionTime,
observation: captureResult.observation
observation: captureResult.observation,
menuBar: menuBarSummary
)
await self.renderResults(context: context)
}
private func emitAnnotationStatus(context: SeeCommandRenderContext) {
let annotationsAllowed = self.allowsAnnotationForCurrentCapture()
if self.annotate, annotationsAllowed, context.annotatedPath == nil, !self.jsonOutput {
print("\(AgentDisplayTokens.Status.warning) No interactive UI elements found to annotate")
} else if self.annotate, annotationsAllowed, let annotatedPath = context.annotatedPath, !self.jsonOutput {
let interactableElements = context.elements.all.filter(\.isEnabled)
print("📝 Created annotated screenshot with \(interactableElements.count) interactive elements")
self.logger.verbose("Annotated screenshot path: \(annotatedPath)")
}
}
func getFileSize(_ path: String) -> Int? {
@ -293,8 +381,8 @@ extension SeeCommand: ParsableCommand {
description: "Capture the frontmost window, print structured output, and save annotations."
),
CommandUsageExample(
command: "peekaboo see --app Safari --window-title \"Login\" --json",
description: "Target a specific Safari window to collect stable element IDs."
command: "peekaboo see --app Safari --window-title \"Login\" --json --path /tmp/safari-login.png",
description: "Target a specific Safari window to collect fresh element IDs and keep the capture artifact in /tmp."
),
CommandUsageExample(
command: "peekaboo see --mode screen --screen-index 0 --analyze 'Summarize the dashboard'",

View File

@ -45,7 +45,9 @@ extension PermissionCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.prepare(using: runtime)
if await self.renderIfAlreadyGranted() { return }
let result = await self.requestScreenRecordingPermission()
let result = await PermissionHelpers.performInteractivePermissionRequest(using: runtime) {
await self.requestScreenRecordingPermission()
}
self.render(result: result)
}
@ -161,7 +163,9 @@ extension PermissionCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.prepare(using: runtime)
if await self.renderIfAlreadyGranted() { return }
let granted = self.promptAccessibilityDialog()
let granted = await PermissionHelpers.performInteractivePermissionRequest(using: runtime) {
self.promptAccessibilityDialog()
}
self.renderAccessibilityResult(granted: granted)
}
@ -280,7 +284,10 @@ extension PermissionCommand {
}
private func requestEventSynthesizingPermission() async throws -> AgentPermissionActionResult {
let result = try await PermissionHelpers.requestEventSynthesizingPermission(services: self.services)
let result = try await PermissionHelpers.requestEventSynthesizingPermission(
services: self.services,
runtime: self.resolvedRuntime
)
return AgentPermissionActionResult(
action: result.action,
source: result.source,

View File

@ -76,22 +76,22 @@ extension ErrorHandlingCommand {
}
private func mapPeekabooErrorToCode(_ error: PeekabooError) -> ErrorCode {
if let lookupCode = self.lookupErrorCode(for: error) {
if let lookupCode = lookupErrorCode(for: error) {
return lookupCode
}
if let permissionCode = self.permissionErrorCode(for: error) {
if let permissionCode = permissionErrorCode(for: error) {
return permissionCode
}
if let timeoutCode = self.timeoutErrorCode(for: error) {
if let timeoutCode = timeoutErrorCode(for: error) {
return timeoutCode
}
if let automationCode = self.automationErrorCode(for: error) {
if let automationCode = automationErrorCode(for: error) {
return automationCode
}
if let inputCode = self.inputErrorCode(for: error) {
if let inputCode = inputErrorCode(for: error) {
return inputCode
}
if let credentialCode = self.credentialErrorCode(for: error) {
if let credentialCode = credentialErrorCode(for: error) {
return credentialCode
}
return .UNKNOWN_ERROR
@ -232,6 +232,15 @@ func errorMessage(for error: any Error) -> String {
return error.localizedDescription
}
func applicationLaunchErrorCode(for error: any Error) -> ErrorCode? {
guard let bridgeError = error as? PeekabooBridgeErrorEnvelope,
bridgeError.code == .notFound
else {
return nil
}
return .APP_NOT_FOUND
}
func errorDetails(for error: any Error) -> String? {
guard let bridgeError = error as? PeekabooBridgeErrorEnvelope else {
return nil

View File

@ -20,11 +20,21 @@ struct CommandRuntimeOptions {
var captureEnginePreference: String?
var inputStrategy: UIInputStrategy?
var preferRemote = true
var remoteIsolationRequested = false
var autoStartDaemon = true
var bridgeSocketPath: String?
var requiresElementActions = false
var requiresInspectAccessibilityTree = false
var requiresBrowserMCP = false
var requiresApplicationLaunchOptions = false
var requiresApplicationRelaunch = false
var requiresSurvivingApplicationHost = false
var requiresHostApplicationInventory = false
var requiresImplicitSnapshotInvalidation = false
var requiresCallerDesktopMutationBarrier = false
var usesPerToolSnapshotInvalidation = false
var requiresExactWindowTargetedClicks = false
var requiresPostEventClickPermission = false
func makeConfiguration() -> CommandRuntime.Configuration {
CommandRuntime.Configuration(
@ -41,7 +51,9 @@ struct CommandRuntimeOptions {
if options.captureEnginePreference == nil,
let captureEngine = Self.captureEnginePreference(environment: environment) {
options.captureEnginePreference = captureEngine
options.preferRemote = false
if !options.requiresApplicationLaunchOptions && !options.requiresHostApplicationInventory {
options.preferRemote = false
}
}
return options
}
@ -73,14 +85,32 @@ struct CommandRuntime {
let configuration: Configuration
let hostDescription: String
let selectedRemoteSocketPath: String?
let selectedRemoteHostProcessIdentifier: pid_t?
let snapshotInvalidationRemoteSocketPaths: [String]
let applicationRelaunchAllowed: Bool
let interactionMutationTracker: InteractionMutationTracker
@MainActor let services: any PeekabooServiceProviding
@MainActor let logger: Logger
@MainActor
var observationTimeoutMutationTracker: InteractionMutationTracker? {
if self.selectedRemoteSocketPath == nil || self.interactionMutationTracker.hasPendingDurableMutation {
return self.interactionMutationTracker
}
return nil
}
@MainActor
init(
configuration: Configuration,
services: any PeekabooServiceProviding,
hostDescription: String = "local (in-process)"
hostDescription: String = "local (in-process)",
selectedRemoteSocketPath: String? = nil,
selectedRemoteHostProcessIdentifier: pid_t? = nil,
snapshotInvalidationRemoteSocketPaths: [String] = [],
applicationRelaunchAllowed: Bool = true,
interactionMutationTracker: InteractionMutationTracker = InteractionMutationTracker()
) {
// Keep Tachikoma credential/profile resolution aligned with Peekaboo CLI storage.
PeekabooCore.ConfigurationManager.configureTachikomaProfileDirectory()
@ -88,6 +118,11 @@ struct CommandRuntime {
self.configuration = configuration
self.services = services
self.hostDescription = hostDescription
self.selectedRemoteSocketPath = selectedRemoteSocketPath
self.selectedRemoteHostProcessIdentifier = selectedRemoteHostProcessIdentifier
self.snapshotInvalidationRemoteSocketPaths = snapshotInvalidationRemoteSocketPaths
self.applicationRelaunchAllowed = applicationRelaunchAllowed
self.interactionMutationTracker = interactionMutationTracker
self.logger = Logger.shared
services.installAgentRuntimeDefaults()
@ -150,15 +185,19 @@ extension CommandRuntime {
@MainActor
static func makeDefaultAsync(options: CommandRuntimeOptions) async -> CommandRuntime {
let effectiveOptions = options.applyingEnvironmentOverrides(environment: ProcessInfo.processInfo.environment)
if let override = self.serviceOverride {
if let override = serviceOverride {
return CommandRuntime(options: effectiveOptions, services: override)
}
let resolution = await self.resolveServices(options: effectiveOptions)
let resolution = await resolveServices(options: effectiveOptions)
return CommandRuntime(
configuration: effectiveOptions.makeConfiguration(),
services: resolution.services,
hostDescription: resolution.hostDescription
hostDescription: resolution.hostDescription,
selectedRemoteSocketPath: resolution.selectedRemoteSocketPath,
selectedRemoteHostProcessIdentifier: resolution.selectedRemoteHostProcessIdentifier,
snapshotInvalidationRemoteSocketPaths: resolution.snapshotInvalidationRemoteSocketPaths,
applicationRelaunchAllowed: resolution.applicationRelaunchAllowed
)
}
@ -178,8 +217,7 @@ extension CommandRuntime {
}
@MainActor
private static func resolveServices(options: CommandRuntimeOptions)
async -> (services: any PeekabooServiceProviding, hostDescription: String) {
private static func resolveServices(options: CommandRuntimeOptions) async -> RuntimeHostResolver.Resolution {
await RuntimeHostResolver.resolveServices(options: options)
}
@ -241,6 +279,18 @@ extension CommandRuntime {
BridgeCapabilityPolicy.supportsTargetedClicks(for: handshake)
}
static func supportsApplicationLaunchOptions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsApplicationLaunchOptions(for: handshake)
}
static func supportsApplicationRelaunch(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsApplicationRelaunch(for: handshake)
}
static func supportsImplicitSnapshotInvalidation(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsImplicitSnapshotInvalidation(for: handshake)
}
static func supportsElementActions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
BridgeCapabilityPolicy.supportsElementActions(for: handshake)
}
@ -285,7 +335,7 @@ protocol RuntimeOptionsConfigurable {
extension RuntimeOptionsConfigurable {
mutating func setRuntimeOptions(_ options: CommandRuntimeOptions) {
self.runtimeOptions = options
runtimeOptions = options
}
}

View File

@ -42,7 +42,8 @@ enum AutomationServiceBridge {
target: ClickTarget,
clickType: ClickType,
snapshotId: String?,
targetProcessIdentifier: pid_t
targetProcessIdentifier: pid_t,
targetWindowID: Int? = nil
) async throws {
try await Task { @MainActor in
guard let targetedClickService = automation as? any TargetedClickServiceProtocol else {
@ -55,12 +56,28 @@ enum AutomationServiceBridge {
throw self.targetedClickUnavailableError(service: targetedClickService)
}
try await targetedClickService.click(
target: target,
clickType: clickType,
snapshotId: snapshotId,
targetProcessIdentifier: targetProcessIdentifier
)
if let targetWindowID {
guard let exactWindowService = targetedClickService as? any ExactWindowTargetedClickServiceProtocol
else {
throw PeekabooError.serviceUnavailable(
"Background clicks with an exact window require a compatible automation service"
)
}
try await exactWindowService.click(
target: target,
clickType: clickType,
snapshotId: snapshotId,
targetProcessIdentifier: targetProcessIdentifier,
targetWindowID: targetWindowID
)
} else {
try await targetedClickService.click(
target: target,
clickType: clickType,
snapshotId: snapshotId,
targetProcessIdentifier: targetProcessIdentifier
)
}
}.value
}

View File

@ -1,5 +1,6 @@
import CoreGraphics
import Foundation
import PeekabooAutomationKit
import PeekabooCore
import PeekabooFoundation
@ -126,7 +127,11 @@ func withCommandTimeout<T: Sendable>(
}
let timeoutTask = Task.detached {
try? await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
do {
try await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
} catch {
return
}
race.resume(with: Result<T, any Error>.failure(PeekabooError.timeout(
operation: operationName,
duration: seconds
@ -139,6 +144,7 @@ func withCommandTimeout<T: Sendable>(
race.setContinuation(continuation)
}
} onCancel: {
race.resume(with: Result<T, any Error>.failure(CancellationError()))
workTask.cancel()
timeoutTask.cancel()
}
@ -148,6 +154,9 @@ func withCommandTimeout<T: Sendable>(
func withMainActorCommandTimeout<T: Sendable>(
seconds: TimeInterval,
operationName: String,
timeoutError: (@Sendable () -> any Error)? = nil,
desktopMutationWatermarkStore: DesktopMutationWatermarkStore? = nil,
interactionMutationTracker: InteractionMutationTracker? = nil,
operation: @escaping @MainActor () async throws -> T
) async throws -> T {
guard seconds > 0 else {
@ -155,21 +164,37 @@ func withMainActorCommandTimeout<T: Sendable>(
}
let race = TimeoutRace()
let workTask = Task { @MainActor in
do {
let value = try await operation()
race.resume(with: .success(value))
} catch {
race.resume(with: Result<T, any Error>.failure(error))
let pendingMutation = try desktopMutationWatermarkStore?.beginMutation()
do {
try interactionMutationTracker?.retainDurableMutationLease()
} catch {
if let desktopMutationWatermarkStore, let pendingMutation {
try? desktopMutationWatermarkStore.cancelMutation(pendingMutation)
}
throw error
}
let workTask = Task { @MainActor in
let result: Result<T, any Error>
do {
result = try await .success(operation())
} catch {
result = .failure(error)
}
if let desktopMutationWatermarkStore, let pendingMutation {
_ = try? desktopMutationWatermarkStore.completeMutation(pendingMutation)
}
_ = try? interactionMutationTracker?.completeDurableMutation(through: Date())
race.resume(with: result)
}
let timeoutTask = Task.detached {
try? await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
race.resume(with: Result<T, any Error>.failure(PeekabooError.timeout(
operation: operationName,
duration: seconds
)))
do {
try await Task.sleep(nanoseconds: UInt64(seconds * 1_000_000_000))
} catch {
return
}
let error = timeoutError?() ?? PeekabooError.timeout(operation: operationName, duration: seconds)
race.resume(with: Result<T, any Error>.failure(error))
workTask.cancel()
}
@ -178,6 +203,7 @@ func withMainActorCommandTimeout<T: Sendable>(
race.setContinuation(continuation)
}
} onCancel: {
race.resume(with: Result<T, any Error>.failure(CancellationError()))
workTask.cancel()
timeoutTask.cancel()
}

View File

@ -10,7 +10,7 @@ enum CommanderCLIBinder {
parsedValues: ParsedValues
) throws -> any ParsableCommand {
var command = type.init()
let runtimeOptions = try self.makeRuntimeOptions(from: parsedValues, commandType: type)
let runtimeOptions = try makeRuntimeOptions(from: parsedValues, commandType: type)
if var bindable = command as? any CommanderBindableCommand {
try bindable.applyCommanderValues(.init(parsedValues: parsedValues))
guard let rebound = bindable as? any ParsableCommand else {
@ -43,6 +43,31 @@ enum CommanderCLIBinder {
commandType: (any ParsableCommand.Type)? = nil
) throws -> CommandRuntimeOptions {
var options = CommandRuntimeOptions()
options.requiresApplicationLaunchOptions = Self.requiresApplicationLaunchOptions(commandType)
options.requiresApplicationRelaunch = commandType == AppCommand.RelaunchSubcommand.self
options.requiresSurvivingApplicationHost = commandType == AppCommand.QuitSubcommand.self
options.requiresHostApplicationInventory = Self.requiresHostApplicationInventory(commandType)
options.requiresImplicitSnapshotInvalidation = Self.requiresImplicitSnapshotInvalidation(
commandType,
parsedValues: parsedValues
)
let clipboardMayMutate = commandType == ClipboardCommand.self &&
Self.clipboardMayMutate(parsedValues)
options.requiresCallerDesktopMutationBarrier = commandType == SwitchSubcommand.self ||
commandType == MoveWindowSubcommand.self ||
commandType == CaptureActionCommand.self ||
clipboardMayMutate
options.requiresExactWindowTargetedClicks = Self.requiresExactWindowTargetedClicks(
commandType,
parsedValues: parsedValues
)
options.requiresPostEventClickPermission = Self.requiresPostEventClickPermission(
commandType,
parsedValues: parsedValues
)
options.usesPerToolSnapshotInvalidation = commandType == AgentCommand.self ||
commandType == MCPCommand.Serve.self ||
commandType == InspectUICommand.self
options.verbose = parsedValues.flags.contains("verbose")
options.jsonOutput = parsedValues.flags.contains("jsonOutput")
let values = CommanderBindableValues(parsedValues: parsedValues)
@ -53,7 +78,9 @@ enum CommanderCLIBinder {
.trimmingCharacters(in: .whitespacesAndNewlines),
!captureEngine.isEmpty {
options.captureEnginePreference = captureEngine
options.preferRemote = false
if !options.requiresApplicationLaunchOptions && !options.requiresHostApplicationInventory {
options.preferRemote = false
}
}
if let rawInputStrategy = values.singleOption("inputStrategy")?
.trimmingCharacters(in: .whitespacesAndNewlines),
@ -66,10 +93,10 @@ enum CommanderCLIBinder {
)
}
options.inputStrategy = strategy
options.preferRemote = false
}
if values.flag("no-remote") {
options.preferRemote = false
options.remoteIsolationRequested = true
}
let explicitBridgeSocket = values.singleOption("bridge-socket")?.trimmingCharacters(in: .whitespacesAndNewlines)
if commandType == AgentCommand.self && !values.flag("no-remote") {
@ -80,8 +107,10 @@ enum CommanderCLIBinder {
options.preferRemote = false
options.autoStartDaemon = false
}
if Self.prefersLocalRuntime(commandType), !values.flag("no-remote"),
explicitBridgeSocket?.isEmpty ?? true {
if Self.requiresCallerLocalRuntime(commandType) {
options.preferRemote = false
} else if Self.prefersLocalRuntime(commandType), !values.flag("no-remote"),
explicitBridgeSocket?.isEmpty ?? true {
options.preferRemote = false
}
if let socketPath = explicitBridgeSocket, !socketPath.isEmpty {
@ -99,6 +128,232 @@ enum CommanderCLIBinder {
return options
}
private static func requiresApplicationLaunchOptions(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == OpenCommand.self ||
commandType == AppCommand.LaunchSubcommand.self ||
commandType == AppCommand.RelaunchSubcommand.self
}
private static func requiresHostApplicationInventory(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == ListCommand.AppsSubcommand.self ||
commandType == AppCommand.ListSubcommand.self
}
private static func requiresImplicitSnapshotInvalidation(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
if commandType == ClipboardCommand.self {
return self.clipboardMayMutate(parsedValues)
}
if commandType == MenuBarCommand.self {
return parsedValues.positional.first?.lowercased() == "click"
}
if commandType == BrowserCommand.self {
return BrowserCommand.actionMayMutate(parsedValues.positional.first ?? "status")
}
if commandType == SeeCommand.self {
return true
}
if self.isInteractivePermissionRequest(commandType) {
return true
}
if commandType == DialogCommand.ListSubcommand.self {
return self.dialogListMayFocus(parsedValues)
}
if commandType == MenuCommand.ListSubcommand.self {
return self.menuListMayFocus(parsedValues)
}
if commandType == ImageCommand.self ||
commandType == CaptureLiveCommand.self ||
commandType == CaptureWatchAlias.self {
return self.captureCommandMayFocus(commandType, parsedValues: parsedValues)
}
return commandType == OpenCommand.self ||
commandType == AppCommand.LaunchSubcommand.self ||
commandType == AppCommand.RelaunchSubcommand.self ||
commandType == AppCommand.QuitSubcommand.self ||
commandType == AppCommand.HideSubcommand.self ||
commandType == AppCommand.UnhideSubcommand.self ||
commandType == AppCommand.SwitchSubcommand.self ||
commandType == ClickCommand.self ||
commandType == MoveCommand.self ||
commandType == TypeCommand.self ||
commandType == PressCommand.self ||
commandType == HotkeyCommand.self ||
commandType == PasteCommand.self ||
commandType == ScrollCommand.self ||
commandType == SwipeCommand.self ||
commandType == DragCommand.self ||
commandType == SetValueCommand.self ||
commandType == PerformActionCommand.self ||
commandType == CaptureActionCommand.self ||
commandType == WindowCommand.FocusSubcommand.self ||
commandType == WindowCommand.CloseSubcommand.self ||
commandType == WindowCommand.MinimizeSubcommand.self ||
commandType == WindowCommand.MaximizeSubcommand.self ||
commandType == WindowCommand.MoveSubcommand.self ||
commandType == WindowCommand.ResizeSubcommand.self ||
commandType == WindowCommand.SetBoundsSubcommand.self ||
commandType == DialogCommand.ClickSubcommand.self ||
commandType == DialogCommand.DismissSubcommand.self ||
commandType == DialogCommand.InputSubcommand.self ||
commandType == DialogCommand.FileSubcommand.self ||
commandType == MenuCommand.ClickSubcommand.self ||
commandType == MenuCommand.ClickExtraSubcommand.self ||
commandType == DockCommand.LaunchSubcommand.self ||
commandType == DockCommand.RightClickSubcommand.self ||
commandType == DockCommand.HideSubcommand.self ||
commandType == DockCommand.ShowSubcommand.self ||
commandType == SwitchSubcommand.self ||
commandType == MoveWindowSubcommand.self ||
commandType == RunCommand.self
}
private static func isInteractivePermissionRequest(
_ commandType: (any ParsableCommand.Type)?
) -> Bool {
commandType == PermissionsCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionsCommand.RequestEventSynthesizingSubcommand.self ||
commandType == PermissionCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestAccessibilitySubcommand.self ||
commandType == PermissionCommand.RequestEventSynthesizingSubcommand.self
}
private static func clipboardMayMutate(_ parsedValues: ParsedValues) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
let positionalAction = values.positionalValue(at: 0)?
.trimmingCharacters(in: .whitespacesAndNewlines)
let action = (positionalAction?.isEmpty == false ? positionalAction : nil) ??
values.singleOption("actionOption") ??
values.singleOption("action")
return ClipboardCommand.actionMayMutate(action)
}
private static func menuListMayFocus(_ parsedValues: ParsedValues) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
return !values.flag("noAutoFocus")
}
private static func dialogListMayFocus(_ parsedValues: ParsedValues) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
let hasWindowTarget = values.singleOption("windowId") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil
if hasWindowTarget {
return true
}
guard !values.flag("noAutoFocus") else { return false }
let app = values.singleOption("app")?
.trimmingCharacters(in: .whitespacesAndNewlines)
return app?.isEmpty == false ||
values.singleOption("pid") != nil
}
private static func captureCommandMayFocus(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
let values = CommanderBindableValues(parsedValues: parsedValues)
let focus = values.singleOption("captureFocus")?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased()
guard focus != "background" else { return false }
let app = values.singleOption("app")?
.trimmingCharacters(in: .whitespacesAndNewlines)
let hasApplicationTarget = app?.isEmpty == false || values.singleOption("pid") != nil
if commandType == ImageCommand.self {
let normalizedApp = app?.lowercased()
guard normalizedApp != "menubar", normalizedApp != "frontmost" else { return false }
let mode = values.singleOption("mode")?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased() ?? Self.inferredImageCaptureMode(values)
switch mode {
case "window":
return values.singleOption("windowId") == nil && hasApplicationTarget
case "multi":
return hasApplicationTarget
default:
return false
}
}
let mode = values.singleOption("mode")?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased() ?? Self.inferredLiveCaptureMode(values)
return mode == "window" && hasApplicationTarget
}
private static func inferredImageCaptureMode(_ values: CommanderBindableValues) -> String {
if values.singleOption("region") != nil { return "area" }
if values.singleOption("app") != nil ||
values.singleOption("pid") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil ||
values.singleOption("windowId") != nil {
return "window"
}
return "frontmost"
}
private static func inferredLiveCaptureMode(_ values: CommanderBindableValues) -> String {
if values.singleOption("region") != nil { return "area" }
if values.singleOption("app") != nil ||
values.singleOption("pid") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil {
return "window"
}
return "frontmost"
}
private static func requiresExactWindowTargetedClicks(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
guard commandType == ClickCommand.self else { return false }
let values = CommanderBindableValues(parsedValues: parsedValues)
guard self.usesBackgroundClickDelivery(values) else { return false }
let hasWindowSelector = values.singleOption("windowId") != nil ||
values.singleOption("windowTitle") != nil ||
values.singleOption("windowIndex") != nil
if hasWindowSelector {
return true
}
let hasProcessTarget = values.singleOption("app") != nil || values.singleOption("pid") != nil
return values.singleOption("coords") != nil && hasProcessTarget && !values.flag("globalCoords")
}
private static func requiresPostEventClickPermission(
_ commandType: (any ParsableCommand.Type)?,
parsedValues: ParsedValues
) -> Bool {
guard commandType == ClickCommand.self else { return false }
let values = CommanderBindableValues(parsedValues: parsedValues)
guard self.usesBackgroundClickDelivery(values) else { return false }
if values.singleOption("coords") != nil {
return true
}
// ClickCommand resolves conflicting flags as right-click first, then double-click.
return values.flag("double") && !values.flag("right")
}
private static func usesBackgroundClickDelivery(_ values: CommanderBindableValues) -> Bool {
if values.flag("focusBackground") { return true }
return !values.flag("foreground") &&
!values.flag("noAutoFocus") &&
!values.flag("spaceSwitch") &&
!values.flag("bringToCurrentSpace") &&
values.singleOption("focusTimeoutSeconds") == nil &&
values.singleOption("focusRetryCount") == nil
}
private static func prefersLocalRuntime(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == MCPCommand.Serve.self ||
commandType == ToolsCommand.self ||
@ -117,9 +372,16 @@ enum CommanderCLIBinder {
commandType == ConfigCommand.TestProviderCommand.self ||
commandType == ConfigCommand.RemoveProviderCommand.self ||
commandType == ConfigCommand.ModelsProviderCommand.self ||
commandType == AppCommand.ListSubcommand.self ||
commandType == ListCommand.AppsSubcommand.self ||
commandType == ListCommand.ScreensSubcommand.self
commandType == ListCommand.ScreensSubcommand.self ||
commandType == PermissionsCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestAccessibilitySubcommand.self
}
private static func requiresCallerLocalRuntime(_ commandType: (any ParsableCommand.Type)?) -> Bool {
commandType == PermissionsCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestScreenRecordingSubcommand.self ||
commandType == PermissionCommand.RequestAccessibilitySubcommand.self
}
private static func isDaemonCommand(_ commandType: (any ParsableCommand.Type)?) -> Bool {

View File

@ -12,15 +12,46 @@ enum BridgeCapabilityPolicy {
return false
}
if options.requiresElementActions && !self.supportsElementActions(for: handshake) {
if options.requiresElementActions, !self.supportsElementActions(for: handshake) {
return false
}
if options.requiresInspectAccessibilityTree && !self.supportsInspectAccessibilityTree(for: handshake) {
if options.requiresInspectAccessibilityTree, !self.supportsInspectAccessibilityTree(for: handshake) {
return false
}
if options.requiresBrowserMCP && !self.supportsBrowserMCP(for: handshake) {
if options.requiresBrowserMCP, !self.supportsBrowserMCP(for: handshake) {
return false
}
if options.requiresApplicationLaunchOptions, !self.supportsApplicationLaunchOptions(for: handshake) {
return false
}
if options.requiresApplicationRelaunch, !self.supportsApplicationRelaunch(for: handshake) {
return false
}
if options.requiresSurvivingApplicationHost, handshake.hostKind != .onDemand {
return false
}
if options.requiresHostApplicationInventory, !self.supportsHostApplicationInventory(for: handshake) {
return false
}
if options.requiresExactWindowTargetedClicks,
!self.supportsExactWindowTargetedClicks(for: handshake) {
return false
}
if options.requiresPostEventClickPermission,
handshake.permissions?.postEvent != true {
return false
}
if options.requiresImplicitSnapshotInvalidation || options.usesPerToolSnapshotInvalidation,
!self.supportsImplicitSnapshotInvalidation(for: handshake) {
return false
}
@ -39,6 +70,45 @@ enum BridgeCapabilityPolicy {
self.targetedClickAvailability(for: handshake).isEnabled
}
static func supportsApplicationLaunchOptions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 9) &&
handshake.supportedOperations.contains(.launchApplicationWithOptions)
}
static func supportsApplicationRelaunch(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
guard handshake.hostKind == .onDemand,
handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 9),
handshake.supportedOperations.contains(.relaunchApplicationWithOptions)
else {
return false
}
let enabledOperations = handshake.enabledOperations ?? handshake.supportedOperations
return enabledOperations.contains(.relaunchApplicationWithOptions)
}
static func supportsHostApplicationInventory(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
guard handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 0),
handshake.supportedOperations.contains(.listApplications)
else {
return false
}
let enabledOperations = handshake.enabledOperations ?? handshake.supportedOperations
return enabledOperations.contains(.listApplications)
}
static func supportsImplicitSnapshotInvalidation(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
guard handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 9),
handshake.supportedOperations.contains(.invalidateImplicitLatestSnapshot)
else {
return false
}
let enabledOperations = handshake.enabledOperations ?? handshake.supportedOperations
return enabledOperations.contains(.invalidateImplicitLatestSnapshot)
}
static func supportsElementActions(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 3) &&
handshake.supportedOperations.contains(.setValue) &&
@ -82,7 +152,7 @@ enum BridgeCapabilityPolicy {
return (true, nil, [])
}
let missingPermissions = self.missingPermissions(for: .targetedHotkey, handshake: handshake)
let missingPermissions = missingPermissions(for: .targetedHotkey, handshake: handshake)
guard !missingPermissions.isEmpty else {
return (
false,
@ -110,10 +180,25 @@ enum BridgeCapabilityPolicy {
let enabledOperations = handshake.enabledOperations ?? handshake.supportedOperations
if enabledOperations.contains(.targetedClick) {
return (true, nil, [])
let missingVariantPermissions: Set<PeekabooBridgePermissionKind> =
handshake.permissions?.postEvent == false ? [.postEvent] : []
return (true, nil, missingVariantPermissions)
}
let missingPermissions = self.missingPermissions(for: .targetedClick, handshake: handshake)
let requestAwarePermissions =
handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 9) &&
handshake.permissionTags[PeekabooBridgeOperation.targetedClick.rawValue]?.isEmpty == true
if requestAwarePermissions,
handshake.permissions?.accessibility == false,
handshake.permissions?.postEvent == false {
return (
false,
"Remote bridge host background clicks require Accessibility or Event Synthesizing permission",
[]
)
}
let missingPermissions = missingPermissions(for: .targetedClick, handshake: handshake)
guard !missingPermissions.isEmpty else {
return (
false,
@ -130,6 +215,16 @@ enum BridgeCapabilityPolicy {
)
}
static func supportsExactWindowTargetedClicks(for handshake: PeekabooBridgeHandshakeResponse) -> Bool {
guard handshake.negotiatedVersion >= PeekabooBridgeProtocolVersion(major: 1, minor: 9),
handshake.supportedOperations.contains(.exactWindowTargetedClick)
else {
return false
}
return (handshake.enabledOperations ?? handshake.supportedOperations)
.contains(.exactWindowTargetedClick)
}
static func targetedTypeAvailability(for handshake: PeekabooBridgeHandshakeResponse)
-> (isEnabled: Bool, unavailableReason: String?, missingPermissions: Set<PeekabooBridgePermissionKind>) {
guard
@ -144,7 +239,7 @@ enum BridgeCapabilityPolicy {
return (true, nil, [])
}
let missingPermissions = self.missingPermissions(for: .targetedTypeActions, handshake: handshake)
let missingPermissions = missingPermissions(for: .targetedTypeActions, handshake: handshake)
guard !missingPermissions.isEmpty else {
return (
false,
@ -168,7 +263,7 @@ enum BridgeCapabilityPolicy {
let requiredPermissions = Set(
handshake.permissionTags[operation.rawValue] ?? Array(operation.requiredPermissions)
)
let grantedPermissions = self.grantedPermissions(from: handshake.permissions)
let grantedPermissions = grantedPermissions(from: handshake.permissions)
return requiredPermissions.subtracting(grantedPermissions)
}

View File

@ -1,8 +1,34 @@
import Darwin
import Foundation
import MachO
import PeekabooBridge
enum DaemonLaunchPolicy {
enum ImplicitRuntimeCandidateRole: Equatable {
case reusableDaemon
case defaultAppFallback
}
struct LaunchResult {
let status: PeekabooDaemonStatus
let processID: pid_t
var ownsObservedDaemon: Bool {
self.status.pid == self.processID
}
}
enum SocketAvailability: Equatable {
case available
case reusableDaemon
case timedOut
}
enum LegacyStopRaceResolution: Equatable {
case keepReplacement
case useLegacy(socketPath: String)
}
static func shouldAutoStartDaemon(
options: CommandRuntimeOptions,
environment: [String: String]
@ -16,7 +42,201 @@ enum DaemonLaunchPolicy {
!socket.isEmpty {
return socket
}
return PeekabooBridgeConstants.peekabooSocketPath
return PeekabooBridgeConstants.daemonSocketPath
}
static func runtimeBuildIdentity(
executableURL: URL? = Bundle.main.executableURL,
executableUUIDProvider: (URL) -> [String] = executableUUIDs
) -> String {
let protocolVersion = PeekabooBridgeConstants.protocolVersion
let identityPrefix = "\(protocolVersion.major).\(protocolVersion.minor)|" +
PeekabooBridgeConstants.buildIdentifier
let resolvedURL = executableURL?.resolvingSymlinksInPath()
if let resolvedURL {
let executableUUIDs = executableUUIDProvider(resolvedURL).sorted()
if !executableUUIDs.isEmpty {
return "\(identityPrefix)|\(executableUUIDs.joined(separator: ","))"
}
}
let executablePath = resolvedURL?.path ?? CommandLine.arguments.first ?? "unknown"
let attributes = try? FileManager.default.attributesOfItem(atPath: executablePath)
let fileSize = (attributes?[.size] as? NSNumber)?.uint64Value ?? 0
let modificationBits = (attributes?[.modificationDate] as? Date)?
.timeIntervalSinceReferenceDate.bitPattern ?? 0
return [
identityPrefix,
executablePath,
"\(fileSize)",
"\(modificationBits)",
].joined(separator: "|")
}
private enum ByteOrder {
case little
case big
}
private nonisolated static func executableUUIDs(_ executableURL: URL) -> [String] {
guard let data = try? Data(contentsOf: executableURL, options: .mappedIfSafe) else {
return []
}
return self.machoUUIDs(in: data)
}
nonisolated static func machoUUIDs(in data: Data) -> [String] {
guard let magic = readUInt32(data, at: 0, order: .little) else { return [] }
switch magic {
case UInt32(FAT_CIGAM), UInt32(FAT_CIGAM_64):
return self.fatMachOUUIDs(
in: data,
order: .big,
uses64BitArchitectureRecords: magic == UInt32(FAT_CIGAM_64)
)
case UInt32(FAT_MAGIC), UInt32(FAT_MAGIC_64):
return self.fatMachOUUIDs(
in: data,
order: .little,
uses64BitArchitectureRecords: magic == UInt32(FAT_MAGIC_64)
)
default:
return self.machOUUID(in: data, sliceOffset: 0).map { [$0] } ?? []
}
}
private nonisolated static func fatMachOUUIDs(
in data: Data,
order: ByteOrder,
uses64BitArchitectureRecords: Bool
) -> [String] {
guard let architectureCount = readUInt32(data, at: 4, order: order) else { return [] }
let recordSize = uses64BitArchitectureRecords ? 32 : 20
guard architectureCount <= 64 else { return [] }
var uuids: [String] = []
for index in 0..<Int(architectureCount) {
let recordOffset = 8 + index * recordSize
let rawSliceOffset: UInt64? = if uses64BitArchitectureRecords {
self.readUInt64(data, at: recordOffset + 8, order: order)
} else {
self.readUInt32(data, at: recordOffset + 8, order: order).map(UInt64.init)
}
guard let rawSliceOffset, rawSliceOffset <= UInt64(Int.max) else { return [] }
if let uuid = machOUUID(in: data, sliceOffset: Int(rawSliceOffset)) {
uuids.append(uuid)
}
}
return uuids
}
private nonisolated static func machOUUID(in data: Data, sliceOffset: Int) -> String? {
guard let magic = readUInt32(data, at: sliceOffset, order: .little) else { return nil }
let order: ByteOrder
let headerSize: Int
switch magic {
case UInt32(MH_MAGIC):
order = .little
headerSize = 28
case UInt32(MH_MAGIC_64):
order = .little
headerSize = 32
case UInt32(MH_CIGAM):
order = .big
headerSize = 28
case UInt32(MH_CIGAM_64):
order = .big
headerSize = 32
default:
return nil
}
guard let commandCount = readUInt32(data, at: sliceOffset + 16, order: order),
let commandBytes = readUInt32(data, at: sliceOffset + 20, order: order),
commandCount <= 16384
else { return nil }
var commandOffset = sliceOffset + headerSize
let commandsEnd = commandOffset + Int(commandBytes)
guard commandsEnd >= commandOffset, commandsEnd <= data.count else { return nil }
for _ in 0..<Int(commandCount) {
guard let command = readUInt32(data, at: commandOffset, order: order),
let rawCommandSize = readUInt32(data, at: commandOffset + 4, order: order)
else { return nil }
let commandSize = Int(rawCommandSize)
guard commandSize >= 8, commandOffset + commandSize <= commandsEnd else { return nil }
if command == UInt32(LC_UUID), commandSize >= 24 {
let uuidRange = (commandOffset + 8)..<(commandOffset + 24)
return data[uuidRange].map { String(format: "%02x", $0) }.joined()
}
commandOffset += commandSize
}
return nil
}
private nonisolated static func readUInt32(_ data: Data, at offset: Int, order: ByteOrder) -> UInt32? {
guard offset >= 0, offset + 4 <= data.count else { return nil }
let bytes = data[offset..<(offset + 4)]
return bytes.enumerated().reduce(UInt32(0)) { partial, pair in
let shift = switch order {
case .little: pair.offset * 8
case .big: (3 - pair.offset) * 8
}
return partial | UInt32(pair.element) << UInt32(shift)
}
}
private nonisolated static func readUInt64(_ data: Data, at offset: Int, order: ByteOrder) -> UInt64? {
guard offset >= 0, offset + 8 <= data.count else { return nil }
let bytes = data[offset..<(offset + 8)]
return bytes.enumerated().reduce(UInt64(0)) { partial, pair in
let shift = switch order {
case .little: pair.offset * 8
case .big: (7 - pair.offset) * 8
}
return partial | UInt64(pair.element) << UInt64(shift)
}
}
static func autoStartSocketPath(
daemonSocketPath: String,
defaultSocketWasOccupiedAndRejected: Bool,
runtimeBuildIdentity: String
) -> String {
guard defaultSocketWasOccupiedAndRejected,
let buildScopedSocketPath = buildScopedDaemonSocketPath(
daemonSocketPath: daemonSocketPath,
runtimeBuildIdentity: runtimeBuildIdentity
)
else {
return daemonSocketPath
}
return buildScopedSocketPath
}
static func buildScopedDaemonSocketPath(
daemonSocketPath: String,
runtimeBuildIdentity: String
) -> String? {
guard self.standardizedSocketPath(daemonSocketPath) ==
self.standardizedSocketPath(PeekabooBridgeConstants.daemonSocketPath)
else { return nil }
return URL(fileURLWithPath: daemonSocketPath)
.deletingLastPathComponent()
.appendingPathComponent("daemon-\(self.stableHash(runtimeBuildIdentity)).sock")
.path
}
private static func stableHash(_ value: String) -> String {
var hash: UInt64 = 14_695_981_039_346_656_037
for byte in value.utf8 {
hash ^= UInt64(byte)
hash &*= 1_099_511_628_211
}
return String(format: "%016llx", hash)
}
static func daemonIdleTimeoutSeconds(environment: [String: String]) -> TimeInterval {
@ -29,20 +249,96 @@ enum DaemonLaunchPolicy {
return value
}
static func shouldMigrateLegacyDaemon(targetSocketPath: String) -> Bool {
self.standardizedSocketPath(targetSocketPath) ==
self.standardizedSocketPath(PeekabooBridgeConstants.daemonSocketPath)
}
static func implicitRuntimeCandidateRole(
socketPath: String,
daemonSocketPath: String,
buildScopedDaemonSocketPath: String? = nil
) -> ImplicitRuntimeCandidateRole? {
let candidate = self.standardizedSocketPath(socketPath)
if candidate == self.standardizedSocketPath(daemonSocketPath) ||
buildScopedDaemonSocketPath.map(self.standardizedSocketPath) == candidate {
return .reusableDaemon
}
if self.shouldMigrateLegacyDaemon(targetSocketPath: daemonSocketPath),
candidate == self.standardizedSocketPath(PeekabooBridgeConstants.peekabooSocketPath) {
return .defaultAppFallback
}
return nil
}
static func isSelectableImplicitRuntimeCandidate(
role: ImplicitRuntimeCandidateRole,
handshake: PeekabooBridgeHandshakeResponse,
daemonStatus: PeekabooDaemonStatus?
) -> Bool {
switch role {
case .reusableDaemon:
daemonStatus.map(DaemonControlClient.isReusableDaemonStatus) == true
case .defaultAppFallback:
handshake.hostKind == .gui ||
daemonStatus.map(DaemonControlClient.isReusableDaemonStatus) == true
}
}
static func onDemandDaemonArguments(socketPath: String, idleTimeoutSeconds: TimeInterval) -> [String] {
[
self.daemonArguments(
socketPath: socketPath,
mode: .auto,
idleTimeoutSeconds: idleTimeoutSeconds
)
}
static func daemonArguments(
socketPath: String,
mode: PeekabooDaemonMode,
pollIntervalMs: Int? = nil,
idleTimeoutSeconds: TimeInterval
) -> [String] {
var arguments = [
"daemon",
"run",
"--mode",
"auto",
mode.rawValue,
"--bridge-socket",
socketPath,
"--idle-timeout-seconds",
String(format: "%.3f", idleTimeoutSeconds),
]
if let pollIntervalMs, pollIntervalMs > 0 {
arguments.append(contentsOf: [
"--poll-interval-ms",
"\(pollIntervalMs)",
])
}
if mode == .auto {
arguments.append(contentsOf: [
"--idle-timeout-seconds",
String(format: "%.3f", idleTimeoutSeconds),
])
}
return arguments
}
static func startOnDemandDaemon(socketPath: String, environment: [String: String]) async -> Bool {
static func migratedDaemonArguments(
socketPath: String,
status: PeekabooDaemonStatus,
fallbackIdleTimeoutSeconds: TimeInterval
) -> [String]? {
guard let mode = DaemonControlClient.migrationMode(for: status) else { return nil }
let idleTimeoutSeconds = status.activity?.idleTimeoutSeconds.flatMap { $0 > 0 ? $0 : nil }
?? fallbackIdleTimeoutSeconds
return self.daemonArguments(
socketPath: socketPath,
mode: mode,
pollIntervalMs: status.windowTracker?.cgPollIntervalMs,
idleTimeoutSeconds: idleTimeoutSeconds
)
}
static func startOnDemandDaemon(socketPath: String, environment: [String: String]) async -> String? {
let client = DaemonControlClient(socketPath: socketPath)
let lockHandle = DaemonPaths.openDaemonStartupLock()
if let fileDescriptor = lockHandle?.fileDescriptor {
@ -55,17 +351,201 @@ enum DaemonLaunchPolicy {
try? lockHandle?.close()
}
if await client.fetchStatus() != nil {
return true
if await client.fetchReusableDaemonStatus() != nil {
return socketPath
}
switch await self.waitForDaemonSocketAvailability(
socketPath: socketPath,
client: client,
timeout: TimeInterval(DaemonControlClient.defaultShutdownWaitSeconds)
) {
case .available:
break
case .reusableDaemon:
return socketPath
case .timedOut:
return nil
}
let fallbackIdleTimeoutSeconds = self.daemonIdleTimeoutSeconds(environment: environment)
var launchArguments = self.daemonArguments(
socketPath: socketPath,
mode: .auto,
idleTimeoutSeconds: fallbackIdleTimeoutSeconds
)
let legacyClient = DaemonControlClient(socketPath: PeekabooBridgeConstants.peekabooSocketPath)
if self.shouldMigrateLegacyDaemon(targetSocketPath: socketPath),
let legacyStatus = await legacyClient.fetchReusableDaemonStatus(),
let migrationArguments = migratedDaemonArguments(
socketPath: socketPath,
status: legacyStatus,
fallbackIdleTimeoutSeconds: fallbackIdleTimeoutSeconds
) {
if DaemonControlClient.supportsSafeMigration(legacyStatus),
DaemonControlClient.isIdleForMigration(legacyStatus) {
launchArguments = migrationArguments
guard let replacement = await launchDaemon(
socketPath: socketPath,
arguments: launchArguments
)
else {
return await self.compatibleLegacyFallbackSocketPath {
await legacyClient.fetchReusableDaemonStatus()
}
}
do {
let stopped = try await legacyClient.stopAndWait(
waitSeconds: DaemonControlClient.defaultShutdownWaitSeconds,
expectedPID: legacyStatus.pid,
requireIdentityMatch: true
)
if !stopped {
if let currentLegacyStatus = await legacyClient.fetchReusableDaemonStatus() {
return await self.resolveLegacyStopRace(
legacyStatus: currentLegacyStatus,
client: client,
replacement: replacement,
replacementSocketPath: socketPath
)
}
}
} catch {
if let currentLegacyStatus = await legacyClient.fetchReusableDaemonStatus() {
return await self.resolveLegacyStopRace(
legacyStatus: currentLegacyStatus,
client: client,
replacement: replacement,
replacementSocketPath: socketPath
)
}
}
return await client.fetchReusableDaemonStatus() != nil ? socketPath : nil
}
if let fallback = self.compatibleLegacyFallbackSocketPath(for: legacyStatus) {
return fallback
}
// An incompatible legacy host cannot satisfy this caller. Leave it running and
// start the current daemon on the free canonical socket instead.
launchArguments = self.daemonArguments(
socketPath: socketPath,
mode: .auto,
idleTimeoutSeconds: fallbackIdleTimeoutSeconds
)
}
return await self.launchDaemon(
socketPath: socketPath,
arguments: launchArguments
) != nil ? socketPath : nil
}
static func compatibleLegacyFallbackSocketPath(for status: PeekabooDaemonStatus) -> String? {
guard DaemonControlPlanner.supportsCurrentDaemon(status) else {
return nil
}
return PeekabooBridgeConstants.peekabooSocketPath
}
static func compatibleLegacyFallbackSocketPath(
refreshingWith fetchStatus: () async -> PeekabooDaemonStatus?
) async -> String? {
guard let currentStatus = await fetchStatus() else { return nil }
return self.compatibleLegacyFallbackSocketPath(for: currentStatus)
}
static func legacyStopRaceResolution(for status: PeekabooDaemonStatus) -> LegacyStopRaceResolution {
if let fallback = self.compatibleLegacyFallbackSocketPath(for: status) {
return .useLegacy(socketPath: fallback)
}
return .keepReplacement
}
static func legacyStopRaceSocketPath(
replacementCleanupSucceeded: Bool,
replacementIsReusable: Bool,
legacySocketPath: String,
replacementSocketPath: String
) -> String? {
if replacementCleanupSucceeded {
return legacySocketPath
}
return replacementIsReusable ? replacementSocketPath : nil
}
private static func resolveLegacyStopRace(
legacyStatus: PeekabooDaemonStatus,
client: DaemonControlClient,
replacement: LaunchResult,
replacementSocketPath: String
) async -> String? {
switch self.legacyStopRaceResolution(for: legacyStatus) {
case .keepReplacement:
return await client.fetchReusableDaemonStatus() != nil ? replacementSocketPath : nil
case let .useLegacy(socketPath):
let cleanedUp = await self.stopReplacement(client: client, replacement: replacement)
var replacementIsReusable = false
if !cleanedUp {
replacementIsReusable = await client.fetchReusableDaemonStatus() != nil
}
return self.legacyStopRaceSocketPath(
replacementCleanupSucceeded: cleanedUp,
replacementIsReusable: replacementIsReusable,
legacySocketPath: socketPath,
replacementSocketPath: replacementSocketPath
)
}
}
static func waitForDaemonSocketAvailability(
socketPath: String,
client: DaemonControlClient,
timeout: TimeInterval
) async -> SocketAvailability {
guard self.bridgeLeaseIsHeld(socketPath: socketPath) else {
return .available
}
let deadline = Date().addingTimeInterval(timeout)
while Date() < deadline {
if await client.fetchReusableDaemonStatus() != nil {
return .reusableDaemon
}
if !self.bridgeLeaseIsHeld(socketPath: socketPath) {
return .available
}
try? await Task.sleep(nanoseconds: 100_000_000)
}
return self.bridgeLeaseIsHeld(socketPath: socketPath) ? .timedOut : .available
}
private static func bridgeLeaseIsHeld(socketPath: String) -> Bool {
let fd = open(
"\(socketPath).lock",
O_RDWR | O_CLOEXEC | O_NOFOLLOW
)
guard fd >= 0 else { return false }
defer { close(fd) }
if flock(fd, LOCK_EX | LOCK_NB) == 0 {
flock(fd, LOCK_UN)
return false
}
return errno == EWOULDBLOCK || errno == EAGAIN
}
static func launchDaemon(
socketPath: String,
arguments: [String],
timeout: TimeInterval = 3
) async -> LaunchResult? {
let executable = CommandLine.arguments.first ?? "/usr/local/bin/peekaboo"
let process = Process()
process.executableURL = URL(fileURLWithPath: executable)
process.arguments = self.onDemandDaemonArguments(
socketPath: socketPath,
idleTimeoutSeconds: self.daemonIdleTimeoutSeconds(environment: environment)
)
process.arguments = arguments
let logHandle = DaemonPaths.openDaemonLogForAppend() ?? FileHandle.nullDevice
process.standardOutput = logHandle
process.standardError = logHandle
@ -74,16 +554,52 @@ enum DaemonLaunchPolicy {
do {
try process.run()
} catch {
return false
return nil
}
let deadline = Date().addingTimeInterval(3)
let deadline = Date().addingTimeInterval(timeout)
let client = DaemonControlClient(socketPath: socketPath)
while Date() < deadline {
if await client.fetchStatus() != nil {
return true
if let status = await client.fetchReusableDaemonStatus() {
let processID = process.processIdentifier
if status.pid != processID, process.isRunning {
process.terminate()
}
return LaunchResult(status: status, processID: processID)
}
try? await Task.sleep(nanoseconds: 100_000_000)
}
return false
if process.isRunning {
process.terminate()
}
return nil
}
static func stopReplacement(
client: DaemonControlClient,
replacement: LaunchResult
) async -> Bool {
guard replacement.ownsObservedDaemon else { return true }
let expectedPID = replacement.processID
let deadline = Date().addingTimeInterval(
TimeInterval(DaemonControlClient.defaultShutdownWaitSeconds)
)
while Date() < deadline {
guard let status = await client.fetchControllableDaemonStatus(),
status.pid == expectedPID
else {
return true
}
_ = try? await client.stopDaemon(expectedPID: expectedPID)
try? await Task.sleep(nanoseconds: 200_000_000)
}
return await client.fetchControllableDaemonStatus()?.pid != expectedPID
}
private static func standardizedSocketPath(_ path: String) -> String {
let expanded = (path as NSString).expandingTildeInPath
return (expanded as NSString).standardizingPath
}
}

View File

@ -6,31 +6,85 @@ import PeekabooCore
@MainActor
enum RuntimeHostResolver {
static func resolveServices(options: CommandRuntimeOptions)
async -> (services: any PeekabooServiceProviding, hostDescription: String) {
struct Resolution {
let services: any PeekabooServiceProviding
let hostDescription: String
let selectedRemoteSocketPath: String?
let selectedRemoteHostProcessIdentifier: pid_t?
let snapshotInvalidationRemoteSocketPaths: [String]
let applicationRelaunchAllowed: Bool
}
struct ImplicitRemoteCandidate: Equatable {
let socketPath: String
let requireReusableDaemon: Bool
let requiredHostKind: PeekabooBridgeHostKind?
let requiresValidatedHistoricalDaemon: Bool
}
struct RemoteCandidatePlan {
let explicitSocket: String?
let daemonSocketPath: String
let runtimeBuildIdentity: String
let buildScopedDaemonSocketPath: String?
let historicalBuildScopedDaemonSocketPaths: [String]
let candidates: [ImplicitRemoteCandidate]
}
struct RemoteCandidateValidation {
let reusableDaemonStatus: PeekabooDaemonStatus?
}
enum InitialRoutingDecision: Equatable {
case local(snapshotInvalidationRemoteSocketPaths: [String])
case remote
}
static func resolveServices(options: CommandRuntimeOptions) async -> Resolution {
let environment = ProcessInfo.processInfo.environment
let envNoRemote = environment["PEEKABOO_NO_REMOTE"]
guard options.preferRemote,
envNoRemote == nil,
options.inputStrategy == nil,
!RuntimeInputPolicyResolver.hasEnvironmentOverride(environment: environment),
!RuntimeInputPolicyResolver.hasConfigOverride(
input: PeekabooAutomation.ConfigurationManager.shared.getConfiguration()?.input
)
else {
return (
let configurationInput = PeekabooAutomation.ConfigurationManager.shared.getConfiguration()?.input
guard self.shouldResolveKnownRemoteEndpoints(
options: options,
environment: environment,
configurationInput: configurationInput
) else {
return Resolution(
services: RuntimeServiceFactory.makeLocalServices(options: options),
hostDescription: "local (in-process)"
hostDescription: "local (in-process)",
selectedRemoteSocketPath: nil,
selectedRemoteHostProcessIdentifier: nil,
snapshotInvalidationRemoteSocketPaths: [],
applicationRelaunchAllowed: true
)
}
let explicitSocket = BridgeSocketResolver.explicitBridgeSocket(options: options, environment: environment)
let candidatePlan = await self.remoteCandidatePlan(options: options, environment: environment)
let explicitSocket = candidatePlan.explicitSocket
let daemonSocketPath = candidatePlan.daemonSocketPath
let runtimeBuildIdentity = candidatePlan.runtimeBuildIdentity
let buildScopedDaemonSocketPath = candidatePlan.buildScopedDaemonSocketPath
let historicalBuildScopedDaemonSocketPaths = candidatePlan.historicalBuildScopedDaemonSocketPaths
let snapshotInvalidationRemoteSocketPaths = snapshotInvalidationRemoteSocketPaths(
explicitSocket: explicitSocket,
daemonSocketPath: daemonSocketPath,
buildScopedDaemonSocketPath: buildScopedDaemonSocketPath,
historicalBuildScopedDaemonSocketPaths: historicalBuildScopedDaemonSocketPaths
)
let daemonSocketPath = DaemonLaunchPolicy.daemonSocketPath(environment: environment)
let candidates: [String] = if let explicitSocket, !explicitSocket.isEmpty {
[explicitSocket]
} else {
[daemonSocketPath]
if case let .local(localSnapshotInvalidationPaths) = initialRoutingDecision(
options: options,
environment: environment,
configurationInput: configurationInput,
knownSnapshotInvalidationRemoteSocketPaths: snapshotInvalidationRemoteSocketPaths
) {
return Resolution(
services: RuntimeServiceFactory.makeLocalServices(options: options),
hostDescription: "local (in-process)",
selectedRemoteSocketPath: nil,
selectedRemoteHostProcessIdentifier: nil,
snapshotInvalidationRemoteSocketPaths: localSnapshotInvalidationPaths,
applicationRelaunchAllowed: true
)
}
let identity = PeekabooBridgeClientIdentity(
@ -40,51 +94,289 @@ enum RuntimeHostResolver {
hostname: Host.current().name
)
if let resolved = await self.resolveRemoteServices(
candidates: candidates,
if let resolved = await resolveRemoteServices(
candidates: candidatePlan.candidates,
identity: identity,
options: options
options: options,
snapshotInvalidationRemoteSocketPaths: snapshotInvalidationRemoteSocketPaths
) {
return resolved
}
if options.autoStartDaemon,
DaemonLaunchPolicy.shouldAutoStartDaemon(options: options, environment: environment),
await DaemonLaunchPolicy.startOnDemandDaemon(socketPath: daemonSocketPath, environment: environment),
let resolved = await self.resolveRemoteServices(
candidates: [daemonSocketPath],
identity: identity,
options: options
) {
return resolved
if DaemonLaunchPolicy.shouldAutoStartDaemon(options: options, environment: environment) {
let rejectedDefaultSocketOccupant =
await DaemonControlClient(socketPath: daemonSocketPath).fetchStatus() != nil
let autoStartSocketPath = DaemonLaunchPolicy.autoStartSocketPath(
daemonSocketPath: daemonSocketPath,
defaultSocketWasOccupiedAndRejected: rejectedDefaultSocketOccupant,
runtimeBuildIdentity: runtimeBuildIdentity
)
if let resolvedDaemonSocket = await DaemonLaunchPolicy.startOnDemandDaemon(
socketPath: autoStartSocketPath,
environment: environment
),
let resolved = await resolveRemoteServices(
candidates: [ImplicitRemoteCandidate(
socketPath: resolvedDaemonSocket,
requireReusableDaemon: true,
requiredHostKind: nil,
requiresValidatedHistoricalDaemon: false
)],
identity: identity,
options: options,
snapshotInvalidationRemoteSocketPaths: snapshotInvalidationRemoteSocketPaths
) {
return resolved
}
}
return (
return Resolution(
services: RuntimeServiceFactory.makeLocalServices(options: options),
hostDescription: "local (in-process)"
hostDescription: "local (in-process fallback)",
selectedRemoteSocketPath: nil,
selectedRemoteHostProcessIdentifier: nil,
snapshotInvalidationRemoteSocketPaths: snapshotInvalidationRemoteSocketPaths,
applicationRelaunchAllowed: !options.requiresApplicationRelaunch
)
}
static func remoteRoutingAllowed(
options: CommandRuntimeOptions,
environment: [String: String],
configurationInput: PeekabooAutomation.Configuration.InputConfig?
) -> Bool {
self.initialRoutingDecision(
options: options,
environment: environment,
configurationInput: configurationInput,
knownSnapshotInvalidationRemoteSocketPaths: []
) == .remote
}
static func remoteCandidatePlan(
options: CommandRuntimeOptions,
environment: [String: String]
) async -> RemoteCandidatePlan {
let explicitSocket = BridgeSocketResolver.explicitBridgeSocket(options: options, environment: environment)
let daemonSocketPath = DaemonLaunchPolicy.daemonSocketPath(environment: environment)
let runtimeBuildIdentity = DaemonLaunchPolicy.runtimeBuildIdentity()
let buildScopedDaemonSocketPath = DaemonLaunchPolicy.buildScopedDaemonSocketPath(
daemonSocketPath: daemonSocketPath,
runtimeBuildIdentity: runtimeBuildIdentity
)
let historicalBuildScopedDaemonSocketPaths: [String] = if self.shouldDiscoverHistoricalDaemons(
explicitSocket: explicitSocket,
daemonSocketPath: daemonSocketPath
) {
await DaemonControlResolver.validatedHistoricalTargets(
daemonSocketPath: daemonSocketPath,
currentBuildScopedSocketPath: buildScopedDaemonSocketPath
)
.filter { DaemonControlPlanner.supportsCurrentDaemon($0.status) }
.map(\.client.socketPath)
} else {
[]
}
let candidates: [ImplicitRemoteCandidate] = if let explicitSocket, !explicitSocket.isEmpty {
[ImplicitRemoteCandidate(
socketPath: explicitSocket,
requireReusableDaemon: false,
requiredHostKind: nil,
requiresValidatedHistoricalDaemon: false
)]
} else {
self.implicitRemoteCandidates(
options: options,
daemonSocketPath: daemonSocketPath,
buildScopedDaemonSocketPath: buildScopedDaemonSocketPath,
historicalBuildScopedDaemonSocketPaths: historicalBuildScopedDaemonSocketPaths
)
}
return RemoteCandidatePlan(
explicitSocket: explicitSocket,
daemonSocketPath: daemonSocketPath,
runtimeBuildIdentity: runtimeBuildIdentity,
buildScopedDaemonSocketPath: buildScopedDaemonSocketPath,
historicalBuildScopedDaemonSocketPaths: historicalBuildScopedDaemonSocketPaths,
candidates: candidates
)
}
static func initialRoutingDecision(
options: CommandRuntimeOptions,
environment: [String: String],
configurationInput: PeekabooAutomation.Configuration.InputConfig?,
knownSnapshotInvalidationRemoteSocketPaths: [String]
) -> InitialRoutingDecision {
guard !self.remoteIsolationRequested(options: options, environment: environment) else {
return .local(snapshotInvalidationRemoteSocketPaths: [])
}
if self.inputPolicyRequiresLocal(
options: options,
environment: environment,
configurationInput: configurationInput
) {
return .local(
snapshotInvalidationRemoteSocketPaths: knownSnapshotInvalidationRemoteSocketPaths
)
}
if !options.preferRemote,
options.requiresImplicitSnapshotInvalidation || options.usesPerToolSnapshotInvalidation {
return .local(
snapshotInvalidationRemoteSocketPaths: knownSnapshotInvalidationRemoteSocketPaths
)
}
guard options.preferRemote else {
return .local(snapshotInvalidationRemoteSocketPaths: [])
}
return .remote
}
static func shouldResolveKnownRemoteEndpoints(
options: CommandRuntimeOptions,
environment: [String: String],
configurationInput: PeekabooAutomation.Configuration.InputConfig?
) -> Bool {
guard !self.remoteIsolationRequested(options: options, environment: environment) else {
return false
}
return options.preferRemote ||
options.requiresImplicitSnapshotInvalidation ||
options.usesPerToolSnapshotInvalidation ||
self.inputPolicyRequiresLocal(
options: options,
environment: environment,
configurationInput: configurationInput
)
}
static func remoteIsolationRequested(
options: CommandRuntimeOptions,
environment: [String: String]
) -> Bool {
options.remoteIsolationRequested || environment["PEEKABOO_NO_REMOTE"] != nil
}
static func snapshotInvalidationRemoteSocketPaths(
explicitSocket: String?,
daemonSocketPath: String,
buildScopedDaemonSocketPath: String? = nil,
historicalBuildScopedDaemonSocketPaths: [String] = []
) -> [String] {
var seen = Set<String>()
var candidatePaths = [
explicitSocket,
PeekabooBridgeConstants.peekabooSocketPath,
daemonSocketPath,
buildScopedDaemonSocketPath,
]
.compactMap(\.self)
candidatePaths.append(contentsOf: historicalBuildScopedDaemonSocketPaths)
return candidatePaths
.map { NSString(string: $0).standardizingPath }
.filter { !$0.isEmpty && seen.insert($0).inserted }
}
static func shouldDiscoverHistoricalDaemons(
explicitSocket: String?,
daemonSocketPath: String
) -> Bool {
explicitSocket == nil && DaemonLaunchPolicy.shouldMigrateLegacyDaemon(targetSocketPath: daemonSocketPath)
}
static func inputPolicyRequiresLocal(
options: CommandRuntimeOptions,
environment: [String: String],
configurationInput: PeekabooAutomation.Configuration.InputConfig?
) -> Bool {
guard !options.requiresApplicationLaunchOptions,
!options.requiresHostApplicationInventory
else {
return false
}
return options.inputStrategy != nil ||
RuntimeInputPolicyResolver.hasEnvironmentOverride(environment: environment) ||
RuntimeInputPolicyResolver.hasConfigOverride(input: configurationInput)
}
static func implicitRemoteCandidates(
options: CommandRuntimeOptions,
daemonSocketPath: String,
buildScopedDaemonSocketPath: String? = nil,
historicalBuildScopedDaemonSocketPaths: [String] = []
) -> [ImplicitRemoteCandidate] {
var seenDaemonPaths = Set<String>()
var daemons: [ImplicitRemoteCandidate] = []
for socketPath in [daemonSocketPath, buildScopedDaemonSocketPath].compactMap(\.self) {
guard seenDaemonPaths.insert(NSString(string: socketPath).standardizingPath).inserted else { continue }
daemons.append(ImplicitRemoteCandidate(
socketPath: socketPath,
requireReusableDaemon: true,
requiredHostKind: nil,
requiresValidatedHistoricalDaemon: false
))
}
for socketPath in historicalBuildScopedDaemonSocketPaths {
guard seenDaemonPaths.insert(NSString(string: socketPath).standardizingPath).inserted else { continue }
daemons.append(ImplicitRemoteCandidate(
socketPath: socketPath,
requireReusableDaemon: true,
requiredHostKind: .onDemand,
requiresValidatedHistoricalDaemon: true
))
}
let gui = ImplicitRemoteCandidate(
socketPath: PeekabooBridgeConstants.peekabooSocketPath,
requireReusableDaemon: false,
requiredHostKind: .gui,
requiresValidatedHistoricalDaemon: false
)
if options.requiresApplicationRelaunch || options.requiresSurvivingApplicationHost {
return daemons
}
if options.requiresApplicationLaunchOptions || options.requiresHostApplicationInventory {
return [gui] + daemons
}
if DaemonLaunchPolicy.shouldMigrateLegacyDaemon(targetSocketPath: daemonSocketPath) {
return daemons + [gui]
}
return daemons
}
private static func resolveRemoteServices(
candidates: [String],
candidates: [ImplicitRemoteCandidate],
identity: PeekabooBridgeClientIdentity,
options: CommandRuntimeOptions
options: CommandRuntimeOptions,
snapshotInvalidationRemoteSocketPaths: [String]
)
async -> (services: any PeekabooServiceProviding, hostDescription: String)? {
for socketPath in candidates {
async -> Resolution? {
for candidate in candidates {
let socketPath = candidate.socketPath
let client = PeekabooBridgeClient(socketPath: socketPath)
do {
let handshake = try await client.handshake(client: identity, requestedHost: nil)
guard BridgeCapabilityPolicy.supportsRemoteRequirements(for: handshake, options: options) else {
continue
}
guard let validation = await self.validateRemoteCandidate(
candidate,
handshake: handshake,
options: options
) else { continue }
let reusableDaemonStatus = validation.reusableDaemonStatus
let targetedHotkeyAvailability = BridgeCapabilityPolicy.targetedHotkeyAvailability(for: handshake)
let targetedTypeAvailability = BridgeCapabilityPolicy.targetedTypeAvailability(for: handshake)
let targetedClickAvailability = BridgeCapabilityPolicy.targetedClickAvailability(for: handshake)
let hostDescription = "remote \(handshake.hostKind.rawValue) via \(socketPath)" +
(handshake.build.map { " (build \($0))" } ?? "")
return (
return Resolution(
services: RemotePeekabooServices(
client: client,
supportsTargetedHotkeys: targetedHotkeyAvailability.isEnabled,
@ -99,6 +391,8 @@ enum RuntimeHostResolver {
targetedClickUnavailableReason: targetedClickAvailability.unavailableReason,
targetedClickRequiresEventSynthesizingPermission:
targetedClickAvailability.missingPermissions.contains(.postEvent),
supportsExactWindowTargetedClicks:
BridgeCapabilityPolicy.supportsExactWindowTargetedClicks(for: handshake),
supportsInspectAccessibilityTree: BridgeCapabilityPolicy.supportsInspectAccessibilityTree(
for: handshake
),
@ -107,9 +401,20 @@ enum RuntimeHostResolver {
),
supportsElementActions: BridgeCapabilityPolicy.supportsElementActions(for: handshake),
supportsDesktopObservation: BridgeCapabilityPolicy.supportsDesktopObservation(for: handshake),
allowLocalApplicationFallback: handshake.hostKind == .onDemand
supportsImplicitLatestSnapshotInvalidation:
BridgeCapabilityPolicy.supportsImplicitSnapshotInvalidation(for: handshake),
supportsApplicationLaunchOptions:
BridgeCapabilityPolicy.supportsApplicationLaunchOptions(for: handshake),
supportsApplicationRelaunch:
BridgeCapabilityPolicy.supportsApplicationRelaunch(for: handshake),
allowLocalApplicationFallback: handshake.hostKind == .onDemand,
desktopMutationWatermarkStore: DesktopMutationWatermarkStore()
),
hostDescription: hostDescription
hostDescription: hostDescription,
selectedRemoteSocketPath: NSString(string: socketPath).standardizingPath,
selectedRemoteHostProcessIdentifier: reusableDaemonStatus?.pid,
snapshotInvalidationRemoteSocketPaths: snapshotInvalidationRemoteSocketPaths,
applicationRelaunchAllowed: BridgeCapabilityPolicy.supportsApplicationRelaunch(for: handshake)
)
} catch {
continue
@ -117,4 +422,47 @@ enum RuntimeHostResolver {
}
return nil
}
static func validateRemoteCandidate(
_ candidate: ImplicitRemoteCandidate,
handshake: PeekabooBridgeHandshakeResponse,
options: CommandRuntimeOptions,
fetchReusableDaemonStatus: (String) async -> PeekabooDaemonStatus? = { socketPath in
await DaemonControlClient(socketPath: socketPath).fetchReusableDaemonStatus()
}
) async -> RemoteCandidateValidation? {
guard candidate.requiredHostKind == nil || handshake.hostKind == candidate.requiredHostKind else {
return nil
}
guard BridgeCapabilityPolicy.supportsRemoteRequirements(for: handshake, options: options) else {
return nil
}
let requiresReusableHost = candidate.requireReusableDaemon ||
options.requiresApplicationRelaunch ||
options.requiresSurvivingApplicationHost
let reusableDaemonStatus: PeekabooDaemonStatus? = if requiresReusableHost {
await fetchReusableDaemonStatus(candidate.socketPath)
} else {
nil
}
guard !requiresReusableHost || reusableDaemonStatus != nil else { return nil }
if candidate.requiresValidatedHistoricalDaemon {
guard let reusableDaemonStatus,
DaemonControlResolver.isValidatedHistoricalTarget(
status: reusableDaemonStatus,
socketPath: candidate.socketPath
),
DaemonControlPlanner.supportsCurrentDaemon(reusableDaemonStatus)
else {
return nil
}
}
if options.requiresApplicationRelaunch || options.requiresSurvivingApplicationHost,
reusableDaemonStatus?.pid == nil {
return nil
}
return RemoteCandidateValidation(reusableDaemonStatus: reusableDaemonStatus)
}
}

View File

@ -5,6 +5,9 @@ import PeekabooCore
enum RuntimeServiceFactory {
static func makeLocalServices(options: CommandRuntimeOptions) -> PeekabooServices {
PeekabooServices(
snapshotManager: SnapshotManager(
desktopMutationWatermarkStore: DesktopMutationWatermarkStore()
),
inputPolicy: PeekabooAutomation.ConfigurationManager.shared.getUIInputPolicy(
cliStrategy: options.inputStrategy
)

View File

@ -1,4 +1,5 @@
import Foundation
import PeekabooAutomation
import PeekabooBridge
import Security
@ -11,11 +12,14 @@ struct BridgeDiagnostics {
@MainActor
func run(runtimeOptions: CommandRuntimeOptions) async -> BridgeStatusReport {
let envNoRemote = ProcessInfo.processInfo.environment["PEEKABOO_NO_REMOTE"]
let shouldSkipRemote = !runtimeOptions.preferRemote || envNoRemote != nil
let remoteSkipReason = shouldSkipRemote
? (!runtimeOptions.preferRemote ? "--no-remote" : "PEEKABOO_NO_REMOTE")
: nil
let environment = ProcessInfo.processInfo.environment
let effectiveOptions = runtimeOptions.applyingEnvironmentOverrides(environment: environment)
let configurationInput = PeekabooAutomation.ConfigurationManager.shared.getConfiguration()?.input
let remoteSkipReason = Self.remoteSkipReason(
runtimeOptions: effectiveOptions,
environment: environment,
configurationInput: configurationInput
)
let identity = PeekabooBridgeClientIdentity(
bundleIdentifier: Bundle.main.bundleIdentifier,
@ -24,9 +28,12 @@ struct BridgeDiagnostics {
hostname: Host.current().name
)
let candidates = self.candidateSocketPaths(runtimeOptions: runtimeOptions)
if shouldSkipRemote {
self.logger.debug("Bridge status: remote skipped (\(remoteSkipReason ?? "unknown reason"))")
if let remoteSkipReason {
let candidates = Self.diagnosticSocketPaths(
runtimeOptions: effectiveOptions,
environment: environment
)
self.logger.debug("Bridge status: remote skipped (\(remoteSkipReason))")
return BridgeStatusReport(
remoteSkipped: true,
remoteSkipReason: remoteSkipReason,
@ -36,6 +43,23 @@ struct BridgeDiagnostics {
)
}
let candidatePlan = await RuntimeHostResolver.remoteCandidatePlan(
options: effectiveOptions,
environment: environment
)
let runtimeCandidates = candidatePlan.candidates
let candidates = Self.diagnosticSocketPaths(
runtimeCandidateSocketPaths: runtimeCandidates.map(\.socketPath),
hasExplicitSocket: candidatePlan.explicitSocket != nil
)
var runtimeCandidateByPath: [String: RuntimeHostResolver.ImplicitRemoteCandidate] = [:]
for candidate in runtimeCandidates {
let path = NSString(string: candidate.socketPath).standardizingPath
if runtimeCandidateByPath[path] == nil {
runtimeCandidateByPath[path] = candidate
}
}
var results: [BridgeCandidateReport] = []
var selected: BridgeSelectionReport?
@ -50,9 +74,17 @@ struct BridgeDiagnostics {
)
results.append(.init(socketPath: socketPath, result: .success(report)))
let enabledOps = handshake.enabledOperations ?? handshake.supportedOperations
if selected == nil, enabledOps.contains(.captureScreen) {
selected = .remote(socketPath: socketPath, handshake: report)
let candidatePath = NSString(string: socketPath).standardizingPath
if selected == nil,
let runtimeCandidate = runtimeCandidateByPath[candidatePath] {
let validation = await RuntimeHostResolver.validateRemoteCandidate(
runtimeCandidate,
handshake: handshake,
options: effectiveOptions
)
if validation != nil {
selected = .remote(socketPath: socketPath, handshake: report)
}
}
} catch let envelope as PeekabooBridgeErrorEnvelope {
self.logger.debug(
@ -78,21 +110,90 @@ struct BridgeDiagnostics {
)
}
private func candidateSocketPaths(runtimeOptions: CommandRuntimeOptions) -> [String] {
let envSocket = ProcessInfo.processInfo.environment["PEEKABOO_BRIDGE_SOCKET"]
let explicitSocket = runtimeOptions.bridgeSocketPath ?? envSocket
static func remoteSkipReason(
runtimeOptions: CommandRuntimeOptions,
environment: [String: String],
configurationInput: PeekabooAutomation.Configuration.InputConfig?
) -> String? {
let decision = RuntimeHostResolver.initialRoutingDecision(
options: runtimeOptions,
environment: environment,
configurationInput: configurationInput,
knownSnapshotInvalidationRemoteSocketPaths: []
)
guard case .local = decision else { return nil }
let rawCandidates: [String] = if let explicitSocket, !explicitSocket.isEmpty {
[explicitSocket]
} else {
[
PeekabooBridgeConstants.peekabooSocketPath,
PeekabooBridgeConstants.claudeSocketPath,
PeekabooBridgeConstants.clawdbotSocketPath,
]
if environment["PEEKABOO_NO_REMOTE"] != nil {
return "PEEKABOO_NO_REMOTE"
}
if runtimeOptions.remoteIsolationRequested {
return "--no-remote"
}
if RuntimeHostResolver.inputPolicyRequiresLocal(
options: runtimeOptions,
environment: environment,
configurationInput: configurationInput
) {
return "input strategy policy"
}
return "local runtime policy"
}
static func runtimeCandidateSocketPaths(
runtimeOptions: CommandRuntimeOptions,
environment: [String: String],
historicalBuildScopedDaemonSocketPaths: [String] = []
) -> [String] {
if let explicitPath = BridgeSocketResolver.explicitBridgeSocket(
options: runtimeOptions,
environment: environment
) {
return [explicitPath]
}
return rawCandidates.map { NSString(string: $0).expandingTildeInPath }
let daemonPath = DaemonLaunchPolicy.daemonSocketPath(environment: environment)
let buildScopedPath = DaemonLaunchPolicy.buildScopedDaemonSocketPath(
daemonSocketPath: daemonPath,
runtimeBuildIdentity: DaemonLaunchPolicy.runtimeBuildIdentity()
)
return RuntimeHostResolver.implicitRemoteCandidates(
options: runtimeOptions,
daemonSocketPath: daemonPath,
buildScopedDaemonSocketPath: buildScopedPath,
historicalBuildScopedDaemonSocketPaths: historicalBuildScopedDaemonSocketPaths
).map(\.socketPath)
}
static func diagnosticSocketPaths(
runtimeOptions: CommandRuntimeOptions,
environment: [String: String],
historicalBuildScopedDaemonSocketPaths: [String] = []
) -> [String] {
let runtimePaths = self.runtimeCandidateSocketPaths(
runtimeOptions: runtimeOptions,
environment: environment,
historicalBuildScopedDaemonSocketPaths: historicalBuildScopedDaemonSocketPaths
)
return self.diagnosticSocketPaths(
runtimeCandidateSocketPaths: runtimePaths,
hasExplicitSocket: BridgeSocketResolver.explicitBridgeSocket(
options: runtimeOptions,
environment: environment
) != nil
)
}
private static func diagnosticSocketPaths(
runtimeCandidateSocketPaths runtimePaths: [String],
hasExplicitSocket: Bool
) -> [String] {
if hasExplicitSocket { return runtimePaths }
let additionalPaths = [
PeekabooBridgeConstants.peekabooSocketPath,
PeekabooBridgeConstants.claudeSocketPath,
PeekabooBridgeConstants.clawdbotSocketPath,
]
return runtimePaths + additionalPaths.filter { !runtimePaths.contains($0) }
}
private static func currentTeamIdentifier() -> String? {

View File

@ -9,11 +9,8 @@ struct BridgeCommand: ParsableCommand {
Peekaboo Bridge lets the CLI run permission-bound operations (Screen Recording, Accessibility,
AppleScript) via a host app that already has the needed TCC grants.
By default, Peekaboo prefers a remote host when available:
1) Peekaboo.app
2) Claude.app
3) ClawdBot.app
4) Local in-process fallback (caller needs permissions)
By default, automation commands use the dedicated Peekaboo daemon and fall back to local execution.
Peekaboo.app, Claude.app, and ClawdBot.app sockets are shown for diagnostics and can be selected explicitly.
Examples:
peekaboo bridge status

View File

@ -74,6 +74,10 @@ RuntimeOptionsConfigurable {
self.resolvedRuntime.services
}
func withCaptureFocusMutation(_ operation: () async throws -> Void) async rethrows {
try await self.resolvedRuntime.withCaptureFocusMutation(operation)
}
var jsonOutput: Bool {
self.resolvedRuntime.configuration.jsonOutput
}
@ -92,16 +96,16 @@ RuntimeOptionsConfigurable {
throw ValidationError("Pass the action command after --")
}
let scope = try await self.resolveScope()
let options = try self.buildOptions()
let timing = try self.resolveActionTiming(durationLimit: options.duration)
let scope = try await resolveScope()
let options = try buildOptions()
let timing = try resolveActionTiming(durationLimit: options.duration)
if scope.kind == .window, let identifier = scope.applicationIdentifier {
try await self.focusIfNeeded(appIdentifier: identifier)
try await focusIfNeeded(appIdentifier: identifier)
}
let outputDir = try self.resolveOutputDirectory()
let outputDir = try resolveOutputDirectory()
let deps = WatchCaptureDependencies(
screenCapture: self.services.screenCapture,
screenCapture: services.screenCapture,
screenService: self.services.screens,
frameSource: nil
)
@ -109,7 +113,7 @@ RuntimeOptionsConfigurable {
scope: scope,
options: options,
outputRoot: outputDir,
autoclean: WatchAutocleanConfig(minutes: self.autocleanMinutes ?? 120, managed: self.path == nil),
autoclean: WatchAutocleanConfig(minutes: autocleanMinutes ?? 120, managed: path == nil),
sourceKind: .live,
videoIn: nil,
videoOut: CaptureCommandPathResolver.filePath(from: self.videoOut),
@ -125,6 +129,7 @@ RuntimeOptionsConfigurable {
) != nil {
throw ValidationError("Capture ended before action started")
}
self.resolvedRuntime.beginInteractionMutation()
let action = try await CaptureActionProcessRunner.run(
command: self.command,
timeoutSeconds: timing.actionTimeout
@ -133,7 +138,7 @@ RuntimeOptionsConfigurable {
session.requestStop()
let capture = try await captureTask.value
let validation = self.validateArtifacts(capture)
let validation = validateArtifacts(capture)
let result = CaptureActionCommandResult(
success: action.succeeded && validation.ok,
action: action,
@ -158,7 +163,7 @@ RuntimeOptionsConfigurable {
} catch let exit as ExitCode {
throw exit
} catch {
self.handleError(error)
handleError(error)
self.logger.operationComplete(
"capture_action",
success: false,
@ -175,9 +180,9 @@ RuntimeOptionsConfigurable {
let runSession: @MainActor @Sendable () async throws -> CaptureSessionResult = {
try await session.run()
}
let enginePreference = self.liveCaptureEnginePreference(for: scope)
let enginePreference = liveCaptureEnginePreference(for: scope)
return Task { @MainActor in
if let engineAware = self.services.screenCapture as? any EngineAwareScreenCaptureServiceProtocol {
if let engineAware = services.screenCapture as? any EngineAwareScreenCaptureServiceProtocol {
try await engineAware.withCaptureEngine(enginePreference, operation: runSession)
} else {
try await runSession()
@ -220,17 +225,17 @@ RuntimeOptionsConfigurable {
}
private func buildOptions() throws -> CaptureOptions {
let duration = max(1, min(self.durationLimit ?? 60, 180))
let idle = min(max(self.idleFps ?? 2, 0.1), 5)
let active = min(max(self.activeFps ?? 8, 0.5), 15)
let threshold = min(max(self.threshold ?? 2.5, 0), 100)
let heartbeat = max(self.heartbeatSec ?? 5, 0)
let quiet = max(self.quietMs ?? 1000, 0)
let maxFrames = max(self.maxFrames ?? 800, 1)
let resolutionCap = self.resolutionCap ?? 1440
let diffStrategy = try CaptureCommandOptionParser.diffStrategy(self.diffStrategy)
let diffBudgetMs = self.diffBudgetMs ?? (diffStrategy == .quality ? 30 : nil)
let maxMb = self.maxMb.flatMap { $0 > 0 ? $0 : nil }
let duration = max(1, min(durationLimit ?? 60, 180))
let idle = min(max(idleFps ?? 2, 0.1), 5)
let active = min(max(activeFps ?? 8, 0.5), 15)
let threshold = min(max(threshold ?? 2.5, 0), 100)
let heartbeat = max(heartbeatSec ?? 5, 0)
let quiet = max(quietMs ?? 1000, 0)
let maxFrames = max(maxFrames ?? 800, 1)
let resolutionCap = resolutionCap ?? 1440
let diffStrategy = try CaptureCommandOptionParser.diffStrategy(diffStrategy)
let diffBudgetMs = diffBudgetMs ?? (diffStrategy == .quality ? 30 : nil)
let maxMb = maxMb.flatMap { $0 > 0 ? $0 : nil }
return CaptureOptions(
duration: duration,
@ -250,14 +255,14 @@ RuntimeOptionsConfigurable {
}
private func resolveActionTiming(durationLimit: TimeInterval) throws -> CaptureActionTiming {
let preRoll = max(self.preRollMs ?? 250, 0)
let postRoll = max(self.postRollMs ?? 500, 0)
let preRoll = max(preRollMs ?? 250, 0)
let postRoll = max(postRollMs ?? 500, 0)
let rollSeconds = Double(preRoll + postRoll) / 1000.0
guard rollSeconds < durationLimit else {
throw ValidationError("--pre-roll-ms + --post-roll-ms must be less than --duration-limit")
}
let defaultActionTimeout = max(0.1, durationLimit - rollSeconds)
let actionTimeout = max(0.1, min(self.actionTimeout ?? defaultActionTimeout, durationLimit - rollSeconds))
let actionTimeout = max(0.1, min(actionTimeout ?? defaultActionTimeout, durationLimit - rollSeconds))
return CaptureActionTiming(
preRollMs: preRoll,
postRollMs: postRoll,
@ -371,7 +376,7 @@ extension CaptureActionCommand {
checked.append(contentsOf: result.frames.map(\.path))
if let videoOut = result.videoOut {
checked.append(videoOut)
} else if let expectedVideoOut = CaptureCommandPathResolver.filePath(from: self.videoOut) {
} else if let expectedVideoOut = CaptureCommandPathResolver.filePath(from: videoOut) {
checked.append(expectedVideoOut)
}
@ -400,10 +405,10 @@ extension CaptureActionCommand {
@MainActor
extension CaptureActionCommand {
func resolveScope() async throws -> CaptureScope {
let mode = try self.resolveMode()
let mode = try resolveMode()
switch mode {
case .screen:
let displayInfo = try await self.displayInfo(for: self.screenIndex)
let displayInfo = try await displayInfo(for: screenIndex)
return CaptureScope(
kind: .screen,
screenIndex: displayInfo?.index,
@ -416,8 +421,8 @@ extension CaptureActionCommand {
case .frontmost:
return CaptureScope(kind: .frontmost)
case .window:
let identifier = try self.resolveApplicationIdentifier()
let windowReference = try await self.resolveWindowReference(for: identifier)
let identifier = try resolveApplicationIdentifier()
let windowReference = try await resolveWindowReference(for: identifier)
return CaptureScope(
kind: .window,
screenIndex: nil,
@ -428,7 +433,7 @@ extension CaptureActionCommand {
region: nil
)
case .area:
let rect = try self.parseRegion()
let rect = try parseRegion()
return CaptureScope(kind: .region, region: rect)
case .multi:
throw ValidationError("capture action does not support multi-mode captures")
@ -436,7 +441,7 @@ extension CaptureActionCommand {
}
func resolveMode() throws -> LiveCaptureMode {
if let explicit = self.mode {
if let explicit = mode {
let normalized = explicit.trimmingCharacters(in: .whitespacesAndNewlines).lowercased()
if normalized == "region" { return .area }
guard let mode = LiveCaptureMode(rawValue: normalized) else {
@ -452,7 +457,7 @@ extension CaptureActionCommand {
}
func parseRegion() throws -> CGRect {
guard let region = self.region?.trimmingCharacters(in: .whitespacesAndNewlines),
guard let region = region?.trimmingCharacters(in: .whitespacesAndNewlines),
!region.isEmpty
else {
throw PeekabooError.invalidInput("Region must be provided when --mode area is set")
@ -486,12 +491,14 @@ extension CaptureActionCommand {
spaceSwitch: false,
bringToCurrentSpace: false
)
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
try await withCaptureFocusMutation {
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
}
case .foreground:
let options = FocusOptions(
autoFocus: true,
@ -500,17 +507,19 @@ extension CaptureActionCommand {
spaceSwitch: true,
bringToCurrentSpace: true
)
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
try await withCaptureFocusMutation {
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
}
}
}
private func liveCaptureEnginePreference(for scope: CaptureScope) -> CaptureEnginePreference {
let value = (self.captureEngine ?? self.resolvedRuntime.configuration.captureEnginePreference)?
let value = (captureEngine ?? self.resolvedRuntime.configuration.captureEnginePreference)?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased()
@ -544,11 +553,11 @@ extension CaptureActionCommand {
)
let renderable = ObservationTargetResolver.captureCandidates(from: windows)
let selectedWindow: ServiceWindowInfo? = if let title = self.windowTitle?
let selectedWindow: ServiceWindowInfo? = if let title = windowTitle?
.trimmingCharacters(in: .whitespacesAndNewlines),
!title.isEmpty {
renderable.first { $0.title.localizedCaseInsensitiveContains(title) }
} else if let explicitIndex = self.windowIndex {
} else if let explicitIndex = windowIndex {
renderable.first { $0.index == explicitIndex }
} else {
nil

View File

@ -69,6 +69,10 @@ struct CaptureLiveCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFo
self.resolvedRuntime.services
}
func withCaptureFocusMutation(_ operation: () async throws -> Void) async rethrows {
try await self.resolvedRuntime.withCaptureFocusMutation(operation)
}
var jsonOutput: Bool {
self.resolvedRuntime.configuration.jsonOutput
}
@ -86,14 +90,14 @@ struct CaptureLiveCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFo
// The capture service performs the authoritative permission check inside
// the serialized capture transaction; an extra CLI-side SCK probe can race
// with concurrent screenshot commands and report transient TCC denial.
let scope = try await self.resolveScope()
let options = try self.buildOptions()
let scope = try await resolveScope()
let options = try buildOptions()
if scope.kind == .window, let identifier = scope.applicationIdentifier {
try await self.focusIfNeeded(appIdentifier: identifier)
try await focusIfNeeded(appIdentifier: identifier)
}
let outputDir = try self.resolveOutputDirectory()
let outputDir = try resolveOutputDirectory()
let deps = WatchCaptureDependencies(
screenCapture: self.services.screenCapture,
screenCapture: services.screenCapture,
screenService: self.services.screens,
frameSource: nil
)
@ -101,7 +105,7 @@ struct CaptureLiveCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFo
scope: scope,
options: options,
outputRoot: outputDir,
autoclean: WatchAutocleanConfig(minutes: self.autocleanMinutes ?? 120, managed: self.path == nil),
autoclean: WatchAutocleanConfig(minutes: autocleanMinutes ?? 120, managed: path == nil),
sourceKind: .live,
videoIn: nil,
videoOut: CaptureCommandPathResolver.filePath(from: self.videoOut),
@ -111,21 +115,21 @@ struct CaptureLiveCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFo
let runSession: @MainActor @Sendable () async throws -> CaptureSessionResult = {
try await session.run()
}
let enginePreference = self.liveCaptureEnginePreference(for: scope)
let result: CaptureSessionResult = if let engineAware = self.services.screenCapture
let enginePreference = liveCaptureEnginePreference(for: scope)
let result: CaptureSessionResult = if let engineAware = services.screenCapture
as? any EngineAwareScreenCaptureServiceProtocol {
try await engineAware.withCaptureEngine(enginePreference, operation: runSession)
} else {
try await runSession()
}
self.output(result)
output(result)
self.logger.operationComplete(
"capture_live",
success: true,
metadata: ["frames_kept": result.stats.framesKept]
)
} catch {
self.handleError(error)
handleError(error)
self.logger.operationComplete(
"capture_live",
success: false,
@ -138,7 +142,7 @@ struct CaptureLiveCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFo
extension CaptureLiveCommand {
private func liveCaptureEnginePreference(for scope: CaptureScope) -> CaptureEnginePreference {
let value = (self.captureEngine ?? self.resolvedRuntime.configuration.captureEnginePreference)?
let value = (captureEngine ?? self.resolvedRuntime.configuration.captureEnginePreference)?
.trimmingCharacters(in: .whitespacesAndNewlines)
.lowercased()

View File

@ -3,7 +3,7 @@ import PeekabooCore
@MainActor
extension CaptureLiveCommand {
func focusIfNeeded(appIdentifier: String) async throws {
switch self.captureFocus {
switch captureFocus {
case .background: return
case .auto:
let options = FocusOptions(
@ -13,12 +13,14 @@ extension CaptureLiveCommand {
spaceSwitch: false,
bringToCurrentSpace: false
)
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
try await withCaptureFocusMutation {
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
}
case .foreground:
let options = FocusOptions(
autoFocus: true,
@ -27,12 +29,14 @@ extension CaptureLiveCommand {
spaceSwitch: true,
bringToCurrentSpace: true
)
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
try await withCaptureFocusMutation {
try await ensureFocused(
applicationName: appIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
}
}
}
}

View File

@ -4,43 +4,51 @@ import PeekabooCore
@MainActor
extension ImageCommand {
func focusIfNeeded(appIdentifier: String) async throws {
switch self.captureFocus {
switch captureFocus {
case .background:
return
case .auto:
if await self.hasVisibleCaptureWindow(appIdentifier: appIdentifier) {
if try await self.hasVisibleCaptureWindow(appIdentifier: appIdentifier) {
return
}
if self.windowTitle == nil, await self.isAlreadyFrontmost(appIdentifier: appIdentifier) {
if windowTitle == nil, try await self.isAlreadyFrontmost(appIdentifier: appIdentifier) {
return
}
let focusIdentifier = await self.resolveFocusIdentifier(appIdentifier: appIdentifier)
let focusIdentifier = try await resolveFocusIdentifier(appIdentifier: appIdentifier)
let options = FocusOptions(autoFocus: true, spaceSwitch: false, bringToCurrentSpace: false)
try await ensureFocused(
applicationName: focusIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
try await withCaptureFocusMutation {
try await ensureFocused(
applicationName: focusIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
}
case .foreground:
let focusIdentifier = await self.resolveFocusIdentifier(appIdentifier: appIdentifier)
let focusIdentifier = try await resolveFocusIdentifier(appIdentifier: appIdentifier)
let options = FocusOptions(autoFocus: true, spaceSwitch: true, bringToCurrentSpace: true)
try await ensureFocused(
applicationName: focusIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
try await withCaptureFocusMutation {
try await ensureFocused(
applicationName: focusIdentifier,
windowTitle: self.windowTitle,
options: options,
services: self.services
)
}
}
}
private func hasVisibleCaptureWindow(appIdentifier: String) async -> Bool {
guard let app = try? await self.services.applications.findApplication(identifier: appIdentifier) else {
private func hasVisibleCaptureWindow(appIdentifier: String) async throws -> Bool {
guard let app = try await FocusFailurePolicy.optional({
try await services.applications.findApplication(identifier: appIdentifier)
}) else {
return false
}
let lookupIdentifier = app.bundleIdentifier ?? app.name
guard let response = try? await self.services.applications.listWindows(for: lookupIdentifier, timeout: 1) else {
guard let response = try await FocusFailurePolicy.optional({
try await services.applications.listWindows(for: lookupIdentifier, timeout: 1)
}) else {
return false
}
@ -51,7 +59,7 @@ extension ImageCommand {
return false
}
guard let windowTitle = self.windowTitle?.trimmingCharacters(in: .whitespacesAndNewlines),
guard let windowTitle = windowTitle?.trimmingCharacters(in: .whitespacesAndNewlines),
!windowTitle.isEmpty
else {
return true
@ -62,9 +70,13 @@ extension ImageCommand {
}
}
private func isAlreadyFrontmost(appIdentifier: String) async -> Bool {
guard let frontmost = try? await self.services.applications.getFrontmostApplication(),
let target = try? await self.services.applications.findApplication(identifier: appIdentifier)
private func isAlreadyFrontmost(appIdentifier: String) async throws -> Bool {
guard let frontmost = try await FocusFailurePolicy.optional({
try await services.applications.getFrontmostApplication()
}),
let target = try await FocusFailurePolicy.optional({
try await services.applications.findApplication(identifier: appIdentifier)
})
else {
return false
}
@ -72,8 +84,10 @@ extension ImageCommand {
return frontmost.processIdentifier == target.processIdentifier
}
private func resolveFocusIdentifier(appIdentifier: String) async -> String {
guard let app = try? await self.services.applications.findApplication(identifier: appIdentifier) else {
private func resolveFocusIdentifier(appIdentifier: String) async throws -> String {
guard let app = try await FocusFailurePolicy.optional({
try await services.applications.findApplication(identifier: appIdentifier)
}) else {
return appIdentifier
}
return "PID:\(app.processIdentifier)"

View File

@ -78,6 +78,10 @@ struct ImageCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFormatta
self.resolvedRuntime.services
}
func withCaptureFocusMutation(_ operation: () async throws -> Void) async rethrows {
try await self.resolvedRuntime.withCaptureFocusMutation(operation)
}
var jsonOutput: Bool {
self.runtime?.configuration.jsonOutput ?? self.runtimeOptions.jsonOutput
}
@ -95,7 +99,7 @@ struct ImageCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFormatta
self.runtime = runtime
self.logger.setJsonOutputMode(self.jsonOutput)
let startMetadata: [String: Any] = [
"mode": self.mode?.rawValue ?? "auto",
"mode": mode?.rawValue ?? "auto",
"app": self.app ?? "none",
"pid": self.pid ?? 0,
"hasAnalyzePrompt": self.analyze != nil
@ -103,28 +107,28 @@ struct ImageCommand: ApplicationResolvable, ErrorHandlingCommand, OutputFormatta
self.logger.operationStart("image_command", metadata: startMetadata)
do {
try self.validateStdoutStreamingOptions()
try validateStdoutStreamingOptions()
// ScreenCaptureService performs the authoritative permission check inside each capture path.
// Avoid preflighting here too; it adds fixed latency to every one-shot screenshot.
let captures = try await CrossProcessOperationGate.withExclusiveOperation(
named: CrossProcessOperationGate.desktopObservationName
) {
try await self.performCapture()
try await performCapture()
}
if self.streamsImageToStdout {
try self.outputImageToStdout(captures)
} else if let prompt = self.analyze, let firstFile = captures.first?.file {
let analysis = try await self.analyzeImage(at: firstFile.path, with: prompt)
self.outputResultsWithAnalysis(captures, analysis: analysis)
if streamsImageToStdout {
try outputImageToStdout(captures)
} else if let prompt = analyze, let firstFile = captures.first?.file {
let analysis = try await analyzeImage(at: firstFile.path, with: prompt)
outputResultsWithAnalysis(captures, analysis: analysis)
} else {
self.outputResults(captures)
outputResults(captures)
}
self.logger.operationComplete("image_command", success: true)
} catch {
self.handleError(error)
handleError(error)
self.logger.operationComplete(
"image_command",
success: false,
@ -155,7 +159,7 @@ extension ImageCommand: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {
self.app = values.singleOption("app")
self.pid = try values.decodeOption("pid", as: Int32.self)
self.path = values.singleOption("path")
path = values.singleOption("path")
if let parsedMode: CaptureMode = try values.decodeOptionEnum("mode") {
self.mode = parsedMode
}
@ -169,7 +173,7 @@ extension ImageCommand: CommanderBindableCommand {
if let parsedFormat {
self.format = parsedFormat
}
if let path = self.path?.trimmingCharacters(in: .whitespacesAndNewlines),
if let path = path?.trimmingCharacters(in: .whitespacesAndNewlines),
!path.isEmpty {
let expanded = (path as NSString).expandingTildeInPath
let ext = URL(fileURLWithPath: expanded).pathExtension.lowercased()

View File

@ -37,8 +37,7 @@ extension ListCommand {
self.logger.setJsonOutputMode(self.jsonOutput)
do {
try await requireScreenRecordingPermission(services: self.services)
let output = try await self.services.applications.listApplications()
let output = try await services.applications.listApplications()
if self.jsonOutput {
outputSuccessCodable(data: output.data, logger: self.outputLogger)
@ -46,7 +45,7 @@ extension ListCommand {
print(CLIFormatter.format(output))
}
} catch {
self.handleError(error)
handleError(error)
throw ExitCode(1)
}
}

View File

@ -32,6 +32,12 @@ extension PermissionsCommand.GrantSubcommand: CommanderSignatureProviding {
}
}
extension PermissionsCommand.RequestScreenRecordingSubcommand: CommanderSignatureProviding {
static func commanderSignature() -> CommandSignature {
CommandSignature()
}
}
extension PermissionsCommand.RequestEventSynthesizingSubcommand: CommanderSignatureProviding {
static func commanderSignature() -> CommandSignature {
CommandSignature()

View File

@ -11,6 +11,7 @@ struct PermissionsCommand: ParsableCommand {
subcommands: [
StatusSubcommand.self,
GrantSubcommand.self,
RequestScreenRecordingSubcommand.self,
RequestEventSynthesizingSubcommand.self,
],
defaultSubcommand: StatusSubcommand.self
@ -134,6 +135,57 @@ extension PermissionsCommand {
}
}
@MainActor
struct RequestScreenRecordingSubcommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConfigurable {
struct Result: Codable {
let action: String
let granted: Bool
}
@RuntimeStorage private var runtime: CommandRuntime?
var runtimeOptions = CommandRuntimeOptions()
private var resolvedRuntime: CommandRuntime {
guard let runtime else {
preconditionFailure("CommandRuntime must be configured before accessing runtime resources")
}
return runtime
}
var outputLogger: Logger {
self.resolvedRuntime.logger
}
var jsonOutput: Bool {
self.runtime?.configuration.jsonOutput ?? self.runtimeOptions.jsonOutput
}
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let granted = await PermissionHelpers.performInteractivePermissionRequest(using: runtime) {
runtime.services.permissions.requestScreenRecordingPermission(interactive: true)
}
let result = Result(action: "request-screen-recording", granted: granted)
if self.jsonOutput {
outputSuccessCodable(data: result, logger: self.outputLogger)
return
}
if granted {
print("Screen Recording permission is granted.")
} else {
print("Screen Recording permission was not granted.")
print(
"If no prompt appeared, open System Settings > Privacy & Security > " +
"Screen & System Audio Recording."
)
print("Add or enable the current Peekaboo binary, then restart Peekaboo.")
}
}
}
@MainActor
struct RequestEventSynthesizingSubcommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConfigurable {
@RuntimeStorage private var runtime: CommandRuntime?
@ -158,7 +210,10 @@ extension PermissionsCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
do {
let result = try await PermissionHelpers.requestEventSynthesizingPermission(services: runtime.services)
let result = try await PermissionHelpers.requestEventSynthesizingPermission(
services: runtime.services,
runtime: runtime
)
self.render(result)
} catch {
self.handleError(error)
@ -233,6 +288,27 @@ extension PermissionsCommand.GrantSubcommand: CommanderBindableCommand {
}
}
@MainActor
extension PermissionsCommand.RequestScreenRecordingSubcommand: ParsableCommand {
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
CommandDescription(
commandName: "request-screen-recording",
abstract: "Request Screen Recording permission for the local Peekaboo process"
)
}
}
}
extension PermissionsCommand.RequestScreenRecordingSubcommand: AsyncRuntimeCommand {}
@MainActor
extension PermissionsCommand.RequestScreenRecordingSubcommand: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {
_ = values
}
}
@MainActor
extension PermissionsCommand.RequestEventSynthesizingSubcommand: ParsableCommand {
nonisolated(unsafe) static var commandDescription: CommandDescription {

View File

@ -54,7 +54,7 @@ extension ClickCommand: CommanderSignatureProviding {
),
.commandOption(
"on",
help: "Element ID to click (e.g., B1, T2)",
help: "Opaque element ID copied from current see or inspect-ui output",
long: "on"
),
.commandOption(

View File

@ -48,7 +48,7 @@ extension ClickCommand {
💡 Hints:
Run 'peekaboo see' first to capture UI elements
Check that the element ID is correct (e.g., B1, T2)
Copy the opaque element ID exactly from current see or inspect-ui output
Element may have disappeared or changed
"""
}

View File

@ -14,7 +14,7 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
@Option(help: "Snapshot ID, or 'latest' (uses latest if not specified)")
var snapshot: String?
@Option(help: "Element ID to click (e.g., B1, T2)")
@Option(help: "Opaque element ID copied from current see or inspect-ui output")
var on: String?
@Option(name: .customLong("id"), help: "Element ID to click (alias for --on)")
@ -89,14 +89,14 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
let startTime = Date()
do {
try self.validate()
try validate()
// Determine click target first to check if we need a snapshot
let clickTarget: ClickTarget
let waitResult: WaitForElementResult
var activeSnapshotId: String
var observationForInvalidation: InteractionObservationContext?
var coordinateResolution: InteractionCoordinateResolution?
var explicitWindowResolution: InteractionWindowResolution?
// Check if we're clicking by coordinates (doesn't need snapshot)
if let coordString = coords {
@ -114,6 +114,9 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
clickTarget = .coordinates(resolvedCoordinates.screenPoint)
waitResult = WaitForElementResult(found: true, element: nil, waitTime: 0)
activeSnapshotId = "" // Not needed for coordinate clicks
if !self.usesBackgroundDelivery {
self.resolvedRuntime.beginInteractionMutation()
}
try await self.focusApplicationIfNeeded(
snapshotId: nil,
coordinateResolution: resolvedCoordinates
@ -124,7 +127,7 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
// so if the target window is not frontmost, the click will land on
// whatever window is at that position (see #90).
if !self.usesBackgroundDelivery {
try await self.verifyFocusForCoordinateClick(coordinateResolution: resolvedCoordinates)
try await verifyFocusForCoordinateClick(coordinateResolution: resolvedCoordinates)
}
} else {
@ -137,6 +140,12 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
)
try await observation.validateIfExplicit(using: self.services.snapshots)
explicitWindowResolution = try await self.resolveExplicitWindowSelection(
observation: observation
)
if !self.usesBackgroundDelivery {
self.resolvedRuntime.beginInteractionMutation()
}
try await self.focusApplicationIfNeeded(snapshotId: observation.focusSnapshotId(for: self.target))
// Use whichever element ID parameter was provided
@ -144,20 +153,23 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
if let elementId {
if !self.usesBackgroundDelivery {
let refreshRuntime = self.resolvedRuntime
observation = try await InteractionObservationRefresher.refreshForMissingElementsIfNeeded(
observation,
elementIds: [elementId],
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
refreshRuntime.beginInteractionMutation(at: startedAt)
}
)
}
observationForInvalidation = observation
activeSnapshotId = observation.snapshotId ?? ""
clickTarget = .elementId(elementId)
if self.usesBackgroundDelivery {
let element = try await self.cachedElementById(elementId, observation: observation)
let element = try await cachedElementById(elementId, observation: observation)
waitResult = WaitForElementResult(found: true, element: element, waitTime: 0)
} else {
// Click by element ID with auto-wait
@ -177,11 +189,10 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
if !self.usesBackgroundDelivery {
observation = try await self.refreshObservationIfQueryMissing(observation, query: searchQuery)
}
observationForInvalidation = observation
activeSnapshotId = observation.snapshotId ?? ""
if self.usesBackgroundDelivery {
let element = try await self.cachedElementMatching(searchQuery, observation: observation)
let element = try await cachedElementMatching(searchQuery, observation: observation)
clickTarget = .elementId(element.id)
waitResult = WaitForElementResult(found: true, element: element, waitTime: 0)
} else {
@ -209,24 +220,50 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
}
}
let backgroundProcessIdentifier: pid_t? = if self.usesBackgroundDelivery {
try await self.resolveBackgroundClickProcessIdentifier(
snapshotId: activeSnapshotId.isEmpty ? nil : activeSnapshotId,
coordinateResolution: coordinateResolution,
explicitWindowResolution: explicitWindowResolution
)
} else {
nil
}
// Determine click type
let clickType: ClickType = self.right ? .right : (self.double ? .double : .single)
self.resolvedRuntime.beginInteractionMutation()
try await self.performClick(
clickTarget,
clickType: clickType,
snapshotId: activeSnapshotId,
coordinateResolution: coordinateResolution
coordinateResolution: coordinateResolution,
explicitWindowResolution: explicitWindowResolution,
backgroundProcessIdentifier: backgroundProcessIdentifier
)
// Brief delay to ensure click is processed
try await Task.sleep(nanoseconds: 20_000_000) // 0.02 seconds
try? await Task.sleep(nanoseconds: 20_000_000) // 0.02 seconds
// Result formatting can await bridge lookups. Freeze the mutation boundary first so
// observations created after the click remain eligible as the next implicit latest.
let snapshotInvalidationCutoff = Date()
let appName = await self.resultApplicationName(
// The click already happened. Advance every host watermark before diagnostics that can
// fail if the action closed, moved, or resized its target window.
await InteractionObservationInvalidator.invalidateAfterClickMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "click",
through: snapshotInvalidationCutoff
)
try Task.checkCancellation()
let appName = await resultApplicationName(
snapshotId: activeSnapshotId,
coordinateResolution: coordinateResolution
)
let details = try await self.clickOutputDetails(
let details = try await clickOutputDetails(
clickTarget: clickTarget,
waitResult: waitResult,
snapshotId: activeSnapshotId,
@ -241,8 +278,9 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
waitTime: waitResult.waitTime,
executionTime: Date().timeIntervalSince(startTime),
targetApp: appName,
targetWindowId: coordinateResolution?.targetWindowID,
targetWindowTitle: coordinateResolution?.targetWindowTitle,
targetWindowId: explicitWindowResolution?.windowInfo.windowID ?? coordinateResolution?.targetWindowID,
targetWindowTitle: explicitWindowResolution?.windowInfo.title ?? coordinateResolution?
.targetWindowTitle,
coordinateSpace: coordinateResolution?.coordinateSpace.rawValue,
inputCoordinates: coordinateResolution?.inputPoint,
screenCoordinates: coordinateResolution?.screenPoint,
@ -250,19 +288,10 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
deliveryMode: self.deliveryMode.rawValue
)
if let observationForInvalidation {
await InteractionObservationInvalidator.invalidateAfterMutation(
observationForInvalidation,
snapshots: self.services.snapshots,
logger: self.logger,
reason: "click"
)
}
self.outputSuccess(result)
} catch {
self.handleError(error)
handleError(error)
throw ExitCode.failure
}
}
@ -279,13 +308,11 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
guard let element = waitResult.element else {
return (.zero, "Element ID: \(id)", nil)
}
let resolution = try await InteractionTargetPointResolver.elementCenterResolution(
return try await self.elementOutputDetails(
element: element,
elementId: id,
snapshotId: snapshotId.isEmpty ? nil : snapshotId,
snapshots: self.services.snapshots
snapshotId: snapshotId
)
return (resolution.point, self.formatElementInfo(element), resolution.diagnostics)
case let .coordinates(point):
let diagnostics = if let coordinateResolution {
@ -307,13 +334,44 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
guard let element = waitResult.element else {
return (.zero, "Element matching: \(query)", nil)
}
let resolution = try await InteractionTargetPointResolver.elementCenterResolution(
return try await self.elementOutputDetails(
element: element,
elementId: element.id,
snapshotId: snapshotId.isEmpty ? nil : snapshotId,
snapshotId: snapshotId
)
}
}
private func elementOutputDetails(
element: DetectedElement,
elementId: String,
snapshotId: String
) async throws
-> (location: CGPoint, clickedElement: String?, targetPointDiagnostics: InteractionTargetPointDiagnostics?) {
let resolvedSnapshotId = snapshotId.isEmpty ? nil : snapshotId
do {
let resolution = try await InteractionTargetPointResolver.elementCenterResolution(
element: element,
elementId: elementId,
snapshotId: resolvedSnapshotId,
snapshots: self.services.snapshots
)
return (resolution.point, self.formatElementInfo(element), resolution.diagnostics)
return (resolution.point, formatElementInfo(element), resolution.diagnostics)
} catch let error as CancellationError {
throw error
} catch {
// The click already succeeded; its target may have closed or moved before result formatting.
self.logger.debug("Post-click target diagnostics unavailable: \(error.localizedDescription)")
let point = CGPoint(x: element.bounds.midX, y: element.bounds.midY)
let diagnostics = InteractionTargetPointDiagnostics(
source: InteractionTargetPointSource.element.rawValue,
elementId: elementId,
snapshotId: resolvedSnapshotId,
original: InteractionPoint(point),
resolved: InteractionPoint(point),
windowAdjustment: nil
)
return (point, formatElementInfo(element), diagnostics)
}
}
@ -329,7 +387,7 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
return targetApplicationName
}
if let processIdentifier = coordinateResolution?.targetProcessIdentifier {
return await self.applicationName(processIdentifier: processIdentifier) ?? "PID \(processIdentifier)"
return await applicationName(processIdentifier: processIdentifier) ?? "PID \(processIdentifier)"
}
if let windowID = coordinateResolution?.targetWindowID {
return "window \(windowID)"
@ -339,25 +397,25 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
return await self.frontmostApplicationName()
}
if let pid = self.target.pid {
return await self.applicationName(processIdentifier: pid) ?? "PID \(pid)"
if let pid = target.pid {
return await applicationName(processIdentifier: pid) ?? "PID \(pid)"
}
if let appIdentifier = self.target.app?.trimmingCharacters(in: .whitespacesAndNewlines),
if let appIdentifier = target.app?.trimmingCharacters(in: .whitespacesAndNewlines),
!appIdentifier.isEmpty {
return await (try? self.services.applications.findApplication(identifier: appIdentifier).name) ??
appIdentifier
}
guard !snapshotId.isEmpty,
let snapshot = try? await self.services.snapshots.getUIAutomationSnapshot(snapshotId: snapshotId)
let snapshot = try? await services.snapshots.getUIAutomationSnapshot(snapshotId: snapshotId)
else {
if let detectionResult = try? await self.services.snapshots.getDetectionResult(snapshotId: snapshotId) {
if let detectionResult = try? await services.snapshots.getDetectionResult(snapshotId: snapshotId) {
if let applicationName = detectionResult.metadata.windowContext?.applicationName {
return applicationName
}
if let processId = detectionResult.metadata.windowContext?.applicationProcessId {
return await self.applicationName(processIdentifier: processId) ?? "PID \(processId)"
return await applicationName(processIdentifier: processId) ?? "PID \(processId)"
}
}
return await self.frontmostApplicationName()
@ -368,14 +426,14 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
}
if let processId = snapshot.applicationProcessId {
return await self.applicationName(processIdentifier: processId) ?? "PID \(processId)"
return await applicationName(processIdentifier: processId) ?? "PID \(processId)"
}
return await self.frontmostApplicationName()
}
private func applicationName(processIdentifier: Int32) async -> String? {
guard let output = try? await self.services.applications.listApplications() else {
guard let output = try? await services.applications.listApplications() else {
return nil
}
return output.data.applications.first { $0.processIdentifier == processIdentifier }?.name
@ -420,10 +478,37 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
query: query,
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
self.resolvedRuntime.beginInteractionMutation(at: startedAt)
}
)
}
private func resolveExplicitWindowSelection(
observation: InteractionObservationContext
) async throws -> InteractionWindowResolution? {
guard self.target.windowId != nil || self.target.windowTitle != nil || self.target.windowIndex != nil else {
return nil
}
let resolution = try await InteractionCoordinateResolver.resolveTargetWindow(
target: self.target,
services: self.services
)
guard self.usesBackgroundDelivery else {
return resolution
}
let snapshotId = try observation.requireSnapshot()
let detectionResult = try await observation.requireDetectionResult(using: self.services.snapshots)
try InteractionWindowSelectionValidator.validate(
resolution: resolution,
snapshotContext: detectionResult.metadata.windowContext,
snapshotId: snapshotId
)
return resolution
}
private func cachedElementById(
_ elementId: String,
observation: InteractionObservationContext
@ -491,7 +576,9 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
_ target: ClickTarget,
clickType: ClickType,
snapshotId: String,
coordinateResolution: InteractionCoordinateResolution?
coordinateResolution: InteractionCoordinateResolution?,
explicitWindowResolution: InteractionWindowResolution?,
backgroundProcessIdentifier: pid_t?
) async throws {
let effectiveSnapshotId: String? = if case .coordinates = target {
nil
@ -500,16 +587,16 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
}
if self.usesBackgroundDelivery {
let pid = try await self.resolveBackgroundClickProcessIdentifier(
snapshotId: effectiveSnapshotId,
coordinateResolution: coordinateResolution
)
guard let backgroundProcessIdentifier else {
preconditionFailure("Background process identifier must be resolved before click delivery")
}
try await AutomationServiceBridge.click(
automation: self.services.automation,
target: target,
clickType: clickType,
snapshotId: effectiveSnapshotId,
targetProcessIdentifier: pid
targetProcessIdentifier: backgroundProcessIdentifier,
targetWindowID: explicitWindowResolution?.windowInfo.windowID ?? coordinateResolution?.targetWindowID
)
} else {
try await AutomationServiceBridge.click(
@ -574,22 +661,27 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
private func resolveBackgroundClickProcessIdentifier(
snapshotId: String?,
coordinateResolution: InteractionCoordinateResolution?
coordinateResolution: InteractionCoordinateResolution?,
explicitWindowResolution: InteractionWindowResolution?
) async throws -> pid_t {
if self.target.pid != nil, self.target.app != nil {
throw ValidationError("Background click accepts one process target: use --app or --pid")
}
if let pid = self.target.pid {
if let processId = explicitWindowResolution?.targetProcessIdentifier {
return pid_t(processId)
}
if let pid = target.pid {
guard pid > 0 else {
throw ValidationError("--pid must be greater than 0")
}
return pid_t(pid)
}
if let appIdentifier = self.target.app?.trimmingCharacters(in: .whitespacesAndNewlines),
if let appIdentifier = target.app?.trimmingCharacters(in: .whitespacesAndNewlines),
!appIdentifier.isEmpty {
let app = try await self.services.applications.findApplication(identifier: appIdentifier)
let app = try await services.applications.findApplication(identifier: appIdentifier)
return pid_t(app.processIdentifier)
}
@ -598,13 +690,13 @@ struct ClickCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
}
if let snapshotId,
let snapshot = try? await self.services.snapshots.getUIAutomationSnapshot(snapshotId: snapshotId),
let snapshot = try? await services.snapshots.getUIAutomationSnapshot(snapshotId: snapshotId),
let processId = snapshot.applicationProcessId {
return pid_t(processId)
}
if let snapshotId,
let detectionResult = try? await self.services.snapshots.getDetectionResult(snapshotId: snapshotId),
let detectionResult = try? await services.snapshots.getDetectionResult(snapshotId: snapshotId),
let processId = detectionResult.metadata.windowContext?.applicationProcessId {
return pid_t(processId)
}

View File

@ -80,12 +80,16 @@ struct DragCommand: ErrorHandlingCommand, OutputFormattable {
fallbackToLatest: needsSnapshot,
snapshots: self.services.snapshots
)
let refreshRuntime = self.resolvedRuntime
observation = try await InteractionObservationRefresher.refreshForMissingElementsIfNeeded(
observation,
elementIds: [self.from, self.to],
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
refreshRuntime.beginInteractionMutation(at: startedAt)
}
)
if needsSnapshot {
_ = try await observation.requireDetectionResult(using: self.services.snapshots)
@ -93,6 +97,7 @@ struct DragCommand: ErrorHandlingCommand, OutputFormattable {
try await observation.validateIfExplicit(using: self.services.snapshots)
}
self.resolvedRuntime.beginInteractionMutation()
try await ensureFocused(
snapshotId: observation.focusSnapshotId(for: self.target),
target: self.target,
@ -157,13 +162,13 @@ struct DragCommand: ErrorHandlingCommand, OutputFormattable {
+ "profile=\(movement.profileName)"
)
try await Task.sleep(nanoseconds: 100_000_000)
try? await Task.sleep(nanoseconds: 100_000_000)
await InteractionObservationInvalidator.invalidateAfterMutation(
observation,
snapshots: self.services.snapshots,
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "drag"
)
try Task.checkCancellation()
let result = DragResult(
success: true,
@ -254,11 +259,11 @@ extension DragCommand: ParsableCommand {
Execute click-and-drag operations for moving elements, selecting text, or dragging files.
EXAMPLES:
peekaboo drag --from B1 --to T2
peekaboo drag --from "$SOURCE_ID" --to "$TARGET_ID"
peekaboo drag --from-coords "100,200" --to-coords "400,300"
peekaboo drag --from B1 --to-app Trash
peekaboo drag --from S1 --to-coords "500,250" --duration 2000
peekaboo drag --from T1 --to T5 --modifiers shift
peekaboo drag --from "$SOURCE_ID" --to-app Trash
peekaboo drag --from "$SOURCE_ID" --to-coords "500,250" --duration 2000
peekaboo drag --from "$SOURCE_ID" --to "$TARGET_ID" --modifiers shift
""",
version: "2.0.0",
showHelpOnEmptyInvocation: true

View File

@ -96,6 +96,7 @@ struct HotkeyCommand: ErrorHandlingCommand, OutputFormattable {
let targetPID: pid_t?
let backgroundPID = try await self.backgroundProcessIdentifier(snapshotId: observation.snapshotId)
self.resolvedRuntime.beginInteractionMutation()
if let backgroundPID {
try self.validateBackgroundHotkeyOptions(snapshotId: observation.snapshotId)
@ -124,9 +125,8 @@ struct HotkeyCommand: ErrorHandlingCommand, OutputFormattable {
targetPID = nil
}
await InteractionObservationInvalidator.invalidateAfterMutationOrLatest(
observation,
snapshots: self.services.snapshots,
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "hotkey"
)

View File

@ -16,7 +16,7 @@ extension MoveCommand: ParsableCommand {
EXAMPLES:
peekaboo move 100,200 # Move to coordinates
peekaboo move --to "Submit Button" # Move to element by text
peekaboo move --on B3 # Move to element by ID
peekaboo move --on "$ELEMENT_ID" # ID copied from current output
peekaboo move 500,300 --smooth # Smooth movement
peekaboo move --center # Move to screen center
@ -84,7 +84,7 @@ extension MoveCommand: CommanderSignatureProviding {
),
.commandOption(
"on",
help: "Element ID to move to (e.g., B1, T2)",
help: "Opaque element ID copied from current see or inspect-ui output",
long: "on"
),
.commandOption(

View File

@ -17,7 +17,7 @@ struct MoveCommand: ErrorHandlingCommand, OutputFormattable {
@Option(help: "Move to element by text/label")
var to: String?
@Option(help: "Element ID to move to (e.g., B1, T2)")
@Option(help: "Opaque element ID copied from current see or inspect-ui output")
var on: String?
@Option(name: .customLong("id"), help: "Element ID to move to (alias for --on)")
@ -130,6 +130,7 @@ struct MoveCommand: ErrorHandlingCommand, OutputFormattable {
)
// Perform the movement
self.resolvedRuntime.beginInteractionMutation()
try await AutomationServiceBridge.moveMouse(
automation: self.services.automation,
to: targetLocation,
@ -220,6 +221,7 @@ struct MoveCommand: ErrorHandlingCommand, OutputFormattable {
}
private func focusForCoordinateTarget() async throws {
self.resolvedRuntime.beginInteractionMutation()
try await ensureFocused(
snapshotId: nil,
target: self.target,
@ -239,8 +241,12 @@ struct MoveCommand: ErrorHandlingCommand, OutputFormattable {
elementIds: [elementId],
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
self.resolvedRuntime.beginInteractionMutation(at: startedAt)
}
)
self.resolvedRuntime.beginInteractionMutation()
try await ensureFocused(
snapshotId: observation.focusSnapshotId(for: self.target),
target: self.target,
@ -277,9 +283,13 @@ struct MoveCommand: ErrorHandlingCommand, OutputFormattable {
query: query,
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
self.resolvedRuntime.beginInteractionMutation(at: startedAt)
}
)
let activeSnapshotId = try observation.requireSnapshot()
self.resolvedRuntime.beginInteractionMutation()
try await ensureFocused(
snapshotId: observation.focusSnapshotId(for: self.target),
target: self.target,

View File

@ -90,6 +90,7 @@ struct PasteCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
if let targetPID,
let text = self.resolvedText {
let setResult = try Self.readResult(for: request)
self.resolvedRuntime.beginInteractionMutation()
_ = try await AutomationServiceBridge.typeActions(
automation: self.services.automation,
request: TypeActionsRequest(
@ -99,8 +100,8 @@ struct PasteCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
),
targetProcessIdentifier: targetPID
)
await InteractionObservationInvalidator.invalidateLatestSnapshot(
using: self.services.snapshots,
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "paste"
)
@ -126,6 +127,7 @@ struct PasteCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
return
}
self.resolvedRuntime.beginInteractionMutation()
if targetPID == nil {
try await ensureFocused(
snapshotId: nil,
@ -170,8 +172,8 @@ struct PasteCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
holdDuration: 50
)
}
await InteractionObservationInvalidator.invalidateLatestSnapshot(
using: self.services.snapshots,
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "paste"
)

View File

@ -52,6 +52,7 @@ struct PerformActionCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOpt
let observation = await self.resolveObservationContext()
try await observation.validateIfExplicit(using: self.services.snapshots)
let startTime = Date()
self.resolvedRuntime.beginInteractionMutation()
let result = try await AutomationServiceBridge.performAction(
automation: self.services.automation,
target: target,
@ -59,8 +60,7 @@ struct PerformActionCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOpt
snapshotId: observation.snapshotId
)
await InteractionObservationInvalidator.invalidateAfterMutation(
observation,
snapshots: self.services.snapshots,
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "perform-action"
)
@ -116,7 +116,7 @@ extension PerformActionCommand: ParsableCommand {
Invokes an accessibility action without synthesizing a mouse or keyboard event.
EXAMPLES:
peekaboo perform-action --on B1 --action AXPress
peekaboo perform-action --on "$ELEMENT_ID" --action AXPress
peekaboo perform-action --on Stepper --action AXIncrement
""",
showHelpOnEmptyInvocation: true

View File

@ -80,6 +80,7 @@ struct PressCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
try await observation.validateIfExplicit(using: self.services.snapshots)
let targetPID = try await self.backgroundProcessIdentifier(snapshotId: observation.snapshotId)
self.resolvedRuntime.beginInteractionMutation()
if targetPID == nil {
try await ensureFocused(
snapshotId: observation.focusSnapshotId(for: self.target),
@ -126,9 +127,8 @@ struct PressCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
}
}
await InteractionObservationInvalidator.invalidateAfterMutationOrLatest(
observation,
snapshots: self.services.snapshots,
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "press"
)

View File

@ -76,12 +76,16 @@ struct ScrollCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsCon
)
if let elementId = self.on {
let refreshRuntime = self.resolvedRuntime
observation = try await InteractionObservationRefresher.refreshForMissingElementsIfNeeded(
observation,
elementIds: [elementId],
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
refreshRuntime.beginInteractionMutation(at: startedAt)
}
)
_ = try await observation.requireDetectionResult(using: self.services.snapshots)
} else {
@ -89,6 +93,7 @@ struct ScrollCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsCon
}
// Ensure window is focused before scrolling
self.resolvedRuntime.beginInteractionMutation()
try await ensureFocused(
snapshotId: observation.focusSnapshotId(for: self.target),
target: self.target,
@ -109,6 +114,11 @@ struct ScrollCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsCon
automation: self.services.automation,
request: scrollRequest
)
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "scroll"
)
AutomationEventLogger.log(
.scroll,
"direction=\(self.direction) amount=\(self.amount) smooth=\(self.smooth) "
@ -140,13 +150,6 @@ struct ScrollCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsCon
}
let scrollLocation = scrollResolution.point
await InteractionObservationInvalidator.invalidateAfterMutation(
observation,
snapshots: self.services.snapshots,
logger: self.logger,
reason: "scroll"
)
// Output results
let outputPayload = ScrollResult(
success: true,

View File

@ -52,6 +52,7 @@ struct SetValueCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsC
let observation = await self.resolveObservationContext()
try await observation.validateIfExplicit(using: self.services.snapshots)
let startTime = Date()
self.resolvedRuntime.beginInteractionMutation()
let result = try await AutomationServiceBridge.setValue(
automation: self.services.automation,
target: target,
@ -59,8 +60,7 @@ struct SetValueCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsC
snapshotId: observation.snapshotId
)
await InteractionObservationInvalidator.invalidateAfterMutation(
observation,
snapshots: self.services.snapshots,
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "set-value"
)
@ -119,7 +119,7 @@ extension SetValueCommand: ParsableCommand {
Sets a settable accessibility value without synthesizing keystrokes.
EXAMPLES:
peekaboo set-value "hello" --on T1
peekaboo set-value "hello" --on "$ELEMENT_ID"
peekaboo set-value "42" --on "Search"
""",
showHelpOnEmptyInvocation: true

View File

@ -98,12 +98,16 @@ struct SwipeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
fallbackToLatest: needsSnapshotForElements,
snapshots: self.services.snapshots
)
let refreshRuntime = self.resolvedRuntime
observation = try await InteractionObservationRefresher.refreshForMissingElementsIfNeeded(
observation,
elementIds: [self.from, self.to],
target: self.target,
services: self.services,
logger: self.logger
logger: self.logger,
beforeRefresh: { startedAt in
refreshRuntime.beginInteractionMutation(at: startedAt)
}
)
if needsSnapshotForElements {
@ -112,6 +116,7 @@ struct SwipeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
try await observation.validateIfExplicit(using: self.services.snapshots)
}
self.resolvedRuntime.beginInteractionMutation()
try await ensureFocused(
snapshotId: observation.focusSnapshotId(for: self.target),
target: self.target,
@ -173,13 +178,13 @@ struct SwipeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConf
)
// Small delay to ensure swipe is processed
try await Task.sleep(nanoseconds: 100_000_000) // 0.1 seconds
try? await Task.sleep(nanoseconds: 100_000_000) // 0.1 seconds
await InteractionObservationInvalidator.invalidateAfterMutation(
observation,
snapshots: self.services.snapshots,
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "swipe"
)
try Task.checkCancellation()
let outputPayload = SwipeResult(
success: true,
@ -244,20 +249,20 @@ extension SwipeCommand: ParsableCommand {
EXAMPLES:
# Swipe between UI elements
peekaboo swipe --from B1 --to B5 --snapshot 12345
peekaboo swipe --from "$SOURCE_ID" --to "$TARGET_ID" --snapshot "$SNAPSHOT_ID"
# Swipe with coordinates
peekaboo swipe --from-coords 100,200 --to-coords 300,400
# Mixed mode: element to coordinates
peekaboo swipe --from T1 --to-coords 500,300 --duration 1000
peekaboo swipe --from "$SOURCE_ID" --to-coords 500,300 --duration 1000
# Slow swipe for precise gesture
peekaboo swipe --from-coords 50,50 --to-coords 400,400 --duration 2000
USAGE:
You can specify source and destination using either:
- Element IDs from a previous 'see' command
- Opaque element IDs copied from current 'see' or 'inspect-ui' output
- Direct coordinates
- A mix of both

View File

@ -28,7 +28,7 @@ extension TypeCommand: CommanderSignatureProviding {
),
.commandOption(
"profile",
help: "Typing profile: human (default) or linear",
help: "Typing profile: linear (default) or human",
long: "profile"
),
.commandOption(

View File

@ -17,13 +17,13 @@ struct TypeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConfi
var snapshot: String?
@Option(help: "Delay between keystrokes in milliseconds")
var delay: Int = 2
var delay: Int = 0
@Option(name: .customLong("wpm"), help: "Approximate human typing speed (words per minute)")
var wordsPerMinute: Int?
@Option(name: .customLong("profile"), help: "Typing profile: human (default) or linear")
var profileOption: String? = TypingProfile.human.rawValue
@Option(name: .customLong("profile"), help: "Typing profile: linear (default) or human")
var profileOption: String?
@Flag(names: [.customLong("return"), .long], help: "Press return/enter after typing")
var pressReturn = false
@ -86,7 +86,7 @@ struct TypeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConfi
let selection = TypingProfile(rawValue: profileOption.lowercased()) {
return selection
}
return .human
return self.wordsPerMinute == nil ? .linear : .human
}
private var resolvedWordsPerMinute: Int {
@ -112,6 +112,7 @@ struct TypeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConfi
let observation = await self.resolveObservationContext()
try await observation.validateIfExplicit(using: self.services.snapshots)
let targetPID = try await self.backgroundProcessIdentifier(snapshotId: observation.snapshotId)
self.resolvedRuntime.beginInteractionMutation()
if targetPID == nil {
self.warnIfFocusUnknown(snapshotId: observation.snapshotId)
try await self.focusIfNeeded(snapshotId: observation.focusSnapshotId(for: self.target))
@ -122,8 +123,7 @@ struct TypeCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsConfi
targetProcessIdentifier: targetPID
)
await InteractionObservationInvalidator.invalidateAfterMutation(
observation,
snapshots: self.services.snapshots,
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "type"
)
@ -392,9 +392,9 @@ extension TypeCommand: ParsableCommand {
Use --foreground only when the target requires focused keyboard input.
Without a target, keys are injected into the current focused element.
HUMAN TYPING:
Use --profile human (default) for realistic cadence; override speed with --wpm (80-220).
Use --profile linear for deterministic timing via --delay.
TYPING CADENCE:
Linear typing is the default and uses --delay (0ms by default).
Use --profile human or --wpm (80-220) for realistic cadence.
""",
showHelpOnEmptyInvocation: true

View File

@ -101,9 +101,13 @@ struct BrowserCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsCo
self.logger.setJsonOutputMode(self.jsonOutput)
do {
let arguments = try self.arguments()
if Self.actionMayMutate(self.action) {
self.resolvedRuntime.beginInteractionMutation()
}
let context = MCPToolContext(services: self.services)
let tool = BrowserTool(context: context)
let response = try await tool.execute(arguments: ToolArguments(raw: self.arguments()))
let response = try await tool.execute(arguments: ToolArguments(raw: arguments))
try MCPToolCommandOutput.output(
tool: tool.name,
response: response,
@ -118,6 +122,20 @@ struct BrowserCommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptionsCo
}
}
static func actionMayMutate(_ rawAction: String) -> Bool {
let normalized = rawAction
.trimmingCharacters(in: .whitespacesAndNewlines)
.replacingOccurrences(of: "-", with: "_")
guard let action = BrowserAction(rawValue: normalized) else { return false }
switch action {
case .status, .connect, .disconnect, .listPages, .waitFor, .snapshot, .console, .network, .screenshot:
return false
case .selectPage, .closePage, .newPage, .navigate, .click, .fill, .fillForm, .drag, .hover, .type,
.pressKey, .uploadFile, .handleDialog, .performanceTrace, .call:
return true
}
}
private func arguments() throws -> [String: Any] {
let normalizedAction = self.action
.trimmingCharacters(in: .whitespacesAndNewlines)

View File

@ -66,9 +66,15 @@ struct InspectUICommand: ErrorHandlingCommand, OutputFormattable, RuntimeOptions
self.logger.setJsonOutputMode(self.jsonOutput)
do {
let context = MCPToolContext(services: self.services)
let context = MCPToolContext(
services: self.services,
snapshotMutationCoordinator: runtime.toolSnapshotMutationCoordinator
)
let tool = InspectUITool(context: context)
let response = try await tool.execute(arguments: ToolArguments(raw: self.arguments()))
let response = try await context.execute(
tool: tool,
arguments: ToolArguments(raw: self.arguments())
)
try MCPToolCommandOutput.output(
tool: tool.name,
response: response,

View File

@ -36,6 +36,7 @@ extension MCPCommand {
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
var localDaemon: PeekabooDaemon?
do {
guard let transportType = Self.transportType(named: self.transport) else {
runtime.logger.setJsonOutputMode(runtime.configuration.jsonOutput)
@ -51,20 +52,53 @@ extension MCPCommand {
if runtime.services is RemotePeekabooServices {
runtime.logger.debug("MCP: using remote Bridge host; skipping local daemon startup")
} else {
let daemon = PeekabooDaemon(configuration: .mcp())
await daemon.start()
let daemon = PeekabooDaemon(configuration: .embeddedMCP())
localDaemon = daemon
try await daemon.startChecked()
}
let server = try await PeekabooMCPServer()
let mutationCoordinator = runtime.toolSnapshotMutationCoordinator
let toolContext = Self.makeToolContext(
services: runtime.services,
snapshotMutationCoordinator: mutationCoordinator
)
let server = try await PeekabooMCPServer(toolContext: toolContext)
try await server.serve(transport: transportType, port: self.port)
await Self.stopLocalDaemon(localDaemon)
} catch let exitCode as ExitCode {
await Self.stopLocalDaemon(localDaemon)
throw exitCode
} catch {
await Self.stopLocalDaemon(localDaemon)
runtime.logger.error("Failed to start MCP server: \(error)")
throw ExitCode.failure
}
}
private static func stopLocalDaemon(_ daemon: PeekabooDaemon?) async {
guard let daemon, await daemon.requestStop() else { return }
await daemon.waitUntilStopped()
}
static func makeToolContext(
services: any PeekabooServiceProviding,
snapshotMutationCoordinator: (any MCPToolSnapshotMutationCoordinating)?
) -> MCPToolContext {
let snapshotExecutionGate: MCPToolSnapshotExecutionGate
if let agent = services.agent as? PeekabooAgentService {
agent.configureSnapshotMutationCoordinator(snapshotMutationCoordinator)
snapshotExecutionGate = agent.snapshotExecutionGate
} else {
snapshotExecutionGate = MCPToolSnapshotExecutionGate()
}
return MCPToolContext(
services: services,
snapshotMutationCoordinator: snapshotMutationCoordinator,
snapshotExecutionGate: snapshotExecutionGate
)
}
static func transportType(named name: String) -> PeekabooCore.TransportType? {
switch name.lowercased() {
case "stdio": .stdio

View File

@ -34,7 +34,41 @@ enum FocusTargetResolver {
}
}
enum FocusFailurePolicy {
static func optional<T>(_ operation: () async throws -> T) async throws -> T? {
do {
try Task.checkCancellation()
let result = try await operation()
try Task.checkCancellation()
return result
} catch {
try self.rethrowCancellation(error)
return nil
}
}
static func flatteningOptional<T>(_ operation: () async throws -> T?) async throws -> T? {
do {
try Task.checkCancellation()
let result = try await operation()
try Task.checkCancellation()
return result
} catch {
try self.rethrowCancellation(error)
return nil
}
}
static func rethrowCancellation(_ error: any Error) throws {
if error is CancellationError {
throw error
}
try Task.checkCancellation()
}
}
/// Ensure the target window is focused before executing a command.
@MainActor
func ensureFocused(
snapshotId: String? = nil,
windowID: CGWindowID? = nil,
@ -43,6 +77,7 @@ func ensureFocused(
options: any FocusOptionsProtocol,
services: any PeekabooServiceProviding
) async throws {
try Task.checkCancellation()
guard options.autoFocus else {
return
}
@ -54,6 +89,7 @@ func ensureFocused(
} else {
nil as UIAutomationSnapshot?
}
try Task.checkCancellation()
let targetRequest = FocusTargetResolver.resolve(
windowID: windowID,
@ -66,7 +102,9 @@ func ensureFocused(
case let .windowId(windowID):
windowID
case let .bestWindow(applicationName, windowTitle):
try? await focusService.findBestWindow(applicationName: applicationName, windowTitle: windowTitle)
try await FocusFailurePolicy.flatteningOptional {
try await focusService.findBestWindow(applicationName: applicationName, windowTitle: windowTitle)
}
case nil:
nil
}
@ -74,7 +112,9 @@ func ensureFocused(
guard let windowID = targetWindow else {
if case let .bestWindow(applicationName, _) = targetRequest {
_ = try await services.applications.findApplication(identifier: applicationName)
try Task.checkCancellation()
try await services.applications.activateApplication(identifier: applicationName)
try Task.checkCancellation()
}
return
}
@ -86,8 +126,10 @@ func ensureFocused(
bringToCurrentSpace: options.bringToCurrentSpace
)
try Task.checkCancellation()
do {
try await focusService.focusWindow(windowID: windowID, options: focusOptions)
try Task.checkCancellation()
} catch let error as FocusError {
switch error {
case .windowNotFound, .axElementNotFound:
@ -99,19 +141,25 @@ func ensureFocused(
fallbackTargets.append(.frontmost)
for target in fallbackTargets {
try Task.checkCancellation()
do {
try await WindowServiceBridge.focusWindow(windows: services.windows, target: target)
try Task.checkCancellation()
return
} catch {
try FocusFailurePolicy.rethrowCancellation(error)
fallbackErrors.append(error)
}
}
if let appName = applicationName {
try Task.checkCancellation()
do {
try await services.applications.activateApplication(identifier: appName)
try Task.checkCancellation()
return
} catch {
try FocusFailurePolicy.rethrowCancellation(error)
fallbackErrors.append(error)
}
}
@ -124,6 +172,7 @@ func ensureFocused(
}
/// Ensure focus using shared interaction target flags (`--app/--pid/--window-title/--window-index`).
@MainActor
func ensureFocused(
snapshotId: String? = nil,
target: InteractionTargetOptions,

View File

@ -12,7 +12,7 @@ enum InteractionCoordinateResolver {
services: any PeekabooServiceProviding,
forceGlobal: Bool = false
) async throws -> InteractionCoordinateResolution {
guard target.hasAnyTarget, !forceGlobal else {
guard target.hasAnyTarget else {
return InteractionCoordinateResolution(
inputPoint: inputPoint,
screenPoint: inputPoint,
@ -22,8 +22,33 @@ enum InteractionCoordinateResolver {
)
}
let hasWindowSelector = target.windowId != nil || target.windowTitle != nil || target.windowIndex != nil
if forceGlobal, !hasWindowSelector {
return InteractionCoordinateResolution(
inputPoint: inputPoint,
screenPoint: inputPoint,
coordinateSpace: .global,
windowInfo: nil,
targetApplication: nil
)
}
let windowResolution = try await self.resolveTargetWindow(target: target, services: services)
return try self.resolveTargetWindowCoordinates(
inputPoint,
windowInfo: windowResolution.windowInfo,
targetApplication: windowResolution.targetApplication,
forceGlobal: forceGlobal
)
}
static func resolveTargetWindow(
target: InteractionTargetOptions,
services: any PeekabooServiceProviding
) async throws -> InteractionWindowResolution {
guard let windowTarget = try target.toWindowTarget() else {
throw ValidationError("Coordinate target could not be resolved from the supplied target options.")
throw ValidationError("Window target could not be resolved from the supplied target options.")
}
let windowInfo = try await self.resolveWindowInfo(
@ -31,18 +56,12 @@ enum InteractionCoordinateResolver {
target: target,
services: services
)
let targetApplication = await self.resolveTargetApplication(
let targetApplication = try await self.resolveTargetApplication(
windowInfo: windowInfo,
target: target,
services: services
)
return try self.resolveTargetWindowCoordinates(
inputPoint,
windowInfo: windowInfo,
targetApplication: targetApplication
)
return InteractionWindowResolution(windowInfo: windowInfo, targetApplication: targetApplication)
}
static func resolveTargetWindowCoordinates(
@ -51,7 +70,7 @@ enum InteractionCoordinateResolver {
targetApplication: ServiceApplicationInfo?,
forceGlobal: Bool = false
) throws -> InteractionCoordinateResolution {
guard let windowInfo, !forceGlobal else {
guard let windowInfo else {
return InteractionCoordinateResolution(
inputPoint: inputPoint,
screenPoint: inputPoint,
@ -61,6 +80,16 @@ enum InteractionCoordinateResolver {
)
}
if forceGlobal {
return InteractionCoordinateResolution(
inputPoint: inputPoint,
screenPoint: inputPoint,
coordinateSpace: .global,
windowInfo: windowInfo,
targetApplication: targetApplication
)
}
try self.validate(inputPoint: inputPoint, within: windowInfo)
let screenPoint = CGPoint(
@ -109,9 +138,19 @@ enum InteractionCoordinateResolver {
windowInfo: ServiceWindowInfo,
target: InteractionTargetOptions,
services: any PeekabooServiceProviding
) async -> ServiceApplicationInfo? {
if let identifier = try? target.resolveApplicationIdentifierOptional(),
let application = try? await services.applications.findApplication(identifier: identifier) {
) async throws -> ServiceApplicationInfo? {
if let identifier = try target.resolveApplicationIdentifierOptional() {
let application = try await services.applications.findApplication(identifier: identifier)
if target.windowId != nil {
let applicationWindows = try await services.windows.listWindows(
target: .application("PID:\(application.processIdentifier)")
)
try self.validateWindowOwnership(
windowInfo: windowInfo,
application: application,
applicationWindows: applicationWindows
)
}
return application
}
@ -137,6 +176,19 @@ enum InteractionCoordinateResolver {
return nil
}
static func validateWindowOwnership(
windowInfo: ServiceWindowInfo,
application: ServiceApplicationInfo,
applicationWindows: [ServiceWindowInfo]
) throws {
guard applicationWindows.contains(where: { $0.windowID == windowInfo.windowID }) else {
throw ValidationError(
"Window \(windowInfo.windowID) does not belong to \(application.name) " +
"(PID \(application.processIdentifier))"
)
}
}
private static func targetDescription(_ target: InteractionTargetOptions) -> String {
if let windowId = target.windowId {
return "window id \(windowId)"
@ -165,6 +217,46 @@ enum InteractionCoordinateResolver {
}
}
struct InteractionWindowResolution {
let windowInfo: ServiceWindowInfo
let targetApplication: ServiceApplicationInfo?
var targetProcessIdentifier: Int32? {
self.targetApplication?.processIdentifier
}
}
enum InteractionWindowSelectionValidator {
static func validate(
resolution: InteractionWindowResolution,
snapshotContext: WindowContext?,
snapshotId: String
) throws {
guard let snapshotWindowID = snapshotContext?.windowID else {
throw ValidationError(
"Snapshot '\(snapshotId)' does not identify an exact window; " +
"capture a fresh snapshot for the selected window"
)
}
guard snapshotWindowID == resolution.windowInfo.windowID else {
throw ValidationError(
"Snapshot '\(snapshotId)' belongs to window \(snapshotWindowID), but the explicit selector " +
"resolved window \(resolution.windowInfo.windowID)"
)
}
if let snapshotPID = snapshotContext?.applicationProcessId,
let selectedPID = resolution.targetProcessIdentifier,
snapshotPID != selectedPID {
throw ValidationError(
"Snapshot '\(snapshotId)' belongs to PID \(snapshotPID), but the selected window " +
"belongs to PID \(selectedPID)"
)
}
}
}
struct InteractionCoordinateResolution {
let inputPoint: CGPoint
let screenPoint: CGPoint

View File

@ -106,6 +106,17 @@ struct InteractionObservationContext {
struct InteractionObservationRefreshDependencies {
let desktopObservation: any DesktopObservationServiceProtocol
let snapshots: any SnapshotManagerProtocol
let beginMutation: ((Date) -> Void)?
init(
desktopObservation: any DesktopObservationServiceProtocol,
snapshots: any SnapshotManagerProtocol,
beginMutation: ((Date) -> Void)? = nil
) {
self.desktopObservation = desktopObservation
self.snapshots = snapshots
self.beginMutation = beginMutation
}
}
@MainActor
@ -115,7 +126,8 @@ enum InteractionObservationRefresher {
elementIds: [String?],
target: InteractionTargetOptions,
services: any PeekabooServiceProviding,
logger: Logger
logger: Logger,
beforeRefresh: ((Date) -> Void)? = nil
) async throws -> InteractionObservationContext {
var refreshed = observation
for elementId in elementIds.compactMap(\.self) {
@ -124,7 +136,8 @@ enum InteractionObservationRefresher {
elementId: elementId,
target: target,
services: services,
logger: logger
logger: logger,
beforeRefresh: beforeRefresh
)
}
return refreshed
@ -135,7 +148,8 @@ enum InteractionObservationRefresher {
query: String,
target: InteractionTargetOptions,
services: any PeekabooServiceProviding,
logger: Logger
logger: Logger,
beforeRefresh: ((Date) -> Void)? = nil
) async throws -> InteractionObservationContext {
try await self.refreshForMissingQueryIfNeeded(
observation,
@ -143,7 +157,8 @@ enum InteractionObservationRefresher {
target: target,
dependencies: InteractionObservationRefreshDependencies(
desktopObservation: services.desktopObservation,
snapshots: services.snapshots
snapshots: services.snapshots,
beginMutation: beforeRefresh
),
logger: logger
)
@ -180,7 +195,8 @@ enum InteractionObservationRefresher {
elementId: String,
target: InteractionTargetOptions,
services: any PeekabooServiceProviding,
logger: Logger
logger: Logger,
beforeRefresh: ((Date) -> Void)? = nil
) async throws -> InteractionObservationContext {
try await self.refreshForMissingElementIfNeeded(
observation,
@ -188,7 +204,8 @@ enum InteractionObservationRefresher {
target: target,
dependencies: InteractionObservationRefreshDependencies(
desktopObservation: services.desktopObservation,
snapshots: services.snapshots
snapshots: services.snapshots,
beginMutation: beforeRefresh
),
logger: logger
)
@ -228,31 +245,87 @@ enum InteractionObservationRefresher {
logger: Logger
) async throws -> InteractionObservationContext {
let requestTarget = try target.observationTargetRequest()
let result = try await dependencies.desktopObservation.observe(DesktopObservationRequest(
target: requestTarget,
capture: DesktopCaptureOptions(
engine: .auto,
scale: .logical1x,
visualizerMode: .screenshotFlash
),
detection: DesktopDetectionOptions(mode: .accessibility, allowWebFocusFallback: true),
output: DesktopObservationOutputOptions(saveSnapshot: true)
))
guard let refreshedSnapshotId = result.elements?.snapshotId else {
return observation
let observationStartedAt = Date()
dependencies.beginMutation?(observationStartedAt)
let snapshotID = try await dependencies.snapshots.createSnapshot(pendingAt: observationStartedAt)
let result: DesktopObservationResult
do {
result = try await dependencies.desktopObservation.observe(DesktopObservationRequest(
target: requestTarget,
capture: DesktopCaptureOptions(
engine: .auto,
scale: .logical1x,
visualizerMode: .screenshotFlash
),
detection: DesktopDetectionOptions(mode: .accessibility, allowWebFocusFallback: true),
output: DesktopObservationOutputOptions(
saveSnapshot: true,
snapshotID: snapshotID
)
))
guard result.elements != nil else {
try? await dependencies.snapshots.cleanSnapshot(snapshotId: snapshotID)
_ = try? await dependencies.snapshots.invalidateImplicitLatestSnapshot(through: Date())
return observation
}
let publication = try self.certifiedPublicationBoundary(
for: result,
observationStartedAt: observationStartedAt
)
_ = try await dependencies.snapshots.invalidateImplicitLatestSnapshot(
through: publication.cutoff,
preserving: snapshotID,
preservedAt: publication.preservedAt
)
guard await dependencies.snapshots.getMostRecentSnapshot() == snapshotID else {
throw PeekabooError.snapshotStale(
"The refreshed observation was superseded before it could be published"
)
}
} catch {
if !PendingSnapshotCleanupPolicy.shouldPreserveReservation(after: error) {
try? await dependencies.snapshots.cleanSnapshot(snapshotId: snapshotID)
}
_ = try? await dependencies.snapshots.invalidateImplicitLatestSnapshot(through: Date())
throw error
}
logger.debug(
"Refreshed implicit observation snapshot '\(refreshedSnapshotId)' for \(reason)"
"Refreshed implicit observation snapshot '\(snapshotID)' for \(reason)"
)
return InteractionObservationContext(
explicitSnapshotId: nil,
snapshotId: refreshedSnapshotId,
snapshotId: snapshotID,
source: .latest
)
}
private static func certifiedPublicationBoundary(
for result: DesktopObservationResult,
observationStartedAt: Date
) throws -> (cutoff: Date, preservedAt: Date) {
let completedAtValues = [
result.diagnostics.desktopMutationCompletedAt,
result.elements?.metadata.desktopMutationCompletedAt,
].compactMap(\.self)
let preservationValues = [
result.diagnostics.desktopMutationPreservationAllowed,
result.elements?.metadata.desktopMutationPreservationAllowed,
].compactMap(\.self)
let hasCertificate = !completedAtValues.isEmpty || !preservationValues.isEmpty
guard hasCertificate else { return (observationStartedAt, Date()) }
guard let completedAt = completedAtValues.max(),
!preservationValues.isEmpty,
preservationValues.allSatisfy(\.self)
else {
throw PeekabooError.snapshotStale(
"The refreshed observation overlapped another desktop mutation"
)
}
let cutoff = max(observationStartedAt, completedAt)
return (cutoff, cutoff)
}
private static func containsElement(
matching query: String,
in detectionResult: ElementDetectionResult

View File

@ -1,38 +1,440 @@
import Foundation
import PeekabooBridge
import PeekabooCore
import PeekabooFoundation
@MainActor
final class InteractionMutationTracker {
private let desktopMutationWatermarkStore: DesktopMutationWatermarkStore
private var pendingDesktopMutation: DesktopMutationWatermarkStore.PendingMutation?
private var durableMutationLeaseCount = 0
private(set) var mutationStartedAt: Date?
private(set) var mutationSequence: UInt64 = 0
private var successfulCompletionCutoff: Date?
private var failedInvalidationCutoff: Date?
private(set) var preservedSnapshotID: String?
private(set) var preservedAt: Date?
init(desktopMutationWatermarkStore: DesktopMutationWatermarkStore = DesktopMutationWatermarkStore()) {
self.desktopMutationWatermarkStore = desktopMutationWatermarkStore
}
var hasFailedInvalidationAttempt: Bool {
self.failedInvalidationCutoff != nil
}
var hasPendingDurableMutation: Bool {
self.pendingDesktopMutation != nil
}
@discardableResult
func begin(
at cutoff: Date = Date(),
preservingSnapshotsCreatedAfterBoundary: Bool = false
) -> Date {
if self.mutationSequence < UInt64.max {
self.mutationSequence += 1
}
if let failedInvalidationCutoff, cutoff > failedInvalidationCutoff {
self.failedInvalidationCutoff = nil
}
if self.mutationStartedAt == nil {
self.mutationStartedAt = cutoff
}
if preservingSnapshotsCreatedAfterBoundary {
self.successfulCompletionCutoff = max(self.successfulCompletionCutoff ?? cutoff, cutoff)
} else {
self.successfulCompletionCutoff = nil
}
self.preservedSnapshotID = nil
self.preservedAt = nil
return self.mutationStartedAt ?? cutoff
}
func preserveFreshObservation(
snapshotId: String,
startedAt: Date,
preservedAt: Date,
preservationAllowed: Bool = true
) {
guard self.mutationStartedAt != nil else { return }
guard preservationAllowed else {
self.successfulCompletionCutoff = nil
self.preservedSnapshotID = nil
self.preservedAt = nil
return
}
self.successfulCompletionCutoff = max(self.successfulCompletionCutoff ?? startedAt, startedAt)
self.preservedSnapshotID = snapshotId
self.preservedAt = preservedAt
}
@discardableResult
func beginDurableMutation(at startedAt: Date = Date()) throws -> Bool {
guard self.pendingDesktopMutation == nil else { return false }
self.pendingDesktopMutation = try self.desktopMutationWatermarkStore.beginMutation(at: startedAt)
self.durableMutationLeaseCount = 1
return true
}
func retainDurableMutationLease(at startedAt: Date = Date()) throws {
if self.pendingDesktopMutation == nil {
self.pendingDesktopMutation = try self.desktopMutationWatermarkStore.beginMutation(at: startedAt)
self.durableMutationLeaseCount = 1
} else {
self.durableMutationLeaseCount += 1
}
}
func completeDurableMutation(
through cutoff: Date
) throws -> DesktopMutationWatermarkStore.MutationCompletion? {
guard let pendingDesktopMutation else { return nil }
if self.durableMutationLeaseCount > 1 {
self.durableMutationLeaseCount -= 1
return nil
}
let completion = try self.desktopMutationWatermarkStore.completeMutation(
pendingDesktopMutation,
through: cutoff
)
self.pendingDesktopMutation = nil
self.durableMutationLeaseCount = 0
return completion
}
func cancelDurableMutation() throws {
guard let pendingDesktopMutation else { return }
if self.durableMutationLeaseCount > 1 {
self.durableMutationLeaseCount -= 1
return
}
try self.desktopMutationWatermarkStore.cancelMutation(pendingDesktopMutation)
self.pendingDesktopMutation = nil
self.durableMutationLeaseCount = 0
}
func withPendingDurableMutationVisible<T>(
createdByCurrentCommand: Bool,
operation: () async throws -> T
) async rethrows -> T {
guard createdByCurrentCommand, let pendingDesktopMutation else {
return try await operation()
}
return try await DesktopMutationWatermarkStore.withPendingMutationVisible(
pendingDesktopMutation,
operation: operation
)
}
func invalidationCutoff(commandCompletedAt completion: Date, succeeded: Bool) -> Date? {
guard self.mutationStartedAt != nil else { return nil }
if let failedInvalidationCutoff {
return failedInvalidationCutoff
}
if succeeded, let successfulCompletionCutoff {
return successfulCompletionCutoff
}
return completion
}
func markInvalidationFailed(through cutoff: Date) {
guard let mutationStartedAt, mutationStartedAt <= cutoff else { return }
self.failedInvalidationCutoff = min(self.failedInvalidationCutoff ?? cutoff, cutoff)
}
func markInvalidated(through cutoff: Date) {
guard let mutationStartedAt, mutationStartedAt <= cutoff else { return }
self.mutationStartedAt = nil
self.successfulCompletionCutoff = nil
self.failedInvalidationCutoff = nil
self.preservedSnapshotID = nil
self.preservedAt = nil
}
}
@MainActor
extension InteractionObservationContext {
@discardableResult
func invalidateAfterMutation(using snapshots: any SnapshotManagerProtocol) async throws -> String? {
guard self.source == .latest, let snapshotId else {
func invalidateAfterMutation(
using snapshots: any SnapshotManagerProtocol,
through cutoff: Date = Date()
) async throws -> String? {
guard source == .latest, let snapshotId else {
return nil
}
try await snapshots.cleanSnapshot(snapshotId: snapshotId)
guard try await snapshots.invalidateImplicitLatestSnapshot(through: cutoff) != nil else {
return nil
}
return snapshotId
}
static func invalidateLatestSnapshot(using snapshots: any SnapshotManagerProtocol) async throws -> String? {
guard let latestSnapshotId = await snapshots.getMostRecentSnapshot() else {
return nil
}
try await snapshots.cleanSnapshot(snapshotId: latestSnapshotId)
return latestSnapshotId
static func invalidateLatestSnapshot(
using snapshots: any SnapshotManagerProtocol,
through cutoff: Date = Date(),
preserving snapshotId: String? = nil,
preservedAt: Date? = nil
) async throws -> String? {
try await snapshots.invalidateImplicitLatestSnapshot(
through: cutoff,
preserving: snapshotId,
preservedAt: preservedAt
)
}
}
@MainActor
enum InteractionObservationInvalidator {
struct MutationTargets {
let snapshots: any SnapshotManagerProtocol
let selectedRemoteSocketPath: String?
let remoteSocketPaths: [String]
let socketExists: (String) -> Bool
let makeLocalSnapshotManager: () -> any SnapshotManagerProtocol
let makeRemoteSnapshotManager: (String) async throws -> (any SnapshotManagerProtocol)?
let mutationTracker: InteractionMutationTracker?
init(
snapshots: any SnapshotManagerProtocol,
selectedRemoteSocketPath: String?,
remoteSocketPaths: [String],
socketExists: @escaping (String) -> Bool = { FileManager.default.fileExists(atPath: $0) },
makeLocalSnapshotManager: @escaping () -> any SnapshotManagerProtocol = {
SnapshotManager(desktopMutationWatermarkStore: DesktopMutationWatermarkStore())
},
makeRemoteSnapshotManager: @escaping (String) async throws -> (any SnapshotManagerProtocol)? = {
try await InteractionObservationInvalidator.makeRemoteSnapshotManager(socketPath: $0)
},
mutationTracker: InteractionMutationTracker? = nil
) {
self.snapshots = snapshots
self.selectedRemoteSocketPath = selectedRemoteSocketPath
self.remoteSocketPaths = remoteSocketPaths
self.socketExists = socketExists
self.makeLocalSnapshotManager = makeLocalSnapshotManager
self.makeRemoteSnapshotManager = makeRemoteSnapshotManager
self.mutationTracker = mutationTracker
}
}
@discardableResult
static func invalidateAfterClickMutation(
targets: MutationTargets,
logger: Logger,
reason: String,
through cutoff: Date = Date()
) async -> Bool {
await self.invalidateAfterMutation(
targets: targets,
logger: logger,
reason: reason,
through: cutoff
)
}
@discardableResult
static func invalidateAfterMutation(
targets: MutationTargets,
logger: Logger,
reason: String,
through cutoff: Date = Date(),
preserving snapshotId: String? = nil,
preservedAt: Date? = nil
) async -> Bool {
let succeeded = await invalidateLatestSnapshotsAcrossKnownHosts(
using: targets.snapshots,
selectedRemoteSocketPath: targets.selectedRemoteSocketPath,
remoteSocketPaths: targets.remoteSocketPaths,
logger: logger,
reason: reason,
through: cutoff,
preserving: snapshotId,
preservedAt: preservedAt,
logFailures: targets.mutationTracker?.mutationStartedAt == nil,
socketExists: targets.socketExists,
makeLocalSnapshotManager: targets.makeLocalSnapshotManager,
makeRemoteSnapshotManager: targets.makeRemoteSnapshotManager
)
if succeeded {
targets.mutationTracker?.markInvalidated(through: cutoff)
} else {
targets.mutationTracker?.markInvalidationFailed(through: cutoff)
}
return succeeded
}
@discardableResult
static func invalidateLatestSnapshotsAcrossKnownHosts(
using snapshots: any SnapshotManagerProtocol,
selectedRemoteSocketPath: String?,
remoteSocketPaths: [String],
logger: Logger,
reason: String,
through cutoff: Date = Date(),
preserving snapshotId: String? = nil,
preservedAt: Date? = nil,
logFailures: Bool = true,
socketExists: (String) -> Bool = { FileManager.default.fileExists(atPath: $0) },
makeLocalSnapshotManager: () -> any SnapshotManagerProtocol = {
SnapshotManager(desktopMutationWatermarkStore: DesktopMutationWatermarkStore())
},
makeRemoteSnapshotManager: (String) async throws -> (any SnapshotManagerProtocol)? = {
try await InteractionObservationInvalidator.makeRemoteSnapshotManager(socketPath: $0)
}
) async -> Bool {
var requiredManagers: [any SnapshotManagerProtocol] = [snapshots]
let selectedPath = selectedRemoteSocketPath.map { NSString(string: $0).standardizingPath }
if selectedPath != nil {
requiredManagers.append(makeLocalSnapshotManager())
}
let requiredSucceeded = await self.invalidateLatestSnapshots(
using: requiredManagers,
logger: logger,
reason: reason,
through: cutoff,
preserving: snapshotId,
preservedAt: preservedAt,
logFailures: logFailures
)
var alternateManagers: [(path: String, manager: any SnapshotManagerProtocol)] = []
var seenPaths = Set<String>()
for rawPath in remoteSocketPaths {
let path = NSString(string: rawPath).standardizingPath
guard !path.isEmpty,
path != selectedPath,
seenPaths.insert(path).inserted,
socketExists(path)
else { continue }
do {
if let manager = try await makeRemoteSnapshotManager(path) {
alternateManagers.append((path: path, manager: manager))
}
} catch {
if self.isStaleSocketProbeFailure(
error,
socketPath: path,
socketExists: socketExists
) {
logger.debug(
"Skipping stale snapshot invalidation endpoint at \(path) after \(reason)"
)
continue
}
if logFailures {
logger.warn(
"Skipping unavailable alternate snapshot endpoint at \(path) after \(reason): " +
error.localizedDescription
)
} else {
logger.debug(
"Skipping unavailable alternate snapshot endpoint at \(path) after \(reason)"
)
}
}
}
for alternate in alternateManagers {
let succeeded = await self.invalidateLatestSnapshot(
using: alternate.manager,
logger: logger,
reason: reason,
through: cutoff,
preserving: snapshotId,
preservedAt: preservedAt,
logFailures: false
)
if !succeeded {
logger.debug(
"Skipping unavailable alternate snapshot endpoint at \(alternate.path) after \(reason)"
)
}
}
return requiredSucceeded
}
private static func isStaleSocketProbeFailure(
_ error: any Error,
socketPath: String,
socketExists: (String) -> Bool
) -> Bool {
guard socketExists(socketPath) else {
return true
}
guard let posixError = error as? POSIXError else {
return false
}
return switch posixError.code {
case .ECONNREFUSED, .ENOENT, .ENOTSOCK:
true
default:
false
}
}
private static func makeRemoteSnapshotManager(
socketPath: String
) async throws -> (any SnapshotManagerProtocol)? {
let client = PeekabooBridgeClient(socketPath: socketPath, requestTimeoutSec: 1)
let identity = PeekabooBridgeClientIdentity(
bundleIdentifier: Bundle.main.bundleIdentifier,
teamIdentifier: nil,
processIdentifier: getpid(),
hostname: Host.current().name
)
let handshake = try await client.handshake(client: identity, requestedHost: nil)
guard BridgeCapabilityPolicy.supportsImplicitSnapshotInvalidation(for: handshake) else {
return nil
}
return RemoteSnapshotManager(
client: client,
supportsImplicitLatestSnapshotInvalidation: true,
desktopMutationWatermarkStore: DesktopMutationWatermarkStore()
)
}
@discardableResult
static func invalidateLatestSnapshots(
using snapshotManagers: [any SnapshotManagerProtocol],
logger: Logger,
reason: String,
through cutoff: Date = Date(),
preserving snapshotId: String? = nil,
preservedAt: Date? = nil,
logFailures: Bool = true
) async -> Bool {
var succeeded = true
for snapshots in snapshotManagers {
guard await self.invalidateLatestSnapshot(
using: snapshots,
logger: logger,
reason: reason,
through: cutoff,
preserving: snapshotId,
preservedAt: preservedAt,
logFailures: logFailures
) else {
succeeded = false
continue
}
}
return succeeded
}
static func invalidateAfterMutation(
_ observation: InteractionObservationContext,
snapshots: any SnapshotManagerProtocol,
logger: Logger,
reason: String
reason: String,
through cutoff: Date = Date()
) async {
do {
if let invalidatedSnapshotId = try await observation.invalidateAfterMutation(using: snapshots) {
if let invalidatedSnapshotId = try await observation.invalidateAfterMutation(
using: snapshots,
through: cutoff
) {
logger.debug(
"Invalidated implicit latest snapshot '\(invalidatedSnapshotId)' after \(reason)"
)
@ -48,7 +450,8 @@ enum InteractionObservationInvalidator {
_ observation: InteractionObservationContext,
snapshots: any SnapshotManagerProtocol,
logger: Logger,
reason: String
reason: String,
through cutoff: Date = Date()
) async {
switch observation.source {
case .explicit:
@ -58,34 +461,221 @@ enum InteractionObservationInvalidator {
observation,
snapshots: snapshots,
logger: logger,
reason: reason
reason: reason,
through: cutoff
)
case .none:
await self.invalidateLatestSnapshot(
using: snapshots,
logger: logger,
reason: reason
reason: reason,
through: cutoff
)
}
}
@discardableResult
static func invalidateLatestSnapshot(
using snapshots: any SnapshotManagerProtocol,
logger: Logger,
reason: String
) async {
reason: String,
through cutoff: Date = Date(),
preserving snapshotId: String? = nil,
preservedAt: Date? = nil,
logFailures: Bool = true
) async -> Bool {
do {
if let invalidatedSnapshotId = try await InteractionObservationContext.invalidateLatestSnapshot(
using: snapshots
using: snapshots,
through: cutoff,
preserving: snapshotId,
preservedAt: preservedAt
) {
logger.debug(
"Invalidated latest snapshot '\(invalidatedSnapshotId)' after \(reason)"
"Invalidated implicit latest snapshot '\(invalidatedSnapshotId)' after \(reason)"
)
}
return true
} catch {
logger.warn(
"Failed to invalidate latest snapshot after \(reason): \(error.localizedDescription)"
)
if logFailures {
logger.warn(
"Failed to invalidate latest snapshot after \(reason): \(error.localizedDescription)"
)
}
return false
}
}
}
@MainActor
extension CommandRuntime {
func withCaptureFocusMutation(
_ operation: () async throws -> Void
) async rethrows {
self.beginInteractionMutation()
try await operation()
self.beginInteractionMutation(preservingSnapshotsCreatedAfterBoundary: true)
}
@discardableResult
func beginInteractionMutation(
at cutoff: Date = Date(),
preservingSnapshotsCreatedAfterBoundary: Bool = false
) -> Date {
interactionMutationTracker.begin(
at: cutoff,
preservingSnapshotsCreatedAfterBoundary: preservingSnapshotsCreatedAfterBoundary
)
}
func preserveFreshObservation(
snapshotId: String,
startedAt: Date,
preservedAt: Date,
preservationAllowed: Bool = true
) {
interactionMutationTracker.preserveFreshObservation(
snapshotId: snapshotId,
startedAt: startedAt,
preservedAt: preservedAt,
preservationAllowed: preservationAllowed
)
}
var interactionMutationTargets: InteractionObservationInvalidator.MutationTargets {
.init(
snapshots: services.snapshots,
selectedRemoteSocketPath: selectedRemoteSocketPath,
remoteSocketPaths: snapshotInvalidationRemoteSocketPaths,
mutationTracker: interactionMutationTracker
)
}
var toolSnapshotMutationCoordinator: any MCPToolSnapshotMutationCoordinating {
RuntimeMCPToolSnapshotMutationCoordinator(
targets: .init(
snapshots: services.snapshots,
selectedRemoteSocketPath: selectedRemoteSocketPath,
remoteSocketPaths: snapshotInvalidationRemoteSocketPaths
),
logger: logger,
mutationTracker: interactionMutationTracker
)
}
}
@MainActor
private final class RuntimeMCPToolSnapshotMutationCoordinator: MCPToolSnapshotMutationCoordinating {
private let targets: InteractionObservationInvalidator.MutationTargets
private let logger: Logger
private let mutationTracker: InteractionMutationTracker
private let hasRemoteSelection: Bool
private var preparedLocalMutationIDs: Set<UUID> = []
private var completedPreparedMutationIDs: Set<UUID> = []
init(
targets: InteractionObservationInvalidator.MutationTargets,
logger: Logger,
mutationTracker: InteractionMutationTracker
) {
self.targets = targets
self.logger = logger
self.mutationTracker = mutationTracker
self.hasRemoteSelection = targets.selectedRemoteSocketPath != nil
}
func prepareMutation(_ scope: MCPToolSnapshotMutationScope) throws {
guard scope.effect != .freshObservation else { return }
let needsCallerBarrier = !self.hasRemoteSelection || scope.effect != .mutationProducingFreshObservation
if needsCallerBarrier {
guard try self.mutationTracker.beginDurableMutation(at: scope.startedAt) else {
throw PeekabooError.operationError(
message: "A previous local desktop mutation barrier is still pending"
)
}
self.preparedLocalMutationIDs.insert(scope.id)
}
self.mutationTracker.begin(
at: scope.startedAt,
preservingSnapshotsCreatedAfterBoundary: scope.effect == .mutationProducingFreshObservation
)
}
func completeMutationBarrier(
_ scope: MCPToolSnapshotMutationScope
) throws -> MCPToolMutationBarrierCompletion? {
guard self.preparedLocalMutationIDs.contains(scope.id) else { return nil }
let completion = try self.mutationTracker.completeDurableMutation(
through: scope.completedAt ?? Date()
)
self.preparedLocalMutationIDs.remove(scope.id)
self.completedPreparedMutationIDs.insert(scope.id)
return completion.map {
MCPToolMutationBarrierCompletion(
cutoff: $0.cutoff,
allowsObservationPreservation: $0.allowsObservationPreservation
)
}
}
@discardableResult
func completeMutation(_ scope: MCPToolSnapshotMutationScope, succeeded: Bool) async -> Bool {
let completedPreparedMutation = self.completedPreparedMutationIDs.remove(scope.id) != nil
let defersToOuterCommandBarrier = !self.hasRemoteSelection &&
scope.effect == .mutationProducingFreshObservation &&
!completedPreparedMutation &&
self.mutationTracker.hasPendingDurableMutation
if defersToOuterCommandBarrier {
if succeeded,
let preservedSnapshotID = scope.preservedSnapshotID,
let completedAt = scope.completedAt {
self.mutationTracker.preserveFreshObservation(
snapshotId: preservedSnapshotID,
startedAt: scope.confirmedMutationCompletedAt ?? scope.startedAt,
preservedAt: completedAt
)
}
return true
}
let sharedWatermark = self.targets.snapshots.effectiveImplicitLatestInvalidationWatermark
let wantsPreservation = succeeded &&
scope.effect == .mutationProducingFreshObservation &&
scope.preservedSnapshotID != nil
let preservationBoundary = scope.confirmedMutationCompletedAt ?? scope.startedAt
let publicationAllowed = !wantsPreservation ||
((scope.observationPreservationAllowed ?? true) &&
(sharedWatermark.map { $0 <= preservationBoundary } ?? true))
let effectiveSucceeded = succeeded && publicationAllowed
let requestedCutoff = scope.invalidationCutoff(succeeded: effectiveSucceeded)
let cutoff = max(requestedCutoff, sharedWatermark ?? requestedCutoff)
let preservedSnapshotID = effectiveSucceeded ? scope.preservedSnapshotID : nil
if let preservedSnapshotID, let completedAt = scope.completedAt {
self.mutationTracker.preserveFreshObservation(
snapshotId: preservedSnapshotID,
startedAt: cutoff,
preservedAt: completedAt
)
}
let invalidated = await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.targets,
logger: self.logger,
reason: "\(scope.toolName) tool execution",
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedSnapshotID == nil ? nil : scope.completedAt
)
if !invalidated {
let retried = await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.targets,
logger: self.logger,
reason: "\(scope.toolName) tool execution retry",
through: cutoff,
preserving: preservedSnapshotID,
preservedAt: preservedSnapshotID == nil ? nil : scope.completedAt
)
return retried && (!succeeded || effectiveSucceeded)
}
return !succeeded || effectiveSucceeded
}
}

View File

@ -9,11 +9,6 @@ extension AppCommand {
@MainActor
struct LaunchSubcommand {
@MainActor
static var launcher: any ApplicationLaunching = ApplicationLaunchEnvironment.launcher
@MainActor
static var resolver: any ApplicationURLResolving = ApplicationURLResolverEnvironment.resolver
static let commandDescription = CommandDescription(
commandName: "launch",
abstract: "Launch an application",
@ -88,14 +83,12 @@ extension AppCommand {
self.prepare(using: runtime)
do {
try self.validateInputs()
let url = try self.resolveApplicationURL()
let launchedApp = try await self.launchApplication(at: url, name: self.displayName(for: url))
try await self.waitIfNeeded(for: launchedApp)
self.activateIfNeeded(launchedApp)
await self.invalidateFocusSnapshotIfNeeded()
self.resolvedRuntime.beginInteractionMutation()
let launchedApp = try await launchApplication()
await invalidateSnapshotsAfterLaunch()
self.renderLaunchSuccess(app: launchedApp)
} catch {
self.handleError(error)
handleError(error, customCode: applicationLaunchErrorCode(for: error))
throw ExitCode(1)
}
}
@ -112,41 +105,19 @@ extension AppCommand {
}
}
private func resolveApplicationURL() throws -> URL {
try Self.resolver.resolveApplication(appIdentifier: self.requestedAppIdentifier, bundleId: self.bundleId)
}
private func displayName(for url: URL) -> String {
(try? url.resourceValues(forKeys: [.localizedNameKey]).localizedName) ?? self.requestedAppIdentifier
}
private var requestedAppIdentifier: String {
self.app ?? self.bundleId ?? "unknown"
self.bundleId ?? self.app ?? "unknown"
}
private func waitIfNeeded(for app: any RunningApplicationHandle) async throws {
guard self.waitUntilReady else { return }
try await self.waitForApplicationReady(app)
}
private func activateIfNeeded(_ app: any RunningApplicationHandle) {
guard self.shouldFocusAfterLaunch else { return }
if !app.activate(options: []) {
self.logger
.error("Launch succeeded but failed to focus \(app.localizedName ?? self.requestedAppIdentifier)")
}
}
private func invalidateFocusSnapshotIfNeeded() async {
guard self.shouldFocusAfterLaunch else { return }
await InteractionObservationInvalidator.invalidateLatestSnapshot(
using: self.services.snapshots,
private func invalidateSnapshotsAfterLaunch() async {
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "app launch focus"
reason: "app launch"
)
}
private func renderLaunchSuccess(app: any RunningApplicationHandle) {
private func renderLaunchSuccess(app: ServiceApplicationInfo) {
struct LaunchResult: Codable {
let action: String
let app_name: String
@ -157,10 +128,10 @@ extension AppCommand {
let data = LaunchResult(
action: "launch",
app_name: app.localizedName ?? self.requestedAppIdentifier,
app_name: app.name,
bundle_id: app.bundleIdentifier ?? "unknown",
pid: app.processIdentifier,
is_ready: app.isFinishedLaunching
is_ready: app.isFinishedLaunching ?? !self.waitUntilReady
)
AutomationEventLogger.log(
.app,
@ -168,34 +139,22 @@ extension AppCommand {
)
output(data) {
print("✓ Launched \(app.localizedName ?? self.requestedAppIdentifier) (PID: \(app.processIdentifier))")
print("✓ Launched \(app.name) (PID: \(app.processIdentifier))")
}
}
private func launchApplication(at url: URL, name: String) async throws -> any RunningApplicationHandle {
if self.openTargets.isEmpty {
return try await Self.launcher.launchApplication(at: url, activates: self.shouldFocusAfterLaunch)
} else {
let urls = try self.openTargets.map { try Self.resolveOpenTarget($0) }
return try await Self.launcher.launchApplication(
url,
opening: urls,
activates: self.shouldFocusAfterLaunch
)
}
}
private func waitForApplicationReady(
_ app: any RunningApplicationHandle,
timeout: TimeInterval = 10
) async throws {
let startTime = Date()
while !app.isFinishedLaunching {
if Date().timeIntervalSince(startTime) > timeout {
throw PeekabooError.timeout("Application did not become ready within \(Int(timeout)) seconds")
}
try await Task.sleep(nanoseconds: 100_000_000) // 0.1 second
}
private func launchApplication() async throws -> ServiceApplicationInfo {
let urls = try openTargets.map { try Self.resolveOpenTarget($0) }
let applicationIdentifier = self.bundleId == nil
? self.app.map { ApplicationIdentifierResolver.resolve($0) }
: nil
return try await self.services.applications.launchApplication(request: ApplicationLaunchRequest(
applicationIdentifier: applicationIdentifier,
applicationBundleIdentifier: self.bundleId,
openURLs: urls,
activates: self.shouldFocusAfterLaunch,
waitUntilReady: self.waitUntilReady
))
}
static func resolveOpenTarget(
@ -229,10 +188,10 @@ extension AppCommand.LaunchSubcommand: AsyncRuntimeCommand, ErrorHandlingCommand
@MainActor
extension AppCommand.LaunchSubcommand: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {
self.app = try values.decodeOptionalPositional(0, label: "app")
self.bundleId = values.singleOption("bundleId")
self.waitUntilReady = values.flag("waitUntilReady")
self.noFocus = values.flag("noFocus")
self.openTargets = values.optionValues("open")
app = try values.decodeOptionalPositional(0, label: "app")
bundleId = values.singleOption("bundleId")
waitUntilReady = values.flag("waitUntilReady")
noFocus = values.flag("noFocus")
openTargets = values.optionValues("open")
}
}

View File

@ -45,6 +45,23 @@ extension AppCommand {
self.resolvedRuntime.configuration.jsonOutput
}
static func filteredApplications(
_ applications: [ServiceApplicationInfo],
includeHidden: Bool,
includeBackground: Bool
) -> [ServiceApplicationInfo] {
applications.filter { app in
if !includeHidden, app.isHidden {
return false
}
if !includeBackground,
app.activationPolicy == .accessory || app.activationPolicy == .prohibited {
return false
}
return true
}
}
/// Enumerate running applications, apply filtering flags, and emit the chosen output representation.
@MainActor
mutating func run(using runtime: CommandRuntime) async throws {
@ -53,12 +70,11 @@ extension AppCommand {
do {
let appsOutput = try await self.services.applications.listApplications()
// Filter based on flags
let filtered = appsOutput.data.applications.filter { app in
if !self.includeHidden && app.isHidden { return false }
if !self.includeBackground && app.name.isEmpty { return false }
return true
}
let filtered = Self.filteredApplications(
appsOutput.data.applications,
includeHidden: self.includeHidden,
includeBackground: self.includeBackground
)
struct AppInfo: Codable {
let name: String

View File

@ -114,6 +114,12 @@ extension AppCommand {
var results: [AppQuitInfo] = []
for target in quitApps {
if target.pid == self.resolvedRuntime.selectedRemoteHostProcessIdentifier {
throw PeekabooError.invalidInput(
"Cannot quit the daemon host executing this command; use a different runtime host"
)
}
self.resolvedRuntime.beginInteractionMutation()
let success = await (try? self.services.applications.quitApplication(
identifier: target.identifier,
force: self.force

View File

@ -9,11 +9,6 @@ extension AppCommand {
@MainActor
struct RelaunchSubcommand {
@MainActor
static var launcher: any ApplicationLaunching = ApplicationLaunchEnvironment.launcher
@MainActor
static var resolver: any ApplicationURLResolving = ApplicationURLResolverEnvironment.resolver
static let commandDescription = CommandDescription(
commandName: "relaunch",
abstract: "Quit and relaunch an application"
@ -68,41 +63,45 @@ extension AppCommand {
self.runtime = runtime
do {
guard self.resolvedRuntime.applicationRelaunchAllowed else {
throw PeekabooError.serviceUnavailable(
"Relaunch requires a surviving daemon host; the selected bridge is unavailable or GUI-hosted"
)
}
// Find the application first
let appIdentifier = try self.resolveApplicationIdentifier()
let appInfo = try await resolveApplication(appIdentifier, services: self.services)
let appIdentifier = try resolveApplicationIdentifier()
let appInfo = try await resolveApplication(appIdentifier, services: services)
let originalPID = appInfo.processIdentifier
guard originalPID != self.resolvedRuntime.selectedRemoteHostProcessIdentifier else {
throw PeekabooError.serviceUnavailable(
"Cannot relaunch the selected daemon through itself; use another bridge host"
)
}
let processIdentifier = "PID:\(originalPID)"
// Step 1: Quit the app
let quitSuccess = try await self.services.applications.quitApplication(
identifier: processIdentifier,
force: self.force
guard self.wait.isFinite, self.wait >= 0 else {
throw PeekabooError.invalidInput("Relaunch wait must be a finite, non-negative number of seconds")
}
let launchIdentifier = appInfo.bundleIdentifier == nil ? (appInfo.bundlePath ?? appInfo.name) : nil
self.resolvedRuntime.beginInteractionMutation()
let launchedApp = try await services.applications.relaunchApplication(
request: ApplicationRelaunchRequest(
targetIdentifier: processIdentifier,
launchRequest: ApplicationLaunchRequest(
applicationIdentifier: launchIdentifier,
applicationBundleIdentifier: appInfo.bundleIdentifier,
activates: true,
waitUntilReady: self.waitUntilReady
),
force: self.force,
waitSeconds: self.wait
)
)
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "app relaunch focus"
)
if !quitSuccess {
throw PeekabooError
.commandFailed(
"Failed to quit \(appInfo.name) (PID: \(originalPID)). The app may have unsaved changes."
)
}
// Wait for the app to actually terminate
try await self.waitUntilTerminated(identifier: processIdentifier, appName: appInfo.name)
// Step 2: Wait the specified duration
if self.wait > 0 {
try await Task.sleep(nanoseconds: UInt64(self.wait * 1_000_000_000))
}
// Step 3: Launch the app
let appURL = try self.resolveLaunchURL(for: appInfo)
let launchedApp = try await Self.launcher.launchApplication(at: appURL, activates: true)
// Wait until ready if requested
if self.waitUntilReady {
try await self.waitUntilReady(launchedApp)
}
struct RelaunchResult: Codable {
let action: String
@ -123,53 +122,22 @@ extension AppCommand {
bundle_id: appInfo.bundleIdentifier,
quit_forced: self.force,
wait_time: self.wait,
launch_success: launchedApp.isFinishedLaunching || !self.waitUntilReady
launch_success: !self.waitUntilReady || launchedApp.isFinishedLaunching == true
)
output(data) {
print("✓ Relaunched \(appInfo.name)")
print(" Old PID: \(originalPID) → New PID: \(launchedApp.processIdentifier)")
if self.waitUntilReady {
print(" Status: \(launchedApp.isFinishedLaunching ? "Ready" : "Launching...")")
print(" Status: \(launchedApp.isFinishedLaunching == true ? "Ready" : "Launching...")")
}
}
} catch {
handleError(error)
handleError(error, customCode: applicationLaunchErrorCode(for: error))
throw ExitCode(1)
}
}
private func waitUntilTerminated(identifier: String, appName: String) async throws {
var terminateWaitTime = 0.0
while await self.services.applications.isApplicationRunning(identifier: identifier),
terminateWaitTime < 5.0 {
try await Task.sleep(nanoseconds: 100_000_000) // 0.1 seconds
terminateWaitTime += 0.1
}
if await self.services.applications.isApplicationRunning(identifier: identifier) {
throw PeekabooError.timeout("App \(appName) did not terminate within 5 seconds")
}
}
private func resolveLaunchURL(for appInfo: ServiceApplicationInfo) throws -> URL {
if let bundleID = appInfo.bundleIdentifier {
return try Self.resolver.resolveBundleIdentifier(bundleID)
}
if let bundlePath = appInfo.bundlePath {
return URL(fileURLWithPath: bundlePath)
}
throw PeekabooError.commandFailed("No bundle ID or path available to relaunch \(appInfo.name)")
}
private func waitUntilReady(_ app: any RunningApplicationHandle) async throws {
var readyWaitTime = 0.0
while !app.isFinishedLaunching && readyWaitTime < 10.0 {
try await Task.sleep(nanoseconds: 100_000_000) // 0.1 seconds
readyWaitTime += 0.1
}
}
}
}
@ -180,12 +148,12 @@ extension AppCommand.RelaunchSubcommand: AsyncRuntimeCommand, ErrorHandlingComma
@MainActor
extension AppCommand.RelaunchSubcommand: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {
self.app = try values.decodePositional(0, label: "app")
self.pid = try values.decodeOption("pid", as: Int32.self)
app = try values.decodePositional(0, label: "app")
pid = try values.decodeOption("pid", as: Int32.self)
if let wait: TimeInterval = try values.decodeOption("wait", as: TimeInterval.self) {
self.wait = wait
}
self.force = values.flag("force")
self.waitUntilReady = values.flag("waitUntilReady")
force = values.flag("force")
waitUntilReady = values.flag("waitUntilReady")
}
}

View File

@ -98,6 +98,7 @@ struct AppCommand: ParsableCommand {
let appIdentifier = try self.resolveApplicationIdentifier()
let appInfo = try await resolveApplication(appIdentifier, services: self.services)
self.resolvedRuntime.beginInteractionMutation()
try await self.services.applications.hideApplication(identifier: appIdentifier)
let data = [
@ -178,6 +179,7 @@ struct AppCommand: ParsableCommand {
let appIdentifier = try self.resolveApplicationIdentifier()
let appInfo = try await resolveApplication(appIdentifier, services: self.services)
self.resolvedRuntime.beginInteractionMutation()
try await self.services.applications.unhideApplication(identifier: appIdentifier)
// Activate if requested
@ -271,6 +273,7 @@ struct AppCommand: ParsableCommand {
if self.verify {
throw ValidationError("Verify is only supported with --to (not --cycle)")
}
self.resolvedRuntime.beginInteractionMutation()
try await self.services.automation.hotkey(keys: "cmd,tab", holdDuration: 0)
struct CycleResult: Codable {
@ -280,8 +283,8 @@ struct AppCommand: ParsableCommand {
let data = CycleResult(action: "cycle", success: true)
await InteractionObservationInvalidator.invalidateLatestSnapshot(
using: self.services.snapshots,
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "app switch cycle"
)
@ -291,7 +294,13 @@ struct AppCommand: ParsableCommand {
AutomationEventLogger.log(.app, "switch action=cycle success=true")
} else if let targetApp = to {
let appInfo = try await resolveApplication(targetApp, services: self.services)
self.resolvedRuntime.beginInteractionMutation()
try await self.services.applications.activateApplication(identifier: appInfo.name)
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "app switch"
)
if self.verify {
try await self.verifyFrontmostApp(expected: appInfo)
}
@ -310,11 +319,6 @@ struct AppCommand: ParsableCommand {
success: true
)
await InteractionObservationInvalidator.invalidateLatestSnapshot(
using: self.services.snapshots,
logger: self.logger,
reason: "app switch"
)
output(data) {
print("✓ Switched to \(appInfo.name)")
}

View File

@ -0,0 +1,17 @@
import Foundation
enum ApplicationIdentifierResolver {
static func resolve(
_ value: String,
cwd: String = FileManager.default.currentDirectoryPath
) -> String {
let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines)
guard trimmed.contains("/") else { return trimmed }
let expanded = NSString(string: trimmed).expandingTildeInPath
let absolutePath = expanded.hasPrefix("/")
? expanded
: NSString(string: cwd).appendingPathComponent(expanded)
return URL(fileURLWithPath: absolutePath).standardizedFileURL.path
}
}

View File

@ -1,125 +0,0 @@
import AppKit
import Foundation
import PeekabooCore
// MARK: - Running Application Handle
@MainActor
protocol RunningApplicationHandle {
var localizedName: String? { get }
var bundleIdentifier: String? { get }
var processIdentifier: Int32 { get }
var isFinishedLaunching: Bool { get }
var isActive: Bool { get }
@discardableResult
func activate(options: NSApplication.ActivationOptions) -> Bool
}
@MainActor
extension NSRunningApplication: RunningApplicationHandle {}
// MARK: - Launcher abstraction
@MainActor
protocol ApplicationLaunching {
func launchApplication(at url: URL, activates: Bool) async throws -> any RunningApplicationHandle
func launchApplication(_ url: URL, opening documents: [URL], activates: Bool) async throws
-> any RunningApplicationHandle
func openTarget(_ targetURL: URL, handlerURL: URL?, activates: Bool) async throws -> any RunningApplicationHandle
}
@MainActor
enum ApplicationLaunchEnvironment {
static var launcher: any ApplicationLaunching = NSWorkspaceApplicationLauncher()
}
@MainActor
final class NSWorkspaceApplicationLauncher: ApplicationLaunching {
func launchApplication(at url: URL, activates: Bool) async throws -> any RunningApplicationHandle {
let configuration = NSWorkspace.OpenConfiguration()
configuration.activates = activates
return try await NSWorkspace.shared.openApplication(at: url, configuration: configuration)
}
func launchApplication(
_ url: URL,
opening documents: [URL],
activates: Bool
) async throws -> any RunningApplicationHandle {
let configuration = NSWorkspace.OpenConfiguration()
configuration.activates = activates
return try await NSWorkspace.shared.open(documents, withApplicationAt: url, configuration: configuration)
}
func openTarget(_ targetURL: URL, handlerURL: URL?, activates: Bool) async throws -> any RunningApplicationHandle {
if let handlerURL {
return try await self.launchApplication(handlerURL, opening: [targetURL], activates: activates)
} else {
let configuration = NSWorkspace.OpenConfiguration()
configuration.activates = activates
return try await NSWorkspace.shared.open(targetURL, configuration: configuration)
}
}
}
// MARK: - Application URL resolver
@MainActor
protocol ApplicationURLResolving {
func resolveApplication(appIdentifier: String, bundleId: String?) throws -> URL
func resolveBundleIdentifier(_ bundleId: String) throws -> URL
}
@MainActor
enum ApplicationURLResolverEnvironment {
static var resolver: any ApplicationURLResolving = DefaultApplicationURLResolver()
}
@MainActor
final class DefaultApplicationURLResolver: ApplicationURLResolving {
func resolveApplication(appIdentifier: String, bundleId: String?) throws -> URL {
if let bundleId {
return try self.resolveBundleIdentifier(bundleId)
}
if let bundleURL = NSWorkspace.shared.urlForApplication(withBundleIdentifier: appIdentifier) {
return bundleURL
}
if let namedURL = self.findApplicationByName(appIdentifier) {
return namedURL
}
if appIdentifier.contains("/") {
return URL(fileURLWithPath: appIdentifier)
}
throw NotFoundError.application(appIdentifier)
}
func resolveBundleIdentifier(_ bundleId: String) throws -> URL {
guard let url = NSWorkspace.shared.urlForApplication(withBundleIdentifier: bundleId) else {
throw NotFoundError.application("Bundle ID: \(bundleId)")
}
return url
}
private func findApplicationByName(_ name: String) -> URL? {
let searchPaths = [
"/Applications",
"/System/Applications",
"/System/Library/CoreServices",
"~/Applications",
"/Applications/Utilities"
].map { NSString(string: $0).expandingTildeInPath }
for path in searchPaths {
let appPath = "\(path)/\(name).app"
if FileManager.default.fileExists(atPath: appPath) {
return URL(fileURLWithPath: appPath)
}
}
return nil
}
}

View File

@ -101,6 +101,9 @@ struct ClipboardCommand: OutputFormattable, RuntimeOptionsConfigurable {
self.logger.setJsonOutputMode(self.jsonOutput)
let action = try self.resolvedAction()
if Self.actionMayMutate(action) {
self.resolvedRuntime.beginInteractionMutation()
}
switch action.lowercased() {
case "get":
try self.handleGet()
@ -121,6 +124,13 @@ struct ClipboardCommand: OutputFormattable, RuntimeOptionsConfigurable {
// MARK: - Actions
static func actionMayMutate(_ action: String?) -> Bool {
guard let action = action?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() else {
return false
}
return ["set", "load", "clear", "restore"].contains(action)
}
private func resolvedAction() throws -> String {
let positionalAction = self.action?.trimmingCharacters(in: .whitespacesAndNewlines)
let optionAction = self.actionOption?.trimmingCharacters(in: .whitespacesAndNewlines)

View File

@ -15,7 +15,7 @@ extension DaemonCommand {
}
}
@Option(name: .long, help: "Daemon mode (manual, mcp)")
@Option(name: .long, help: "Daemon mode (manual, auto)")
var mode: String = "manual"
@Option(name: .long, help: "Override bridge socket path")
@ -33,23 +33,41 @@ extension DaemonCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let pollInterval = TimeInterval(Double(self.pollIntervalMs ?? 1000) / 1000.0)
let socketPath = self.bridgeSocket ?? PeekabooBridgeConstants.peekabooSocketPath
let normalizedMode = self.mode.lowercased()
let config: PeekabooDaemon.Configuration = if normalizedMode == "auto" {
.auto(
bridgeSocketPath: socketPath,
windowPollInterval: pollInterval,
idleTimeout: self.idleTimeoutSeconds ?? CommandRuntime.defaultDaemonIdleTimeoutSeconds
)
} else if normalizedMode == "mcp" {
.mcp(bridgeSocketPath: socketPath, windowPollInterval: pollInterval)
} else {
.manual(bridgeSocketPath: socketPath, windowPollInterval: pollInterval)
}
let config = try Self.configuration(
mode: self.mode,
bridgeSocket: self.bridgeSocket,
pollInterval: pollInterval,
idleTimeoutSeconds: self.idleTimeoutSeconds
)
let daemon = PeekabooDaemon(configuration: config)
await daemon.runUntilStop()
try await daemon.runUntilStopChecked()
}
static func configuration(
mode: String,
bridgeSocket: String?,
pollInterval: TimeInterval,
idleTimeoutSeconds: Double?
) throws -> PeekabooDaemon.Configuration {
let normalizedMode = mode.lowercased()
if normalizedMode == "mcp" {
throw ValidationError(
"Standalone MCP daemon mode is unavailable; use `peekaboo mcp` so the MCP transport owns its lifecycle."
)
}
return if normalizedMode == "auto" {
.auto(
bridgeSocketPath: bridgeSocket ?? PeekabooBridgeConstants.daemonSocketPath,
windowPollInterval: pollInterval,
idleTimeout: idleTimeoutSeconds ?? CommandRuntime.defaultDaemonIdleTimeoutSeconds
)
} else {
.manual(
bridgeSocketPath: bridgeSocket ?? PeekabooBridgeConstants.daemonSocketPath,
windowPollInterval: pollInterval
)
}
}
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {

View File

@ -1,4 +1,5 @@
import Commander
import Darwin
import Foundation
import PeekabooBridge
import PeekabooFoundation
@ -48,60 +49,198 @@ extension DaemonCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let socketPath = self.bridgeSocket ?? PeekabooBridgeConstants.peekabooSocketPath
let defaultSocketPath = PeekabooBridgeConstants.daemonSocketPath
let buildScopedSocketPath = DaemonLaunchPolicy.buildScopedDaemonSocketPath(
daemonSocketPath: defaultSocketPath,
runtimeBuildIdentity: DaemonLaunchPolicy.runtimeBuildIdentity()
)
let lockHandle = DaemonPaths.openDaemonStartupLock()
if let fileDescriptor = lockHandle?.fileDescriptor {
flock(fileDescriptor, LOCK_EX)
}
defer {
if let fileDescriptor = lockHandle?.fileDescriptor {
flock(fileDescriptor, LOCK_UN)
}
try? lockHandle?.close()
}
let targets = await DaemonControlResolver.targets(explicitSocket: self.bridgeSocket)
let action = DaemonControlPlanner.startAction(
targets: targets,
explicitSocket: self.bridgeSocket,
defaultSocketPath: defaultSocketPath,
buildScopedSocketPath: buildScopedSocketPath
)
guard let destination = try await self.resolveDestination(action: action, targets: targets) else { return }
let socketPath = destination.socketPath
let promotionTarget = destination.promotionTarget
let client = DaemonControlClient(socketPath: socketPath)
if let status = await client.fetchStatus() {
self.output(status) {
DaemonStatusPrinter.render(status: status)
let migratesLegacyTarget = DaemonControlPlanner.shouldMigrateLegacyTarget(
explicitSocket: self.bridgeSocket,
destinationSocketPath: socketPath,
defaultSocketPath: defaultSocketPath,
targets: targets
)
let legacyTarget = migratesLegacyTarget
? targets.first {
$0.isLegacyDefault && DaemonControlClient.isReusableDaemonStatus($0.status)
}
: nil
if let legacyTarget {
guard DaemonControlClient.supportsSafeMigration(legacyTarget.status) else {
throw PeekabooError.operationError(
message: "Legacy daemon predates safe migration; run `peekaboo daemon stop`, then retry start"
)
}
guard DaemonControlClient.isIdleForMigration(legacyTarget.status) else {
throw PeekabooError.operationError(
message: "Legacy daemon has active requests; retry start after they finish"
)
}
return
}
let executable = Self.resolveExecutablePath()
let process = Process()
process.executableURL = URL(fileURLWithPath: executable)
var args = ["daemon", "run", "--mode", "manual"]
if let bridgeSocket {
args.append(contentsOf: ["--bridge-socket", bridgeSocket])
}
if let pollIntervalMs {
args.append(contentsOf: ["--poll-interval-ms", "\(pollIntervalMs)"])
}
process.arguments = args
let logHandle = DaemonPaths.openDaemonLogForAppend() ?? FileHandle.nullDevice
process.standardOutput = logHandle
process.standardError = logHandle
process.standardInput = FileHandle.nullDevice
try process.run()
let deadline = Date().addingTimeInterval(TimeInterval(self.waitSeconds))
while Date() < deadline {
if let status = await client.fetchStatus() {
switch await DaemonLaunchPolicy.waitForDaemonSocketAvailability(
socketPath: socketPath,
client: client,
timeout: TimeInterval(max(self.waitSeconds, DaemonControlClient.defaultShutdownWaitSeconds))
) {
case .available:
break
case .reusableDaemon:
if let status = await client.fetchReusableDaemonStatus() {
guard status.mode == .manual else {
throw PeekabooError.operationError(
message: "Daemon at \(socketPath) remained in auto mode; retry start when it is idle"
)
}
self.output(status) {
DaemonStatusPrinter.render(status: status)
}
return
}
try await Task.sleep(nanoseconds: 200_000_000)
case .timedOut:
throw PeekabooError.operationError(message: "Daemon socket is still shutting down")
}
throw PeekabooError.operationError(message: "Daemon did not start within \(self.waitSeconds)s")
}
private static func resolveExecutablePath() -> String {
if let path = CommandLine.arguments.first {
return path
let arguments = DaemonLaunchPolicy.daemonArguments(
socketPath: socketPath,
mode: .manual,
pollIntervalMs: self.pollIntervalMs ?? promotionTarget?.status.windowTracker?.cgPollIntervalMs
?? legacyTarget?.status.windowTracker?.cgPollIntervalMs,
idleTimeoutSeconds: CommandRuntime.defaultDaemonIdleTimeoutSeconds
)
guard let replacement = await DaemonLaunchPolicy.launchDaemon(
socketPath: socketPath,
arguments: arguments,
timeout: TimeInterval(self.waitSeconds)
)
else {
throw PeekabooError.operationError(message: "Daemon did not start within \(self.waitSeconds)s")
}
if let legacyTarget {
do {
let stopped = try await legacyTarget.client.stopAndWait(
waitSeconds: max(self.waitSeconds, DaemonControlClient.defaultShutdownWaitSeconds),
expectedPID: legacyTarget.status.pid,
requireIdentityMatch: true
)
if !stopped,
await legacyTarget.client.fetchReusableDaemonStatus() != nil {
throw PeekabooError.operationError(message: "Legacy daemon refused migration stop request")
}
} catch {
if await legacyTarget.client.fetchReusableDaemonStatus() != nil {
let cleanedUp = await DaemonLaunchPolicy.stopReplacement(
client: client,
replacement: replacement
)
if !cleanedUp {
throw PeekabooError.operationError(
message: "Legacy migration failed and replacement cleanup timed out"
)
}
throw error
}
}
}
let status = replacement.status
self.output(status) {
DaemonStatusPrinter.render(status: status)
}
return "/usr/local/bin/peekaboo"
}
}
}
extension DaemonCommand.Start: AsyncRuntimeCommand {}
private struct DaemonStartDestination {
let socketPath: String
let promotionTarget: DaemonControlTarget?
}
@MainActor
extension DaemonCommand.Start {
fileprivate func resolveDestination(
action: DaemonStartAction,
targets: [DaemonControlTarget]
) async throws -> DaemonStartDestination? {
switch action {
case let .useExisting(socketPath):
guard let target = targets.first(where: { $0.client.socketPath == socketPath }) else {
throw PeekabooError.operationError(message: "Selected daemon disappeared; retry start")
}
self.output(target.status) {
DaemonStatusPrinter.render(status: target.status)
}
return nil
case let .launchManual(socketPath):
return DaemonStartDestination(socketPath: socketPath, promotionTarget: nil)
case let .promoteAutoToManual(socketPath, pid):
guard let target = targets.first(where: { $0.client.socketPath == socketPath }) else {
throw PeekabooError.operationError(message: "Selected daemon disappeared; retry start")
}
do {
guard try await target.client.stopAndWait(
waitSeconds: max(self.waitSeconds, DaemonControlClient.defaultShutdownWaitSeconds),
expectedPID: pid,
requireIdentityMatch: true
)
else {
throw PeekabooError.operationError(
message: "Daemon at \(socketPath) refused a safe stop; retry when it is idle"
)
}
} catch let error as PeekabooError {
throw error
} catch {
throw PeekabooError.operationError(
message: "Could not safely promote daemon at \(socketPath): \(error.localizedDescription)"
)
}
return DaemonStartDestination(socketPath: socketPath, promotionTarget: target)
case let .rejectBusy(socketPath):
throw PeekabooError.operationError(
message: "Daemon at \(socketPath) has active requests; retry start after they finish"
)
case let .rejectUnsafe(socketPath):
throw PeekabooError.operationError(
message: "Daemon at \(socketPath) cannot be safely promoted; " +
"stop it explicitly with `peekaboo daemon stop --bridge-socket \(socketPath)`, then retry"
)
case let .rejectIncompatible(socketPath):
throw PeekabooError.operationError(
message: "Daemon at \(socketPath) is incompatible with this build; " +
"stop it with `peekaboo daemon stop --bridge-socket \(socketPath)`, then retry"
)
}
}
}
@MainActor
extension DaemonCommand.Start: CommanderBindableCommand {
mutating func applyCommanderValues(_ values: CommanderBindableValues) throws {

View File

@ -40,12 +40,24 @@ extension DaemonCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let socketPath = self.bridgeSocket ?? PeekabooBridgeConstants.peekabooSocketPath
let client = DaemonControlClient(socketPath: socketPath)
let targets = await DaemonControlResolver.targets(explicitSocket: self.bridgeSocket)
if let status = await client.fetchStatus() {
self.output(status) {
DaemonStatusPrinter.render(status: status)
if let target = DaemonControlPlanner.preferredStatusTarget(
targets,
explicitSocket: self.bridgeSocket
) {
let additionalSocketPaths = DaemonControlPlanner.additionalSocketPaths(
in: targets,
excluding: target
)
if !additionalSocketPaths.isEmpty {
self.logger.warn(
"Additional Peekaboo daemon detected at \(additionalSocketPaths.joined(separator: ", ")); " +
"reporting \(target.client.socketPath)"
)
}
self.output(target.status) {
DaemonStatusPrinter.render(status: target.status)
}
} else {
let stopped = PeekabooDaemonStatus(running: false)

View File

@ -18,8 +18,8 @@ extension DaemonCommand {
@Option(name: .long, help: "Override bridge socket path")
var bridgeSocket: String?
@Option(name: .long, help: "Seconds to wait for daemon shutdown (default 3)")
var waitSeconds: Int = 3
@Option(name: .long, help: "Seconds to wait for daemon shutdown (default 12)")
var waitSeconds: Int = DaemonControlClient.defaultShutdownWaitSeconds
@RuntimeStorage private var runtime: CommandRuntime?
var runtimeOptions = CommandRuntimeOptions()
@ -45,10 +45,9 @@ extension DaemonCommand {
mutating func run(using runtime: CommandRuntime) async throws {
self.runtime = runtime
let socketPath = self.bridgeSocket ?? PeekabooBridgeConstants.peekabooSocketPath
let client = DaemonControlClient(socketPath: socketPath)
let targets = await DaemonControlResolver.targets(explicitSocket: self.bridgeSocket)
guard let status = await client.fetchStatus() else {
guard !targets.isEmpty else {
let stopped = PeekabooDaemonStatus(running: false)
self.output(stopped) {
DaemonStatusPrinter.render(status: stopped)
@ -56,28 +55,25 @@ extension DaemonCommand {
return
}
if status.mode == nil {
if targets.contains(where: { $0.status.mode == nil }) {
throw PeekabooError.operationError(message: "Connected host does not support daemon stop")
}
let stopped = try await client.stopDaemon()
guard stopped else {
throw PeekabooError.operationError(message: "Daemon refused stop request")
}
let deadline = Date().addingTimeInterval(TimeInterval(self.waitSeconds))
while Date() < deadline {
if await client.fetchStatus() == nil {
let stopped = PeekabooDaemonStatus(running: false)
self.output(stopped) {
DaemonStatusPrinter.render(status: stopped)
}
return
for target in targets {
guard try await target.client.stopAndWait(
waitSeconds: self.waitSeconds,
expectedPID: target.status.pid,
requireIdentityMatch: DaemonControlClient.supportsSafeMigration(target.status)
)
else {
throw PeekabooError.operationError(message: "Daemon refused stop request")
}
try await Task.sleep(nanoseconds: 200_000_000)
}
throw PeekabooError.operationError(message: "Daemon did not stop within \(self.waitSeconds)s")
let stopped = PeekabooDaemonStatus(running: false)
self.output(stopped) {
DaemonStatusPrinter.render(status: stopped)
}
}
}
}

View File

@ -1,4 +1,5 @@
import Commander
import Darwin
import Foundation
import PeekabooBridge
@ -23,10 +24,22 @@ struct DaemonCommand: ParsableCommand {
}
struct DaemonControlClient {
static let defaultShutdownWaitSeconds =
Int(ceil(PeekabooBridgeConstants.defaultRequestTimeoutSeconds)) + 2
let socketPath: String
let requestTimeoutSec: TimeInterval
init(
socketPath: String,
requestTimeoutSec: TimeInterval = PeekabooBridgeConstants.defaultRequestTimeoutSeconds
) {
self.socketPath = socketPath
self.requestTimeoutSec = requestTimeoutSec
}
func fetchStatus() async -> PeekabooDaemonStatus? {
let client = PeekabooBridgeClient(socketPath: self.socketPath)
let client = PeekabooBridgeClient(socketPath: socketPath, requestTimeoutSec: self.requestTimeoutSec)
do {
return try await client.daemonStatus()
} catch let envelope as PeekabooBridgeErrorEnvelope {
@ -39,11 +52,96 @@ struct DaemonControlClient {
}
}
func stopDaemon() async throws -> Bool {
let client = PeekabooBridgeClient(socketPath: self.socketPath)
func stopDaemon(expectedPID: pid_t? = nil) async throws -> Bool {
let client = PeekabooBridgeClient(socketPath: socketPath, requestTimeoutSec: self.requestTimeoutSec)
if let expectedPID {
return try await client.daemonStop(expectedPID: expectedPID)
}
return try await client.daemonStop()
}
func fetchControllableDaemonStatus() async -> PeekabooDaemonStatus? {
guard let status = await fetchStatus(),
Self.isControllableDaemonStatus(status)
else {
return nil
}
return status
}
func fetchReusableDaemonStatus() async -> PeekabooDaemonStatus? {
guard let status = await fetchStatus(),
Self.isReusableDaemonStatus(status)
else {
return nil
}
return status
}
static func isControllableDaemonStatus(_ status: PeekabooDaemonStatus) -> Bool {
status.mode != nil
}
static func isReusableDaemonStatus(_ status: PeekabooDaemonStatus) -> Bool {
status.mode == .auto || status.mode == .manual
}
static func migrationMode(for status: PeekabooDaemonStatus) -> PeekabooDaemonMode? {
self.isReusableDaemonStatus(status) ? status.mode : nil
}
static func isIdleForMigration(_ status: PeekabooDaemonStatus) -> Bool {
status.activity?.activeRequests ?? 0 == 0
}
static func supportsSafeMigration(_ status: PeekabooDaemonStatus) -> Bool {
status.supportsConditionalStop == true
}
func stopAndWait(
waitSeconds: Int,
expectedPID: pid_t?,
requireIdentityMatch: Bool = false
) async throws -> Bool {
var requestError: (any Error)?
var accepted = false
do {
accepted = try await self.stopDaemon(
expectedPID: requireIdentityMatch ? expectedPID : nil
)
} catch {
requestError = error
}
if !accepted, requestError == nil {
return false
}
let deadline = Date().addingTimeInterval(TimeInterval(waitSeconds))
while Date() < deadline {
if await self.fetchControllableDaemonStatus() == nil {
if let expectedPID {
if !Self.isProcessAlive(expectedPID) {
return true
}
} else if requestError == nil {
return true
}
}
try await Task.sleep(nanoseconds: 200_000_000)
}
if let requestError {
throw requestError
}
return false
}
private static func isProcessAlive(_ pid: pid_t) -> Bool {
if kill(pid, 0) == 0 { return true }
return errno != ESRCH
}
private func fallbackHandshake(client: PeekabooBridgeClient) async -> PeekabooDaemonStatus? {
let identity = PeekabooBridgeClientIdentity(
bundleIdentifier: Bundle.main.bundleIdentifier,
@ -54,9 +152,10 @@ struct DaemonControlClient {
do {
let handshake = try await client.handshake(client: identity)
let bridge = PeekabooDaemonBridgeStatus(
socketPath: self.socketPath,
socketPath: socketPath,
hostKind: handshake.hostKind,
allowedOperations: handshake.supportedOperations
allowedOperations: handshake.supportedOperations,
availableOperationNames: handshake.supportedOperations.map(\.rawValue).sorted()
)
return PeekabooDaemonStatus(
running: true,
@ -74,6 +173,308 @@ struct DaemonControlClient {
}
}
struct DaemonControlTarget {
let client: DaemonControlClient
let status: PeekabooDaemonStatus
let role: DaemonControlTargetRole
var isLegacyDefault: Bool {
self.role == .legacyDefault
}
}
enum DaemonControlTargetRole: Equatable {
case explicit
case defaultDaemon
case buildScopedDaemon
case legacyDefault
}
struct DaemonSocketFileCandidate: Equatable {
let path: String
let isSocket: Bool
let ownerUID: uid_t
}
enum DaemonStartAction: Equatable {
case useExisting(socketPath: String)
case launchManual(socketPath: String)
case promoteAutoToManual(socketPath: String, pid: pid_t)
case rejectBusy(socketPath: String)
case rejectUnsafe(socketPath: String)
case rejectIncompatible(socketPath: String)
}
enum DaemonControlPlanner {
private static let currentOperationNames: Set<String> = [
PeekabooBridgeOperation.launchApplicationWithOptions.rawValue,
PeekabooBridgeOperation.relaunchApplicationWithOptions.rawValue,
PeekabooBridgeOperation.invalidateImplicitLatestSnapshot.rawValue,
]
static func supportsCurrentDaemon(_ status: PeekabooDaemonStatus) -> Bool {
guard let bridge = status.bridge else { return false }
let availableNames = Set(
bridge.availableOperationNames ?? bridge.allowedOperations.map(\.rawValue)
)
return self.currentOperationNames.isSubset(of: availableNames)
}
static func preferredStatusTarget(
_ targets: [DaemonControlTarget],
explicitSocket: String?
) -> DaemonControlTarget? {
if explicitSocket != nil {
return targets.first
}
let defaultTarget = targets.first { $0.role == .defaultDaemon }
if let defaultTarget, self.isCurrentReusableTarget(defaultTarget) {
return defaultTarget
}
let scopedTargets = targets.filter { $0.role == .buildScopedDaemon }
if let scopedTarget = scopedTargets.first(where: self.isCurrentReusableTarget) {
return scopedTarget
}
return defaultTarget ?? scopedTargets.first ?? targets.first
}
static func additionalSocketPaths(
in targets: [DaemonControlTarget],
excluding selected: DaemonControlTarget
) -> [String] {
targets
.filter { $0.client.socketPath != selected.client.socketPath }
.map(\.client.socketPath)
}
static func startAction(
targets: [DaemonControlTarget],
explicitSocket: String?,
defaultSocketPath: String,
buildScopedSocketPath: String?
) -> DaemonStartAction {
if let explicitSocket {
guard let target = targets.first else {
return .launchManual(socketPath: explicitSocket)
}
return self.action(forExisting: target)
}
let defaultTarget = targets.first { $0.role == .defaultDaemon }
let scopedTargets = targets.filter { $0.role == .buildScopedDaemon }
if let defaultTarget, self.isCurrentReusableTarget(defaultTarget) {
return self.action(forExisting: defaultTarget)
}
if let scopedTarget = scopedTargets.first(where: self.isCurrentReusableTarget) {
return self.action(forExisting: scopedTarget)
}
if defaultTarget != nil, let buildScopedSocketPath {
if scopedTargets.contains(where: { $0.client.socketPath == buildScopedSocketPath }) {
return .rejectIncompatible(socketPath: buildScopedSocketPath)
}
return .launchManual(socketPath: buildScopedSocketPath)
}
if let defaultTarget {
return .rejectIncompatible(socketPath: defaultTarget.client.socketPath)
}
return .launchManual(socketPath: defaultSocketPath)
}
static func shouldMigrateLegacyTarget(
explicitSocket: String?,
destinationSocketPath: String,
defaultSocketPath: String,
targets: [DaemonControlTarget]
) -> Bool {
explicitSocket == nil &&
NSString(string: destinationSocketPath).standardizingPath ==
NSString(string: defaultSocketPath).standardizingPath &&
!targets.contains { $0.role == .defaultDaemon }
}
private static func action(forExisting target: DaemonControlTarget) -> DaemonStartAction {
guard DaemonControlClient.isReusableDaemonStatus(target.status) else {
return .rejectIncompatible(socketPath: target.client.socketPath)
}
guard target.status.mode == .auto else {
return .useExisting(socketPath: target.client.socketPath)
}
guard DaemonControlClient.isIdleForMigration(target.status) else {
return .rejectBusy(socketPath: target.client.socketPath)
}
guard DaemonControlClient.supportsSafeMigration(target.status),
let pid = target.status.pid
else {
return .rejectUnsafe(socketPath: target.client.socketPath)
}
return .promoteAutoToManual(socketPath: target.client.socketPath, pid: pid)
}
private static func isCurrentReusableTarget(_ target: DaemonControlTarget) -> Bool {
DaemonControlClient.isReusableDaemonStatus(target.status) && self.supportsCurrentDaemon(target.status)
}
}
enum DaemonControlResolver {
private static let historicalProbeTimeoutSeconds: TimeInterval = 1
static func defaultSocketPaths() -> [String] {
let buildScopedPath = DaemonLaunchPolicy.buildScopedDaemonSocketPath(
daemonSocketPath: PeekabooBridgeConstants.daemonSocketPath,
runtimeBuildIdentity: DaemonLaunchPolicy.runtimeBuildIdentity()
)
return [PeekabooBridgeConstants.daemonSocketPath, buildScopedPath].compactMap(\.self)
}
static func historicalBuildScopedSocketPaths(
daemonSocketPath: String,
currentBuildScopedSocketPath: String?,
candidates: [DaemonSocketFileCandidate],
currentUID: uid_t = getuid()
) -> [String] {
let daemonDirectory = Self.standardizedSocketPath(
URL(fileURLWithPath: daemonSocketPath).deletingLastPathComponent().path
)
let excludedPaths = Set([daemonSocketPath, currentBuildScopedSocketPath].compactMap { path in
path.map(Self.standardizedSocketPath)
})
return candidates
.filter { candidate in
candidate.isSocket &&
candidate.ownerUID == currentUID &&
Self.standardizedSocketPath(
URL(fileURLWithPath: candidate.path).deletingLastPathComponent().path
) == daemonDirectory &&
Self.isBuildScopedSocketName(URL(fileURLWithPath: candidate.path).lastPathComponent) &&
!excludedPaths.contains(Self.standardizedSocketPath(candidate.path))
}
.map(\.path)
.sorted()
}
static func isValidatedHistoricalTarget(
status: PeekabooDaemonStatus,
socketPath: String
) -> Bool {
guard status.running,
DaemonControlClient.isReusableDaemonStatus(status),
DaemonControlClient.supportsSafeMigration(status),
status.pid.map({ $0 > 0 }) == true,
let bridge = status.bridge,
bridge.hostKind == .onDemand,
standardizedSocketPath(bridge.socketPath) == standardizedSocketPath(socketPath)
else {
return false
}
let operationNames = Set(bridge.availableOperationNames ?? bridge.allowedOperations.map(\.rawValue))
return operationNames.contains(PeekabooBridgeOperation.daemonStatus.rawValue) &&
operationNames.contains(PeekabooBridgeOperation.daemonStop.rawValue)
}
static func targets(explicitSocket: String?) async -> [DaemonControlTarget] {
if let explicitSocket {
let client = DaemonControlClient(socketPath: explicitSocket)
guard let status = await client.fetchStatus() else { return [] }
return [DaemonControlTarget(client: client, status: status, role: .explicit)]
}
var targets: [DaemonControlTarget] = []
let defaultSocketPaths = self.defaultSocketPaths()
for (index, socketPath) in defaultSocketPaths.enumerated() {
let client = DaemonControlClient(socketPath: socketPath)
if let status = await client.fetchControllableDaemonStatus() {
targets.append(DaemonControlTarget(
client: client,
status: status,
role: index == 0 ? .defaultDaemon : .buildScopedDaemon
))
}
}
await targets.append(contentsOf: self.validatedHistoricalTargets(
daemonSocketPath: PeekabooBridgeConstants.daemonSocketPath,
currentBuildScopedSocketPath: defaultSocketPaths.dropFirst().first
))
let legacyClient = DaemonControlClient(socketPath: PeekabooBridgeConstants.peekabooSocketPath)
if let status = await legacyClient.fetchControllableDaemonStatus() {
targets.append(DaemonControlTarget(
client: legacyClient,
status: status,
role: .legacyDefault
))
}
return targets
}
static func validatedHistoricalTargets(
daemonSocketPath: String,
currentBuildScopedSocketPath: String?
) async -> [DaemonControlTarget] {
var targets: [DaemonControlTarget] = []
for socketPath in self.discoveredHistoricalBuildScopedSocketPaths(
daemonSocketPath: daemonSocketPath,
currentBuildScopedSocketPath: currentBuildScopedSocketPath
) {
let client = DaemonControlClient(
socketPath: socketPath,
requestTimeoutSec: self.historicalProbeTimeoutSeconds
)
guard let status = await client.fetchControllableDaemonStatus(),
self.isValidatedHistoricalTarget(status: status, socketPath: socketPath)
else {
continue
}
targets.append(DaemonControlTarget(
client: client,
status: status,
role: .buildScopedDaemon
))
}
return targets
}
private static func discoveredHistoricalBuildScopedSocketPaths(
daemonSocketPath: String,
currentBuildScopedSocketPath: String?
) -> [String] {
let directoryURL = URL(fileURLWithPath: daemonSocketPath).deletingLastPathComponent()
guard let urls = try? FileManager.default.contentsOfDirectory(
at: directoryURL,
includingPropertiesForKeys: nil,
options: [.skipsHiddenFiles]
)
else {
return []
}
let candidates = urls.compactMap { url -> DaemonSocketFileCandidate? in
var info = stat()
guard lstat(url.path, &info) == 0 else { return nil }
return DaemonSocketFileCandidate(
path: url.path,
isSocket: info.st_mode & mode_t(S_IFMT) == mode_t(S_IFSOCK),
ownerUID: info.st_uid
)
}
return self.historicalBuildScopedSocketPaths(
daemonSocketPath: daemonSocketPath,
currentBuildScopedSocketPath: currentBuildScopedSocketPath,
candidates: candidates
)
}
private static func isBuildScopedSocketName(_ name: String) -> Bool {
guard name.hasPrefix("daemon-"), name.hasSuffix(".sock") else { return false }
let hash = name.dropFirst("daemon-".count).dropLast(".sock".count)
return hash.count == 16 && hash.allSatisfy { ("0"..."9").contains($0) || ("a"..."f").contains($0) }
}
private static func standardizedSocketPath(_ path: String) -> String {
NSString(string: path).standardizingPath
}
}
enum DaemonPaths {
static func daemonLogURL() -> URL {
let root = FileManager.default.homeDirectoryForCurrentUser
@ -131,7 +532,7 @@ enum DaemonStatusPrinter {
print("------")
print("Socket: \(bridge.socketPath)")
print("Host: \(bridge.hostKind.rawValue)")
print("Ops: \(bridge.allowedOperations.count)")
print("Ops: \(bridge.availableOperationNames?.count ?? bridge.allowedOperations.count)")
}
if let permissions = status.permissions {

View File

@ -44,6 +44,9 @@ extension DialogCommand {
do {
try self.target.validate()
if self.focusOptions.autoFocus {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocused(
snapshotId: nil,
target: self.target,
@ -54,6 +57,7 @@ extension DialogCommand {
let resolvedWindowTitle = try await self.target.resolveWindowTitleOptional(services: self.services)
let appHint = try await DialogCommand.resolveDialogAppHint(target: self.target, services: self.services)
self.resolvedRuntime.beginInteractionMutation()
let result = try await self.services.dialogs.clickButton(
buttonText: self.button,
windowTitle: resolvedWindowTitle,

View File

@ -49,6 +49,9 @@ extension DialogCommand {
do {
try self.target.validate()
if self.focusOptions.autoFocus {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocused(
snapshotId: nil,
target: self.target,
@ -56,9 +59,10 @@ extension DialogCommand {
services: self.services
)
let resolvedWindowTitle = try await self.target.resolveWindowTitleOptional(services: self.services)
let resolvedWindowTitle = try await target.resolveWindowTitleOptional(services: self.services)
let appHint = try await DialogCommand.resolveDialogAppHint(target: self.target, services: self.services)
let result = try await self.services.dialogs.dismissDialog(
self.resolvedRuntime.beginInteractionMutation()
let result = try await services.dialogs.dismissDialog(
force: self.force,
windowTitle: resolvedWindowTitle,
appName: appHint
@ -142,6 +146,9 @@ extension DialogCommand {
do {
try self.target.validate()
if self.focusOptions.autoFocus, self.target.hasAnyTarget {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocused(
snapshotId: nil,
target: self.target,
@ -149,10 +156,10 @@ extension DialogCommand {
services: self.services
)
let resolvedWindowTitle = try await self.target.resolveWindowTitleOptional(services: self.services)
let resolvedWindowTitle = try await target.resolveWindowTitleOptional(services: self.services)
let appHint = try await DialogCommand.resolveDialogAppHint(target: self.target, services: self.services)
let dialogService = self.services.dialogs
let timeoutSeconds = self.timeoutSeconds
let timeoutSeconds = timeoutSeconds
let elements = try await withMainActorCommandTimeout(
seconds: timeoutSeconds,
operationName: "dialog list"

View File

@ -62,6 +62,9 @@ extension DialogCommand {
do {
try self.target.validate()
if self.focusOptions.autoFocus {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocused(
snapshotId: nil,
target: self.target,
@ -76,9 +79,11 @@ extension DialogCommand {
let select = self.select
let ensureExpanded = self.ensureExpanded
self.resolvedRuntime.beginInteractionMutation()
let result = try await withMainActorCommandTimeout(
seconds: self.timeoutSeconds,
operationName: "dialog file"
operationName: "dialog file",
desktopMutationWatermarkStore: DesktopMutationWatermarkStore()
) {
try await dialogs.handleFileDialog(
path: path,

View File

@ -58,6 +58,9 @@ extension DialogCommand {
do {
try self.target.validate()
if self.focusOptions.autoFocus {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocused(
snapshotId: nil,
target: self.target,
@ -69,6 +72,7 @@ extension DialogCommand {
let appHint = try await DialogCommand.resolveDialogAppHint(target: self.target, services: self.services)
let fieldIdentifier = self.field ?? self.index.map { String($0) }
self.resolvedRuntime.beginInteractionMutation()
let result = try await self.services.dialogs.enterText(
text: self.text,
fieldIdentifier: fieldIdentifier,

View File

@ -44,6 +44,7 @@ extension DockCommand {
self.logger.setJsonOutputMode(self.jsonOutput)
do {
self.resolvedRuntime.beginInteractionMutation()
try await DockServiceBridge.launchFromDock(dock: self.services.dock, appName: self.app)
let dockItem = try await DockServiceBridge.findDockItem(dock: self.services.dock, name: self.app)
if self.verify {

View File

@ -43,6 +43,7 @@ extension DockCommand {
do {
let dockItem = try await DockServiceBridge.findDockItem(dock: self.services.dock, name: self.app)
self.resolvedRuntime.beginInteractionMutation()
try await DockServiceBridge.rightClickDockItem(
dock: self.services.dock,
appName: self.app,

View File

@ -37,6 +37,7 @@ extension DockCommand {
self.logger.setJsonOutputMode(self.jsonOutput)
do {
self.resolvedRuntime.beginInteractionMutation()
try await DockServiceBridge.hideDock(dock: self.services.dock)
AutomationEventLogger.log(.dock, "hide")
@ -91,6 +92,7 @@ extension DockCommand {
self.logger.setJsonOutputMode(self.jsonOutput)
do {
self.resolvedRuntime.beginInteractionMutation()
try await DockServiceBridge.showDock(dock: self.services.dock)
AutomationEventLogger.log(.dock, "show")

View File

@ -137,8 +137,10 @@ struct MenuBarCommand: ParsableCommand, ErrorHandlingCommand, OutputFormattable
let focusSnapshot = self.verify ? try await verifier.captureFocusSnapshot() : nil
let result: PeekabooCore.ClickResult
if let idx = self.index {
self.resolvedRuntime.beginInteractionMutation()
result = try await MenuServiceBridge.clickMenuBarItem(at: idx, menu: self.services.menu)
} else if let name = self.itemName {
self.resolvedRuntime.beginInteractionMutation()
result = try await MenuServiceBridge.clickMenuBarItem(named: name, menu: self.services.menu)
} else {
throw PeekabooError.invalidInput("Please provide either a menu bar item name or use --index")

View File

@ -76,6 +76,9 @@ extension MenuCommand {
try self.target.validate()
let appIdentifier = try await self.resolveTargetApplicationIdentifier()
let windowID = try await self.target.resolveWindowID(services: self.services)
if self.focusOptions.autoFocus {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocusIgnoringMissingWindows(
request: FocusIgnoringMissingWindowsRequest(
windowID: windowID,
@ -92,6 +95,7 @@ extension MenuCommand {
try await self.ensureMenuItemEnabled(appIdentifier: appIdentifier, menuPath: canonicalPath)
}
self.resolvedRuntime.beginInteractionMutation()
if let itemName = normalizedItem {
try await MenuServiceBridge.clickMenuItemByName(
menu: self.services.menu,

View File

@ -50,6 +50,7 @@ extension MenuCommand {
let verifier = MenuBarClickVerifier(services: self.services)
let verifyTarget = self.verify ? try await self.resolveVerificationTarget() : nil
let preFocus = self.verify ? try await verifier.captureFocusSnapshot() : nil
self.resolvedRuntime.beginInteractionMutation()
let clickResult = try await MenuServiceBridge
.clickMenuBarItem(named: self.title, menu: self.services.menu)

View File

@ -48,6 +48,9 @@ extension MenuCommand {
try self.target.validate()
let appIdentifier = try await self.resolveTargetApplicationIdentifier()
let windowID = try await self.target.resolveWindowID(services: self.services)
if self.focusOptions.autoFocus {
self.resolvedRuntime.beginInteractionMutation()
}
try await ensureFocusIgnoringMissingWindows(
request: FocusIgnoringMissingWindowsRequest(
windowID: windowID,

View File

@ -6,11 +6,6 @@ import PeekabooFoundation
@available(macOS 14.0, *)
@MainActor
struct OpenCommand: ParsableCommand, OutputFormattable, ErrorHandlingCommand, RuntimeOptionsConfigurable {
@MainActor
static var launcher: any ApplicationLaunching = ApplicationLaunchEnvironment.launcher
@MainActor
static var resolver: any ApplicationURLResolving = ApplicationURLResolverEnvironment.resolver
nonisolated(unsafe) static var commandDescription: CommandDescription {
MainActorCommandDescription.describe {
CommandDescription(
@ -64,6 +59,10 @@ struct OpenCommand: ParsableCommand, OutputFormattable, ErrorHandlingCommand, Ru
self.resolvedRuntime.logger
}
private var services: any PeekabooServiceProviding {
self.resolvedRuntime.services
}
var outputLogger: Logger {
self.logger
}
@ -82,13 +81,21 @@ struct OpenCommand: ParsableCommand, OutputFormattable, ErrorHandlingCommand, Ru
do {
let targetURL = try Self.resolveTarget(self.target)
let handlerURL = try self.resolveHandlerApplication()
let appInstance = try await self.openTarget(targetURL: targetURL, handlerURL: handlerURL)
try await self.waitIfNeeded(for: appInstance)
let didFocus = self.activateIfNeeded(appInstance)
self.renderSuccess(app: appInstance, targetURL: targetURL, didFocus: didFocus)
let handlerIdentifier = self.bundleId == nil
? app.map { ApplicationIdentifierResolver.resolve($0) }
: nil
self.resolvedRuntime.beginInteractionMutation()
let app = try await services.applications.launchApplication(request: ApplicationLaunchRequest(
applicationIdentifier: handlerIdentifier,
applicationBundleIdentifier: self.bundleId,
openURLs: [targetURL],
activates: self.shouldFocus,
waitUntilReady: self.waitUntilReady
))
await self.invalidateSnapshotsAfterOpen()
self.renderSuccess(app: app, targetURL: targetURL)
} catch {
self.handleError(error)
handleError(error, customCode: applicationLaunchErrorCode(for: error))
throw ExitCode.failure
}
}
@ -98,6 +105,14 @@ struct OpenCommand: ParsableCommand, OutputFormattable, ErrorHandlingCommand, Ru
self.logger.setJsonOutputMode(self.jsonOutput)
}
private func invalidateSnapshotsAfterOpen() async {
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "open"
)
}
static func resolveTarget(_ target: String, cwd: String = FileManager.default.currentDirectoryPath) throws -> URL {
let trimmed = target.trimmingCharacters(in: .whitespacesAndNewlines)
guard !trimmed.isEmpty else {
@ -118,52 +133,17 @@ struct OpenCommand: ParsableCommand, OutputFormattable, ErrorHandlingCommand, Ru
return URL(fileURLWithPath: absolutePath)
}
private func resolveHandlerApplication() throws -> URL? {
if let bundleId {
return try Self.resolver.resolveBundleIdentifier(bundleId)
}
if let app {
return try Self.resolver.resolveApplication(appIdentifier: app, bundleId: nil)
}
return nil
}
private func openTarget(targetURL: URL, handlerURL: URL?) async throws -> any RunningApplicationHandle {
try await Self.launcher.openTarget(targetURL, handlerURL: handlerURL, activates: self.shouldFocus)
}
private func waitIfNeeded(for app: any RunningApplicationHandle) async throws {
guard self.waitUntilReady else { return }
try await self.waitForApplicationReady(app)
}
private func activateIfNeeded(_ app: any RunningApplicationHandle) -> Bool {
guard self.shouldFocus else { return false }
if app.isActive {
return true
}
let activated = app.activate(options: [])
if !activated {
self.logger.warn("Open succeeded but failed to focus \(app.localizedName ?? "application")")
}
return activated
}
private func renderSuccess(app: any RunningApplicationHandle, targetURL: URL, didFocus: Bool) {
private func renderSuccess(app: ServiceApplicationInfo, targetURL: URL) {
let result = OpenResult(
success: true,
action: "open",
target: self.target,
resolved_target: self.normalizedTargetString(for: targetURL),
handler_app: app.localizedName ?? app.bundleIdentifier ?? "unknown",
target: target,
resolved_target: normalizedTargetString(for: targetURL),
handler_app: app.name,
bundle_id: app.bundleIdentifier,
pid: app.processIdentifier,
is_ready: app.isFinishedLaunching,
focused: didFocus && self.shouldFocus
is_ready: app.isFinishedLaunching ?? !self.waitUntilReady,
focused: self.shouldFocus && app.isActive
)
AutomationEventLogger.log(
.open,
@ -172,18 +152,7 @@ struct OpenCommand: ParsableCommand, OutputFormattable, ErrorHandlingCommand, Ru
)
output(result) {
let handler = app.localizedName ?? app.bundleIdentifier ?? "application"
print("✅ Opened \(result.resolved_target) with \(handler)")
}
}
private func waitForApplicationReady(_ app: any RunningApplicationHandle, timeout: TimeInterval = 10) async throws {
let start = Date()
while !app.isFinishedLaunching {
if Date().timeIntervalSince(start) > timeout {
throw PeekabooError.timeout("Application did not become ready within \(Int(timeout)) seconds")
}
try await Task.sleep(nanoseconds: 100_000_000)
print("✅ Opened \(result.resolved_target) with \(app.name)")
}
}

View File

@ -63,14 +63,28 @@ struct RunCommand: OutputFormattable {
var didEmitJSONResponse = false
do {
let resolvedScriptPath = self.resolvedScriptPath()
let resolvedScriptPath = resolvedScriptPath()
let script = try await ProcessServiceBridge.loadScript(services: self.services, path: resolvedScriptPath)
switch Self.finalSnapshotEffect(in: script) {
case .none:
break
case .mutation, .freshObservation:
self.resolvedRuntime.beginInteractionMutation()
}
let results = try await ProcessServiceBridge.executeScript(
services: self.services,
script,
failFast: !self.noFailFast,
verbose: self.isVerbose
)
if let freshObservation = Self.terminalFreshObservation(in: script, results: results) {
self.resolvedRuntime.preserveFreshObservation(
snapshotId: freshObservation.snapshotId,
startedAt: freshObservation.confirmedMutationCompletedAt ?? freshObservation.startedAt,
preservedAt: freshObservation.completedAt,
preservationAllowed: freshObservation.preservationAllowed
)
}
let output = ScriptExecutionResult(
success: results.allSatisfy(\.success),
@ -84,7 +98,7 @@ struct RunCommand: OutputFormattable {
)
if let outputPath = self.output {
let resolvedOutputPath = self.resolvedOutputPath(from: outputPath)
let resolvedOutputPath = resolvedOutputPath(from: outputPath)
let data = try JSONEncoder().encode(output)
try data.write(to: URL(fileURLWithPath: resolvedOutputPath))
if !self.jsonOutput {
@ -131,6 +145,106 @@ struct RunCommand: OutputFormattable {
PathResolver.expandPath(outputPath)
}
enum ScriptFinalSnapshotEffect: Equatable {
case none
case mutation
case freshObservation
}
static func finalSnapshotEffect(in script: PeekabooScript) -> ScriptFinalSnapshotEffect {
var effect = ScriptFinalSnapshotEffect.none
for step in script.steps {
switch step.command.lowercased() {
case "sleep":
continue
case "see":
effect = self.isFreshObservationStep(step) ? .freshObservation : .mutation
case "dock":
if self.isReadOnlyDockStep(step) {
continue
}
effect = .mutation
case "clipboard":
if self.isReadOnlyClipboardStep(step) {
continue
}
effect = .mutation
default:
effect = .mutation
}
}
return effect
}
private static func isFreshObservationStep(_ step: ScriptStep) -> Bool {
guard step.command.lowercased() == "see" else { return false }
let annotate: Bool? = switch step.params {
case let .screenshot(parameters):
parameters.annotate
case let .generic(parameters):
parameters["annotate"].flatMap { Bool($0) }
default:
nil
}
return annotate ?? true
}
private static func isReadOnlyClipboardStep(_ step: ScriptStep) -> Bool {
let action: String? = switch step.params {
case let .clipboard(parameters):
parameters.action
case let .generic(parameters):
parameters["action"]
default:
nil
}
guard let action else { return false }
return ["get", "save"].contains(action.trimmingCharacters(in: .whitespacesAndNewlines).lowercased())
}
private static func isReadOnlyDockStep(_ step: ScriptStep) -> Bool {
let action: String? = switch step.params {
case let .dock(parameters):
parameters.action
case let .generic(parameters):
parameters["action"]
default:
nil
}
return action?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() == "list"
}
struct TerminalFreshObservation {
let snapshotId: String
let startedAt: Date
let completedAt: Date
let confirmedMutationCompletedAt: Date?
let preservationAllowed: Bool
}
static func terminalFreshObservation(
in script: PeekabooScript,
results: [StepResult]
) -> TerminalFreshObservation? {
guard self.finalSnapshotEffect(in: script) == .freshObservation,
results.count == script.steps.count,
results.allSatisfy(\.success),
let observationIndex = script.steps.lastIndex(where: { self.isFreshObservationStep($0) })
else { return nil }
let result = results[observationIndex]
guard result.command.lowercased() == "see",
let snapshotId = result.snapshotId,
let startedAt = result.startedAt
else { return nil }
return TerminalFreshObservation(
snapshotId: snapshotId,
startedAt: startedAt,
completedAt: startedAt.addingTimeInterval(result.executionTime),
confirmedMutationCompletedAt: result.desktopMutationCompletedAt,
preservationAllowed: result.desktopMutationPreservationAllowed ?? true
)
}
@MainActor
private func printSummary(_ result: ScriptExecutionResult) {
if result.success {

View File

@ -86,6 +86,7 @@ struct MoveWindowSubcommand: ApplicationResolvable, ErrorHandlingCommand, Output
let spaceService = SpaceCommandEnvironment.service
if self.toCurrent {
self.resolvedRuntime.beginInteractionMutation()
try await spaceService.moveWindowToCurrentSpace(windowID: windowID)
AutomationEventLogger.log(
.space,
@ -119,6 +120,7 @@ struct MoveWindowSubcommand: ApplicationResolvable, ErrorHandlingCommand, Output
}
let targetSpace = spaces[spaceNum - 1]
self.resolvedRuntime.beginInteractionMutation()
try await spaceService.moveWindowToSpace(windowID: windowID, spaceID: targetSpace.id)
if self.follow {
try await spaceService.switchToSpace(targetSpace.id)

View File

@ -43,6 +43,7 @@ struct SwitchSubcommand: ErrorHandlingCommand, OutputFormattable {
}
let targetSpace = spaces[self.to - 1]
self.resolvedRuntime.beginInteractionMutation()
try await spaceService.switchToSpace(targetSpace.id)
AutomationEventLogger.log(
.space,

View File

@ -97,6 +97,7 @@ extension WindowCommand {
guard hasWindowTarget || snapshotContext != nil else { preconditionFailure("validated above") }
// Use enhanced focus with space support
self.resolvedRuntime.beginInteractionMutation()
if let windowID = windowInfo?.windowID {
try await ensureFocused(
windowID: CGWindowID(windowID),
@ -117,6 +118,11 @@ extension WindowCommand {
} else {
throw ValidationError("Either --app, --pid, --window-id, or --snapshot must be specified")
}
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: self.resolvedRuntime.interactionMutationTargets,
logger: self.logger,
reason: "window focus"
)
let refreshedWindowInfo: ServiceWindowInfo? = if hasWindowTarget {
await self.windowOptions.refetchWindowInfo(
@ -141,12 +147,6 @@ extension WindowCommand {
expectedApp: appInfo
)
}
await InteractionObservationInvalidator.invalidateAfterMutationOrLatest(
observation,
snapshots: self.services.snapshots,
logger: self.logger,
reason: "window focus"
)
logWindowAction(
action: "focus",
appName: appName,

View File

@ -65,10 +65,10 @@ extension WindowCommand {
// Move the window
let newOrigin = CGPoint(x: x, y: y)
self.resolvedRuntime.beginInteractionMutation()
try await WindowServiceBridge.moveWindow(windows: self.services.windows, target: target, to: newOrigin)
await invalidateLatestSnapshotAfterWindowMutation(
services: self.services,
logger: self.logger,
runtime: self.resolvedRuntime,
reason: "window move"
)
@ -180,10 +180,10 @@ extension WindowCommand {
// Resize the window
let newSize = CGSize(width: width, height: height)
self.resolvedRuntime.beginInteractionMutation()
try await WindowServiceBridge.resizeWindow(windows: self.services.windows, target: target, to: newSize)
await invalidateLatestSnapshotAfterWindowMutation(
services: self.services,
logger: self.logger,
runtime: self.resolvedRuntime,
reason: "window resize"
)
@ -284,14 +284,14 @@ extension WindowCommand {
// Set bounds
let newBounds = CGRect(x: x, y: y, width: width, height: height)
self.resolvedRuntime.beginInteractionMutation()
try await WindowServiceBridge.setWindowBounds(
windows: self.services.windows,
target: target,
bounds: newBounds
)
await invalidateLatestSnapshotAfterWindowMutation(
services: self.services,
logger: self.logger,
runtime: self.resolvedRuntime,
reason: "window set-bounds"
)

View File

@ -55,10 +55,10 @@ extension WindowCommand {
}
// Perform the action
self.resolvedRuntime.beginInteractionMutation()
try await WindowServiceBridge.closeWindow(windows: self.services.windows, target: target)
await invalidateLatestSnapshotAfterWindowMutation(
services: self.services,
logger: self.logger,
runtime: self.resolvedRuntime,
reason: "window close"
)
@ -137,10 +137,10 @@ extension WindowCommand {
}
// Perform the action
self.resolvedRuntime.beginInteractionMutation()
try await WindowServiceBridge.minimizeWindow(windows: self.services.windows, target: target)
await invalidateLatestSnapshotAfterWindowMutation(
services: self.services,
logger: self.logger,
runtime: self.resolvedRuntime,
reason: "window minimize"
)
logWindowAction(
@ -218,10 +218,10 @@ extension WindowCommand {
}
// Perform the action
self.resolvedRuntime.beginInteractionMutation()
try await WindowServiceBridge.maximizeWindow(windows: self.services.windows, target: target)
await invalidateLatestSnapshotAfterWindowMutation(
services: self.services,
logger: self.logger,
runtime: self.resolvedRuntime,
reason: "window maximize"
)
logWindowAction(

View File

@ -177,13 +177,12 @@ func logWindowAction(
@MainActor
func invalidateLatestSnapshotAfterWindowMutation(
services: any PeekabooServiceProviding,
logger: Logger,
runtime: CommandRuntime,
reason: String
) async {
await InteractionObservationInvalidator.invalidateLatestSnapshot(
using: services.snapshots,
logger: logger,
await InteractionObservationInvalidator.invalidateAfterMutation(
targets: runtime.interactionMutationTargets,
logger: runtime.logger,
reason: reason
)
}

View File

@ -44,20 +44,25 @@ enum PermissionHelpers {
/// Try to fetch permissions from a remote Peekaboo Bridge host; falls back to local services on failure.
@MainActor
private static func remotePermissionsStatus(socketPath override: String? = nil) async -> PermissionsStatus? {
let envSocket = ProcessInfo.processInfo.environment["PEEKABOO_BRIDGE_SOCKET"]
private static func remotePermissionsStatus(
services: any PeekabooServiceProviding,
socketPath override: String? = nil
) async -> PermissionsStatus? {
let environment = ProcessInfo.processInfo.environment
let envSocket = environment["PEEKABOO_BRIDGE_SOCKET"]
let resolvedOverride = override ?? envSocket
let candidates: [String] = if let explicit = resolvedOverride, !explicit.isEmpty {
[explicit]
} else {
[
PeekabooBridgeConstants.peekabooSocketPath,
PeekabooBridgeConstants.claudeSocketPath,
PeekabooBridgeConstants.clawdbotSocketPath,
]
if resolvedOverride == nil,
let remoteServices = services as? RemotePeekabooServices,
let status = try? await remoteServices.permissionsStatus() {
return status
}
let candidates = self.remotePermissionSocketPaths(
explicitSocket: resolvedOverride,
environment: environment
)
let identity = PeekabooBridgeClientIdentity(
bundleIdentifier: Bundle.main.bundleIdentifier,
teamIdentifier: nil,
@ -69,6 +74,28 @@ enum PermissionHelpers {
let client = PeekabooBridgeClient(socketPath: socketPath)
do {
let handshake = try await client.handshake(client: identity, requestedHost: nil)
if resolvedOverride == nil {
guard let role = DaemonLaunchPolicy.implicitRuntimeCandidateRole(
socketPath: socketPath,
daemonSocketPath: DaemonLaunchPolicy.daemonSocketPath(environment: environment),
buildScopedDaemonSocketPath: DaemonLaunchPolicy.buildScopedDaemonSocketPath(
daemonSocketPath: DaemonLaunchPolicy.daemonSocketPath(environment: environment),
runtimeBuildIdentity: DaemonLaunchPolicy.runtimeBuildIdentity()
)
) else {
continue
}
let daemonStatus = handshake.hostKind == .gui
? nil
: await DaemonControlClient(socketPath: socketPath).fetchStatus()
guard DaemonLaunchPolicy.isSelectableImplicitRuntimeCandidate(
role: role,
handshake: handshake,
daemonStatus: daemonStatus
) else {
continue
}
}
guard handshake.supportedOperations.contains(.permissionsStatus) else { continue }
return try await client.permissionsStatus()
} catch {
@ -78,13 +105,31 @@ enum PermissionHelpers {
return nil
}
static func remotePermissionSocketPaths(
explicitSocket: String?,
environment: [String: String] = ProcessInfo.processInfo.environment
) -> [String] {
if let explicitSocket, !explicitSocket.isEmpty {
return [explicitSocket]
}
let daemonPath = DaemonLaunchPolicy.daemonSocketPath(environment: environment)
guard DaemonLaunchPolicy.shouldMigrateLegacyDaemon(targetSocketPath: daemonPath) else {
return [daemonPath]
}
let buildScopedPath = DaemonLaunchPolicy.buildScopedDaemonSocketPath(
daemonSocketPath: daemonPath,
runtimeBuildIdentity: DaemonLaunchPolicy.runtimeBuildIdentity()
)
return [daemonPath, buildScopedPath, PeekabooBridgeConstants.peekabooSocketPath].compactMap(\.self)
}
/// Get current permission status for all Peekaboo permissions
static func getCurrentPermissions(
services: any PeekabooServiceProviding,
allowRemote: Bool = true,
socketPath: String? = nil
) async -> [PermissionInfo] {
let response = await self.getCurrentPermissionsWithSource(
let response = await getCurrentPermissionsWithSource(
services: services,
allowRemote: allowRemote,
socketPath: socketPath
@ -100,7 +145,7 @@ enum PermissionHelpers {
) async -> PermissionStatusResponse {
// Prefer remote host when available so sandboxes can reuse existing TCC grants.
let remoteStatus = allowRemote
? await self.remotePermissionsStatus(socketPath: socketPath)
? await remotePermissionsStatus(services: services, socketPath: socketPath)
: nil
let status: PermissionsStatus
@ -121,9 +166,9 @@ enum PermissionHelpers {
socketPath: String? = nil
) async -> PermissionSourcesResponse {
let remoteStatus = allowRemote
? await self.remotePermissionsStatus(socketPath: socketPath)
? await remotePermissionsStatus(services: services, socketPath: socketPath)
: nil
let localStatus = await self.localPermissionsStatus(services: services)
let localStatus = await localPermissionsStatus(services: services)
let selectedSource = remoteStatus != nil ? "bridge" : "local"
var sources: [PermissionSourceStatus] = []
@ -148,9 +193,12 @@ enum PermissionHelpers {
private static func localPermissionsStatus(services: any PeekabooServiceProviding) async -> PermissionsStatus {
await Task { @MainActor in
let screenRecording = await services.screenCapture.hasScreenRecordingPermission()
let accessibility = await services.automation.hasAccessibilityPermission()
let postEvent = services.permissions.checkPostEventPermission()
let localServices: any PeekabooServiceProviding = services is RemotePeekabooServices
? PeekabooServices()
: services
let screenRecording = await localServices.screenCapture.hasScreenRecordingPermission()
let accessibility = await localServices.automation.hasAccessibilityPermission()
let postEvent = localServices.permissions.checkPostEventPermission()
return PermissionsStatus(
screenRecording: screenRecording,
accessibility: accessibility,
@ -184,7 +232,8 @@ enum PermissionHelpers {
@MainActor
static func requestEventSynthesizingPermission(
services: any PeekabooServiceProviding
services: any PeekabooServiceProviding,
runtime: CommandRuntime
) async throws -> EventSynthesizingPermissionRequestResult {
if let remoteServices = services as? RemotePeekabooServices {
let status = try await remoteServices.permissionsStatus()
@ -199,7 +248,9 @@ enum PermissionHelpers {
}
do {
let granted = try await remoteServices.requestPostEventPermission()
let granted = try await self.performInteractivePermissionRequest(using: runtime) {
try await remoteServices.requestPostEventPermission()
}
return .init(
action: "request-event-synthesizing",
source: "bridge",
@ -223,7 +274,9 @@ enum PermissionHelpers {
)
}
let granted = permissions.requestPostEventPermission(interactive: true)
let granted = await self.performInteractivePermissionRequest(using: runtime) {
permissions.requestPostEventPermission(interactive: true)
}
return .init(
action: "request-event-synthesizing",
source: "local",
@ -233,6 +286,15 @@ enum PermissionHelpers {
)
}
@MainActor
static func performInteractivePermissionRequest<T>(
using runtime: CommandRuntime,
_ request: @MainActor () async throws -> T
) async rethrows -> T {
runtime.beginInteractionMutation()
return try await request()
}
/// Format permission status for display
static func formatPermissionStatus(_ permission: PermissionInfo) -> String {
let status = permission.isGranted ? "Granted" : "Not Granted"
@ -257,7 +319,7 @@ enum PermissionHelpers {
services: any PeekabooServiceProviding
) async -> String {
// Format permissions for help display with dynamic status
let permissions = await self.getCurrentPermissions(services: services)
let permissions = await getCurrentPermissions(services: services)
var output = ["PERMISSIONS:"]
for permission in permissions {

View File

@ -7,9 +7,9 @@
<key>CFBundleName</key>
<string>Peekaboo</string>
<key>CFBundleShortVersionString</key>
<string>3.1.1</string>
<string>3.5.3</string>
<key>CFBundleVersion</key>
<string>3.1.1</string>
<string>3.5.3</string>
<key>LSMinimumSystemVersion</key>
<string>15.0</string>
<key>LSUIElement</key>
@ -23,6 +23,6 @@
<key>NSScreenCaptureUsageDescription</key>
<string>Peekaboo needs screen recording permission to capture screenshots and analyze window content.</string>
<key>PeekabooVersionDisplayString</key>
<string>Peekaboo 3.1.1</string>
<string>Peekaboo 3.5.3</string>
</dict>
</plist>

View File

@ -1,3 +1,3 @@
{
"version": "3.4.1"
"version": "3.5.3"
}

View File

@ -11,9 +11,9 @@
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>3.1.1</string>
<string>3.5.3</string>
<key>CFBundleVersion</key>
<string>3.1.1</string>
<string>3.5.3</string>
<key>LSMinimumSystemVersion</key>
<string>13.0</string>
<key>NSMainStoryboardFile</key>

View File

@ -18,14 +18,14 @@ struct ClickCommandTests {
@Test
func `Click command parses coordinates correctly`() async throws {
let context = await self.makeContext()
let context = await makeContext()
let result = try await InProcessCommandRunner.run(
["click", "--coords", "100,200", "--foreground", "--json"],
services: context.services
)
#expect(result.exitStatus == 0)
let calls = await self.automationState(context) { $0.clickCalls }
let calls = await automationState(context) { $0.clickCalls }
let call = try #require(calls.first)
if case let .coordinates(point) = call.target {
#expect(point == CGPoint(x: 100, y: 200))
@ -44,14 +44,14 @@ struct ClickCommandTests {
@Test
func `Click command defaults to background coordinate clicks when pid is supplied`() async throws {
let context = await self.makeContext()
let context = await makeContext()
let result = try await InProcessCommandRunner.run(
["click", "--coords", "100,200", "--pid", "12345", "--global-coords", "--json"],
services: context.services
)
#expect(result.exitStatus == 0)
let calls = await self.automationState(context) { $0.targetedClickCalls }
let calls = await automationState(context) { $0.targetedClickCalls }
let call = try #require(calls.first)
if case let .coordinates(point) = call.target {
#expect(point == CGPoint(x: 100, y: 200))
@ -59,18 +59,65 @@ struct ClickCommandTests {
Issue.record("Expected coordinates click target")
}
#expect(call.targetProcessIdentifier == 12345)
#expect(call.targetWindowID == nil)
}
@Test
func `Click command pins background coordinate clicks to exact window`() async throws {
let application = Self.makeApplication()
let selectedWindow = Self.makeWindow(id: 42, title: "Editor", index: 0)
let context = await makeContext(application: application, windows: [selectedWindow])
let result = try await InProcessCommandRunner.run(
["click", "--coords", "10,10", "--window-id", "42", "--json"],
services: context.services
)
#expect(result.exitStatus == 0)
let calls = await automationState(context) { $0.targetedClickCalls }
let call = try #require(calls.first)
if case let .coordinates(point) = call.target {
#expect(point == CGPoint(x: 20, y: 30))
} else {
Issue.record("Expected coordinates click target")
}
#expect(call.targetProcessIdentifier == application.processIdentifier)
#expect(call.targetWindowID == selectedWindow.windowID)
}
@Test
func `Click command preserves exact window routing for global coordinates`() async throws {
let application = Self.makeApplication()
let selectedWindow = Self.makeWindow(id: 42, title: "Editor", index: 0)
let context = await makeContext(application: application, windows: [selectedWindow])
let result = try await InProcessCommandRunner.run(
["click", "--coords", "100,200", "--window-id", "42", "--global-coords", "--json"],
services: context.services
)
#expect(result.exitStatus == 0)
let calls = await automationState(context) { $0.targetedClickCalls }
let call = try #require(calls.first)
if case let .coordinates(point) = call.target {
#expect(point == CGPoint(x: 100, y: 200))
} else {
Issue.record("Expected coordinates click target")
}
#expect(call.targetProcessIdentifier == application.processIdentifier)
#expect(call.targetWindowID == selectedWindow.windowID)
}
@Test
func `Click command default element click uses cached snapshot without waiting`() async throws {
let context = await self.makeContext()
let context = await makeContext()
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await self.storeSnapshot(element: element, in: context.snapshots)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
let result = try await InProcessCommandRunner.run(
["click", "--on", "B1", "--snapshot", snapshotId, "--json"],
@ -78,8 +125,8 @@ struct ClickCommandTests {
)
#expect(result.exitStatus == 0)
let waitCalls = await self.automationState(context) { $0.waitForElementCalls }
let calls = await self.automationState(context) { $0.targetedClickCalls }
let waitCalls = await automationState(context) { $0.waitForElementCalls }
let calls = await automationState(context) { $0.targetedClickCalls }
#expect(waitCalls.isEmpty)
let call = try #require(calls.first)
#expect(call.snapshotId == snapshotId)
@ -93,14 +140,14 @@ struct ClickCommandTests {
@Test
func `Click command default query click resolves cached snapshot without waiting`() async throws {
let context = await self.makeContext()
let context = await makeContext()
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await self.storeSnapshot(element: element, in: context.snapshots)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
let result = try await InProcessCommandRunner.run(
["click", "Save", "--snapshot", snapshotId, "--json"],
@ -108,8 +155,8 @@ struct ClickCommandTests {
)
#expect(result.exitStatus == 0)
let waitCalls = await self.automationState(context) { $0.waitForElementCalls }
let calls = await self.automationState(context) { $0.targetedClickCalls }
let waitCalls = await automationState(context) { $0.waitForElementCalls }
let calls = await automationState(context) { $0.targetedClickCalls }
#expect(waitCalls.isEmpty)
let call = try #require(calls.first)
#expect(call.snapshotId == snapshotId)
@ -122,15 +169,248 @@ struct ClickCommandTests {
}
@Test
func `Click command reuses latest snapshot for element lookup with app target`() async throws {
let context = await self.makeContext()
func `Background click lookup failures preserve the implicit latest snapshot`() async throws {
let testCases = [
["--on", "missing"],
["Missing query"],
["--on", "B1", "--app", "MissingApp"],
]
for arguments in testCases {
let context = await makeContext()
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
let result = try await InProcessCommandRunner.run(
["click"] + arguments + ["--snapshot", snapshotId, "--json"],
services: context.services
)
#expect(result.exitStatus == 1)
#expect(context.snapshots.invalidationCutoffs.isEmpty)
#expect(await context.snapshots.getMostRecentSnapshot() == snapshotId)
let calls = await automationState(context) { $0.targetedClickCalls }
#expect(calls.isEmpty)
}
}
@Test
func `Click command pins element and query clicks to matching window selectors`() async throws {
let application = Self.makeApplication()
let otherWindow = Self.makeWindow(id: 41, title: "Other", index: 0)
let selectedWindow = Self.makeWindow(id: 42, title: "Editor", index: 1)
let testCases = [
(target: ["--on", "B1"], selector: ["--window-id", "42"]),
(target: ["Save"], selector: ["--app", application.name, "--window-title", "Editor"]),
(target: ["--on", "B1"], selector: ["--app", application.name, "--window-index", "1"]),
]
for testCase in testCases {
let context = await makeContext(application: application, windows: [otherWindow, selectedWindow])
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(
element: element,
windowID: selectedWindow.windowID,
windowTitle: selectedWindow.title,
in: context.snapshots
)
let result = try await InProcessCommandRunner.run(
["click"] + testCase.target + ["--snapshot", snapshotId, "--json"] + testCase.selector,
services: context.services
)
#expect(result.exitStatus == 0)
let calls = await automationState(context) { $0.targetedClickCalls }
let call = try #require(calls.first)
#expect(calls.count == 1)
#expect(call.snapshotId == snapshotId)
#expect(call.targetProcessIdentifier == application.processIdentifier)
#expect(call.targetWindowID == selectedWindow.windowID)
if case let .elementId(id) = call.target {
#expect(id == "B1")
} else {
Issue.record("Expected exact cached element target")
}
}
}
@Test
func `Click command rejects window selectors that contradict cached snapshot`() async throws {
let application = Self.makeApplication()
let selectedWindow = Self.makeWindow(id: 41, title: "Other", index: 0)
let snapshotWindow = Self.makeWindow(id: 42, title: "Editor", index: 1)
let testCases = [
(target: ["--on", "B1"], selector: ["--window-id", "41"]),
(target: ["Save"], selector: ["--app", application.name, "--window-title", "Other"]),
(target: ["--on", "B1"], selector: ["--app", application.name, "--window-index", "0"]),
]
for testCase in testCases {
let context = await makeContext(application: application, windows: [selectedWindow, snapshotWindow])
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(
element: element,
windowID: snapshotWindow.windowID,
windowTitle: snapshotWindow.title,
in: context.snapshots
)
let result = try await InProcessCommandRunner.run(
["click"] + testCase.target + ["--snapshot", snapshotId, "--json"] + testCase.selector,
services: context.services
)
#expect(result.exitStatus == 1)
#expect(result.combinedOutput.contains("window 42"))
#expect(result.combinedOutput.contains("window 41"))
let calls = await automationState(context) { $0.targetedClickCalls }
#expect(calls.isEmpty)
}
}
@Test
func `Click command rejects window selector when snapshot has no exact window`() async throws {
let application = Self.makeApplication()
let selectedWindow = Self.makeWindow(id: 42, title: "Editor", index: 0)
let context = await makeContext(application: application, windows: [selectedWindow])
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await self.storeSnapshot(element: element, in: context.snapshots)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
let result = try await InProcessCommandRunner.run(
["click", "--on", "B1", "--snapshot", snapshotId, "--window-id", "42", "--json"],
services: context.services
)
#expect(result.exitStatus == 1)
#expect(result.combinedOutput.contains("does not identify an exact window"))
let calls = await automationState(context) { $0.targetedClickCalls }
#expect(calls.isEmpty)
}
@Test
func `Foreground click accepts legacy snapshot without exact window metadata`() async throws {
let application = Self.makeApplication()
let selectedWindow = Self.makeWindow(id: 42, title: "Editor", index: 0)
let context = await makeContext(application: application, windows: [selectedWindow])
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
await MainActor.run {
context.automation.setWaitForElementResult(
WaitForElementResult(found: true, element: element, waitTime: 0),
for: .elementId("B1")
)
}
let result = try await InProcessCommandRunner.run(
[
"click", "--on", "B1", "--snapshot", snapshotId,
"--window-id", "42", "--foreground", "--no-auto-focus", "--json",
],
services: context.services
)
#expect(result.exitStatus == 0)
let foregroundCalls = await automationState(context) { $0.clickCalls }
let backgroundCalls = await automationState(context) { $0.targetedClickCalls }
#expect(foregroundCalls.count == 1)
#expect(backgroundCalls.isEmpty)
}
@Test
func `Click command succeeds when post-click diagnostics become stale`() async throws {
let context = await makeContext()
let element = DetectedElement(
id: "B1",
type: .button,
label: "Close",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
context.snapshots.uiAutomationSnapshotError = .snapshotStale("target changed after click")
let result = try await InProcessCommandRunner.run(
["click", "--on", "B1", "--snapshot", snapshotId, "--json"],
services: context.services
)
#expect(result.exitStatus == 0)
let calls = await automationState(context) { $0.targetedClickCalls }
#expect(calls.count == 1)
#expect(context.snapshots.invalidationCutoffs.count == 1)
#expect(await context.snapshots.getMostRecentSnapshot() == nil)
#expect(try await context.snapshots.getDetectionResult(snapshotId: snapshotId) != nil)
}
@Test
func `Foreground click failure after focus invalidates latest snapshot`() async throws {
let application = ServiceApplicationInfo(
processIdentifier: 12345,
bundleIdentifier: "com.example.peekaboo-focus-mutation-fixture",
name: "PeekabooFocusMutationFixture",
isActive: false,
windowCount: 0,
activationPolicy: .regular
)
let context = await makeContext(application: application)
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
let result = try await InProcessCommandRunner.run(
[
"click", "--on", "B1", "--snapshot", snapshotId,
"--app", application.name, "--foreground", "--json",
],
services: context.services
)
#expect(result.exitStatus == 1)
let applications = try #require(context.services.applications as? StubApplicationService)
#expect(applications.activateCalls == [application.name])
#expect(context.snapshots.invalidationCutoffs.count == 1)
#expect(await context.snapshots.getMostRecentSnapshot() == nil)
}
@Test
func `Click command reuses latest snapshot for element lookup with app target`() async throws {
let context = await makeContext()
let element = DetectedElement(
id: "B1",
type: .button,
label: "Save",
bounds: CGRect(x: 20, y: 30, width: 80, height: 30)
)
let snapshotId = try await storeSnapshot(element: element, in: context.snapshots)
await MainActor.run {
context.automation.setWaitForElementResult(
WaitForElementResult(found: true, element: element, waitTime: 0),
@ -144,19 +424,39 @@ struct ClickCommandTests {
)
#expect(result.exitStatus == 0)
let waitCalls = await self.automationState(context) { $0.waitForElementCalls }
let clickCalls = await self.automationState(context) { $0.clickCalls }
let waitCalls = await automationState(context) { $0.waitForElementCalls }
let clickCalls = await automationState(context) { $0.clickCalls }
#expect(waitCalls.first?.snapshotId == snapshotId)
#expect(clickCalls.first?.snapshotId == snapshotId)
}
private func makeContext() async -> TestServicesFactory.AutomationTestContext {
private func makeContext(
application: ServiceApplicationInfo? = nil,
windows: [ServiceWindowInfo] = []
) async -> TestServicesFactory.AutomationTestContext {
await MainActor.run {
TestServicesFactory.makeAutomationTestContext()
let applications = application.map { [$0] } ?? []
var windowsByApp: [String: [ServiceWindowInfo]] = [:]
if let application {
windowsByApp[application.name] = windows
windowsByApp["PID:\(application.processIdentifier)"] = windows
if let bundleIdentifier = application.bundleIdentifier {
windowsByApp[bundleIdentifier] = windows
}
}
return TestServicesFactory.makeAutomationTestContext(
applications: StubApplicationService(applications: applications, windowsByApp: windowsByApp),
windows: StubWindowService(windowsByApp: windowsByApp)
)
}
}
private func storeSnapshot(element: DetectedElement, in snapshots: StubSnapshotManager) async throws -> String {
private func storeSnapshot(
element: DetectedElement,
windowID: Int? = nil,
windowTitle: String? = nil,
in snapshots: StubSnapshotManager
) async throws -> String {
let snapshotId = try await snapshots.createSnapshot()
let detection = ElementDetectionResult(
snapshotId: snapshotId,
@ -169,7 +469,9 @@ struct ClickCommandTests {
windowContext: WindowContext(
applicationName: "TestApp",
applicationBundleId: "com.example.test",
applicationProcessId: 12345
applicationProcessId: 12345,
windowTitle: windowTitle,
windowID: windowID
)
)
)
@ -177,6 +479,27 @@ struct ClickCommandTests {
return snapshotId
}
private static func makeApplication() -> ServiceApplicationInfo {
ServiceApplicationInfo(
processIdentifier: 12345,
bundleIdentifier: "com.example.test",
name: "TestApp",
isActive: false,
windowCount: 2,
activationPolicy: .regular
)
}
private static func makeWindow(id: Int, title: String, index: Int) -> ServiceWindowInfo {
ServiceWindowInfo(
windowID: id,
title: title,
bounds: CGRect(x: 10, y: 20, width: 400, height: 300),
isMainWindow: index == 0,
index: index
)
}
private func automationState<T: Sendable>(
_ context: TestServicesFactory.AutomationTestContext,
_ operation: @MainActor (StubAutomationService) -> T

View File

@ -0,0 +1,81 @@
import Foundation
import PeekabooAutomationKit
import Testing
@testable import PeekabooCLI
@Suite(.tags(.safe), .serialized)
struct ClipboardCommandTests {
@Test
@MainActor
func `Clipboard set invalidates implicit latest only after marking the mutation`() async throws {
let snapshots = StubSnapshotManager()
let originalSnapshot = try await snapshots.createSnapshot()
let clipboard = StubClipboardService()
let tracker = InteractionMutationTracker()
var mutationWasMarkedBeforeWrite = false
clipboard.beforeMutation = {
mutationWasMarkedBeforeWrite = tracker.mutationStartedAt != nil
}
let services = TestServicesFactory.makePeekabooServices(
snapshots: snapshots,
clipboard: clipboard
)
let runtime = CommandRuntime(
configuration: .init(
verbose: false,
jsonOutput: true,
logLevel: nil,
captureEnginePreference: nil,
inputStrategy: nil
),
services: services,
interactionMutationTracker: tracker
)
var command = try ClipboardCommand.parse(["set", "--text", "updated", "--json"])
try await CommanderRuntimeExecutor.runWithImplicitSnapshotInvalidation(
using: runtime,
required: true,
requiresCallerBarrier: true
) {
try await command.run(using: runtime)
}
#expect(mutationWasMarkedBeforeWrite)
#expect(await snapshots.getMostRecentSnapshot() == nil)
#expect(try await snapshots.listSnapshots().map(\.id) == [originalSnapshot])
}
@Test
@MainActor
func `Clipboard get leaves implicit latest unchanged`() async throws {
let snapshots = StubSnapshotManager()
let originalSnapshot = try await snapshots.createSnapshot()
let clipboard = StubClipboardService()
clipboard.current = ClipboardReadResult(
utiIdentifier: "public.utf8-plain-text",
data: Data("current".utf8),
textPreview: "current"
)
let services = TestServicesFactory.makePeekabooServices(
snapshots: snapshots,
clipboard: clipboard
)
let runtime = CommandRuntime(
configuration: .init(
verbose: false,
jsonOutput: true,
logLevel: nil,
captureEnginePreference: nil,
inputStrategy: nil
),
services: services
)
var command = try ClipboardCommand.parse(["get", "--json"])
try await command.run(using: runtime)
#expect(await snapshots.getMostRecentSnapshot() == originalSnapshot)
#expect(snapshots.invalidationCutoffs.isEmpty)
}
}

View File

@ -0,0 +1,344 @@
import Darwin
import Foundation
import PeekabooCore
import PeekabooFoundation
import Testing
@testable import PeekabooCLI
@MainActor
@Suite(.serialized)
struct CommanderRuntimeOutputDeferralTests {
private let clickArguments = [
"click", "--coords", "100,200", "--pid", "12345", "--global-coords",
]
@Test
func `Process output gate serializes concurrent async regions`() async throws {
let gate = InProcessRunGate()
let probe = ConcurrentRegionProbe()
try await withThrowingTaskGroup(of: Void.self) { group in
for _ in 0..<8 {
group.addTask {
try await gate.run {
await probe.enter()
try await Task.sleep(for: .milliseconds(5))
await probe.leave()
}
}
}
try await group.waitForAll()
}
#expect(await probe.maximumActiveRegions == 1)
}
@Test
func `Cancelled output gate waiter does not run or block its successor`() async throws {
let gate = InProcessRunGate()
let holderStarted = AsyncTestLatch()
let releaseHolder = AsyncTestLatch()
let probe = ConcurrentRegionProbe()
let holder = Task {
try await gate.run {
await holderStarted.open()
await releaseHolder.wait()
}
}
await holderStarted.wait()
let cancelledWaiter = Task {
try await gate.run {
await probe.markCancelledRegionRan()
}
}
cancelledWaiter.cancel()
let successor = Task {
try await gate.run { 42 }
}
await releaseHolder.open()
try await holder.value
await #expect(throws: CancellationError.self) {
try await cancelledWaiter.value
}
#expect(try await successor.value == 42)
#expect(await probe.cancelledRegionRan == false)
}
@Test
func `Deferred output preserves original terminal capabilities`() async throws {
let result = try await InProcessCommandRunner.withExclusiveProcessOutput {
var controllerFD: Int32 = -1
var terminalFD: Int32 = -1
var size = winsize()
size.ws_col = 132
size.ws_row = 43
guard openpty(&controllerFD, &terminalFD, nil, nil, &size) == 0 else {
throw POSIXError(POSIXErrorCode(rawValue: errno) ?? .EIO)
}
defer {
close(controllerFD)
close(terminalFD)
}
let originalStdout = dup(STDOUT_FILENO)
guard originalStdout != -1 else {
throw POSIXError(POSIXErrorCode(rawValue: errno) ?? .EIO)
}
_ = fflush(nil)
guard dup2(terminalFD, STDOUT_FILENO) != -1 else {
let code = POSIXErrorCode(rawValue: errno) ?? .EIO
close(originalStdout)
throw POSIXError(code)
}
defer {
_ = fflush(nil)
_ = dup2(originalStdout, STDOUT_FILENO)
close(originalStdout)
}
return try await DeferredCommandOutput.run(bufferingOutput: true) {
let rawDescriptorIsInteractive = isatty(STDOUT_FILENO) != 0
let capabilities = await MainActor.run {
let detected = TerminalDetector.detectCapabilities()
return (
detected.isInteractive,
detected.isPiped,
detected.width,
detected.height
)
}
return (
rawDescriptorIsInteractive,
capabilities.0,
capabilities.1,
capabilities.2,
capabilities.3
)
}
}
#expect(!result.0)
#expect(result.1)
#expect(!result.2)
#expect(result.3 == 132)
#expect(result.4 == 43)
}
@Test
func `JSON success is replayed with one warning when snapshot cleanup fails`() async throws {
let context = TestServicesFactory.makeAutomationTestContext()
context.snapshots.invalidationError = PeekabooError.operationError(
message: "invalidation unavailable"
)
let result = try await InProcessCommandRunner.run(
self.clickArguments + ["--json"],
services: context.services
)
#expect(result.exitStatus == 0)
#expect(result.stderr == "\(CommanderRuntimeExecutorMessage.snapshotInvalidationWarning)\n")
let output = try Self.parseJSONObject(result.stdout)
#expect(output["success"] as? Bool == true)
#expect(context.snapshots.invalidationCutoffs.count == 2)
#expect(context.snapshots.invalidationCutoffs.first == context.snapshots.invalidationCutoffs.last)
}
@Test
func `Text success is replayed with one warning when snapshot cleanup fails`() async throws {
let context = TestServicesFactory.makeAutomationTestContext()
context.snapshots.invalidationError = PeekabooError.operationError(
message: "invalidation unavailable"
)
let result = try await InProcessCommandRunner.run(
self.clickArguments,
services: context.services
)
#expect(result.exitStatus == 0)
#expect(result.stdout.contains("Click successful"))
#expect(!result.stderr.contains("invalidation unavailable"))
#expect(result.stderr == "\(CommanderRuntimeExecutorMessage.snapshotInvalidationWarning)\n")
#expect(!result.stderr.contains("Error:"))
#expect(context.snapshots.invalidationCutoffs.count == 2)
}
@Test
func `Successful command output is replayed after snapshot cleanup`() async throws {
let context = TestServicesFactory.makeAutomationTestContext()
let result = try await InProcessCommandRunner.run(
self.clickArguments + ["--json"],
services: context.services
)
#expect(result.exitStatus == 0)
#expect(result.stderr.isEmpty)
let output = try Self.parseJSONObject(result.stdout)
#expect(output["success"] as? Bool == true)
#expect(context.snapshots.invalidationCutoffs.count >= 1)
}
@Test
func `Failed selected-host catch-up blocks command before side effect`() async throws {
let context = TestServicesFactory.makeAutomationTestContext()
context.snapshots.effectiveImplicitLatestInvalidationWatermark = Date()
context.snapshots.invalidationError = PeekabooError.operationError(
message: "host watermark unavailable"
)
let result = try await InProcessCommandRunner.run(
self.clickArguments,
services: context.services
)
#expect(result.exitStatus == 1)
#expect(context.automation.clickCalls.isEmpty)
#expect(context.snapshots.invalidationCutoffs.count == 1)
#expect(result.combinedOutput.contains("requested command was not executed"))
#expect(result.combinedOutput.contains("retrying later is safe"))
}
@Test
func `Selected-host catch-up preserves cancellation`() async {
let context = TestServicesFactory.makeAutomationTestContext()
context.snapshots.effectiveImplicitLatestInvalidationWatermark = Date()
context.snapshots.invalidationError = CancellationError()
let runtime = CommandRuntime(
configuration: .init(verbose: false, jsonOutput: false, logLevel: nil),
services: context.services
)
await #expect(throws: CancellationError.self) {
try await CommanderRuntimeExecutor.catchUpSelectedHostIfNeeded(
using: runtime,
required: true
)
}
}
@Test
func `Pre-cancelled selected-host catch-up cannot reach command execution`() async {
let context = TestServicesFactory.makeAutomationTestContext()
context.snapshots.effectiveImplicitLatestInvalidationWatermark = Date()
let runtime = CommandRuntime(
configuration: .init(verbose: false, jsonOutput: false, logLevel: nil),
services: context.services
)
let ready = AsyncTestLatch()
let release = AsyncTestLatch()
let task = Task {
await ready.open()
await release.wait()
try await CommanderRuntimeExecutor.catchUpSelectedHostIfNeeded(
using: runtime,
required: true
)
}
await ready.wait()
task.cancel()
await release.open()
await #expect(throws: CancellationError.self) {
try await task.value
}
#expect(context.snapshots.invalidationCutoffs.isEmpty)
}
@Test
func `Command without snapshot dependency skips selected-host catch-up`() async throws {
let context = TestServicesFactory.makeAutomationTestContext()
context.snapshots.effectiveImplicitLatestInvalidationWatermark = Date()
context.snapshots.invalidationError = PeekabooError.operationError(
message: "must not be consulted"
)
let runtime = CommandRuntime(
configuration: .init(verbose: false, jsonOutput: false, logLevel: nil),
services: context.services
)
try await CommanderRuntimeExecutor.catchUpSelectedHostIfNeeded(
using: runtime,
required: false
)
#expect(context.snapshots.invalidationCutoffs.isEmpty)
}
@Test
func `Operation error output remains primary when snapshot cleanup also fails`() async throws {
let context = TestServicesFactory.makeAutomationTestContext()
context.automation.clickError = PeekabooError.operationError(
message: "synthetic click failure"
)
context.snapshots.invalidationError = PeekabooError.operationError(
message: "invalidation unavailable"
)
let result = try await InProcessCommandRunner.run(
self.clickArguments + ["--json"],
services: context.services
)
#expect(result.exitStatus == 1)
#expect(result.stderr.isEmpty)
let output = try Self.parseJSONObject(result.stdout)
#expect(output["success"] as? Bool == false)
let error = try #require(output["error"] as? [String: Any])
let message = try #require(error["message"] as? String)
#expect(message.contains("synthetic click failure"))
#expect(!message.contains("stale UI snapshots"))
#expect(context.snapshots.invalidationCutoffs.count == 2)
}
private static func parseJSONObject(_ output: String) throws -> [String: Any] {
let data = try #require(output.data(using: .utf8))
return try #require(JSONSerialization.jsonObject(with: data) as? [String: Any])
}
}
private actor ConcurrentRegionProbe {
private var activeRegions = 0
private(set) var maximumActiveRegions = 0
private(set) var cancelledRegionRan = false
func enter() {
self.activeRegions += 1
self.maximumActiveRegions = max(self.maximumActiveRegions, self.activeRegions)
}
func leave() {
self.activeRegions -= 1
}
func markCancelledRegionRan() {
self.cancelledRegionRan = true
}
}
private actor AsyncTestLatch {
private var isOpen = false
private var waiters: [CheckedContinuation<Void, Never>] = []
func wait() async {
guard !self.isOpen else { return }
await withCheckedContinuation { continuation in
self.waiters.append(continuation)
}
}
func open() {
guard !self.isOpen else { return }
self.isOpen = true
let waiters = self.waiters
self.waiters.removeAll()
for waiter in waiters {
waiter.resume()
}
}
}

View File

@ -120,7 +120,7 @@ struct DialogCommandTests {
)
let services = self.makeTestServices(dialogs: dialogService)
let (output, status) = try await self.runCommand(
let (output, status) = try await runCommand(
["dialog", "dismiss", "--force", "--json"],
services: services
)
@ -198,7 +198,7 @@ struct DialogCommandTests {
let dialogService = StubDialogService(elements: elements)
let services = self.makeTestServices(dialogs: dialogService)
let (output, status) = try await self.runCommand(
let (output, status) = try await runCommand(
["dialog", "list", "--json"],
services: services
)
@ -211,6 +211,37 @@ struct DialogCommandTests {
#expect(response.data.textFields.first?.placeholder == "File name")
}
@Test
func `untargeted dialog list leaves latest snapshot eligible`() async throws {
let elements = DialogElements(
dialogInfo: DialogInfo(
title: "Open",
role: "AXWindow",
subrole: "AXDialog",
isFileDialog: true,
bounds: .init(x: 0, y: 0, width: 400, height: 300)
),
buttons: [],
textFields: [],
staticTexts: []
)
let snapshots = StubSnapshotManager()
let latestSnapshotID = try await snapshots.createSnapshot()
let services = TestServicesFactory.makePeekabooServices(
dialogs: StubDialogService(elements: elements),
snapshots: snapshots
)
let (_, status) = try await runCommand(
["dialog", "list", "--json"],
services: services
)
#expect(status == 0)
#expect(snapshots.invalidationCutoffs.isEmpty)
#expect(await snapshots.getMostRecentSnapshot() == latestSnapshotID)
}
@Test
func `dialog click emits JSON success when stub succeeds`() async throws {
let dialogService = await MainActor.run { StubDialogService() }
@ -235,7 +266,7 @@ struct DialogCommandTests {
)
let services = self.makeTestServices(dialogs: dialogService)
let (output, status) = try await self.runCommand(
let (output, status) = try await runCommand(
["dialog", "click", "--button", "New Document", "--json"],
services: services
)

View File

@ -0,0 +1,41 @@
import PeekabooCore
import Testing
@testable import PeekabooCLI
@Suite(.tags(.safe))
@MainActor
struct FocusCancellationTests {
@Test
func `Cancellation during fallback activation propagates after the service returns`() async {
let bundleIdentifier = "com.example.focus-cancellation"
let application = ServiceApplicationInfo(
processIdentifier: 4242,
bundleIdentifier: bundleIdentifier,
name: "Focus Cancellation"
)
let applications = StubApplicationService(applications: [application])
applications.activateApplicationHandler = { _ in
withUnsafeCurrentTask { $0?.cancel() }
}
let services = TestServicesFactory.makePeekabooServices(applications: applications)
let options = FocusOptions(
autoFocus: true,
focusTimeout: nil,
focusRetryCount: nil,
spaceSwitch: false,
bringToCurrentSpace: false
)
let task = Task { @MainActor in
try await ensureFocused(
applicationName: bundleIdentifier,
options: options,
services: services
)
}
await #expect(throws: CancellationError.self) {
try await task.value
}
#expect(applications.activateCalls == [bundleIdentifier])
}
}

View File

@ -35,12 +35,12 @@ struct ListCommandCLIHarnessTests {
windowCount: 1
),
]
let context = await self.makeContext(applications: applications)
let context = await makeContext(applications: applications)
let result = try await self.runList(arguments: ["list", "apps", "--json"], services: context.services)
let result = try await runList(arguments: ["list", "apps", "--json"], services: context.services)
#expect(result.exitStatus == 0)
let data = try #require(self.output(from: result).data(using: .utf8))
let data = try #require(output(from: result).data(using: .utf8))
let payload = try JSONDecoder().decode(CodableJSONResponse<ServiceApplicationListData>.self, from: data)
#expect(payload.success == true)
#expect(payload.data.applications.count == 2)
@ -58,11 +58,11 @@ struct ListCommandCLIHarnessTests {
windowCount: 4
),
]
let context = await self.makeContext(applications: applications)
let context = await makeContext(applications: applications)
let result = try await self.runList(arguments: ["list", "apps"], services: context.services)
let result = try await runList(arguments: ["list", "apps"], services: context.services)
#expect(result.exitStatus == 0)
let output = self.output(from: result)
let output = output(from: result)
#expect(output.contains("Viewer"))
#expect(output.contains("PID"))
}
@ -94,9 +94,9 @@ struct ListCommandCLIHarnessTests {
let applicationService = await MainActor.run {
StubApplicationService(applications: applications, windowsByApp: [appName: windows])
}
let context = await self.makeContext(applicationService: applicationService)
let context = await makeContext(applicationService: applicationService)
let result = try await self.runList(
let result = try await runList(
arguments: [
"list", "windows",
"--app", appName,
@ -107,14 +107,14 @@ struct ListCommandCLIHarnessTests {
)
#expect(result.exitStatus == 0)
let output = self.output(from: result)
let output = output(from: result)
#expect(output.contains("\"windowID\""))
#expect(output.contains("\"bounds\""))
#expect(output.contains("\"spaceID\""))
}
@Test
func `list apps fails when screen recording permission missing`() async throws {
func `list apps works when screen recording permission is missing`() async throws {
let applications = [
ServiceApplicationInfo(
processIdentifier: 101,
@ -127,11 +127,11 @@ struct ListCommandCLIHarnessTests {
let screenCapture = await MainActor.run {
StubScreenCaptureService(permissionGranted: false)
}
let context = await self.makeContext(applications: applications, screenCapture: screenCapture)
let context = await makeContext(applications: applications, screenCapture: screenCapture)
let result = try await self.runList(arguments: ["list", "apps"], services: context.services)
#expect(result.exitStatus != 0)
#expect(self.output(from: result).contains("Screen recording permission"))
let result = try await runList(arguments: ["list", "apps"], services: context.services)
#expect(result.exitStatus == 0)
#expect(self.output(from: result).contains("AlphaApp"))
}
// MARK: - Helpers

View File

@ -9,7 +9,7 @@ struct PermissionsCommandTests {
func `permissions command metadata describes current command`() {
#expect(PermissionsCommand.commandDescription.commandName == "permissions")
#expect(PermissionsCommand.commandDescription.abstract == "Check Peekaboo permissions")
#expect(PermissionsCommand.commandDescription.subcommands.count == 3)
#expect(PermissionsCommand.commandDescription.subcommands.count == 4)
}
@Test
@ -27,6 +27,14 @@ struct PermissionsCommandTests {
#expect(command.noRemote == true)
}
@Test
func `permissions request screen recording command binds`() throws {
_ = try CommanderCLIBinder.instantiateCommand(
ofType: PermissionsCommand.RequestScreenRecordingSubcommand.self,
parsedValues: ParsedValues(positional: [], options: [:], flags: [])
)
}
@Test
func `permissions status all sources emits JSON with local source`() async throws {
let automation = StubAutomationService()

Some files were not shown because too many files have changed in this diff Show More