Peekaboo/docs/service-api-reference.md

summary	read_when
Review PeekabooCore Service API Reference guidance	planning work related to peekaboocore service api reference debugging or extending features described here

PeekabooCore Service API Reference

This document provides a comprehensive reference for all services available in PeekabooCore. These services are used by both the CLI and Mac app to provide consistent functionality with optimal performance.

Table of Contents

1. ScreenCaptureService
2. ApplicationService
3. WindowManagementService
4. UIAutomationService
5. MenuService
6. DockService
7. ProcessService
8. DialogService
9. FileService
10. SnapshotManager
11. ConfigurationManager
12. EventGenerator

---

ScreenCaptureService

Handles all screen capture operations including windows, screens, and areas.

Methods

captureWindow(element:savePath:options:)

Captures a screenshot of a specific window.

func captureWindow(
    element: Element,
    savePath: String,
    options: CaptureOptions = .init()
) async throws -> CaptureResult

Parameters:

• element: The window element to capture
• savePath: Path where the image should be saved
• options: Capture options (format, quality, etc.)

Returns: CaptureResult containing capture metadata

Example:

let result = try await service.captureWindow(
    element: windowElement,
    savePath: "~/Desktop/window.png"
)

captureScreen(displayIndex:savePath:options:)

Captures a full screen or specific display.

func captureScreen(
    displayIndex: Int = 0,
    savePath: String,
    options: CaptureOptions = .init()
) async throws -> CaptureResult

captureArea(rect:savePath:options:)

Captures a specific rectangular area of the screen.

func captureArea(
    rect: CGRect,
    savePath: String,
    options: CaptureOptions = .init()
) async throws -> CaptureResult

captureAllWindows(for:savePath:options:)

Captures all windows for a specific application.

func captureAllWindows(
    for app: RunningApplication,
    savePath: String,
    options: CaptureOptions = .init()
) async throws -> [CaptureResult]

---

ApplicationService

Manages application lifecycle and information.

Methods

listApplications()

Lists all running applications.

func listApplications() -> [RunningApplication]

Returns: Array of running applications with metadata

findApplication(identifier:)

Finds an application by name or bundle ID.

func findApplication(identifier: String) throws -> RunningApplication

Parameters:

• identifier: App name or bundle identifier

Throws: ApplicationError.notFound if not found

launchApplication(identifier:)

Launches an application.

func launchApplication(identifier: String) async throws -> RunningApplication

quitApplication(_:force:)

Quits an application gracefully or forcefully.

func quitApplication(_ app: RunningApplication, force: Bool = false) async throws

hideApplication(_:)

Hides an application.

func hideApplication(_ app: RunningApplication) async throws

unhideApplication(_:)

Shows a hidden application.

func unhideApplication(_ app: RunningApplication) async throws

---

WindowManagementService

Handles window manipulation and queries.

Methods

listWindows(for:)

Lists all windows for an application.

func listWindows(for app: RunningApplication) throws -> [WindowInfo]

findWindow(app:title:index:)

Finds a specific window by title or index.

func findWindow(
    app: RunningApplication,
    title: String? = nil,
    index: Int? = nil
) throws -> Element

closeWindow(_:)

Closes a window.

func closeWindow(_ window: Element) async throws

minimizeWindow(_:)

Minimizes a window.

func minimizeWindow(_ window: Element) async throws

maximizeWindow(_:)

Maximizes a window.

func maximizeWindow(_ window: Element) async throws

moveWindow(_:to:)

Moves a window to a specific position.

func moveWindow(_ window: Element, to position: CGPoint) async throws

resizeWindow(_:to:)

Resizes a window.

func resizeWindow(_ window: Element, to size: CGSize) async throws

focusWindow(_:)

Brings a window to the front and focuses it.

func focusWindow(_ window: Element) async throws

---

UIAutomationService

Provides UI element interaction and automation.

Methods

findElement(matching:in:timeout:)

Finds UI elements matching criteria.

func findElement(
    matching criteria: ElementCriteria,
    in container: Element? = nil,
    timeout: TimeInterval = 5.0
) async throws -> Element

clickElement(_:at:clickCount:)

Clicks on a UI element.

func clickElement(
    _ element: Element,
    at point: CGPoint? = nil,
    clickCount: Int = 1
) async throws

typeText(_:in:clearFirst:)

Types text into an element.

func typeText(
    _ text: String,
    in element: Element? = nil,
    clearFirst: Bool = false
) async throws

scrollElement(_:direction:amount:)

Scrolls within an element.

func scrollElement(
    _ element: Element,
    direction: ScrollDirection,
    amount: CGFloat
) async throws

dragElement(from:to:duration:)

Performs a drag operation.

func dragElement(
    from startPoint: CGPoint,
    to endPoint: CGPoint,
    duration: TimeInterval = 0.5
) async throws

swipeElement(_:direction:distance:)

Performs a swipe gesture.

func swipeElement(
    _ element: Element,
    direction: SwipeDirection,
    distance: CGFloat
) async throws

---

MenuService

Handles menu bar and context menu interactions.

Methods

clickMenuItem(app:menuPath:)

Clicks a menu item by path.

func clickMenuItem(
    app: RunningApplication,
    menuPath: [String]
) async throws

Example:

try await service.clickMenuItem(
    app: app,
    menuPath: ["File", "Save As..."]
)

listMenuItems(app:)

Lists all menu items for an application.

func listMenuItems(app: RunningApplication) throws -> [MenuItemInfo]

openContextMenu(at:)

Opens a context menu at a specific location.

func openContextMenu(at point: CGPoint) async throws

---

DockService

Manages Dock interactions.

Methods

listDockItems()

Lists all items in the Dock.

func listDockItems() throws -> [DockItem]

clickDockItem(identifier:)

Clicks a Dock item.

func clickDockItem(identifier: String) async throws

rightClickDockItem(identifier:)

Right-clicks a Dock item to show its menu.

func rightClickDockItem(identifier: String) async throws

---

ProcessService

Manages system processes and shell commands.

Methods

runCommand(_:arguments:environment:currentDirectory:)

Executes a shell command.

func runCommand(
    _ command: String,
    arguments: [String] = [],
    environment: [String: String]? = nil,
    currentDirectory: String? = nil
) async throws -> ProcessResult

killProcess(pid:signal:)

Terminates a process.

func killProcess(pid: Int32, signal: Int32 = SIGTERM) throws

checkProcessRunning(name:)

Checks if a process is running.

func checkProcessRunning(name: String) -> Bool

---

DialogService

Handles system dialogs and alerts.

Methods

findDialog(withTitle:timeout:)

Finds a dialog by title.

func findDialog(
    withTitle title: String? = nil,
    timeout: TimeInterval = 5.0
) async throws -> Element

clickDialogButton(_:in:)

Clicks a button in a dialog.

func clickDialogButton(
    _ buttonTitle: String,
    in dialog: Element
) async throws

dismissDialog(_:)

Dismisses a dialog using keyboard shortcuts.

func dismissDialog(_ dialog: Element) async throws

handleFileDialog(_:path:)

Handles file selection dialogs.

func handleFileDialog(
    _ dialog: Element,
    path: String
) async throws

---

FileService

Provides file system operations.

Methods

cleanFiles(at:matching:dryRun:)

Cleans files matching criteria.

func cleanFiles(
    at path: String,
    matching criteria: CleanCriteria,
    dryRun: Bool = false
) async throws -> CleanResult

listFiles(at:recursive:)

Lists files in a directory.

func listFiles(
    at path: String,
    recursive: Bool = false
) throws -> [FileInfo]

createDirectory(at:)

Creates a directory.

func createDirectory(at path: String) throws

---

SnapshotManager

Persists UI automation snapshots created by peekaboo see so follow-up commands (click, type, scroll, …) can resolve element IDs and reliably refocus the same window.

Snapshots are stored under ~/.peekaboo/snapshots/<snapshot-id>/ and typically include:

• snapshot.json (the serialized UIAutomationSnapshot, including uiMap + window metadata)
• raw.png (the stored screenshot copied into the snapshot)
• annotated.png (optional, when annotations are generated)

Methods

createSnapshot()

Creates a new empty snapshot and returns its ID.

func createSnapshot() async throws -> String

getMostRecentSnapshot()

Returns the most recent valid snapshot ID (if any).

func getMostRecentSnapshot() async -> String?

storeScreenshot(snapshotId:screenshotPath:applicationName:windowTitle:windowBounds:)

Stores a raw screenshot in the snapshot directory and records basic window metadata.

func storeScreenshot(
    snapshotId: String,
    screenshotPath: String,
    applicationName: String?,
    windowTitle: String?,
    windowBounds: CGRect?
) async throws

storeAnnotatedScreenshot(snapshotId:annotatedScreenshotPath:)

Stores an annotated screenshot as annotated.png inside the snapshot directory (optional companion to raw.png).

func storeAnnotatedScreenshot(
    snapshotId: String,
    annotatedScreenshotPath: String
) async throws

storeDetectionResult(snapshotId:result:)

Persists detected UI elements into snapshot.json.

func storeDetectionResult(
    snapshotId: String,
    result: ElementDetectionResult
) async throws

getDetectionResult(snapshotId:)

Loads the persisted snapshot and returns a reconstructed ElementDetectionResult (if present).

func getDetectionResult(snapshotId: String) async throws -> ElementDetectionResult?

getElement(snapshotId:elementId:)

Fetches a single UIElement from the snapshot’s uiMap.

func getElement(snapshotId: String, elementId: String) async throws -> UIElement?

findElements(snapshotId:matching:)

Searches the snapshot’s uiMap for elements matching a query string.

func findElements(snapshotId: String, matching query: String) async throws -> [UIElement]

listSnapshots()

Returns metadata for all snapshot directories.

func listSnapshots() async throws -> [SnapshotInfo]

cleanSnapshot(snapshotId:)

Deletes a specific snapshot directory.

func cleanSnapshot(snapshotId: String) async throws

cleanAllSnapshots()

Deletes all snapshot directories and returns the number removed.

func cleanAllSnapshots() async throws -> Int

---

ConfigurationManager

Manages application configuration.

Properties

static let shared: ConfigurationManager
var currentConfiguration: Configuration { get }

Methods

loadConfiguration()

Loads configuration from disk.

func loadConfiguration() -> Configuration

saveConfiguration(_:)

Saves configuration to disk.

func saveConfiguration(_ config: Configuration) throws

resetToDefaults()

Resets configuration to defaults.

func resetToDefaults() throws

---

EventGenerator

Low-level event generation for automation.

Methods

createMouseEvent(type:at:)

Creates mouse events.

static func createMouseEvent(
    type: CGEventType,
    at point: CGPoint
) -> CGEvent?

createKeyboardEvent(keyCode:down:)

Creates keyboard events.

static func createKeyboardEvent(
    keyCode: UInt16,
    down: Bool
) -> CGEvent?

typeText(_:)

Types text using keyboard events.

static func typeText(_ text: String) async throws

---

Error Handling

All services throw typed errors for better error handling:

enum ScreenCaptureError: Error {
    case permissionDenied
    case invalidWindow
    case captureF ailed
    case fileWriteError(Error)
}

enum ApplicationError: Error {
    case notFound(String)
    case ambiguousIdentifier([RunningApplication])
    case launchFailed(Error)
}

enum UIAutomationError: Error {
    case elementNotFound
    case interactionFailed
    case timeout
}

Usage Example

Here's a complete example showing how to use multiple services together:

import PeekabooCore

// Initialize services
let appService = ApplicationService()
let windowService = WindowManagementService()
let captureService = ScreenCaptureService()
let uiService = UIAutomationService()

// Find and focus Safari
let safari = try appService.findApplication(identifier: "Safari")
let windows = try windowService.listWindows(for: safari)
if let firstWindow = windows.first {
    try await windowService.focusWindow(firstWindow.element)
}

// Capture the window
let result = try await captureService.captureWindow(
    element: firstWindow.element,
    savePath: "~/Desktop/safari.png"
)

// Click on a button
let criteria = ElementCriteria(role: .button, title: "Reload")
let button = try await uiService.findElement(matching: criteria)
try await uiService.clickElement(button)

Performance Notes

• Services are designed to be lightweight and efficient
• They eliminate process spawning overhead compared to CLI invocations
• All async operations use Swift's native concurrency
• Services maintain minimal state for optimal performance
• The Mac app sees 100x+ performance improvement using services directly

Thread Safety

• All services are thread-safe and can be used from any thread
• UI operations are automatically dispatched to the main thread
• Async methods use Swift's concurrency model for safety
• Shared state is protected with appropriate synchronization