- Default AI to GPT-5 across PeekabooCore; include gpt‑5 mini/nano in lists
- Hydrate Tachikoma with OPENAI/ANTHROPIC keys and Ollama URL from config/env
- Wire CLI SeeCommand to real PeekabooAIService (removed placeholder)
- Make MCP Analyze tool read providers from config; use GPT‑5 for OpenAI
- Update Image tool text and model reporting to GPT‑5
- Fix OpenAI chat content encoding (type/text/image_url) for multimodal
- Prefer providers default to `openai/gpt-5` first
This re-enables OCR/vision analysis out of the box using env or config credentials.
BREAKING CHANGE: pgrun command renamed to polter
- Update wrapper script to use global polter command
- Simplify wrapper to 3 lines by removing path detection
- Update all documentation references from pgrun to polter
- Update examples and commands throughout CLAUDE.md
- Maintain PEEKABOO_WAIT_DEBUG environment variable compatibility
Users should now install polter globally:
npm install -g @steipete/poltergeist
Then use: polter peekaboo [args...]
Or create alias: alias pb='polter peekaboo'
- Fix Poltergeist config: rename target from "peekaboo-cli" to "peekaboo"
- Simplify wrapper from 36 lines to 5 lines (86% reduction)
- Remove symlink workaround and directory context switching
- Eliminate hardcoded paths and complex logic
- Maintain PEEKABOO_WAIT_DEBUG environment variable support
- Remove obsolete peekaboo-cli directory
The target name now matches the actual Swift executable name,
eliminating the need for workarounds and making the configuration
more intuitive.
- Replace complex 229-line shell script with simple 36-line pgrun wrapper
- Use Poltergeist's pgrun for superior build management and diagnostics
- Maintain PEEKABOO_WAIT_DEBUG environment variable compatibility
- Create symlink to handle target name mismatch (peekaboo-cli -> peekaboo)
- Keep original script as backup (.original)
- Add .crush/ to .gitignore
This simplifies the wrapper while providing better build status detection,
graceful fallback when Poltergeist is not running, and clearer error messages.
The pgrun fallback ensures the wrapper never completely blocks workflows.
This is a complete rewrite of the Peekaboo MCP server in Swift, removing all TypeScript dependencies
and providing a native, high-performance implementation that integrates directly with PeekabooCore.
## Major Changes
### Architecture
- Removed entire TypeScript/Node.js server implementation (Server/ directory)
- Implemented native Swift MCP server using modelcontextprotocol/swift-sdk
- Direct integration with PeekabooCore services for ~10x performance improvement
- All operations now run on MainActor for thread safety with UI/AppKit APIs
### MCP Tools Implementation
- Implemented all 23 MCP tools in Swift with full feature parity
- Added comprehensive input validation and error handling
- Improved type safety with Swift's strong type system
- Better integration with macOS accessibility and UI automation APIs
### Key Improvements
- Performance: ~10x faster by eliminating CLI subprocess overhead
- Type Safety: Compile-time checking for all tool parameters
- Thread Safety: Proper @MainActor usage for UI operations
- Memory Efficiency: No more Node.js runtime overhead
- Better Error Messages: More descriptive errors for debugging
### Testing
- Added comprehensive test suite with 200+ tests
- Unit tests for all MCP tools and components
- Integration tests for server functionality
- Mock implementations for testing without side effects
### Fixes Included
- Fixed threading violations by ensuring UI operations run on main thread
- Fixed API errors with proper media type detection for images
- Fixed UI element detection using correct property mappings
- Added Sendable conformance for Swift concurrency compliance
### Installation
- New installation script for Claude Desktop integration
- Simplified deployment with single binary
- No npm dependencies or Node.js runtime required
## Breaking Changes
- Server/ directory and all TypeScript code removed
- npm scripts updated to reflect Swift-only build
- MCP server now starts with 'peekaboo mcp serve' command
Co-authored-by: Previous Claude session <claude-3-5-sonnet@anthropic.com>
- Add new build-swift-arm.sh script that builds only for arm64 architecture
- Change npm run build:swift to use ARM-only builds (2s vs 2min)
- Add npm run build:swift:all for universal builds used in releases
- Update Server/package.json to use universal builds for prepublishOnly
- Update documentation to reflect new build commands
This significantly improves development iteration speed while preserving
universal binary support for production releases via npm publish.
- Create AudioInputService for recording and transcription via Whisper API
- Extend MessageContent with audio case and AudioContent struct
- Add audio handling to all AI providers (Anthropic, OpenAI, Grok, Ollama)
- Integrate audio flags (--audio, --audio-file) into CLI agent command
- Add comprehensive tests for audio infrastructure
- Update build script to copy binary to project root for Poltergeist
Each provider converts audio content to transcript with duration metadata.
Audio recording uses 16kHz mono WAV format optimized for AI transcription.
- Added @MainActor to all UI service classes: ApplicationService, MenuService, DialogService, DockService, UIAutomationService, WindowManagementService, ScreenCaptureService, PermissionsService, ProcessService, PeekabooAgentService
- Added @MainActor to all UI/AX protocol definitions to ensure compile-time thread safety
- Removed all unnecessary MainActor.run blocks from @MainActor classes (100+ instances removed)
- Changed ProcessService from actor to @MainActor class for proper UI thread execution
- Kept ModelProvider and AI model implementations off MainActor for network operations
- Fixed variable naming issues in ApplicationService (hiddenCount/unhiddenCount)
This ensures all UI and accessibility API calls happen on the main thread as required by macOS, preventing crashes and race conditions while simplifying the codebase.
- Mark executeTools method with @MainActor to ensure all AX operations run on main thread
- This prevents segfaults when accessing NSWorkspace.shared.runningApplications
- Increase peekaboo-wait.sh timeout from 3 to 5 minutes for longer builds
The crash was happening because even though individual services were @MainActor,
the tool execution pipeline itself could run on background threads created by
the actor runtime. This ensures the entire tool execution chain stays on the
main thread where all Accessibility and AppKit APIs must run.
Changed connectionLock from a stored property to a lazy property to avoid
initialization timing issues in @MainActor classes. This prevents the
crash that occurred when NSLock() was initialized during class instantiation.
- Fix excessive newlines between tool commands and text output
- Add rich contextual information to all tool outputs:
- Show which specific menu items were clicked (not just 'menu item')
- Display which app was actually launched (not just 'launched app')
- Include frontmost app context for click, type, and hotkey actions
- Show actual coordinates clicked (properly parse wrapped values)
- Display keyboard shortcuts pressed with modifiers
- Enhanced tool summaries for better user feedback:
- menu_click: Shows full menu path clicked
- launch_app: Shows app name launched
- type: Shows text typed and target app
- click: Shows element/coordinates and target app
- hotkey: Shows keys pressed and target app
- scroll: Shows direction and amount
- see: Shows capture target and resolution
- And many more tools with contextual info
- Fix coordinate display to handle wrapped value format
- Add metadata capture for frontmost app in UI automation tools
- Improve argument display in compact tool summaries
- Add menu bar synchronization showing current tool with animated SF Symbol icons
- Display token usage (prompt/completion/total) in menu bar header with hover details
- Move time formatter from CLI to PeekabooCore for consistent '1m 30s' format across app
- Implement native SwiftUI Markdown rendering for assistant messages
- Fix tool execution UI: remove green checkmark, fix double-tap expansion, add live duration
- Add AI-powered session title generation (2-4 word summaries instead of 'New Session')
- Remove unnecessary macOS 14 availability checks throughout codebase
- Created ToolFormatter utility for consistent formatting logic
- Updated ToolExecutionRow to show tool-specific summaries
- Added three-level expansion (collapsed/summary/full)
- Implemented symbol replacements for keyboard shortcuts (⌘⇧⌥⌃)
- Added duration formatting with ⌖ symbol
- Enhanced visual presentation with proper tool icons and status indicators
- Modified AgentEvent.completed to include usage information
- Updated PeekabooAgentService to emit token counts in completion events
- Enhanced Mac app to display token usage in task completion summary
- Shows total tokens and breakdown (input/output) when available
- Format: '✅ Task completed in Xs with Y tool calls • 🤖 Z tokens (A in, B out)'
- Rename vtlog.sh to pblog.sh throughout the project
- Consolidate logging documentation into docs/logging-profiles/README.md
- Add configuration profile for enabling private data logging
- Update all references from vtlog to pblog
- Add comprehensive guide for dealing with macOS log privacy redaction
The pblog (Peekaboo Log) name better represents the tool's purpose
and avoids confusion with other tools.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit unifies the codebase under the new boo.peekaboo bundle ID namespace
and improves logging capabilities across all Peekaboo components.
Changes:
- Replace all com.steipete bundle IDs with boo.peekaboo throughout the codebase
- Fix typo in OverlayManager subsystem (boo.pekaboo.inspector → boo.peekaboo.app)
- Enhance vtlog.sh to monitor logs from ALL Peekaboo subsystems
- Add subsystem filtering and proper documentation for vtlog
- Update all Logger instances to use the new bundle ID namespace
- Fix dialog detection in ElementDetectionService for file/save dialogs
- Create comprehensive documentation for vtlog usage
The new bundle ID structure:
- boo.peekaboo.core - Core services
- boo.peekaboo.inspector - Inspector app
- boo.peekaboo.playground - Playground app
- boo.peekaboo.app - Mac app
- boo.peekaboo - Mac app CLI
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Integrated VibeTunnel for dynamic terminal title updates during agent execution
- Terminal titles show current tool being executed (e.g., "click: Submit button")
- Shows task completion status: "Completed: [task]" or "Error: [task]"
- Created global Claude configuration at ~/.claude/CLAUDE.md for all sessions
- Disabled Poltergeist build start notifications (only show completion)
- Added test script to demonstrate VibeTunnel integration
This improves visibility across multiple Claude Code sessions and reduces
notification noise during development.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add robust build failure detection in peekaboo-wait.sh wrapper script
- Script now detects build failures and prompts Claude to fix them automatically
- Show recent build logs and exit with code 1 on failures
- Disable build failure notifications in Poltergeist (success notifications remain)
- Fix concurrency issue in AgentCommand by adding @MainActor to GhostAnimator
- Update CLAUDE.md to document the enhanced build failure detection
- Set o3 reasoning effort to "high" for maximum capability
This allows Claude to automatically detect and fix build errors when using the wrapper script.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add explicit Task Completion Requirements section
- Emphasize literal instruction following (e.g., 'say' command)
- Require full action completion (send email, not just draft)
- Add verification steps for all actions
- Add Tool Selection Guidelines
- Clarify 'command not found' is definitive
- No retry attempts for missing tools
- Immediate fallback to alternatives required
- Add UI Automation Best Practices
- Complete full user journeys (Draft → Send)
- Verify UI state changes after actions
- Handle multi-step workflows properly
- Add Shell Command Best Practices
- Clear guidance for text-to-speech requests
- Binary command availability handling
- Proper escaping and quoting rules
These improvements address issues where the agent would:
- Skip 'say' commands when requested
- Create email drafts without sending
- Retry unavailable commands multiple times
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
When Poltergeist cancels an in-progress build due to detecting new file changes,
it now exits with code 0 instead of 1. This prevents the cleanup function from
showing a failure notification for what is normal behavior. Also added a "Build
Started" notification to provide better feedback about build status.
Users will now see:
- "Build Started" notification when Poltergeist begins building
- "Build Succeeded" notification with build time
- "Build Failed" notification only for real failures
- No notification for cancelled builds (normal operation)
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Include short commit hash in success/failure notifications
- Format: 'Build completed (Xs) - abc1234' for success
- Format: 'Build failed (exit X) - abc1234' for failure
- Also add Git hash to log messages for better tracking
This helps identify which commit was built, especially useful
during rapid development when multiple builds are triggered.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
Added native macOS notifications when builds complete:
- Success notifications with Glass sound and build time
- Failure notifications with Basso sound and error details
- Can be disabled with POLTERGEIST_NOTIFICATIONS=false
Also updated documentation to explain the notification feature.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added explicit instructions in the agent system prompt to use the macOS
`say` command for text-to-speech when users request to "say" something.
This prevents confusion and ensures the agent properly executes speech
output requests like "say YOWZA YOWZA BO-BOWZA".
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Changed MAX_WAIT from 30s to 180s (3 minutes) to accommodate real Swift build times
- Updated progress messages to show every 10s instead of 5s
- Added remaining time in progress updates
- Improved timeout message to suggest checking logs
- Updated CLAUDE.md to reflect the 3-minute timeout
Swift builds, especially universal builds, can take 1-2 minutes or more, so the previous 30-second timeout was too short and would often result in running stale binaries.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
CLAUDE.md improvements:
- Added clear explanation of what Poltergeist is
- Critical instructions for AI agents to NEVER manually rebuild
- Emphasized ALWAYS using the wrapper script
- Explained the efficiency benefits
- Deprecated manual build commands section
Poltergeist handler improvements:
- Added build cancellation when newer changes detected
- Kills outdated builds to start fresh ones immediately
- Process tree killing to ensure clean cancellation
- Cancel flag mechanism for graceful shutdown
- Improved logging for build cancellations
OpenAI o3 model refinements:
- Removed reasoning_summary parameter (not needed)
- Cleaned up parameter handling
- Proper null handling for temperature
This ensures agents use Poltergeist efficiently and builds are always for the latest code changes.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Check for any Swift build processes before starting new build
- Exit early if builds are already running to avoid cascading builds
- Fixes issue where multiple file changes could trigger many parallel builds
This prevents the scenario where Poltergeist could spawn dozens of concurrent builds.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Enhanced system prompt to explicitly request thinking out loud
- Added instructions for models to share their reasoning process
- Increased temperature for o3 model to encourage more verbose output
- Set maxTokens to 4096 to ensure room for explanations
This should help make o3's thought process visible to users.
- Enhanced stop_watcher to properly remove both trigger and watch
- Added SwiftPM conflict detection to prevent concurrent build issues
- Improved status messages with success/warning indicators
- Fixed issue where Poltergeist wouldn't properly restart after stopping
The watcher now handles edge cases better and provides clearer feedback about its state.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Enhanced system prompt with explicit screenshot-after-launch guidance
- Added comprehensive AppleScript quoting rules and examples
- Strengthened task completion requirements with checklist
- Added dialog handling best practices
- Introduced new 'see' tool that combines screenshot + UI detection
- Updated CLI agent command to support 'see' tool with proper emoji
- Fixed compilation issues with DetectedElements
This improves agent's ability to handle app dialogs, complete all task
requirements (including specific output phrases), and use proper
AppleScript syntax.
* feat: Add build staleness detection for debug CLI
- Add debug-only staleness check using git config 'peekaboo.check-build-staleness'
- CLI will exit with error if current git commit differs from build commit
- Helps prevent Claude Code from using outdated binaries after source changes
* feat: Enhance build staleness detection with file modification checks
- Add buildDate timestamp to Version.swift generation
- Create separate BuildStalenessChecker.swift file
- Add comprehensive file modification time checking using git status
- Parse git status --porcelain=1 output to identify modified files
- Compare file modification times against build timestamp
- Provide clear error messages for both commit and file staleness
- Support clean/comprehensive staleness detection for Claude Code workflows
* docs: Add Debug Build Staleness Detection section to README
- Document how to enable/disable staleness checking via git config
- Explain both git commit and file modification staleness detection
- Provide clear examples and benefits
- Highlight usefulness for AI-assisted development workflows
* docs: Simplify staleness detection README section
Replace verbose documentation with single concise paragraph as requested
* fix: Remove test comments from main.swift
Clean up debugging comments that were accidentally left in the code
* Update Apps/CLI/Sources/peekaboo/main.swift
- Modified build scripts to extract git commit, date, branch, and dirty state
- Enhanced Version.swift with fullVersion property containing git metadata
- Updated help menu to display version with git info (branch/commit, date)
- Created build-swift-debug.sh for quick debug builds with version info
- Now shows version like: Peekaboo 3.0.0-beta.1 (spec-v3/6c4adea, 2025-07-26 04:00:23 +0200)
This makes it easy to verify if running the latest version based on git commit.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Created SwiftUI test app at Playground/ for testing all Peekaboo automation features
- Includes comprehensive UI elements: clicks, text input, controls, gestures, drag/drop, keyboard
- Added OSLog integration with categorized logging (Click, Text, Menu, Window, etc.)
- Created playground-log.sh utility inspired by vtlog for easy log viewing
- Features: color-coded output, category filtering, search, JSON export, time ranges
- Added wrapper script at scripts/playground-log.sh for project root access
- Updated CLAUDE.md with comprehensive Playground documentation
- All UI elements have accessibility identifiers for automation testing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Move core libraries to Core/ directory (PeekabooCore, AXorcist)
- Move applications to Apps/ directory (Mac, CLI)
- Move TypeScript server to Server/ directory
- Move scripts to Scripts/ directory
- Archive deprecated PeekabooInspector (now integrated into Mac app)
- Update all build configurations and paths
- Update CI/CD workflows for new structure
- Fix build scripts to use new paths
This reorganization provides:
- Clear separation between core libraries, apps, and server
- Flattened Mac app structure (removed double nesting)
- Consistent naming conventions
- Better code sharing through PeekabooCore
- Easier maintenance and development
- Adopted vtlog script from VibeTunnel project for PeekabooInspector
- Configured to work with com.steipete.PeekabooInspector subsystem
- Added comprehensive documentation to CLAUDE.md
- Provides easy access to macOS unified logging output with filtering options
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Homebrew expects the version string to include the "Peekaboo" prefix.
This commit:
- Reverts the version generation to include "Peekaboo" prefix
- Updates all version tests to expect the prefix format
- Ensures compatibility with Homebrew's version requirements
All tests now pass with the expected format: "Peekaboo X.Y.Z"
The Swift tests expect Version.current to contain only the semantic version
number (e.g. "2.0.3") without the "Peekaboo" prefix. This was causing the
version format tests to fail in CI.
- Updated build-swift-universal.sh to inject only the version number
- Regenerated Version.swift with the correct format
- All version tests now pass
- Updated CLI to output "Peekaboo X.X.X" instead of just version number
- Fixes Homebrew formula test that expects "Peekaboo" in --version output
- No functional changes, just formatting improvement
- Actually fixed LC_UUID load command generation (v2.0.1 fix was incomplete)
- Binary now includes LC_UUID for both x86_64 and arm64 architectures
- Verified with otool that LC_UUID is present in the universal binary
- This ensures proper dyld loading on macOS 26+
- Fixed LC_UUID load command preservation during binary stripping
- Updated strip command to use -u flag to retain UUID for macOS 26+ compatibility
- Ensures proper debugging and crash reporting support on newer macOS versions
- Generate enum instead of struct in build script
- Prevents formatting conflicts during release process
- Maintains alignment with SwiftFormat enumNamespaces rule
- Add npm run inspector script for MCP inspector tool
- Synchronize Swift CLI version with package.json (1.0.0-beta.9)
- Update macOS version requirement to v14 (Sonoma) for n-1 support
- Add Swift compiler warnings check in prepare-release script
- Convert tests/setup.ts from Jest to Vitest syntax
- Update server status tests to match new format
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
