computer-use-agents

1# Computer Use Agents
2 
3## Patterns
4 
5### Perception-Reasoning-Action Loop
6 
7The fundamental architecture of computer use agents: observe screen,
8reason about next action, execute action, repeat. This loop integrates
9vision models with action execution through an iterative pipeline.
10 
11Key components:
121. PERCEPTION: Screenshot captures current screen state
132. REASONING: Vision-language model analyzes and plans
143. ACTION: Execute mouse/keyboard operations
154. FEEDBACK: Observe result, continue or correct
16 
17Critical insight: Vision agents are completely still during "thinking"
18phase (1-5 seconds), creating a detectable pause pattern.
19 
20 
21**When to use**: ['Building any computer use agent from scratch', 'Integrating vision models with desktop control', 'Understanding agent behavior patterns']
22 
23```python
24from anthropic import Anthropic
25from PIL import Image
26import base64
27import pyautogui
28import time
29 
30class ComputerUseAgent:
31    """
32    Perception-Reasoning-Action loop implementation.
33    Based on Anthropic Computer Use patterns.
34    """
35 
36    def __init__(self, client: Anthropic, model: str = "claude-sonnet-4-20250514"):
37        self.client = client
38        self.model = model
39        self.max_steps = 50  # Prevent runaway loops
40        self.action_delay = 0.5  # Seconds between actions
41 
42    def capture_screenshot(self) -> str:
43        """Capture screen and return base64 encoded image."""
44        screenshot = pyautogui.screenshot()
45…