Labs showed models that click, type, and navigate UIs like humans, extending automation without APIs.
Anthropic, OpenAI, and Google showed computer-use / browser agents that can manipulate GUIs. Useful without APIs; risky via errors and security.
Enterprises limited domains and required confirmations. A “neural RPA” category emerged.
Computer use closes the loop with multimodal vision: seeing the screen means acting on legacy software.
Enterprises deployed agents with real tools—CRM, code, browser—under tracing, permissions, and continuous eval.
Full-duplex realtime audio APIs enabled low-latency call centers and spoken assistants with tool use.
Open-source projects chained LLMs with tools and memory for multi-step goals—high hype, mixed reliability.