AI agents aren’t just indexing your content anymore. With tools like OpenAI’s Operator, Anthropic’s Computer Use API, and BrowserBase’s Open Operator, these agents can navigate the web, mimic real users, and even take actions at scale. The challenge? Knowing whether they’re enhancing your user experience—or opening the door to abuse.

In some scenarios, apps might encourage agent use if it improves usability and adoption, but in other cases, it could present unacceptable risks for application developers or be used as a method for malicious attacks (e.g. credential stuffing or fake account creation).

In either scenario, observability is paramount. Applications need to know what traffic is on their site (is this a human? A bot? A good bot or a bad one?) in order to make intelligent decisions about how to shape traffic and enforce desired usage patterns. AI agents add an additional wrinkle as users are already sharing their credentials with tools like Operator, meaning even a well-intentioned agent creates potential risk for these applications and their users.

The key question is: Can you detect AI agent traffic on your application today?

We tested multiple AI agent toolkits across high-traffic consumer sites, and the results were clear—legacy detection techniques (CAPTCHAs, IP blocking, user-agent filtering) are largely ineffective. Here’s what we found.