Anthropic’s latest development reportedly allows its Claude AI assistant to control PCs just like a human would. The feature, called “Use Computer,” entered public beta today. Using the computer, Claude can be instructed to perform tasks such as “looking at the screen, moving the cursor, clicking buttons, and entering text,” according to the company’s announcement.
We built an API that allows Claude to understand and interact with computer interfaces.
This API allows Claude to translate prompts into computer commands. Developers can use it to automate repetitive tasks, conduct testing and QA, and perform free-form research. pic.twitter.com/eK0UCGEozm
— Anthropic (@AnthropicAI) October 22, 2024
In theory, this could make AI even more useful in automating repetitive computer tasks. However, in a second blog post focused on computer usage, Anthropic admits that this application of its AI models is still in the early stages of development and, in other words, very buggy. During an internal test, Claude stopped in the middle of an assigned coding task and began opening an image of Yellowstone National Park, the company said. While this is creepy as human behavior (who wouldn’t want to take a break during the workday to gaze at the beauty of nature?), even the best AI models can have errors. It also serves as a reminder.
In addition to revealing the use of the computer, Anthropic has also released an upgraded version of its Claude 3.5 Sonnet model alongside a new model called the Claude 3.5 Haiku that will be released in late October. In August, Anthropic joined OpenAI and agreed to share its work with the American Institute for AI Safety.
