Microsoft's Copilot learns to speak, see and think

In the future, there will be a test environment for the Copilot. Here, people can try out how the AI assistant performs in logical thinking. Copilot Vison is also moving in: It should then also be possible to see and process images –, for example of websites that you have opened. The Copilot is to become a friend, writes Mustafa Suleyman, CEO of Microsoft AI. A friend that adapts to the needs of humans.

Microsoft explains that AI is not yet a mature technology. That is why they want to make new functions available in a new test environment. "We want to fully utilize the capabilities of AI Copilot to provide the most useful and rich experience for users, in the safest way we can," the blog post states. Copilot Labs will be accessible to selected people who can test new features there. Copilot Pro users have apparently been selected, as they should see the offer as a glimpse into the "work-in-progress" projects. So instead of making beta versions generally available, there is now a closed area. Google also calls its test environment for AI applications Google Labs.

Copilot sees the screen and listens to speech

A reasoning function will be the first to be available in Copilot Labs, Microsoft writes further. Think Deeper is designed to help Copilot gain a better understanding of more complex tasks – from math to task management. Initially, however, the function will not only be limited to Pro subscribers, but also to the USA, Australia, Canada, New Zealand and the UK.

Subsequently, Copilot Vision will appear in the Labs. Language is limited, writes Microsoft. Copilot Vision can see what people see. This means at least what can be seen on the screen. You can also ask questions about it. And now also in spoken language.

The functions are opt-in, content is not saved in the preview and is not fed back into the training of the AI models behind it. For websites with paywalls and those with sensitive content, the eyes and ears do not yet work. However, there is a whitelist of websites where it does work on purpose.

Copilot as a partner

Mustafa Suleyman, Head of AI at Microsoft, who was previously at Google, has also published a letter in which he explains how Copilot should become a companion. "Copilot is there for you, in every corner, by your side and always strongly focused on your interests. It understands the context of your life, protects your privacy, your data and your security, and remembers the details that are most helpful in any given situation."

The AI assistant should adapt to its user – apparently in the same way that a dog adapts to its master. It is important, writes Suleyman, that we don't just talk about what's in the technology, not just about parameters and computing power. The focus should be on how we humans use technology and how we feel about it.

(emw)