Microsoft Copilot gets Voice and Vision features

Microsoft on Tuesday announced several updates to its free Microsoft Copilot and paid-for Copilot Pro services aimed at making the personal AI assistant more powerful and easier to converse with.

Among the updates is Copilot Vision. Built natively into Microsoft’s Edge browser, the Vision feature lets Copilot see what a user sees when surfing the web. It can then respond to queries about the contents of a web page in natural language — highlighting reviews to help choose a film on Rotten Tomatoes, to give one of Microsoft’s examples, or assisting with research. 

“We believe Copilot can go beyond answering basic questions and generating content, to offering more complete support for you and your tasks,” Yusuf Mehdi, Microsoft’s corporate vice president and consumer chief marketing officer, said in a pre-recorded press briefing.

The service will be limited to a list of pre-approved websites initially, said Microsoft. The company’s AI models won’t be trained on the content Copilot views.

“Increasingly, generative AI assistants are becoming multi-modal (language, vision and voice) and have personalities that can be configured by the consumers,” said Jason Wong, distinguished vice president analyst at Gartner. “We will see even more anthropomorphism of AI in the coming year.” 

Mehdi said Microsoft has taken steps to “respect and protect” user privacy when accessing Copilot Vision, which is turned off by default. “You must actively choose to enable the Copilot feature,” he said. “You have clear notification it is on, no conversations or content are stored beyond the active session, and none of the Copilot Vision interactions will be used for training.”

The feature will initially roll out in the United States via Copilot Labs, a new service where paid Copilot Pro subscribers can test upcoming AI capabilities. Copilot Pro costs $20 per month. 

Another experimental feature available in Copilot Labs is Think Deeper, which enables Copilot to “reason” and answer more complex questions. 

“Think Deeper takes more time before responding, allowing Copilot to deliver detailed, step-by-step answers to challenging questions,” Microsoft’s Copilot Team said in a blog post. “We’ve designed it to be helpful for all kinds of practical, everyday challenges like comparing two complex options side by side. Should I move to this city or that? What type of car best suits my needs? And so on.”

Think Deeper is available now to a limited number of Copilot Pro customers in Australia, Canada, New Zealand, the United Kingdom and the United States. 

Microsoft has also announced a refresh of the Microsoft Copilot mobile app, with a UI that is “leaner, simpler, warmer, and all around more approachable,” said Mehdi. The new Copilot app rolls out today. 

Conversations with the AI assistant will be more realistic and natural with the introduction of Copilot Voice, Microsoft said. The revamped voice interface promises faster responses and the ability to interrupt when the Copilot is speaking; users can also now select from four different voices to interact with when talking with the Copilot assistant.

“With the new Copilot Voice, you’ll have a smoother and more engaging conversation, because responses are faster and you can easily interrupt and direct your experience,” said Mehdi.

One of the Copilot voices can also be chosen to read out a Copilot Daily news digest — a summary of news from authorized content sources. (Microsoft has partnered with Reuters, Financial Times, German publisher Axel Springer and others.) It will also provide weather forecasts, with a reminder function also in the works. 

Copilot Voice is initially available in English in Australia, Canada, New Zealand, the United Kingdom, and the United States. It will expand to more regions and languages soon, Microsoft said. Copilot Daily is rolling out now starting in the United States and the United Kingdom with more countries coming soon.

To help new users get started, Microsoft has released Copilot Discover, which provides guidance on the AI assistant’s features and “conversation starter” suggestions.

Microsoft

The introduction of realistic AI assistants is part of a wider trend, said Wong. Gartner predicts that, by 2026, 80% of the top 100 consumer brands will offer anthropomorphized generative AI agents to drive consumer loyalty. 

It’s not just Microsoft Copilot; Google Gemini, OpenAI’s ChatGPT, and X.ai Grok are all developing multi-modal agents that will “entertain, inform and connect the consumer to relevant services and products,” said Wong. “This is the next frontier — and battleground — of customer experience.”