Copilot Vision, Voice, and Weather: An Updated Experience for Providing a User-Friendly Artificial Intelligence Companion
Notepad is a weekly newsletter written by Tom Warren which shines a light on the secrets and strategy behind Microsoft’s bets on computing, gaming and artificial intelligence. Receive the latest in your inbox.
Microsoft will be ramping up its work on its vision of an artificial intelligence companion for everyone, similar to the one Openai has put in place, by adding voice capabilities that are very similar to what it has already done. It is now possible to talk with the assistant, ask it questions, and interrupt it if you wish. When you first use the updated Copilot experience, you are encouraged to pick one of the four voice options.
This updated Copilot experience will be available today in the mobile iOS and Android apps, on the web at copilot.microsoft.com, and through the Copilot Windows app. Copilot Voice will be initially available in English in Australia, Canada, New Zealand, the UK, and the US, before expanding to more regions and languages in the future. Copilot Daily is limited to the US and the UK before it expands elsewhere, and Copilot Vision will be limited to a number of Copilot Pro subscribers in the US.
Microsoft is putting Think Deeper into Copilot Labs to give it a chance to see experimental features the company is still developing. Copilot Vision will be included in the Labs feature initially, and participants can give feedback on the experiences. Microsoft is clearly treading carefully with Copilot Vision after the backlash around its initial Recall security and privacy issues. Microsoft revealed last week that Recall has been overhauled with improved security and privacy options, and you’ll even be able to fully uninstall the feature or not turn it on in the first place.
Copilot Daily is a summary of news and weather that is read out by Copilot as if it were a CNN anchor. It’s designed as a short clip you can listen to in the mornings, and it only uses content from news and weather providers that have authorized Copilot to use their content. Microsoft is collaborating with several other sources initially with plans to add more over time.
Copilot Vision sessions are opt-in and ephemeral, and Microsoft says none of the content Copilot Vision engages with is stored or used for training. Microsoft restricts the types of websites Copilot Vision works with, so this new experience will not work on all websites yet. The Copilot team says they are starting with a limited list of popular websites to make sure it is safe for everyone. During preview, Copilot Vision won’t work on paywalled and sensitive content, either.
Voice: Can Artificial Intelligence Help us Live Better? A Conversational Approach to Natural Language Processing and Memory Loss in the 21st Century
He says that they are making a huge bet on voice. “When you use it with the way we’ve designed it, you really start to let yourself go and have conversations. You can see the possibilities with vision, where the artificial intelligence can help and help if you want it to.
“At Microsoft AI, we are creating an AI companion for everyone,” says Suleyman in an open letter today. “I truly believe we can create a calmer, more helpful and supportive era of technology, quite unlike anything we’ve seen before.”
The best known appearance of Microsoft Clippy was when users opened Word with the line, “It looks like you’re writing a letter…” The product was unpopular; Microsoft concluded this was in part because the program failed to deliver on the humanlike intelligence it promised, forgetting users’ preferences and repeating itself endlessly. Large language models are better at mimicking intelligence than it is, but their behavior is still odd and unpredictable, which may prove a factor in Copilot’s popularity.
“One of the things that seems to be most common is that people ask it for aesthetic advice,” Suleyman says. They’re on a fashion website, and they’re wondering what the pattern is. What do you call that dress?”
From Practice to Reality: How to Make Sense of Your Companion’s Real-Time Experiences in a Collaborative Environment, with Application to AlphaGo
He co-authored a paper on Combining Practice with Positive and Negative feedback for computers to tackle seemingly impossible problems as a founding co-owner of DeepMind. DeepMind demonstrated the approach with AlphaGo, a program that defeated the best Go player in the world.
Suleyman says it’s all part of a plan to make users fall back in love with the PC. He spoke with Will Knight, a senior writer at WIRED, over Microsoft Teams. The conversation has been lightly edited.
The new design material is all about persistence, about relationship, about emotion. I create experiences which are about long-term interaction with a companion.
Microsoft was interested in building supportive and compassionate artificial intelligence. It seems that you brought that with you to your new job.
That’s the beauty of this technological moment. To see what it feels like to engage with one of these experiences over a sustained period of time—this companion that really gets to know you. It’s teaching you, encouraging you, and coaching you. I think that isn’t going to feel like a computer anymore.
There are a lot of little moments when you’re at the computer. It’s amazing to have an automated companion that can see what you see and talk to you in real time. It reduces the need to type in things and makes it easier to navigate through your digital life.