Microsoft announced a big new AI feature on Windows 11 later this year coming exclusively to Copilot+ PCs later this year called Click To Do. This feature adds an intelligent overlay to the OS, that can understand what’s on-screen and provide contextual actions at any time. To invoke the overlay hold down the Windows key and click with your mouse.
- Click To Do works similarly to Android’s Circle To Search,. It allows users to quickly search the web based on an element selected on screen.
- Click To Do goes a step further with contextual actions based on what’s currently on screen.
- Click To Do is exclusive to Copilot+ PCs.
The feature is called Click To Do and is a clear competitor to Android’s Circle To Search function, and it is more capable. While a quick way to visually search the web,
Click To Do also provides contextual suggestions based on what it can see on screen.
For example, select some text, and then Click To Do will offer actions such as summarizing what is highlighted, or rewriting it. When you select an image, Click To Do will provide quick actions like blurring the background ,or erasing an object within the image that has been selected.
In the Click To Do overlay, search for anything on screen with a dedicated search bar that floats at the top of the screen. This interface offers quick access to Windows Recall, because both Recall, and Click To Do use similar technology to identify what’s on-screen.
If you’re working on a presentation, Copilot might offer to help expand the text in one section, remove the background of an image, or even email someone if you have their address in the presentation.
The new Recall tool, helps you find things you were looking at recently on your computer, document, webpage, video, or anything else. The AI can remember and recognize what it was and pulls it back up again for you, even if you don’t remember the location or file name. Windows Search on Copilot+ PCs will even find files and settings that you haven’t opened yet by using a description you write in.
Click To Do is exclusive to Copilot+ PCs with an NPU of 40+ TOPS. So, if you’re not using a Copilot+ PC, then you won’t get access to this feature his year. Copilot+ PCs will be able to test Click To Do in October, and it is expected to begin rolling out in waves from November onwards. Microsoft further announced more coming to Copilot+ PCs before year end,, including adding AI to Windows Search. Iand a brand-new Copilot experience that’s more human and interactive, and can browse the web alongside you and help with tasks with Copilot Vision.
The AI can suggest ideas for related information or analysis, even bring up products that might better suit your interests and budget if you ask, simply using the vision on the screen. It won’t work universally yet, but it can handle most of the more popular websites. This is an opt-in feature, so none of the data is stored or used for training unless you agree. Notably, Copilot Vision isn’t limited to printed text; it can even read handwriting
An AI companion for everyone – The Official Microsoft Blog
The loudest upgrade is the new Copilot Voice feature that allows you to converse vocally with Copilot. It’s a bit like OpenAI’s new AdvancedVoice Mode in that you can speak to the AI like you would a human, even interrupting mid-word, and hear what it has to say back.
You can select four voices for the experience, with multiple accents, though only in English for now. Copilot Voice can also be your newscaster with the Copilot Daily feature. You can get audio summaries of the latest news and weather personalized to your location and interests. The AI relies on Microsft’s partners, including Reuters, Axel Springer, Hearst, and the Financial Times, to source and summarize the news.
Copilot Daily: Copilot Daily helps you kick off your morning with a summary of news and weather, all read in your favorite Copilot Voice, with more options like reminders of what’s coming soon. It’s an antidote for that familiar feeling of information overload. Clean, simple and easy to digest. Copilot Daily will only pull from authorized content sources
