Inside Look: I Experienced Every New AI Update on Copilot PCs & Here’s Why It Might Just Live Up to Its Hype Insider Report

Sabrina Ortiz/ZDNET

As generative AI evolves, tech giants are developing new solutions that make solving everyday problems a little faster and easier. Microsoft’s latest Copilot+ PC announcement featured such AI solutions from the start to the end of the presentation, and I got a chance to test them all out.

Also: Every Copilot+ PC Microsoft just announced to take on Apple’s M3 MacBooks

The new AI features, including Recall, Cocreator in Paint, Live captions with live translation, Restyle in Photos, and Windows Studio Effects, are only available on Copilot+ PCs, which include the new Surface Laptop and Surface Pro , as well as PCs from partners such as Lenovo, HP, and Dell.

My thoughts on the latest AI tools, their performance and usability, and more below.

Newsletters

ZDNET Tech Today

ZDNET’s Tech Today newsletter is a daily briefing of the newest, most talked about stories, five days a week.

Subscribe

See all

Disclaimer: This post includes affiliate links

If you click on a link and make a purchase, I may receive a commission at no extra cost to you.

1. Recall

Sabrina Ortiz/ZDNET

How often have you exited out of a tab, application, or project that you later wanted to access but couldn’t find? Microsoft yesterday unveiled a new “Recall” feature for Copilot+ PCs that solves that exact issue, while taking things a step further.

With Recall, users can locate just about anything they’ve ever interacted with on their PC – including files, images, text, and more – simply by typing in what they want to locate and having their PC populate possible matches for them. You can also scroll back through possible timelines to see what and where you were browsing in the past.

Also: 3 AI features coming to Copilot+ PCs that I wish were on my MacBook

The feature works by storing a semantic index of your activity locally on your PC. If you are like me, your first thoughts were, “Do I want everything stored?” Microsoft addresses that concern by allowing users to personalize their snapshots, delete specified information, pause any tracking, and filter websites and apps that they don’t want to be saved.

One example that a Microsoft member shared with me was when he asked his colleague if they were familiar with another colleague. To help show him what the other colleague looked like, he used the Recall feature to access a Zoom call he had with the individual.

2. Live captions with live translation

Sabrina Ortiz/ZDNET

As someone who grew up in a Spanish-speaking household, I am familiar with language barriers and how much they can hinder everyday communication. Naturally, the AI-supported Live captions with live translation feature stood out to me the most.

As the lengthy name implies, the feature can generate captions for all audio that passes through your PC in over 40 languages into English subtitles, all in real time. For the demo, I spoke into the Surface Laptop’s microphone and was pleasantly surprised with how accurate and speedy the Spanish-to-English translation was.

Also: Microsoft’s Surface Pro and Laptop are the ultimate ‘AI PCs’, and I’m worried for Apple

When I switched from speaking English to Spanish, the captions automatically shifted from one language to the other nearly instantaneously with minimal lag. Most importantly, the translations were still accurate and better than the standard Google Translate. Not having to switch between languages manually is a game-changer because, in real-world conversations, you don’t have the time or foresight to predict when the other person will start speaking.

When using it, you can choose whether the computer’s microphone or the internal audio from a video, song, or meeting should be used. This makes the feature suitable for various applications, such as translating all-hands meetings, conversations with customer service representatives, and more.

3. Cocreator in Paint

Sabrina Ortiz/ZDNET

Will Cocreator win awards for being the most useful AI feature ever? Probably not. Was it tons of fun to test? Definitely. With Cocreator, users can use an AI canvas to quickly transform their sketches into more polished art. The Creativity bar lets you choose how much of the original sketch you’d like to keep and how much more you’d like to be AI-generated.

Also: I went hands-on with Microsoft’s Surface Pro and Laptop, and I’m ready to ditch my MacBook

For example, in the demo, I drew a terrible sketch of a puppy, as seen in the image above. When I turned Creativity up to 100, Cocreator generated a much cuter rendition of the dog, keeping some basic features such as big beady eyes, floppy ears, and even my color scheme.

This feature appeals to me more than the traditional AI image generators since the mock-up process feels more genuine and personal. Additionally, some creatives, such as marketers or interior designers, can benefit from adding a tool like this to their workflow – when an idea comes to mind, and they just don’t have the time to map it all out.

4. Restyle Image in Photos

Sabrina Ortiz/ZDNET

Almost equally as fun is Restyle in Photos. With this feature, users can modify their photos with generative AI, changing elements such as the background or even the full picture to match a specific style or theme.

Also: What is Veo? Meet Google’s next-gen AI video creator

As seen in my demo, I was able to completely change the background of a photo of the couple by typing in a descriptive text prompt with what I wanted to see instead. Like Cocreator, users can set a Creativity level to adjust how much of the image is generated with AI.

While Restyle in Photos may not be a feature I would necessarily use every day, I can see it being useful for creating lock screens, greeting cards, seasonal messages, and more.

5. Image Creator in Photos

Sabrina Ortiz/ZDNET

Copilot+ PCs also have Image Creator in Photos, which, like other image generators, transforms a user text prompt into an image. However, the impressive part here is that the tool runs on-device instead of in the cloud like DALL-E. As a result, generated images come out faster, and the whole process happens locally.

Also: The best AI image generators of 2024: Tested and reviewed

In my demo, I prompted Image Creator in Photos to generate 50 images, and I was shocked by how quickly it completed the task. As seen in the image above, Image Creator relied entirely on Qualcomm’s Snapdragon X Elite NPU to generate the images, lessening the power required from the laptop’s GPU and CPU.

6. Windows Studio Effects

Sabrina Ortiz/ZDNET

Lastly, Windows Studio Effects has been enhanced to deliver improved video calls, with features such as Portrait Light that enhances how you or your background are illuminated, an eye contact teleprompter, which helps you read your screen while keeping contact, and, my favorite, creative filters.

Also: How to use Image Creator from Microsoft Designer (formerly Bing Image Creator)

While creative filters are perhaps the least practical addition to Windows Studio Effects, it was really fun to see myself being transformed into an animated version with impressive detail, as seen in the image above. Work and video calls can be boring, so why not have some fun?

For more serious conversations, voice focus, a feature that uses AI to detect and cancel noises around you, and portrait blur, which blurs your background in calls, will be of great use.

Artificial Intelligence

How I used ChatGPT to scan 170k lines of code in seconds and save me hours of detective work

6 ways to write better ChatGPT prompts - and get the results you want faster

6 digital twin building blocks businesses need - and how AI fits in

Google’s Gems are a gentle introduction to AI prompt engineering

• How I used ChatGPT to scan 170k lines of code in seconds and save me hours of detective work
• 6 ways to write better ChatGPT prompts - and get the results you want faster
• 6 digital twin building blocks businesses need - and how AI fits in
• Google’s Gems are a gentle introduction to AI prompt engineering

Also read: