ChatGPT will soon be able to see everything happening on your screen

Yesterday, in its spring update, OpenAI released a native Mac app for ChatGPT and also debuted its new flagship model, GPT-4o . The demos generated a lot of excitement, especially among Apple users who were looking forward to the Apple-OpenAI partnership that is rumored to be coming to iOS 18 .

One important thing in the full event video was that the upcoming ChatGPT app feature was demoed but no real details were shared. That is, ChatGPT's ability to see everything that's happening on the user's device screen.

Are you ready to trust ChatGPT and share your screen content?

At launch, the ChatGPT Mac app supports taking a screenshot manually or uploading a file to then query ChatGPT for it. But in the future, as seen in the video below, you'll be able to click a button to give the app access to everything on your screen.

This feature apparently won't be limited to Macs, as iOS and iPadOS apps have demonstrated the same technology in the example below.

It’s unclear when these screen-viewing features will arrive in ChatGPT. However, the change could make for a more seamless, interactive experience. Apple even provides an API for developers to tap into the device’s screen content—with user consent, of course. So OpenAI isn’t doing anything nefarious by adding these features.

Having ChatGPT access to the entire content of your screen can be a bit scary, but it can also be extremely useful.

There will likely be a time limit on how long an app can access a user's screen, requiring a button to be pressed each time you want to grant access. But in theory, it makes sense that one day AI tools will have permanent access to everything you can see, allowing for more seamless assistance with minimal friction.

An AI assistant that can see everything you see on your devices seems like something Apple is aiming for at a system level across its platforms. If Siri gets a new AI boost, with deep system integration, it could prove to be a real asset for iOS 18 and later users.

Sign up and earn $1000 a day ⋙

Leave a Comment

Why use Llama 3.2 instead of ChatGPT?

Why use Llama 3.2 instead of ChatGPT?

Amidst all the AI ​​hype, Llama 3.2 is a gentle option that pays attention to the finer details and it's time you switched to this tool.

5 AI Tools to Help You Write Your First Song

5 AI Tools to Help You Write Your First Song

The following tools will create a quick sketch to inspire or even help you create a complete track.

Apple: Siri Isnt Really Ready to Beat ChatGPT

Apple: Siri Isnt Really Ready to Beat ChatGPT

With the launch of the new generation iPad Mini, Apple has clearly affirmed that “AI-filled” software experiences are the way the company is heading in the future.

What makes ChatGPTs search feature superior to Perplexity?

What makes ChatGPTs search feature superior to Perplexity?

While Perplexity has been reliable for a long time, ChatGPT's new capabilities have users slowly returning to the original Generative AI chatbot.

15 Things You Can Do With ChatGPT

15 Things You Can Do With ChatGPT

Since its launch in November 2022, ChatGPT has received a lot of attention for its myriad uses. To help you get the most out of this tool, here are 15 ways you can use ChatGPT.

How to Use Canva and ChatGPT to Create Mass Content Designs

How to Use Canva and ChatGPT to Create Mass Content Designs

Along with ChatGPT, this tutorial will show you how the Bulk Create app works in Canva. You'll be amazed at how you can create a month's worth of content in just a few minutes.

ChatGPT May Be Running Silently on Your Phone and How to Fix It

ChatGPT May Be Running Silently on Your Phone and How to Fix It

Background Conversations keep a user's conversation with ChatGPT going while using other apps or when the screen is off. This means ChatGPT can listen or monitor a user's phone usage behavior 24/7.

4 Ways to Use ChatGPTs Canvas Mode

4 Ways to Use ChatGPTs Canvas Mode

ChatGPT's new Canvas mode adds a new dimension to writing and editing in the world's leading Generative AI engine.

ChatGPT Will Kill These 4 Types of Websites

ChatGPT Will Kill These 4 Types of Websites

AI chatbots are completely reshaping the way users interact with websites and making some types of websites obsolete in the process.

The first person to have ChatGPT implanted in the brain to help manipulate electronic devices

The first person to have ChatGPT implanted in the brain to help manipulate electronic devices

ChatGPT is being tested by Synchron, one of the pioneers of brain implant (BCI) chips to help patients manipulate electronic devices.

7 Ways ChatGPTs Image Import Function Helps You Solve Everyday Problems

7 Ways ChatGPTs Image Import Function Helps You Solve Everyday Problems

Did you know ChatGPT can see images? By using the image import function, it can analyze images and help you solve daily challenges seamlessly.

3 Apple Visual Intelligence Alternatives for Older iPhones

3 Apple Visual Intelligence Alternatives for Older iPhones

If you have an older iPhone, don't worry — there are great Apple Visual Intelligence alternatives that offer similar features without requiring the latest hardware.

ChatGPT experienced widespread system outage

ChatGPT experienced widespread system outage

According to OpenAI's system status page, ChatGPT is currently down for a small group of users, many of whom are unable to use the Android mobile app or other services.

Custom GPT is the least noticed feature of ChatGPT

Custom GPT is the least noticed feature of ChatGPT

Custom GPT is a popular in-app feature and even without using any other ChatGPT tools, many people will still sign up just for this feature.

Collection of easy-to-make, delicious, quality coffee cocktail recipes for everyone

Collection of easy-to-make, delicious, quality coffee cocktail recipes for everyone

Do you love coffee and want to try making your own coffee cocktail? Then the article below will summarize for you simple, delicious and attractive coffee cocktail recipes.

Bleach: Characters with the most changed appearances

Bleach: Characters with the most changed appearances

While many of the characters in the Bleach series are recognizable, that doesn't necessarily mean they've had the same look from start to finish. Here are some Bleach characters who've had significant changes in appearance.

How to Optimize Google Chrome for Super Fast Startup

How to Optimize Google Chrome for Super Fast Startup

We've all been there: Clicking the Chrome icon, then waiting for the browser to launch. The seemingly endless wait for the home page to load can be frustrating.

How to create stickers from photos on Samsung

How to create stickers from photos on Samsung

On some Samsung Galaxy phones, there is an option to create stickers from photos in the album, allowing users to freely create stickers to use in messages.

How to fix Task Manager not working in Windows

How to fix Task Manager not working in Windows

Users cannot use Task Manager when it is not working. Here is how you can fix Task Manager not working on Windows 11/10 PC.

Latest Code of Absolute God of War

Latest Code of Absolute God of War

The latest Code Dau Than Tuyet The gives players coins, gold ingots and many other items including Nguyen Phach, gift boxes, chests, Trac Viet Stones...

Legendary Dragon God Code and how to enter code

Legendary Dragon God Code and how to enter code

The rewards of the Legendary Dragon God Giftcode will mostly be gold and diamonds. Along with that are some bonus chests, stones, recovery items...

What is NanoCell? Should I choose NanoCell or OLED TV?

What is NanoCell? Should I choose NanoCell or OLED TV?

Two of the newest technologies are OLED displays and LG's NanoCell displays. These are two quite different types of TVs that are often marketed with similar features.

How to use the washing machine cleaning mode correctly and effectively

How to use the washing machine cleaning mode correctly and effectively

The article below will help you understand how to use the washing machine cleaning mode properly to help remove bacteria and dirt.

Chromecast is officially dead

Chromecast is officially dead

Google's Chromecast line has long been a popular choice if you want to replace your smart TV experience or turn any TV into a smart TV.

This Android phone just broke the record for smartphone battery life

This Android phone just broke the record for smartphone battery life

Not the iPhone 15 Pro Max, the new Asus ROG Phone 8 Pro Android gaming phone is the phone with the longest battery life today, according to the latest review from Toms Guide experts.

How to turn a photo into a painting using the Generative Fill function in Photoshop

How to turn a photo into a painting using the Generative Fill function in Photoshop

Whether you want to turn your photo into a watercolor, oil painting, or some other type of painting, here's how to create the effect using Generative Fill in Photoshop.

Apple Watch Saved a Pregnant Cardiologists Life

Apple Watch Saved a Pregnant Cardiologists Life

A pediatric cardiologist has spoken out, explaining that she had to perform an emergency cesarean section after the woman's Apple Watch advised her to seek medical help.

Dont buy a Nintendo Switch now, wait for the Switch 2!

Dont buy a Nintendo Switch now, wait for the Switch 2!

The Nintendo Switch is a great console, and there are plenty of great Switch games to choose from. But if you're still on the fence about whether or not you should buy one, you're right to be concerned.

How to Edit Photos in Paint from the Snipping Tool

How to Edit Photos in Paint from the Snipping Tool

If you take a screenshot using Snipping Tool and want to edit it further, you can edit the screenshot in Paint from Snipping Tool.