ChatGPT will soon be able to see everything happening on your screen

Yesterday, in its spring update, OpenAI released a native Mac app for ChatGPT and also debuted its new flagship model, GPT-4o . The demos generated a lot of excitement, especially among Apple users who were looking forward to the Apple-OpenAI partnership that is rumored to be coming to iOS 18 .

One important thing in the full event video was that the upcoming ChatGPT app feature was demoed but no real details were shared. That is, ChatGPT's ability to see everything that's happening on the user's device screen.

Are you ready to trust ChatGPT and share your screen content?

At launch, the ChatGPT Mac app supports taking a screenshot manually or uploading a file to then query ChatGPT for it. But in the future, as seen in the video below, you'll be able to click a button to give the app access to everything on your screen.

This feature apparently won't be limited to Macs, as iOS and iPadOS apps have demonstrated the same technology in the example below.

It’s unclear when these screen-viewing features will arrive in ChatGPT. However, the change could make for a more seamless, interactive experience. Apple even provides an API for developers to tap into the device’s screen content—with user consent, of course. So OpenAI isn’t doing anything nefarious by adding these features.

Having ChatGPT access to the entire content of your screen can be a bit scary, but it can also be extremely useful.

There will likely be a time limit on how long an app can access a user's screen, requiring a button to be pressed each time you want to grant access. But in theory, it makes sense that one day AI tools will have permanent access to everything you can see, allowing for more seamless assistance with minimal friction.

An AI assistant that can see everything you see on your devices seems like something Apple is aiming for at a system level across its platforms. If Siri gets a new AI boost, with deep system integration, it could prove to be a real asset for iOS 18 and later users.

Sign up and earn $1000 a day ⋙

Leave a Comment

Why ChatGPT is better than DeepSeek

Why ChatGPT is better than DeepSeek

DeepSeek initially had high hopes. As an AI chatbot marketed as a strong competitor to ChatGPT, it promised intelligent conversational capabilities and experiences.

ChatGPT Starts Adding Google Drive Integration, But Not for Everyone

ChatGPT Starts Adding Google Drive Integration, But Not for Everyone

ChatGPT has started adding support for Google Drive integration but this feature is not available to all users.

Each ChatGPT reply consumes an average of 2.5 Wh of electricity.

Each ChatGPT reply consumes an average of 2.5 Wh of electricity.

OpenAI's ChatGPT chatbot uses more than half a million kWh of electricity every day, 17,241 times more than the average US household (about 29 kWh/day).

Why use Llama 3.2 instead of ChatGPT?

Why use Llama 3.2 instead of ChatGPT?

Amidst all the AI ​​hype, Llama 3.2 is a gentle option that pays attention to the finer details and it's time you switched to this tool.

5 AI Tools to Help You Write Your First Song

5 AI Tools to Help You Write Your First Song

The following tools will create a quick sketch to inspire or even help you create a complete track.

Apple: Siri Isnt Really Ready to Beat ChatGPT

Apple: Siri Isnt Really Ready to Beat ChatGPT

With the launch of the new generation iPad Mini, Apple has clearly affirmed that “AI-filled” software experiences are the way the company is heading in the future.

What makes ChatGPTs search feature superior to Perplexity?

What makes ChatGPTs search feature superior to Perplexity?

While Perplexity has been reliable for a long time, ChatGPT's new capabilities have users slowly returning to the original Generative AI chatbot.

15 Things You Can Do With ChatGPT

15 Things You Can Do With ChatGPT

Since its launch in November 2022, ChatGPT has received a lot of attention for its myriad uses. To help you get the most out of this tool, here are 15 ways you can use ChatGPT.

How to Use Canva and ChatGPT to Create Mass Content Designs

How to Use Canva and ChatGPT to Create Mass Content Designs

Along with ChatGPT, this tutorial will show you how the Bulk Create app works in Canva. You'll be amazed at how you can create a month's worth of content in just a few minutes.

ChatGPT May Be Running Silently on Your Phone and How to Fix It

ChatGPT May Be Running Silently on Your Phone and How to Fix It

Background Conversations keep a user's conversation with ChatGPT going while using other apps or when the screen is off. This means ChatGPT can listen or monitor a user's phone usage behavior 24/7.

4 Ways to Use ChatGPTs Canvas Mode

4 Ways to Use ChatGPTs Canvas Mode

ChatGPT's new Canvas mode adds a new dimension to writing and editing in the world's leading Generative AI engine.

ChatGPT Will Kill These 4 Types of Websites

ChatGPT Will Kill These 4 Types of Websites

AI chatbots are completely reshaping the way users interact with websites and making some types of websites obsolete in the process.

The first person to have ChatGPT implanted in the brain to help manipulate electronic devices

The first person to have ChatGPT implanted in the brain to help manipulate electronic devices

ChatGPT is being tested by Synchron, one of the pioneers of brain implant (BCI) chips to help patients manipulate electronic devices.

7 Ways ChatGPTs Image Import Function Helps You Solve Everyday Problems

7 Ways ChatGPTs Image Import Function Helps You Solve Everyday Problems

Did you know ChatGPT can see images? By using the image import function, it can analyze images and help you solve daily challenges seamlessly.

Difference between regular TV and Smart TV

Difference between regular TV and Smart TV

Smart TVs have really taken the world by storm. With so many great features and the ability to connect to the Internet, technology has changed the way we watch TV.

Why doesnt the freezer have a light but the refrigerator does?

Why doesnt the freezer have a light but the refrigerator does?

Refrigerators are familiar appliances in families. Refrigerators usually have 2 compartments, the cool compartment is spacious and has a light that automatically turns on every time the user opens it, while the freezer compartment is narrow and has no light.

2 Ways to Fix Network Congestion That Slows Down Wi-Fi

2 Ways to Fix Network Congestion That Slows Down Wi-Fi

Wi-Fi networks are affected by many factors beyond routers, bandwidth, and interference, but there are some smart ways to boost your network.

How to Downgrade from iOS 17 to iOS 16 without Losing Data using Tenorshare Reiboot

How to Downgrade from iOS 17 to iOS 16 without Losing Data using Tenorshare Reiboot

If you want to go back to stable iOS 16 on your phone, here is the basic guide to uninstall iOS 17 and downgrade from iOS 17 to 16.

What happens to the body when you eat yogurt every day?

What happens to the body when you eat yogurt every day?

Yogurt is a great food. Is it good to eat yogurt every day? What will happen to your body when you eat yogurt every day? Let's find out together!

Which type of rice is best for health?

Which type of rice is best for health?

This article discusses the most nutritious types of rice and how to maximize the health benefits of whichever rice you choose.

How to wake up on time in the morning

How to wake up on time in the morning

Establishing a sleep schedule and bedtime routine, changing your alarm clock, and adjusting your diet are some of the measures that can help you sleep better and wake up on time in the morning.

Rent Please! Landlord Sim Tips for Beginners

Rent Please! Landlord Sim Tips for Beginners

Rent Please! Landlord Sim is a simulation mobile game on iOS and Android. You will play as a landlord of an apartment complex and start renting out an apartment with the goal of upgrading the interior of your apartments and getting them ready for rent.

Latest Bathroom Tower Defense Codes and How to Enter Codes

Latest Bathroom Tower Defense Codes and How to Enter Codes

Get Bathroom Tower Defense Roblox game codes and redeem them for exciting rewards. They will help you upgrade or unlock towers with higher damage.

Structure, symbols and operating principles of transformers

Structure, symbols and operating principles of transformers

Let's learn about the structure, symbols and operating principles of transformers in the most accurate way.

4 Ways AI Is Making Smart TVs Better

4 Ways AI Is Making Smart TVs Better

From better picture and sound quality to voice control and more, these AI-powered features are making smart TVs so much better!

Why ChatGPT is better than DeepSeek

Why ChatGPT is better than DeepSeek

DeepSeek initially had high hopes. As an AI chatbot marketed as a strong competitor to ChatGPT, it promised intelligent conversational capabilities and experiences.

Meet Fireflies.ai: The Free AI Secretary That Saves You Hours of Work

Meet Fireflies.ai: The Free AI Secretary That Saves You Hours of Work

It's easy to miss important details when you're jotting down other essentials, and trying to take notes while chatting can be distracting. Fireflies.ai is the solution.

How to raise Axolotl Minecraft, tame Minecraft Salamander

How to raise Axolotl Minecraft, tame Minecraft Salamander

Axolot Minecraft will be a great assistant for players when operating underwater if they know how to use them.

A Quiet Place: The Road Ahead PC Game Configuration

A Quiet Place: The Road Ahead PC Game Configuration

A Quiet Place: The Road Ahead's configuration is rated quite highly, so you will need to consider the configuration before deciding to download.