Outstanding changes and features in iOS 18.3 beta 3

The latest iOS 18 adds a feature in the Camera app settings that lets you choose to pause or resume audio on your iPhone when you start recording in Video mode.

Table of Contents

• iOS 18.3 beta 3 features
• Apple Intelligence in iOS 18.2
• 7 Apple Intelligence Features in iOS 18.1
• New features on iOS 18
• List of iPhones supporting iOS 18

iOS 18.3 beta 3 features

Yesterday, Apple released iOS 18.3 beta 3, two days after similar betas for watchOS and tvOS were released. Apple has been working to improve Notification Briefing on platforms like iOS, powered by Apple Intelligence. This update comes after more than one AI-generated summary has been flawed, leading to seemingly false and/or misleading headlines from organizations like BBC News.

To demonstrate the seriousness of these notifications, Apple has made some significant changes to how these summaries work and how they are presented to users.

Problems with Notification Summary

As part of the iOS 18.1 rollout, which includes phase one of Apple Intelligence, Apple released Notification Summary. Notification Summary attempts to concisely summarize multiple notifications from a single source into one easy-to-understand, easy-to-scan alert. Many times, these summaries are not only helpful, but also accurate in representing the content of the entire notification.

Like everything related to AI, it is not perfect and can make mistakes. However, receiving a summary of a text message from a family member or friend is one thing. Even if the content of the message is summarized incorrectly, it can often be humorous. It is another matter entirely when the message summary condenses information from a news organization, resulting in a summary that is misleading or sometimes outright wrong.

In the case of BBC News, here's what happened when the Apple Intelligence-powered summary feature pushed this misleading message to users:

It’s a serious issue that could easily damage the reputation of the original source. Needless to say, BBC News was not pleased and called out Apple for the misleading summary. Notably, BBC News did not share the original headlines used to generate the news summary, but it was clearly not the same as what was generated by the AI.

This isn’t the first time this has happened, and it won’t be the last. However, Apple is taking steps, as promised a few weeks ago, to mitigate potential issues in the future.

Clear indication of beta status

The first thing Apple wants users to know is that Notification Summaries is a beta feature. While this may not have been obvious in previous versions of iOS, all Apple Intelligence features are in beta so far. Apple made this clear when enabling notification summaries in the latest iOS 18.3 beta.

Warning

Not only will Apple Intelligence's beta status be made clearer, iOS 18.3 will also warn users that these message summaries "may contain errors." Again, public awareness of the limitations of AI-generated headlines is a good thing.

Add permission to turn off notification summary

Additionally, Apple is giving users more power to turn off summaries for individual apps directly. You could always do this through Settings → Notifications, but now you can do it directly from Notification Center or from the lock screen by swiping left on a notification → Options → Turn Off Summaries.

Italic text

The summary on the lock screen will now be displayed in italicized text. This may seem like a small change, but it will make it clear, at first glance, that what is being displayed is AI-generated, thus distinguishing it from regular notifications.

Temporarily disabled categories

The biggest problem with summaries is coming from apps in the News & Entertainment category, like the BBC. As a result, notification summaries will be disabled indefinitely for apps in this category. Apple says a future software update will re-enable summaries for apps in this category as it works to refine the experience.

Other features

There aren’t many new features in iOS 18.3 beta 3 beyond what we’ve discussed above. One thing that has been spotted is a new AXFeatureOverrideServer component. From what we can tell, this appears to be a feature that allows users to temporarily enable or disable accessibility features. This could just be a developer tool, or it could hint at an upcoming addition to iOS’s accessibility capabilities.

Apple Intelligence in iOS 18.2

Apple released iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2, introducing a whole new set of Apple Intelligence features that will enhance the user experience with iPhone, iPad, and Mac, and build on the first set of capabilities that were introduced.

Now, users can explore creative new ways to express themselves visually with Image Playground, create the perfect emoji for any situation with Genmoji, and make their writing more dynamic with new improvements in Writing Tools.

Powered by Apple Intelligence, iPhone 16 or iPhone 16 Pro users can instantly learn about their surroundings using intuitive intelligence with Camera Control. And now, with ChatGPT integrated into Writing Tools and Siri, users can tap into ChatGPT’s expertise without having to switch between apps, helping them get things done faster and easier than ever.

Design fun, unique images with Image Playground

The Image Playground experience allows users to easily create fun and unique images, with concepts such as themes, costumes, accessories, and locations. Users can add their own text descriptions and can even create a lookalike of a family member or friend using photos from their library.

Image Playground creates images in distinct styles, including Animation - a modern 3D animation look - and Illustration, which offers images with simple shapes, clean lines, and solid colors.

The experience is built right into Messages, making it easier than ever to create images for conversations, as well as into apps like Freeform, Keynote, and more. Image Playground is also available as a brand new dedicated app.

Create a Genmoji to suit every moment

With the power of Apple Intelligence, emojis are taken to the next level with Genmoji, making conversations with family and friends more fun and playful, and opening up entirely new ways to communicate.

Simply type a description into the emoji keyboard and a Genmoji will appear, with a variety of options to choose from. Using images from their photo library, users can take Genmoji even further by creating images inspired by friends or family members.

Personalized Genmoji can be customized with accessories, like hats or sunglasses, and can reflect a theme or activity to make them more personal and unique. Like emojis, Genmoji can be added online to messages or shared as stickers or reactions in Tapback.

Take note-taking to the next level with Image Wand

The Notes app has new tools that make note-taking more intuitive and dynamic. With the Image Wand in the tools panel, users can quickly create images in their notes using text or image context captured in the note.

Image Wand turns a rough sketch into a finished image simply by circling. Users can even circle a blank space in a note and Image Wand will gather context from the surrounding area - using generative models on the device that analyze handwritten or typed text - to create a relevant image that complements the note and makes it more intuitive. Users can create images using Animation, Illustration, and additional Sketch styles in Image Wand.

Describe changes in Writing Tools

Writing Tools builds on the existing Rewrite, Proofread, and Summarize options with the new ability to let users specify the changes they want to make with the new Describe Your Change option. Describe Your Change gives users more flexibility and control when they want to make their writing more expressive, such as adding more dynamic action words to a resume or even rewriting a dinner party invitation in verse, etc. Like all Writing Tools features, the new Describe Your Change option is available system-wide across Apple and many third-party apps.

Learn more about your surroundings with just one click with Visual Intelligence

The new Visual Intelligence experience builds on Apple Intelligence and helps users learn about objects and places instantly, thanks to the new Camera Control on the iPhone 16 lineup. Visual Intelligence can summarize and transcribe text, translate text between languages, detect phone numbers or email addresses with the option to add to contacts, and more.

Camera Control also lets users search Google to see where they can buy an item, or tap into ChatGPT's problem-solving skills to ask for an explanation of a complex diagram, such as from class notes. Users control when third-party tools are used and what information is shared.

Access ChatGPT with Siri and Writing Tools

Apple is enabling access to ChatGPT within the Siri and Writing Tools experience on iOS, iPadOS, and macOS, allowing users to access the tool's image and document understanding capabilities without having to switch between apps. With ChatGPT integration, Siri can suggest users access ChatGPT for certain requests, and Siri can provide direct responses.

With Compose, users can ask ChatGPT to generate content for any topic they're writing about from Writing Tools across the system. They can also use ChatGPT's image generation capabilities to add images alongside their written content.

Users can choose whether or not to enable the ChatGPT integration and have full control over when to use it and what information is shared with ChatGPT. By default, a ChatGPT account is not required to use this integration. When using ChatGPT without an account, OpenAI will not store requests and will not use data to train the model. Additionally, the user's IP address will be masked to prevent their sessions from being linked. For those who choose to connect their accounts, OpenAI's data usage policy will apply.

More possibilities coming soon

Additional Apple Intelligence capabilities will be available in the coming months. Siri will be even more capable, with the ability to provide personalized intelligence based on a user’s personal context. Siri will also be aware of everything that’s happening on the screen, and will be able to perform hundreds of new actions in and across Apple and third-party apps. Priority notifications will also surface what’s most important. Additionally, users will be able to create images in Image Playground in the Sketch style, an academic and highly detailed style that uses a vibrant color palette combined with technical lines to create realistic drawings.

A Breakthrough in Privacy in AI

Designed to protect user privacy at every step, Apple Intelligence uses on-device processing, meaning many of the models that power it run entirely on the device. For requests that need access to larger models, Private Cloud Compute extends the privacy and security of iPhone into the cloud to unlock even more intelligence.

When using Private Cloud Compute, user data is never stored or shared with Apple; it is only used to fulfill their requests. Independent experts can audit the code running on Apple silicon servers to repeatedly verify this privacy promise, and they do. This is an extraordinary step forward for privacy in AI.

7 Apple Intelligence Features in iOS 18.1

Apple has started rolling out the first wave of Apple Intelligence features to iPhone users. As long as you have an iPhone 15 Pro or later, you can start taking advantage of them. Here's a look at all the AI ​​features you'll get with iOS 18.1.

1. Photos app improvements, including Clean Up feature

Apple Intelligence is adding some new features to the Photos app. When you try to edit a photo, you'll now see a Clean Up tool that can remove unwanted objects from your photo. Like Google's Magic Eraser, you can use this on any photo—whether it's a screenshot or a photo taken with another camera.

To use this feature, open a photo, tap the Edit icon (represented by three sliders) at the bottom, and select Clean Up . Apple Intelligence will highlight areas of the photo you might want to clean up. You can also tap, brush, or circle something you want to remove. Pinch to move and zoom in closer to the image. This feature works best with smaller parts of the image or objects against a plain background.

You can also use natural language in the Photos app to search for something specific, like “Maddie near dolphins.” You can also search for a specific event in a movie. In Memories, you can create a movie by entering a description like “Trip to Orlando.”

2. Writing Tools

With Apple Intelligence’s Writing Tools, you can edit anything you write. These tools are available in any app, including Mail and Messages. Just highlight the text and select Writing Tools.

In the Writing Tools menu that appears at the bottom, you can choose Proofread to correct selected text, including spelling errors. You can go through each change Apple Intelligence makes, or choose to roll back all changes at once.

Another option is Rewrite . This will change the tone of the text with a choice of Friendly, Professional , or Concise .

Finally, you can select text and summarize in a variety of ways. You can copy and paste the summary wherever you like.

3. Better Siri

When you activate Siri with a voice command or by pressing the Side button, you'll see a new light around the edges of your iPhone's screen. As you speak, the light will also move with the sound of your voice.

There's also a new way to activate your personal assistant now. Just double-tap the bottom of the screen to bring up the Type to Siri interface. You can type your command instead of speaking it. As you type, Siri will provide a list of suggested requests.

When you talk to Siri, the personal assistant will also remember the context between questions. For example, you can ask "What time is it in Los Angeles?" and follow it up with "What's the weather like there?" and Siri will remember which city you're asking about.

If you change what you're saying or stumble over a phrase when making a request, Siri can now follow along and still provide the correct answer.

Siri has also become an Apple expert. You can ask Siri a variety of questions about your iPhone, iPad, or Mac, such as specific settings or customization tools. For example, you can ask Siri for instructions on how to schedule text messages, one of the useful Messages features Apple introduced with iOS 18.

4. Summary and more in Mail and Safari

Apple Intelligence can help summarize content in both Safari and Mail. When in Reader Mode in Safari, select Summarize at the top of the page to see a summary of the article.

In the Mail app, you'll now see a quick summary of each email in your inbox, along with the sender and subject. Previously, you saw the first few lines of the email. You can also open a specific email and select Summarize in the upper-right corner to see a summary of the email's contents.

Additionally, whenever you receive a time-sensitive email, the Mail app will automatically push it to the top of your inbox for easy access. When replying to an email, the Smart Reply option will provide a quick response based on the context of the email.

5. Transcription for phone calls and voice recordings

In iOS 18.1, you can take advantage of transcription when recording phone calls or audio in apps like Notes and Voice Memos. You can see exactly what was said and even share or use the text in another app.

With Apple Intelligence, you can also see summaries of those transcriptions.

6. Smart replies to messages

Like the Mail app, you can tap the Smart Reply option in Messages. This will look at the text of the message and give you some suggestions for a quick reply. This feature often works best in situations where there are questions that you can answer "Yes" or "No."

7. Summary of announcement

Your iPhone's lock screen can be filled with notifications from almost any app. That's where Notification Summaries come in handy; they help you better manage all that information.

Instead of having to read multiple notifications from a single app, this feature provides a summary of what's happening from each app. It works for all apps. During a short setup process, you can turn it on for all apps or select specific apps.

Additionally, iOS 18.1 has a new Priority Notifications feature that uses Apple Intelligence to scan all notifications and prioritize any that need immediate attention. This feature also works with the new Reduce Interruptions Focus mode, which you can enable by going to Settings > Focus and selecting Reduce interruptions .

Once set up, Apple Intelligence will allow important notifications while silencing any that it deems unimportant. You can further customize Focus Mode to allow specific people and apps to interrupt you.

While this is just the first batch of all the Apple Intelligence features announced at WWDC 2024, Apple has already made a good start by adding essential features to improve the overall iPhone user experience.

New features on iOS 18

Customize home screen interface

iPhone users can customize their Home screen, Lock screen, and Control Center in a variety of ways.

Users can arrange apps and widgets anywhere on the screen, set app and widget icons to have dark or colored effects, and zoom in for the best experience.

Users can add controls from third-party apps to Control Center to quickly unlock their car or take social media photos.

Apple Intelligence

Apple Intelligence, a personal artificial intelligence system deeply integrated into iOS 18 and built with privacy in mind, helps users improve their writing and communication. Users can rewrite, proofread, and summarize text almost anywhere, including Mail, Notes, and third-party apps.

With Image Playground, users can create fun images in seconds that make communication and self-expression more fun, through three styles: Animation, Illustration or Sketch.

The Memories feature in the Photos app lets users enter descriptions to create their own stories. Apple Intelligence does the rest, arranging the images into a movie with its own content.

The new Clean Up tool can identify and remove distracting objects in the background of photos.

Siri is also getting an upgrade thanks to Apple's Intelligence capabilities to become more personal and natural. Users can communicate with Siri in any way they want, including text or voice.

Users can use ChatGPT without switching between tools thanks to system-wide integration across Apple platforms.

Photos App

The photo app has a simple interface that lets users browse by topic without having to organize content into albums.

Messages App

iMessage gets new text effects including formatting like bold, underline, italic, and strikethrough that bring life to conversations.

iOS 18 also allows users to send messages via satellite when cellular or Wi-Fi connections are not available.

Mail App

Later this year, Mail will introduce new ways for users to manage their inboxes and stay up to date. On-device categorization will organize and sort emails into categories like Primary, Transactional, Updates, and Promotions. A new summary view aggregates important emails from a business, allowing users to quickly scan through what matters.

Safari

With Highlights and a redesigned Reader experience, Safari on iOS 18 provides an easier way for users to discover information on the web.

Using machine learning, Safari can display key information about a web page so users can see a summary to get the most important information of the article.

Password Application

The new Passwords app on iOS 18 gives users easy access to passwords, passkeys, Wi-Fi passwords, and verification codes. The app also alerts users when it detects common weaknesses like easily guessed or used passwords multiple times.

Security Features

With new management tools, iOS 18 gives users control over who can see their apps, how to share contacts, and how their iPhone connects to accessories.

Tap to Cash

The Wallet app in iOS 18 allows users to simply enter the amount of money they want to transfer to the other person, then tap their iPhone on the recipient's iPhone to transfer money without needing an account number, phone number or email address.

However, this feature is only available in the US.

Call recording

iOS 18 will bring a recording button right in the call interface, displaying a sound wave icon and the duration of the recording.

This feature will automatically notify call participants that they are being recorded to ensure privacy.

The recorded call will be transcribed into text in the Notes app, where Apple's new artificial intelligence (AI) system - Apple Intelligence will create a call summary for the user.

Additionally, users can also record and translate audio directly from within the Notes app. However, the audio translation feature currently only supports the following languages: English, Spanish, German, French, Japanese, Chinese, Cantonese, and Portuguese.

Lock apps on iPhone with Face ID

Apple has just added the feature of locking apps on iPhone with Face ID, including built-in Apple apps and third-party apps to help users better protect data, limiting the leakage of important information. This feature helps locked apps remain safe and locked when you give your phone to someone else, even if your phone is unlocked.

Vocal Shortcuts

Take action based on any voice command even when iPhone is locked

Voice computing is still in its early stages, for one reason: voice assistants often can't understand a user's speech unless it fits a predefined pattern.

Siri, with Apple Intelligence, will hopefully get a lot smarter in this regard. If not, there's another feature in iOS 18 that should help: Vocal Shortcuts.

The Power of Vocal Shortcuts

Vocal Shortcuts is a powerful feature of iOS 18 that lets you trigger any action on your iPhone or iPad with a custom voice command of your choice. Apple describes it as follows:

With Vocal Shortcuts, iPhone and iPad users can assign custom utterances that Siri can understand to launch shortcuts and complete complex tasks.

Vocal Shortcuts are available in developer beta 1 of iOS and iPadOS 18, and they're super easy to set up.

Set up Vocal Shortcuts

First, you need to enable this feature on your iPhone or iPad. It's a simple process:

1. Open the Settings app
2. Navigate to Accessibility ⇾ Vocal Shortcuts
3. Click Set Up Vocal Shortcuts

Once this feature is enabled, you can create your first Vocal Shortcut by doing the following:

1. Select an action it will perform
2. Define custom trigger phrase
3. Say the phrase out loud 3 times

All Vocal Shortcuts processing happens entirely on the device, as confirmed by the feature description inside Settings:

Teach iPhone to recognize a custom phrase you can say to quickly perform an action. Audio is processed on iPhone.

Enable Vocal Shortcuts in iOS 18

Once Vocal Shortcuts are configured, you can say the trigger phrase at any time while using your device, even when the device is locked, to perform the specified action.

And importantly, there's no need to call Siri first, just say the trigger words and the action will be performed.

Types of actions supported by Vocal Shortcuts

Actions that can be selected when setting up Vocal Shortcuts include:

• Anything Siri can do - Select a command, then assign it a custom phrase and Siri will perform that command when you say that phrase
• Any shortcuts you have in the Shortcuts app
• Or one of many basic functions like activating Control Center, the device's flashlight, adjusting volume, etc.

There's a lot of power here to perform complex multi-step actions, like by activating the Shortcuts app. But Vocal Shortcuts can also do really simple things.

For example, you can use Vocal Shortcuts to open the Camera app when your device hears the word 'Camera'. But of course, you have to be careful not to use really common words, or your device will accidentally start performing actions you didn't intend.

A better example might be opening another app with a voice command. You could say 'Fantastical' and that third-party calendar app would open instantly.

Remember, there's no need to press a button first or activate Siri by voice: All you do is say 'Fantastical' and the app will open.

Vocal Shortcuts. is an incredibly powerful tool with a lot of potential. There are a ton of accessibility benefits to this feature, but the average power user would also benefit by building Vocal Shortcuts to make everyday computing a little more convenient.

List of iPhones supporting iOS 18

• iPhone 15
• iPhone 15 Plus
• iPhone 15 Pro
• iPhone 15 Pro Max
• iPhone 14
• iPhone 14 Plus
• iPhone 14 Pro
• iPhone 14 Pro Max
• iPhone 13
• iPhone 13 mini
• iPhone 13 Pro
• iPhone 13 Pro Max
• iPhone 12
• iPhone 12 mini
• iPhone 12 Pro
• iPhone 12 Pro Max
• iPhone 11
• iPhone 11 Pro
• iPhone 11 Pro Max
• iPhone XS
• iPhone XS Max
• iPhone XR
• iPhone SE (2nd generation)
• iPhone SE (3rd generation)