WWDC 2024 Keynote Summary

VisionOS
iOS
Apple Watch
iPadOS
MacOS
AI

VisionOS 2

! Vision Pro coming to following countries!

• Automatically create spatial photos from 2D photos
• SharePlay with others while viewing photos
• Hold your hand to check the time and go to the home screen
• Display the screen as a wide display
• New FrameworkTabletopKit Enterprise API
• Canon cameras can now capture spatial video
• New Vimeo app now supports VisionOS

iOS

A new way to customize your iPhone

Place app icons anywhere on the screen

You can change the color of the app icon

Customizing the Control Center

• Multiple Pages in Control Center
• Customizable control center.
• App developers can use the Controls API to provide widgets, and these controls can be used on the lock screen.

privacy

Lock or hide an app

You can hide the app

Control contact access for apps
Similar to image access control, you can choose which contacts you want to share with an app.

Do not ask for local network permission when pairing accessories
(uses special accessory connection dialog)

iMessage

Send Later

You can use emojis as reactions to messages.

Text formatting

iMessage via Satellite

Email app

On-device categorization

Categories include social, promotions, shopping orders, updates, etc. Sortable by company, so you can see all emails from United Airlines.

Available later this year

New Map Features

Hiking Maps (USA)

Send money with Tap to Cash (no phone number required)

When paying online with Apple Pay, you can choose between points and installment payments

Game Modes

• Minimize background activity
• Improved responsiveness when connecting to AirPods

Photo App

Photos app redesigned

Collections based on topics, people, and travel displayed below the photo grid

Separate password apps for iOS and MacOS

Apple Watch

Track your workout intensity

(Evaluated based on age, heart rate, and other data)

Customize the overview screen of your iPhone’s fitness app

New Vitals App

See metrics like your heart rate, compare trends, and get notified about any major changes

Smart Stack

Widgets are automatically added. For example, if it rains, a precipitation chart will be displayed.

Double Tap API

Implement double tap actions within the app (double tap to perform an action)
Find the perfect photo to use for your watch interface

iPadOS 18

Floating tab bar (expandable into sidebar), you can drag items from the sidebar to the floating bar

SharePlay Screen Sharing: Tap and draw to tell others where to tap

Calculator app on iPad

History function included

Formula notes, results display

Use Apple Pencil to write a math equation in a note, and the result will be automatically calculated and displayed on screen.

Able to display graphs and calculation results based on variable values

Smart scripting within the Notes app

Handwriting Improvement (uses local machine learning to learn how you write)

MacOS Sequoia

iPhone Mirroring

Even if your iPhone is in another room, you can view and control the contents of your iPhone. Wireless operation is possible.

iPhone notifications appear on MacOS

When you tap the notification, the app will appear on your Mac and sound will play through your Mac’s speakers.

Use your iPhone on your Mac even when it’s charging and in standby mode

Drag and drop files from your Mac to your iPhone

Automatically place windows in grid positions

Presenter Preview

Preview windows before sharing

Add background to your video

(Using MacOS native API)

Safari

(AI) Uses machine learning to detect and highlight information (music, people, movies, places)

(AI) Get article summaries and display auto-generated table of contents

game

Gaming Porting Toolkit 2 makes it easy to port your Windows games to Mac. Easily export your MacOS games to iOS.

Apple Intelligence AI

Personalized intelligence.

Important notifications automatically come to the foreground.

Help with rewriting emails. You can request rewrites based on the tone of the text.

Works with third-party apps, Mail, and Pages.

An image generated from a sketch based on a photo of a person in your photo library.

Perform actions across different apps: For example, if you ask to play a song a friend sent you in iMessage, Siri will find the message from that specific person and play the song using the Music app.

Understand personal context (from the app on your device).

Call recording and conversion to notes

Privacy-focused architecture.

On-device processing on iPhone, iPad, and Mac.

On-device semantic indexing to organize and find relevant information. Private Cloud Compute.

Fulfills server requests while respecting privacy and security.

If Apple intelligence determines that online processing is required, it will access Apple servers. No data will be stored.

If a server doesn’t meet the privacy standards, your iPhone will refuse to communicate with it.

Siri appears shining across the entire screen.

Siri can now understand previous context.

You can type to Siri at any time by double-tapping the bottom of the lock screen.

Siri can also answer questions about your Apple products, for example, ask how to send a scheduled iMessage or email and Siri will show you the steps.

Take action on the screen

You can ask Siri to show you photos.
For example, you can ask Siri to show you all photos of a particular person wearing a particular color of clothing in a particular place. You can also ask Siri to retouch images and apply different filters.

(API) Can be used by declaring the function within the app.

Siri can find and ask for information.

For example, if you ask a friend when their flight is arriving, it will automatically find the information by searching iMessages and emails.

Siri can summarize selected text.

Answer questions and help Siri generate a reply.
For example, in this case, Siri prompts you with an on-screen UI to choose between driving or taking an Uber, and generates a reply based on your answer.

Siri can summarize emails

(Not just the first few lines of the email).

You can integrate this feature into your app using our API.

Generate emoji images based on prompts.

Image Generation

The app can generate images by calling the Image Playground API.

Convert your sketch to an image

Remove people from the background

Ask ChatGPT (4o) a question

It can also help answer questions and write paragraphs in Pages.

Due to appear later this year.

Apps can declare their app’s intents. For example, a photo editor app can declare that it supports filters and allow the user to ask Siri to apply the filters.

Xcode can use on-device intelligence to perform on-device code completion.

Audio and Home

• Shake your head to reject a call, nod to answer
• Spatial Audio in Games (Personalized Spatial Audio API for Developers)
• Add music from TV shows to your library
• Subtitles improved
• Portrait TV Desktop Wallpaper, Apple TV+ shows can be set as wallpaper