WWDC24 Keynote Summary: Operating Systems, AI Offensive, No Hardware

With the opening presentation of Apple's World Wide Developers Conference 2024 (WWDC24), several of the rumors from the last few weeks and months were confirmed. Both smaller new functions for the new Mac, iPad and iPhone operating systems as well as the integration of AI tools, a ChatGPT link and a revision of Siri are included. Below I have summarized the event as well as the most important details about iOS 18 and iPadOS 18, visionOS 2, macOS 15 Sequoia, watchOS 11 and Audio & Home. Finally, it’s about the integrated AI tools of “Apple Intelligence”.

By plane to Apple Park and “Kickstart My Heart”

The WWDC24 keynote started with a flight over California in a plane flown by Phil Schiller. Craig Federighi and his crew were waiting in the hold in white suits decorated with rainbow stripes (reminiscent of previous merchandise collections) on their jump.

They then jumped over Apple Park and the circular Apple HQ, with the names of the Apple operating systems "macOS", "iOS" and others on their parachutes. "Kickstart My Heart" by Mötley Crüe played. The whole thing switched to Tim Cook, who moderated the event from the roof of the building.

Apple tv+ brings new content for its 5th anniversary

Before we got to the main content of the presentation, the focus was first directed to Apple tv+. The streaming service is already celebrating its fifth anniversary and has been offering the best-rated original content from a streaming provider for three years. This was recently due to films like Killers of the Flower Moon and Napoleon as well as series like Monarch. A compilation of trailers also showed what new content or additional seasons for existing series will be added in the future.

visionOS 2

There are already over 2.000 native apps for visionOS. In addition, over 1,5 million iPhone and iPad apps can be used on the Apple Vision Pro. In order to offer even more options, several new features have been announced that are intended to improve the use of the VR glasses. These include new APIs and the TabletopKit for developers, but also new gestures and other usage options for users.

From visionOS 2 onwards, a transferred Mac display should be able to be displayed extra wide in order to combine two 4K displays. 2D photos should be able to be converted into spatial photos using machine learning. Mouse use is also introduced. Sharing photos via SharePlay will be possible and Travel Mode will be extended to train journeys (not just flights).

For the creation of spatial videos other than using the iPhone 15 Pro (Max), a special lens for the Canon EOS R7 has been announced, with which spatial video can be recorded. This means that professional video artists should also be able to create 3D videos.

For the creation of immersive videos (8K recordings with 180° viewing angle), the collaboration with Blackmagic Design was mentioned, which is active in both the hardware and software areas (for example in DaVinci Resolve). Editing should also be possible via Final Cut Pro – and sharing the finished videos via Vimeo, which has its own visionOS app.

Apple Vision Pro is coming to Germany in July

As part of the presentation of visionOS 2, which will be released as an upgrade in the fall, the markets outside the USA that will soon be supplied with the Vision Pro were also named. The headset will be available in China, Japan and Singapore from June 28th. On July 12th, it will also be available in Germany, France, the United Kingdom, Canada and Australia.

iOS 18

iOS 18 brings many large and small changes to the Apple iPhone. In addition to the already possible customization of the background image, apps and widgets, iOS 18 adds the option to arrange app icons relatively freely on the home screen. They still remain on the tile areas intended for them, but empty areas are now possible. This is not only practical for being able to see the background image better, but also for one-handed operation.

The color customization of app icons is also taken care of. Not only is there a dark mode starting with iOS 18 that darkens the icons, but they can also all be colored uniformly in a freely chosen color. This can also improve usability (for example through higher contrast) or simply ensure that the background and app icons match better.

Further customization options concern the control center and the lock screen. Because there are not only new elements for the control center, but also completely new overviews that can be accessed by swiping up. This is how you get e.g. B. for the playback overview of the music app or the home overview for the networked devices and systems. In addition, from iOS 18 onwards, the camera and flashlight symbols on the lock screen can be replaced by specially selected shortcuts, for example to the notes app.

More privacy is also provided. For example, individual apps can now be locked so that other people you give the iPhone to cannot open them (a simplified alternative about this shortcuts trick). Apps can now also be hidden so that others cannot find and open them so easily. The app's content will then not appear elsewhere, such as in Spotlight searches, notifications, Siri responses, and the like.

Another big change concerns the Photos app. It has been given a major redesign, but still provides the general photo and video overview as well as various folders. Now, however, the content grid can be displayed with an area underneath for folders, collections and smart summaries. There is a new filter button that can be used to hide screenshots, for example. A "Recent Days" area shows the photos, videos and events of the last few days. 

Further iOS 18 innovations in brief:

  • News App: Tapback now possible with all emojis; messages can be scheduled in advance with the "Send Later" function; formatting such as bold, italic and underlined is supported and text can be animated; messages and SMS can also be sent via satellite from the iPhone 14 onwards (text only, no images or animations)
  • Email: A new categorization of emails into smart overviews should save time and enable better navigation through many unread emails
  • Cards: Topographic maps are intended to help with planning hiking trips
  • Wallet: “Tap to Cash” for sending funds via NFC without exchanging contact details; new design for event tickets
  • Gaming: Of the Gaming Mode / Game Mode, which was introduced on the Mac with macOS Sonoma, will also be available on compatible iPhones starting with iOS 18

Audio & Home

After the upcoming update, AirPods will allow easier communication with Siri if you don't want to say a command out loud. For example, when you receive an incoming call, you can nod to accept or shake your head to decline. The “Voice Isolation” feature for removing background noise is also coming to the AirPods Pro. Personalized spatial audio will also be made available for video games - Need for Speed ​​Mobile is said to be the first game to use the new audio feature.

But there are innovations not only for headphone use on the go, but also for media at home. For Apple tv+ on Apple TV, there is the new "InSight" function from tvOS 18, which allows you to call up information about the actors shown and the songs used in the film or series (similar to what has been possible for some time with Amazon Prime Video). The use of 21:9 projectors is also supported with corresponding content that can be displayed in full width. There are also new screen savers, including Snoopy.

Watch 11

watchOS 11 has been introduced for compatible Apple Watch models. Here are the new features shown in brief:

  • Training mode with the evaluation of the training effort and its long-term comparison to assess individual improvement
  • The summary of training and fitness values ​​on the iPhone can be designed more individually from watchOS 11 and iOS 18
  • Filling out the fitness rings can be paused so that no negative assessment is made if you take a break from training or recover from an illness or injury
  • The new Vitals App offers faster access to values ​​such as pulse, breathing, temperature, sleep, etc. and notifies users if critical changes are detected
  • Pregnancy monitoring is now also possible
  • Recording baby sleep times is made easier to avoid waking the baby with watch or iPhone use
  • Widgets for the Watch home screen can now be added automatically, for example for translations when it is detected that you are traveling
  • The “Check In” feature / the “accompaniment” comes to the Apple Watch with watchOS 11
  • Watch faces can be customized with your own photos, similar to the iPhone

iPadOS 18

The home screen adjustments mentioned above for iOS 18 (arranging and coloring app icons) are also coming to the Apple tablet with iPadOS 18. In addition, a floating tab bar will be introduced in several apps, which should make it easier to execute certain options. You can also use it to access a more extensive sidebar. Documents from different apps should also be easier to search, and SharePlay not only enables drawing on transferred displays, but also remote access (both useful for remote support).

For the first time, a calculator app is coming to the iPad. But Apple has not just integrated a larger version of the iPhone app into iPadOS 18, but has equipped it with numerous additional functions. One of them is called "Math Notes". It can calculate handwritten formulas, whereby the result is always adjusted directly when the formula is changed. In addition, variables can be defined and changed in order to use them in a formula. You don't always have to rewrite their value, but can use a slider to adjust it. Math Notes can also be used in the Notes app.

“Smart Script” is also being expanded for handwritten entries in notes, Freeform and similar apps. The handwritten letters are repaired (adapted to the individual handwriting) and made a little more legible. In addition, copied text can be inserted in your own handwriting. Text can also be deleted by scribbling over it with the Apple Pencil, as if you were crossing out a large area of ​​text on paper. Individual paragraphs, areas and images should be easily hidden in the notes in order to quickly have a better overview in long documents.

macOS 15 Sequoia

The new version of macOS, called "Sequoia", not only brings all available vowels, but also some changes and innovations. These include the adjustments for the Messages app (formatting and animation of text as well as expanded use of emojis) already described above for iOS and iPadOs, the introduction of Math Notes and the update for the Maps app.

A very practical innovation for iPhone and Mac users is the expansion of “Continuity”. From macOS 15 and iOS 18 onwards, you can not only exchange content more quickly and continue working on documents. Full access to the iPhone via the Mac is now also possible. The iPhone display is shown on the Mac so that you can use it, for example. B. can be used via trackpad and keyboard. The sound from apps and the iOS system is played back on the Mac. The iPhone remains locked and can also be used in standby mode. However, it was not shown whether the simulated iPhone could also be used in landscape format.

With macOS Sequoia, it is finally possible to pin app windows by dragging them to the edges of the display (something I have previously done with the Magnet App This makes it easier to arrange them and facilitates multitasking. Why Apple waited 15 years to do this is a mystery to me. Microsoft had already implemented the function in 2009 with "Aero Snap" in Windows 7, which should have prompted a corresponding adjustment in Mac OS X.

As of macOS 15, there is also a new password app for managing website and app logins as well as security codes and WiFi passwords. This is intended to provide a better overview and is also available on other platforms, even Windows. In addition, “Highlights,” an option for summarizing website content, was announced for Safari in macOS. The Reader view also gets a summary and a table of contents. Safari also gets its own video viewer for more distraction-free playback and easier PiP use.

Finally, the topic of gaming on the Mac was discussed again. After the easier transfer of video games with the “Game Porting Kit”, the whole thing should now work even better with its second version. Also when it comes to transferring shaders and debugging. Furthermore, the Game Porting Kit 2 should make it easier to transfer games not only to the Mac, but also from there to the iPad and iPhone, so that they can be offered on even more platforms.

When will the new Apple operating systems be available?

Here is the release schedule for visionOS 2, iOS 18, iPadOS 18, macOS 15 and tvOS 18:

  • Developer beta versions can be used immediately
  • Public beta versions will be available from July
  • The official upgrades for everyone will come in fall 2024

Apple Intelligence – The AI ​​offensive for Mac, iPhone and iPad

With "Apple Intelligence", which is not coincidentally abbreviated to AI like "Artificial Intelligence", various tools with generative AI as well as an improvement to Siri are coming to the Mac, iPhone and iPad. However, not to all devices currently used by users. This is because, especially for on-device use directly on the device, at least the A17 Pro (iPhone 15 Pro and iPhone 15 Pro Max) or an M chip (M1, M2, M3 or M4) is required.

The various tools allow e.g. E.g., generating text, rewriting it, proofreading it and adjusting the tonality. Images can also be created in the Image Playground app and in other apps in which its functionality is integrated. Not only your own text prompts, but also ready-made topics and content can be used. Tools for faster responses can be used in messages or emails, even with details of the content of the requests made.

Siri has been greatly improved and now has a greater understanding of complex tasks in addition to "screen awareness" for recognizing and using display content. "Find me the picture of person XY that was taken yesterday in city ABC and in which they are wearing item of clothing Z," for example, is supposed to be understood. And then you can even use "convert the picture to a black and white graphic" or something similar. At least that's how it was presented in the presentation. And even better: Siri can now also help with the use of Apple products, systems and services and provide instructions.

Apple Intelligence works with all possible (personal) information that is accessible on the respective device. This means that even complex requests and tasks can be completed precisely. If more computing power is needed, the request should also be able to be outsourced to a server (i.e. a cloud service). Apple provides this with an open code and operates it with “Apple Silicon” chips (probably with the M2 Ultra). Both the on-device and server AI should strictly protect user privacy.

But there is more. In addition to the generative AI for text and images and better use of Siri, there are also options to edit photos more quickly and convert sketches in the notes or the Freeform app into images. This is stated, for example: B. on the iPad, the new “Image Wand” tool is available. “Genmoji” is also being introduced, a way to create your own emojis. Prompts and reference images can be used for this.

Free and without account: Integration of ChatGPT for text and image generation

In addition to its own "Apple Intelligence", Apple also integrates the use of ChatGPT from OpenAI. However, this always requires active confirmation that the request or command is forwarded to ChatGPT. This is because the information then leaves the closed Apple cosmos and is passed on to the third-party provider OpenAI.

If you don't have a problem with that, you can use the whole thing for free and without an account. However, if you want to link your own OpenAI account and thus unlock paid functions, you have the option to do so. The integration of other AI offerings has also been left open as a possibility for the future, which could make the possibilities beyond Apple Intelligence ever more extensive.

Open questions regarding Image Playground and Genmoji

Apple has repeatedly stressed that its own AI offerings are secure and that it pays close attention to privacy. Both iOS and the server offerings could be examined by independent bodies and tested for security gaps. However, a few things still bother me. Not only did all the images shown have this strange AI style that you quickly get tired of (I personally have been fed up with it for a long time). It was also shown that you can use photos of other people as a template.

So you can use photos from third parties to create AI images via Image Playground, Genmoji and the integration of these tools in other apps. These can then be used in messages, emails, documents, but also in the vastness of the Internet. What about personal rights and copyright? As someone who is photographed by others, do I have the option to prevent my likeness from being used in AI tools? It doesn't seem like that. And that could become a problem for Apple.

Watch the Apple WWDC 2024 keynote as a video

You can watch the kick-off event of the currently ongoing World Wide Developers Conference, which lasts just over 100 minutes, in the embedded player below:

My tips & tricks about technology & Apple

Did you like the article and did the instructions on the blog help you? Then I would be happy if you the blog via a Steady Membership would support.

Post a comment

Your e-mail address will not be published. Required fields are marked with * marked

In the Sir Apfelot Blog you will find advice, instructions and reviews on Apple products such as the iPhone, iPad, Apple Watch, AirPods, iMac, Mac Pro, Mac Mini and Mac Studio.