Before Google’s developer conference on Tuesday, Apple revealed new accessibility updates enhanced by Apple Intelligence for features such as VoiceOver, voice control, live recognition, and real-time video captions. Additionally, Vision Pro users will now be able to operate compatible wheelchairs using eye control with this update.
Apple stated that the image recognition feature in VoiceOver will offer a more detailed understanding of images, like reading a bill’s amount and due date, and will better describe photos and personal records.
Users can activate the Live Recognition feature using an iPhone, which utilizes the camera to identify content in the frame and allows follow-up questions for more details. Magnifier can be assigned to the action button for a high-contrast interface, and voice commands such as “zoom in” or “turn on flashlight” can be used to access features.
Apple’s voice command updates extend beyond the Magnifier app. Users can now describe tasks in natural language for actions on their screen. For example, in Apple Maps, they might say, “tap the guide about best restaurants,” or in Files, “tap the purple folder.”
The Reader feature has been improved to handle documents like scientific papers with columns, images, and tables. Users can receive AI-generated summaries or read text in the original language with custom fonts and colors retained. The update caters to disabilities such as dyslexia and low vision.
Furthermore, Apple is introducing AI-generated subtitles for videos without pre-existing captions, including iPhone-recorded videos or clips from friends or family. These subtitles are compatible across iPhone, iPad, Mac, Apple TV, and Apple Vision Pro, and users can manage their appearance.
Apple also announced a feature for Vision Pro users to control compatible wheelchairs with their eyes, functioning under various lighting conditions without recalibration. This will launch with Tolt and LUCI alternative drive systems in the U.S., supporting Bluetooth and wired connections.
The name recognition feature, alerting users with hearing disabilities when someone says their name, now supports 50 languages. Additionally, Apple will introduce large text support to tvOS, and Made for iPhone hearing aids will improve their device handoff capability.
These features are expected to be released to users later this year, likely as part of the upcoming iOS 27 release.