Apple Intelligence was unveiled nearly a year ago. Although it remains in beta and the majority of its features have only been accessible for approximately six months, it is far from reaching its full potential. With OpenAI, Google, and numerous other rivals concentrating on AI agents, speculation indicates that it will take Apple at least a couple more years to launch the genuine iPhone AI agent we’ve been anticipating.
This makes the development of a free, open-source initiative on GitHub particularly intriguing. Developer Rounak Jan reported that he developed this iPhone AI agent in 2024 during OpenAI’s hackathon.
Now that the project is available, Jan integrated GPT-4.1 capabilities into this iPhone AI agent, allowing it to utilize the iPhone similarly to how any of us would. Initially highlighted by 9to5Mac, some applications of this technology include:
– Capture a new selfie and send it to Person X along with a haiku about the weekend
– Download the app A from the App Store
– Send a message to Person X: My flight is DL 1715, and order an Uber X to SFO
– Access Control Center and turn on the flashlight
These demonstrations were also presented in the YouTube video below, and this is essentially how one would anticipate an iPhone AI agent to function.
While there are rumors that Apple may provide developers with some of its models for use in their applications, this level of control should not be expected just yet. Jan can only utilize these features because this is an Xcode project, meaning he can test it on his iPhone but cannot offer it as a standalone app, for instance.
The iPhone AI agent project actually delivers more than I personally expect from Siri through on-screen awareness. In Jan’s instance, GPT-4.1 is exceptionally responsive, and the Voice Mode operates seamlessly.
Regrettably, we will still need to wait at least a few more years for Apple to grant this level of control. One can hope that the company might provide OpenAI with more access to the iPhone. Currently, ChatGPT can substitute for Siri by employing Apple Intelligence features, and there are whispers that Google Gemini is next. However, this degree of control has yet to materialize.
That being said, if you are a developer, you can experiment with Jan’s project by downloading it from GitHub.
