Apple is making significant strides with its latest research breakthrough: ReaLM (Reference Resolution As Language Modeling). This innovation promises to propel Siri into a realm of understanding that surpasses mere image recognition, potentially revolutionizing how we interact with voice assistants.
Spoiler alert: It already beats GPT-4, according to Apple’s benchmark tests.
| Model | Conv | Synth | Screen | Unseen |
|---|---|---|---|---|
| MARRS | 92.1 | 99.4 | 83.5 | 84.5 |
| GPT-3.5 | 84.1 | 34.2 | 74.1 | 67.5 |
| GPT-4 | 97.0 | 58.7 | 90.1 | 98.4 |
| ReALM-80M | 96.7 | 99.5 | 88.9 | 99.3 |
| ReALM-250M | 97.8 | 99.8 | 90.6 | 97.2 |
| ReALM-1B | 97.9 | 99.7 | 91.4 | 94.8 |
| ReALM-3B | 97.9 | 99.8 | 93.0 | 97.8 |
ReaLM by Apple: Revolutionizing Siri’s intelligence
According to its paper, Apple’s ReaLM is like giving Siri a super brain. It helps Siri understand what you’re saying in a much smarter way. Here are its early impressions about it:
- Looking at your screen: ReaLM checks out what’s on your screen, like texts or pictures, to understand what you might need help with.
- Listening to your conversations: It also pays attention to what you’re talking about with Siri. So, if you mention something earlier, Siri will remember and help you better.
- Noticing what’s happening around you: ReaLM is like having eyes and ears everywhere. It can tell if there’s an alarm going off or music playing, and adjust its responses accordingly. Here is an example dialogue from the paper:
| Speaker | Dialogue |
|---|---|
| User | Show me pharmacies near me. |
| Agent | Here is a list I found. |
| Agent | … (list presented) |
| User | (eg 1) Call the one on Rainbow Rd. |
| User | (eg 2) Call the bottom one. |
| User | (eg 3) Call this number (present onscreen). |
- Being really good at its job: Apple tested ReaLM against other smart systems and found out it’s really good at figuring things out, even better than some others out there.
- Working right on your device: ReaLM does all this smart features right on your iPhone or other Apple devices. That means your information stays private, and everything works super fast.
In simple terms, ReaLM will make Siri much smarter by helping you understand more about what you’re doing and saying.
The release date for Apple’s ReaLM hasn’t been announced yet. However, it’s possible that elements of ReaLM could be integrated into future updates of Apple’s software, such as iOS or macOS. We might see more information during events like the Worldwide Developers Conference (WWDC) in June. Keep an eye out for updates!
Featured image credit: cottonbro studio/Pexels
