New AI sorting: Apple Mail praises phishing messages
The controversial AI functions in Apple Mail can also be a security risk under certain circumstances. Malicious messages are sometimes flushed to the top.
Phishing mail is "important": Apple's mail categorization sometimes misses the mark.
(Image: Rafael Zeier / YouTube)
The automatic sorting of messages in Apple Mail introduced with iOS 18.4 and macOS 15.4, which is activated independently of Apple Intelligence, not only causes confusion among users due to incorrect categorization, but can also push spam and phishing messages to the top. This was discovered by Swiss YouTuber and technology journalist Rafael Zeier. An alleged "refund" of 376 francs landed at the top of the "Important" inbox of Apple Mail on the iPhone. At the same time, Apple Intelligence also prioritized the message.
"Time-critical transaction"
All of this makes it difficult for users to recognize a scam message as such. If they trust Apple Mail's sorting, they may click on it more quickly. When the message is opened, even more pressure is built up – by Apple Mail itself. "Time-sensitive transaction" with a shopping icon appeared in Zeier's spam message. Interestingly, there is no indication in the mail itself as to which company is involved. "Your refund is waiting: Act now!" it just says, plus the amount of money and a link, as well as "Delays on your part may lead to unnecessary complications".
Videos by heise
If you click on the email, you are taken to a strange URL (which does not match the email) that pretends to be the website of an insurance company. You are then supposed to enter your customer information there, which the attackers can then use to commit fraud. In Apple Mail, a URL can be detected by clicking on it for a longer period of time, after which a preview including the address appears. There is no way to detect the URL just by moving the mouse pointer over it, as is the case in Apple Mail for macOS, as there is no mouse operation on the iPhone.
Apple needs better filters – and more training
The incident shows that Apple clearly has some catching up to do with its AI systems. Phishing emails like this should actually be recognized and stopped by spam filters (which exist in iCloud Mail, Gmail and many companies). However, when it does go through, Apple's language model sometimes seems to tend to misinterpret messages.
Since in this case, there are many features that indicate phishing (such as no indication of the company, pressure build-up, strange/inappropriate URL), it is very surprising that the categorization as "Important" (plus prioritization in Apple Intelligence) works. Apple is currently trying to obtain better training data, but this has been criticized by data protectionists. Categories in Apple Mail can be turned off. The feature was first introduced in English in iOS 18.2 / macOS 15.2. By clicking on "All mails", you will not see the categories; alternatively, you can simply reactivate the classic list display using the button with the three dots.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externer Preisvergleich (heise Preisvergleich) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (heise Preisvergleich) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
(bsc)
