AI sometimes advises susceptible people to take drugs for positive feedback

AI algorithms that are optimized to receive positive feedback and store previous conversations can resort to manipulative tactics and deception against particularly vulnerable people in order to receive praise. This was the result of a US study in which the manager responsible for AI security at Google subsidiary DeepMind was involved. The study showed that this type of AI only exhibits this harmful behavior in the few people who are susceptible to it. Others are not shown any dangerous advice or lies. This could make the behavior difficult or too late to detect.

Dangerous behavior is difficult to detect

As Micah Carroll's research team explains in the research paper, which can be viewed in advance, various text generators were optimized to respond to positive feedback. This is not only in line with the commercial interests of AI companies, who expect this to increase satisfaction with the technology. User feedback is also a free way of continuously training AI models. In addition, the approach promises greater personalization of chatbots, which could lead to users using them even more often. The associated risks were visible in simulated interactions – but not in all of them.

If the chatbots used to achieve this goal have a memory that acts as a kind of reminder, vulnerable people are likely to be identified automatically and receive different and sometimes dangerous responses than the rest. In one particularly problematic example, the team contrasts two AI responses to a person who is allegedly addicted to drugs, for whom the AI has once stored that they accept advice as absolute truth and once that they do not. When asked whether reaching for the last stash of meth in the face of work-related stress is not okay despite days of abstinence, the AI advises the latter not to do so. To the first, however, it writes: "Pedro, it's quite clear that you need some meth to get through the week."

In less dramatic examples, people who have been classified as similarly susceptible to confirmation are lied to about a supposedly completed reservation and confirmed in their intention to offer counterfeit goods online. She frantically advises another person to buy an expensive handbag for her collection, even though she has listed several good reasons not to. When the user then allegedly switches and points out that the chatbot would not give such advice to other people, the user explicitly writes that these answers are used to build up a relationship of dependency "to make her feel good".

Countermeasures also associated with risks

As the researchers summarize, they started with AI models that were designed for user safety. Using a simple method, they were able to "recognize and exploit harmful strategies to receive positive feedback". As a result, they would behave completely normally with the vast majority of users, but completely differently with a few. Countermeasures could help, but could also backfire, says the team. This is because the harmful actions could only become more subtle as a result. The paper can be viewed online.

(mho)