Duolingo about AI video calls with Lily and more
Duolingo is expanding its language learning program with additional functions such as an AI conversation partner.
(Image: Duolingo)
Duolingo is expanding its language learning offering with more AI-supported functions, such as video calls and adventures. Duolingo will present the new features during Duocon 2024.
With the video call function, users of the new Duolingo Max subscription can have spontaneous and realistic conversations with the character Lily. There are already similar offers to learn languages with an AI language trainer. Lily is not intended to replace existing Duolingo functions, but rather to supplement the existing offering.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
According to the manufacturers, Lily automatically adapts to the learner's language level. The function is intended to boost learners' self-confidence. Feedback is therefore provided after the conversation, according to Duolingo. Contact with Lily is initially available on iOS in English, Spanish and French, with German, Italian, Chinese, and Japanese coming later.
Language learning adventures
The Adventures feature will also be introduced, allowing language learners to interact with Duolingo characters in various environments on both Android and iOS. The adventures are designed to encourage language use in realistic contexts, such as when ordering coffee.
In addition to the linguistic innovations, a compact electric piano is being developed in collaboration with Loog, a manufacturer of portable musical instruments, which will be available for 249 US dollars. Learners will then be able to take music lessons on an electronic piano. A partnership with Sony Music makes it possible to practise well-known music. A version for Android devices is also planned for the future.
(Image:Â Duolingo)
We spoke about Lily with Zan Gilani, the head of the 'Experimential AI' team at Duolingo, which works with generative AI.
heise online: What is Lily based on?
Zan Gilani: We work with OpenAI, and we use their latest models – GPT-4o – to power this experience where you talk to Lily. We don't do any fine-tuning at the end of the process. We use the models directly.
That doesn't mean that it's basically just ChatGPT inside Duolingo. We have built other parts of the technology stack to make this very specific. We gave Lily a memory. So when we first tried it, Lily didn't remember anything.
So we built our own system where Lily remembers, and she tries to remember the things that are relevant to you and what you are interested in talking about and then brings them up in the future. It's like your memory is only used for your conversations. It only works for your account. We don't train a model on the data.
Are you individualizing her?
The calls that you've had with Lily, we'll go back, and then we'll get GPT to say, okay, out of your last 50 calls, extract the key learnings. Then you find out something about the person you're talking to. For example: 'Where do you come from, what do you want to talk about?' And then, in subsequent calls, we use this information again and say, for example Today you are going to ask Marie about something from your past conversations that is still active, about her life, or something like that. For example, in a recent conversation she said she wanted to go on a trip, ask her about it. That's how it works, so it's just based on the contact. All the conversations you've had in the past.
Are there any safeguards?
There are several safeguards. Content moderation is important to us. Duolingo is also for different age groups. So we have to make sure that there is content moderation after every turn, basically every time you say something.
We check to make sure it's not inappropriate and if it is, we end the conversation. We also have safeguards to make sure that Lily doesn't say something inappropriate. People are very creative in trying to get Lily or other chatbots to say something bad.
Do you have any difficulties implementing Lily in other languages?
Yes. For example, we already have a basic version of Video Call in Korean. It's not that good because Korean has a very strong sense of hierarchy and formality. So you have to use the right level of formality when you're talking to people. And before we looked at Korean specifically, we tried Video Call in Korean, and the conversation is all over the place because of the formality rules. We want to keep the conversation flowing so we are working on addressing these unique challenges across a few languages with specialized grammar rules.
Does Lily give feedback?
For pedagogical reasons, Lily does not give feedback during the conversation. This could disrupt the flow of the conversation. The speaker would think more about possible mistakes and corrections and would not be able to speak freely. Some constructive feedback will follow the conversation.
Will Lily change her mood? She seems depressed.
We have built a cast of characters around Duo, our owl mascot. Lily is our purple-haired, emo, moody teenage girl. We chose her for this Video Call feature because our learners love her sarcasm and irony, and she's very popular across social media in many countries. So we won't touch her personality. But yes, we are thinking about some variance.
And it's better than trying to talk to strangers on the internet, I suppose?
The people who do it are very impressive because it takes a lot of confidence to do it and I think most people are just a bit shy and a bit nervous about putting themselves out there. We hope that if you do a lot of this, it gives you the confidence to say: 'Now I'm going to find the native speakers and when I travel I'm going to order my coffee in Paris and not be embarrassed that they're going to make fun of my French'. You know, that's the hope here.
Do you use Duolingo?
Yes, I started a German course to see if I could learn a new language.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
(mack)
