US-based entertainment technology and audience insights firm TiVo has lifted the lid on its award-winning TiVo Conversation product*, which allows TiVo-equipped users to search and discover entertainment content by voice using a microphone-equipped remote control.
Charles Dawes, Senior Director, International Marketing at TiVo, explained that the integration of machine learning (ML), an AI technique, into the product responded to the fact that “we see more and more content coming, and one of the challenges is actually having the ability to be able to generate the metadata for that content.” Traditionally, this work has been done by human beings, but “actually that’s not practical and it’s not cost-effective,” says Dawes. So “how do we use machine learning and semantic graphs [which map the semantic relationships between concepts] to actually auto-generate keywords and generate the right keywords?”
The answer to this question was the topic of a technical paper one of TiVo’s senior engineers, Lijin Chungapalli, presented to IBC delegates this year. It transpires that one of the techniques involves the use of an AI using ML to analyse a ‘long-form plot synopsis’, which, for instance, would detail every scene in a movie and run to thousands of words, before deciding which are the appropriate keywords to extract. “It’s able to understand which ones are important, the relevancy of those and actually which ones you want to discard,” says Dawes.
An example of one of the skills exhibited by the system is the ability to contextually replace pronouns in a sentence such as ‘Jack is a doctor; he helped Mary,” by understanding that ‘he’ refers back to ‘Jack’.
Meanwhile, at the user end of TiVo Conversation Services, a technology called NLU (for Natural Language Understanding) is used to interpret what the viewer says into the remote’s mike. Dawes notes that the system doesn’t just recognise the user’s voice and translate their words into a piece of text, but goes on to ask “What do the words mean in context? What is the intent of the words?”
He illustrates this with the word ‘play’: “If you say, ‘play the movie’, that’s a very direct command: do you want to play a particular movie? If you say, ‘when do Arsenal next play?’, actually, you’re asking a question. You’re not saying, ‘play me the Arsenal game from yesterday’ – you’re saying, ‘when’s the next game on?’ Also, ‘play’ is another word that can mean ‘play’ as what happens at a theatre. Understanding exactly what ‘play’ means in the context of the sentence and everything – that’s the additional value we’re bringing on top of just having a system that can translate a word.”
Dawes reveals that TiVo sees “24% more usage of conversational systems like ours compared to systems that just take a word and have it as a word and don’t really understand what that word means in context.”
In a subsequent demonstration on TiVo’s IBC booth of its Next-Gen user interface – the latest version of which integrates TiVo Conversation – Dawes uses the example of a viewer forgetting the title of a film they wanted to watch (in this case the 2000 film Castaway), remembering only that it involved ‘Tom Hanks with a volleyball’.
Dawes speaks this phrase into the remote’s mic, and the UI successfully retrieves the film Castaway. However, Dawes notes that because of the high background noise level on the booth, the remote misunderstood the phrase as ‘Hanks in a volleyball’. “A lot of systems would go ‘what on earth do you mean?’,” notes Dawes. “But because we have that innate knowledge of the entertainment piece and we have those techniques that generate keywords, [it still knows that] actually Tom Hanks is the star of this film and he’s on the island with this volleyball.”
In May, TiVo potentially removed the need for users to be restricted to the mic in the remote control by offering support for Amazon’s Alexa technology in its DVR line-up – allowing the smart speaker to interact with the box in the same way using so-called ‘far-field’ voice control.
* TiVo’s Conversation Services won the UX Award at the Content Innovation Awards, held on October 15, 2017 in Cannes
