During a workshop with the European Broadcasting Union (EBU), the Danish Broadcasting Corporation (DR) and Media Distillery presented the results of a Proof-of-Concept on Binge Markers: the technology that automatically detects the end credits in video playback, which viewers may utilise to skip the end credits of a show while binge-viewing to enable a smooth, quick transition between programmes. DR has identified this aspect of television viewership as an area to improve upon within their market, and discussed with Media Distillery both the challenges and possible solutions that lay ahead.
Read below as Danish Broadcast Corporation’s Birgitte Stannius (Consultant, DR Enterprise Architecture) and Martin Prins of Media Distillery (Head of Product) answer questions regarding the use of Binge Markers, the preliminary results of working together, the current state of the technology, business considerations, and more.
Q: What are the main challenges for a broadcaster in this connected digital world?
Birgitte Stannius, DR:
For a public broadcaster, the transformation to public service media is a challenge. Not only programme formats, but also the presentation, the way it’s disseminated and consumed, and marketing are aspects that must be rethought.
Q: What was the main reason to look for a solution for skipping credits?
Birgitte Stannius, DR:
We relaunched DR’s streaming service – DRTV – in 2019 as part of a larger strategic effort. DR´s goal with this and other actions is to increase the usage of DRTV. At the moment, four-out-of-ten Danes use DRTV weekly. However, the goal is that six-out-of-ten Danes use the service. One of the actions we are looking for is that the viewer/user remains engaged in watching another programme after the programme the viewer is currently watching comes to an end. Basically, we need solutions to retain the viewer on our service by offering them programmes when he or she is most receptive to suggestions. A Binge Marker identifies a point in time where the viewer is receptive to suggestions, and is most likely to remain tuned in.
Q: What approaches did you consider?
Birgitte Stannius, DR:
Currently, we are using a solution where the view of the programme is shrunk five seconds before it ends – while offering a new programme. This is a safe way – the user will presumably not miss any of the content. However, it is not a particularly effective way to do it – especially not if you compare to the average duration of an end credit.
Q: How could Binge Markers be created?
Birgitte Stannius, DR:
Binge Markers could be created manually as part of different production and application processes; e.g., in the editing process or in the quality control of the programme before publishing. You could also consider making a separate process – where you update your catalogue of active programmes manually with Binge Markers. However, this means changes in current processes and development of technical support of this new task.
Martin Prins, Media Distillery:
While you could create Binge Markers manually, that would require people to view all videos, scroll until the end, and note the time at which the credits start. This does not scale well for large collections of video or where fast processing is needed, and it is quite tedious.
But with advances in Artificial Intelligence and Computer Vision, it is now feasible to automate this by training an AI model, or a combination of AI models, to detect end credit sequences. This can be achieved by looking at several features of a video. For example, the appearance and amount of text, what the text says, or by looking at the audio, the start of music, or repetition of a sequence. We opted to go for the visual aspects, the main reason being we are already using this for other products, but another being: not all videos have audio during the credits, so it may not be a reliable source.
Q: From a tech point of view, what are the advantages of Binge Markers?
Birgitte Stannius, DR:
You don’t have to incorporate a new task with existing processes or create a new process. This can be complicated, e.g. if you choose to get data from the editing process you have to bear in mind that you have different sources – internal/external production – and it could be difficult to build a streamlined production chain for the Binge Markers.
Martin Prins, Media Distillery:
We see several advantages: Because of automation, it can be scaled easily. It doesn’t matter if you need to create end credits for hundreds of programmes or thousands. Additionally, it can be performed really quickly and at any time.
Q: What are the main outcomes of your partnership so far?
Birgitte Stannius, DR:
We have really gained some insight into the preliminary quality, the potential, and pitfalls for AI/ML. We have become more aware of the way programmes are designed, specifically the way they end, and provided substance for reflection on how Binge Markers should be used in relation to different types of end credit.
Martin Prins, Media Distillery:
In the PoC, DR asked us to evaluate end credit detections on a variety of content. What we discovered was that we needed a large pool of examples to evaluate accuracy – where the types of end credits varied. Looking at it from a user experience perspective, any detections before the start of the end credits could lead to a “skip credits” appearing during the cliffhanger of a programme, and ideally this is something we want to prevent.
Looking at all the videos, our end credit detection was successful 85% of the time, which is pretty good, but also shows there is room for improvement. For example, we ran into some challenges with our AI model identifying text in the background of the programme as an end credit sequence, mistakenly identifying a textual epilogue as an end credit sequence, detecting end credits when there were none, and identifying a proper end credit when there was additional content playing subsequent to the initial end credits. Think of a blooper reel that begins shortly after a movie ends, and after the credits start rolling.
Now we are looking to improve the results and expect to be able to increase the accuracy to 95%, especially if we have more data about the content. As it relates to DR’s current solution with the shrunken view five seconds before the end of the programme, Media Distillery’s Binge Marker solution can solve DR’s viewer retention issue by identifying the exact point to run a new programme, thereby increasing the likelihood that the viewer remains engaged with the content and tuned in to the channel.
Of course, implementing this advanced solution on a scalable level, and at pace, comes with its challenges, but Media Distillery has always accepted these challenges to innovate and improve, while helping TV operators enhance their consumer’s experience. We have already enabled this for catch-up and Replay TV for millions of TV viewers with our EPG Correction product; now we want to do the same for Video on Demand.
