ProSiebenSat.1. Digital Data cooperates with Berklee Valencia on data and AI topics
ProSiebenSat.1 Digital Data (short: PSDD) is a 100% subsidiary of ProSiebenSat.1 Media SE, which sets its focus on data-driven products and artificial Intelligence (AI). Berklee College of Music, Valencia Campus is the European branch of Berklee in Boston, USA. Alexandre Perrin, Associate Professor at Berklee Valencia, who initiated the module “Data Analytics in the Music Industry” in the Master of Arts in Global Entertainment and Music Business, received his doctorate in 2008 in Management Science while attending University of Nice, France and taught at Audencia Business School and EDHEC Business school before he started to work for Berklee in 2013. The following collaboration between Berklee Valencia and PSDD was led by Stefanie John, Associate Director Data Strategy & Sourcing, in the team of Fahim Alefi, VP Data Strategy and Sourcing. Stefanie John, who has worked in the entertainment and music business focusing on data-related topics since 2009, joined PSDD in 2019. Generating new impulses the project is considered promising due to collaboration of digital natives with a leading entertainment house. From January through May 2020 graduate students in ‘Global Entertainment and Music Business’ attending the class “Data Analytics in the Music Industry“ worked on topics with a strong data and AI alignment.
During theoretical and practical stages the students tackled a problem and contributed to finding innovative solutions. PSDD selected the topics, but didn’t set too many limitations, letting the students pick their own focus area. As the second stage also covered technological parameters, the students were given several data sets to firstly understand data available and secondly to define data sources to be added. In total 5 groups of 4 students worked on the 4 different subjects. In biweekly video calls PSDD provided guidance and continuously challenged results. The students delivered their results using data analytics software to visualize and interpret data and packaged them within a presentation:
1. Music, entertainment & augmented reality (AR):
AR is a subset of computer vision that enables us to combine reality with digital data, creating a completely new content experience. AR can be an entertainment feature which is already applied, e.g. in ProSiebenSat.1’s show Galileo to visualize ideas, technology, etc. Consumers get further information and can interact or even become protagonists themselves in an innovative way.
Group 1 reviewed solutions in the market, looking at big US players such as USA Network, Nickelodeon and Niantic, the company behind ‘Pokemon Go’. Having identified several interaction mechanisms as well as underlying technologies, they evaluated and prioritized those relevant to ProSiebenSat.1’s apps. Most promising were those features that are built around facial recognition, location mapping and gamification approaches. Subsequently the students decided to develop further app features for The Masked Singer to be used during and after the live shows on linear TV. After analyzing the given data set they could prove the relevance and strategic priority of this technology in general and current trends in its respective context. They drafted innovative face filters, trivia games and an Avatar creator for The Masked Singer.
2. Content strategy & trend detection:
A big challenge is to identify potential trends early and therefore predict what content to source or produce. Social Buzz can provide an indicator for future success. What matters to different target groups might appear in social networks such as Twitter, Facebook, Instagram, TikTok, YouTube, etc. — an entertainment company needs to be aware of these trends at a very early stage and therefore understand what led to success in the past and what is going to be successful in the future.
Group 2 decided to set themselves the target of identifying content with wide mass appeal rather than focusing on niche markets. They analyzed potential data types from three areas: socio-demographics, overarching content themes and interactions. Their methodology was a smart combination of these data sets to identify patterns in consumption and connect them with main audiences of corresponding broadcasting stations. They broke down all the data types required and analyzed a Social Listening data set. As a result, the students described an algorithm for creating a dashboard that identifies and forecasts trends for ‘the new mainstream’. Applying KPIs like e.g. virality score, the final dashboard draft visualizes different dimensions of content such as level of emotional interaction, type of engagement or preferences of visual content (colors, animation, style, etc.)
3. AI based content production
Rules and patterns of production can be recognized and applied using AI. How production is influenced in the future by natural language processing that analyzes, understands and generates human languages is still not defined. Will AI be the driver or still human beings? There might be limits in terms of creativity, but an AI can certainly function as a support system for a producer.
Group 3 looked into existing technologies from other players and markets and applied their approach to FYEO, the recently launched audio streaming platform by ProSiebenSat.1. Spotify’s EchoNest categories that help to characterize a song (speechiness, valence, etc.) were analyzed to develop a proprietary audio fingerprinting for podcasts. They identified various categories such as: host (solo vs. co-hosted), interview vs. story, speaker gender(s), pitch, pace, loudness, genre, topics, hashtag words, mood and sponsorship. The students suggested a spectrogram analysis to show different audio features and combine it with language content features such as genre, topic, sponsors, etc. They built a recommendation algorithm for FYEO users consisting of demographics, user behavior and sonic metadata of podcasts. As a final step the students defined how to use that information for assistance in production. A creator would get insights about most relevant characteristics for a certain audience and vice versa. In the future natural language processing could create a completely AI based script for a new audio format.
4. AI based content individualization
What has already been established in advertising over the past decade is still new in terms of content: personalization and individualization. How can content — e.g. a news format or a movie — be individualized or personalized? Consumer, socio-demographic or behavioral data can help to anticipate personal needs and should therefore be taken into consideration for an AI-based solution. A real-time interaction can optimize an individualized experience.
This topic was executed by two groups: Group 4 focused on a self-curated TV channel; Group 5 worked on options for Joyn, a video-on-demand service which is a joint venture of Discovery Communications and ProSiebenSat.1.
Group 4 chose the approach to define personas anticipating their high need of individualization mechanics. They identified: Older people and such with health impairments, busy parents with young children as well as TV or video fanatics. Based on their known preferences and behavior, an AI should be gathering and analyzing data to curate a dedicated ‘TV playlist,’ choosing the best fitting content in the optimum time sequence. Add-ons could be a child lock to prevent changing the program, as well as an app, skill or desktop version to expand market and business cooperation with mobility companies. In the practical stage they examined channel programming data. They analyzed ProSieben’s TV grid to understand anticipated consumer needs. An algorithm that used these as a basis and combined it with actual consumer behavior could program your personal TV channel. This channel would improve over time using individualization features offering e.g. content with a match rate of above 90%. An AI would improve this self-curated channel by analyzing behavior as well as cross-relations between contents and consumers of different contents.
Group 5 defined a three-way approach for Joyn customers to further improve their user experience applying AI algorithms. They also looked into other entertainment players’ solutions such as HBO and Netflix, Spotify or movie recommendation apps such as Mubi or Tinsel and elaborated on the following approaches: the first one was individual content engagement which requires a software to follow complex storylines and produce different versions for interactive content; the second idea was based on social features to engage with content, a new trait is an AI as a recommendation behind these features. As a third option this group suggested a skill for a voice-driven device to ask the consumer relevant questions, e.g. about their current mood, and make recommendations to them based on a small selection of movies or TV series. They worked further on content individualization by engagement and by curation via voice-driven devices. Several storylines are created so that different proceedings and endings can be generated. Based on this every consumer could interact with a movie and create his or her own individualized content experience. By understanding metadata of a TV grid, they broke down parameters to select from in a user’s conversation with Alexa and Co.
This cooperation emphasized PSDD is focusing on highly relevant future topics. Their chosen focus areas and most ideas are very much in line with existing ProSiebenSat.1’s innovation approaches. The students created also new impulses like the AR Avatars, the new mainstream dashboard, audio fingerprinting for podcasts, self-created TV channels or a movie recommendation skill. While trend detection is one topic PSDD is already working on, they will liaise with corresponding business units to evaluate opportunities of the other ideas. The students appreciated working on ‘real-world’ subjects facing opportunities and challenges of a leading European entertainment player in 2020. They learned how to approach topics from a data strategy perspective as well as the role and power of data and AI in general. Getting access to actual data and learning how to process them gave them a deep understanding and knowledge for their future careers. They also got the chance to connect with ProSiebenSat.1’s HR team. Next year’s run could focus more on trend detection and further AI topics. As new technologies and platforms quickly evolve in this highly dynamic, digital world, PSDD should introduce other cutting-edge topics of 2021. Who knows if we will need entertainment for self-driving-cars soon, AI that easily detects fake video material or robots that do stunts in our shows?
Stay tuned.
