by Myrthe Reuver & Nicolas Mattis | Reading Time: 7 Minutes
Note: This blog is based on the hackathon work by the authors as well as Marijn Sax, Felicia Loecherbach and Sanne Vrijenhoek during the Media Hack Day Hackathon on Diverse Recommenders on 9-10 October 2020 . Felicia wrote a personal blog about her experiences at this hackathon as well.
Who doesn’t want to be a hacker? The word Hackathon, which combines the words hacker and marathon, might evoke images of nerdy looking guys in hoodies, coding away with green letters on a large black screen. However, we were hackers not conforming to this stereotype: our most experienced programmers were the women in the team, and we were not primarily coders but problem-solvers from different scientific domains (from communication science, philosophy, computer science and computational linguistics). Moreover, we did not hack banks, governments or elections. We hacked a societal problem: a lack of diversity in news recommendation, in a 24-hour hackathon organized by public broadcaster NPO! The public broadcaster has their own “Netflix”-like recommendation system: a video portal where users can watch public TV programs and documentaries. With “diverse recommendation”, the NPO meant users of this system would get recommended different items than ones only based on their personalized preferences. This means, for instance, users getting recommendations on different topics, genres, and ideas than their usual preference of NPO videos.
This is especially important because non-diverse (news) recommender systems can actually damage democracy, leading to online filter bubbles or echo chambers. For instance, personalization of online news can lead to the user only consuming news that immediately interests them: about football, or only about one perspective on the U.S. elections. Such a limited perspective harms the deliberative aspect of a functioning democracy, where different perspectives and ideas should be heard and debated.
We attempted to diversify the NPO Start recommendation system in a 24-hour hackathon and learned much and more not only about diverse recommendation, but also about science and interdisciplinary collaboration!
Why is diversity in recommender systems important?
Diversity in recommender systems is a ‘hot topic’ nowadays. You might know it as the “fabeltjesfuik” recently discussed in the Dutch television show ‘Zondag met Lubach’, or as talk about Filter Bubbles (Pariser, 2011) and echo chambers (Sunstein, 2009). A system that only gives users more of the same content can lead to polarization or a very limited perspective on current issues. This can be dangerous for democratic participation in a society (Helberger, 2019). For instance, a sports fan only gets more recommended news articles or videos on sports, but new topics and other ideas are excluded from this user’s news consumption.
As a public broadcaster, the NPO is ultimately driven by the public value of diversity and providing a public discussion space. And from this perspective, it is not hard to see the need for diverse news exposure. Only when we, as members of a society, are aware of the different voices and opinions that are out there we can constructively discuss and decide on the best way forward.
Therefore, NPO asked 5 teams to design a diverse recommender system for their video platform NPO Plus. Instead of getting more recommended news based on one’s personal preferences, users should have the opportunity to encounter diverse content. We participated as team “Geeky Griffins”, an interdisciplinary team of scientists (PhDs and postdocs from the VU and UvA) from the fields of computational linguistics, philosophy, and communication science, and took on the challenge of a 24-hour hackathon.
And as an interdisciplinary team of five junior researchers, we did not just hack recommender system diversity. We also hacked the scientific disciplines! We identified some lessons we learned from working with different disciplines on one complex topic. Is such a disciplinary shake-up perhaps the answer to complex problems like recommender system diversity? We feel like such an approach would also work well for other complex societal problems, where technical, philosophical, and experimental knowledge is all needed.
What did we do during the hackathon?
After hearing the word “hackathon”, you might think of other kinds of hackers who write complex computer programmes, and ‘hacking’ taking place mostly behind computer screens and in code. The reality is more complex though: hackathons can be about any kind of non-computational problem. These range from problems in wildlife preservation to education, and non-programmers make also valuable contributions to a team. The idea of a hackathon is to explore a problem, and build a prototypical solution, in a very short time frame. Teams compete against each other and present their solutions to a jury at the end of the Hackathon. The best solution (and team) wins!
Usually, a hackathon is in one location where food and drinks are provided so ‘hackers’ can focus on the problem at hand, but because of the COVID-19 pandemic we had a virtual hackathon from our homes. This was a strange experience for us: food was delivered to our house so we could spend all our time working on the problem, for which NPO provided video data such as subtitles and metadata on the videos. We worked throughout the night in our separate rooms and houses, and only connected through video chats in Microsoft Teams. This was especially strange because for most of the 5 team members, this was the first time we met each other or worked together. In 24 hours, we not only got to know each other’s preferred working styles, but also experienced quite some stress together.
This hackathon was like an intense hazing period for our “Rethinking News Algorithms” project. For this NWO-funded project, the authors of this blog and their fellow hackathon teammates intend to work together during the coming 4 years. In 24 hours, we already experienced all kinds of common pitfalls of interdisciplinary science, from miscommunication to the threat of not communicating at all.
What did we learn?
Our virtual video-calls started with mapping our respective strengths, with each of us coming from different fields. This was particularly valuable, as the problem at hand involved unique challenges that required the perspective and know-how from more than just one academic discipline. For example, when it comes to designing a diverse recommender, there is the technical programming perspective (‘how’), but also the philosophical and ethical perspective (‘why’ are we doing this). We feel such an approach might be more useful to other complex societal problems as well.
In addition, when tackling complex societal issues, the participation of (non-academic) stake-holders is essential. Throughout the hackathon day, we were able to talk with experts (such as data scientists and programmers) working with the current NPO recommendation system. Their ideas on, and terminology about, diversity in recommender systems were not the ones common in academic research. However, these terms and ideas were closer to how our designed systems would be used in non-academic contexts. This meant speaking to these non-academic stakeholders was especially useful!
We decided to split into two teams, one working on the technical implementation and the other on theoretical argumentation of our ideas. While this approach played into everyone’s strengths, it also created a certain disconnect. Due to our set-up the theory and tech were treated as two separate parts of the project. This was something that we only realized in hindsight and we might have wanted to avoid. We had some meetings where everyone came together during the day. That was nice. However, during long stretches we made choices on our ideas without leveraging advantages of each other’s expertise. This might have been less than ideal. Our takeaway is that the connection and collaboration is the most important to this interdisciplinary project and should be at the core of such projects on complex societal problems.
Our diverse understandings of science and recommendations were useful for solving the problem. One example was how to explain the recommender’s diversification to users. Should we visualize cluster and/or topic distance between items, which is a more computational approach? Or should we keep things simple by merely providing user-friendly content suggestions, because offering a technical visualization might overwhelm users? This latter idea came from the communication scientists in our team. We also extensively discussed the implications of our definitions and approaches thanks to the philosopher. In the end, all these ideas came together in our prototype. Our idea consisted of (among other aspects) a “peek behind the curtain” idea, where users would be enticed to see what other users and perhaps famous people are seeing, and embedding the subtitles of every video in a multi-dimensional space to distance between videos was visible and calculatable.
In the end, there were some differences in approaches and ideas, but our final product contained aspects of each of them. We presented our prototype to the jury, consisting of experts in recommender systems at the NPO and at Media Perspectives, and to our own surprise our team won! The jury was impressed with our story, where we put the user experience central but also used the newest computational techniques.
We recently heard other good news: the NPO is possibly interested in further developing our concept! This strengthened us in our idea: a shake-up of traditional scientific disciplines, and involving people with different perspectives, could help solve complex societal problems such as recommender system diversity!
- Helberger, N. 2019. On the democratic role of news recommenders. Digital Journalism, 7(8), 993-1012.
- Pariser, E. 2011. The Filter Bubble: What the Internet Is Hiding from You. London: Viking/Penguin Press.
- Sunstein, C. R. 2009. Republic.com 2.0. Princeton, NJ: Princeton University Press.
Myrthe Reuver (MA MSc) is a PhD Candidate in Computational Linguistics at CLTL at the Vrije Universiteit Amsterdam (VU). She researches how to automatically capture news diversity in recommendation systems in the UvA/VU project “Rethinking News Algorithms”. One of her interests is combining knowledge from linguistics and other fields in the humanities and social sciences with new techniques related to machine learning, AI, and data analysis. If you want to know more, you can follow her on Twitter.
Nicolas Mattis is a PhD candidate at the department of Communication Science. He studies news selection and exposure diversity in news recommender systems. His research is part of the interdisciplinary project “Rethinking News Algorithms: Nudging users towards diverse news exposure”.