- Keynote presentation at the American Physical Society’s 2022 Annual Meeting January 2022, Virtual: https://leadership.aps.org/
- Keynote presentation at the American Association for the Advancement of Science’s 2022 Annual Meeting February 2022, Virtual: https://aaas.confex.com/aaas/2022/meetingapp.cgi
- Keynote presentation at The National Academies for Sciences, Engineering, and Medicine’s 2022 Strategic Council April 2022, Virtual: https://www.nationalacademies.org/event/04-21-2022/the-strategic-council-for-research-excellence-integrity-and-trust-meeting-3
Machine Learning Reveals Adaptive COVID-19 Narratives in Online Anti-Vaccination Network
Proceedings of the 2021 Conference of The Computational Social Science Society of the Americas
The COVID-19 pandemic sparked an online “infodemic” of potentially dangerous misinformation. We use machine learning to quantify COVID-19 content from opponents of establishment health guidance, in particular vaccination. We quantify this content in two different ways: number of topics and evolution of keywords. We find that, even in the early stages of the pandemic, the anti-vaccination community had the infrastructure to more effectively garner support than their pro-vaccination counterparts by exhibiting a broader array of discussion topics. This provided an advantage in terms of attracting new users seeking COVID-19 guidance online. We also find that our machine learning framework can pick up on the adaptive nature of discussions within the anti-vaccination community, tracking distrust of authorities, opposition to lockdown orders, and an interest in early vaccine trials. Our approach is scalable and hence tackles the urgent problem facing social media platforms of having to analyze huge volumes of online health misinformation. With vaccine booster shots being approved and vaccination rates stagnating, such an automated approach is key in understanding how to combat the misinformation that slows the eradication of the pandemic.
Richard Sear, Rhys Leahy, Nicholas Johnson Restrepo, Yonatan Lupu, Neil Johnson
Dynamic Latent Dirichlet Allocation Tracks Evolution of Online Hate Topics
Advances in Artificial Intelligence and Machine Learning
Not only can online hate content spread easily between social media platforms, but its focus can also evolve over time. Machine learning and other artificial intelligence (AI) tools could play a key role in helping human moderators understand how such hate topics are evolving online. Latent Dirichlet Allocation (LDA) has been shown to be able to identify hate topics from a corpus of text associated with online communities that promote hate. However, applying LDA to each day’s data is impractical since the inferred topic list from the optimization can change abruptly from day to day, even though the underlying text and hence topics do not typically change this quickly. Hence, LDA is not well suited to capture the way in which hate topics evolve and morph. Here we solve this problem by showing that a dynamic version of LDA can help capture this evolution of topics surrounding online hate. Specifically, we show how standard and dynamical LDA models can be used in conjunction to analyze the topics over time emerging from extremist communities across multiple moderated and unmoderated social media platforms. Our dataset comprises material that we have gathered from hate-related communities on Facebook, Telegram, and Gab during the time period January-April 2021. We demonstrate the ability of dynamic LDA to shed light on how hate groups use different platforms in order to propagate their cause and interests across the online multiverse of social media platforms.
Richard Sear, Rhys Leahy, Nicholas Johnson Restrepo, Yonatan Lupu, Neil F. Johnson
Facebook Parenting Groups Are The New Target For Extremist Misinformation
Forbes
Any parent knows that Facebook mom groups are a confusing mix of support, criticism and downright bad advice. But they were still mainstream communities where parents could find connection around what can often be the lonely job of raising kids.
Preventing the Spread of Online Harms: Physics of Contagion across Multi-Platform Social Media and Metaverses
We present a minimal yet empirically-grounded theory for the spread of online harms (e.g. misinformation, hate) across current multi-platform social media and future Metaverses. New physics emerges from the interplay between the intrinsic heterogeneity among online communities and platforms, their clustering dynamics generated through user-created links and sudden moderator shutdowns, and the contagion process. The theory provides an online `R-nought’ criterion to prevent system-wide spreading; it predicts re-entrant spreading phases; it establishes the level of digital vaccination required for online herd immunity; and it can be applied at multiple scales.
Chen Xu, Pak Ming Hui, Om K. Jha, Chenkai Xia, Neil F. Johnson
How Social Media Machinery Pulled Mainstream Parenting Communities Closer to Extremes and Their Misinformation During Covid-19
IEEE
We reveal hidden social media machinery that has allowed misinformation to thrive among mainstream users, but which is missing from current policy discussions. Specifically, we show how mainstream parenting communities on Facebook have been subject to a powerful, two-pronged misinformation machinery during the pandemic, that has pulled them closer to extreme communities and their misinformation. The first prong involves a strengthening of the bond between mainstream parenting communities and pre-Covid conspiracy theory communities that promote misinformation about climate change, fluoride, chemtrails and 5G. Alternative health communities have acted as the critical conduits. The second prong features an adjacent core of tightly bonded, yet largely under-the-radar, anti-vaccination communities that continually supplied Covid-19 and vaccine misinformation to the mainstream parenting communities. Our findings show why Facebook’s own efforts to post reliable information about vaccines and Covid-19 have not been efficient; why targeting the largest communities does not work; and how this machinery could generate new pieces of misinformation perpetually. We provide a simple yet exactly solvable mathematical theory for the system’s dynamics. It predicts a new strategy for controlling mainstream community tipping points. Our conclusions should be applicable to any social media platform with in-built community features, and open up a new engineering approach to addressing online misinformation and other harms at scale.
Nicholas J. Restrepo, Lucia Illari, Rhys Leahy, Richard Sear, Yonatan Lupu, Neil F. Johnson
Machine Learning Language Models: Achilles Heel for Social Media Platforms and a Possible Solution
Advances in Artificial Intelligence and Machine Learning
Any uptick in new misinformation that casts doubt on COVID-19 mitigation strategies, such as vaccine boosters and masks, could reverse society’s recovery from the pandemic both nationally and globally. This study demonstrates how machine learning language models can automatically generate new COVID-19 and vaccine misinformation that appears fresh and realistic (i.e. human-generated) even to subject matter experts. The study uses the latest version of the GPT model that is public and freely available, GPT-2, and inputs publicly available text collected from social media communities that are known for their high levels of health misinformation. The same team of subject matter experts that classified the original social media data used as input, are then asked to categorize the GPT-2 output without knowing about its automated origin. None of them successfully identified all the synthetic text strings as being a product of the machine model. This presents a clear warning for social media platforms: an unlimited volume of fresh and seemingly human-produced misinformation can be created perpetually on social media using current, off-the-shelf machine learning algorithms that run continually. We then offer a solution: a statistical approach that detects differences in the dynamics of this output as compared to typical human behavior.
Richard Sear, Rhys Leahy, Nicholas Johnson Restrepo, Yonatan Lupu, Neil F. Johnson
New Math to Manage Online Misinformation
SIAM News Blogs
Social media continues to amplify the spread of misinformation and other malicious material. Even before the COVID-19 pandemic, a significant amount of misinformation circulated every day on topics like vaccines, the U.S. elections, and the U.K. Brexit vote. Researchers have linked the rise in online hate and extremist narratives to real-world attacks, youth suicides, and mass shootings such as the 2019 mosque attacks in Christchurch, New Zealand. The ongoing pandemic added to this tumultuous online battlefield with misinformation about COVID-19 remedies and vaccines. Misinformation about the origin of COVID-19 has also resulted in real-world attacks against members of the Asian community. In addition, news stories frequently describe how social media misinformation negatively impacts the lives of politicians, celebrities, athletes, and members of the public.
Neil F. Johnson
Online Group Dynamics Reveal New Gel Science
A better understanding of how support evolves online for undesirable behaviors such as extremism and hate, could help mitigate future harms. Here we show how the highly irregular growth curves of groups supporting two high-profile extremism movements, can be accurately described if we generalize existing gelation models to account for the facts that the number of potential recruits is time-dependent and humans are heterogeneous. This leads to a novel generalized Burgers equation that describes these groups’ temporal evolution, and predicts a critical influx rate for potential recruits beyond which such groups will not form. Our findings offer a new approach to managing undesirable groups online — and more broadly, managing the sudden appearance and growth of large macroscopic aggregates in a complex system — by manipulating their onset and engineering their growth curves.
Pedro D. Manrique, Sara El Oud, Neil F. Johnson
A Public Health Research Agenda for Managing Infodemics: Methods and Results of the First WHO Infodemiology Conference
JMIR Infodemiology
An infodemic is an overflow of information of varying quality that surges across digital and physical environments during an acute public health event. It leads to confusion, risk-taking, and behaviors that can harm health and lead to erosion of trust in health authorities and public health responses. Owing to the global scale and high stakes of the health emergency, responding to the infodemic related to the pandemic is particularly urgent. Building on diverse research disciplines and expanding the discipline of infodemiology, more evidence-based interventions are needed to design infodemic management interventions and tools and implement them by health emergency responders.
Calleja et al.