GW Press Release
AI models like ChatGPT have amazed the world with their ability to write poetry, solve equations and even pass medical exams. But they also can churn out harmful content, or promote disinformation.
Unlocking the Secrets of AI: Why Understanding Attention in Large Language Models Matters
Ministry of AI
As we zoom deeper into the remarkable world of Artificial Intelligence (AI), specifically Large Language Models (LLMs) like ChatGPT, understanding how they work becomes increasingly vital. Ever wondered why these systems sometimes repeat themselves, imagine bizarre things, or appear biased? A recent research article dives into the core mechanics of these models, explaining their magic through the lens of physics—a fascinating twist that makes this topic both enlightening and accessible!
Capturing AIs Attention Physics of Repetition, Hallucination, Bias and Beyond
Ministry of AI
A detailed breakdown of the AI research paper: Capturing AI’s Attention: Physics of Repetition, Hallucination, Bias and Beyond
Capturing AI’s Attention: Physics of Repetition, Hallucination, Bias and Beyond
We derive a first-principles physics theory of the AI engine at the heart of LLMs’ ‘magic’ (e.g. ChatGPT, Claude): the basic Attention head. The theory allows a quantitative analysis of outstanding AI challenges such as output repetition, hallucination and harmful content, and bias (e.g. from training and fine-tuning). Its predictions are consistent with large-scale LLM outputs. Its 2-body form suggests why LLMs work so well, but hints that a generalized 3-body Attention would make such AI work even better. Its similarity to a spin-bath means that existing Physics expertise could immediately be harnessed to help Society ensure AI is trustworthy and resilient to manipulation.
Frank Yingjie Huo, Neil F. Johnson
Meta Launches Community Notes For Improved Post Context
The Pinnacle Gazette
The new feature encourages user collaboration to add insights and combat misinformation.
Meta Announces Community Notes Will Use Open-source Technology From X
TechBullion
Meta’s upcoming Community Notes feature for monitoring misinformation through crowdsourcing will use some technology developed by Elon Musk’s X for its similar service.
Meta’s Community Notes will use tech from Elon Musk’s X
NBC News
The feature will roll out on March 18 on Facebook, Instagram and Threads in the United States.
Meta’s Community Notes will use open-source technology from Elon Musk’s X
CNBC
Meta’s upcoming Community Notes feature for monitoring misinformation through crowdsourcing will use some technology developed by Elon Musk’s X for its similar service.
Meta’s Community Notes To Use X’s Algorithm
Silicon
Community Notes testing across Facebook, Instagram and Threads to begin next week in US, using algorithm from Elon Musk’s X
Coevolution of network and attitudes under competing propaganda machines
NPJ Complexity
Politicization of the COVID-19 vaccination debate has lead to a polarization of opinions regarding this topic. We present a theoretical model of this debate on Facebook. In this model, agents form opinions through information that they receive from other agents with flexible opinions and from politically motivated entities such as media or interest groups. The model captures the co-evolution of opinions and network structure under similarity-dependent social influence, as well as random network re-wiring and opinion change. We show that attitudinal polarization can be avoided if agents (1) connect to agents all across the opinion spectrum, (2) receive information from many sources before changing their opinions, (3) frequently change opinions at random, and (4) frequently connect to friends of friends. High Kleinberg authority scores among politically motivated media and two network components that are comparable in size can indicate the onset of attitudinal polarization.
Mikhail Lipatov, Lucia Illari, Neil Johnson, Sergey Gavrilets