Welcome to the second edition of the Human Rights Foundation’s (HRF) monthly newsletter on the intersection of AI and individual rights!
A few weeks ago, I visited Norway as part of the 17th annual Oslo Freedom Forum (OFF), a gathering affectionately known as “Davos for dissidents,” which once again brought together an inspiring constellation of human rights defenders, philanthropists, and tech innovators from across the globe. This year marked my fourth time at OFF, but it was particularly special as it was my first year participating as a member of the HRF team. With each successive year, the forum’s impact grows even more profound for me, serving as a vital nexus for renewing old friendships, forging new connections, and collectively strategizing on the most pressing challenges facing freedom worldwide.
It was within this dynamic atmosphere that we proudly launched the AI for Individual Rightsinitiative, complete with hands-on workshops, office hours for activists, talks and panel discussions, and extra-long days as we discussed how to amplify the work of nonprofits with AItools late into the evening, with the sun only finally setting around 11pm each day.
A growing interest in AI was undeniably one of the standout themes at this year’s Oslo Freedom Forum. Every AI-focused event and workshop was not just well-attended, but truly packed, with standing room only becoming the norm. The overwhelming engagement clearly signals a critical need for larger venues dedicated to AI discussions at next year’s event. For me, it was clear that attendees shared concern for both the potential risks and the vast opportunities presented by AI tools.
On the one hand, a significant portion of the discourse centered on the inherent dangers of AI, with potential for surveillance, control, and the erosion of individual liberties being cited by our speakers as primary concerns. The specter of authoritarian regimes leveraging AI for oppressive purposes was on full display with talks from speakers on repression in China, Iran, and beyond, which HRF will expose later this year with groundbreaking reports on how some of these dictators are using AI for repression. At the same time, there was genuine excitement regarding the capacity of open-source AI tools to empower human rights defenders, especially around areas of productivity and scale for organizations that too often have not enough resources.
Two topics generated a lot of buzz: the growing threat of “LLM pollution” and the intriguing potential of “vibe coding.”
LLM pollution, which we will discuss in this newsletter, refers to the coordinated degradation of information quality and the spread of misinformation driven by state-actors making every effort to influence the responses from large language models. Vibe coding, on the other hand, emerged as an exciting opportunity, exploring how AI tools can save huge amounts of time for activists, and expedite and scale their work in an unprecedented way. Stay tuned for next month’s newsletter for a video of our first “vibe coding for human rights workshop” starring our technical lead, Justin Moon.
If you have any questions about this newsletter, please write to me directly at [email protected].
Stay safe, and fight hard!
Craig Vachon
Director, AI for Individual Rights
P.S. Let’s answer one popular question from the community: Do LLMs capture all of our prompts, and could that information be used against dissidents?
Answer: Yes. The best advice for individuals concerned about state-actor surveillance when using LLMs is to assume that anything you type into a public LLM can be collected, analyzed, and potentially linked back to you. The only truly secure approach is to avoid entering sensitive information at all, or to rely on thoroughly vetted, offline, and self-hosted LLM solutions that are isolated from external networks (we’ll talk more about this challenge in the next newsletter).
In the meantime:
Use a VPN when accessing a corporate LLM (and don’t sign in): If you use a Zero Trust product like Twingate to access a public large language model (LLM) and do not sign in to the LLM service, the LLM itself generally cannot directly identify you as an individual or track your specific identity or urges.
Avoid identifiable language: Train oneself to avoid unique phrases, personal anecdotes, or specific details that could link to your identity or activities;
Mix up communication channels: Don’t rely solely on a specific LLM for sensitive discussions;
Use multiple identities/personas: Interact with LLMs using different, uncorrelated personas for different topics;
Understand LLM capabilities: Be aware of what LLMs are good at (pattern recognition, summarization, inference) and how that might be used against you.
If you can, use an LLM that like trymaple.ai is encrypted end to end, where your account has its own private key that encrypts your chats and the responses from the AImodel. Or, try running a local LLM offline.
Deep Dive – Coordinated LLM Pollution from State Actors
LLMs are becoming increasingly integrated into key parts of our lives, ranging from the dissemination of public information to the news we ingest from the media. In this week’s “deep dive,” located at the bottom of this email, HRF will report on the growing threat of “LLM pollution,” which is when state actors (especially dictators) deliberately manipulate AI models and their underlying data. Such manipulation can subtly alter model behaviors, introduce insidious biases, or propagate fabricated information, thereby eroding public trust, promoting censorship, and undermining the foundational utility of AI systems. State actors, leveraging significant resources and strategic intent, employ LLMs as a new, highly scalable, and often stealthy vectors for information warfare, geopolitical influence, and intelligence gathering. Our mini-report dissects the sophisticated methods and strategic objectives of state-sponsored LLM pollution, and moving forward, we’ll help you understand how best to stay clear of intelligence-led propaganda.
China | MapleAI banned from Apple App Store
The Cyberspace Administration of China banned Maple AI, a privacy-protecting ChatGPT-like AItool, from China’s Apple App Store. Maple AI leverages open source encryption technology to ensure users greater privacy when using AI. The Cyberspace Administration of China demanded the removal of Maple AI from the Apple App Store for having the potential to lead to “major changes in public opinion properties or capacity for social mobilization”. Cofounder and CEO of Maple AI writes, “We are both unsurprised and validated that China demanded the removal of Maple from the App Store. Maple is the confidential AI solution for individuals and organizations that protects the fundamental human right to privacy, and by extension protects freedom of thought. We hope the Chinese government reconsiders their decision because we won’t weaken our encryption just to be in that market.”
Global | OpenAI Requires Government-Issued ID to Access Latest Models
OpenAI recently announced it will require users to upload a government-issued ID to access their latest Application Programming Interface (API). API access allows users to use ChatGPT with third party platforms, democratizing access to the technology and in many cases enhancing privacy. In some instances, users have also reported being required to undergo facial scans in order to access OpenAI’s API. The requirement to reveal sensitive, personal information to access OpenAI’s services unfortunately forces political dissidents to choose between access to cutting edge technologies and making their sensitive data vulnerable to the state-led surveillance, where, like any other large company, OpenAI will be met with requests from authoritarian regimes to access user data, often in a way that targets dissidents and human rights activists.
Türkiye | Scales up AI-powered Surveillance Infrastructure
The Turkish government, which implements repressive measures to crackdown on peaceful protests, continues to scale up its AI-powered surveillance infrastructure. In the wake of peaceful student protestors being detained and prosecuted for protesting the arrest of Istanbul’s mayor and presidential candidate of the opposition party, the Istanbul police headquarters increased their facial recognition equipment with a contract for 5.7 million liras. This follows a consistent pattern of scaling up AI-powered surveillance infrastructure, including the launch of 13,000 facial recognition cameras and 2,500 mounting units in March of this year. Journalists have reported on police using the Chinese-made Hikvision camera, which can detect 120 faces simultaneously. The ongoing expansion of AI-powered surveillance systems has grave human rights implications for the region and other jurisdiction’s that may seek to model the Turkish government’s approach.
China | Use of AI to Censor Results of the Tiananmen Square Massacre
ABC News reports on hundreds of pages of leaked documents which reveal how China uses AItools to censor posts and results about the Tiananmen Square massacre. The documents reveal that every post on popular social media platforms are first scanned by AI systems for any references to the Tiananmen anniversary. Protocol goes so far as to censor visual metaphors referencing Tiananmen Square, such as “one banana and four apples in a line.” Additionally, researchers have demonstrated that widely adopted AI tools such as Deepseek, Weibo Smart Search and Doubao AI will refuse to answer questions about the Tiananmen Square massacre or acknowledge its existence. The use of these AI tools by the Chinese government to censor posts and results about Tiananmen Square speak to the power authoritarian regimes can wield to “control” the recollection of the past and thus influence the future.
Routstr | Launches a Decentralized LLM Routing Marketplace
A decentralized LLM Routing Marketplace named Roustr has recently been launched, which allows users to use LLMs pseudonymously. Rather than paying for a service like ChatGPT through the provider, where a user may be required to reveal their identity, users can now access these cutting edge models without disclosing any personal information. This service leverages two technologies to accomplish this: the decentralized protocol “nostr” (Notes and Other Stuff Transmitted by Relays) to provide users pseudonymously access to the tool, and e-cash, an anonymous payment solution built on top of the Bitcoin protocol. While this service still relies on being granted access to third party LLMs, and can’t offer the guarantee the LLM providers won’t log the user’s queries, the ability for the user to break the linkage between their identities and their inputted data is a crucial step in protecting the privacy of dissidents and activists when using cutting edge tools.
Ollama | Creates Confidential Computing Mode for Greater Privacy
In collaboration with Ollama, an open source application that allows users to run LLMs on their machines locally, Stanford’s Hazy Research recently created an end-to-end encrypted protocol that allows users to privately communicate with more powerful cloud providers (often called “frontier models”). The team built a comprehensive security protocol using the new “confidential computing mode” introduced by hardware released by NVIDIA. Typically, the best open weight models are too powerful to run on consumer hardware, and third party servers risk exposing users’ data. This technology allows users to run more powerful open weight models like Google’s Gemma on third party servers in a way that is verifiably private. This development represents a promising move forward to allow users to use LLMs privately and securely without sacrificing on the strength of the tool.
Signal | Counters Microsoft’s AI Recall Privacy Risks
Microsoft recently released a new AI-powered feature called “Recall,” which takes a screenshot of users’ active window every 10 seconds to help them “recall” information they’ve handled in the past using human language queries. Privacy experts have cited concern over this tool causing users unknowingly leaking sensitive information. Signal, the encrypted messaging app, has responded to this privacy risk by implementing the default setting of disabling users’ Windows devices to take screenshots of Signal chats. This newly implemented feature by Signal ensures greater privacy to users and acts as a great example of developers proactively developing privacy features in the face of newly created privacy risks.
OpenSecret | Joins NVIDIA Inception Program
OpenSecret, the company that develops the privacy-protecting LLM Maple AI, has joinedNVIDIA’s Inception program for startups. NVIDIA’s Inception program supports startups in developing, prototyping, and deploying products, and startups receive benefits such as Deep Learning credits and preferred pricing on hardware, software, and technological assistance. OpenSecret writes “NVIDIA Inception provides the mentorship, resources, and community that will help us deliver secure, encrypted services — at scale — to developers and end users alike.” The next 12-18 months are critical for AI tools as they rapidly expand into the hands of individuals, and it’s vital that privacy-focused AI tools are part of the industry conversation and dialogue.
RECOMMENDED CONTENT
Andrej Kaparthy Gives Keynote on Software in the Era of AI
In this keynote presentation to the AI Startup School in San Francisco, founding member of OpenAI and former senior director of AI at Tesla Andrej Kaparthy offers his perspective on software in the era of AI. Kaparthy explains that for the last 70 years, software has not changed much on a fundamental level, but that we’ve seen two major shifts in the last few years. Kaparthy thinks of there now being three distinct forms of software: Human-written code, AI neural nets with programmable weights (which inform AI’s output), and now programmable neural nets with LLMs. Kaparthy explains that crucially, the ability to program with LLMs using a natural language like English (often called “vibe coding” — a term Kaparthy coined) offers unprecedented access for the individual to write software. This talk is worth watching for insights into how today, individuals with no knowledge of computer programming can finally, and what that might mean for individual liberty in the future.
Want to contribute to the newsletter? Submit tips, stories, news, and ideas by emailing ai@hrf.org
DEEP DIVE: The Evolving Threat of LLM Pollution by State Actors
LLMs like ChatGPT have rapidly become integral for all kinds of businesses and organizations, ranging in format from simple chatbots to sophisticated enterprise systems. With the proliferation of these LLMs comes a new threat: “LLM pollution,” which is the intentional and coordinated corruption of the models and their training data to achieve specific malicious objectives. The goals of “LLM pollution” might include political censorship, the transmission of false information, and ultimately the elimination of certain truths. Dictatorships are actively targeting LLMs as a clandestine vector for achieving their censorious aims.
One of the most insidious forms of LLM manipulation is data poisoning. This involves malicious actors manipulating the training data to influence a model’s behavior, often requiring minimal changes to the dataset itself. During the training phase, harmful data can be introduced, leading to biased or manipulated outputs. Even a small percentage of corrupted data, sometimes less than 5%, can significantly reduce model accuracy or introduce specific “backdoor” behaviors that activate only under certain conditions. This is particularly concerning given that LLMs are frequently trained on vast amounts of scraped web data, making inadvertent contamination possible, or deliberate contamination by malicious model providers seeking to evade detection. The problem extends beyond isolated malicious inputs; it represents a systemic vulnerability where even minor, coordinated efforts can have widespread, cascading effects across multiple downstream models and applications that rely on these foundational models. The “amplification effect” observed in data poisoning implies that a single compromised dataset can infect numerous derived models, rendering the entire AI ecosystem susceptible to a few well-placed attacks. Beyond direct manipulation, backdoor attacks inject data that allows attackers to manipulate the model’s output when specific conditions are met, remaining dormant until triggered. Stealth attacks involve gradually altering the training dataset or injecting harmful data surreptitiously over time to avoid detection, leading to subtle yet impactful biases that are difficult to trace. Such subtle alterations are virtually impossible to detect through manual inspection, and their impact is amplified by the increasing complexity of models and reliance on automated data collection processes. This means vulnerabilities can be deeply embedded within the model’s core before it is even deployed.
State actors are increasingly using generative AI tools to produce larger quantities (scale) of disinformation. LLMs have proven significantly more effective than humans in using personal information to persuade individuals, enabling highly targeted “microtargeting” or hyper-personalization for mass disinformation campaigns. This capability allows for the generation of content that criticizes movements or spreads false narratives tailored to specific audiences (often to the granularity of individual users). The combination of LLMs’ ability to generate vast quantities of content and their enhanced persuasive capabilities through personalization creates a “weapon of mass persuasion” that regimes will use both home and abroad.
This leads to the broader objective of creating chaos via the erosion of trust and the undermining of institutions. The deliberate manipulation of LLMs by authoritarian networks aims to distort AI outputs, infecting them with false claims to deliver propaganda worldwide. Such efforts can subtly shift the sentiment of a model’s response without resorting to outright censorship, making the manipulation harder to detect. The proliferation of unreliable or fabricated information generated by LLMs risks eroding trust in digital communication, traditional media, government institutions, and ultimately, the foundations of democratic discourse. The objective of infecting AI models with false narratives and subtly shifting sentiment points to a deeper, more insidious goal than just spreading specific lies. It is about systematically eroding public trust in all sources of information, including AI, traditional media, and governmental institutions. This creates a pervasive climate of skepticism and cynicism, making populations more vulnerable to future manipulation and undermining societal cohesion and the ability to respond to crises. The ultimate goal is to make it impossible for citizens to discern truth from fiction, thereby creating a fertile ground for authoritarian narratives.
Espionage and sensitive information disclosure represent another critical objective. Adversaries craft jailbreak prompts to manipulate LLMs into revealing sensitive information, such as personally identifiable information (PII) or proprietary algorithms and data. Attackers can also utilize malicious tools injected into LLM systems to collect user queries and dynamically update their attack strategies based on the hyper-personalized stolen information, enhancing subsequent attacks. LLMs, especially when augmented with internal knowledge bases for enterprise applications, become high-value targets for data exfiltration. The fact that LLMs can be manipulated to reveal sensitive data or even collect user queries highlights their “dual-use” nature. A technology designed for helpfulness can be transformed into a potent espionage tool.
The impact of data poisoning is significantly amplified by several factors inherent to modern AIdevelopment, including model complexity, transfer learning and fine-tuning, and automated data collection processes. Generative AI tools have already begun to dramatically alter the size and scope of state-backed propaganda campaigns, enabling a scale previously unimaginable.
LLMs are demonstrably more effective than humans at using personal information to persuade individuals, with potentially alarming consequences for mass disinformation campaigns that can be microtargeted to specific demographics (and eventually hyper-personalization). The sheer volume and velocity of content generated and disseminated in the digital ecosystem often outpace traditional human-centric fact-checking efforts, making it difficult for accurate information to gain traction against a flood of false narratives.
In conclusion, authoritarian regimes are likely to continue to pursue LLM pollution to censor political information, influence the behavior of citizens at home and abroad, weaken trust in global AI systems, and exfiltrate sensitive data about AI system users.
During our next few newsletters, we will discuss ways that you can avoid LLM pollution and use open source AI tools instead to advance freedom and democratic values.