You’ve hate-read a comment section. You’ve watched an argument spiral for hours. You’ve seen someone say something reasonable and get torn apart for it. You know the internet can be miserable.

But you might not know how much more miserable it is for some people than others, or why the tools meant to fix it keep failing.

I use the term ‘online toxicity’ here to refer to a state. A state in which being online is actively detrimental to your mental health.

The levels of trolling, hate speech and harassment outweigh the legitimate discourse to a point where it is toxic to your well-being.

You have to take breaks from being online. You have to use apps that show you how to hide yourself from the public.

You self-censor to avoid backlash. This isn’t just an individual problem; it degrades public discourse for everyone.

And yet, the experience of it is very different for different communities, while the only solutions to it are one-size-fits-all.

Beauty is in the eye of the beholder

Beauty is in the eye of the beholder — and so is toxicity. What is toxic to one person or community isn’t toxic to another. Yet one-size-fits-all moderation treats it as universal: the same algorithm flags a Black woman reclaiming a slur and a white supremacist wielding it. These two situations are not the same.

One community is using language to fight oppression, whilst the other seeks to enact violence and oppress. Context matters. Toxic for whom matters.

What constitutes harm depends on who’s speaking, who’s listening, and what history they carry. A phrase that’s benign political commentary in one community is a dog whistle telling another community they don’t belong.

This means that toxicity classification must be contextual, too. Not one model trained on one dataset reflecting one set of assumptions — but classification that can be configured by the communities who’ll use it, based on their context, their protected characteristics, their threshold for harm.

Make MPs British Again

We’ve built a classifier that does this. It uses large language models to classify hate speech, but the classification is shaped by community-defined configurations.

Here’s what that looks like: we tested “Make MPs British Again” against different configurations. For a collective that values open political debate and regularly analyses hate speech tropes, this wasn’t flagged as toxic. For a collective of South Asian journalists who configured their model to filter out content that tells them they don’t belong, the same phrase was flagged as toxic.

Same content. Different contexts. Different classifications.

This breaks the toxic/not-toxic binary. Content can be labelled as falling under multiple categories — racism, sexism, religious discrimination — as many as apply. Intersectional classification instead of false simplicity.

We’ve proven the core insight works. Context-aware classification is possible.

Building With, Not For

What we haven’t built yet is the governance layer — and we’re not going to build it alone.

We plan to design the next phase with users: how communities define what toxicity means for their space, onboard their own data, vote on edge cases the model gets wrong, and refine the classifier over time. Ultimately, communities can decide how their data informs research and policy.

We’re taking this approach because we believe you can’t design technology without designing the social context it sits in.

Researchers call this “sociotechnical design” — treating technology as inseparable from the power structures and human practices surrounding it. Most AI tools ignore this. They claim neutrality while embedding the assumptions of whoever built them.

We want to make the human input visible, not hide it. To build with communities, not for them.

Moreover, research shows that participation makes the science better. Involving communities with lived experience doesn’t just make AI evaluation more democratic or legitimate; it makes the findings more accurate and rigorous (Matias & Price, 2025).

In the project Beneath the Surface, community members in Chicago reviewed police complaint records and found allegations of sexual violence buried in files the department had categorised as “illegal search.” Lived experience revealed what ‘official’ labels had obscured.

Others are already building this way — some by creating alternative infrastructure entirely, like Te Hiku Media, a Māori radio station. Rather than let tech companies train on their elders’ voices, they created their own speech recognition and data license.

Other projects are contesting governance on existing systems.

For example, the Drivers Cooperative has an elected board of fellow drivers who review complaints and make discipline decisions.

The people being governed choose their governors to make these critical decisions. This is in stark contrast to Uber, where an algorithm can deactivate you with no explanation and no appeal.

We, too, are contesting governance on existing systems. We don’t control the foundation models. We’re trying to shift who controls what they’re used to deciding.

Tensions We’re Holding

Building this tool means sitting with contradictions we haven’t resolved.

Beyond AI Solutionism

There’s a rush to apply AI to every problem. I’m sceptical of that impulse. WeLivedIt.AI wasn’t born from wanting to leverage AI to solve a problem. My lived experience of learning and unlearning systems of oppression, as well as challenging my assumptions around technology, led me here.

I became interested in the social side of technology when I was living in Bangladesh. I spoke to rickshaw pullers who showed me how they used their smartphones to send money back to their families in the village.

They couldn’t do much else on these phones, but they used them for this one life-saving purpose.

I reflected on why I was so surprised and realised I’d absorbed this idea of technology as progress. I needed to unlearn the neoliberal metaphor of the ladder you climb rung by rung.

A concept that frames access as achievement and treats anyone not climbing as behind — subtly encoding hierarchy and judgment.

That experience changed how I thought about design. If technology serves different people differently, then building well requires more than technical skill; it requires care for who’s affected. And that’s why this project centres governance — who decides, who’s impacted, who benefits from the technology – not just the AI.

Using the Tools We Critique

We’re building on top of Big Tech’s models, not because we trust them, but because we want to contest them. Their training data encodes one version of ‘neutral.’ Our tool lets communities overwrite that with their own ground truth. We’re not claiming to fix AI. We’re trying to shift who controls it.

Ruha Benjamin asks who’s at the table when systems get designed. Cathy O’Neil argues models should be contestable, not opaque. Wendy Chun says networks segregate by design, and we need public rights, not just privacy. We’re trying to build accordingly: community-controlled classifiers, transparent about their limits, governed by the people they affect.

Who Draws the Line? Openness and Misuse

What stops someone from configuring their classifier to treat any disagreement as toxic? I don’t believe in a free-market approach to trauma.

I believe that online discourse is a commons — a shared resource that journalists, activists, and citizens depend on. And so I refer to Elinor Ostrom’s work in identifying design principles for the successful, long-term self-governance of shared resources. She identified clearly defined boundaries as the first and most critical of her eight principles. So we’re building boundaries in the form of a rights-based values framework.

We aren’t claiming false neutrality with what we’re building.

The tool protects people from being targeted for who they are, through violence, threats, discrimination, erasure — not from encountering disagreement.

These aren’t boundaries we invented.

They align with rights-based frameworks used by human rights institutions and encoded in emerging legislation like the EU AI Act and the UK’s Online Safety Act. We want to build on these emerging standards.

Communities of Care: Participation Without Burnout

There’s a tension between agency and accountability. We want governance to feel like care, not like extra work piled onto people already drowning in abuse.

To navigate this tension, I refer back to Elinor Ostrom. In her research, she found that governance wasn’t experienced as a burden when rules emerged from existing relationships, and people followed rules they’d made themselves.

For us, the bar is higher: the people governing are the same people being harmed.

Nevertheless, we’re trying to replicate these conditions — designing for existing collectives who already share lived experience, distributing the labour of refinement so no one carries too much, exploring models like cooperative ownership, cross-subsidy or incentivising organisations to fund access for their staff.

The classifier will never be “done.” Hate speech evolves. Political contexts shift. New slurs emerge.

The system must evolve with them, shaped continuously by the communities using it.

What’s Next

I believe a safer internet is possible. Designing for the people who experience the worst of a system, as a way of protecting all of us who might be next, is one way of practising solidarity.

We’re looking for collectives who want to shape what this becomes — not just use a tool we’ve built, but help design how it’s governed.

We’re starting with journalists, but we know this applies beyond newsrooms. If you are an individual or organisation passionate about solving this problem, please reach out at contact@welivedit.ai — I’d love to hear from you.