Who Is Missing From the Data? Structural Zero 09: Sampling Bias

Every dataset is produced by a set of choices—about technology, language, geography, incentives, and risk. hrdag.substack.com/p/who-is-mis...

Who Is Missing From the Data? Structural Zero 09: Sampling Bias

The question isn’t just “what does the data say?”

It’s “who couldn’t speak?”

Read about sampling bias in this month’s Structural Zero.
hrdag.substack.com/p/who-is-mis...

What is Sampling Bias in Statistics? Who is missing from the data? That’s the central question in this edition of How to Tell the Truth with Statistics, a series from the Human Rights Data Analysis Group (HRDAG). In this conversation,…

Who is missing from the data? That’s the central question in this edition of How to Tell the Truth with Statistics.

Watch to learn about South Africa’s transition from apartheid and how data collection shapes knowledge. www.youtube.com/watch?v=d2Cs...

Green and white graphic with the text: understanding sampling bias

Green and white graphic with the text: Systems of data collection shape knowledge. Every dataset is produced by a set of choices—about technology, language, geography, incentives, and risk. When certain voices don’t appear in the data, their absence can be misinterpreted.

Green and white graphic with the text: We might think these voices weren’t impacted when the truth story could be these voices were afraid to speak out. Gaps in the data don’t happen randomly; they reflect fear, power, access, and trust. When people are missing from a dataset, that absence is itself information.

Green and white graphic with the text: If we ignore those absences, we risk telling incomplete or even misleading stories. We may conclude that violence is rare where reporting is dangerous, or that certain communities were unaffected when, in reality, they were unheard. In this way, sampling bias can quietly reinforce the very inequalities we aim to expose.

We bump into sampling bias a lot in our work at HRDAG. So we know:

Systems of data collection shape knowledge. Every dataset is produced by a set of choices—about technology, language, geography, incentives, and risk.

What is Sampling Bias in Statistics? Who is missing from the data? That’s the central question in this edition of How to Tell the Truth with Statistics, a series from the Human Rights Data Analysis Group (HRDAG). In this conversation,…

Check out our most recent episode of "How to Tell the Truth with Statistics," where Dr. Patrick Ball explains sampling bias and talks about his work with South Africa's Truth and Reconciliation Commission. www.youtube.com/watch?v=d2Cs...

Who Is Missing From the Data? Structural Zero 09: Sampling Bias

Data doesn’t just reflect reality. It reflects who was able or willing to be counted. Fear, distrust, and access all shape the dataset before analysis even begins. hrdag.substack.com/p/who-is-mis...

White text on orange background: Systems of data collection shape knowledge. Every dataset is produced by a set of choices -- about technology, language, geography, and risk. When certain voices don't appear in the data, their absence can be misinterpreted. We might think these voices weren't impacted when the true story could be these voices were afraid to speak out.

Every dataset carries the imprint of how it was collected: incentives, geography, language, and risk. Understanding that is the first step toward truth. hrdag.substack.com/p/who-is-mis...

You're invited to "round table talk: ai in journalism." Tap here to RSVP - Paperless Post Flyer Join trina reynolds-tyler and Tarak Shah from Human Rights Data Analysis Group at the Invisible Institute office for a discussion with other media makers on A…

Tuesday, join trina reynolds-tyler and Tarak Shah from Human Rights Data Analysis Group at the @Invisible.Institute office to discuss AI in journalism www.paperlesspost.com/go/83vYxWpCM...

You're invited to "round table talk: ai in journalism." Tap here to RSVP - Paperless Post Flyer Join trina reynolds-tyler and Tarak Shah from Human Rights Data Analysis Group at the Invisible Institute office for a discussion with other media makers on A…

In Chicago? Come to our roundtable discussion on AI in journalism this Tuesday. www.paperlesspost.com/go/83vYxWpCM...

Who Is Missing From the Data? Structural Zero 09: Sampling Bias

Data doesn’t just reflect reality. It reflects who was able or willing to be counted. Fear, distrust, and access all shape the dataset before analysis even begins. hrdag.substack.com/p/who-is-mis...

White text on orange: systems of data collection shape knowledge. Every dataset is produced by a set of choices—about technology, language, geography, incentives and risk. When certain voices don't appear in the data, their absence can be misinterpreted. We might think these voices weren't impacted when the true story could be these voices were afraid to speak out.

Every dataset carries the imprint of how it was collected: incentives, geography, language, and risk. Understanding that is the first step toward truth. hrdag.org/2026/04/02/w...

Black text on red background: The New National Police Index

Black and white text on red: The National Police Index is a project and data tool showing police employment history data obtained from state police training and certification boards across the U.S. Through our newly relaunched website, users can search a police officer by name and by agency to learn more about their employment history.

When a police officer kills someone, they may leave their role—only to get hired at another police department.

The National Police Index helps track wandering officers, thanks to @invisible.institute + @hrdag.org + Machine Learning Justice Lab + Innocence & Justice Louisiana. national.cpdp.co

The Life and Legacy of Efraín Arévalo Ibarra This is the story of Efraín Arévalo Ibarra: a teacher, father, and union member who was “disappeared” by the Salvadoran government in November 1977.

Incredible multi-media report from our friends at @uwchr.bsky.social telling the story of Efraín Arévalo Ibarra: a teacher, father, and union member who was “disappeared” by the Salvadoran government in November 1977. unfinishedsentences.org/reports/efra...

Yes, Your Dataset is Probably Biased Structural Zero 08: What statisticians mean by bias—and why it matters

Statisticians love to have perfectly-designed, random samples to study. But reality is messy and many datasets have some inherent bias.

That doesn't mean we can't use them. It means we need to acknowledge and account for bias. hrdag.substack.com/p/yes-your-d...

Informes — Kilómetro 0 En esta sección encontrarás informes que hemos desarrollado con datos y análisis sobre la Policía de Puerto Rico.

New from our partners at Kilómetro Cero: has Puerto Rico actually reduced excessive and discriminatory police use of force? www.kilometro0.org/informes

Resource spotlight: Data projects tracking police misconduct, use of force, and employment histories The need for law enforcement transparency, oversight, and accountability has never been clearer. We highlight data projects that have helped document and investigate misconduct, as ...

The @prisonpolicy.org spotlights data projects tracking police misconduct, use of force, and employment histories—including several open data and public records projects supported by HRDAG's research. www.prisonpolicy.org/blog/2026/01...

Informes — Kilómetro 0 En esta sección encontrarás informes que hemos desarrollado con datos y análisis sobre la Policía de Puerto Rico.

Incredible new report from our partners at Kilómetro Cero examines police abuses in Puerto Rico. www.kilometro0.org/informes

Such a powerful event. Happy to help co-host.

An Introduction to Bias in Statistics with Dr. Megan Price YouTube video by Human Rights Data Analysis Group

If you see a statistic online, try asking two questions:

What’s included in the data?

What might be missing?

In this episode of How to Tell the Truth With Statistics, Dr. Megan Price explains why those questions are the key to spotting bias in data.

Watch now: www.youtube.com/watch?v=9Bqp...

Informes — Kilómetro 0 En esta sección encontrarás informes que hemos desarrollado con datos y análisis sobre la Policía de Puerto Rico.

13 years ago, Puerto Rico adopted reforms to try to rein in police abuses.

Today, Kilómetro O, with assistance from @hrdag.org, published a report showing a pattern of ongoing police violence in Puerto Rico marked by racial bias.

www.kilometro0.org/informes

How communities are using databases to hold police accountable YouTube video by Human Rights Data Analysis Group

"Wandering officers" - police officers who kill someone, lie, or engage in other unethical behavior, and then move jurisdictions to evade accountability.

A new, searchable database is helping track them down. www.youtube.com/watch?v=_mwP...

An Introduction to Bias in Statistics with Dr. Megan Price YouTube video by Human Rights Data Analysis Group

We’re living in a moment where people either believe everything they see online... or nothing at all.

What we need is discernment.

In this conversation with Dr. Megan Price, we talk about how understanding bias in data can help us evaluate the numbers we encounter.

www.youtube.com/watch?v=9Bqp...

How communities are using databases to hold police accountable YouTube video by Human Rights Data Analysis Group

Watch now: families, activists, and data scientists came together for a powerful event in Oakland, CA where we explored how a new database supports efforts toward police accountability. www.youtube.com/watch?v=_mwP...

New Paper: Towards a science of AI agent reliability Quantifying the capability-reliability gap

The AI industry doesn’t have good tools for measuring reliability, or even a good definition of reliability. @sayash.bsky.social and @randomwalker.bsky.social seek to define reliability in a functionally useful way. www.normaltech.ai/p/new-paper-...

HRDAG's Year in Review: 2025 This year at HRDAG we re-dedicated ourselves to a shared clarity of purpose, grounded in the Universal Declaration of Human Rights. Our recommitment has been compelled by a dramatic shift in the…

In a year when truth felt fragile, @hrdag.org doubled down on a simple idea: evidence matters.

Our newly published annual review shows how we bring statistical rigor and scientific values to the defense of human rights—including here in the United States. hrdag.org/2026/03/03/h...

Assassination of WHRD Yanar Mohammed, Emblematic Figure of the Feminist Movement, Condemned - Gulf Centre for Human Rights The assassination of woman human rights defender Yanar Mohammed, a prominent Iraqi feminist and President of the Organisation of Women’s Freedom in Iraq (OWFI), on 02 March 2026 in Baghdad,…

Today we mourn the assassination of feminist and human rights defender Yanar Mohammed, killed in her own home March 2 in Baghdad.

We join the human rights community in calling for justice and an independent investigation.

www.gc4hr.org/assassinatio...

Colorful poster that says "Justice for Yanar" and a photo of Yanar Mohammed

On International Women's Day, we join our partners around the world in decrying the assassination of Iraqi human rights defender Yanar Mohammed. Her death is part of a broader pattern of attacks targeting activists, journalists, and human rights defenders in Iraq. www.gc4hr.org/assassinatio...

Straight Outta The Gang Database: Challenging Gang Allegations through the RJA Since the California Racial Justice Act (CRJA)  passed in 2020, San Mateo County has seen very few cases move in the interest of justice, meaning, we have not seen any case move past the prima facie s...

Menlo Park’s gang database included over 100 people listed as “Taliban” members, 93% of whom were Black.

One of them is using California's Racial Justice Act to fight back. www.rjaction.org/post/straigh...

In 377 HIV clinics in Malawi, statistical sampling is key to assessing the data quality on patient treatment cards.

Done right, sampling can be a powerful tool for catching errors. statmodeling.stat.columbia.edu/2026/02/24/s...

HRDAG's Year in Review: 2025 This year at HRDAG we re-dedicated ourselves to a shared clarity of purpose, grounded in the Universal Declaration of Human Rights. Our recommitment has been compelled by a dramatic shift in the polit...

What happens when the patterns we’ve studied in other countries start showing up closer to home?

HRDAG’s 2025 review reflects on how the rise of tyranny in the United States is changing our work and our threat model.

Read it here: hrdag.org/2026/03/03/h...