More details can be found at our website here: 2026.qanta.org

Questions? DM us or e-mail us at qanta@googlegroups.com.

Sign up to play here: qanta-org.github.io/competition/...
Submit systems here: qanta-org.github.io/competition/...
Submit adversarial questions here: qanta-org.github.io/competition/...
Take part in our ICML 2026 workshop: qanta-org.github.io/competition/...

Human teams playing multimodal quiz bowl at George Washington University, staring at a screen with a picture of two Mormon missionaries on bikes. A moderator is reading the question, the players are looking confused and are trying to interpret computer output.

QANTA is back for 2026, introducing images into our questions! You can play quizbowl with AI, build systems, or write adversarial questions… and win cool prizes! All happening on 27. June at UMD, online 28. June, or at an ICML 2026 workshop in Seoul – completely free with cash prizes!

Logo of a human and a computer looking at a Korean painting together, advertising the ICML 2026 workshop.

Can you spot when AI bluffs? Can you outguess AI or work with one to dominate trivia? Can you make an AI that can figure out tricky visual questions?

1) Low-brow but fun: Harry Turtledove, e.g. Worldwar (Aliens disrupt WW2).

2) Fatherland by Richard Harris, Nazis win WW2

3) Pashazade : The First Arabesk by Jon Courtenay Grimwood, the central powers won WW1

4) Fire on the Mountain by Terry Bisson: Harper's Ferry was a victory

Liebe Medien, ich kann die Schlagzeile "Keine Einigung zwischen USA und Dänemark" nicht mehr sehen. Wenn ein Bewaffneter eine Bank stürmt, titelt ihr doch auch nicht: "Räuber und Kassiererin finden keinen Konsens über Geldübergabe." Hört auf, imperiale Aggression als normale Diplomatie zu framen.

Don't forget Snider's!

I agree that Mallet >> Gensim, but VI can also work well. It's just more finicky, and Gensim's implementation doesn't do the things that Mallet does to optimize hyperparameters. So just like we shouldn't let LDA get a bad rap because of Gensim, we shouldn't let VI get a bad rap because of Gensim.

I'm so ahead of the curve on AI, I've been using this strategy for a decade.

Um, actually, that's a Diaeresis:

www.frathwiki.com/Diaeresis_an...

I believe the planet Diaeresis is also the planet where Reva Sevander kidnapped Leia until she was rescued by Obi-Wan.

Der Meister des jüngsten Tages - Krimi Hörspiel - Leo Perutz YouTube video by Hörspiel Teutone

Audio book?

www.youtube.com/watch?v=Rgkf...

I think you mean: Terror has spread. Even in The Loop, hundreds run from gunshots. Traffic has ground to a halt. Unable to do anything to prevent the stampede, hundreds scream from the sidewalks. "Police" stand by, doing nothing.

We had come at it more from the position of trying to use as few dev examples as possible (to keep them secret). I.e., use the best items you could and every model uses the exact same. But it makes sense to use the adaptive testing scenario if you don't mind potentially exposing more dev.

Link to paper since I ran out of room:

users.umiacs.umd.edu/~ying/docs/2...

In 2021, we proposed using IRT to find bad examples and to create more targeted leaderboards (Evaluation
Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?).

From my reading, the big difference seems to be that they're also using the agent's skill, which is super cool!

We also found that it's helpful for improving uncertainty estimation of models:

arxiv.org/abs/2205.12507

If it said that 1990 was "about 10 years ago", I would say that it has reached tenured faculty-level intelligence.

Today's the deadline to apply for an AI-specific teaching track position at UMD:

umd.wd1.myworkdayjobs.com/UMCP/job/Uni...

Please join us!

A couple of weeks ago I left my family behind at a cable car station to finish climbing to the peak of a mountain because they were too scared to continue. When I reached the top, my phone gave a notification: new podcasts available for download. Apparently LMU has an observatory on Wendelstein.

Do you mean salary, physical facilities, work environment, or funding ecosystem?

At the risk of picking out one of my favorite children, this was the paper with our best traditional video of this cycle (thanks to Jon May for playing along):

t.co/QQlgwzo6jf
t.co/2G6kwAAPMy

Joy Wongkamjan on X: "Our paper CTRL-D is accepted to ACL Findings and will be presented at ACL 2025! 🗓️Poster session: 18:00–19:30 (Level 0 Exhibit Halls X4/X5) I’m sad I can’t be there, but Jordan (@boydgraber) will! You’ll enjoy learning about CTRL-D from him. Now… what is CTRL-D? 🔍 https://t.co/ucIPZRHBF1" / X Our paper CTRL-D is accepted to ACL Findings and will be presented at ACL 2025! 🗓️Poster session: 18:00–19:30 (Level 0 Exhibit Halls X4/X5) I’m sad I can’t be there, but Jordan (@boydgraber) will! You’ll enjoy learning about CTRL-D from him. Now… what is CTRL-D? 🔍 https://t.co/ucIPZRHBF1

Finally, this evening I'll be standing in for
@wwongkamjan.bsky.social
at the Findings poster (18:00, Hall X4/X5): Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL

x.com/joywwong/sta...

t.co/j3Iibs9hEn
www.youtube.com/watch?v=NJKd...

Yoo Yeon Sung@ACL2025 on X: "I’ll be presenting this work in Room 1.62 today! If you're curious about how calibration errors in LLMs can be measured through human calibration, come find me and @enfleisig! 📍Oral Session 3 - HC: Human-centered NLP 📅Monday, July 28@ 2PM" / X I’ll be presenting this work in Room 1.62 today! If you're curious about how calibration errors in LLMs can be measured through human calibration, come find me and @enfleisig! 📍Oral Session 3 - HC: Human-centered NLP 📅Monday, July 28@ 2PM

In the second oral paper (14:22 PM, Room 1.62),
@yysung.bsky.social is presenting: GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibration

x.com/YooYeonSung1...

(Short version: quiz bowl, a dumb trivia game, shows humans' calibration > LLMs'.)

t.co/LagmrMjVgi
t.co/aGM7LC2m0q

Nishant is ill-prepared for ACL2025 on X: "While personalization is great, it's not perfect. We found our strategy could easily let users jailbreak the model 😭 With some extra safeguards (e.g. refusal training), we think inferred personas could become a promising way to boost personalization in post-training recipes! 🧑‍🍳 https://t.co/Fpu1Bl34fI" / X While personalization is great, it's not perfect. We found our strategy could easily let users jailbreak the model 😭 With some extra safeguards (e.g. refusal training), we think inferred personas could become a promising way to boost personalization in post-training recipes! 🧑‍🍳 https://t.co/Fpu1Bl34fI

In the first poster session (11AM Monday, Hall X4/X5),
@nbalepur.bsky.social is presenting: Whose Boat Does it Float? Improving Personalization in Preference Tuning via Inferred User Personas

x.com/NishantBalep...

My students and I are presenting three papers on Monday at #ACL2025 and this thread will recap them (including their videos).

The precursor to this paper "The Incoherence of Coherence" had our most-watched paper video ever, so I thought we had to surpass it somehow ... so we decided to do a song parody (of Roxanne, obviously):

youtu.be/87OBxEM8a9E

Which makes this:
users.umiacs.umd.edu/~ying/docs/n...

"The Hobbit"

2025 QANTA Player Signup Sign up for the human competition for our 2025 QANTA event. More information: https://sites.google.com/view/qanta/2025-competition/2025-human-teams

And you can signup for online mirror (June 21, 12:00 EST) here:
docs.google.com/forms/d/e/1F...

[Signup deadline: June 18 Anywhere on Earth]