Advertisement · 728 × 90

Posts by Ian Arawjo

PSA for anyone submitting to HCI conferences: Do not leave the "suggested reviewers" section blank. It is your greatest (and only) way to heighten the chances your paper will be reviewed by someone who really cares about your research topic. Plus, it helps the 1AC!

17 hours ago 0 0 0 0

Looking forward to the open source release, but it sounds like they’re treating model weights as static variables in code rather than data in an ML library, which means the Rust compiler can optimize 1000x better

3 days ago 58 9 3 0

Thanks for this. What’s crazy too is that I can’t even access the podcast to double-check if it might be grossly misrepresenting my work. Checked my papers from CHI, but blocked from viewing the podcast under just “Basic Edition“ access to ACM.

3 days ago 2 0 0 0

I had heard people at #CHI2026 say that they didn't want AI-generated podcasts to be made on top of their work. While this perspective is totally understandable, I thought the podcasts could potentially be useful. But after looking up one of mine, I can't believe this feature was allowed to launch.

3 days ago 19 11 2 0

“Everyone I met knew, at some level, that AI either means that nothing matters—a kind of creeping techno-nihilism—or that everything that has always mattered—humanism, human values—is all that ever mattered, and our tool tinkering had always been a distraction.” 🙏

4 days ago 0 0 0 0
Post image

the first paper from my phd project is now out and i'm presenting it tomorrow morning at #chi2026! it's an explorative design project about supporting players' rule experimentation through game design. you can read/download from this link dl.acm.org/doi/10.1145/...

1 week ago 29 10 2 0

The paper is now officially out in the #chi2026 proceedings!

You can download the dataset, as well as the autoethnographic memos, as Supplementary Material

dl.acm.org/doi/10.1145/...

1 week ago 93 39 3 1

Retweeting this, just because it seems not enough papers at #CHI2026 are declaring their relevance to AI: #makeit100percent #ensureAIrelevanceNow

1 week ago 9 1 0 0

“The real threat is a slow, comfortable drift toward not understanding what you're doing. Not a dramatic collapse. Not Skynet. Just a generation of researchers who can produce results but can't produce understanding.”

1 week ago 2 0 0 0
Advertisement

@devamarh.bsky.social knows

1 week ago 0 0 0 0
Preview
GitHub - ianarawjo/promptstats: Statistical analysis methods for comparing prompt and model performance in LLM evaluations. Statistical analysis methods for comparing prompt and model performance in LLM evaluations. - ianarawjo/promptstats

The Stats for LLM Evals guide uses `promptstats` library, which we're building out together with the guide. github.com/ianarawjo/pr... I'll also post investigations and updates over at our Substack, but probably after the CHI conference: substack.com/@statsforevals

2 weeks ago 2 0 0 0
Preview
Statistics for LLM Evals A research-backed guide to statistical methods for LLM and AI model evaluations. Learn to compare models, prompts, and agents with confidence intervals, bootstrap methods, and hypothesis tests.

Stats for Evals is now live, and we got a site, too: statsforevals.com We'll be posting regular investigations across the summer. For now, we're starting with the basics: comparing models and prompts. Also has resources, principles, example code, and guidance for others:

2 weeks ago 19 2 2 0

It would be cool if HCI had: 1) an open reviewing platform, 2) a quid pro quo credit system like CritiqueCircle with added kudos by experts for quality reviews, 3) anonymization of reviewers (but where you can see fuzzy metrics of reviewer quality)

2 weeks ago 0 0 0 0

"you cheated not only the game but yourself" is not how i feel about video games at all but it is how i feel about people using LLMs to "write" essays for them

2 weeks ago 363 42 3 0

“*Claude Code used regex instead of…” Fixed that for you.

2 weeks ago 0 0 0 0
Post image

Also, Montréal HCI is hiring! Looking for one PhD student (~Winter 2027). Also looking for stats/coding collaborators for our LLM evals project (the latter, can be anyone—doesn't matter as long as you know your stuff!). See us at CHI and say "bonjour hi"! 5/5

3 weeks ago 2 1 0 0

I’ll also attend two workshops: HEAL (evaluating LLMs) and one on LLMs as simulated participants—presenting a position paper "That’s Enough about AI Replacing Users in User Research" (and trying not to step on too many toes in the process!) 🧵 4/5

3 weeks ago 0 0 1 0
Advertisement
Post image

Paper 2: "Reporting and Reviewing LLM-Integrated Systems: Challenges and Considerations." An interview study of how authors report systems with LLM components, and how reviewers grill them, with guidelines for HCI scholars when reporting LLM components in HCI papers. 🧵 3/5

3 weeks ago 2 1 1 0
Post image

First up: "How Notations Evolve: A Historical Analysis with Implications for Supporting User-Defined Abstractions." A historical analysis of notational systems, from quantum circuits to dance notations to SignWriting, with implications for abstraction co-creation. 🧵 2/5

3 weeks ago 2 2 1 0

Montréal HCI is headed to sunny Barcelona for #CHI2026! ☀️ We’re presenting two exciting, rather unique papers and joining two workshops. Details below! 🧵 1/5

3 weeks ago 6 0 1 0
Preview
Stats for Evals | Substack Musing about statistical analysis for LLM evals. Follow for updates on the Stats for Evals project and promptstats library, including concrete investigations and recommendations.

I've made a Substack for Stats for LLM Evals progress. We'll be releasing a website soon, but progress will be piecemeal (first release is focused on model comparison, prompt comparison, and model x prompt). Subscribe here for regular updates: substack.com/@statsforevals

3 weeks ago 4 1 0 0
Preview
It Sucks to Work in the Video-Game Industry Right Now Even developers who successfully release big hits, like Fortnite and Battlefield, are losing their jobs

I don't think people understand how hard it is to work in the video-game industry right now. If you've been laid off, it can take months if not years to find new work. If you haven't been laid off, you're anxious that you will be laid off.

This week's column: www.bloomberg.com/news/newslet...

3 weeks ago 2285 574 57 71
Preview
Reporting and Reviewing LLM-Integrated Systems in HCI: Challenges and Considerations What should HCI scholars consider when reporting and reviewing papers that involve LLM-integrated systems? We interview 18 authors of LLM-integrated system papers on their authoring and reviewing expe...

Submitting an AI-powered system paper to #UIST2026 ? Wish you could sense what reviewers were thinking, and how to maximize your chance of acceptance? Check out our #CHI2026 paper, “Reporting and Reviewing LLM-integrated Systems in HCI”, for tips and guidelines: arxiv.org/abs/2602.05128

3 weeks ago 3 1 0 0

now desk reject the papers of reviewers whose human-written reviews are worse than an LLM 😂 … #icml2026

1 month ago 13 1 0 1
Advertisement
Post image

I want my workflow to feel more like this. One big reconfigurable space for deep work

1 month ago 4 2 0 1

HCI summer research opportunity 📣 My group has two openings for research assistants this summer, both in scientific tools for thought. Applicants are welcome at any level. Please help me share the news! andrewhead.info/positions/20...

1 month ago 11 4 0 0
Preview
I’m writing an HCI paper about an AI-powered system. What should I report? Eight Guidelines to Improve Research Quality and Enhance Chance of Acceptance

Writing an HCI paper about an AI-powered system to a venue like UIST 2026 or CHI 2027? Wondering what reviewers expect you to report, and how to approach paper framing and writing? Check out our reporting guidelines: medium.com/p/7c3ae86341...

1 month ago 2 1 0 0
Conference Programs

#CHI2026 program (the draft) is out: programs.sigchi.org/chi/2026/pro... a monster sized CHI that will definitely be fun and intellectually stimulating. Huge kudos to Pablo Cesar and Heloisa Candello, as well as our assistants for making this possible in such a short time ! Check it out!

1 month ago 5 2 0 0
Post image

To date, HCI researchers have had no support on signaling their paper's relevance to AI, esp. when that connection is tenuous at best. We introduce a systematic framework to ensure LLMs are mentioned at every stage of paper reporting—from framing, to evaluation, to implications.

1 month ago 25 3 0 2

Those are great topics. I am reacting more to papers that aren’t about AI directly but where authors shoehorn phrases like “LLM-based” into their titles and framing. If a topic is not directly about LLMs, then, it shouldn’t be pressured to answering “but what is the relevance for AI?”

1 month ago 1 0 1 0