Great human tutors intuitively build up an understanding of what their students know, and leverage that in their teaching. The knowledge models we've built can give AI tutors the power to do something similar by explicitly modeling (and interrogating) a personalized model of the learner's knowledge.
![[MIND 2019] Jeremy Manning: Can we improve real-world learning using scalable AI teachers?](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:6q3hl3ew33eemos5babl7b4l/bafkreia3jhast6j7a6gp77654d5kqk43s7e32umhi6efqzrseem3nzn2g4)
Where we're going next is using these maps as the core representations of AI tutoring systems. By accurately modeling landscape of someone's knowledge, we think we can use that representation to teach people more effectively: www.youtube.com/watch?v=FMmP...
We've also put together other tools for probing and visualizing these maps. Here is a fun project that uses Blender to create synthwave-style 3D renders of knowledge maps: github.com/ContextLab/k...
If you're curious what *your* map looks like, you can map out your own knowledge using our web demo: context-lab.com/mapper/. It usually takes around 30 or so questions to converge on a stable representaiton. It can be a highly revealing and humbling thing to learn what you know!
We also found that each person has a unique knowledge map, which can be revealed by asking just a few multiple-choice questions. We're excited about what this unlocks: imagine matching up study partners, conversation partners, or even social relationships by studying how people's maps align!
Eventually, for concepts that are too far apart, predictive accuracy falls to "chance" and the model ends up predicting that your chances of answering a question correctly are equal to whatever your overall average performance is, across all questions (regardless of content).
One finding is that knowledge is fundamentally *smooth*. Given that you've answered a question correctly, the probability of answering another question correctly falls off smoothly with distance. (Same with *incorrect* responses.)
So these tests showed that the maps we get out using this approach reflect people's actual conceptual understanding of real course content! Next, we looked at the geometric properties of the maps. We found a bunch of interesting things.
And even harder: construct a map using all but one question about domain X, and predict performance on the held-out domain X question. This is a highly conservative test of the model's *specificity*, because it requires the predictive model to distinguish between very similar concepts.
Harder is: construct a map using domain X and predict performance on questions from domain Y. This tests the model's ability to *generalize* to unseen, unrelated content.
We tested this in a few different ways, and all of the tests indicated that the maps were predictive of quiz performance! The simplest test is: given a map constructed from all but one question, can we predict how you'll answer that last question?
First, and most importantly: given that someone's knowledge map shows that they have X level of knowledge about some concept, does that correspond to their actual knowledge? We show that the predicted knowledge track closely with people's chances of correctly answering held-out quiz questions.
We developed a process for efficiently building a "map" of one person's knowledge using responses to short multiple-choice quizzes. In our paper, we tested two things: (1) the quality of those maps and (2) their geometric properties.
We had our participants answer multiple-choice quiz questions before and after watching a series of @khanacademy.org lectures.
We use text embeddings to define a coordinate system for human knowledge. Any text (video transcripts, quiz questions, etc.) can be localized in the embedding space, which lets us capture the idea that knowing about one concept also increases the chances that you know about other related concepts.
Curious what a representation of "everything" you know might look like? Wonder how you might fill it in?
Check out our demo and paper (led by @paxt0n4.bsky.social and now out in @natcomms.nature.com ), or read on to learn more!
Demo: context-lab.com/mapper/
Paper: www.doi.org/10.1038/s414...
And if you actually want to see the diagram, here you go-- there was some weird blurring in the previous message.
More info: cdl-scheduler.readthedocs.io/en/latest/ar...
Curious how it works? We use @github.com pages to host a static frontend site, along with @developers.google.com apps scripts as a serverless backend. GitHub actions do scheduled maintenance and Google Calendar, Gmail, and Sheets provide the scheduling, notification, and database infrastructure.
Do you use @youcanbookme.bsky.social or @calendly-official.bsky.social for scheduling? We were frustrated with recent increases in pricing, so we tapped @anthropic.com's Claude to help! You can easily use it yourself: 100% free to set up and host: cdl-scheduler.readthedocs.io/en/latest/
With our v1.0 release, dream-stream now comes in Android 🤖!
context-lab.com/dream-stream...
I made a quirky little web app to help guide your lucid dreams: context-lab.com/dream-stream/
It's kind of like a "netflix" or "spotify" for lucid dreaming-- you select different narratives to form a playlist, and then it uses your device's microphone to start playing when it detects you're in REM.
Excited to be teaching a new undergraduate course on Models of Language and Conversation this term!
Check it out here: context-lab.com/llm-course/
I've added lots of fun interactive demos of chatbots and NLP techniques that let students dig into the approaches.
Remember when grinding leetcode was still a thing? If you'd like to hone your coding skills, or even just return to that simpler time for nostalgia's sake, you might enjoy this project from our group: github.com/ContextLab/l...
Happy hacking! 👩💻
A plot showing a 3D projection of 8 "authors" (each represented with a differently colored and labeled dot). Stylistic distances between authors are reflected by spatial distances in the plot.
🚨 New preprint alert!
We use trained-from-scratch GPT-2 models to characterize & capture the unique writing styles of individual authors. We also develop a new LLM-based relative stylometric measure.
Paper: arxiv.org/abs/2510.21958
Code/data: github.com/ContextLab/l...
🤗: huggingface.co/contextlab
🤠 New release announcement for our datawrangler package! Try it using:
pip install --upgrade pydata-wrangler
Lots of awesome performance improvements (including native polars support!), simplified API, support for @hf.co text embeddings, etc. More info here: data-wrangler.readthedocs.org
We're super excited to announce that we've officially convinced @cgonciulea.bsky.social to join our rag-tag (but VERY classy) team of science nerds this fall as a @dartmouthpbs.bsky.social PhD student 🎉🥳🤓🧠🧑🔬🎓!!
Just finished a draft of my Models of Memory (grad) course that I'm teaching this spring! Please share/borrow/re-use/follow along as desired, and if you have feedback or suggestions I'd really love to hear (especially while I can still change it)!
All materials are here: github.com/ContextLab/m...
Re-sharing from Xitter-- new paper out in @pnas.org (lead: Lucy Owen)!
We show how the informativeness and compressibility of brain activity patterns change under different levels of cognitive engagement: www.pnas.org/doi/10.1073/...
Lots of neat stuff in there!
Excited to have this out in @natureportfolio.bsky.social!
We found that real & fictional people communicate roughly 1.5x more about the past than the future.
In turn, this influences the interferences we make about past & future events in *other* people's lives.
www.nature.com/articles/s41...
Now live in SoftwareX!
www.sciencedirect.com/science/arti...