Every lens leaves a blur signature—a hidden fingerprint in every photo.

In our new #TPAMI paper, we show how to learn it fast (5 mins of capture!) with Lens Blur Fields ✨

With it, we can tell apart ‘identical’ phones by their optics, deblur images, and render realistic blurs.

This is happening today at 2:10 PM during the Classroom Technology Session in G302!

Come check out how mixed reality supports note-taking and facilitates engagement! #CHI2025

Huge thanks to my amazing collaborators: Erin Kim, Sangho Suh, Ludwig Sidenmark, and Tovi Grossman. I'll present MaRginalia as my first CHI paper in the "Classroom Technology" session on Tuesday at 2:10PM-3:40PM in Room G302. If you are a note-taker, come check out our work and chat with us! 😉 (8/8)

MaRginalia introduces a new way to take notes and opens up further research for leveraging mixed reality for productivity and learning.
📄 Paper: arxiv.org/pdf/2501.16010
🎥 Video (4min): youtu.be/RBWh2WZriDY
🎙️ Talk (10min): youtu.be/EycV33cabZ0
👇 (7/8)

We conducted an exploratory user study with 12 participants to understand the system's usability and usefulness. Results found MaRginalia supports diverse workflows, has "good" usability on the System Usability Scale, and helps students stay engaged.👇 (6/8)

We introduced a Gaze+Pen interaction that operates similarly to a Wacom tablet. 👁️ Gaze is used as the initial cursor position. ✍️ Pen is used for adjusting the cursor, pushing buttons, and handwriting.👇 (5/8)

If the user misses something, they can quickly flip through prior slides and transcripts to catch up and see their annotations.👇 (4/8)

During the lecture, Marginalia automatically creates 📸 snapshots of slides and transcripts. When the user sees something important, they can directly annotate, and the content is seamlessly added to their notes.👇 (3/8)

Let us follow a user to see MaRginalia in action! The user can take handwritten notes on the tablet like they usually do, or take notes with mixed reality so that they can look at the lecture and stay engaged.👇 (2/8)

Illustration of the use case scenario.]{Illustration of the use case scenario. The user wears a mixed reality head-mounted display holding a stylus over a tablet. In front of the user is a room with a big display of slides and a lecturer. The MR spatial panels are overlaid in the environment. To the left of the display is the Notes Panel, which has notes that mirror what is on the tablet. The display has the Slides Panel overlaid with a navigator of previous slides and some control buttons. To the speaker's right is the Transcripts Panel, which displays the lecture transcript.

Take hand-written notes in mixed reality! Our #CHI2025 paper "MaRginalia" explores how mixed reality can better support university students’ note-taking during in-person lectures. Students can stay engaged without needing to look down at their devices. #HCI #XR 🧵 (1/8)

And to 4o's Ghibli style image recreations 👀

Is this going to be the phone to realize the a-world-without-apps vision? Curious of how they will market this to people walking into a T-Mobile store looking to replace their broken iPhone 👀

Need to convince my collaborators to join Bluesky ✊

I'm happy to chat more about the project here and at
@chi.acm.org #CHI2025!

MaRginalia: Enabling In-person Lecture Capturing and Note-taking Through Mixed Reality Students often take digital notes during live lectures, but current methods can be slow when capturing information from lecture slides or the instructor's speech, and require them to focus on their de...

📄 Preprint available on arXiv: arxiv.org/abs/2501.16010

MaRginalia: Enabling In-person Lecture Capturing and Note-taking Through Mixed Reality YouTube video by Leping Qiu

✍️ Introducing MaRginalia: A new way to take handwritten notes using #HoloLens and #iPad. Now, with fewer distractions! - #CHI2025 Paper.

👉 Check out our project video to learn more: www.youtube.com/watch?v=RBWh...

🤝 In collaboration with Erin Kim, Sangho Suh, Ludwig Sidenmark and Tovi Grossman