Transformers: Origins An unofficial origin story of the transformer neural network architecture.

I have converted a portion of my NLP Online Masters course to blog form. This is the progression I present that takes one from recurrent neural network to seq2seq with attention to transformer. mark-riedl.medium.com/transformers...

Open RL Benchmark: Comprehensive Tracked Experiments for... In many Reinforcement Learning (RL) papers, learning curves are useful indicators to measure the effectiveness of RL algorithms. However, the complete raw data of the learning curves are rarely...

Neurips reviews are now publicly available.
Don't forget to check out Open RL benchmark, very useful when implementing algorithms or checking performance/impact of hyperparameters.

openreview.net/forum?id=ZDv...

ALTA: Compiler-Based Analysis of Transformers We propose a new programming language called ALTA and a compiler that can map ALTA programs to Transformer weights. ALTA is inspired by RASP, a language proposed by Weiss et al. (2021), and Tracr (Lin...

I’m pretty excited about this one!

ALTA is A Language for Transformer Analysis.

Because ALTA programs can be compiled to transformer weights, it provides constructive proofs of transformer expressivity. It also offers new analytic tools for *learnability*.

arxiv.org/abs/2410.18077

AI Safety Events and Training: 2024 Week 46 update This is a weekly newsletter listing newly announced AI safety events and training programs.

AI Safety Events and Training: 2024 Week 46 update

aisafetyeventsandtraining.substack.com/p/ai-safety-...

A tweet from Tim van der Zee, from August 10, 2017, that reads: "Academia is a bunch of people emailing "sorry for the late response" back and forth until one of them gets tenure."

This was seven years ago. I think about this often.

On Evaluating Explanation Utility for Human-AI Decision Making in NLP Is explainability a false promise? This debate has emerged from the insufficient evidence that explanations help people in situations they are introduced for. More human-centered, application-grounded...

I will be at #EMNLP2024! My student 𝙁𝙖𝙩𝙚𝙢𝙚 𝙃𝙖𝙨𝙝𝙚𝙢𝙞 𝘾𝙝𝙖𝙡𝙚𝙨𝙝𝙩𝙤𝙧𝙞 will present "On Evaluating Explanation Utility for Human-AI Decision Making in NLP" in the poster session on 𝗪𝗲𝗱𝗻𝗲𝘀𝗱𝗮𝘆 𝟭𝟬:𝟯𝟬𝗮𝗺: arxiv.org/abs/2407.03545 1/

The AI Interdisciplinary Institute at the University of Maryland (AIM) is hiring

40 new faculty members

in all areas of AI, particularly:
- accessibility,
- sustainability,
- social justice, and
- learning;

building on computational, humanistic, or social scientific approaches to AI.

>

Schmidt Sciences is outlining the timeline for a new program to support research at the intersection of artificial intelligence and the humanities. Open call for proposals to come Dec 15. www.schmidtsciences.org/humanities-a...

This one is a study on voting-based evaluation to comparisons of models in LMSYS Chatbot Arena leaderboard, by independent researcher Nick Ryan. Simulations show that two Condorcet-consistent methods (Copeland and Ranked Pairs) can be robust to uncertain/noisy evals.

nickcdryan.com/2024/09/06/u...

Honestly very disappointed since joining BlueSky, this is not the weather app I was hoping for

Text Shot: Further experiments reveal two key insights about the generalization mechanisms of these models: (1) the models fail to abstract general physical rules and instead exhibit "case-based" generalization behavior, i.e., mimicking the closest training example; (2) when generalizing to new cases, models are observed to prioritize different factors when referencing training data: color > size > velocity > shape. Our study suggests that scaling alone is insufficient for video generation models to uncover fundamental physical laws, despite its role in Sora's broader success.

How Far is Video Generation from World Model: A Physical Law Perspective https://arxiv.org/abs/2411.02385v1 #AI #video

NSF makes you say who you got conflicts (coauthored) with. We (really just Jordan Matelsky) just built you a tool for that. Literally one click: bib.experiments.kordinglab.com/nsf-coa

New York Theory Day finally returns on December 6, 2024, after being put on hiatus during COVID.

Will be held at @nyutandon.bsky.social in Brooklyn. Registration is free!

Ft stellar speakers Amir Abboud, Sanjeev Khanna, Rotem Oshman, and
Ron Rothblum!

sites.google.com/view/nyctheo...

Hello… world?