CSE Colloquium: Relational Attention for Data-Efficient Language Modeling
Title: Relational Attention for Data-Efficient Language Modeling Presenter: Adrian Brasoveanu, UCSC Linguistics (Joint work with Ece Takmaz and Jakub Dotlačil) Abstract: We present Relational BabyLM, a system submission to the BabyLM 2026 challenge that combines two cognitively motivated inductive biases in a single decoder-only Transformer. Architecturally, we replace standard self-attention with a Dual Attention Transformer (DAT), […]