I am currently a doctoral student in the Electrical Engineering and Computer Science (EECS) Department at MIT, advised by Prof. Antonio Torralba. My research focuses on interpretability and generative AI.

Prior to my PhD, I earned a Bachelor’s degree in Mathematics from the University of Vienna and the Autonomous University of Barcelona. I then worked as an AI Engineer at Twenty Billion Neurons, later acquired by Qualcomm. I went on to complete an MSc in Computer Science at the University of Oxford, where I also contributed as a research intern at the Torr Vision Group.

In addition to my academic and professional commitments, I am actively engaged in the academic community. I serve as a reviewer for several esteemed conferences and journals, including ICCV (2021, 2023), CVPR (2022–2025), ICLR (2022), NeurIPS (2021), ACM FAccT (2024), TPAMI, and IJCV. Furthermore, I co-organize workshops at major conferences, such as CVG at ICML 2024 and MIV at CVPR 2025. I have also been co-orgnizing the MIT Vision and Graphic Seminar since 2022.

News

2024

• October Our work “NewMove: Customizing text-to-video models with novel motions” was accepted at ACCV 2024!
• Our work “Concept sliders: Lora adaptors for precise control in diffusion models” was presented at ECCV 2024!
• We presented our paper “AirLetters: An Open Video Dataset of Characters Drawn in the Air” at HANDS Workshop in ECCV 2024!
• The work in collaboration with the Data Provenance Initiative got accepted at Neurips 2024!
• April Our workshop Text, Camera, Action! Frontiers in Controllable Video Generation got accepted at the ICML 2024’ Conference in Vienna!
• February Honoured to be invited as a speaker at the Rising Stars in AI Symposium 2024 in Saudi Arabia!

2023

• December Our work “FIND: A Function Description Benchmark for Evaluating Interpretability Methods” was accepted at NeurIPS 2023!
• December Our work “Unified Concept Editing in Diffusion Models” was accepted at WACV 2023!
• June - August I had the pleasure to intern at Adobe Research under the supervision of Bryan Russell, Richard Zhang, Josef Sivic and Eli Shechtman.
• May: Our paper “Erasing Concepts from Diffusion Models” has been chosen for a spotlight presentation at the Machine Learning Advances Symposium.

2022

• July: Proud to have won the second place of the poster competition at the International Computer Vision Summer School .
• June: Our paper “Disentangling visual and written concepts in CLIP” got awarded an Oral (~4% acceptance rate) at CVPR 22’ in New Orleans.
• June-August I was fortunate to intern at Netflix Research under the supervision of Mahdi M Kalayeh

NewMove: Customizing text-to-video models with novel motions

Joanna Materzyńska, Josef Sivic, Eli Shechtman, Antonio Torralba, Richard Zhang, Bryan Russell

ACCV 2024

[website] [code]

Consent in Crisis: The Rapid Decline of the AI Data Commons

Shayne Longpre, Robert Mahari, Ariel Lee, Campbell Lund, Hamidah Oderinwale, William Brannon, Nayan Saxena, Naana Obeng-Marnu, Tobin South, Cole Hunter, Kevin Klyman, Christopher Klamm, Hailey Schoelkopf, Nikhil Singh, Manuel Cherep, Ahmad Anis, An Dinh, Caroline Chitongo, Da Yin, Damien Sileo, Deividas Mataciunas, Diganta Misra, Emad Alghamdi, Enrico Shippole, Jianguo Zhang, Joanna Materzyńska, Kun Qian, Kush Tiwary, Lester Miranda, Manan Dey, Minnie Liang, Mohammed Hamdy, Niklas Muennighoff, Seonghyeon Ye, Seungone Kim, Shrestha Mohanty, Vipul Gupta, Vivek Sharma, Vu Minh Chien, Xuhui Zhou, Yizhi Li, Caiming Xiong, Luis Villa, Stella Biderman, Hanlin Li, Daphne Ippolito, Sara Hooker, Jad Kabbara, Sandy Pentland

NeurIPS 2024 Track on Datasets and Benchmarks

[paper]

AirLetters: An Open Video Dataset of Characters Drawn in the Air

Rishit Dagli, Guillaume Berger, Joanna Materzyńska, Ingo Bax, Roland Memisevic

ECCVW 2024 HANDS Workshop

[paper]

Concept sliders: Lora adaptors for precise control in diffusion models

Rohit Gandikota, Joanna Materzyńska, David Bau, Antonio Torralba

ECCV 2024

[paper] [code]

Unified concept editing in diffusion models

Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyńska, David Bau

WACV 2024

[paper] [code]

Find: A function description benchmark for evaluating interpretability methods

Sarah Schwettmann, Tamar Shaham, Joanna Materzyńska, Neil Chowdhury, Shuang Li, Jacob Andreas, David Bau, Antonio Torralba

NeurIPS 2023 Track on Datasets and Benchmarks

[paper] [code]

Erasing Concepts from Diffusion Models

Joanna Materzyńska, Rohit Gandikota*, Jaden Fiotto-Kaufman, David Bau

ICCV 2023

[paper] [code]

Disentangling visual and written concepts in CLIP

Joanna Materzyńska, Antonio Torralba, David Bau

CVPR 2022 (Oral)

[paper] [code]

Re-enacting video shots with fictional characters

Joanna Materzynska, David Bau, Antonio Torralba

ICCVW 2021 CVEU Workshop (Spotlight)

[paper]

Something-else: Compositional action recognition with spatial-temporal interaction networks

Joanna Materzyńska Tete Xiao, Roei Herzig, Huijuan Xu, Xiaolong Wang, Trevor Darrell

CVPR 2020

[paper] [code] [website]

The jester dataset: A large-scale video dataset of human gestures

Joanna Materzyńska Guillaume Berger, Ingo Bax, Roland Memisevic

ICCVW 2019 HANDS Workshop

[paper] [dataset]

The "something something" video database for learning and evaluating visual common sense

Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzyńska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, Florian Hoppe, Christian Thurau, Ingo Bax, Roland Memisevic

ICCV 2017

[paper] [dataset]

The SYNTHIA dataset: A large collection of synthetic images for semantic segmentation of urban scenes

German Ros, Laura Sellart, Joanna Materzyńska, David Vazquez, Antonio M Lopez

CVPR 2016

[paper]

To learn more about Captioning & Accessibility resources at the MIT please refer to the website.