Yonatan Bitton

Yonatan Bitton

Senior Research Scientist at Google, CS PhD

The Hebrew University of Jerusalem

Biography

I am a Research Scientist at Google Research in Tel-Aviv where I work on multimodal consistency.

My research is centered on improving large vision-and-language models. I develop feedback models for text-to-image and text-to-video applications, specifically designed to enhance the alignment of visual outputs with their corresponding textual prompts. Additionally, I work on multimodal factuality, including visual understanding and image or video-to-text evaluation, ensuring that the generated text is factually correct and attributable to trustworthy textual or visual sources.

I completed my PhD in The Hebrew University of Jerusalem, Israel. During my time there, I had the privilege of being advised by Dr. Roy Schwartz and Dr. Gabriel Stanovsky. My PhD talk "Bridging Vision and Language with Data: From Perception to Understanding" 🎬 record is available here. I did my MSc with Prof. Michael Elhadad and Prof. Eitan Bachmat, at the Ben Gurion University.

Download my complete CV: link.
📄 Download my bio: link.

Education

  • PhD in Computer Science (Vision-and-Language), 2020-2023

    The Hebrew University of Jerusalem, Israel

  • MSc in Computer Science (Natural Language Processing), Magna cum laude, 2018-2019

    Ben Gurion University of the Negev, Israel

  • BSc in Computer Science, 2015-2018

    Ben Gurion University of the Negev, Israel

Students

I've had the opportunity to collaborate with several MSc and PhD students towards their publication goals:

1. Wenbo (Gordon) Hu (University of California, Los Angeles) 1 3DLLM-Mem
2. Brian Gordon (Tel-Aviv University) 2 Mismatch Quest Unblocking Detailed Captions
3. Aviv Slobodkin (Bar-Ilan University) 1 RefVNLI
4. Moran Yanuka (Tel-Aviv University) 1 Bridging the Visual Gap
5. Mor Ventura (Technion – Israel Institute of Technology) 1 NL-Eye
6. Orr Zohar (Stanford University) 1 Video-STaR
7. Hritik Bansal (University of California, Los Angeles) 4 VideoPhy2 VideoPhy TALC Video-Con
8. Nitzan Bitton-Guetta (Ben-Gurion University of the Negev) 2 WHOOPS! Visual Riddles
9. Ron Yosef (The Hebrew University of Jerusalem) 2 IRFL EditInspector
10. Oren Sultan (The Hebrew University of Jerusalem) 1 ParallelPARC
11. Netta Madvil (The Hebrew University of Jerusalem) 1 Read, Look or Listen?

If you’d like to work together on vision-and-language research, send me an email.

Papers by Venue

24 peer-reviewed papers · 2021 – 2025

2025 5

ICLR 3 NL-Eye VideoSTaR VideoPhy
NAACL 1 Bridging the Visual Gap
WACV 1 Contrastive Sequential-Diffusion Learning

2024 10

NeurIPS 2 DataComp-LM Visual Riddles
ACL 2 Chain-of-Thought Verifier Visual Illustrations Sequences
ECCV 2 DOCCI Mismatch Quest
EMNLP 1 ImageInWords
CVPR 1 VideoCon
NAACL 1 ParallelPARC
AIES 1 Beyond Thumbs Up/Down

2023 6

NeurIPS 4 VisIT-Bench DataComp OpenFlamingo WYSIWYR
EMNLP 1 q2d
ICCV 1 WHOOPS!

2022 1

NeurIPS 1 WinoGAViL

2021 2

EMNLP 1 Data Efficient MLM for V&L
NAACL 1 Auto Contrast Sets from Scene Graphs

Publications

Unblocking Fine-Grained Evaluation of Detailed Captions: An Explaining AutoRater and Critic-and-Revise Pipeline
Unblocking Fine-Grained Evaluation of Detailed Captions: An Explaining AutoRater and Critic-and-Revise Pipeline
Brian Gordon*, Yonatan Bitton*, Andreea Marzoca, Yasumasa Onoe, Xiao Wang, Daniel Cohen-Or, Idan Szpektor
June 2025 arXiv preprint
arXiv Project Website
3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model
3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model
Wenbo Hu, Yining Hong, Yanjun Wang, Leison Gao, Zibu Wei, Xingcheng Yao, Nanyun Peng, Yonatan Bitton, Idan Szpektor, Kai-Wei Chang
May 2025 arXiv preprint 🎉 Best Paper Award at the Foundation Models Meet Embodied Agents Workshop @ CVPR 2025
arXiv Project Website
EditInspector: A Benchmark for Evaluation of Text-Guided Image Edits
EditInspector: A Benchmark for Evaluation of Text-Guided Image Edits
Ron Yosef, Moran Yanuka, Yonatan Bitton, Dani Lischinski
June 2025 arXiv preprint
arXiv Project Website
RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation
RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation
Aviv Slobodkin, Hagai Taitelbaum, Yonatan Bitton, Brian Gordon, Michal Sokolik, Nitzan Bitton Guetta, Almog Gueta, Royi Rassin, Itay Laish, Dani Lischinski, Idan Szpektor
April 2025 arXiv preprint
arXiv
VideoPhy2: Challenging Action-Centric Physical Commonsense Evaluation of Video Generation
VideoPhy2: Challenging Action-Centric Physical Commonsense Evaluation of Video Generation
Hritik Bansal*, Clark Peng*, Yonatan Bitton*, Roman Goldenberg, Aditya Grover, Kai-Wei Chang,
March 2025 arXiv preprint 🎉 Best Paper Award at the Building Physically Plausible World Models Workshop @ ICML 2025
arXiv Project Website
PaliGemma 2: A Family of Versatile VLMs for Transfer
PaliGemma 2: A Family of Versatile VLMs for Transfer
Andreas Steiner, André Susano Pinto, Michael Tschannen, Daniel Keysers, Xiao Wang, Yonatan Bitton, Alexey Gritsenko, Matthias Minderer, Anthony Sherbondy, Shangbang Long, Siyang Qin, Reeve Ingle, Emanuele Bugliarello, Sahar Kazemzadeh, Thomas Mesnard, Ibrahim Alabdulmohsin, Lucas Beyer, Xiaohua Zhai
December 2024 arXiv preprint
arXiv Blog HuggingFace
Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions
Bridging the Visual Gap: Fine-Tuning Multimodal Models with Knowledge-Adapted Captions
Moran Yanuka, Assaf Ben Kish, Yonatan Bitton, Idan Szpektor, Raja Giryes
November 2024 NAACL 2025
arXiv
KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities
KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities
Hsin-Ping Huang, Xinyi Wang, Yonatan Bitton, Hagai Taitelbaum, Gaurav Singh Tomar, Ming-Wei Chang, Xuhui Jia, Kelvin C.K. Chan, Hexiang Hu, Yu-Chuan Su, Ming-Hsuan Yang
October 2024 arXiv preprint
arXiv
NL-Eye: Abductive NLI for Images
NL-Eye: Abductive NLI for Images
Mor Ventura, Michael Toker, Nitay Calderon, Zorik Gekhman, Yonatan Bitton, Roi Reichart
October 2024 ICLR 2025
arXiv Website Code Dataset
DataComp-LM: In search of the next generation of training sets for language models
DataComp-LM: In search of the next generation of training sets for language models
Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng-Yu Hsieh, Dhruba Ghosh, Josh Gardner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo, Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexandros G. Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, Vaishaal Shankar
June 2024 NeurIPS 2024
arXiv Project Website Code Data
Contrastive Sequential-Diffusion Learning: An approach to Multi-Scene Instructional Video Synthesis
Contrastive Sequential-Diffusion Learning: An approach to Multi-Scene Instructional Video Synthesis
Vasco Ramos, Yonatan Bitton, Michal Yarom, Idan Szpektor, Joao Magalhaes
July 2024 WACV 2025
arXiv
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
Katherine M. Collins, Najoung Kim, Yonatan Bitton, Verena Rieser, Shayegan Omidshafiei, Yushi Hu, Sherol Chen, Senjuti Dutta, Minsuk Chang, Kimin Lee, Youwei Liang, Georgina Evans, Sahil Singla, Gang Li, Adrian Weller, Junfeng He, Deepak Ramachandran, Krishnamurthy Dj Dvijotham
June 2024 AIES 2024
arXiv
Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Models
Visual Riddles: a Commonsense and World Knowledge Challenge for Large Vision and Language Models
Nitzan Bitton-Guetta, Aviv Slobodkin, Aviya Maimon, Eliya Habba, Royi Rassin, Yonatan Bitton, Idan Szpektor, Amir Globerson, Yuval Elovici
July 2024 NeurIPS 2024, Datasets and Benchmarks
arXiv Project Website Explorer
VideoSTaR: Bootstrapping Weak Video Supervision for Visual Instruction Tuning
VideoSTaR: Bootstrapping Weak Video Supervision for Visual Instruction Tuning
Orr Zohar, Xiaohan Wang Yonatan Bitton, Idan Szpektor, Serena Yeung-Levy
July 2024 ICLR 2025
arXiv Project Website GitHub Dataset Demo
VideoPhy: Evaluating Physical Commonsense In Video Generation
VideoPhy: Evaluating Physical Commonsense In Video Generation
Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, Aditya Grover
June 2024 ICLR 2025
arXiv Project Website GitHub Dataset Model Demo
TALC: Time-Aligned Captions for Multi-Scene Text-to-Video Generation
TALC: Time-Aligned Captions for Multi-Scene Text-to-Video Generation
Hritik Bansal, Yonatan Bitton, Michal Yarom, Idan Szpektor, Aditya Grover, Kai-Wei Chang
May 2024 arXiv preprint
arXiv Project Website GitHub Dataset Model Demo PDF
ImageInWords: Unlocking Hyper-Detailed Image Descriptions
ImageInWords: Unlocking Hyper-Detailed Image Descriptions
Roopal Garg, Andrea Burns, Burcu Karagol Ayan, Yonatan Bitton, Ceslee Montgomery, Yasumasa Onoe, Andrew Bunner, Ranjay Krishna, Jason Baldridge, Radu Soricut
May 2024 EMNLP 2024
arXiv Project Website GitHub Explorer Dataset
DOCCI: Descriptions of Connected and Contrasting Images
DOCCI: Descriptions of Connected and Contrasting Images
Yasumasa Onoe, Sunayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset, Garrett Tanzer, et al.
April 2024 ECCV 2024
arXiv Project Website Explorer Dataset
ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies
ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies
Oren Sultan, Yonatan Bitton, Ron Yosef, Dafna Shahaf
March 2024 NAACL 2024
arXiv PDF
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
Alon Jacovi, Yonatan Bitton, Bernd Bohnet, Jonathan Herzig, Or Honovich, Michael Tseng, Michael Collins, Roee Aharoni, Mor Geva
February 2024 ACL 2024
arXiv Blog PDF Dataset
Generating Coherent Sequences of Visual Illustrations for Real-World Manual Tasks
Generating Coherent Sequences of Visual Illustrations for Real-World Manual Tasks
João Bordalo, Vasco Ramos, Rodrigo Valério, Diogo Glória-Silva, Yonatan Bitton, Michal Yarom, Idan Szpektor, Joao Magalhaes
May 2024 ACL 2024
arXiv
Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment
Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment
Brian Gordon*, Yonatan Bitton*, Yonatan Shafir, Roopal Garg, Xi Chen, Dani Lischinski, Daniel Cohen-Or, Idan Szpektor,
Dec 2023 ECCV 2024
arXiv Website PDF Code Dataset
VideoCon: Robust Video-Language Alignment via Contrast Captions
VideoCon: Robust Video-Language Alignment via Contrast Captions
Hritik Bansal, Yonatan Bitton, Idan Szpektor, Kai-Wei Chang, Aditya Grover,
Nov 2023 CVPR 2024
arXiv Website PDF Code Dataset Demo Model
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use
Yonatan Bitton*, Hritik Bansal*, Jack Hessel*, Rulin Shao, Wanrong Zhu, Anas Awadalla, Josh Gardner, Rohan Taori, Ludwig Schmidt,
Aug 2023 NeurIPS 2023, Datasets and Benchmarks
arXiv Website PDF Code Dataset Leaderboard LAION Blog
Read, Look or Listen? What's Needed for Solving a Multimodal Dataset
Read, Look or Listen? What's Needed for Solving a Multimodal Dataset
Netta Madvil, Yonatan Bitton, Roy Schwartz,
July 2023
arXiv PDF Tweet
Transferring Visual Attributes from Natural Language to Verified Image Generation
Transferring Visual Attributes from Natural Language to Verified Image Generation
Rodrigo Valerio, Joao Bordalo, Michal Yarom, Yonatan Bitton, Idan Szpektor, Joao Magalhaes
May 2023
arXiv PDF
What You See is What You Read? Improving Text-Image Alignment Evaluation
What You See is What You Read? Improving Text-Image Alignment Evaluation
Michal Yarom*, Yonatan Bitton*, Soravit "Beer" Changpinyo, Roee Aharoni, Jonathan Herzig, Oran Lang Eran Ofek Idan Szpektor
May 2023 NeurIPS 2023
arXiv Website PDF Tweet
q2d: Turning Questions into Dialogs to Teach Models How to Search
q2d: Turning Questions into Dialogs to Teach Models How to Search
Yonatan Bitton, Shlomi Cohen-Ganor, Ido Hakimi, Yoad Lewenberg, Roee Aharoni, Enav Weinreb
April 2023 EMNLP 2023
arXiv Website PDF Tweet
DataComp: In search of the next generation of multimodal datasets
DataComp: In search of the next generation of multimodal datasets
Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan Yonatan Bitton Kalyani Marathe Stephen Mussmann Richard Vencu Mehdi Cherti Ranjay Krishna Pang Wei Koh Olga Saukh Alexander Ratner Shuran Song Hannaneh Hajishirzi Ali Farhadi Romain Beaumont Sewoong Oh Alex Dimakis Jenia Jitsev Yair Carmon Vaishaal Shankar Ludwig Schmidt
April 2023 NeurIPS 2023, Datasets and Benchmarks
arXiv PDF Website GitHub LAION Blog Tweet
OpenFlamingo: An open-source framework for training vision-language models with in-context learning
OpenFlamingo: An open-source framework for training vision-language models with in-context learning
Anas Awadalla, Irena Gao, Joshua Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, Ludwig Schmidt
Mar 2023 NeurIPS 2023, Datasets and Benchmarks
Website Code Hugging Face
IRFL: Image Recognition of Figurative Language
IRFL: Image Recognition of Figurative Language
Ron Yosef, Yonatan Bitton, Dafna Shahaf
Mar 2023 Findings of EMNLP 2023
arXiv Website PDF Cite Tweet
WHOOPS! A Vision-and-Language Commonsense Benchmark of Heterogeneous Objects and Situations
WHOOPS! A Vision-and-Language Commonsense Benchmark of Heterogeneous Objects and Situations
Nitzan Bitton-Guetta*, Yonatan Bitton*, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, Roy Schwartz
Mar 2023 ICCV 2023
arXiv Website PDF Cite Evaluation Code Hugging Face
VASR: Visual Analogies of Situation Recognition
VASR: Visual Analogies of Situation Recognition
Yonatan Bitton, Ron Yosef, Eli Strugo, Dafna Shahaf, Roy Schwartz, Gabriel Stanovsky
Dec 2022 AAAI 2023
🎉 Selected as Oral Presentation
arXiv Website PDF Cite Code Hugging Face Tweet
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
Yonatan Bitton*, Nitzan Bitton-Guetta*, Ron Yosef, Yuval Elovici, Mohit Bansal, Gabriel Stanovsky, Roy Schwartz
Jul 2022 NeurIPS 2022, Datasets and Benchmarks
🎉 Selected as Featured Presentation
arXiv Website PDF Cite Code Poster Papers With Code Hugging Face Tweet
Data Efficient Masked Language Modeling for Vision and Language
Data Efficient Masked Language Modeling for Vision and Language
Yonatan Bitton, Gabriel Stanovsky, Michael Elhadad, Roy Schwartz
September 2021 Findings of EMNLP 2021
arXiv PDF Cite Code Poster Video Tweet
Data Efficient Masked Language Modeling for Vision and Language
Automatic Generation of Contrast Sets from Scene Graphs
Yonatan Bitton, Gabriel Stanovsky, Roy Schwartz, Michael Elhadad
March 2021 NAACL 2021
arXiv PDF Cite Code Poster Video Tweet
Cross lingual Unified Medical Language System entity linking in online health communities
Cross lingual Unified Medical Language System entity linking in online health communities
Yonatan Bitton, Raphael Cohen, Tamar Schifter, Eitan Bachmat, Michael Elhadad, Noémie Elhadad,
Sep 2020 JAMIA 2020
PDF Cite Code

Work Experience

 
 
 
 
 
Google Research
Senior Research Scientist
Google Research
April 2024 – Present Israel
Advancing multimodal consistency. Developing feedback models for text-to-image and text-to-video applications and enhance multimodal factuality to ensure the accuracy of text generated from visual sources.
 
 
 
 
 
Google Research
Research Scientist
Google Research
Jun 2023 – April 2024 Israel
Focusing on vision-and-language. Recent works include image-text alignment, improving text-to-image models, and visual instruction tuning.
 
 
 
 
 
Google Research
Research Intern
Google Research
Jul 2022 – Jun 2023 Israel
Cerebra team, Conversational AI, working with LLMs (LaMDA, PaLM, BARD, etc)
 
 
 
 
 
Amazon Lab126
Applied Scientist Intern
Amazon Lab126
Oct 2019 – July 2022 Israel
Visual Fitness - Halo team
Developed a virtual fitness trainer, specializing in 2D/3D pose estimation, action recognition, error correction, on-device deployment and more.
 
 
 
 
 
IBM Research
Research Student
IBM Research
Jun 2017 – Oct 2019 Israel
Using data-science and machine-learning methods in order to detect frauds

Invited Talks

Bridging Vision and Language with Data: From Perception to Understanding
April-June 2023 Hebrew University of Jerusalem, NLP-IL Reading Group, Microsoft Israel (MSAI-HIVE team), Meta AI Research Tel-Aviv, Technion, Ben Gurion University, Google Tel-Aviv, Bar-Ilan University, IBM Research (Israel NLP team), Tel Aviv University
My talk 🎬 record is available here.
Commonsense Benchmarks for Vision and Language
November 2022 NLP Seminar at Cornell Tech, Google Research Israel, the Hebrew University of Jerusalem
q2d: Turning Questions into Dialogs to Teach Models How to Search
September 2022 Conversational applications with LLMs - Summit in Google Zurich
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
June 2022 IBM Research Israel
VASR: Visual Analogies of Situation Recognition
May 2022 Computer Vision Seminar at the Hebrew University of Jerusalem

Others

Managing Research
This talk deals with several research related questions. For example findings new research ideas, choose a research topic, staying updated with new research, working with your supervisors, and more.
Yonatan Bitton
Jan 5, 2022
Managing Research
AirPal
A platform that connects drone pilots with people in need of drone services.
This project participated in Starter - Jump course and won 1st place in the final Demo Day event.
Press coverage: telecomnews, israeldefense, sheva7.
Yonatan Bitton
June, 2018
AirPal