Rayat Imtiaz Hossain retweeted
📊 Why do Large Vision-Language Models struggle with charts?
Even the most advanced LVLMs often look at the wrong parts of a chart—fixating on irrelevant axes or labels—leading to wrong answers and making their reasoning hard to interpret. In domains like finance, medicine, or scientific research, this lack of interpretability is a huge barrier to trust.
✨ That’s where our new work ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement comes in.
We introduce ChartGaze, the first eye-tracking dataset for chart reasoning, capturing how humans actually look at charts when answering questions. With this, we:
🔹 Show that LVLM attention diverges significantly from human gaze.
🔹 Propose gaze-guided attention refinement to teach models where to look.
🔹 Improve both accuracy (+2.56%) and interpretability, making attention maps more human-like and reasoning more trustworthy.
What’s inside ChartGaze?
✅ 1,620 real-world charts from Statista, Pew, and beyond
✅ 4,638 high-precision human gaze maps
✅ 4,638 chart QA pairs across trend analysis, extremum finding, and comparison reasoning
🚀 The result: LVLMs that don’t just answer better—but reason in a way that’s more aligned with human cognition.
👉 Paper: arxiv.org/pdf/2509.13282
💻 Code: github.com/alisalamatian1/Ch…
📊 Dataset: huggingface.co/datasets/alis…
A huge thanks to my amazing co-authors and mentors: Ali Salamatian, @CyuanWan , @RayatHoss , and my supervisors Leonid Sigal and @careninigiusepp — for making this work possible! 🙏
#AI #MachineLearning #DeepLearning #NLP #ComputerVision #MultimodalAI #Interpretability #Charts #Visualization #Research #ChartQA #LVLMs
Rayat Imtiaz Hossain retweeted
I’m angry to learn that women wearing hijabs were the target of a violent attack in London, and a mosque in Winnipeg was the target of hateful harassment.
Islamophobia kills. Whenever we see it, we’ll confront it — and we’ll work to keep our communities safe from it.
Please come by our poster on 21st June, Friday at 10:30 AM. Our poster number is 383. Additionally we will also present our work in L3D-IVU workshop on 18th June, Tuesday at 1:15 PM.
Our @CVPR 2024 work is out where we discuss mutliscale visual prompting, novel-to-base causal attention and transductive prompt tuning to utilize the unlabelled data when prompting image segmentation models. #CVPR2024. arxiv.org/pdf/2404.11732.pdf, Kudos to
@RayatHoss
Rayat Imtiaz Hossain retweeted
Eid Adha Mubarak everyone which is coinciding with @CVPR this year in Seattle. I will there in 3 days there. Looking forward to the talks, workshops, posters and meeting good people ^_^. Come visit us in our poster 383, June 21st 10:30-12:00 @RayatHoss. arxiv.org/abs/2404.11732
Rayat Imtiaz Hossain retweeted
Our @CVPR 2024 work is out where we discuss mutliscale visual prompting, novel-to-base causal attention and transductive prompt tuning to utilize the unlabelled data when prompting image segmentation models. #CVPR2024. arxiv.org/pdf/2404.11732.pdf, Kudos to
@RayatHoss
Happy to share that our paper titled 'Visual Prompting for Generalized Few-shot Segmentation: A Multi-scale Approach,' in collaboration with @mennatullahSiam and my supervisors Leonid Sigal and James J. Little, has been accepted at #CVPR2024.
In this paper we proposed:
- Fine-tuning the visual prompts in a transductive manner utilizing the unlabelled query images in addition to few-shot support images to improve performance on novel categories in addition to retaining base class performance.
1
1
92
Rayat Imtiaz Hossain retweeted
Phenaki. Text to video using VQ encoders and masked transformers. Simple yet powerful. Love the 2 min video generation. phenaki.video/






