Ph.D. student @UMich @Michigan_AI studying language grounding, embodied AI and situated dialogue.
Joined November 2017
- Tweets23
- Following159
- Followers194
- Likes59
Yichi Zhang retweeted
Seed 2.0 Pro released. Reaching the Frontier.
Model Card: lf3-static.bytednsdoc.com/ob…
Attending #CVPR2024. Open to chat about multimodal LLMs, embodied AI, Agent and so on, pls DM me!
We will present GROUNDHOG, a multimodal large language model with pixel-level visual grounding on Thu 20 Jun 10:30 - 12 am at Arch 4A-E Poster #442.
arxiv.org/abs/2402.16846
Yichi Zhang retweeted
Introducing GPT-4o, our new model which can reason across text, audio, and video in real time.
It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):
Yichi Zhang retweeted
New paper from @Berkeley_AI on Autonomous Evaluation and Refinement of Digital Agents!
We show that VLM/LLM-based evaluators can significantly improve the performance of agents for web browsing and device control, advancing sotas by 29% to 75%.
arxiv.org/abs/2404.06474 [🧵]
1/ Excited to share our latest research "Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?" at #EMNLP2023 🎉 Discover how VLMs fare against tricky visual illusions👀➡️ vl-illusion.github.io/
5/ A test with GPT-4 Vision hints at its skill in simple illusion tasks, but complex ones remain a challenge. Does this point to a generalization gap? 🤖🧩 More on our site!
6/ This work aims to bridge the gap in understanding visual illusions towards more aligned communications between humans and AI. For an in-depth analysis, check out our paper 🚀📄 arxiv.org/pdf/2311.00047.pdf
Yichi Zhang retweeted
1/ 🎉 After a 2 years of hard work, #TeamSEAGULL, representing @UMichCSE @SLED_AI, has won First Place🥇 in the #AlexaPrize SimBot Challenge organized by @AmazonScience! This victory is a testament to our collective dedication to advancing #EmbodiedAI. 🏆💻🚀
Congrats to the winning team of the #AlexaPrize SimBot Challenge! 🎉 The @UMichCSE team earned $500,000 for their performance in the challenge, which focused on advancing next-generation virtual assistants that help humans complete real-world tasks. #ConvAI
Yichi Zhang retweeted
I agree with Oren. If the violation gives the paper “an unfair” advantage (squeezes more text in) or is unreadable, then I agree with a desk rejection. If it only violates a formatting rule but doesn’t impact anything else, I wouldn’t desk reject. We are trying to do science.
💭Ever wondered how deliberative symbolic planning can help in Embodied AI🤖? Check out our new #EMNLP2022 paper *DANLI* from the #SLEDLab (@michigan_AI). Meet us on Friday, Dec. 9 in Oral Session 3!
Pre-print: arxiv.org/abs/2210.12485
Video: bit.ly/3ur9iJT
A thread🧵
A key benefit of this approach is that it enables powerful exception-handling mechanisms that can be continuously improved. Consequently, our deliberative approach sharply outperforms past reactive approaches on TEACh EDH in terms of success rate (over 70% gain) and efficiency.
Come chat with our presenters @shanestorks and @ziqiao_ma about DANLI on Friday, Dec. 9 in Oral Session 3, Speech, Vision, Robotics, Multimodal Grounding 1!
Thanks to all collaborators @jed_yang, @pan_jiayipan, @nikdevraj, Keunwoo Peter Yu, @baobaoyaobaobao and Joyce Chai!