Assistant Professor of Computer Science at Auburn University. Communication Chair of ACL Rolling Reviews (ARR)
Auburn, AL
Joined October 2014
- Tweets148
- Following305
- Followers247
- Likes176
Shubhra K. Karmaker ("Santu") retweeted
Thanks for the kind invitation to the inaugural UCF AI Institute's Distinguished Lecture, Amrit+Mubarak et al. -- looking forward to meeting the awesome faculty+students! 🙂
Will discuss 3 important AI challenges: Trustworthy Collaboration, World Discovery, Long-Horizon Memory Planning (across diverse domains & modalities, and real-world applications such as early medical diagnosis and classroom education engagement).
Feel free to join/stop by (virtual option also available)! 👇👇
🚀UCF IAI Distinguished Speaker Series
We’re thrilled to welcome Prof. @mohitban47 @UNC as our inaugural speaker!
🎙️ AI Challenges: Trustworthy Collaboration, World Discovery & Long-Horizon Memory
📅 Sept. 16, 2026
⏰ 11 AM
📍 SPRK 101 @UCF + Zoom
#AI #AgenticAI #TrustworthyAI
Shubhra K. Karmaker ("Santu") retweeted
Theory of Mind 🧠 is a core part of social intelligence.
But most LLM benchmarks only test the final answer, not whether the model actually built the right mental model.
We introduce OmniToM, a benchmark for multi-actor mental state reasoning.
adam-12-0.github.io/omnitom-…
ALT A model can answer “Where will Bob look?” correctly while still tracking the wrong beliefs internally. OmniToM evaluates the mental state representations directly.
I will be participating as an AI Expert Panelist at the following webinar tomorrow. The @UCF and the U.S. Army DEVCOM Army Research Laboratory will jointly host the webinar. Please RSVP in case you are interested in the topic. @UCFCECS @usarmy_devcom #AI #HumanAICollaboration
Honored to receive the 2026 Young Investigator Award from the Office of Naval Research. Time for some celebrations! @ReviewAcl @USNavyResearch @UCFCECS @UCFComputerSci @UCFIAI
onr.navy.mil/2026-young-inve…
Do you like csrankings.org? If yes, how about a similar ranking platform for LLMs? Check out our latest work on the LLM Ranking based on their psychological biases.
bridgeai-lab.github.io/LLM-R…
@ReviewAcl #LLM #NLP @emnlpmeeting @aclmeeting
Job Posting: Postdoc Position Available ucf.wd1.myworkdayjobs.com/op…
@ReviewAcl @emnlpmeeting @naaclmeeting @aclmeeting #NLP #postdoc #UCF
Shubhra K. Karmaker ("Santu") retweeted
We are glad to share our latest NAACL 2025 paper, "LLMs as Meta-Reviewers' Assistants: A Case Study".
Read the full paper at: arxiv.org/pdf/2402.15589v2
(1/2)
Shubhra K. Karmaker ("Santu") retweeted
🚨 We are hiring! 🚨
We are looking for two new members to join our communication team and help support the ARR initiative. If you’re interested, please fill out the following form: forms.gle/e2F6nASMaUD7juDi8
Our latest paper introduces the pre-evaluation of sentence encoders, similar to pertaining language models!! Pre-evaluation performs evaluation without a specific downstream task, that's why we call it taskless evaluation as well. @ReviewAcl
A big thanks to @eduardo_nlp for his collaboration and significant contributions to this work.
Our paper introduces ALIGN-SIM, a novel task-free framework for evaluating and interpreting sentence embeddings based on various semantic similarity alignment criteria.
2/2
Shubhra K. Karmaker ("Santu") retweeted
🚨 Attention authors! The CFP has been updated for the December cycle. Be sure to double-check the latest requirements: aclrollingreview.org/cfp 📝
We have also updated the author checklist to help you avoid common issues: aclrollingreview.org/authorc…
The Bangla LLaMA (BongLLaMA) is here.
We are excited to collaborate with BangaLLM to introduce BongLLaMA, the first ever and, of course, the state-of-the-art Bangla LLM (Based on Meta LLaMA). Try it out.
@ReviewAcl #EMNLP #ACL #NLP #BanglaAI
@UCFComputerSci
arxiv.org/abs/2410.21200
UCF CS has Ph.D. openings for Fall 2025. Consider applying to @UCFComputerSci @ReviewAcl
linkedin.com/feed/update/urn…
Auburn CSSE is a very collegial department. Please consider applying: @AuburnEngineers
auemployment.com/postings/49…
@ReviewAcl #facultyhiring #AIFaculty #TTFaculty
Please try SEM-F1 and let us know if you see a better correlation with human judgments too!
We are happy to release our text summarization metric, Sem-F1, for public use. huggingface.co/spaces/Bridge…
Sem-F1 achieves a 25% higher correlation with human judgments than Bertscore on the Overlap Summarization task. Benchmarking paper coming soon.
@ReviewAcl
Here goes the Ph.D. # 4 from my lab, Congratulations Naman Bansal @ImBNaman ! Naman joined @Google as a software engineer last month.
@AuburnEngineers
I am thrilled as well.
We are thrilled that @karmake2 has joined #UCFCS for Fall 2024! He is one of the 17 new faculty members at @ucfcecs this semester. Join us in welcoming him and learn more about his research and expertise: bit.ly/3TyXdPM
ALT graphic with text overlay "Meet our newest star. We're welcoming Santu Karmaker for Fall 2024" with UCF Department of Computer Science logo and headshot of Santu Karmaker
I am starting a new chapter of my career by joining UCF Artificial Intelligence Initiative (Aii) – AII as an assistant professor.
@UCFComputerSci @UCFCECS
Suparna is fighting TNBC and needs our support to cover medical expenses. Please consider donating and sharing her GoFundMe. Every little bit helps. Let's stand with her during this tough time. 🙏 gofund.me/091c68b4
Shubhra K. Karmaker ("Santu") retweeted
Today 11 AM, I will be presenting our paper on state reconciliation for @ansible (infrastructure-as-code) at Acerola in the @FSEconf . In collaboration with @karmake2 and his students. This work benefitted from discussions at a @dagstuhl seminar that I attended in 2023.
