Yifei Ming

Contact: alvinming5 [at] gmail [dot] com

Hi! I am a research scientist at Salesforce AI Research. I am broadly interested in trustworthy AI that aligns with human values, especially in the era of foundation models. Topics that I currently focus on:

  • Understand and improve reasoning of contextual LLMs and Agents.
  • Obtain domain expert language models.
  • Enable LLMs to handle long contexts effectively.

I obtained my CS Ph.D. in 2024 from the University of Wisconsin-Madison advised by Sharon Li. My thesis covers a spectrum of algorithmic developments for reliable foundation models in the open world. In the past, I have worked on Spatial Reasoning for LLMs and VLMs with Xin Wang, Vibhav Vineet, and Neel Joshi from Microsoft Research. I also worked on Multi-Modal Document OOD with Jiuxiang Gu from Adobe Research.

News

09/2024 Is a picture worth a thousand words? 🤔 Check out our new work on spatial reasoning of LLMs and multi-modal LLMs at NeurIPS 2024 ✨
05/2024 Starting a new position as a Research Scientist at Salesforce! Super excited to explore the frontiers of LLM, VLM, reliable ML, among other fascinating topics.
05/2024 Defended my Ph.D. thesis on Reliable Foundation Models in the Open World :mortar_board:
01/2024 Our paper Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models is accepted by ICML 2024. :tada:
01/2024 Our paper Provable Out-of-Distribution Generalization in Hypersphere is accepted by ICLR 2024. :tada:
More News

Publications

  1. NeurIPS
    Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
    Jiayu Wang, Yifei Ming, Zhenmei Shi, Vibhav Vineet, Xin Wang, Yixuan Li, and Neel Joshi
    In Neural Information Processing Systems (NeurIPS) 2024
    Paper
  2. ICML
    Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models
    Yifei Ming, and Yixuan Li
    In International Conference on Machine Learning (ICML) 2024
    Paper
  3. ICLR
    Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models
    Atsuyuki Miyai, Jingkang Yang, Jingyang Zhang, Yifei Ming, Qing Yu, Go Irie, Yixuan Li, Hai Li, Ziwei Liu, and Kiyoharu Aizawa
    In International Conference on Learning Representations (ICLR), Workshop on Reliable and Responsible Foundation Models 2024
    Code Paper
  4. ICLR
    Provable Out-of-Distribution Generalization in Hypersphere
    Haoyue Bai*, Yifei Ming*, Julian Katz-Samuels, and Yixuan Li
    In International Conference on Learning Representations (ICLR) 2024
    Paper
  5. CPAL Oral
    Domain Generalization via Nuclear Norm Regularization
    Zhenmei Shi*, Yifei Ming*, Ying Fan*, Frederic Sala, and Yingyu Liang
    In Conference on Parsimony and Learning (CPAL) 2023
    Paper
  6. EMNLP
    A Critical Analysis of Document Out-of-Distribution Detection
    Jiuxiang Gu*, Yifei Ming*, Yi Zhou, Jason Kuen, Vlad I Morariu, Handong Zhao, Nikolaos Barmpalios Ruiyi Zhang, Anqi Liu, Yixuan Li, Tong Sun, and Ani Nenkova
    In Empirical Methods in Natural Language Processing (EMNLP Findings) 2023
    Paper
  7. IJCV
    How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?
    Yifei Ming, and Yixuan Li
    In International Journal of Computer Vision (IJCV) 2023
    Paper
  8. ICLR
    How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?
    Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li
    In International Conference on Learning Representations (ICLR) 2023
    Code Paper
  9. NeurIPS
    Delving into Out-of-Distribution Detection with Vision-Language Representations
    Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li
    In Neural Information Processing Systems (NeurIPS) 2022
    Code Paper
  10. NeurIPS
    SIREN: Shaping Representations for Detecting Out-of-distribution Objects
    Xuefeng Du, Gabriel Gozum, Yifei Ming, and Yixuan Li
    In Neural Information Processing Systems (NeurIPS) 2022
    Code Paper
  11. NeurIPS
    Domain Generalization with Nuclear Norm Regularization
    Zhenmei Shi*, Yifei Ming*, Ying Fan*, Frederic Sala, and Yingyu Liang
    In Neural Information Processing Systems (NeurIPS’W) DistShift Workshop 2022
    Paper
  12. EMNLP
    Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment
    Tuan Dinh, Jy-yong Sohn, Shashank Rajput, Timothy Ossowski, Yifei Ming, Junjie Hu, Dimitris Papailiopoulos, and Kangwook Lee
    In Empirical Methods in Natural Language Processing (EMNLP Findings) 2022
    Code Paper
  13. ICML Oral
    POEM: Out-of-Distribution Detection with Posterior Sampling
    Yifei Ming*, Ying Fan*, and Yixuan Li
    In International Conference on Machine Learning (ICML) 2022
    Oral Presentation [Top 2%]
    Code Paper
  14. ICML
    Out-of-Distribution Detection with Deep Nearest Neighbors
    Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li
    In International Conference on Machine Learning (ICML) 2022
    Code Paper
  15. ICML
    Are Vision Transformers Robust to Spurious Correlations?
    Soumya Suvra Ghosal, Yifei Ming, and Yixuan Li
    In International Conference on Machine Learning (ICML’W), SCIS Workshop 2022
    Paper
  16. AAAI Oral
    On the Impact of Spurious Correlation for Out-of-distribution Detection
    Yifei Ming, Hang Yin, and Yixuan Li
    In The AAAI Conference on Artificial Intelligence (AAAI) 2022
    Code Paper
  17. ICML Oral
    Model-based Reinforcement Learning for Continuous Control with Posterior Sampling
    Ying Fan, and Yifei Ming
    In International Conference on Machine Learning (ICML) 2021
    Oral Presentation [Top 3%]
    Code Paper