I am a first-year ECE PhD student at University of Georgia, supervised by Prof. Tianming Liu.

I am now working on Multi-modal Large Language Models, Medical Image Analysis. If you are seeking any form of academic cooperation, please feel free to email me at hanqi.jiang@uga.edu.

I graduated from School of Computer and Information Technology, Beijing Jiaotong University with a bachelor’s degree of engineering and School of Computing and Communications, Lancaster University with a bachelor’s degree of science (First-Class Honored). Prior to joining UGA, I interned at Li Auto, CUHKSZ, ICTCAS, Tsinghua University and Baidu.

My research focus is medical image analysis and application of multi-modal large language models (MLLMs) in the field of healthcare. Specifically, I am focusing on 3D medical image encoding and video generation.

🔥 News

• 2024.10.30 🎉 A paper is accepted by IEEE Reviews in Biomedical Engineering (IF=17.2)!
• 2024.9.23 🎉 A paper is accepted by NeurIPS 2024, see you in Vancouver!
• 2024.8.18 🎉 A paper is accepted by ECCV 2024 Workshop!
• 2024.5.30 🎉 A paper is accepted by ICIC 2024 (oral presentation)!
• 2024.4.8 🎉 A paper is accepted by Automotive Innovation (IF=6.1)!
• 2023.10.10 I will join the University of Georgia, College of Engineering as a PhD student, guided by Prof. Tianming Liu.
• 2023.7.1 I joined the Chinese University of Hong Kong (Shenzhen), School of Data Science as a research assistant, guided by Prof. Ruimao Zhang.
• 2022.4.24 I joined the State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences (ICTCAS) as an intern, guided by Prof. Yinhe Han.

📝 Publications

NeurIPS 2024

Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning
Chong Ma, Hanqi Jiang, Wenting Chen, Yiwei Li, Zihao Wu, Xiaowei Yu, Zhengliang Liu, Lei Guo, Dajiang Zhu, Tuo Zhang, Dinggang Shen, Tianming Liu, Xiang Li

• We propose EGMA, a novel framework for medical multi-modal alignment, marking the first attempt to integrate eye-gaze data into vision-language pre-training.
• EGMA outperforms existing state-of-the-art medical multi-modal pre-training methods, and realizes notable enhancements in image classification and image-text retrieval tasks.
• EGMA demonstrates that even a small amount of eye-gaze data can effectively assist in multi-modal pre-training and improve the feature representation ability of the model.

ECCV 2024 Workshop

Advancing Medical Radiograph Representation Learning: A Hybrid Pre-training Paradigm with Multilevel Semantic Granularity
Hanqi Jiang, Xixuan Hao, Yuzhou Huang, Chong Ma, Jiaxun Zhang, Yi Pan, Ruimao Zhang

• we present a medical vision-language pre-training (Med-VLP) framework that incorporates multi-modal contrastive alignment and parallel generative streams with multi-level semantic hierarchies. To accomplish this goal, we effectively leverage the characteristics of medical data. By optimizing elaborate training objectives, our HybridMED is capable of efficiently executing a variety of downstream tasks, including cross-modal, uni-modal, and multi-modal types. Extensive experimental results demonstrate that our HybridMED can deliver highly satisfactory performance across a wide array of downstream tasks, thereby validating the model’s superiority.

Under Review

ECHOPluse: ECG Controlled Echocardiograms Video Generation
Yiwei Li, Sekeun Kim, Zihao Wu, Hanqi Jiang, Yi Pan, Pengfei Jin, Sifan Song, Yucheng Shi, Xiaowei Yu, Tianze Yang, Tianming Liu, Quanzheng Li, Xiang Li

• We propose ECHOPluse, an ECG-conditioned ECHO video generation model. ECHOPluse introduces two key advancements: (1) it accelerates ECHO video generation by leveraging VQ-VAE tokenization and masked visual token modeling for fast decoding, and (2) it conditions on readily accessible ECG signals, which are highly coherent with ECHO videos, bypassing complex conditional prompts. To the best of our knowledge, this is the first work to use time-series prompts like ECG signals for ECHO video generation. ECHOPluse not only enables controllable synthetic ECHO data generation but also provides updated cardiac function information for disease monitoring and prediction beyond ECG alone. Evaluations on three public and private datasets demonstrate state-of-the-art performance in ECHO video generation across both qualitative and quantitative measures. Additionally, ECHOPluse can be easily generalized to other modality generation tasks, such as cardiac MRI, fMRI, and 3D CT generation.
• Demo

• ICIC 2024 (Oral) Depth-NeuS: Neural Implicit Surfaces Learning for Multi-view Reconstruction Based on Depth Information Optimization Hanqi Jiang, Cheng Zeng, Runnan Chen, Shuai Liang, Yinhe Han, Yichao Gao, Conglin Wang
• IEEE RBME Artificial General Intelligence for Medical Imaging Analysis Xiang Li, Lin Zhao, Lu Zhang, Zihao Wu, Zhengliang Liu, Hanqi Jiang, Chao Cao, Shaochen Xu, Yiwei Li, Haixing Dai, Yixuan Yuan, Jun Liu, Gang Li, Dajiang Zhu, Pingkun Yan, Quanzheng Li, Wei Liu, Tianming Liu, Dinggang Shen
• MICCAI 2024 Workshop Radiology-GPT: A Large Language Model for Radiology Zhengliang Liu, Yiwei Li, Peng Shu, Aoxiao Zhong, Hanqi Jiang, Yi Pan, Longtao Yang, Chao Ju, Zihao Wu, Chong Ma, Cheng Chen, Sekeun Kim, Haixing Dai, Lin Zhao, Lichao Sun, Dajiang Zhu, Jun Liu, Wei Liu, Dinggang Shen, Quanzheng Li, Tianming Liu, and Xiang Li
• Under Review Large Language Models for Robotics: Opportunities, Challenges, and Perspectives Jiaqi Wang, Zihao Wu, Yiwei Li, Hanqi Jiang, Peng Shu, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Huaqin Zhao, Zhengliang Liu, Haixing Dai, Lin Zhao, Bao Ge, Xiang Li, Tianming Liu†, and Shu Zhang
• Under Review A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks Jiaqi Wang, Hanqi Jiang, Yiheng Liu, Chong Ma, Xu Zhang, Yi Pan, Mengyuan Liu, Peiran Gu, Sichen Xia, Wenjun Li, Yutong Zhang, Zihao Wu, Zhengliang Liu, Tianyang Zhong, Bao Ge, Tuo Zhang, Ning Qiang, Xintao Hu, Xi Jiang, Xin Zhang, Wei Zhang, Dinggang Shen, Tianming Liu, Shu Zhang
• Under Review Argus: Leveraging Multi-ViewImages for Improved 3D SceneUnderstanding with LargeLanguage Models Yifan Xu, Chao Zhang, Hanqi Jiang, Xiaoyan Wang, Ruifei Ma, Yiwei Li, Zihao Wu, Zeju Li, Xiangde Liu
• Under Review GeoDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation Zhifei Yang, Keyang Lu, Chao Zhang, Jiaxing Qi, Hanqi Jiang, Ruifei Ma, Shenglin Yin, Yifan Xu, Mingzhe Xing, Zhen Xiao, Jieyi Long, Xiangde Liu, Guangyao Zhai
• Under Review BrainSF: A Foundation Model for Whole Brain Functional Signals Forecasting Li Yang, Hanqi Jiang, Xintao Hu, Tianming Liu, Tuo Zhang
• Under Review EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysis Yi Pan, Hanqi Jiang, Junhao Chen, Yiwei Li, Huaqin Zhao, Yifan Zhou, Peng Shu, Zihao Wu, Zhengliang Liu, Dajiang Zhu, Xiang Li, Yohannes Abate, Tianming Liu

🎖 Honors and Awards

• Bronze Award of the 8th National College “Internet +” Innovation and Entrepreneurship Competition
• Provincial Second Prize of the 2022 China College Student Mathematics Modeling Competition
• Second Prize at the municipal level of the 5th Beijing College Students “Energy-saving, Water-saving, Low-carbon, Emission Reduction” Social Practice and Science and Technology Competition
• S-Prize in the COMAP’s Mathematical Contest in Modeling (MCM) for College Students in the United States
• Outstanding Scholarship for Social Work of Beijing Jiaotong University
• Outstanding Volunteer of the Academic Support Center of Beijing Jiaotong University
• Outstanding Scholarship for Academic Excellence of Beijing Jiaotong University
• Outstanding Graduation Project of Beijing Jiaotong University
• Outstanding Graduation Project of Beijing

📖 Educations

• 2024.9.1 - Present, PhD in Engineering, College of Engineering, University of Georgia.
• 2020.09 - 2024.06, Undergraduate, School of Computing and Communications, Lancaster University.
• 2020.09 - 2024.06, Undergraduate, School of Computer and Information Technology, Beijing Jiaotong University.

💻 Internships

• 2024.9.1 - Now Mayo Clinic, Arizona.
• 2023.11 - 2024.02, Li Auto, 3D Vision Group, Beijing.
• 2023.07 - 2023.10, School of Data Science, the Chinese University of Hong Kong, Shenzhen.
• 2023.01 - 2023.07, Tsinghua University, Beijing.
• 2022.04 - 2023.04, China Telecom Corporation Limited Beijing Research Institute, Beijing.
• 2022.04 - 2023.09, Institute of Computing Technology, Chinese Academy of Sciences, Beijing.
• 2022.01 - 2022.03, Baidu Inc.,DuerOS Team, Beijing.