Zhuohao Li

I am a Ph.D. student at UCLA EECS. I help to build SGLang, a fast serving framework for large language models and vision language models with Ying Sheng and Lianmin Zheng. SGLang has been widely adopted by industry and academy. My research interests lie in efficient large language models (LLMs), post-training, and scaling inference. Previously, I obtained my Bachelor's degree from Shanghai Jiao Tong University. I also visited UT Austin computer science and spent an exchange year at HKUST CSE.

During my career, I am fortunate to work and intern at NVIDIA, AWS AI Lab, and Shanghai AI Laboratory. My research is supported by Samueli Fellowship.

Linkedin / CV / Google Scholar / 𝕏 / GitHub

zhuohaol [at] ucla [dot] edu Office: Engineering VI 63-205

I am actively interested in potential collaboration in LLMs and VLMs. If you want to discuss ideas or have a chat, please feel free to shoot me an email and book an appointment with me.

News

[3/2025] 🎉 Our work: Hawkeye on RL-based fast reasoning framework has been open source. Check the technical report: [Link].
[6/2024] 🎉 I join AWS AI Lab as an Applied Scientist Intern, see you in New York!
[3/2024] I gave a talk at UC Irvine
[5/2023] 🎉 My thesis was elected as the Merit Honor at SJTU!
[3/2023] 🎉 I join Shanghai AI Lab as an Research Scientist Intern this summer!
[6/2022] 🎉 I join NVIDIA as a SWE Intern!

Research Interests

I am currently working on efficient large language models. My dream is build interactive, human-friendly, fault-tolerant, and low-cost end-to-end frameworks from models to infrastructure to help enpower AGIs.

In my past years of research experience, I have also explored these research directions:

ML Computing Paradigms: How to enable efficient management of emerging new workloads in heterogeneous datacenters and constructing systems for new computing paradigms on the cloud.

SW-HW Co-design: The way how hardware, software, compiler, and data interact with each other to get optimized in specified tasks. ML tasks get either specifically or generally optimzied on emerging hardware.

LLM Agents: How to apply, retreive, fine-tune general models with other domain specific usages, such as vulnerability extraction, security, and privacy.

Publications

	Hawkeye: Efficient Reasoning with Model Collaboration Jianshu She, Zhuohao Li, Zhemin Huang, Mu Li, Peiran Xu, Qirong Ho arXiv preprint, 2025 code / project page / arXiv / Poster
	Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models with Adaptive Expert Placement Yongji Wu, Wenjie Qu, Tianyang Tao, Zhuang Wang, Wei Bai, Zhuohao Li, Yuan Tian, Jiaheng Zhang, Matthew Lentz, Danyang Zhuo arXiv preprint, 2024 code / project page / arXiv / Poster
	Torpor: GPU-Enabled Serverless Computing for Low-Latency, Resource-Efficient Inference Minchen Yu, Ao Wang, Dong Chen, Haoxuan Yu, Xiaonan Luo, Zhuohao Li, Wei Wang, Ruichuan Chen, Dapeng Nie, Haoran Yang USENIX ATC, 2025 code / project page / arXiv / Poster
	RealNet: Combining Optimized Object Detection with Information Fusion Depth Estimation on IoT Devices Zhuohao Li, Leqi Ding, Yuanhang Zhang, etc. arXiv preprint, 2022 code / project page / arXiv / Poster

Experience

	Amazon Web Services, New York, NY, USA Applied Scientist Intern at AWS AI Lab Jun 2024 ~ Sep 2024
	NVIDIA, Santa Clara, CA, USA CUDA Software Engineering Intern Jun 2022 ~ Dec 2022

Highlighted Honor

Samueli School of Engineering Fellowship, UCLA, 2024

Shanghai Outstanding Graduates, SJTU, 2023

Cockrell School of Engineering Fellowship, UT Austin, 2023

SenseTime Scholarship , SenseTime, 2022

Stanford Irving T Ho Fellowship , Irving T Ho Foundation, 2022

Dean List, SJTU, 2020 - 2022

Google Cup Google, 2020

Misc.

I love snowboarding, tennis, and racing fomula. If you want to have a tennis mate, I am very welcomed (I struggle to find a tennis mate these days). You can find me on 小红书 and instagram.
I was a participant of Mathematics Olympiad in high school.