Publications

Zikun Li, Zhuofu Chen, Remi Delacourt, Gabriele Oliaro, Zeyu Wang, Qinghan Chen, Shuhuai Lin, April Yang, Zhihao Zhang, Zhuoming Chen, Sean Lai, Xupeng Miao, Zhihao Jia (2026). AdaServe: SLO-Customized LLM Serving with Fine-Grained Speculative Decoding. EuroSys 2026.

PDF Cite

Gabriele Oliaro, Xupeng Miao, Xinhao Cheng, Vineeth Kada, Mengdi Wu, Ruohan Gao, Yingyi Huang, Remi Delacourt, April Yang, Yingcheng Wang, Colin Unger, Zhihao Jia (2026). FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees. NSDI 2026.

PDF Cite Code Project

Gabriele Oliaro, Zhihao Jia, Daniel Campos, Aurick Qiao (2025). SuffixDecoding: A Model-Free Approach to Speeding Up Large Language Model Inference. NeurIPS 2025 (Spotlight 🏆).

PDF Cite Code Project

Rui Pan, Yinwei Dai, Zhihao Zhang, Gabriele Oliaro, Zhihao Jia, Ravi Netravali (2025). SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning. NeurIPS 2025.

PDF Cite

Jaeseong Lee, Seung-won Hwang, Aurick Qiao, Gabriele Oliaro, Ye Wang, Samyam Rajbhandari (2025). OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs. ArXiv 2025.

PDF Cite

Zhengxin Zhang, Dan Zhao, Xupeng Miao, Gabriele Oliaro, Qing Li, Yong Jiang, Zhihao Jia (2024). Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models. ACL 2024 Oral (Outstanding paper award 🏆).

PDF Cite

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, Zhihao Jia (2024). SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification. ASPLOS 2024 (Cited 350+ times 🏆).

PDF Cite Code

Muyan Hu, Ashwin Venkatram, Shreyashri Biswas, Balamurugan Marimuthu, Bohan Hou, Gabriele Oliaro, Haojie Wang, Liyan Zheng, Xupeng Miao, Jidong Zhai, Zhihao Jia (2024). Optimal Kernel Orchestration for Tensor Programs with Korch. ASPLOS 2024.

PDF Code

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, Zhihao Jia (2023). Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems. ACM Comput. Surv..

PDF Cite

Jonatan Langlet, Ran Ben Basat, Gabriele Oliaro, Michael Mitzenmacher, Minlan Yu, Gianni Antichi (2023). Direct Telemetry Access. SIGCOMM 2023.

PDF Cite Code

Jonatan Langlet, Ran Ben Basat, Sivaram Ramanathan, Gabriele Oliaro, Michael Mitzenmacher, Minlan Yu, Gianni Antichi (2021). Zero-CPU Collection with Direct Telemetry Access. HotNets 2021.

PDF Cite