1

FastServe: Iteration-Level Preemptive Scheduling for Large Language Model Inference
Large language models (LLMs) power a new generation of interactive AI applications exemplified by ChatGPT. The interactive nature of …
PDF Cite DOI
Optimizing RLHF Training for Large Language Models with Stage Fusion
We present RLHFuse, an efficient training system with stage fusion for Reinforcement Learning from Human Feedback (RLHF). Due to the …
PDF Cite
LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
The context window of large language models (LLMs) is rapidly increasing, leading to a huge variance in resource usage between …
PDF Cite DOI
dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving
Low-rank adaptation (LoRA) is a popular approach to finetune pre-trained large language models (LLMs) to specific domains. This paper …
PDF Cite
XRON: A Hybrid Elastic Cloud Overlay Network for Video Conferencing at Planetary Scale
Quality and cost are two key considerations for video conferencing services. Service providers face a dilemma when selecting network …
PDF Cite DOI
Transparent GPU Sharing in Container Clouds for Deep Learning Workloads
Containers are widely used for resource management in datacenters. A common practice to support deep learning (DL) training in …
PDF Cite
AMOS: Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware Abstraction
Hardware specialization is a promising trend to sustain performance growth. Spatial hardware accelerators that employ specialized and …
PDF Cite DOI