Remote Direct Memory Access (RDMA) has been widely deployed in data centers to improve application performance. However, the characteristic of RDMA to deliver messages in order cannot meet the emerging requirements of applications for scheduling messages within an RDMA connection, making RDMA unable to be fully utilized. Some works try to schedule the data to be transferred in specific applications before delivering to RDMA, or distribute messages to different connections. However, these approaches tightly couple scheduling logic with application logic and may result in high scheduling overhead. In this paper, we propose sRDMA, a general and lowoverhead scheduler working in user-space RDMA driver. sRDMA allows the application to express the expected transfer order to RDMA hardware via work requests (WRs). With priority information in WRs, sRDMA slices and schedules WRs to achieve desired order of message transfer and reduce blocking impact of large messages in the same RDMA connection. Our experiments show that sRDMA can improve the performance of applications, e.g., TensorFlow, by up to 54%, and sRDMA has negligible overhead in terms of CPU and flow throughput.
APNet '23: Proceedings of the 7th Asia-Pacific Workshop on Networking
来聊聊如何使用RDMA Verbs提供的用户态驱动进行RDMA的通信
本文记录了Python虚拟环境的一些使用心得及技巧
本文记录了我在使用Linux服务(Services)的一些心得
记一下DPDK发包工具DPDK-Pktgen的使用以及本人踩的一些坑
Remote Direct Memory Access (RDMA) equips modern data centers (DCs) with high-performance data trans- mission. Yet, its potential is often unmet, as the State-Of-The-Art (SOTA) RDMA traffic load balancers struggle to balance high- performance with practicability. In this paper, we first analyse existing RDMA-aware load balancers, examining their trade-offs across four key metrics: path utilization, packet order preserva- tion, implementation overhead, and deployment complexity. We then present HP3, a host-based load balancer designed to excel in all these dimensions. HP3 combines Proactive Path Perception for real-time path state awareness with Reorder Free Rerouting to guarantee in-order packet delivery. It maintains low intrusiveness by complying with the standard RDMA transport layer and activating only during severe congestion. Implemented purely on end-hosts with no switch modifications, HP3 offers a practical solution with modest additional overhead. We prototype HP3 using DPDK and commodity RNICs, and evaluate it on a P4- programmable switch testbed. Experiments with diverse data center workloads show that HP3 outperforms SOTA RDMA load balancers by up to 3.4× for typical traffic and 1.3× for AI training traffic.
IEEE/ACM International Symposium on Quality of Service (IWQoS) 2026