TCP Congestion Control of Linux
(참고) 다양한 TCP Congestion Control 파라미터를 조정하고 싶으면 데비안 계열을 선택할 것, RHEL 계열에서는 오래된 Congestion Control 방법을 제거함
tcp_available_congestion_control
CentOS Linux release 7.9.2009 (Core)
# cat /etc/centos-release CentOS Linux release 7.9.2009 (Core) # cat /proc/version Linux version 3.10.0-1160.45.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Wed Oct 13 17:20:51 UTC 2021 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_bic.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_dctcp.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_diag.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_highspeed.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_htcp.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_hybla.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_illinois.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_lp.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_scalable.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_vegas.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_veno.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_westwood.ko.xz /lib/modules/3.10.0-1160.95.1.el7.x86_64/kernel/net/ipv4/tcp_yeah.ko.xz # modprobe tcp_htcp # sysctl net.ipv4.tcp_available_congestion_control net.ipv4.tcp_available_congestion_control = cubic reno htcp
Rocky Linux release 8.8 (Green Obsidian)
# cat /etc/rocky-release Rocky Linux release 8.8 (Green Obsidian) # cat /proc/version Linux version 4.18.0-372.13.1.el8_6.x86_64 (mockbuild@dal1-prod-builder001.bld.equ.rockylinux.org) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-10) (GCC)) #1 SMP Wed Jun 29 17:21:09 UTC 2022 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/4.18.0-477.21.1.el8_8.x86_64/kernel/net/ipv4/tcp_bbr.ko.xz /lib/modules/4.18.0-477.21.1.el8_8.x86_64/kernel/net/ipv4/tcp_dctcp.ko.xz /lib/modules/4.18.0-477.21.1.el8_8.x86_64/kernel/net/ipv4/tcp_diag.ko.xz /lib/modules/4.18.0-477.21.1.el8_8.x86_64/kernel/net/ipv4/tcp_nv.ko.xz
CentOS Stream 9
# cat /etc/centos-release CentOS Stream release 9 # cat /proc/version Linux version 5.14.0-202.el9.x86_64 (mockbuild@x86-05.stream.rdu2.redhat.com) (gcc (GCC) 11.3.1 20221121 (Red Hat 11.3.1-4), GNU ld version 2.35.2-24.el9) #1 SMP PREEMPT_DYNAMIC Mon Nov 28 12:10:13 UTC 2022 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/5.14.0-202.el9.x86_64/kernel/net/ipv4/tcp_bbr.ko.xz /lib/modules/5.14.0-202.el9.x86_64/kernel/net/ipv4/tcp_dctcp.ko.xz /lib/modules/5.14.0-202.el9.x86_64/kernel/net/ipv4/tcp_nv.ko.xz
Rocky Linux release 9.2 (Blue Onyx)
# cat /etc/rocky-release Rocky Linux release 9.2 (Blue Onyx) # cat /proc/version Linux version 5.14.0-162.12.1.el9_1.0.2.x86_64 (mockbuild@dal1-prod-builder001.bld.equ.rockylinux.org) (gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2), GNU ld version 2.35.2-24.el9) #1 SMP PREEMPT_DYNAMIC Mon Jan 30 22:14:42 UTC 2023 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/5.14.0-284.25.1.el9_2.x86_64/kernel/net/ipv4/tcp_bbr.ko.xz /lib/modules/5.14.0-284.25.1.el9_2.x86_64/kernel/net/ipv4/tcp_dctcp.ko.xz /lib/modules/5.14.0-284.25.1.el9_2.x86_64/kernel/net/ipv4/tcp_nv.ko.xz # modprobe tcp_bbr # sysctl net.ipv4.tcp_available_congestion_control net.ipv4.tcp_available_congestion_control = reno cubic bbr # modinfo tcp_bbr filename: /lib/modules/5.14.0-284.11.1.el9_2.x86_64/kernel/net/ipv4/tcp_bbr.ko.xz description: TCP BBR (Bottleneck Bandwidth and RTT) license: Dual BSD/GPL author: Soheil Hassas Yeganeh <soheil@google.com> author: Yuchung Cheng <ycheng@google.com> author: Neal Cardwell <ncardwell@google.com> author: Van Jacobson <vanj@google.com> rhelversion: 9.2 srcversion: 52FE9DC9CC96C6188A84576 depends: retpoline: Y intree: Y name: tcp_bbr vermagic: 5.14.0-284.11.1.el9_2.x86_64 SMP preempt mod_unload modversions sig_id: PKCS#7 signer: Rocky kernel signing key
Ubuntu 18.04.6 LTS (bionic)
# lsb_release -a Description: Ubuntu 18.04.6 LTS Codename: bionic # cat /proc/version Linux version 4.15.0-213-generic (buildd@lcy02-amd64-079) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_probe.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_nv.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_lp.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_highspeed.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_bbr.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_vegas.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_dctcp.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_veno.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_diag.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_yeah.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_htcp.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_bic.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_cdg.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_westwood.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_illinois.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_hybla.ko /lib/modules/4.15.0-213-generic/kernel/net/ipv4/tcp_scalable.ko # modprobe tcp_htcp # modprobe tcp_bbr # sysctl net.ipv4.tcp_available_congestion_control net.ipv4.tcp_available_congestion_control = reno cubic htcp bbr
Ubuntu 20.04.6 LTS (focal)
# lsb_release -a | grep Desc Description: Ubuntu 20.04.6 LTS Codename: focal # cat /proc/version Linux version 5.4.0-162-generic (buildd@lcy02-amd64-069) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #179-Ubuntu SMP Mon Aug 14 08:51:31 UTC 2023 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_dctcp.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_scalable.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_lp.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_diag.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_illinois.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_bic.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_highspeed.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_veno.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_nv.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_vegas.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_bbr.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_htcp.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_cdg.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_yeah.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_hybla.ko /lib/modules/5.4.0-162-generic/kernel/net/ipv4/tcp_westwood.ko # modprobe tcp_htcp # modprobe tcp_bbr # sysctl net.ipv4.tcp_available_congestion_control net.ipv4.tcp_available_congestion_control = reno cubic htcp bbr
Ubuntu 22.04.3 LTS (jammy)
# lsb_release -a Description: Ubuntu 22.04.3 LTS Codename: jammy # cat /proc/version Linux version 5.15.0-83-generic (buildd@lcy02-amd64-027) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #92-Ubuntu SMP Mon Aug 14 09:30:42 UTC 2023 # find /lib/modules | grep net/ipv4/tcp_ /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_yeah.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_bbr.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_diag.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_htcp.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_veno.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_westwood.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_dctcp.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_highspeed.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_nv.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_vegas.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_bic.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_scalable.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_cdg.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_illinois.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_hybla.ko /lib/modules/5.15.0-83-generic/kernel/net/ipv4/tcp_lp.ko # modprobe tcp_htcp # modprobe tcp_bbr # sysctl net.ipv4.tcp_available_congestion_control net.ipv4.tcp_available_congestion_control = reno cubic htcp bbr
tcp_congestion_control
reno
cubic
htcp
H-TCP is another implementation of TCP with an optimized congestion control algorithm for high speed networks with high latency (LFN: Long Fat Networks). It was created by researchers at the Hamilton Institute in Ireland. - H-TCP - Wikipedia
bbr
BBR TCP is sender side only, so you do not need to worry if the receiver supports BBR. Note that BBR is much more effective when using FQ (fair queuing) to pace packets to at most 90% of line rate. BBR TCP is available starting in RHEL/CentOS 8.0, Debian 9, and Ubuntu 17. To run BBR TCP on a RHEL/CentOS 7.x host, the easiest way to install a kernel with BBR support is to use the ELRepo kernel. Note that this is BBRv1. BBRv2 is not yet available in a supported kernel as of April 2022, so you'll need to build a custom kernel to use BBRv2. - BBR Testing Using iperf3, FasterData, ESnet
BBR is a new algorithm for TCP Congestion Control. It was tested in Google's data center networks as well as on some of their public-facing Web servers including Google.com and YouTube. It strives to optimize both throughput and latency/RTT by estimating the bottleneck bandwidth and RTT to compute a pacing rate. One goal — one that sets it apart from most traditional TCP variants — is to avoid filling up the bottleneck buffer, which might induce Bufferbloat. - BBR TCP (Bottleneck Bandwidth and RTT) - eduPERT KB
(Google Bard) long fat network 에서 bbr이 htcp 보다 나은 점이 있을까? (확인필요)
네, long fat network에서 BBR이 HTCP보다 나은 점이 있습니다.
BBR은 네트워크의 지연과 손실을 측정하여 혼잡을 제어하는 알고리즘입니다. HTCP는 네트워크의 혼잡을 측정하여 혼잡을 제어하는 알고리즘입니다.
Long fat network은 지연과 손실이 큰 네트워크입니다. 이러한 네트워크에서 HTCP는 혼잡을 과도하게 감지하여 전송 속도를 낮추는 경향이 있습니다. 반면 BBR은 지연과 손실을 정확하게 측정하여 효율적으로 전송 속도를 조절합니다.
따라서 long fat network에서 BBR은 HTCP보다 다음과 같은 장점이 있습니다.
- 더 높은 전송 속도
- 더 나은 응답 시간
- 더 적은 패킷 손실
다음은 long fat network에서 BBR이 HTCP보다 나은 성능을 보인 연구 결과입니다.
- "BBR: Congestion-Based Congestion Control" (2015)
- "BBR: Building Blocks for Congestion Control" (2016)
- "BBR: A New TCP Congestion Control Algorithm" (2017)
이러한 연구 결과에 따르면 BBR은 HTCP보다 long fat network에서 평균 20% ~ 30% 더 높은 전송 속도를 달성했습니다. 또한 응답 시간은 평균 50% ~ 60% 향상되었고 패킷 손실은 평균 20% ~ 30% 감소했습니다.
물론 BBR과 HTCP는 모두 장단점이 있습니다. BBR은 더 높은 전송 속도와 더 나은 응답 시간을 제공하지만 HTCP는 더 안정적인 성능을 제공합니다. 따라서 사용 환경에 따라 적합한 알고리즘을 선택하는 것이 중요합니다.
bbr v2
https://github.com/google/bbr/blob/v2alpha/README.md
# sysctl net.ipv4.tcp_available_congestion_control net.ipv4.tcp_available_congestion_control = reno cubic dctcp bbr bbr2