You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »


Goal

Find minimum hardward requirement of 100G perfSonar


Method

Change num of CPU, freq of CPU → test


Ref

Spec of 100G PS using VM

3.5GHz 2 core, 16GB RAM

http://ps-100g-hpr01.stanford.edu/


Change CPU (num of cpu, on/off multithreading, freq)

# lscpu
CPU(s):                24
On-line CPU(s) list:   0-23
Model name:            Intel(R) Xeon(R) CPU E5-2643 v4 @ 3.40GHz

# cat /proc/cpuinfo | grep processor | wc -l
24

# echo 0 > /sys/devices/system/cpu/cpu{20,21,22,23}/online

# lscpu
CPU(s):                24
On-line CPU(s) list:   0-19
Off-line CPU(s) list:  20-23

# cat /proc/cpuinfo | grep processor | wc -l
20

# echo 1 > /sys/devices/system/cpu/cpu{20,21,22,23}/online


# lscpu
CPU(s):                24
On-line CPU(s) list:   0-23

# cat /sys/devices/system/cpu/smt/active
1

# echo off > /sys/devices/system/cpu/smt/control

# lscpu
CPU(s):                24
On-line CPU(s) list:   0-11
Off-line CPU(s) list:  12-23

# cat /proc/cpuinfo | grep processor | wc -l
12


# echo off > /sys/devices/system/cpu/smt/control

# lscpu
CPU(s):                16
On-line CPU(s) list:   0-15
Off-line CPU(s) list:  8-15
Model name:            Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
CPU MHz:               1243.188
CPU min MHz:           1200.0000
CPU max MHz:           3300.0000

# cpupower frequency-info
analyzing CPU 0:
  driver: intel_pstate
  hardware limits: 1.20 GHz - 3.30 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 1.20 GHz and 3.30 GHz.
                  The governor "powersave" may decide which speed to use  within this range.
  current CPU frequency: 1.20 GHz (asserted by call to hardware)
  boost state support:
    Supported: yes
    Active: yes
    3200 MHz max turbo 4 active cores
    3200 MHz max turbo 3 active cores
    3300 MHz max turbo 2 active cores
    3300 MHz max turbo 1 active cores

# cpupower frequency-set --governor performance
Setting cpu: 0
Setting cpu: 1
...
Setting cpu: 7


# vi /etc/default/grub
# Append "intel_pstate=disable" to GRUB_CMDLINE_LINUX= option
# update grub via some commands
# reboot

# modprobe cpufreq_userspace
# cpupower frequency-set --governor userspace
# cpupower --cpu all frequency-set --freq 1.5GHz


Disable hyperthreading from within Linux (no access to BIOS), https://serverfault.com/questions/235825

Programmatically disable CPU core, https://stackoverflow.com/questions/44907731

How to on/offline CPU on RHEL(CentOS), https://blog.seabow.pe.kr/?p=7303

100g Network Adapter Tuning - https://srcc.stanford.edu/100g-network-adapter-tuning

Red Hat > Monitoring and managing system status and performanceChapter > 17. Tuning CPU frequency to optimize energy consumption - https://access.redhat.com/documentation/enus/red_hat_enterprise_linux/8/html/monitoring_and_managing_system_status_and_performance/tuning-cpu-frequency-to-optimize-energy-consumption_monitoring-and-managing-system-status-and-performance

CPU Frequency Scaling On Linux With cpupower - https://krimson8.github.io/linux/2019/03/13/Configuring-CPU-frequency.html

How to solve the cpufreqset errors - https://stackoverflow.com/questions/23526671


Ref Spec

perfSONAR Deployment Examples > 100G Configuration - https://docs.perfsonar.net/deployment_examples.html

  • 2 x Intel Xeon Haswell E5-2643V3 3.4GHz 6 Cores (Total 12 Threads each)
  • 8 x 16GB DDR4-2133MHz RAM ECC/REG (128GB total)
  • Mellanox MCX455A-ECAT 100Gbps NIC

perfSONAR Node Reference Implementation - https://fasterdata.es.net/performance-testing/perfsonar/perfsonar-node-reference-implementation

  • Current (~2021/22) 40/50/100Gbps Capable Design 1
    • The total cost of this server was around $4K in mid 2021.
    • CPU: AMD EPYC 7302P 16-Core Processor (3.0GHz base clock, 3.3GHz max boost)
    • Memory: 8 x 8GB DDR4 ECC/REG RAM (64G Total)
    • Network: Mellanox MCX556A-ECAT Dual Port Connect X5
  • Current (~2021/22) 40/50/100Gbps Capable Design 2
    • We also tested and liked the following system, but due to supply chain issues, went with the above system.
    • CPU: Intel Xeon Silver 4215R 8C 3.2Ghz
    • Memory: 6 x 8GB DDR4 ECC/REG RAM (48G Total)
    • Network: Mellanox MCX556A-ECAT Dual Port Connect X5


  • No labels