버전 비교

이전 버전 30

changes.mady.by.user 장민석(KN)

에 저장

비교대상

새 버전 현재

changes.mady.by.user 장민석(KN)

에 저장

키

  • 이 줄이 추가되었습니다.
  • 이 줄이 삭제되었습니다.
  • 서식이 변경되었습니다.

...

  • 400GbE 패킷 생성 (dpdk, RoCE, iperf3)
  • 400GbE 링크에서 NVMe 기반 Disk to Disk 전송 실험 (NVMe의 MTBF, TBW 낮아도 됨)
  • PCIe 버스를 최대한 Gen5 NVMe로 채우는 구성
  • 실험 종료 후 추후 HDD 추가하여 



LGA4677Motherboard H12DSUiN

AS -1115CS-TNR

  • SMCi H13SSW

  • 2 PCIe 5.0 x16 FHHL slots

  • 2 PCIe 5.0 x16 AIOM slots (OCP 3.0 compatible)

  • 10 front hot-swap 2.5" PCIe 5.0 NVMe* drive bays

SMCi AS-2015CS-TNR

  • SMCi H13SSW
  • 2 PCIe 5.0 x16 AIOM slots (OCP 3.0 compatible)
  • 2 PCIe 5.0 x16 FHFL slots
    • 2 PCIe 5.0 x16 FHHL slotsDiskDiskCompatibility

    		레퍼런스Type AType B비고

    		2023 ESnet6 200G DTN2025 KREONET 400G DTN Intel2025 KREONET 400G DTN AMD
    CPU

    AMD EPYC 73F3 (2ea) (Total 32C)

    • 16C 24T (per 1ea)
    • Base / Turbo
      : 3.5 / 4.0 GHz
    • L3 256 MB
    • PCIe 4.0 x128
    • EPYC 7003 Series (SP3)
    • TDP 240W
    • Q1'21
    • $3521

    Intel Xeon w7-3565X

    • 32C 64T
    • Base / Turbo
      : 2.5 / 4.8 GHz
    • 82.5 MB
    • PCIe 5.0 x112
    • Xeon W (
    • LGA-4677)
    • TDP 335~402W
    • Q3'24
    • $2699 (390만원)

    AMD EPYC™ 9355P (1ea)

    • 32C 64T
    • Base / All Turbo / Max Turbo
      : 3.55 / 4.2 / 4.4 GHz
    • L3 256 MB
    • PCIe 5.0 x128
    • EPYC 9005 Series (SP5)
    • TDP 280W
    • Q4'24
    • $2998 (435만원)

    싱글소켓 구성 (듀얼소켓 구성 가격 비쌈)

    Chassis

    SMCi

    2124US-

    SMCi X13SWA-TF

    SMCi H13SSW

    • EPYC™ 9005 series (SP5)
    • 1 PCIe 5.0 x16 Right Riser Slot
    • 1 PCIe 5.0 x16 Left Riser Slot
    • 2 PCIe 5.0 x16 AIOM Networking Slot
    • 6 MCIO (PCIe 5.0 x8) Port(s)
    • 2 MCIO (PCIe 5.0 x8/SATA3) Port(s)
    • M.2 Interface: 2 PCIe 3.0 x2
    • IPMI
    • (매뉴얼) SMCi H13SSW

    IPMI 기능 없는 경우

    		Chassis

    SMCi 2124US-TNRP

    • 24 front hot-swap 2.5" NVMe U.2

    TNRP

    • 24 front hot-swap 2.5" NVMe U.2


    SMCi > Chassis > 검색조건

    • Global SKU
    • Drive Form Factor 3.5 inch

    SuperChassis 846BE1C-R1K23B

    • Image Added
    • ATX, E-ATX-, EE-ATX
    • PCIe FHFL 장착 가능

    SuperChassis 743AC-1K26B-SQ

    • Image Added
    • ATX, E-ATX
    • PCIe FHFL 장착 가능
    		추후 디스크 추가를 위해 베이가 많은 샤시 선호
    Motherboard

    SMCi H12DSU-iN

    SMCi > Motherboards > 검색조건

    • Form Factor: ATX, E-ATX
    • Socket: LGA-4677

    SMCi X13SWA-TF

    • Image Added
    • Form Factor: E-ATX
    • LGA-4677
    • Intel W790
    • 6 PCIe 5.0 x16
      • 1 x 400GbE
      • 3 x Hyper M2
      • 1 x Mega RAID
    • Gen5 M.2 onboard
    • VROC key 설치 가능
    • IPMI

    SMCi > Motherboards > 검색조건

    • Form Factor: ATX, E-ATX
    • Socket: SP5

    SMCi H13SSL-NT

    • Image Added
    • Form Factor: ATX
    • EPYC™ 9005 series (SP5)
    • 3 PCIe 5.0 x16
      • 1  x 400GbE
      • 2 x Hyper M2
    • 2 PCIe 5.0 x8
      • 1 x Mega RAID
    • 3 MCIO PCIe 5.0 x8
    • Gen4 M.2 onboard
    • IPMI


    RAM

    DDR4-3200 16GB x 16ea (256GB)

    256GB, 속도 제일 빠른 구성

    DDR5-4800 ECC 64GB x 4ea

    • 46만원/1ea (184만원/4ea)

    256GB, 속도 제일 빠른 구성

    DDR5-4800 ECC 64GB x 4ea

    • 46만원/1ea (184만원/4ea)

    OS_
    M2

    Micron 7300 MAX 800G U.2 (2ea)

    • PCIe 3.0 x4

    M.2 1ea

    		M.2 1ea
    DATA_
    NVMe

    Micron 9300 MAX 3.2TB U.2 (10ea)

    • PCIe 3.0 x4
    • configured as RAID10

    Hynix P51 1TB M.2 (8~12ea)

    • PCIe 5.0 x4
    • 29만원/ea (232만원/8ea, 348만원/12ea)

    ASUS Hyper M.2 x16 Gen5 Card (2~3ea)

    • 12만원/1ea (관세 부가 전)

    비고

    • 총 비용 240만원/8ea, 400만원/12ea
    • RAID0 묶어서 테스트 할 것

    Hynix P51 1TB M.2 (8~12ea)

    • PCIe 5.0 x4
    • 29만원/1ea (232만원/8ea, 348만원/12ea)

    ASUS Hyper M.2 x16 Gen5 Card (2~3ea)

    • 12만원/1ea (관세 부가 전)

    저렴한 M.2 기반 구성

    • U.2는 국내 일반 마켓에서 구매가 어려워 M.2 선호

    Gen4 NVMe 5개, Gen5 NVMe 2~3개면 
    레퍼런스와 비슷한 성능을 낼 수 있을 듯

    목표는 레퍼런스보다 더 빠른 성능

    ASUS Hyper M.2 x16 Gen5 Card

    • Hyper M.2 x16 Gen5 CardImage Modified
    NIC

    MCX613106A-VDAT (1ea)

    • PCIe® 4.0 x16
    • 200GbE Dual

    MCX75310AAS-NEAT (1ea)

    • PCIe 5.0 x16
    • 400GbE Single
    • RoCE 가능
    • $1700~4000 (250~580만원)

    MCX75310AAS-NEAT (1ea)

    • PCIe 5.0 x16
    • 400GbE Single
    • RoCE 가능
    		400GbE 싱글 포트
    PSU1300W (2ea)


    구매가$25K in late 20221.5천 / 1대 (목표)1.5천 / 1대 (목표)

    CPU

    선택 조건

    필터

    순번필터필터값비고
    1base clockover 3.0 GHz너무 느린것 제외
    2Pricingunder $ 5500너무 비싼 것 제외
    3Num of coresover 15 cores
    4Num of PCIe Lanesover 60NVMe 설치 위한 PCIe 레인 확보
    5출시일after Jan 2022
    6PCIe version5.0최신 NIC 설치

    ...

    https://www.intel.com/content/www/us/en/products/compare.html?productIds=240486,240485,233417,236878,240487,233479,233416

    SKUNamePCIe_RevPCIe_LanesCoresThreadsBase_GHzTurbo_GHzCachePricingFamilyLaunchLithographySocket_tScalabilityTDPMem_max비고
    240486w7-3565X 511232642.54.882.5 MB2699W24Q3Intel 7LGA46771S Only　4 TB1순위
    240485w7-3555 511228562.74.875 MB2339W24Q3Intel 7LGA46771S Only　4 TB
    233417w7-3465X 511228562.54.875 MB2899W23Q1Intel 7LGA46771S Only　4 TB
    236878w7-2595X 56426522.84.848.75 MB2049W24Q3Intel 7LGA46771S Only　2 TB
    240487w7-3545 511224482.74.867.5 MB2039W24Q3Intel 7LGA46771S Only　4 TB
    233479w7-3455 511224482.54.867.5 MB2489W23Q1Intel 7LGA46771S Only　4 TB
    233416w7-2495X 56424482.54.845 MB2199W23Q1Intel 7LGA46771S Only　2 TB
    232377Gold 6426Y 58016322.54.137.5 MB1517Scalables 4thQ1'23Intel 7LGA46772S185 W4 TB2소켓
    232383Platinum 8462Y+ 58032642.84.160 MB5945Scalables 4thQ1'23Intel 7LGA46772S300 W4 TB
    237558Platinum 8562Y+ 58032642.84.160 MB
    		Scalables 5thQ4'23Intel 7LGA46772S300 W4 TB
    237559Gold 6542Y 58024482.94.160 MB
    		Scalables 5thQ4'23Intel 7LGA46772S250 W4 TB
    237564Gold 6548Y+ 58032642.54.160 MB3726Scalables 5thQ4'23Intel 7LGA46772S250 W4 TB1소켓
    237566Gold 6558Q 58032643.24.160 MB6416Scalables 5thQ4'23Intel 7LGA46772S350 W4 TB
    237567Gold 6548N 58032642.84.160 MB
    		Scalables 5thQ4'23Intel 7LGA46772S250 W4 TB
    237569Gold 6544Y 58016323.64.145 MB
    		Scalables 5thQ4'23Intel 7LGA46772S270 W4 TB
    231749Platinum 8444H 58016322.9445 MB
    		Scalables 4thQ1'23Intel 7LGA4677S8S270 W4 TB
    232380Gold 6442Y 58024482.6460 MB
    		Scalables 4thQ1'23Intel 7LGA46772S225 W4 TB
    232385Gold 6444Y 58016323.6445 MB
    		Scalables 4thQ1'23Intel 7LGA46772S270 W4 TB
    232386Gold 6458Q 58032643.1460 MB
    		Scalables 4thQ1'23Intel 7LGA46772S350 W4 TB
    237259Platinum 8558P 58048962.74260 MB
    		Scalables 5thQ4'23Intel 7LGA46772S350 W4 TB
    237560Gold 6526Y 58016322.83.937.5 MB
    		Scalables 5thQ4'23Intel 7LGA46772S195 W4 TB
    2407756960P 596721442.73.9432 MB
    		Xeon 6Q3'24Intel 3LGA75292S500 W3 TB
    231742Platinum 8458P 58044882.73.882.5 MB
    		Scalables 4thQ1'23Intel 7LGA46772S350 W4 TB
    232597CPU Max 9462 58032642.73.575 MB
    		CPU MaxQ1'23Intel 7LGA46772S350 W4 TB

    Chassis

    SMCi AS-2015CS-TNR

    ...

    https://docs.nvidia.com/networking/display/connectx6vpi

    NVIDIA SKU

    Legacy OPN

    Marketing Description

    Lifecycle

    900-9X603-0016-DT0

    MCX653105A-EFAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE),  single-port QSFP56,  PCIe3.0/4.0 Socket Direct 2x8 in a row, tall bracket

    Mass Production

    900-9X603-0056-DT0

    MCX653106A-EFAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IBand100GbE),  dual-port QSFP56,  PCIe3.0/4.0 Socket Direct 2x8 in a row, tall bracket

    Mass Production

    900-9X628-0016-ST0

    MCX651105A-EDAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE,  single-port QSFP56,  PCIe4.0 x8, tall bracket

    Mass Production

    900-9X6AF-0016-ST1

    MCX653105A-ECAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE),  single-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

    Mass Production

    900-9X6AF-0056-MT1

    MCX653106A-ECAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE),  dual-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

    Mass Production

    900-9X6AF-0018-MT2

    MCX653105A-HDAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  single-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

    Mass Production

    900-9X6AF-0058-ST1

    MCX653106A-HDAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  dual-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

    Mass Production

    900-9X6B4-0058-DT0

    MCX654106A-HCAT

    ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  dual-port QSFP56,  Socket Direct 2x PCIe3.0/4.0x16, tall bracket

    Mass Production

    900-9X6AF-0018-SS0

    MCX653105A-HDAL

    ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  single-port QSFP56,  PCIe3.0/4.0 x16,  cold plate for liquid-cooled Intel® Server System D50TNP platforms, tall bracket, ROHS R6

    Mass Production

    900-9X6AF-0058-SS0

    MCX653106A-HDAL

    ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  dual-port QSFP56,  PCIe3.0/4.0 x16,  cold plate for liquid-cooled Intel® Server System D50TNP platforms, tall bracket, ROHS R6

    Mass Production

    900-9X0BC-001H-ST1

    MCX683105AN-HDAT

    ConnectX®-6 DE adapter card,  HDR IB  (200Gb/s),  single-port QSFP,  PCIe4.0 x16,  no crypto, tall bracket

    Mass Production

    ConnectX-7 (OSFP)

    https://docs.nvidia.com/networking/display/connectx7vpi

    NVIDIA SKU

    Legacy OPN

    Form Factor

    Data Transmission Rate

    No. of Ports and Type

    PCIe Support

    Secure Boot

    Crypto

    Bracket Type

    Lifecycle

    900-9X766-003N-SQ0

    MCX75310AAS-NEAT

    PCIe Half Height, Half Length

    2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

    InfiniBand: NDR 400Gb/s (Default speed)

    Ethernet: 400GbE

    Single-port OSFP

    PCIe x16 Gen 4.0/5.0 @ SERDES 16GT/s/32GT/s

    ✔

    -

    Tall Bracket

    Mass Production

    900-9X766-003N-SR0

    MCX75310AAC-NEAT

    PCIe Half Height, Half Length

    2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

    InfiniBand: NDR 400Gb/s (Default speed)

    Ethernet: 400GbE

    Single-port OSFP

    PCIe x16 Gen 4.0/5.0 @ SERDES 16GT/s/32GT/s

    ✔

    ✔

    Tall Bracket

    Mass Production

    900-9X766-003N-ST0

    MCX75310AAS-HEAT

    PCIe Half Height, Half Length

    2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

    InfiniBand: NDR200 200Gb/s (Default speed)

    Ethernet: 200GbE

    Single-port OSFP

    PCIe x16 Gen 4.0/5.0 @ SERDES 16GT/s/32GT/s

    ✔

    -

    Tall Bracket

    Mass Production

    900-9X7AO-00C3-STZ

    MCX713104AC-ADAT

    PCIe Half Height, Half Length

    2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

    Ethernet: 50/25GbE

    Quad-port SFP56

    PCIe x16 Gen 4.0 @ SERDES 16GT/s

    ✔

    ✔

    Tall Bracket

    Mass Production

    900-9X7AO-0003-ST0

    MCX713104AS-ADAT

    PCIe Half Height, Half Length

    2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

    Ethernet: 50/25GbE

    Quad-port SFP56

    PCIe x16 Gen 4.0 @ SERDES 16GT/s

    ✔

    -

    Tall Bracket

    Mass Production

    OSFP - QSFP 연결

    OSFP-SR 트랜시버 - OM4 케이블 - QSFP-DD SR 트랜시버 연결

    ...