버전 비교

이전 버전 26

changes.mady.by.user 장민석(KN)

에 저장

비교대상

새 버전 현재

changes.mady.by.user 장민석(KN)

에 저장

키

  • 이 줄이 추가되었습니다.
  • 이 줄이 삭제되었습니다.
  • 서식이 변경되었습니다.

...

  • 400GbE 패킷 생성 (dpdk, RoCE, iperf3)
  • 400GbE 링크에서 NVMe 기반 Disk to Disk 전송 실험 (NVMe의 MTBF, TBW 낮아도 됨)
  • PCIe 버스를 최대한 Gen5 NVMe로 채우는 구성
  • 실험 종료 후 추후 HDD 추가하여 



LGA4677Motherboard H12DSUiN1ea

저렴한 M.2 기반 구성

  • U.2는 국내 일반 마켓에서 구매가 어려워 M.2 선호

Gen4 NVMe 5개, Gen5 NVMe 2~3개면 
레퍼런스와 비슷한 성능을 낼 수 있을 듯

목표는 레퍼런스보다 더 빠른 성능

Hyper M.2 x16 Gen5 CardImage Removed

Compatibility

레퍼런스Type AType B비고

2023 ESnet6 200G DTN2025 KREONET 400G DTN Intel2025 KREONET 400G DTN AMD
CPU

AMD EPYC 73F3 (2ea) (Total 32C)

  • 16C 24T (per 1ea)
  • Base / Turbo
    : 3.5 / 4.0 GHz
  • L3 256 MB
  • PCIe 4.0 x128
  • EPYC 7003 Series (SP3)
  • TDP 240W
  • Q1'21
  • $3521

Intel Xeon w7-3565X

  • 32C 64T
  • Base / Turbo
    : 2.5 / 4.8 GHz
  • 82.5 MB
  • PCIe 5.0 x112
  • Xeon W (
  • LGA-4677)
  • TDP 335~402W
  • Q3'24
  • $2699 (390만원)

AMD EPYC™ 9355P (1ea)

  • 32C 64T
  • Base / All Turbo / Max Turbo
    : 3.55 / 4.2 / 4.4 GHz
  • L3 256 MB
  • PCIe 5.0 x128
  • EPYC 9005 Series (SP5)
  • TDP 280W
  • Q4'24
  • $2998 (435만원)

싱글소켓 구성 (듀얼소켓 구성 가격 비쌈)

Chassis

SMCi

2124US-

SMCi X13SWA-TF

SMCi H13SSW

  • EPYC™ 9005 series (SP5)
  • 1 PCIe 5.0 x16 Right Riser Slot
  • 1 PCIe 5.0 x16 Left Riser Slot
  • 2 PCIe 5.0 x16 AIOM Networking Slot
  • 6 MCIO (PCIe 5.0 x8) Port(s)
  • 2 MCIO (PCIe 5.0 x8/SATA3) Port(s)
  • M.2 Interface: 2 PCIe 3.0 x2
  • IPMI
  • (매뉴얼) SMCi H13SSW

IPMI 기능 없는 경우

Chassis

SMCi 2124US-TNRP

  • 24 front hot-swap 2.5" NVMe U.2

AS -1115CS-TNR

  • SMCi H13SSW

  • 2 PCIe 5.0 x16 FHHL slots

  • 2 PCIe 5.0 x16 AIOM slots (OCP 3.0 compatible)

  • 10 front hot-swap 2.5" PCIe 5.0 NVMe* drive bays
RAM

DDR4-3200 16GB x 16ea (256GB)

256GB, 속도 제일 빠른 구성

DDR5-4800 ECC 64GB x 4ea

  • 46만원/1ea (184만원/4ea)

256GB, 속도 제일 빠른 구성

DDR5-4800 ECC 64GB x 4ea

  • 46만원/1ea (184만원/4ea)
OS_Disk

Micron 7300 MAX 800G U.2 (2ea)

  • PCIe 3.0 x4

M.2 1ea

M.2 1eaDATA_Disk

Micron 9300 MAX 3.2TB U.2 (10ea)

  • PCIe 3.0 x4
  • configured as RAID10

Hynix P51 1TB M.2 (8~12ea)

  • PCIe 5.0 x4
  • 29만원/ea (232만원/8ea, 348만원/12ea)

ASUS Hyper M.2 x16 Gen5 Card (2~3ea)

  • 12만원/1ea (관세 부가 전)

비고

  • 총 비용 240만원/8ea, 400만원/12ea
  • RAID0 묶어서 테스트 할 것

TNRP

  • 24 front hot-swap 2.5" NVMe U.2


SMCi > Chassis > 검색조건

  • Global SKU
  • Drive Form Factor 3.5 inch

SuperChassis 846BE1C-R1K23B

  • Image Added
  • ATX, E-ATX-, EE-ATX
  • PCIe FHFL 장착 가능

SuperChassis 743AC-1K26B-SQ

  • Image Added
  • ATX, E-ATX
  • PCIe FHFL 장착 가능
추후 디스크 추가를 위해 베이가 많은 샤시 선호
Motherboard

SMCi H12DSU-iN

SMCi > Motherboards > 검색조건

  • Form Factor: ATX, E-ATX
  • Socket: LGA-4677

SMCi X13SWA-TF

  • Image Added
  • Form Factor: E-ATX
  • LGA-4677
  • Intel W790
  • 6 PCIe 5.0 x16
    • 1 x 400GbE
    • 3 x Hyper M2
    • 1 x Mega RAID
  • Gen5 M.2 onboard
  • VROC key 설치 가능
  • IPMI

SMCi > Motherboards > 검색조건

  • Form Factor: ATX, E-ATX
  • Socket: SP5

SMCi H13SSL-NT

  • Image Added
  • Form Factor: ATX
  • EPYC™ 9005 series (SP5)
  • 3 PCIe 5.0 x16
    • 1  x 400GbE
    • 2 x Hyper M2
  • 2 PCIe 5.0 x8
    • 1 x Mega RAID
  • 3 MCIO PCIe 5.0 x8
  • Gen4 M.2 onboard
  • IPMI


RAM

DDR4-3200 16GB x 16ea (256GB)

256GB, 속도 제일 빠른 구성

DDR5-4800 ECC 64GB x 4ea

  • 46만원/1ea (184만원/4ea)

256GB, 속도 제일 빠른 구성

DDR5-4800 ECC 64GB x 4ea

  • 46만원/1ea (184만원/4ea)

OS_M2

Micron 7300 MAX 800G U.2 (2ea)

  • PCIe 3.0 x4

M.2 1ea

M.2 1ea
DATA_NVMe

Micron 9300 MAX 3.2TB U.2 (10ea)

  • PCIe 3.0 x4
  • configured as RAID10

Hynix P51 1TB M.2 (8~12ea)

  • PCIe 5.0 x4
  • 29만원/
  • ea (232만원/8ea, 348만원/12ea)

ASUS Hyper M.2 x16 Gen5 Card (2~3ea)

  • 12만원/1ea (관세 부가 전)

비고

  • 총 비용 240만원/8ea, 400만원/12ea
  • RAID0 묶어서 테스트 할 것

Hynix P51 1TB M.2 (8~12ea)

  • PCIe 5.0 x4
  • 29만원/1ea (232만원/8ea, 348만원/12ea)

ASUS Hyper M.2 x16 Gen5 Card

(2~3ea)

  • 12만원/1ea (관세 부가 전)

저렴한 M.2 기반 구성

  • U.2는 국내 일반 마켓에서 구매가 어려워 M.2 선호

Gen4 NVMe 5개, Gen5 NVMe 2~3개면 
레퍼런스와 비슷한 성능을 낼 수 있을 듯

목표는 레퍼런스보다 더 빠른 성능

ASUS Hyper M.2 x16 Gen5 Card

  • Hyper M.2 x16 Gen5 CardImage Added
  • 호환성
NIC

MCX613106A-VDAT (1ea)

  • PCIe® 4.0 x16
  • 200GbE Dual

MCX75310AAS-NEAT (1ea)

  • PCIe 5.0 x16
  • 400GbE Single
  • RoCE 가능
  • $1700~4000 (250~580만원)

MCX75310AAS-NEAT (1ea)

  • PCIe 5.0 x16
  • 400GbE Single
  • RoCE 가능
400GbE 싱글 포트
PSU1300W (2ea)


구매가$25K in late 20221.5천 / 1대 (목표)1.5천 / 1대 (목표)

CPU

선택 조건

필터

순번필터필터값비고
1base clockover 3.0 GHz너무 느린것 제외
2Pricingunder $ 5500너무 비싼 것 제외
3Num of coresover 15 cores
4Num of PCIe Lanesover 60NVMe 설치 위한 PCIe 레인 확보
5출시일after Jan 2022
6PCIe version5.0최신 NIC 설치

...

https://www.intel.com/content/www/us/en/products/compare.html?productIds=240486,240485,233417,236878,240487,233479,233416,240485,233417,236878,240487,233479,233416

5675 289923Q12368782595X 642652848.20492 2404873545 2448767.5 203924Q32334793455 11224485675 248923Q14 2334162495X 64545 219923Q12 Gold 6426Y 8016321371517Q1'232S185 W2소켓8032648160 5945Q1'232S300 W4 Platinum 8562Y+ 860 5thQ4300 Gold 6542Y 24489 5thQ4250 237564Gold 6548Y53726250 1소켓 6558Q 3264326416350 237567 6548N 8 TB237569 6544Y 1632645 270 231749Platinum 8444H 1632945 4thQ1S8S270 232380 6442Y 2448260 4thQ1225 232385Gold 6444Y 362S232386 6458Q 326431350 237259Platinum 8558P 489627260 5thQ4350 237560 6526Y 162.8937.5 5thQ4195 2407756960P 96721443.9432 Xeon 6Q324 3LGA7529500 3 231742Platinum 8458P 44887882 4thQ1350 232597CPU Max 9462 803264575 CPU MaxQ123 7LGA4677350 4
SKUNamePCIe_RevPCIe_LanesCoresThreadsBase_GHzTurbo_GHzCachePricingFamilyLaunchLithographySocket_tScalabilityTDPMem_max비고
240486w7-3565X 51123264
SKUNamePCIe_RevPCIe_LanesCoresThreadsBase_GHzTurbo_GHzCachePricingFamilyLaunchLithographySocket_tScalabilityTDPMem_max비고
240486w7-3565X 511232642.54.882.5 MB2699W24Q3Intel 7LGA46771S Only　4 TB1순위
240485w7-3555 511228562.74.875 MB2339W24Q3Intel 7LGA46771S Only　4 TB233417w7-3465X 511228
2.54.8
82.5 MB
2699W
24Q3Intel 7LGA46771S Only　4 TB1순위
240485w7-
3555 5
112
28
562.
74.8
75 MB
2339W24Q3Intel 7LGA46771S Only　
4 TB

233417w7-
3465X 5112
28
562.
54.8
75 MB
2899W
23Q1Intel 7LGA46771S Only　4 TB

236878w7-
2595X 5
64
26
522.
84.8
48.
75 MB
2049W
24Q3Intel 7LGA46771S Only　
2 TB

240487w7-
3545 5
11224482.
74.8
67.5 MB
2039W
24Q3Intel 7LGA46771S Only　
4 TB
232377

233479w7-3455 
5
112
24
482.54.
8
67.5 MB
2489
Scalables 4th
W23Q1
Intel 7LGA4677
1S Only
　4 TB

233416
232383Platinum 8462Y+ 
w7-2495X 5
64
24
482.
54.
8
45 MB
2199
Scalables 4th
W23Q1
Intel 7LGA4677
1S Only
　
2 TB
237558

232377Gold 6426Y 
5801632
64
2.
54.1
37.5 MB1517Scalables
4th
Q1'23Intel 7LGA46772S
185 W4 TB
237559
2소켓
232383Platinum 8462Y+ 
580
32
642.
84.160 MB5945Scalables
4th
Q1'23Intel 7LGA46772S
300 W4 TB

237558
Platinum 8562Y+ 58032642.
84.160 MB

Scalables 5thQ4'23Intel 7LGA46772S
300 W4 TB

237559
237566
Gold
6542Y 580
24
48
2.
94.160 MB

Scalables 5thQ4'23Intel 7LGA46772S
250 W4 TB

237564Gold
6548Y+ 58032642.
54.160 MB3726Scalables 5thQ4'23Intel 7LGA46772S250 W4
TB1소켓
237566Gold
6558Q 580
32
643.
24.1
60 MB6416Scalables 5thQ4'23Intel 7LGA46772S
350 W4 TB

237567
Gold 6548N 580
32
642.
84.1
60 MB
Scalables
5th
Q4'23Intel 7LGA4677
2S
250 W4 TB

237569Gold
6544Y 580
16
32
3.64.1
45 MB
Scalables
5th
Q4'23Intel 7LGA46772S
270 W4 TB

231749
Platinum 8444H 5801632
2.
9445 MB
Scalables 4thQ1'23Intel 7LGA4677
S8S270 W4 TB

232380Gold
6442Y 580
24
48
2.
6460 MB
Scalables 4thQ1'23Intel 7LGA46772S
225 W4 TB

232385
Gold 6444Y 580
16
32
3.
64
45 MB
Scalables
4th
Q1'23Intel 7LGA46772S
270 W4 TB

232386Gold
6458Q 580
32
643.
1460
MB
Scalables
4th
Q1'23Intel 7LGA46772S
350 W4 TB

237259
Platinum 8558P 5
80
48
962.7
4
260 MB

Scalables 5th
Q4'
23Intel
7
LGA46772S
350 W
4 TB

237560
Gold 6526Y 580
16
322.
83.
9
37.5 MB
Scalables
5th
Q4'23Intel 7LGA46772S
195 W4 TB

240775
6960P 5
96
72
1442.73.
9
432 MB

Xeon 6
Q3'
24Intel
3
LGA75292S
500 W
3 TB

Chassis

SMCi AS-2015CS-TNR


231742Platinum 8458P 58044882.73.882.5 MB
Scalables 4thQ1'23Intel 7LGA46772S350 W4 TB
232597CPU Max 9462 58032642.73.575 MB
CPU MaxQ1'23Intel 7LGA46772S350 W4 TB

Chassis

SMCi AS-2015CS-TNR

Image Removed
CPUMotherboardChassis

AMD EPYC™ 9355P

SMCi H13SSW

SMCi AS-2015CS-TNR

SMCi AS-2015CS-TNR (PDF)


Image Added


Image AddedImage Added




1x16 Full
CPUMotherboardChassis

AMD EPYC™ 9355P

SMCi H13SSW

SMCi AS-2015CS-TNR

Image Removed

Image Removed

커넥터버스Riser위치규격부품 연결PCIe 5.0 x128

Rear

AIOM1

PCIe 5.0 x16후면 A1OCP

ConnectX-7 400GbE

RearAIOM2PCIe 5.0 x16후면 A2OCPRearLWIOPCIe 5.0 x16

RSC-D2-666G5

  • Type: Passive 
  • 2 PCIe 5.0 x16
후면 1FHFL (Full-Height, Full-Length)

ASUS Hyper M.2 x16 Gen5

4 M.2 Gen5

MCIO3PCIe 5.0 x8후면 1FHHL (Full-Height, Half-Length)MCIO4PCIe 5.0 x8RearRWIOPCIe 5.0 x16

RSC-D2-666G5

  • Type: Passive 
  • 2 PCIe 5.0 x16
후면 5FHFL (Full-Height, Full-Length)

ASUS Hyper M.2 x16 Gen5 Card

4 Hynix P51 1TB M.2

Gen5

MCIO5
2
PCIe 5.0
x8 Full
후면 6FHHL (Full-Height, Half-Length)MCIO6PCIe 5.0 x8MCIO1PCIe 5.0 x8Front Backplane전면MCIO2PCIe 5.0 x8MCIO7PCIe 5.0 x8MCIO8PCIe 5.0 x8

...


3x16 Half
4x8 Full


5x16 Full

ASUS Hyper M.2 x16 Gen5 Card

2 Hynix P51 1TB M.2

6x16 Half

NVMe




NVMe 성능


Hynix P31 1TBHynix P41 1TBKIOXIA Plus G4 1TBPhison X200P 1.92TBPhison D200P 1.92TBHynix P51 1TB비고
가격 (만원)141517707929
PCIe3.04.05.05.05.05.05.0

M.2

M.2

M.2

U.2

E1.S

성능

M.2


성능

순차읽기: 3,500MB/s 

순차쓰기: 3,200MB/s

읽기IOPS: 570K

쓰기IOPS: 600K

M.2

순차읽기: 7,000MB/s

순차쓰기: 6,500MB/s

읽기IOPS: 1,400K

쓰기IOPS: 1,300KM.2

순차읽기: 10,000MB/s

순차쓰기: 7,900MB/s

읽기IOPS: 1,300K

쓰기IOPS: 1,400KM.2

순차읽기: 14,800MB/s

순차쓰기: 4,300MB/s

읽기IOPS: 2,400K

쓰기IOPS: 170K

순차읽기: 14,000MB/s

순차쓰기: 8,600MB/s

읽기IOPS: 3,000K

쓰기IOPS: 400K

순차읽기: 14,700MB/s

순차쓰기: 13,400MB/s

읽기IOPS: 2,300K

쓰기IOPS: 2,400K

PCIe 버전 오를 때마다
약 2배씩 성능 향상

NVMe 개수 적게 장착하려면
무조건 최신 PCIe 지원되는 CPU/메인보드 구매

...

https://docs.nvidia.com/networking/display/connectx6vpi

NVIDIA SKU

Legacy OPN

Marketing Description

Lifecycle

900-9X603-0016-DT0

MCX653105A-EFAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE),  single-port QSFP56,  PCIe3.0/4.0 Socket Direct 2x8 in a row, tall bracket

Mass Production

900-9X603-0056-DT0

MCX653106A-EFAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IBand100GbE),  dual-port QSFP56,  PCIe3.0/4.0 Socket Direct 2x8 in a row, tall bracket

Mass Production

900-9X628-0016-ST0

MCX651105A-EDAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE,  single-port QSFP56,  PCIe4.0 x8, tall bracket

Mass Production

900-9X6AF-0016-ST1

MCX653105A-ECAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE),  single-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

Mass Production

900-9X6AF-0056-MT1

MCX653106A-ECAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  100Gb/s  (HDR100, EDR IB and 100GbE),  dual-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

Mass Production

900-9X6AF-0018-MT2

MCX653105A-HDAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  single-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

Mass Production

900-9X6AF-0058-ST1

MCX653106A-HDAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  dual-port QSFP56,  PCIe3.0/4.0 x16, tall bracket

Mass Production

900-9X6B4-0058-DT0

MCX654106A-HCAT

ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  dual-port QSFP56,  Socket Direct 2x PCIe3.0/4.0x16, tall bracket

Mass Production

900-9X6AF-0018-SS0

MCX653105A-HDAL

ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  single-port QSFP56,  PCIe3.0/4.0 x16,  cold plate for liquid-cooled Intel® Server System D50TNP platforms, tall bracket, ROHS R6

Mass Production

900-9X6AF-0058-SS0

MCX653106A-HDAL

ConnectX®-6 InfiniBand/Ethernet adapter card,  HDR IB  (200Gb/s) and 200GbE,  dual-port QSFP56,  PCIe3.0/4.0 x16,  cold plate for liquid-cooled Intel® Server System D50TNP platforms, tall bracket, ROHS R6

Mass Production

900-9X0BC-001H-ST1

MCX683105AN-HDAT

ConnectX®-6 DE adapter card,  HDR IB  (200Gb/s),  single-port QSFP,  PCIe4.0 x16,  no crypto, tall bracket

Mass Production

ConnectX-7 (OSFP)

https://docs.nvidia.com/networking/display/connectx7vpi

NVIDIA SKU

Legacy OPN

Form Factor

Data Transmission Rate

No. of Ports and Type

PCIe Support

Secure Boot

Crypto

Bracket Type

Lifecycle

900-9X766-003N-SQ0

MCX75310AAS-NEAT

PCIe Half Height, Half Length

2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

InfiniBand: NDR 400Gb/s (Default speed)

Ethernet: 400GbE

Single-port OSFP

PCIe x16 Gen 4.0/5.0 @ SERDES 16GT/s/32GT/s

✔

-

Tall Bracket

Mass Production

900-9X766-003N-SR0

MCX75310AAC-NEAT

PCIe Half Height, Half Length

2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

InfiniBand: NDR 400Gb/s (Default speed)

Ethernet: 400GbE

Single-port OSFP

PCIe x16 Gen 4.0/5.0 @ SERDES 16GT/s/32GT/s

✔

✔

Tall Bracket

Mass Production

900-9X766-003N-ST0

MCX75310AAS-HEAT

PCIe Half Height, Half Length

2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

InfiniBand: NDR200 200Gb/s (Default speed)

Ethernet: 200GbE

Single-port OSFP

PCIe x16 Gen 4.0/5.0 @ SERDES 16GT/s/32GT/s

✔

-

Tall Bracket

Mass Production

900-9X7AO-00C3-STZ

MCX713104AC-ADAT

PCIe Half Height, Half Length

2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

Ethernet: 50/25GbE

Quad-port SFP56

PCIe x16 Gen 4.0 @ SERDES 16GT/s

✔

✔

Tall Bracket

Mass Production

900-9X7AO-0003-ST0

MCX713104AS-ADAT

PCIe Half Height, Half Length

2.71 in. x 6.6 in. (68.90mm x 167.65 mm)

Ethernet: 50/25GbE

Quad-port SFP56

PCIe x16 Gen 4.0 @ SERDES 16GT/s

✔

-

Tall Bracket

Mass Production

OSFP - QSFP 연결

OSFP-SR 트랜시버 - OM4 케이블 - QSFP-DD SR 트랜시버 연결

...