Mellanox CX-5 RoCE網(wǎng)絡(luò)性能測試數(shù)據(jù)

做RoCE性能測試時(shí)腺律，在網(wǎng)上沒有找到太多有效的完整的性能參考數(shù)據(jù)侈玄。
故把實(shí)測數(shù)據(jù)貼出逮诲，方便后來者進(jìn)行參考對(duì)比帜平。
系統(tǒng)未做明顯優(yōu)化幽告，僅跑了一遍tuned_adm和mlnx_tune。

結(jié)果

24c439fceb4249ab723698268ff3086.png

39dc6d008189f46073b721a78c50ad8.png

系統(tǒng)信息

Ubuntu20.04

GenuineIntel Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz N/A

Memory Status Total: 1007.53 GB Free: 898.98 GB

MLNX_OFED_LINUX-23.07-0.5.1.2

ConnectX-5 Device Status on PCI 17:00.0 100 Gb/sec (4X EDR)

perftest

測試RoCE v1和v2無明顯差別裆甩。僅列舉v2數(shù)據(jù)冗锁。

ib_atomic_bw 10.10.11.21 -F -n 100000
---------------------------------------------------------------------------------------
                    Atomic FETCH_AND_ADD BW Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Outstand reads  : 16
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0224 PSN 0x5c1839
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x1186 PSN 0xbd5c66
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]   MsgRate[Mpps]
 8          100000           0.00               11.06              1.449399
------------------------------------------------------------------------------------

ib_atomic_lat 10.10.11.21 -F -n 100000
---------------------------------------------------------------------------------------
                    Atomic FETCH_AND_ADD Latency Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 1
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Outstand reads  : 16
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0226 PSN 0xf7135e
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x1188 PSN 0xeff4d8
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]    t_avg[usec]    t_stdev[usec]   99% percentile[usec]   99.9% percentile[usec]
 8       100000          2.65           10.36        2.72              2.73             0.10            2.87                    4.04
---------------------------------------------------------------------------------------


ib_write_lat -a 10.10.11.21 -F
---------------------------------------------------------------------------------------
                    RDMA_Write Latency Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: OFF
 ibv_wr* API     : ON
 TX depth        : 1
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Max inline data : 220[B]
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0227 PSN 0x8f570f RKey 0x1813ab VAddr 0x007f0d4e0b2000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x1189 PSN 0x293652 RKey 0x1871f1 VAddr 0x007f23e376c000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]    t_avg[usec]    t_stdev[usec]   99% percentile[usec]   99.9% percentile[usec]
 2       1000          1.49           2.33         1.52                1.52             0.00            1.58                    2.33
 4       1000          1.39           2.31         1.41                1.41             0.00            1.43                    2.31
 8       1000          1.39           2.55         1.42                1.42             0.00            1.43                    2.55
 16      1000          1.39           2.80         1.42                1.42             0.00            1.43                    2.80
 32      1000          1.42           2.64         1.44                1.44             0.00            1.46                    2.64
 64      1000          1.43           2.42         1.46                1.47             0.00            1.55                    2.42
 128     1000          1.47           1.75         1.50                1.50             0.00            1.62                    1.75
 256     1000          2.22           2.47         2.25                2.26             0.00            2.41                    2.47
 512     1000          2.36           3.65         2.39                2.41             0.00            2.59                    3.65
 1024    1000          2.45           2.82         2.49                2.52             0.00            2.70                    2.82
 2048    1000          2.66           4.68         2.70                2.74             0.00            2.94                    4.68
 4096    1000          3.09           4.15         3.19                3.21             0.00            3.41                    4.15
 8192    1000          3.45           3.81         3.56                3.55             0.00            3.69                    3.81
 16384   1000          4.13           5.93         4.22                4.25             0.00            4.53                    5.93
 32768   1000          5.45           6.96         5.51                5.56             0.00            5.80                    6.96
 65536   1000          8.13           9.88         8.20                8.22             0.00            8.42                    9.88
 131072  1000          13.51          15.79        13.67               13.68            0.00            13.80                   15.79
 262144  1000          24.19          24.66        24.31               24.34            0.00            24.63                   24.66
 524288  1000          45.57          47.44        45.61               45.64            0.03            45.99                   47.44
 1048576 1000          88.37          89.54        89.20               89.12            0.00            89.48                   89.54
 2097152 1000          174.00         177.11       174.77              174.80           0.04            175.06                  177.11
 4194304 1000          345.85         346.20       345.99              345.99           0.00            346.14                  346.20
 8388608 1000          688.12         688.72       688.17              688.22           0.00            688.57                  688.72
---------------------------------------------------------------------------------------


ib_write_bw -a 10.10.11.21 -F --report_gbits
---------------------------------------------------------------------------------------
                    RDMA_Write BW Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x0229 PSN 0xff1ace RKey 0x203dbd VAddr 0x007f2b168e1000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x118b PSN 0xdeb80e RKey 0x027200 VAddr 0x007f1f13f19000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[Gb/sec]    BW average[Gb/sec]   MsgRate[Mpps]
 2          5000             0.11               0.11               6.881392
 4          5000             0.17               0.17               5.328560
 8          5000             0.48               0.48               7.554181
 16         5000             0.97               0.97               7.542184
 32         5000             1.94               1.94               7.570855
 64         5000             3.88               3.88               7.571238
 128        5000             7.74               7.74               7.556555
 256        5000             15.53              15.50              7.570179
 512        5000             30.89              30.84              7.530099
 1024       5000             59.02              58.95              7.195514
 2048       5000             89.85              89.65              5.471935
 4096       5000             97.47              97.40              2.972274
 8192       5000             97.75              97.72              1.491165
 16384      5000             97.89              97.88              0.746774
 32768      5000             97.96              97.96              0.373695
 65536      5000             98.01              98.00              0.186923
 131072     5000             98.02              98.01              0.093474
 262144     5000             98.03              98.03              0.046743
 524288     5000             98.03              98.03              0.023373
 1048576    5000             98.04              98.04              0.011687
 2097152    5000             98.04              98.04              0.005843
 4194304    5000             98.04              98.04              0.002922
 8388608    5000             98.04              98.04              0.001461
---------------------------------------------------------------------------------------


ib_read_lat -a 10.10.11.21 -F
---------------------------------------------------------------------------------------
                    RDMA_Read Latency Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 1
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Outstand reads  : 16
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x022a PSN 0xd157ea OUT 0x10 RKey 0x203dbd VAddr 0x007f87f9bad000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x118c PSN 0x313e2c OUT 0x10 RKey 0x027200 VAddr 0x007fd732cf8000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]    t_avg[usec]    t_stdev[usec]   99% percentile[usec]   99.9% percentile[usec]
 2       1000          2.66           2.95         2.71                2.72             0.00            2.92                    2.95
 4       1000          2.67           3.63         2.71                2.72             0.00            2.84                    3.63
 8       1000          2.72           3.22         2.78                2.79             0.00            3.02                    3.22
 16      1000          2.84           3.30         2.89                2.93             0.00            3.27                    3.30
 32      1000          2.84           3.30         2.89                2.93             0.00            3.26                    3.30
 64      1000          2.84           3.27         2.90                2.93             0.00            3.24                    3.27
 128     1000          2.87           3.74         2.93                2.93             0.00            3.01                    3.74
 256     1000          2.93           3.65         3.00                3.01             0.00            3.18                    3.65
 512     1000          3.03           3.47         3.10                3.13             0.00            3.33                    3.47
 1024    1000          3.16           3.63         3.23                3.26             0.00            3.44                    3.63
 2048    1000          3.39           4.03         3.47                3.51             0.00            3.91                    4.03
 4096    1000          3.93           4.37         4.04                4.05             0.00            4.22                    4.37
 8192    1000          4.48           4.97         4.63                4.65             0.00            4.88                    4.97
 16384   1000          5.28           6.10         5.55                5.59             0.00            5.96                    6.10
 32768   1000          6.66           7.63         6.90                6.97             0.00            7.51                    7.63
 65536   1000          9.36           12.17        9.76                9.80             0.03            10.59                   12.17
 131072  1000          14.06          17.44        14.24               14.41            0.29            16.09                   17.44
 262144  1000          24.75          28.09        24.91               24.95            0.10            25.33                   28.09
 524288  1000          46.15          54.60        46.31               46.60            1.06            53.16                   54.60
 1048576 1000          88.93          102.07       89.07               89.39            1.58            100.08                  102.07
 2097152 1000          174.49         201.24       175.41              175.52           2.04            179.07                  201.24
 4194304 1000          345.65         347.28       346.62              346.63           0.00            346.97                  347.28
 8388608 1000          688.47         781.16       688.81              688.93           1.36            689.32                  781.16
---------------------------------------------------------------------------------------


ib_read_bw -a 10.10.11.21 -F --report_gbits
---------------------------------------------------------------------------------------
                    RDMA_Read BW Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Outstand reads  : 16
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x022b PSN 0x1959d2 OUT 0x10 RKey 0x203dbd VAddr 0x007ff2ed43d000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x118d PSN 0x68625d OUT 0x10 RKey 0x027200 VAddr 0x007fad14218000
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[Gb/sec]    BW average[Gb/sec]   MsgRate[Mpps]
 2          1000           0.069752            0.068694            4.293375
 4          1000             0.17               0.17               5.186768
 8          1000             0.34               0.34               5.290751
 16         1000             0.67               0.67               5.251310
 32         1000             1.35               1.34               5.235130
 64         1000             2.71               2.70               5.282983
 128        1000             5.41               5.41               5.280663
 256        1000             10.73              10.71              5.228649
 512        1000             21.73              21.69              5.295133
 1024       1000             41.26              41.17              5.025470
 2048       1000             61.95              61.89              3.777417
 4096       1000             83.82              83.80              2.557335
 8192       1000             85.08              85.06              1.297944
 16384      1000             88.15              88.13              0.672377
 32768      1000             97.85              97.84              0.373236
 65536      1000             97.94              97.94              0.186813
 131072     1000             97.99              97.99              0.093450
 262144     1000             98.02              98.02              0.046739
 524288     1000             98.02              98.02              0.023370
 1048576    1000             98.03              98.03              0.011686
 2097152    1000             98.04              98.04              0.005843
 4194304    1000             98.04              98.04              0.002922
 8388608    1000             98.04              98.04              0.001461
---------------------------------------------------------------------------------------


ib_send_lat -a 10.10.11.21 -F
---------------------------------------------------------------------------------------
                    Send Latency Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 1
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Max inline data : 236[B]
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x022c PSN 0xb7944e
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x118e PSN 0x3a106
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]    t_avg[usec]    t_stdev[usec]   99% percentile[usec]   99.9% percentile[usec]
 2       1000          1.37           6.00         1.41                1.42             0.06            1.52                    6.00
 4       1000          1.29           2.84         1.33                1.33             0.00            1.36                    2.84
 8       1000          1.29           2.28         1.33                1.32             0.00            1.35                    2.28
 16      1000          1.31           2.28         1.34                1.34             0.00            1.36                    2.28
 32      1000          1.31           2.29         1.34                1.34             0.00            1.37                    2.29
 64      1000          1.36           3.16         1.38                1.39             0.00            1.44                    3.16
 128     1000          1.40           3.67         1.45                1.45             0.00            1.48                    3.67
 256     1000          2.11           4.55         2.17                2.18             0.03            2.36                    4.55
 512     1000          2.21           3.36         2.26                2.27             0.00            2.46                    3.36
 1024    1000          2.32           4.07         2.36                2.38             0.00            2.63                    4.07
 2048    1000          2.52           4.77         2.58                2.61             0.07            2.76                    4.77
 4096    1000          2.96           4.73         3.02                3.07             0.04            3.40                    4.73
 8192    1000          3.32           7.38         3.43                3.44             0.07            3.66                    7.38
 16384   1000          3.99           13.20        4.05                4.10             0.10            4.33                    13.20
 32768   1000          5.33           5.71         5.50                5.47             0.00            5.65                    5.71
 65536   1000          8.00           10.96        8.05                8.11             0.04            8.42                    10.96
 131072  1000          13.36          14.58        13.49               13.49            0.00            13.74                   14.58
 262144  1000          24.05          25.55        24.10               24.16            0.00            24.45                   25.55
 524288  1000          45.45          46.64        45.60               45.62            0.00            46.11                   46.64
 1048576 1000          88.23          90.04        89.11               89.01            0.00            89.46                   90.04
 2097152 1000          174.19         178.18       174.74              174.75           0.04            174.98                  178.18
 4194304 1000          345.46         349.33       345.99              345.99           0.03            346.18                  349.33
 8388608 1000          687.92         689.08       688.07              688.08           0.00            688.25                  689.08
---------------------------------------------------------------------------------------


ib_send_bw -a 10.10.11.21 -F --report_gbits
---------------------------------------------------------------------------------------
                    Send BW Test
 Dual-port       : OFF          Device         : mlx5_0
 Number of qps   : 1            Transport type : IB
 Connection type : RC           Using SRQ      : OFF
 PCIe relax order: ON
 ibv_wr* API     : ON
 TX depth        : 128
 CQ Moderation   : 100
 Mtu             : 4096[B]
 Link type       : Ethernet
 GID index       : 4
 Max inline data : 0[B]
 rdma_cm QPs     : OFF
 Data ex. method : Ethernet
---------------------------------------------------------------------------------------
 local address: LID 0000 QPN 0x022d PSN 0xcc91a
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:11
 remote address: LID 0000 QPN 0x118f PSN 0xdc89f
 GID: 00:00:00:00:00:00:00:00:00:00:255:255:10:10:11:21
---------------------------------------------------------------------------------------
 #bytes     #iterations    BW peak[Gb/sec]    BW average[Gb/sec]   MsgRate[Mpps]
 2          1000           0.052205            0.051348            3.209238
 4          1000             0.17               0.17               5.221836
 8          1000             0.34               0.34               5.304961
 16         1000             0.68               0.68               5.320089
 32         1000             1.37               1.36               5.327195
 64         1000             2.71               2.68               5.226214
 128        1000             5.44               5.44               5.308179
 256        1000             10.89              10.88              5.313214
 512        1000             21.73              21.70              5.298613
 1024       1000             43.10              43.08              5.258549
 2048       1000             79.29              79.22              4.835111
 4096       1000             95.93              95.89              2.926339
 8192       1000             96.92              96.91              1.478727
 16384      1000             97.53              97.52              0.744006
 32768      1000             97.77              97.77              0.372949
 65536      1000             97.91              97.91              0.186742
 131072     1000             97.97              97.97              0.093433
 262144     1000             98.00              97.99              0.046727
 524288     1000             98.02              98.02              0.023370
 1048576    1000             98.03              98.03              0.011686
 2097152    1000             98.03              98.03              0.005843
 4194304    1000             98.03              98.03              0.002922
 8388608    1000             98.04              98.04              0.001461
---------------------------------------------------------------------------------------

OpenMPI

mpirun -np 2 -host gpu01,gpu02 -x UCX_NET_DEVICES=mlx5_0:1 osu_latency
# OSU MPI Latency Test v7.2
# Size          Latency (us)
# Datatype: MPI_CHAR.
1                       1.35
2                       1.33
4                       1.33
8                       1.33
16                      1.33
32                      1.39
64                      1.50
128                     1.51
256                     1.82
512                     1.82
1024                    1.96
2048                    2.93
4096                    3.67
8192                    4.75
16384                   6.94
32768                   8.96
65536                  11.75
131072                 16.81
262144                 26.37
524288                 47.83
1048576                91.22
2097152               176.93
4194304               348.30


 mpirun -np 2 -host gpu01,gpu02 -x UCX_NET_DEVICES=mlx5_0:1 osu_bw
# OSU MPI Bandwidth Test v7.2
# Size      Bandwidth (MB/s)
# Datatype: MPI_CHAR.
1                       5.92
2                      11.76
4                      23.72
8                      47.17
16                     94.75
32                    190.71
64                    357.15
128                   726.07
256                  1269.35
512                  2345.14
1024                 4027.80
2048                 6507.12
4096                 9284.14
8192                10689.40
16384               11354.37
32768               11692.90
65536               11980.76
131072              12118.09
262144              12185.53
524288              12219.85
1048576             12237.13
2097152             12245.03
4194304             12249.79


mpirun -np 2 -host gpu01,gpu02 -x UCX_NET_DEVICES=ens121np0 osu_latency
# OSU MPI Latency Test v7.2
# Size          Latency (us)
# Datatype: MPI_CHAR.
1                      13.48
2                      13.22
4                      13.30
8                      13.38
16                     13.25
32                     13.24
64                     13.32
128                    13.44
256                    13.68
512                    13.87
1024                   14.15
2048                   14.82
4096                   15.76
8192                   53.69
16384                  44.04
32768                  51.87
65536                  67.73
131072                133.40
262144                156.58
524288                237.20
1048576               363.09
2097152               530.48
4194304               858.22


mpirun -np 2 -host gpu01,gpu02 -x UCX_NET_DEVICES=ens121np0 osu_bw
# OSU MPI Bandwidth Test v7.2
# Size      Bandwidth (MB/s)
# Datatype: MPI_CHAR.
1                       0.66
2                       1.71
4                       3.72
8                       7.06
16                     12.35
32                     27.62
64                     42.68
128                    63.58
256                   151.93
512                   402.99
1024                  825.05
2048                 1253.38
4096                 2052.76
8192                 2204.76
16384                2328.53
32768                1932.05
65536                2558.81
131072               4215.51
262144               5911.52
524288               6973.49
1048576              6751.61
2097152              6457.68
4194304              6459.98

最后編輯于：2023.10.10 11:05:18

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者

人面猴
序言：七十年代末，一起剝皮案震驚了整個(gè)濱河市嗤栓，隨后出現(xiàn)的幾起案子冻河，更是在濱河造成了極大的恐慌，老刑警劉巖茉帅，帶你破解...
沈念sama閱讀 222,681評(píng)論 6贊 517
死咒
序言：濱河連續(xù)發(fā)生了三起死亡事件叨叙，死亡現(xiàn)場離奇詭異，居然都是意外死亡担敌，警方通過查閱死者的電腦和手機(jī)摔敛，發(fā)現(xiàn)死者居然都...
沈念sama閱讀 95,205評(píng)論 3贊 399
救了他兩次的神仙讓他今天三更去死
文/潘曉璐我一進(jìn)店門，熙熙樓的掌柜王于貴愁眉苦臉地迎上來全封，“玉大人，你說我怎么就攤上這事桃犬∩层玻” “怎么了？”我有些...
開封第一講書人閱讀 169,421評(píng)論 0贊 362
道士緝兇錄：失蹤的賣姜人
文/不壞的土叔我叫張陵攒暇，是天一觀的道長土匀。經(jīng)常有香客問我，道長形用，這世上最難降的妖魔是什么就轧？我笑而不...
開封第一講書人閱讀 60,114評(píng)論 1贊 300
?港島之戀（遺憾婚禮）
正文為了忘掉前任，我火速辦了婚禮田度，結(jié)果婚禮上妒御，老公的妹妹穿的比我還像新娘。我一直安慰自己镇饺，他們只是感情好乎莉，可當(dāng)我...
茶點(diǎn)故事閱讀 69,116評(píng)論 6贊 398
惡毒庶女頂嫁案：這布局不是一般人想出來的
文/花漫我一把揭開白布。她就那樣靜靜地躺著奸笤，像睡著了一般惋啃。火紅的嫁衣襯著肌膚如雪。梳的紋絲不亂的頭發(fā)上监右，一...
開封第一講書人閱讀 52,713評(píng)論 1贊 312
城市分裂傳說
那天边灭，我揣著相機(jī)與錄音，去河邊找鬼健盒。笑死绒瘦，一個(gè)胖子當(dāng)著我的面吹牛称簿，可吹牛的內(nèi)容都是我干的。我是一名探鬼主播椭坚，決...
沈念sama閱讀 41,170評(píng)論 3贊 422
雙鴛鴦連環(huán)套：你想象不到人心有多黑
文/蒼蘭香墨我猛地睜開眼予跌，長吁一口氣：“原來是場噩夢(mèng)啊……” “哼！你這毒婦竟也來了善茎？” 一聲冷哼從身側(cè)響起券册，我...
開封第一講書人閱讀 40,116評(píng)論 0贊 277
萬榮殺人案實(shí)錄
序言：老撾萬榮一對(duì)情侶失蹤，失蹤者是張志新（化名）和其女友劉穎垂涯，沒想到半個(gè)月后烁焙，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體，經(jīng)...
沈念sama閱讀 46,651評(píng)論 1贊 320
?護(hù)林員之死
正文獨(dú)居荒郊野嶺守林人離奇死亡耕赘，尸身上長有42處帶血的膿包…… 初始之章·張勛以下內(nèi)容為張勛視角年9月15日...
茶點(diǎn)故事閱讀 38,714評(píng)論 3贊 342
?白月光啟示錄
正文我和宋清朗相戀三年骄蝇，在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了。大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片操骡。...
茶點(diǎn)故事閱讀 40,865評(píng)論 1贊 353
活死人
序言：一個(gè)原本活蹦亂跳的男人離奇死亡九火，死狀恐怖，靈堂內(nèi)的尸體忽然破棺而出册招，到底是詐尸還是另有隱情岔激，我是刑警寧澤，帶...
沈念sama閱讀 36,527評(píng)論 5贊 351
?日本核電站爆炸內(nèi)幕
正文年R本政府宣布是掰，位于F島的核電站虑鼎，受9級(jí)特大地震影響，放射性物質(zhì)發(fā)生泄漏键痛。R本人自食惡果不足惜炫彩，卻給世界環(huán)境...
茶點(diǎn)故事閱讀 42,211評(píng)論 3贊 336
男人毒藥：我在死后第九天來索命
文/蒙蒙一、第九天我趴在偏房一處隱蔽的房頂上張望絮短。院中可真熱鬧江兢，春花似錦、人聲如沸戚丸。這莊子的主人今日做“春日...
開封第一講書人閱讀 32,699評(píng)論 0贊 25
一樁弒父案，背后竟有這般陰謀
文/蒼蘭香墨我抬頭看了看天上的太陽限府。三九已至夺颤，卻和暖如春，著一層夾襖步出監(jiān)牢的瞬間胁勺，已是汗流浹背世澜。一陣腳步聲響...
開封第一講書人閱讀 33,814評(píng)論 1贊 274
情欲美人皮
我被黑心中介騙來泰國打工，沒想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留署穗，地道東北人寥裂。一個(gè)月前我還...
沈念sama閱讀 49,299評(píng)論 3贊 379
代替公主和親
正文我出身青樓嵌洼，卻偏偏與公主長得像，于是被迫代替她去往敵國和親封恰。傳聞我的和親對(duì)象是個(gè)殘疾皇子麻养，可洞房花燭夜當(dāng)晚...
茶點(diǎn)故事閱讀 45,870評(píng)論 2贊 361

Mellanox CX-5 RoCE網(wǎng)絡(luò)性能測試數(shù)據(jù)

結(jié)果

系統(tǒng)信息

perftest

OpenMPI

推薦閱讀更多精彩內(nèi)容