How to measure network throughput in Ceph cluster

According to the Ceph documentation, "the journal size should be at least twice the product of the "expected throughput" multiplied by filestore max sync interval" [1] where "expected throughput" is the lower of drive throughput and network throughput. The same documentation gives guidance that 7200 RPM spinners have approximate throughput of 100MB/sec, so we have a value for drive throughput. But what about network throughput? How to measure it?

iperf to the rescue

Measuring throughput is incredibly easy with iperf3.

For openSUSE or SUSE Linux Enterprise (SLE), packages are available in the network:utilities project in OBS - https://software.opensuse.org/download.html?project=network%3Autilities&package=iperf

Once you have the package installed, running a throughput test between two nodes (node1 and node2, hypothetically) is as easy as:

On node1:

# iperf3 -s

On node2:

# iperf3 -c $IP_ADDR_OF_NODE1

[1] http://docs.ceph.com/docs/jewel/rados/configuration/osd-config-ref/