Cloud Performence Testing PerfKitBenchmarker
Google Cloud
./pkb.py --benchmarks=aerospike,hadoop_terasort,cassandra_stress,mongodb_ycsb,copy_throughput,fio,coremark,iperf,netperf,ping,mesh_network
OpenStack
Be sure that you have correct settings wrong readme
export OS_USERNAME=admin export OS_TENANT_NAME=tenant_name export OS_AUTH_URL=https://endpoint:5000/v2.0 export OS_PASSWORD=password
After this i must fix connection retries which originaly comes from issue comment
vim perfkitbenchmarker/providers/openstack/os_virtual_machine.py +181
add decorator on WaitForBootCompletion @vm_util.Retry(log_errors=False, poll_interval=1)
@vm_util.Retry(log_errors=False, poll_interval=1)
def WaitForBootCompletion(self):
# Do one longer sleep, then check at shorter intervals.
if self.boot_wait_time is None:
self.boot_wait_time = 15
time.sleep(self.boot_wait_time)
self.boot_wait_time = 5
resp, _ = self.RemoteCommand('hostname', retries=1)
if self.bootable_time is None:
self.bootable_time = time.time()
if self.hostname is None:
self.hostname = resp[:-1]
Finally i had problems with floating ips refer to this issue must be used master branch now and use --openstack_network instead public/private networks
./pkb.py --zones=dev01 --benchmarks=iperf --cloud=OpenStack --image=ubuntu-14-04-x64-1455869035 --openstack_public_network=publicIPv6 --openstack_private_network=tcp-lab-public-mjk --openstack_floating_ip_pool=public3 --openstack_boot_from_volume=true --openstack_volume_size=25
Read Data
import sys
import json
from prettytable import PrettyTable
SKIPPEND_FIELDS = ['labels', 'sample_uri', 'official', 'run_uri', 'product_name', 'owner']
x = PrettyTable()
with open(sys.argv[1]) as data_file:
# make valid json
wrap_data_file = '{"data": [%s]}' % ','.join(data_file)
data = json.loads(wrap_data_file)
keys = []
for row in data['data']:
row_values = []
for key, value in row.items():
if key in SKIPPEND_FIELDS:
continue
if key not in keys:
keys.append(key)
row_values.append(value)
x.add_row(row_values)
x._set_field_names(keys)
print x
+---------------+--------------------+---------------+-------+-----------+
| timestamp | metric | value | test | unit |
+---------------+--------------------+---------------+-------+-----------+
| 1468065608.22 | Throughput | 867.0 | iperf | Mbits/sec |
| 1468065672.32 | Throughput | 1000.0 | iperf | Mbits/sec |
| 1468065734.64 | Throughput | 950.0 | iperf | Mbits/sec |
| 1468065798.77 | Throughput | 1000.0 | iperf | Mbits/sec |
| 1468065904.15 | End to End Runtime | 543.026502848 | iperf | seconds |
+---------------+--------------------+---------------+-------+-----------+
+---------------+-------------------------------------------------+----------------+------------------+-------------------------+
| timestamp | metric | value | test | unit |
+---------------+-------------------------------------------------+----------------+------------------+-------------------------+
| 1468667548.88 | TCP_RR_Transaction_Rate | 2910.2 | netperf | transactions_per_second |
| 1468667548.88 | TCP_RR_Latency_p50 | 337.0 | netperf | us |
| 1468667548.88 | TCP_RR_Latency_p90 | 378.0 | netperf | us |
| 1468667548.88 | TCP_RR_Latency_p99 | 459.0 | netperf | us |
| 1468667548.88 | TCP_RR_Latency_min | 264.0 | netperf | us |
| 1468667548.88 | TCP_RR_Latency_max | 3938.0 | netperf | us |
| 1468667548.88 | TCP_RR_Latency_stddev | 47.24 | netperf | us |
| 1468667613.0 | TCP_RR_Transaction_Rate | 6503.41 | netperf | transactions_per_second |
| 1468667613.0 | TCP_RR_Latency_p50 | 143.0 | netperf | us |
| 1468667613.0 | TCP_RR_Latency_p90 | 182.0 | netperf | us |
| 1468667613.0 | TCP_RR_Latency_p99 | 295.0 | netperf | us |
| 1468667613.0 | TCP_RR_Latency_min | 110.0 | netperf | us |
| 1468667613.0 | TCP_RR_Latency_max | 11987.0 | netperf | us |
| 1468667613.0 | TCP_RR_Latency_stddev | 110.68 | netperf | us |
| 1468667678.97 | TCP_CRR_Transaction_Rate | 937.43 | netperf | transactions_per_second |
| 1468667678.97 | TCP_CRR_Latency_p50 | 1047.0 | netperf | us |
| 1468667678.97 | TCP_CRR_Latency_p90 | 1195.0 | netperf | us |
| 1468667678.97 | TCP_CRR_Latency_p99 | 1483.0 | netperf | us |
| 1468667678.97 | TCP_CRR_Latency_min | 719.0 | netperf | us |
| 1468667678.97 | TCP_CRR_Latency_max | 58388.0 | netperf | us |
| 1468667678.97 | TCP_CRR_Latency_stddev | 362.88 | netperf | us |
| 1468667745.92 | TCP_CRR_Transaction_Rate | 2658.65 | netperf | transactions_per_second |
| 1468667745.92 | TCP_CRR_Latency_p50 | 352.0 | netperf | us |
| 1468667745.92 | TCP_CRR_Latency_p90 | 453.0 | netperf | us |
| 1468667745.92 | TCP_CRR_Latency_p99 | 678.0 | netperf | us |
| 1468667745.92 | TCP_CRR_Latency_min | 240.0 | netperf | us |
| 1468667745.92 | TCP_CRR_Latency_max | 24836.0 | netperf | us |
| 1468667745.92 | TCP_CRR_Latency_stddev | 181.1 | netperf | us |
| 1468667809.91 | TCP_STREAM_Throughput | 1219.15 | netperf | Mbits/sec |
| 1468667872.03 | TCP_STREAM_Throughput | 1980.57 | netperf | Mbits/sec |
| 1468667934.05 | UDP_RR_Transaction_Rate | 2819.29 | netperf | transactions_per_second |
| 1468667934.05 | UDP_RR_Latency_p50 | 346.0 | netperf | us |
| 1468667934.05 | UDP_RR_Latency_p90 | 391.0 | netperf | us |
| 1468667934.05 | UDP_RR_Latency_p99 | 482.0 | netperf | us |
| 1468667934.05 | UDP_RR_Latency_min | 272.0 | netperf | us |
| 1468667934.05 | UDP_RR_Latency_max | 9159.0 | netperf | us |
| 1468667934.05 | UDP_RR_Latency_stddev | 71.9 | netperf | us |
| 1468667996.12 | UDP_RR_Transaction_Rate | 5676.27 | netperf | transactions_per_second |
| 1468667996.12 | UDP_RR_Latency_p50 | 163.0 | netperf | us |
| 1468667996.12 | UDP_RR_Latency_p90 | 208.0 | netperf | us |
| 1468667996.12 | UDP_RR_Latency_p99 | 317.0 | netperf | us |
| 1468667996.12 | UDP_RR_Latency_min | 104.0 | netperf | us |
| 1468667996.12 | UDP_RR_Latency_max | 17181.0 | netperf | us |
| 1468667996.12 | UDP_RR_Latency_stddev | 162.91 | netperf | us |
| 1468668129.52 | End to End Runtime | 887.944772005 | netperf | seconds |
| 1468668483.88 | Average Latency | 0.2400873 | aerospike | ms |
| 1468668500.75 | Average Latency | 0.3620518 | aerospike | ms |
| 1468668518.17 | Average Latency | 0.4726747 | aerospike | ms |
| 1468668535.14 | Average Latency | 0.6685167 | aerospike | ms |
| 1468668552.03 | Average Latency | 0.8191464 | aerospike | ms |
| 1468668570.19 | Average Latency | 0.9406734 | aerospike | ms |
| 1468668587.13 | Average Latency | 1.1053107 | aerospike | ms |
| 1468668604.0 | Average Latency | 1.3500618 | aerospike | ms |
| 1468668621.05 | Average Latency | 1.625098 | aerospike | ms |
| 1468668638.09 | Average Latency | 1.6105459 | aerospike | ms |
| 1468668654.92 | Average Latency | 1.7278936 | aerospike | ms |
| 1468668671.75 | Average Latency | 1.9469765 | aerospike | ms |
| 1468668688.69 | Average Latency | 2.2629361 | aerospike | ms |
| 1468668705.52 | Average Latency | 2.3112812 | aerospike | ms |
| 1468668722.37 | Average Latency | 2.5033066 | aerospike | ms |
| 1468668739.24 | Average Latency | 2.7107017 | aerospike | ms |
| 1468668739.24 | max_throughput_for_completion_latency_under_1ms | 50991.6 | aerospike | req/s |
| 1468668845.98 | End to End Runtime | 716.450525999 | aerospike | seconds |
| 1468669064.44 | TCP_RR_Average_Latency | 0.193820078949 | mesh_network | ms |
| 1468669096.77 | TCP_STREAM_Total_Throughput | 3722.09 | mesh_network | Mbits/sec |
| 1468669240.75 | End to End Runtime | 394.763776064 | mesh_network | seconds |
| 1468669660.71 | cp throughput | 121.030245747 | copy_throughput | MB/sec |
| 1468669766.69 | End to End Runtime | 525.935447931 | copy_throughput | seconds |
| 1468669971.03 | Min Latency | 0.271 | ping | ms |
| 1468669971.03 | Average Latency | 0.327 | ping | ms |
| 1468669971.03 | Max Latency | 0.432 | ping | ms |
| 1468669971.03 | Latency Std Dev | 0.034 | ping | ms |
| 1468670073.82 | Min Latency | 0.287 | ping | ms |
| 1468670073.82 | Average Latency | 0.346 | ping | ms |
| 1468670073.82 | Max Latency | 1.073 | ping | ms |
| 1468670073.82 | Latency Std Dev | 0.079 | ping | ms |
| 1468670164.99 | End to End Runtime | 398.293880939 | ping | seconds |
| 1468670617.84 | overall RunTime | 136602.0 | mongodb_ycsb | ms |
| 1468670617.84 | overall Throughput | 7320.5370346 | mongodb_ycsb | ops/sec |
| 1468670617.84 | cleanup Operations | 32 | mongodb_ycsb | |
| 1468670617.84 | cleanup LatencyVariance | 116.994609375 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup MinLatency | 0.0 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup MaxLatency | 1.967 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup p50 latency | 0 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup p75 latency | 0 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup p90 latency | 0 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup p95 latency | 0 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup p99 latency | 1 | mongodb_ycsb | ms |
| 1468670617.84 | cleanup p99.9 latency | 1 | mongodb_ycsb | ms |
| 1468670617.84 | insert Operations | 1000000 | mongodb_ycsb | |
| 1468670617.84 | insert LatencyVariance | 480500.996815 | mongodb_ycsb | ms |
| 1468670617.84 | insert MinLatency | 0.215 | mongodb_ycsb | ms |
| 1468670617.84 | insert Return=OK | 1000000 | mongodb_ycsb | |
| 1468670617.84 | insert MaxLatency | 1316.524 | mongodb_ycsb | ms |
| 1468670617.84 | insert p50 latency | 2 | mongodb_ycsb | ms |
| 1468670617.84 | insert p75 latency | 3 | mongodb_ycsb | ms |
| 1468670617.84 | insert p90 latency | 6 | mongodb_ycsb | ms |
| 1468670617.84 | insert p95 latency | 10 | mongodb_ycsb | ms |
| 1468670617.84 | insert p99 latency | 38 | mongodb_ycsb | ms |
| 1468670617.84 | insert p99.9 latency | 63 | mongodb_ycsb | ms |
| 1468670731.86 | overall RunTime | 108966.0 | mongodb_ycsb | ms |
| 1468670731.86 | overall Throughput | 9177.17453151 | mongodb_ycsb | ops/sec |
| 1468670731.86 | cleanup Operations | 32 | mongodb_ycsb | |
| 1468670731.86 | cleanup LatencyVariance | 157.443027344 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup MinLatency | 0.0 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup MaxLatency | 2.282 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup p50 latency | 0 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup p75 latency | 0 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup p90 latency | 0 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup p95 latency | 0 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup p99 latency | 2 | mongodb_ycsb | ms |
| 1468670731.86 | cleanup p99.9 latency | 2 | mongodb_ycsb | ms |
| 1468670731.86 | read Operations | 500301 | mongodb_ycsb | |
| 1468670731.86 | read LatencyVariance | 9973.4822588 | mongodb_ycsb | ms |
| 1468670731.86 | read MinLatency | 0.186 | mongodb_ycsb | ms |
| 1468670731.86 | read Return=OK | 500301 | mongodb_ycsb | |
| 1468670731.86 | read MaxLatency | 207.848 | mongodb_ycsb | ms |
| 1468670731.86 | read p50 latency | 2 | mongodb_ycsb | ms |
| 1468670731.86 | read p75 latency | 3 | mongodb_ycsb | ms |
| 1468670731.86 | read p90 latency | 5 | mongodb_ycsb | ms |
| 1468670731.86 | read p95 latency | 7 | mongodb_ycsb | ms |
| 1468670731.86 | read p99 latency | 14 | mongodb_ycsb | ms |
| 1468670731.86 | read p99.9 latency | 31 | mongodb_ycsb | ms |
| 1468670731.86 | update Operations | 499699 | mongodb_ycsb | |
| 1468670731.86 | update LatencyVariance | 26269.7768856 | mongodb_ycsb | ms |
| 1468670731.86 | update MinLatency | 0.198 | mongodb_ycsb | ms |
| 1468670731.86 | update Return=OK | 499699 | mongodb_ycsb | |
| 1468670731.86 | update MaxLatency | 271.68 | mongodb_ycsb | ms |
| 1468670731.86 | update p50 latency | 2 | mongodb_ycsb | ms |
| 1468670731.86 | update p75 latency | 4 | mongodb_ycsb | ms |
| 1468670731.86 | update p90 latency | 7 | mongodb_ycsb | ms |
| 1468670731.86 | update p95 latency | 9 | mongodb_ycsb | ms |
| 1468670731.86 | update p99 latency | 23 | mongodb_ycsb | ms |
| 1468670731.86 | update p99.9 latency | 48 | mongodb_ycsb | ms |
| 1468670842.22 | overall RunTime | 105395.0 | mongodb_ycsb | ms |
| 1468670842.22 | overall Throughput | 9488.11613454 | mongodb_ycsb | ops/sec |
| 1468670842.22 | cleanup Operations | 32 | mongodb_ycsb | |
| 1468670842.22 | cleanup LatencyVariance | 169.459234375 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup MinLatency | 0.0 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup MaxLatency | 2.368 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup p50 latency | 0 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup p75 latency | 0 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup p90 latency | 0 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup p95 latency | 0 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup p99 latency | 2 | mongodb_ycsb | ms |
| 1468670842.22 | cleanup p99.9 latency | 2 | mongodb_ycsb | ms |
| 1468670842.22 | read Operations | 950095 | mongodb_ycsb | |
| 1468670842.22 | read LatencyVariance | 10461.0373394 | mongodb_ycsb | ms |
| 1468670842.22 | read MinLatency | 0.189 | mongodb_ycsb | ms |
| 1468670842.22 | read Return=OK | 950095 | mongodb_ycsb | |
| 1468670842.22 | read MaxLatency | 264.692 | mongodb_ycsb | ms |
| 1468670842.22 | read p50 latency | 2 | mongodb_ycsb | ms |
| 1468670842.22 | read p75 latency | 3 | mongodb_ycsb | ms |
| 1468670842.22 | read p90 latency | 5 | mongodb_ycsb | ms |
| 1468670842.22 | read p95 latency | 8 | mongodb_ycsb | ms |
| 1468670842.22 | read p99 latency | 15 | mongodb_ycsb | ms |
| 1468670842.22 | read p99.9 latency | 29 | mongodb_ycsb | ms |
| 1468670842.22 | update Operations | 49905 | mongodb_ycsb | |
| 1468670842.22 | update LatencyVariance | 32771.3813952 | mongodb_ycsb | ms |
| 1468670842.22 | update MinLatency | 0.234 | mongodb_ycsb | ms |
| 1468670842.22 | update Return=OK | 49905 | mongodb_ycsb | |
| 1468670842.22 | update MaxLatency | 282.403 | mongodb_ycsb | ms |
| 1468670842.22 | update p50 latency | 5 | mongodb_ycsb | ms |
| 1468670842.22 | update p75 latency | 8 | mongodb_ycsb | ms |
| 1468670842.22 | update p90 latency | 11 | mongodb_ycsb | ms |
| 1468670842.22 | update p95 latency | 13 | mongodb_ycsb | ms |
| 1468670842.22 | update p99 latency | 23 | mongodb_ycsb | ms |
| 1468670842.22 | update p99.9 latency | 49 | mongodb_ycsb | ms |
| 1468670957.31 | End to End Runtime | 792.317909956 | mongodb_ycsb | seconds |
| 1468673996.43 | op rate | 6809.0 | cassandra_stress | operations per second |
| 1468673996.43 | partition rate | 6809.0 | cassandra_stress | operations per second |
| 1468673996.43 | row rate | 6809.0 | cassandra_stress | operations per second |
| 1468673996.43 | latency mean | 22.0 | cassandra_stress | ms |
| 1468673996.43 | latency median | 8.3 | cassandra_stress | ms |
| 1468673996.43 | latency 95th percentile | 77.3 | cassandra_stress | ms |
| 1468673996.43 | latency 99th percentile | 153.1 | cassandra_stress | ms |
| 1468673996.43 | latency 99.9th percentile | 248.2 | cassandra_stress | ms |
| 1468673996.43 | latency max | 1825.1 | cassandra_stress | ms |
| 1468673996.43 | Total partitions | 2000000.0 | cassandra_stress | ms |
| 1468673996.43 | Total errors | 0.0 | cassandra_stress | ms |
| 1468673996.43 | Total operation time | 293.0 | cassandra_stress | seconds |
| 1468674105.34 | End to End Runtime | 1722.00360799 | cassandra_stress | seconds |
| 1468674278.53 | Coremark Score | 14067.665471 | coremark | |
| 1468674373.56 | End to End Runtime | 268.217623949 | coremark | seconds |
| 1468674597.64 | Throughput | 1107.0 | iperf | Mbits/sec |
| 1468674661.5 | Throughput | 1996.0 | iperf | Mbits/sec |
| 1468674723.41 | Throughput | 796.0 | iperf | Mbits/sec |
| 1468674787.24 | Throughput | 1991.0 | iperf | Mbits/sec |
| 1468674902.02 | End to End Runtime | 528.456736088 | iperf | seconds |
| 1468675259.49 | sequential_write:write:bandwidth | 74210 | fio | KB/s |
| 1468675259.49 | sequential_write:write:latency | 437911.31 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:min | 5418 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:max | 871722 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:mean | 437911.31 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:stddev | 113261.73 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p1 | 22400 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p5 | 209920 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p10 | 423936 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p20 | 432128 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p30 | 436224 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p40 | 440320 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p50 | 444416 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p60 | 444416 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p70 | 444416 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p80 | 448512 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p90 | 464896 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p95 | 634880 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p99 | 815104 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p99.5 | 847872 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p99.9 | 872448 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p99.95 | 872448 | fio | usec |
| 1468675259.49 | sequential_write:write:latency:p99.99 | 872448 | fio | usec |
| 1468675259.49 | sequential_write:write:iops | 144.94 | fio | |
| 1468675259.49 | sequential_read:read:bandwidth | 129283 | fio | KB/s |
| 1468675259.49 | sequential_read:read:latency | 251599.36 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:min | 13428 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:max | 524689 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:mean | 251599.36 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:stddev | 86376.62 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p1 | 14656 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p5 | 32128 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p10 | 113152 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p20 | 259072 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p30 | 264192 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p40 | 268288 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p50 | 268288 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p60 | 268288 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p70 | 268288 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p80 | 272384 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p90 | 284672 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p95 | 370688 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p99 | 485376 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p99.5 | 514048 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p99.9 | 528384 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p99.95 | 528384 | fio | usec |
| 1468675259.49 | sequential_read:read:latency:p99.99 | 528384 | fio | usec |
| 1468675259.49 | sequential_read:read:iops | 252.51 | fio | |
| 1468675259.49 | random_write_test:write:bandwidth | 6530 | fio | KB/s |
| 1468675259.49 | random_write_test:write:latency | 586.66 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:min | 257 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:max | 16592 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:mean | 586.66 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:stddev | 360.55 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p1 | 422 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p5 | 442 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p10 | 454 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p20 | 470 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p30 | 482 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p40 | 494 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p50 | 502 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p60 | 516 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p70 | 532 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p80 | 556 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p90 | 628 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p95 | 1592 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p99 | 1736 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p99.5 | 1832 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p99.9 | 3920 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p99.95 | 4960 | fio | usec |
| 1468675259.49 | random_write_test:write:latency:p99.99 | 16512 | fio | usec |
| 1468675259.49 | random_write_test:write:iops | 1632.75 | fio | |
| 1468675259.49 | random_read_test:read:bandwidth | 3239 | fio | KB/s |
| 1468675259.49 | random_read_test:read:latency | 1207.86 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:min | 0 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:max | 50324 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:mean | 1207.86 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:stddev | 1213.69 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p1 | 374 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p5 | 406 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p10 | 426 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p20 | 454 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p30 | 482 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p40 | 510 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p50 | 564 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p60 | 1640 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p70 | 2024 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p80 | 2128 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p90 | 2224 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p95 | 2288 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p99 | 2736 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p99.5 | 4768 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p99.9 | 14272 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p99.95 | 18816 | fio | usec |
| 1468675259.49 | random_read_test:read:latency:p99.99 | 50432 | fio | usec |
| 1468675259.49 | random_read_test:read:iops | 809.79 | fio | |
| 1468675259.49 | random_read_test_parallel:read:bandwidth | 3254 | fio | KB/s |
| 1468675259.49 | random_read_test_parallel:read:latency | 78614.48 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:min | 0 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:max | 163167 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:mean | 78614.48 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:stddev | 11522.41 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p1 | 3248 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p5 | 75264 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p10 | 76288 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p20 | 77312 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p30 | 78336 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p40 | 79360 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p50 | 80384 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p60 | 80384 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p70 | 81408 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p80 | 82432 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p90 | 82432 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p95 | 83456 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p99 | 87552 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p99.5 | 108032 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p99.9 | 152576 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p99.95 | 158720 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:latency:p99.99 | 162816 | fio | usec |
| 1468675259.49 | random_read_test_parallel:read:iops | 813.65 | fio | |
| 1468675367.3 | End to End Runtime | 465.271894932 | fio | seconds |
+---------------+-------------------------------------------------+----------------+------------------+-------------------------+