tree: d9820ff724c84d34232f2548abea55ca7e763357
  1. client_pool.go
  2. download_benchmark.go
  3. go.mod
  4. go.sum
  5. main.go
  6. README.md
  7. upload_benchmark.go
  8. utils.go
  9. w1r3.go
storage/internal/benchmarks/README.md

go-bench-gcs

This is not an officially supported Google product

Run example:

This runs 1000 iterations on 512kib to 2Gib files in the background, sending output to out.log:

go run main -p {PROJECT_ID} -t 72h -max_samples 1000 -o {RESULTS_FILE_NAME}.csv &> out.log &

CLI parameters

ParameterDescriptionPossible valuesDefault
-pprojectIDa project ID*
-credspath to credentials fileany pathfrom environment
-ofile to output results to
if empty, will output to stdout
any file pathstdout
-output_typeoutput results as csv records or cloud monitoringcsv, cloud-monitoringcloud-monitoring
-apiwhich API to useJSON: use JSON to upload and XML to download
XML: use JSON to upload and XML to download
GRPC: use GRPC
MIXED: select an API at random for each upload/download
DirectPath: use GRPC with direct path
MIXED
-rbucket region for benchmarksany GCS regionUS-WEST1
-workersnumber of goroutines to run at once; set to 1 for no concurrencyany positive integer16
-ttimeout (maximum time running benchmarks)
the program may run for longer while it finishes running processes
any time.Duration1h
-min_samplesminimum number of objects to uploadany positive integer10
-max_samplesmaximum number of objects to uploadany positive integer10 000
-gc_fwhether to force garbage collection
before every write or read benchmark
true or false (present/not present)false
-min_sizeminimum object size in bytesany positive integer512
-max_sizemaximum object size in bytesany positive integer2 097 152 (2 GiB)
-defaultsuse default settings for the client
(conn_pool, read, write and chunk size parameters will be ignored)
true or falsefalse
-conn_poolGRPC connection pool sizeany positive integer4
-min_csminimum ChunkSize in bytesany positive integer16 384 (16 MiB)
-max_csmaximum ChunkSize in bytesany positive integer16 384 (16 MiB)
-q_readdownload quantumany positive integer1
-q_writeupload quantumany positive integer1
-min_r_sizeminimum read size in bytesany positive integer4000
-max_r_sizemaximum read size in bytesany positive integer4000
-min_w_sizeminimum write size in bytesany positive integer4000
-max_w_sizemaximum write size in bytesany positive integer4000
-labelslabels added to cloud monitoring output (ignored when outputting as csv)any string; should be in the format:
stringKey=\"value\",intKey=3,boolKey=true
empty

* required values

Note: while the default read/write size for HTTP clients is 4Kb (the default for this benchmarking), the default for GRPC is 32Kb. If you want to capture performance using the defaults for GRPC run the script separately setting the read and write sizes to 32Kb, or run with the defaults parameter set.