tree: 704cc7714386af86c40dddded2fbfca52a8b4d7d [path history] [tgz]
  1. client_pool.go
  2. download_benchmark.go
  3. go.mod
  4. go.sum
  5. main.go
  6. README.md
  7. upload_benchmark.go
  8. utils.go
  9. w1r3.go
storage/internal/benchmarks/README.md

go-bench-gcs

This is not an officially supported Google product

Run example:

This runs 1000 iterations on 512kib to 2Gib files in the background, sending output to out.log:

go run main -project {PROJECT_ID} -samples 1000 -object_size=524288..2147483648 -o {RESULTS_FILE_NAME} &> out.log &

CLI parameters

ParameterDescriptionPossible valuesDefault
-projectprojectIDa project ID*
-bucketbucket supplied for benchmarks
must be initialized
any GCS regionwill create a randomly named bucket
-bucket_regionbucket region for benchmarks
ignored if bucket is explicitly supplied
any GCS regionUS-WEST1
-ofile to output results to
if empty, will output to stdout
any file pathstdout
-output_typeoutput results as csv records or cloud monitoringcsv, cloud-monitoringcloud-monitoring
-samplesnumber of samples to reportany positive integer8000
-workersnumber of goroutines to run at once; set to 1 for no concurrencyany positive integer16
-clientstotal number of Storage clients to be used;
if Mixed APIs, then x3 the number are created
any positive integer1
-apiwhich API to useJSON: use JSON
XML: use JSON to upload and XML to download
GRPC: use GRPC without directpath enabled
Mixed: select an API at random for each object
DirectPath: use GRPC with directpath
Mixed
-object_sizeobject size in bytes; can be a range min..maxany positive integer1 048 576 (1 MiB)
-range_read_sizesize of the range to read in bytesany positive integer
<=0 reads the full object
0
-minimum_read_offsetminimum offset for the start of the range to be read in bytesany integer >00
-maximum_read_offsetmaximum offset for the start of the range to be read in bytesany integer >00
-read_buffer_sizeread buffer size in bytesany positive integer4096 for HTTP
32768 for GRPC
-write_buffer_sizewrite buffer size in bytesany positive integer4096 for HTTP
32768 for GRPC
-min_chunksizeminimum ChunkSize in bytesany positive integer16 384 (16 MiB)
-max_chunksizemaximum ChunkSize in bytesany positive integer16 384 (16 MiB)
-connection_pool_sizeGRPC connection pool sizeany positive integer4
-force_garbage_collectionwhether to force garbage collection
before every write or read benchmark
true or false (present/not present)false
-timeouttimeout (maximum time running benchmarks)
the program may run for longer while it finishes running processes
any time.Duration1h

* required values