spanner: keep better track of max sessions

The session pool did not keep proper track of the max number of sessions in use within one
maintenance window. This change ensures that the max number of sessions in use is actually kept
track of per window, and prevents the maintainer from deleting sessions too eagerly. This will
change the behavior of the session pool and lead to the pool keeping more sessions in the pool for
a longer time, and to less deletion and creation of sessions.

The logic is now:
1. Keep track of the max number of sessions in use when a session is taken from the pool, instead
of probing this only at the moment that the maintainer is running. This prevents the session pool
from missing spikes in session usage.
2. Store the max number of sessions during a health check cycle in a maintenance window struct
containing one value for each cycle in an entire window of 10 cycles.
3. Only delete sessions after at least one full maintenance window has passed. After that,
the maintainer will delete sessions based on a rolling set of maxSessionsInUse for the last
10 minutes.
4. The shrinkPool and replenishPool both take a max number of sessions to delete/create as an
argument in order to prevent the methods from deleting/creating more sessions than intended if the
number of sessions in the pool change as a result of other operations on the pool.
5. If the shrinkPool method notices that the session pool has started creating new sessions,
the method will stop deleting any further sessions.

Fixes #1303, #1382, #1398 and #1406.

Change-Id: I20c0821a45b8ce02cb09a9dc492d389da03cccab
Reviewed-on: https://code-review.googlesource.com/c/gocloud/+/40150
Reviewed-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Shanika Kuruppu <skuruppu@google.com>
Reviewed-by: Jean de Klerk <deklerk@google.com>
2 files changed
tree: eaaa1ab7ddb6db6ef010847cc454cde58cdb8cb8
  1. asset/
  2. automl/
  3. bigquery/
  4. bigtable/
  5. civil/
  6. cloudbuild/
  7. cloudtasks/
  8. cmd/
  9. compute/
  10. container/
  11. containeranalysis/
  12. dataproc/
  13. datastore/
  14. debugger/
  15. dialogflow/
  16. dlp/
  17. errorreporting/
  18. expr/
  19. firestore/
  20. functions/
  21. grafeas/
  22. httpreplay/
  23. iam/
  24. internal/
  25. iot/
  26. irm/
  27. kms/
  28. language/
  29. logging/
  30. longrunning/
  31. monitoring/
  32. oslogin/
  33. phishingprotection/
  34. profiler/
  35. pubsub/
  36. recaptchaenterprise/
  37. recommender/
  38. redis/
  39. rpcreplay/
  40. scheduler/
  41. securitycenter/
  42. spanner/
  43. speech/
  44. storage/
  45. talent/
  46. texttospeech/
  47. trace/
  48. translate/
  49. videointelligence/
  50. vision/
  51. webrisk/
  52. CHANGES.md
  53. CODE_OF_CONDUCT.md
  54. CONTRIBUTING.md
  55. doc.go
  56. examples_test.go
  57. gapics.txt
  58. go.mod
  59. go.sum
  60. issue_template.md
  61. LICENSE
  62. license_test.go
  63. manuals.txt
  64. microgens.csv
  65. README.md
  66. regen-gapic.sh
  67. RELEASING.md
  68. tidyall.sh
  69. tools.go
README.md

Google Cloud Client Libraries for Go

GoDoc

Go packages for Google Cloud Platform services.

import "cloud.google.com/go"

To install the packages on your system, do not clone the repo. Instead:

  1. Change to your project directory:

    cd /my/cloud/project
    
  2. Get the package you want to use. Some products have their own module, so it's best to go get the package(s) you want to use:

    $ go get cloud.google.com/go/firestore # Replace with the package you want to use.
    

NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.

NOTE: Github repo is a mirror of https://code.googlesource.com/gocloud.

Supported APIs

Google APIStatusPackage
Assetalphacloud.google.com/go/asset/v1beta
BigQuerystablecloud.google.com/go/bigquery
Bigtablestablecloud.google.com/go/bigtable
Cloudtasksstablecloud.google.com/go/cloudtasks/apiv2
Containerstablecloud.google.com/go/container/apiv1
ContainerAnalysisbetacloud.google.com/go/containeranalysis/apiv1beta1
Dataprocstablecloud.google.com/go/dataproc/apiv1
Datastorestablecloud.google.com/go/datastore
Debuggeralphacloud.google.com/go/debugger/apiv2
Dialogflowalphacloud.google.com/go/dialogflow/apiv2
Data Loss Preventionalphacloud.google.com/go/dlp/apiv2
ErrorReportingalphacloud.google.com/go/errorreporting
Firestorestablecloud.google.com/go/firestore
IAMstablecloud.google.com/go/iam
IoTalphacloud.google.com/iot/apiv1
KMSstablecloud.google.com/go/kms
Natural Languagestablecloud.google.com/go/language/apiv1
Loggingstablecloud.google.com/go/logging
Monitoringalphacloud.google.com/go/monitoring/apiv3
OS Loginalphacloud.google.com/go/oslogin/apiv1
Pub/Substablecloud.google.com/go/pubsub
Phishing Protectionalphacloud.google.com/go/phishingprotection/apiv1betad1
reCAPTCHA Enterprisealphacloud.google.com/go/recaptchaenterprise/apiv1betad1
Memorystorealphacloud.google.com/go/redis/apiv1
Schedulerstablecloud.google.com/go/scheduler/apiv1
Spannerstablecloud.google.com/go/spanner
Speechstablecloud.google.com/go/speech/apiv1
Storagestablecloud.google.com/go/storage
Talentalphacloud.google.com/go/talent/apiv4beta1
Text To Speechalphacloud.google.com/go/texttospeech/apiv1
Tracealphacloud.google.com/go/trace/apiv2
Translatestablecloud.google.com/go/translate
Video Intelligencealphacloud.google.com/go/videointelligence/apiv1beta1
Visionstablecloud.google.com/go/vision/apiv1

Alpha status: the API is still being actively developed. As a result, it might change in backward-incompatible ways and is not recommended for production use.

Beta status: the API is largely complete, but still has outstanding features and bugs to be addressed. There may be minor backwards-incompatible changes where necessary.

Stable status: the API is mature and ready for production use. We will continue addressing bugs and feature requests.

Documentation and examples are available at godoc.org/cloud.google.com/go

Go Versions Supported

We support the two most recent major versions of Go. If Google App Engine uses an older version, we support that as well.

Authorization

By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.

client, err := storage.NewClient(ctx)

To authorize using a JSON key file, pass option.WithCredentialsFile to the NewClient function of the desired package. For example:

client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))

You can exert more control over authorization by using the golang.org/x/oauth2 package to create an oauth2.TokenSource. Then pass option.WithTokenSource to the NewClient function: snip:# (auth-ts)

tokenSource := ...
client, err := storage.NewClient(ctx, option.WithTokenSource(tokenSource))

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details. We‘re using Gerrit for our code reviews. Please don’t open pull requests against this repo, new pull requests will be automatically closed.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.