fix(bigquery/storage/managedwriter): retry improvements (#9642)

This PR makes two changes to retry behaviors in managedwriter.

In the first, this PR expands the set of conditions that trigger reconnect when sending the initial request to the backend.

In the second, this PR adds some additional handling for context cancellations when reading responses back from the service.  In cases like reconnection, we establish a new Connection, each of which has it's own associated context.  When draining remaining writes from a connection that's being shut down, we now pass the write into a retryer with a status-based error rather than raw context.Canceled, so we can recover more cleanly if the user is leveraging write retries.

Related internal issue:
b/326242484
3 files changed
tree: 68e7efb54b63d1bf293e27aa58b1864d413750a6
  1. .devcontainer/
  2. .github/
  3. accessapproval/
  4. accesscontextmanager/
  5. advisorynotifications/
  6. ai/
  7. aiplatform/
  8. alloydb/
  9. analytics/
  10. apigateway/
  11. apigeeconnect/
  12. apigeeregistry/
  13. apikeys/
  14. appengine/
  15. apphub/
  16. apps/
  17. area120/
  18. artifactregistry/
  19. asset/
  20. assuredworkloads/
  21. auth/
  22. automl/
  23. baremetalsolution/
  24. batch/
  25. beyondcorp/
  26. bigquery/
  27. bigtable/
  28. billing/
  29. binaryauthorization/
  30. certificatemanager/
  31. channel/
  32. civil/
  33. cloudbuild/
  34. cloudcontrolspartner/
  35. clouddms/
  36. cloudprofiler/
  37. cloudquotas/
  38. cloudtasks/
  39. commerce/
  40. compute/
  41. confidentialcomputing/
  42. config/
  43. contactcenterinsights/
  44. container/
  45. containeranalysis/
  46. datacatalog/
  47. dataflow/
  48. dataform/
  49. datafusion/
  50. datalabeling/
  51. dataplex/
  52. dataproc/
  53. dataqna/
  54. datastore/
  55. datastream/
  56. debugger/
  57. deploy/
  58. dialogflow/
  59. discoveryengine/
  60. dlp/
  61. documentai/
  62. domains/
  63. edgecontainer/
  64. edgenetwork/
  65. errorreporting/
  66. essentialcontacts/
  67. eventarc/
  68. filestore/
  69. firestore/
  70. functions/
  71. gkebackup/
  72. gkeconnect/
  73. gkehub/
  74. gkemulticloud/
  75. grafeas/
  76. gsuiteaddons/
  77. httpreplay/
  78. iam/
  79. iap/
  80. ids/
  81. internal/
  82. iot/
  83. kms/
  84. language/
  85. lifesciences/
  86. logging/
  87. longrunning/
  88. managedidentities/
  89. maps/
  90. mediatranslation/
  91. memcache/
  92. metastore/
  93. migrationcenter/
  94. monitoring/
  95. netapp/
  96. networkconnectivity/
  97. networkmanagement/
  98. networksecurity/
  99. notebooks/
  100. optimization/
  101. orchestration/
  102. orgpolicy/
  103. osconfig/
  104. oslogin/
  105. parallelstore/
  106. phishingprotection/
  107. policysimulator/
  108. policytroubleshooter/
  109. privatecatalog/
  110. profiler/
  111. pubsub/
  112. pubsublite/
  113. rapidmigrationassessment/
  114. recaptchaenterprise/
  115. recommendationengine/
  116. recommender/
  117. redis/
  118. resourcemanager/
  119. resourcesettings/
  120. retail/
  121. rpcreplay/
  122. run/
  123. scheduler/
  124. secretmanager/
  125. securesourcemanager/
  126. security/
  127. securitycenter/
  128. securitycentermanagement/
  129. securityposture/
  130. servicecontrol/
  131. servicedirectory/
  132. servicehealth/
  133. servicemanagement/
  134. serviceusage/
  135. shell/
  136. shopping/
  137. spanner/
  138. speech/
  139. storage/
  140. storageinsights/
  141. storagetransfer/
  142. support/
  143. talent/
  144. telcoautomation/
  145. texttospeech/
  146. third_party/
  147. tpu/
  148. trace/
  149. translate/
  150. vertexai/
  151. video/
  152. videointelligence/
  153. vision/
  154. visionai/
  155. vmmigration/
  156. vmwareengine/
  157. vpcaccess/
  158. webrisk/
  159. websecurityscanner/
  160. workflows/
  161. workstations/
  162. .gitignore
  163. .release-please-manifest-individual.json
  164. .release-please-manifest-submodules.json
  165. .release-please-manifest.json
  166. CHANGES.md
  167. CODE_OF_CONDUCT.md
  168. CONTRIBUTING.md
  169. debug.md
  170. doc.go
  171. go.mod
  172. go.sum
  173. go.work
  174. go.work.sum
  175. LICENSE
  176. migration.md
  177. README.md
  178. release-please-config-individual.json
  179. release-please-config-yoshi-submodules.json
  180. release-please-config.json
  181. RELEASING.md
  182. SECURITY.md
  183. testing.md
README.md

Google Cloud Client Libraries for Go

Go Reference

Go packages for Google Cloud Platform services.

import "cloud.google.com/go"

To install the packages on your system, do not clone the repo. Instead:

  1. Change to your project directory: cd /my/cloud/project
  2. Get the package you want to use. Some products have their own module, so it's best to go get the package(s) you want to use:
go get cloud.google.com/go/firestore # Replace with the package you want to use.

NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.

Supported APIs

For an updated list of all of our released APIs please see our reference docs.

Go Versions Supported

Our libraries are compatible with at least the three most recent, major Go releases. They are currently compatible with:

  • Go 1.21
  • Go 1.20
  • Go 1.19

Authorization

By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.

client, err := storage.NewClient(ctx)

To authorize using a JSON key file, pass option.WithCredentialsFile to the NewClient function of the desired package. For example:

client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))

You can exert more control over authorization by using the golang.org/x/oauth2 package to create an oauth2.TokenSource. Then pass option.WithTokenSource to the NewClient function:

tokenSource := ...
client, err := storage.NewClient(ctx, option.WithTokenSource(tokenSource))

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.

Links