all: add per-chunk retry deadline

In https://code-review.googlesource.com/c/google-api-go-client/+/43470,
we accidentally removed a piece of logic that gave each chunk* a
deadline. This adds it back.

This CL also adds tests for retry logic, so that we don't miss it next
time.

This CL also removes contextDone(ctx context.Context) bool, since it's
only used in one place, and in that one place we do the exact same logic
right after contextDone is used.

* Note: the single for loop in the aforementioned CL is not a for loop
that only handles retries for "each chunk". That for loop handles both
retries as well as multiple chunks (which could occur by way of
incomplete transfers). When a new chunk was started, the for loop wasn't
quit: instead, the backoff and retryDeadline was re-initialized.

In this CL, that single for loop is split into an outer (per chunk) and
inner (retry for a single chunk) for loop. Subsequently the
backoff/retry reset logic is removed. This makes it more clear what's
happening.

Fixes #389

Change-Id: Ib66905efb5eaca5f9a40dccd343bff7c7d2b5aa3
Reviewed-on: https://code-review.googlesource.com/c/google-api-go-client/+/44070
Reviewed-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Tyler Bui-Palsulich <tbp@google.com>
3 files changed
tree: 32cfb9517733b25e5c67cefd24d91ff6f4be7e02
  1. abusiveexperiencereport/
  2. acceleratedmobilepageurl/
  3. accessapproval/
  4. accesscontextmanager/
  5. adexchangebuyer/
  6. adexchangebuyer2/
  7. adexchangeseller/
  8. adexperiencereport/
  9. admin/
  10. adsense/
  11. adsensehost/
  12. alertcenter/
  13. analytics/
  14. analyticsreporting/
  15. androiddeviceprovisioning/
  16. androidenterprise/
  17. androidmanagement/
  18. androidpublisher/
  19. appengine/
  20. appsactivity/
  21. appstate/
  22. bigquery/
  23. bigquerydatatransfer/
  24. bigtableadmin/
  25. binaryauthorization/
  26. blogger/
  27. books/
  28. calendar/
  29. chat/
  30. civicinfo/
  31. classroom/
  32. cloudasset/
  33. cloudbilling/
  34. cloudbuild/
  35. clouddebugger/
  36. clouderrorreporting/
  37. cloudfunctions/
  38. cloudidentity/
  39. cloudiot/
  40. cloudkms/
  41. cloudprivatecatalog/
  42. cloudprivatecatalogproducer/
  43. cloudprofiler/
  44. cloudresourcemanager/
  45. cloudscheduler/
  46. cloudsearch/
  47. cloudshell/
  48. cloudtasks/
  49. cloudtrace/
  50. commentanalyzer/
  51. composer/
  52. compute/
  53. consumersurveys/
  54. container/
  55. containeranalysis/
  56. content/
  57. customsearch/
  58. dataflow/
  59. datafusion/
  60. dataproc/
  61. datastore/
  62. deploymentmanager/
  63. dfareporting/
  64. dialogflow/
  65. digitalassetlinks/
  66. discovery/
  67. dlp/
  68. dns/
  69. docs/
  70. doubleclickbidmanager/
  71. doubleclicksearch/
  72. drive/
  73. driveactivity/
  74. examples/
  75. factchecktools/
  76. fcm/
  77. file/
  78. firebase/
  79. firebasedynamiclinks/
  80. firebasehosting/
  81. firebaseremoteconfig/
  82. firebaserules/
  83. firestore/
  84. fitness/
  85. fusiontables/
  86. games/
  87. gamesconfiguration/
  88. gamesmanagement/
  89. genomics/
  90. gensupport/
  91. gmail/
  92. google-api-go-generator/
  93. googleapi/
  94. groupsmigration/
  95. groupssettings/
  96. healthcare/
  97. iam/
  98. iamcredentials/
  99. iap/
  100. identitytoolkit/
  101. indexing/
  102. integration-tests/
  103. internal/
  104. iterator/
  105. jobs/
  106. kgsearch/
  107. language/
  108. lib/
  109. libraryagent/
  110. licensing/
  111. logging/
  112. manufacturers/
  113. mirror/
  114. ml/
  115. monitoring/
  116. oauth2/
  117. option/
  118. oslogin/
  119. pagespeedonline/
  120. partners/
  121. people/
  122. playcustomapp/
  123. playmoviespartner/
  124. plus/
  125. plusdomains/
  126. poly/
  127. prediction/
  128. proximitybeacon/
  129. pubsub/
  130. qpxexpress/
  131. redis/
  132. remotebuildexecution/
  133. replicapool/
  134. replicapoolupdater/
  135. reseller/
  136. run/
  137. runtimeconfig/
  138. safebrowsing/
  139. script/
  140. searchconsole/
  141. securitycenter/
  142. servicebroker/
  143. serviceconsumermanagement/
  144. servicecontrol/
  145. servicemanagement/
  146. servicenetworking/
  147. serviceusage/
  148. serviceuser/
  149. sheets/
  150. siteverification/
  151. slides/
  152. sourcerepo/
  153. spanner/
  154. spectrum/
  155. speech/
  156. sqladmin/
  157. storage/
  158. storagetransfer/
  159. streetviewpublish/
  160. support/
  161. surveys/
  162. tagmanager/
  163. tasks/
  164. testing/
  165. texttospeech/
  166. toolresults/
  167. tpu/
  168. tracing/
  169. translate/
  170. transport/
  171. urlshortener/
  172. vault/
  173. videointelligence/
  174. vision/
  175. webfonts/
  176. webmasters/
  177. websecurityscanner/
  178. youtube/
  179. youtubeanalytics/
  180. youtubereporting/
  181. .gitignore
  182. .hgtags
  183. api-list.json
  184. AUTHORS
  185. CHANGES.md
  186. CONTRIBUTING.md
  187. CONTRIBUTORS
  188. doc.go
  189. GettingStarted.md
  190. go.mod
  191. go.sum
  192. LICENSE
  193. NOTES
  194. README.md
  195. RELEASING.md
  196. TODO
  197. tools.go
README.md

Google APIs Client Library for Go

Getting Started

$ go get google.golang.org/api/tasks/v1
$ go get google.golang.org/api/moderator/v1
$ go get google.golang.org/api/urlshortener/v1
... etc ...

and using:

package main

import (
	"net/http"

	"google.golang.org/api/urlshortener/v1"
)

func main() {
	svc, err := urlshortener.New(http.DefaultClient)
	// ...
}

Status

GoDoc

These are auto-generated Go libraries from the Google Discovery Service's JSON description files of the available “new style” Google APIs.

Due to the auto-generated nature of this collection of libraries, complete APIs or specific versions can appear or go away without notice. As a result, you should always locally vendor any API(s) that your code relies upon.

These client libraries are officially supported by Google. However, the libraries are considered complete and are in maintenance mode. This means that we will address critical bugs and security issues but will not add any new features.

If you're working with Google Cloud Platform APIs such as Datastore or Pub/Sub, consider using the Cloud Client Libraries for Go instead. These are the new and idiomatic Go libraries targeted specifically at Google Cloud Platform Services.

The generator itself and the code it produces are beta. Some APIs are alpha/beta, and indicated as such in the import path (e.g., “google.golang.org/api/someapi/v1alpha”).

Application Default Credentials Example

Application Default Credentials provide a simplified way to obtain credentials for authenticating with Google APIs.

The Application Default Credentials authenticate as the application itself, which make them great for working with Google Cloud APIs like Storage or Datastore. They are the recommended form of authentication when building applications that run on Google Compute Engine or Google App Engine.

Default credentials are provided by the golang.org/x/oauth2/google package. To use them, add the following import:

import "golang.org/x/oauth2/google"

Some credentials types require you to specify scopes, and service entry points may not inject them. If you encounter this situation you may need to specify scopes as follows:

import (
        "context"
        "golang.org/x/oauth2/google"
        "google.golang.org/api/compute/v1"
)

func main() {
        // Use oauth2.NoContext if there isn't a good context to pass in.
        ctx := context.Background()

        client, err := google.DefaultClient(ctx, compute.ComputeScope)
        if err != nil {
                //...
        }
        computeService, err := compute.New(client)
        if err != nil {
                //...
        }
}

If you need a oauth2.TokenSource, use the DefaultTokenSource function:

ts, err := google.DefaultTokenSource(ctx, scope1, scope2, ...)
if err != nil {
        //...
}
client := oauth2.NewClient(ctx, ts)

See also: golang.org/x/oauth2/google package documentation.