commit | b9fe3880dbb71b287cbb217f36a9e65c1e4ccb83 | [log] [tgz] |
---|---|---|
author | Jonathan Amsterdam <jba@google.com> | Wed Apr 18 22:19:26 2018 -0400 |
committer | Jonathan Amsterdam <jba@google.com> | Fri Apr 20 14:02:03 2018 +0000 |
tree | b2e6de559c22e9a3a39c0bd0556d39185e68df0d | |
parent | d8617376fb720650127a022cc3dfe39a15b54f16 [diff] |
pubsub: don't sleep on stream retry after waiting When sending or receiving from a StreamingPull stream, don't sleep on retry if we've already been blocked on the send/recv for a while. Motivation: sometimes the service will send a retryable error (like UNAVAILABLE) for a "normal" situation, like killing a stream that has been idle a while. If the stream really is idle for a long time, then we'll keep sleeping with a larger duration each time through the loop. If a message is actually available during one of the long sleeps, we won't receive it for a while. Example: say the server kills a stream that has been idle for 1m. We use the default gax.Backoff, with an initial sleep of 1s, doubling to a max of 30s. Assume the stream is idle and we backed off on every retryable error. We call Recv and block. At 1m, Recv returns with a UNAVAILABLE because the server killed it. We sleep 1s and call Recv again. After another 1m of idleness, we again get UNAVAILABLE. Now we sleep 2s. This continues until we are sleeping 30s between calls. Say a message is available 1s into a 30s sleep. We will wait 29s before calling Recv and getting the message. With this CL, we don't sleep if the call has blocked for 30s or more. So if all the errors are from idleness, we never sleep and are always ready for incoming messages. If, however, the service is really unavailable, then calls should fail very quickly, and we will back off. Fixes #949. Change-Id: I460ba5d6ab5c8f0488bb72f5799a6bce7ea3cfaa Reviewed-on: https://code-review.googlesource.com/26550 Reviewed-by: Jean de Klerk <deklerk@google.com>
Go packages for Google Cloud Platform services.
import "cloud.google.com/go"
To install the packages on your system,
$ go get -u cloud.google.com/go/...
NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.
NOTE: Github repo is a mirror of https://code.googlesource.com/gocloud.
April 9, 2018
v0.21.0
bigquery:
firestore:
spanner:
CommitTimestamp
, which supports inserting the commit timestamp of a transaction into a column.March 22, 2018
v0.20.0
bigquery: Support SchemaUpdateOptions for load jobs.
bigtable:
datastore: Add OpenCensus tracing.
firestore:
logging: Add a WriteTimeout option.
spanner: Support Batch API.
storage: Add OpenCensus tracing.
February 26, 2018
v0.19.0
bigquery:
bigtable:
datastore:
firestore:
logging:
profiler:
pubsub:
storage:
January 18, 2018
v0.18.0
bigquery:
firestore: Data provided to DocumentRef.Set with a Merge option can contain Delete sentinels.
logging: Clients can accept parent resources other than projects.
pubsub:
oslogin/apiv1beta: New client for the Cloud OS Login API.
rpcreplay: A package for recording and replaying gRPC traffic.
spanner:
storage: Clarify checksum validation for gzipped files (it is not validated when the file is served uncompressed).
December 11, 2017
v0.17.0
Remove UpdateMap and UpdateStruct; rename UpdatePaths to Update. Change docref.UpdateMap(ctx, map[string]interface{}{"a.b", 1})
to docref.Update(ctx, []firestore.Update{{Path: "a.b", Value: 1}})
Change docref.UpdateStruct(ctx, []string{"Field"}, aStruct)
to docref.Update(ctx, []firestore.Update{{Path: "Field", Value: aStruct.Field}})
Rename MergePaths to Merge; require args to be FieldPaths
A value stored as an integer can be read into a floating-point field, and vice versa.
Alpha status: the API is still being actively developed. As a result, it might change in backward-incompatible ways and is not recommended for production use.
Beta status: the API is largely complete, but still has outstanding features and bugs to be addressed. There may be minor backwards-incompatible changes where necessary.
Stable status: the API is mature and ready for production use. We will continue addressing bugs and feature requests.
Documentation and examples are available at https://godoc.org/cloud.google.com/go
Visit or join the google-api-go-announce group for updates on these packages.
We support the two most recent major versions of Go. If Google App Engine uses an older version, we support that as well. You can see which versions are currently supported by looking at the lines following go:
in .travis.yml
.
By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.
client, err := storage.NewClient(ctx)
To authorize using a JSON key file, pass option.WithServiceAccountFile
to the NewClient
function of the desired package. For example:
client, err := storage.NewClient(ctx, option.WithServiceAccountFile("path/to/keyfile.json"))
You can exert more control over authorization by using the golang.org/x/oauth2
package to create an oauth2.TokenSource
. Then pass option.WithTokenSource
to the NewClient
function: snip:# (auth-ts)
tokenSource := ...
client, err := storage.NewClient(ctx, option.WithTokenSource(tokenSource))
First create a datastore.Client
to use throughout your application:
client, err := datastore.NewClient(ctx, "my-project-id")
if err != nil {
log.Fatal(err)
}
Then use that client to interact with the API:
type Post struct {
Title string
Body string `datastore:",noindex"`
PublishedAt time.Time
}
keys := []*datastore.Key{
datastore.NameKey("Post", "post1", nil),
datastore.NameKey("Post", "post2", nil),
}
posts := []*Post{
{Title: "Post 1", Body: "...", PublishedAt: time.Now()},
{Title: "Post 2", Body: "...", PublishedAt: time.Now()},
}
if _, err := client.PutMulti(ctx, keys, posts); err != nil {
log.Fatal(err)
}
First create a storage.Client
to use throughout your application:
client, err := storage.NewClient(ctx)
if err != nil {
log.Fatal(err)
}
// Read the object1 from bucket. rc, err := client.Bucket("bucket").Object("object1").NewReader(ctx) if err != nil { log.Fatal(err) } defer rc.Close() body, err := ioutil.ReadAll(rc) if err != nil { log.Fatal(err) }
First create a pubsub.Client
to use throughout your application:
client, err := pubsub.NewClient(ctx, "project-id")
if err != nil {
log.Fatal(err)
}
Then use the client to publish and subscribe:
// Publish "hello world" on topic1. topic := client.Topic("topic1") res := topic.Publish(ctx, &pubsub.Message{ Data: []byte("hello world"), }) // The publish happens asynchronously. // Later, you can get the result from res: ... msgID, err := res.Get(ctx) if err != nil { log.Fatal(err) } // Use a callback to receive messages via subscription1. sub := client.Subscription("subscription1") err = sub.Receive(ctx, func(ctx context.Context, m *pubsub.Message) { fmt.Println(m.Data) m.Ack() // Acknowledge that we've consumed the message. }) if err != nil { log.Println(err) }
First create a bigquery.Client
to use throughout your application: snip:# (bq-1)
c, err := bigquery.NewClient(ctx, "my-project-ID") if err != nil { // TODO: Handle error. }
Then use that client to interact with the API: snip:# (bq-2)
// Construct a query. q := c.Query(` SELECT year, SUM(number) FROM [bigquery-public-data:usa_names.usa_1910_2013] WHERE name = "William" GROUP BY year ORDER BY year `) // Execute the query. it, err := q.Read(ctx) if err != nil { // TODO: Handle error. } // Iterate through the results. for { var values []bigquery.Value err := it.Next(&values) if err == iterator.Done { break } if err != nil { // TODO: Handle error. } fmt.Println(values) }
First create a logging.Client
to use throughout your application: snip:# (logging-1)
ctx := context.Background() client, err := logging.NewClient(ctx, "my-project") if err != nil { // TODO: Handle error. }
Usually, you'll want to add log entries to a buffer to be periodically flushed (automatically and asynchronously) to the Stackdriver Logging service. snip:# (logging-2)
logger := client.Logger("my-log")
logger.Log(logging.Entry{Payload: "something happened!"})
Close your client before your program exits, to flush any buffered log entries. snip:# (logging-3)
err = client.Close() if err != nil { // TODO: Handle error. }
First create a spanner.Client
to use throughout your application:
client, err := spanner.NewClient(ctx, "projects/P/instances/I/databases/D")
if err != nil {
log.Fatal(err)
}
// Simple Reads And Writes _, err = client.Apply(ctx, []*spanner.Mutation{ spanner.Insert("Users", []string{"name", "email"}, []interface{}{"alice", "a@example.com"})}) if err != nil { log.Fatal(err) } row, err := client.Single().ReadRow(ctx, "Users", spanner.Key{"alice"}, []string{"email"}) if err != nil { log.Fatal(err) }
Contributions are welcome. Please, see the CONTRIBUTING document for details. We‘re using Gerrit for our code reviews. Please don’t open pull requests against this repo, new pull requests will be automatically closed.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.