httpreplay: set request body to nil if empty

httpreplay had a bug when a non-GET request had a zero-length body.

It always reads the request body to log it, then replaces it with
```
ioutil.NopCloser(bytes.NewReader(data))
```

(See snapshotBody.) When the body is empty, this interacts with the
way the net/http package constructs the Content-Length header.  When
the http.Request.ContentLength field is non-zero, that is used as the
header value. But when it is zero, the Content-Length header is only
transmitted if Request.Body is nil. Since httpreplay set the body to
something non-nil, Content-Length was not transmitted.n

The end result is that a request made from the client with
"Content-Length: 0" would be forwarded by the proxy without the
header. This broke writing an empty object to Amazon S3.

The fix is to explicitly set the body to nil for outgoing requests.
It would be incorrect to do so in snapshotBody, because that function
is also used for responses, and a response body must never be nil.

Also:
- Fix a bug in storage.TestIntegration_BucketPolicyOnly. It was
  using time.Now, breaking replay.

- Re-record the storage integration test.

- Re-record the bigquery integration test. I recently incremented
  the httpreplay log version, and it was using an older version.

Change-Id: Ie9fb0915d2fac444ce63821f8e6167fdfbd7aa04
Reviewed-on: https://code-review.googlesource.com/c/38270
Reviewed-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jean de Klerk <deklerk@google.com>
5 files changed
tree: 90e9fc02a1679160605a3314e5edba5bad5d0f37
  1. AUTHORS
  2. CHANGES.md
  3. CODE_OF_CONDUCT.md
  4. CONTRIBUTING.md
  5. CONTRIBUTORS
  6. LICENSE
  7. README.md
  8. RELEASING.md
  9. asset/
  10. authexample_test.go
  11. bigquery/
  12. bigtable/
  13. civil/
  14. cloud.go
  15. cloudtasks/
  16. cmd/
  17. compute/
  18. container/
  19. containeranalysis/
  20. dataproc/
  21. datastore/
  22. debugger/
  23. dialogflow/
  24. dlp/
  25. errorreporting/
  26. examples_test.go
  27. expr/
  28. firestore/
  29. functions/
  30. go.mod
  31. go.sum
  32. httpreplay/
  33. iam/
  34. internal/
  35. irm/
  36. issue_template.md
  37. keys.tar.enc
  38. kms/
  39. language/
  40. license_test.go
  41. logging/
  42. longrunning/
  43. monitoring/
  44. old-news.md
  45. oslogin/
  46. profiler/
  47. pubsub/
  48. redis/
  49. regen-gapic.sh
  50. rpcreplay/
  51. run-tests.sh
  52. scheduler/
  53. securitycenter/
  54. spanner/
  55. speech/
  56. storage/
  57. talent/
  58. texttospeech/
  59. tools.go
  60. trace/
  61. translate/
  62. videointelligence/
  63. vision/
README.md

Google Cloud Client Libraries for Go

GoDoc

Go packages for Google Cloud Platform services.

import "cloud.google.com/go"

To install the packages on your system, do not clone the repo. Instead use

$ go get -u cloud.google.com/go/...

NOTE: Some of these packages are under development, and may occasionally make backwards-incompatible changes.

NOTE: Github repo is a mirror of https://code.googlesource.com/gocloud.

News

7 August 2018

As of November 1, the code in the repo will no longer support Go versions 1.8 and earlier. No one other than AppEngine users should be on those old versions, and AppEngine Standard and Flex will stop supporting new deployments with those versions on that date.

Changes have been moved to CHANGES.

Supported APIs

Google APIStatusPackage
Assetalphacloud.google.com/go/asset/v1beta
BigQuerystablecloud.google.com/go/bigquery
Bigtablestablecloud.google.com/go/bigtable
Cloudtasksbetacloud.google.com/go/cloudtasks/apiv2beta3
Containerstablecloud.google.com/go/container/apiv1
ContainerAnalysisbetacloud.google.com/go/containeranalysis/apiv1beta1
Dataprocstablecloud.google.com/go/dataproc/apiv1
Datastorestablecloud.google.com/go/datastore
Debuggeralphacloud.google.com/go/debugger/apiv2
Dialogflowalphacloud.google.com/go/dialogflow/apiv2
Data Loss Preventionalphacloud.google.com/go/dlp/apiv2
ErrorReportingalphacloud.google.com/go/errorreporting
Firestorebetacloud.google.com/go/firestore
IAMstablecloud.google.com/go/iam
KMSstablecloud.google.com/go/kms
Natural Languagestablecloud.google.com/go/language/apiv1
Loggingstablecloud.google.com/go/logging
Monitoringalphacloud.google.com/go/monitoring/apiv3
OS Loginalphacloud.google.com/compute/docs/oslogin/rest
Pub/Substablecloud.google.com/go/pubsub
Memorystorestablecloud.google.com/go/redis/apiv1beta1
Spannerstablecloud.google.com/go/spanner
Speechstablecloud.google.com/go/speech/apiv1
Storagestablecloud.google.com/go/storage
Text To Speechalphacloud.google.com/go/texttospeech/apiv1
Tracealphacloud.google.com/go/trace/apiv2
Translationstablecloud.google.com/go/translate
Video Intelligencealphacloud.google.com/go/videointelligence/apiv1beta1
Visionstablecloud.google.com/go/vision/apiv1

Alpha status: the API is still being actively developed. As a result, it might change in backward-incompatible ways and is not recommended for production use.

Beta status: the API is largely complete, but still has outstanding features and bugs to be addressed. There may be minor backwards-incompatible changes where necessary.

Stable status: the API is mature and ready for production use. We will continue addressing bugs and feature requests.

Documentation and examples are available at https://godoc.org/cloud.google.com/go

Visit or join the google-api-go-announce group for updates on these packages.

Go Versions Supported

We support the two most recent major versions of Go. If Google App Engine uses an older version, we support that as well.

Authorization

By default, each API will use Google Application Default Credentials for authorization credentials used in calling the API endpoints. This will allow your application to run in many environments without requiring explicit configuration.

client, err := storage.NewClient(ctx)

To authorize using a JSON key file, pass option.WithCredentialsFile to the NewClient function of the desired package. For example:

client, err := storage.NewClient(ctx, option.WithCredentialsFile("path/to/keyfile.json"))

You can exert more control over authorization by using the golang.org/x/oauth2 package to create an oauth2.TokenSource. Then pass option.WithTokenSource to the NewClient function: snip:# (auth-ts)

tokenSource := ...
client, err := storage.NewClient(ctx, option.WithTokenSource(tokenSource))

Cloud Datastore GoDoc

Example Usage

First create a datastore.Client to use throughout your application:

client, err := datastore.NewClient(ctx, "my-project-id")
if err != nil {
	log.Fatal(err)
}

Then use that client to interact with the API:

type Post struct {
	Title       string
	Body        string `datastore:",noindex"`
	PublishedAt time.Time
}
keys := []*datastore.Key{
	datastore.NameKey("Post", "post1", nil),
	datastore.NameKey("Post", "post2", nil),
}
posts := []*Post{
	{Title: "Post 1", Body: "...", PublishedAt: time.Now()},
	{Title: "Post 2", Body: "...", PublishedAt: time.Now()},
}
if _, err := client.PutMulti(ctx, keys, posts); err != nil {
	log.Fatal(err)
}

Cloud Storage GoDoc

Example Usage

First create a storage.Client to use throughout your application:

client, err := storage.NewClient(ctx)
if err != nil {
	log.Fatal(err)
}
// Read the object1 from bucket.
rc, err := client.Bucket("bucket").Object("object1").NewReader(ctx)
if err != nil {
	log.Fatal(err)
}
defer rc.Close()
body, err := ioutil.ReadAll(rc)
if err != nil {
	log.Fatal(err)
}

Cloud Pub/Sub GoDoc

Example Usage

First create a pubsub.Client to use throughout your application:

client, err := pubsub.NewClient(ctx, "project-id")
if err != nil {
	log.Fatal(err)
}

Then use the client to publish and subscribe:

// Publish "hello world" on topic1.
topic := client.Topic("topic1")
res := topic.Publish(ctx, &pubsub.Message{
	Data: []byte("hello world"),
})
// The publish happens asynchronously.
// Later, you can get the result from res:
...
msgID, err := res.Get(ctx)
if err != nil {
	log.Fatal(err)
}

// Use a callback to receive messages via subscription1.
sub := client.Subscription("subscription1")
err = sub.Receive(ctx, func(ctx context.Context, m *pubsub.Message) {
	fmt.Println(m.Data)
	m.Ack() // Acknowledge that we've consumed the message.
})
if err != nil {
	log.Println(err)
}

BigQuery GoDoc

Example Usage

First create a bigquery.Client to use throughout your application: snip:# (bq-1)

c, err := bigquery.NewClient(ctx, "my-project-ID")
if err != nil {
	// TODO: Handle error.
}

Then use that client to interact with the API: snip:# (bq-2)

// Construct a query.
q := c.Query(`
    SELECT year, SUM(number)
    FROM [bigquery-public-data:usa_names.usa_1910_2013]
    WHERE name = "William"
    GROUP BY year
    ORDER BY year
`)
// Execute the query.
it, err := q.Read(ctx)
if err != nil {
	// TODO: Handle error.
}
// Iterate through the results.
for {
	var values []bigquery.Value
	err := it.Next(&values)
	if err == iterator.Done {
		break
	}
	if err != nil {
		// TODO: Handle error.
	}
	fmt.Println(values)
}

Stackdriver Logging GoDoc

Example Usage

First create a logging.Client to use throughout your application: snip:# (logging-1)

ctx := context.Background()
client, err := logging.NewClient(ctx, "my-project")
if err != nil {
	// TODO: Handle error.
}

Usually, you'll want to add log entries to a buffer to be periodically flushed (automatically and asynchronously) to the Stackdriver Logging service. snip:# (logging-2)

logger := client.Logger("my-log")
logger.Log(logging.Entry{Payload: "something happened!"})

Close your client before your program exits, to flush any buffered log entries. snip:# (logging-3)

err = client.Close()
if err != nil {
	// TODO: Handle error.
}

Cloud Spanner GoDoc

Example Usage

First create a spanner.Client to use throughout your application:

client, err := spanner.NewClient(ctx, "projects/P/instances/I/databases/D")
if err != nil {
	log.Fatal(err)
}
// Simple Reads And Writes
_, err = client.Apply(ctx, []*spanner.Mutation{
	spanner.Insert("Users",
		[]string{"name", "email"},
		[]interface{}{"alice", "a@example.com"})})
if err != nil {
	log.Fatal(err)
}
row, err := client.Single().ReadRow(ctx, "Users",
	spanner.Key{"alice"}, []string{"email"})
if err != nil {
	log.Fatal(err)
}

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details. We‘re using Gerrit for our code reviews. Please don’t open pull requests against this repo, new pull requests will be automatically closed.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.