top

Package storage contains a Google Cloud Storage client.

This package is experimental and may make backwards-incompatible changes.

Package storage provides an easy way to work with Google Cloud Storage. Google Cloud Storage stores data in named objects, which are grouped into buckets.

More information about Google Cloud Storage is available at https://cloud.google.com/storage/docs.

All of the methods of this package use exponential backoff to retry calls that fail with certain errors, as described in https://cloud.google.com/storage/docs/exponential-backoff.

Note: This package is in beta. Some backwards-incompatible changes may occur.

Creating a Client

To start working with this package, create a client:

ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
    // TODO: Handle error.
}

Buckets

A Google Cloud Storage bucket is a collection of objects. To work with a bucket, make a bucket handle:

bkt := client.Bucket(bucketName)

A handle is a reference to a bucket. You can have a handle even if the bucket doesn't exist yet. To create a bucket in Google Cloud Storage, call Create on the handle:

if err := bkt.Create(ctx, projectID, nil); err != nil {
    // TODO: Handle error.
}

Note that although buckets are associated with projects, bucket names are global across all projects.

Each bucket has associated metadata, represented in this package by BucketAttrs. The third argument to BucketHandle.Create allows you to set the intial BucketAttrs of a bucket. To retrieve a bucket's attributes, use Attrs:

attrs, err := bkt.Attrs(ctx)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("bucket %s, created at %s, is located in %s with storage class %s\n",
    attrs.Name, attrs.Created, attrs.Location, attrs.StorageClass)

Objects

An object holds arbitrary data as a sequence of bytes, like a file. You refer to objects using a handle, just as with buckets. You can use the standard Go io.Reader and io.Writer interfaces to read and write object data:

obj := bkt.Object("data")
// Write something to obj.
// w implements io.Writer.
w := obj.NewWriter(ctx)
// Write some text to obj. This will overwrite whatever is there.
if _, err := fmt.Fprintf(w, "This object contains text.\n"); err != nil {
    // TODO: Handle error.
}
// Close, just like writing a file.
if err := w.Close(); err != nil {
    // TODO: Handle error.
}

// Read it back.
r, err := obj.NewReader(ctx)
if err != nil {
    // TODO: Handle error.
}
defer r.Close()
if _, err := io.Copy(os.Stdout, r); err != nil {
    // TODO: Handle error.
}
// Prints "This object contains text."

Objects also have attributes, which you can fetch with Attrs:

objAttrs, err := obj.Attrs(ctx)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("object %s has size %d and can be read using %s\n",
    objAttrs.Name, objAttrs.Size, objAttrs.MediaLink)

ACLs

Both objects and buckets have ACLs (Access Control Lists). An ACL is a list of ACLRules, each of which specifies the role of a user, group or project. ACLs are suitable for fine-grained control, but you may prefer using IAM to control access at the project level (see https://cloud.google.com/storage/docs/access-control/iam).

To list the ACLs of a bucket or object, obtain an ACLHandle and call its List method:

acls, err := obj.ACL().List(ctx)
if err != nil {
    // TODO: Handle error.
}
for _, rule := range acls {
    fmt.Printf("%s has role %s\n", rule.Entity, rule.Role)
}

You can also set and delete ACLs.

Conditions

Every object has a generation and a metageneration. The generation changes whenever the content changes, and the metageneration changes whenever the metadata changes. Conditions let you check these values before an operation; the operation only executes if the conditions match. You can use conditions to prevent race conditions in read-modify-write operations.

For example, say you've read an object's metadata into objAttrs. Now you want to write to that object, but only if its contents haven't changed since you read it. Here is how to express that:

w = obj.If(storage.Conditions{GenerationMatch: objAttrs.Generation}).NewWriter(ctx)
// Proceed with writing as above.

Signed URLs

You can obtain a URL that lets anyone read or write an object for a limited time. You don't need to create a client to do this. See the documentation of SignedURL for details.

url, err := storage.SignedURL(bucketName, "shared-object", opts)
if err != nil {
    // TODO: Handle error.
}
fmt.Println(url)

Imported by 91 package(s)

  1. camlistore.org/cmd/camtool
  2. camlistore.org/misc
  3. camlistore.org/misc/docker
  4. camlistore.org/pkg/blobserver/google/cloudstorage
  5. camlistore.org/pkg/deploy/gce
  6. camlistore.org/website
  7. cloud.google.com/go/examples/storage/appengine
  8. cloud.google.com/go/examples/storage/appenginevm
  9. github.com/30x/haystack/storage
  10. github.com/30x/haystack/test
  11. github.com/FiloSottile/camlistore/cmd/camtool
  12. github.com/FiloSottile/camlistore/misc/docker
  13. github.com/FiloSottile/camlistore/pkg/blobserver/google/cloudstorage
  14. github.com/FiloSottile/camlistore/pkg/deploy/gce
  15. github.com/FiloSottile/camlistore/website
  16. github.com/GoogleCloudPlatform/golang-samples/docs/managed_vms/storage
  17. github.com/GoogleCloudPlatform/golang-samples/getting-started/bookshelf
  18. github.com/GoogleCloudPlatform/golang-samples/getting-started/bookshelf/app
  19. github.com/Nitecon/gcloud-golang/examples/storage/appengine
  20. github.com/Nitecon/gcloud-golang/examples/storage/appenginevm
  21. github.com/acoshift/acourse
  22. github.com/acoshift/acourse/app
  23. github.com/areed/convertiff
  24. github.com/badgerodon/stack/storage
  25. github.com/broady/cdbuild
  26. github.com/builderscon/octav/octav/service
  27. github.com/camlistore/camlistore/pkg/deploy/gce
  28. github.com/camlistore/go4/cloud/cloudlaunch
  29. github.com/camlistore/go4/cloud/google/gcsutil
  30. github.com/camlistore/go4/wkfs/gcs
  31. github.com/codegp/cloud-persister
  32. github.com/flynn/flynn/blobstore/backend
  33. github.com/gebv/storage
  34. github.com/goadesign/swagger-service
  35. github.com/google/skia-buildbot/go/gs
  36. github.com/google/skia-buildbot/go/ingestion
  37. github.com/google/skia-buildbot/go/isolate
  38. github.com/google/skia-buildbot/golden/go/diffstore
  39. github.com/google/skia-buildbot/golden/go/pdfingestion
  40. github.com/google/syzkaller/syz-gce
  41. github.com/jba/google-cloud-go/examples/storage/appengine
  42. github.com/jba/google-cloud-go/examples/storage/appenginevm
  43. github.com/kfl/file-transit
  44. github.com/liangx8/gcloud-helper/gcs
  45. github.com/liangx8/goapp_cms/web
  46. github.com/luci/luci-go/common/gcloud/gs
  47. github.com/luci/luci-go/logdog/appengine/coordinator
  48. github.com/luci/luci-go/logdog/appengine/coordinator/coordinatorTest
  49. github.com/lytics/cloudstorage
  50. github.com/lytics/cloudstorage/storeutils
  51. github.com/mattharden/vitess/go/vt/mysqlctl/gcsbackupstorage
  52. github.com/mjrmyke/CSCI150Project
  53. github.com/mpl/camlistore/pkg/blobserver/google/cloudstorage
  54. github.com/nlfiedler/akashita
  55. github.com/nyaxt/otaru/gcloud/auth
  56. github.com/raphael/goa-swagger
  57. github.com/sajari/storage
  58. github.com/tchaik/tchaik/store
  59. github.com/tebeka/selenium/vendor
  60. github.com/tortuoise/blackout
  61. github.com/tortuoise/fiandsri
  62. github.com/tortuoise/gcloud-golang/examples/storage/appengine
  63. github.com/tortuoise/gcloud-golang/examples/storage/appenginevm
  64. github.com/tortuoise/golang-samples/docs/managed_vms/storage
  65. github.com/tortuoise/golang-samples/getting-started/bookshelf
  66. github.com/tortuoise/golang-samples/getting-started/bookshelf/app
  67. github.com/xairy/syzkaller/syz-gce
  68. github.com/youtube/vitess/go/vt/mysqlctl/gcsbackupstorage
  69. go.skia.org/infra/go/gcs
  70. go.skia.org/infra/go/gs
  71. go.skia.org/infra/go/ingestion
  72. go.skia.org/infra/go/isolate
  73. go.skia.org/infra/go/mockgcsclient
  74. go.skia.org/infra/golden/go/diffstore
  75. go.skia.org/infra/golden/go/pdfingestion
  76. go4.org/cloud/cloudlaunch
  77. go4.org/cloud/google/gcsutil
  78. go4.org/wkfs/gcs
  79. golang.org/x/build/cmd/coordinator
  80. golang.org/x/build/cmd/release
  81. golang.org/x/build/cmd/upload
  82. golang.org/x/perf/storage/fs/gcs
  83. google.golang.org/cloud/examples/storage/appengine
  84. google.golang.org/cloud/examples/storage/appenginevm
  85. kythe.io/kythe/go/platform/vfs/gcs
  86. skia.googlesource.com/buildbot.git/go/gcs
  87. skia.googlesource.com/buildbot.git/go/gs
  88. skia.googlesource.com/buildbot.git/go/ingestion
  89. skia.googlesource.com/buildbot.git/go/isolate
  90. skia.googlesource.com/buildbot.git/go/mockgcsclient
  91. tchaik.com/store

Imported only in test by 15 package(s)

  1. cloud.google.com/go/logging/logadmin
  2. cloud.google.com/go/preview/logging
  3. cloud.google.com/go/preview/logging/logadmin
  4. cloud.google.com/go/trace
  5. github.com/GoogleCloudPlatform/gcloud-golang/storage
  6. github.com/GoogleCloudPlatform/gcloud-golang/trace
  7. github.com/Nitecon/gcloud-golang/storage
  8. github.com/Nitecon/gcloud-golang/trace
  9. github.com/aseure/gcloud-golang/storage
  10. github.com/aseure/gcloud-golang/trace
  11. github.com/jba/google-cloud-go/preview/logging
  12. github.com/jba/google-cloud-go/storage
  13. github.com/jba/google-cloud-go/trace
  14. github.com/tortuoise/gcloud-golang/storage
  15. github.com/tortuoise/gcloud-golang/trace

Imports 9 package(s)

  1. cloud.google.com/go/internal/optional
  2. github.com/googleapis/gax-go
  3. golang.org/x/net/context
  4. google.golang.org/api/storage/v1
  5. cloud.google.com/go/internal
  6. google.golang.org/api/googleapi
  7. google.golang.org/api/option
  8. google.golang.org/api/transport
  9. google.golang.org/api/iterator

Test imports 2 package(s)

  1. cloud.google.com/go/internal/testutil
  2. google.golang.org/api/iterator/testing