top

Package storage contains a Google Cloud Storage client.

This package is experimental and may make backwards-incompatible changes.

Package storage provides an easy way to work with Google Cloud Storage. Google Cloud Storage stores data in named objects, which are grouped into buckets.

More information about Google Cloud Storage is available at https://cloud.google.com/storage/docs.

All of the methods of this package use exponential backoff to retry calls that fail with certain errors, as described in https://cloud.google.com/storage/docs/exponential-backoff.

Note: This package is in beta. Some backwards-incompatible changes may occur.

Creating a Client

To start working with this package, create a client:

ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
    // TODO: Handle error.
}

Buckets

A Google Cloud Storage bucket is a collection of objects. To work with a bucket, make a bucket handle:

bkt := client.Bucket(bucketName)

A handle is a reference to a bucket. You can have a handle even if the bucket doesn't exist yet. To create a bucket in Google Cloud Storage, call Create on the handle:

if err := bkt.Create(ctx, projectID, nil); err != nil {
    // TODO: Handle error.
}

Note that although buckets are associated with projects, bucket names are global across all projects.

Each bucket has associated metadata, represented in this package by BucketAttrs. The third argument to BucketHandle.Create allows you to set the intial BucketAttrs of a bucket. To retrieve a bucket's attributes, use Attrs:

attrs, err := bkt.Attrs(ctx)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("bucket %s, created at %s, is located in %s with storage class %s\n",
    attrs.Name, attrs.Created, attrs.Location, attrs.StorageClass)

Objects

An object holds arbitrary data as a sequence of bytes, like a file. You refer to objects using a handle, just as with buckets. You can use the standard Go io.Reader and io.Writer interfaces to read and write object data:

obj := bkt.Object("data")
// Write something to obj.
// w implements io.Writer.
w := obj.NewWriter(ctx)
// Write some text to obj. This will overwrite whatever is there.
if _, err := fmt.Fprintf(w, "This object contains text.\n"); err != nil {
    // TODO: Handle error.
}
// Close, just like writing a file.
if err := w.Close(); err != nil {
    // TODO: Handle error.
}

// Read it back.
r, err := obj.NewReader(ctx)
if err != nil {
    // TODO: Handle error.
}
defer r.Close()
if _, err := io.Copy(os.Stdout, r); err != nil {
    // TODO: Handle error.
}
// Prints "This object contains text."

Objects also have attributes, which you can fetch with Attrs:

objAttrs, err := obj.Attrs(ctx)
if err != nil {
    // TODO: Handle error.
}
fmt.Printf("object %s has size %d and can be read using %s\n",
    objAttrs.Name, objAttrs.Size, objAttrs.MediaLink)

ACLs

Both objects and buckets have ACLs (Access Control Lists). An ACL is a list of ACLRules, each of which specifies the role of a user, group or project. ACLs are suitable for fine-grained control, but you may prefer using IAM to control access at the project level (see https://cloud.google.com/storage/docs/access-control/iam).

To list the ACLs of a bucket or object, obtain an ACLHandle and call its List method:

acls, err := obj.ACL().List(ctx)
if err != nil {
    // TODO: Handle error.
}
for _, rule := range acls {
    fmt.Printf("%s has role %s\n", rule.Entity, rule.Role)
}

You can also set and delete ACLs.

Conditions

Every object has a generation and a metageneration. The generation changes whenever the content changes, and the metageneration changes whenever the metadata changes. Conditions let you check these values before an operation; the operation only executes if the conditions match. You can use conditions to prevent race conditions in read-modify-write operations.

For example, say you've read an object's metadata into objAttrs. Now you want to write to that object, but only if its contents haven't changed since you read it. Here is how to express that:

w = obj.If(storage.Conditions{GenerationMatch: objAttrs.Generation}).NewWriter(ctx)
// Proceed with writing as above.

Signed URLs

You can obtain a URL that lets anyone read or write an object for a limited time. You don't need to create a client to do this. See the documentation of SignedURL for details.

url, err := storage.SignedURL(bucketName, "shared-object", opts)
if err != nil {
    // TODO: Handle error.
}
fmt.Println(url)

Imported by 78 package(s)

  1. camlistore.org/cmd/camtool
  2. camlistore.org/misc
  3. camlistore.org/misc/docker
  4. camlistore.org/pkg/blobserver/google/cloudstorage
  5. camlistore.org/pkg/deploy/gce
  6. camlistore.org/website
  7. cloud.google.com/go/examples/storage/appengine
  8. cloud.google.com/go/examples/storage/appenginevm
  9. github.com/FiloSottile/camlistore/cmd/camtool
  10. github.com/FiloSottile/camlistore/misc/docker
  11. github.com/FiloSottile/camlistore/pkg/deploy/gce
  12. github.com/FiloSottile/camlistore/website
  13. github.com/GoogleCloudPlatform/golang-samples/docs/managed_vms/storage
  14. github.com/GoogleCloudPlatform/golang-samples/getting-started/bookshelf
  15. github.com/GoogleCloudPlatform/golang-samples/getting-started/bookshelf/app
  16. github.com/Nitecon/gcloud-golang/examples/storage/appengine
  17. github.com/Nitecon/gcloud-golang/examples/storage/appenginevm
  18. github.com/areed/convertiff
  19. github.com/badgerodon/stack/storage
  20. github.com/broady/cdbuild
  21. github.com/builderscon/octav/octav/service
  22. github.com/camlistore/camlistore/pkg/deploy/gce
  23. github.com/camlistore/go4/cloud/cloudlaunch
  24. github.com/camlistore/go4/cloud/google/gcsutil
  25. github.com/camlistore/go4/wkfs/gcs
  26. github.com/codegp/cloud-persister
  27. github.com/flynn/flynn/blobstore/backend
  28. github.com/gebv/storage
  29. github.com/goadesign/swagger-service
  30. github.com/google/skia-buildbot/go/gs
  31. github.com/google/skia-buildbot/go/ingestion
  32. github.com/google/skia-buildbot/go/isolate
  33. github.com/google/skia-buildbot/golden/go/diffstore
  34. github.com/google/skia-buildbot/golden/go/pdfingestion
  35. github.com/google/syzkaller/syz-gce
  36. github.com/jba/google-cloud-go/examples/storage/appengine
  37. github.com/jba/google-cloud-go/examples/storage/appenginevm
  38. github.com/kfl/file-transit
  39. github.com/liangx8/gcloud-helper/gcs
  40. github.com/liangx8/goapp_cms/web
  41. github.com/luci/luci-go/common/gcloud/gs
  42. github.com/luci/luci-go/logdog/appengine/coordinator
  43. github.com/lytics/cloudstorage
  44. github.com/lytics/cloudstorage/storeutils
  45. github.com/mjrmyke/CSCI150Project
  46. github.com/nlfiedler/akashita
  47. github.com/nyaxt/otaru/gcloud/auth
  48. github.com/raphael/goa-swagger
  49. github.com/sajari/storage
  50. github.com/tchaik/tchaik/store
  51. github.com/tortuoise/blackout
  52. github.com/tortuoise/fiandsri
  53. github.com/tortuoise/golang-samples/getting-started/bookshelf
  54. github.com/xairy/syzkaller/syz-gce
  55. github.com/youtube/vitess/go/vt/mysqlctl/gcsbackupstorage
  56. go.skia.org/infra/go/gcs
  57. go.skia.org/infra/go/gs
  58. go.skia.org/infra/go/ingestion
  59. go.skia.org/infra/go/isolate
  60. go.skia.org/infra/go/mockgcsclient
  61. go.skia.org/infra/golden/go/diffstore
  62. go.skia.org/infra/golden/go/pdfingestion
  63. go4.org/cloud/cloudlaunch
  64. go4.org/cloud/google/gcsutil
  65. go4.org/wkfs/gcs
  66. golang.org/x/build/cmd/coordinator
  67. golang.org/x/build/cmd/release
  68. golang.org/x/build/cmd/upload
  69. golang.org/x/perf/storage/fs/gcs
  70. google.golang.org/cloud/examples/storage/appengine
  71. google.golang.org/cloud/examples/storage/appenginevm
  72. kythe.io/kythe/go/platform/vfs/gcs
  73. skia.googlesource.com/buildbot.git/go/gcs
  74. skia.googlesource.com/buildbot.git/go/gs
  75. skia.googlesource.com/buildbot.git/go/ingestion
  76. skia.googlesource.com/buildbot.git/go/isolate
  77. skia.googlesource.com/buildbot.git/go/mockgcsclient
  78. tchaik.com/store

Imported only in test by 13 package(s)

  1. cloud.google.com/go/logging/logadmin
  2. cloud.google.com/go/preview/logging
  3. cloud.google.com/go/preview/logging/logadmin
  4. cloud.google.com/go/trace
  5. github.com/GoogleCloudPlatform/gcloud-golang/storage
  6. github.com/GoogleCloudPlatform/gcloud-golang/trace
  7. github.com/Nitecon/gcloud-golang/storage
  8. github.com/Nitecon/gcloud-golang/trace
  9. github.com/jba/google-cloud-go/preview/logging
  10. github.com/jba/google-cloud-go/storage
  11. github.com/jba/google-cloud-go/trace
  12. github.com/tortuoise/gcloud-golang/storage
  13. github.com/tortuoise/gcloud-golang/trace

Imports 9 package(s)

  1. cloud.google.com/go/internal/optional
  2. github.com/googleapis/gax-go
  3. golang.org/x/net/context
  4. google.golang.org/api/storage/v1
  5. cloud.google.com/go/internal
  6. google.golang.org/api/googleapi
  7. google.golang.org/api/option
  8. google.golang.org/api/transport
  9. google.golang.org/api/iterator

Test imports 2 package(s)

  1. cloud.google.com/go/internal/testutil
  2. google.golang.org/api/iterator/testing