An R library for interacting with the Google Cloud Storage JSON API (api docs).

Setup

Google Cloud Storage charges you for storage (prices here).

You can use your own Google Project with a credit card added to create buckets, where the charges will apply. This can be done in the Google API Console

Configuring your own Google Project

The instructions below are for when you visit the Google API console (https://console.developers.google.com/apis/)

Activate API

  1. Click on “APIs”
  2. Select and activate the Cloud Storage JSON API if not already active

Set environment variables

By default, all cloudyr packages look for the access key ID and secret access key in environment variables. You can also use this to specify a default bucket, and auto-authentication upon attaching the library. For example:

Sys.setenv("GCS_DEFAULT_BUCKET" = "my-default-bucket",
           "GCS_AUTH_FILE" = "/fullpath/to/service-auth.json")

These can alternatively be set on the command line or via an Renviron.site or .Renviron file (https://cran.r-project.org/web/packages/httr/vignettes/api-packages.html).

e.g.

In your .Renviron:

GCS_AUTH_FILE="/fullpath/to/service-auth.json"
GCS_DEFAULT_BUCKET=my-default-bucket

Auto-authentication

The best method for authentication is to use your own Google Cloud Project. You can specify the location of a service account JSON file taken from your Google Project:

    Sys.setenv("GCS_AUTH_FILE" = "/fullpath/to/auth.json")

This file will then used for authentication via gcs_auth() when you load the library:

Setting a default Bucket

To avoid specifying the bucket in the functions below, you can set the name of your default bucket via environmental variables or via the function gcs_global_bucket(). See the Setting environment variables section for more details.

Uploading objects < 5MB

Objects can be uploaded via files saved to disk, or passed in directly if they are data frames or list type R objects. By default, data frames will be converted to CSV via write.csv(), lists to JSON via jsonlite::toJSON.

If you want to use other functions for transforming R objects, for example setting row.names = FALSE or using write.csv2, pass the function through object_function

Upload metadata

You can pass metadata with an object via the function gcs_metadata_object().

the name you pass to the metadata object will override the name if it is also set elsewhere.

Updating user access to objects

You can change who can access objects via gcs_update_acl to one of READER or OWNER, on a user, group, domain, project or public for all users or authenticated users.

By default you are “OWNER” of all the objects and buckets you upload and create.

Deleting an object

Delete an object by passing its name (and bucket if not default)

R Session helpers

Versions of save.image(), save() and load() are implemented called gcs_save_image(), gcs_save() and gcs_load(). These functions save and load the global R session to the cloud.

Save specific objects:

You can also upload .R code files and source them directly using gcs_source:

Uploading via a Shiny app

The library is also compatible with Shiny authentication flows, so you can create Shiny apps that lets users log in and upload their own data.

An example of that is shown below:

Bucket administration

There are various functions to manipulate Buckets:

  • gcs_list_buckets
  • gcs_get_bucket
  • gcs_create_bucket

Object administration

You can get meta data about an object by passing meta=TRUE to gcs_get_object

gcs_get_object("your-object", "your-bucket", meta = TRUE)

Explanation of Google Project access

googleCloudStorageR has its own Google project which is used to call the Google Cloud Storage API, but does not have access to the objects or buckets in your Google Project unless you give permission for the library to access your own buckets during the OAuth2 authentication process.

No other user, including the owner of the Google Cloud Storage API project has access unless you have given them access, but you may want to change to use your own Google Project (that could or could not be the same as the one that holds your buckets).