Convenience function for walking through data in batches
gar_batch_walk( f, walk_vector, gar_pars = NULL, gar_paths = NULL, the_body = NULL, pars_walk = NULL, path_walk = NULL, body_walk = NULL, batch_size = 10, batch_function = NULL, data_frame_output = TRUE, ..., batch_endpoint = getOption("googleAuthR.batch_endpoint", default = "https://www.googleapis.com/batch") )
| f | a function from |
|---|---|
| walk_vector | a vector of the parameter or path to change |
| gar_pars | a list of parameter arguments for f |
| gar_paths | a list of path arguments for f |
| the_body | a list of body arguments for f |
| pars_walk | a character vector of the parameter(s) to modify for each walk of f |
| path_walk | a character vector of the path(s) to modify for each walk of f |
| body_walk | a character vector of the body(s) to modify for each walk of f |
| batch_size | size of each request to Google /batch API |
| batch_function | a function that will act on the result list of each batch API call |
| data_frame_output | if the list of lists are dataframes, you can bind them all by setting to TRUE |
| ... | further arguments passed to the data parse function of f |
| batch_endpoint | the batch API endpoint to send |
if data_frame_output is FALSE: A list of lists. Outer list the length of number of batches required, inner lists the results from the calls
if data_frame_output is TRUE: The list of lists will attempt to rbind all the results
You can modify more than one parameter or path arg,
but it must be the same walked vector e.g. start = end = x
Many Google APIs have batch_size limits greater than 10, 1000 is common.
The `f` function needs to be a `gar_api_generator()` function that uses one of `path_args`, `pars_args` or `body_args` to construct the URL (rather than say using `sprintf()` to create the API URL).
You don't need to set the headers in the Google docs for batching API functions - those are done for you.
The argument `walk_vector` needs to be a vector of the values of the arguments to walk over, which you indicate will walk over the pars/path or body arguments on the function via on of the `*_walk` arguments e.g. if walking over id=1, id=2, for a path argument then it would be `path_walk="id"` and `walk_vector=c(1,2,3,4)`
The `gar_*` parameter is required to pass intended for other arguments to the function `f` you may need to pass through.
`gar_batch_walk()` only supports changing one value at a time, for one or multiple arguments (I think only changing the `start-date`, `end-date` example would be the case when you walk through more than one per call)
`batch_size` should be over 1 for batching to be of any benefit at all
The `batch_function` argument gives you a way to operate on the parsed output of each call
Other batch functions:
gar_batch()
if (FALSE) { # get a webproperty per account getAccountInfo <- gar_api_generator( "https://www.googleapis.com/analytics/v3/management/accounts", "GET", data_parse_function = function(x) unique(x$items$id)) getWebpropertyInfo <- gar_api_generator( "https://www.googleapis.com/analytics/v3/management/", # don't use sprintf to construct this "GET", path_args = list(accounts = "default", webproperties = ""), data_parse_function = function(x) x$items) walkData <- function(){ # here due to R lazy evaluation accs <- getAccountInfo() gar_batch_walk(getWebpropertyInfo, walk_vector = accs, gar_paths = list("webproperties" = ""), path_walk = "accounts", batch_size = 100, data_frame_output = FALSE) } # do the walk walkData() # to walk body data, be careful to modify a top level body name: changed_emails <- lapply(email, function(x){userRef = list(email = x)}) batched <- gar_batch_walk(users, walk_vector = changed_emails, the_body = list( permissions = list( local = list(permissions) ), userRef = list( email = email[[1]] ) ), body_walk = "userRef", batch_size = 300, data_frame_output = FALSE) }