A helper function to help with the common task of paging through large API results.

gar_api_page(f, page_f = function(x) x$nextLink, page_method = c("url",
  "param", "path", "body"), page_arg = NULL, body_list = NULL)

Arguments

f

a function created by gar_api_generator

page_f

A function that will extract the next page information from f(). Should return NULL if no paging is required, or the value for page_arg if it is.

page_method

Method of paging: url will fetch by changing the fetch URL; param will fetch the next page via a parameter set in page_arg; path will change a path variable set in page_arg

page_arg

If page_method="param", you need to set this to the parameter that will change for each API page.

body_list

If page_method="body", you need to set the body that will be used in each API call, including the top level parameter page_arg that will be modified by page_f

Value

A list of the API page responses, that you may need to process further into one object.

Details

The page_f function operates on the object returned from the data_parse_function of the function f

If using page_method="url" then then page_f function needs to return the URL that will fetch the next page of results. The default finds this via x$nextLink. This is the easiest to implement if available and is recommended.

If using page_method = "param", then page_f needs to extract the parameter specified in page_arg that will fetch the next page of the results, or NULL if no more pages are required. e.g. if response is x, page_f should extract the next value for the parameter of page_arg that fetches the next results. It should also return NULL if no (more) paging is necessary. See examples. Remember to add the paging argument (e.g. start-index) to the generated function too, so it can be modified.

Examples

# NOT RUN { # demos the two methods for the same function. # The example is for the Google Analytics management API, # you need to authenticate with that to run them. # paging by using nextLink that is returned in API response ga_segment_list1 <- function(){ # this URL will be modified by using the url_override argument in the generated function segs <- gar_api_generator("https://www.googleapis.com/analytics/v3/management/segments", "GET", pars_args = list("max-results"=10), data_parse_function = function(x) x) gar_api_page(segs, page_method = "url", page_f = function(x) x$nextLink) } # paging by looking for the next start-index parameter ## start by creating the function that will output the correct start-index paging_function <- function(x){ next_entry <- x$startIndex + x$itemsPerPage # we have all results e.g. 1001 > 1000 if(next_entry > x$totalResults){ return(NULL) } next_entry } ## remember to add the paging argument (start-index) to the generated function too, ## so it can be modified. ga_segment_list2 <- function(){ segs <- gar_api_generator("https://www.googleapis.com/analytics/v3/management/segments", "GET", pars_args = list("start-index" = 1, "max-results"=10), data_parse_function = function(x) x) gar_api_page(segs, page_method = "param", page_f = paging_function, page_arg = "start-index") } identical(ga_segment_list1(), ga_segment_list2()) # }