CCurlWithPagination

Description

Makes one or more REST API calls and returns the responses received from all the pages requested from the server. Requests are authenticated using an Integrate.io ETL connection.

Syntax

CCurlWithPagination(url, method[, headers[, request_body[, connection_id[, pagination_scheme[, sleep_interval[, max_pages]]]]]])

Arguments

url - HTTP or HTTPS url of the server.

method - HTTP method, such as GET, POST, PUT, DELETE.

headers (optional) - A json string that identifies the header key and value in the following manner: '{"header1" : "value1" , "header2" : "value2"}'

request_body (optional) - The string body of the request (most relevant in the case of POST)

connection_id (optional string) - The unique identifier of the connection to use to execute the query. You can find it in the connection's page.

pagination_scheme (optional) - The pagination method to use, see options below. If empty, the function will try to detect the pagination method according to the url's domain. See the supported domains here.

sleep_interval (optional) - number of milliseconds to wait between requests. Default is 0.

max_pages (optional) - stop after X number of pages. Default is unlimited, 0 or 1 means single page call.

Examples

CCurlWithPagination('https://api.github.com/repos/integrate.io ETL/integrate.io ETL-api-doc-v2/events','GET','','','gitub_86')

Return value datatype

The returned value is a bag of map objects that contains the following keys:

  • status - response code (integer)
  • body - the body of the response (string)
  • headers - map object of response headers

Notes

Supported pagination methods are:

  • Automatic - default, same as null or empty string. Detects pagination method by domain.
  • NoPagination - Simply make one request, don't follow pagination.
  • LinkHeader - Conforms with RFC 5988.

Impact of null value

The url and method arguments are required. The rest are optional and default to null.