Pagination

Some RippleNet API operations paginate their responses to make the result set easier to handle. For example, if you request a list of objects that is potentially too large to handle efficiently, the operation returns the first batch of results along with a marker that you use to access the next batch of results. In RippleNet this marker is a URL parameter, page, indicating the starting point.

For example, if you call GET /v4/transfers, the default response includes 5 transfers (the default size) on page=0 (the default page). If there are more transfers to fetch (indicated by last=false in the API response), fetch the next set of transfers by using the same endpoint with page=1.

We recommend that you fetch all the results on all pages before processing them. If you paginate over a moving list, you may miss some results.

Pagination flowchart

1. Create the request

Choose the data you need to retrieve (payments, transfers, etc.). The HTTP method for all the data types is GET.

For example:

Copy

Copied!

GET {ripplenet_base_url}/v4/transfers

Authorization: Bearer eyJhbGciO...l0ZSIwIm

To use pagination, add the page parameter to the request. Its default value is 0.

Your request becomes:

Copy

Copied!

GET {ripplenet_base_url}/v4/transfers?page=X

Authorization: Bearer eyJhbGciO...l0ZSIwIm

Logging checklist

Action	Completed
The full request

Pseudocode

Action	Completed
Choose the data to retrieve (payments, transfers...)
Include the `page` parameter in the request

2. Read the response

Before processing the data you received with page=0, read and analyze the response.

This is a sample response from GET /v4/transfers?page=0&size=5:

Click to show complete response

Copy

Copied!

    "first": true,
    "last": false,
    "number": "0",
    "numberOfElements": "5",
    "size": "5",
    "totalElements": "18",
    "totalPages": "4",
    "sort": [
        {
            "direction": "DESC",
            "property": "lastModifiedDate",
            "ignoreCase": false,
            "nullHandling": "NATIVE",
            "ascending": false,
            "descending": true
        }
    ],
    "content": [
        {
            "transfer_id": "59d96a63-1c41-43ab-af10-7bf961d73457",
            "state": "EXECUTED",
            "owner_address": "test.cloud.blueprint2",
            "sender_address": "trans_eur_receiver@test.cloud.blueprint2",
            "receiver_address": "conct_eur_sender@test.cloud.blueprint2",
            "amount": "1000000000.000000000",
            "internal_id": null,
            "end_to_end_id": null,
            "sending_fee": null,
            "created_at": "2021-02-16T08:20:59.842Z",
            "user_info": [],
            "labels": []
        },
        {
            "transfer_id": "2e6f1167-e91a-43dd-be81-716088884549",
            "state": "EXECUTED",
            "owner_address": "test.cloud.blueprint2",
            "sender_address": "conct_eur_sender@test.cloud.blueprint2",
            "receiver_address": "trans_eur_receiver@test.cloud.blueprint2",
            "amount": "900000.000000000",
            "internal_id": null,
            "end_to_end_id": null,
            "sending_fee": null,
            "created_at": "2021-02-16T07:29:35.036Z",
            "user_info": [],
            "labels": []
        },
        {
            "transfer_id": "89b6668c-7a81-485b-99b4-bc924f29140a",
            "state": "EXECUTED",
            "owner_address": "test.cloud.blueprint2",
            "sender_address": "conct_eur_sender@test.cloud.blueprint2",
            "receiver_address": "trans_eur_receiver@test.cloud.blueprint2",
            "amount": "99999000000.000000000",
            "internal_id": null,
            "end_to_end_id": null,
            "sending_fee": null,
            "created_at": "2021-02-16T07:29:07.781Z",
            "user_info": [],
            "labels": []
        },
        {
            "transfer_id": "594ceb52-4db1-4bbd-9956-141e7d3f032a",
            "state": "COMPLETED",
            "owner_address": "test.cloud.blueprint1",
            "sender_address": "trans_usd_sender@test.cloud.blueprint1",
            "receiver_address": "conct_usd_sender@test.cloud.blueprint1",
            "amount": "3.000000000",
            "internal_id": "456",
            "end_to_end_id": "123",
            "sending_fee": null,
            "created_at": "2021-01-04T09:33:35.105Z",
            "user_info": [
                {
                    "node_address": "test.cloud.blueprint1",
                    "state": "EXECUTED",
                    "json": {},
                    "created_at": "2021-01-04T09:33:35.106Z"
                }
            ],
            "labels": []
        },
        {
            "transfer_id": "827da53b-8086-4d1b-8d0f-d8d18ac46051",
            "state": "COMPLETED",
            "owner_address": "test.cloud.blueprint1",
            "sender_address": "trans_usd_sender@test.cloud.blueprint1",
            "receiver_address": "conct_usd_sender@test.cloud.blueprint1",
            "amount": "3.000000000",
            "internal_id": "456",
            "end_to_end_id": "123",
            "sending_fee": null,
            "created_at": "2020-12-02T13:52:40.909Z",
            "user_info": [
                {
                    "node_address": "test.cloud.blueprint1",
                    "state": "EXECUTED",
                    "json": {},
                    "created_at": "2020-12-02T13:52:40.909Z"
                }
            ],
            "labels": []
        }
    ]
}

The first fields of the response are important:

Copy

Copied!

{
    "first": true,
    "last": false,
    "number": "0",
    "numberOfElements": "5",
    "size": "5",
    "totalElements": "18",
    "totalPages": "4",
}
...

The above response indicates that 18 transfers (totalElements) are available on 4 pages (totalPages).

last is a boolean indicating whether this is the last page of the response data.

Implement logic in your middleware to keep retrieving your data until last=true, and increment page by 1 for each call.

Logging checklist

Action	Completed
The full request

Pseudocode

Action	Completed
Receive the successful response
Read `last` boolean field
If `last=false`, use pagination
Else, process data

Example Java code for pagination of GET /v4/payments response:

Copy

Copied!

Payments payments = paymentsServices.getPaymentsFromAllPages(getPaymentsParams);

...

@Override
public Payments getPaymentsFromAllPages(GetPaymentsParams getPaymentsParams)
        throws RippleNetProblemException, RnAuthException {

    int page = 0;
    getPaymentsParams.setPage(page);

    // We retrieve the payments on page 0 first
    Payments payments = getPayments(getPaymentsParams);
    int numberOfElements = payments.getNumberOfElements();

    // If the page is not the last one
    while (!payments.isLast()) {
        // We increase the page by 1
        page++;
        getPaymentsParams.setPage(page);

        // We retrieve the payments on the incremented page
        Payments paymentsTemporary = getPayments(getPaymentsParams);

        // And for each payment on the new page
        for (Payment payment : paymentsTemporary.getContent()) {
            // we add it to the global list of Payments
            payments.addContentItem(payment);
            // we also update the total number of payments/elements in the payments object
            numberOfElements++;
            payments.setNumberOfElements(numberOfElements);
        }
        // Finally, at the end of the while loop, we update the first/last booleans
        // If page 1, for example, is the last, then the while loop will exit
        payments.setLast(paymentsTemporary.isLast());
        payments.setFirst(paymentsTemporary.isFirst());
    }

    LOGGER.info("Total amount of payments retrieved: {}", numberOfElements);
    LOGGER.info("Last page where payments were retrieved is page {}",
            getPaymentsParams.getPage());

    // Return the list of Payments
    return payments;
}

3. Process data

After retrieving all the data on all pages, process the list containing your data in the content response field.

Pseudocode

For example for payments.

Action	Completed
If total number of payments > 0, then send them for processing
else do nothing, wait for next polling

Copy

Copied!

if (payments != null) {
    if (payments.getNumberOfElements() > 0) {
        paymentsProcessor.processPayments(payments, hasSubStates);
    } else {
        LOGGER.info("No payments in {} states retrieved", getPaymentsParams.getStates());
    }
}

Logging checklist

Action	Completed
Total number of data to process
Last `page` used

HTTP errors

HTTP Status Code	Error Message	Description
`400`	`invalid_request`	Invalid Request parameters
`401`	Error 401 Unauthorized	Bad credentials, wrong authentication method OR the token can be expired
`403`	Forbidden	No handler found for GET
`500`	Server not available	Service is temporarily unavailable.

Handlers

Name	Code	Action
Unauthorized	`401`	Credentials could have been altered by an administrator or renew the token.
Server not available	`500`	Retry three times every two seconds. If it fails, raise an alert.

Handlers sample code

To handle retries with a 500 HTTP status code, the RippleNet Reference Implementation uses the Spring @Retryable annotation.

Copy

Copied!

@Retryable(value = {RippleNetProblemTemporaryException.class},
            maxAttempts = 3,
            backoff = @Backoff(maxDelay = 2000, random = true))
    Payments getPayments(GetPaymentsParams getPaymentsParams)
            throws RippleNetProblemException, RnAuthException;