Pagination

Some Ripple Payments API operations paginate their responses to make the result set easier to handle. For example, if you request a list of objects that is potentially too large to handle efficiently, the operation returns the first batch of results along with a marker that you use to access the next batch of results. In Ripple Payments, this marker is a URL parameter, page, indicating the starting point.

For example, if you call GET /v4/payments, the default response includes five payments (the default size) on page=0 (the default page). If there are more payments to fetch (indicated by last=false in the API response), fetch the next set of payments by using the same endpoint with page=1.

We recommend that you fetch all the results on all pages before processing them. If you paginate over a moving list, you may miss some results.

Pagination flowchart

Pagination flowchart

Step-by-step implementation

  1. Create the request
  2. Read the response
  3. Process data

1. Create the request

Choose the data you need to retrieve (payments, etc.). The HTTP method for all the data types is GET.

For example:

Copy
Copied!
GET {ripplepayments_base_url}/v4/payments

Authorization: Bearer eyJhbGciO...l0ZSIwIm

To use pagination, add the page parameter to the request. Its default value is 0.

Your request becomes:

Copy
Copied!
GET {ripplepayments_base_url}/v4/payments?page=X

Authorization: Bearer eyJhbGciO...l0ZSIwIm

Logging checklist

Action Completed
The full request

Pseudocode

Action Completed
Choose the data to retrieve (payments, ...)
Include the page parameter in the request

2. Read the response

Before processing the data you received with page=0, read and analyze the response.

This is a sample response from GET /v4/payments?page=0&size=5:

Click to show complete response
Copy
Copied!
    "first": true,
    "last": false,
    "number": "0",
    "numberOfElements": "5",
    "size": "5",
    "totalElements": "18",
    "totalPages": "4",
    "sort": [
        {
            "direction": "DESC",
            "property": "lastModifiedDate",
            "ignoreCase": false,
            "nullHandling": "NATIVE",
            "ascending": false,
            "descending": true
        }
    ],
    "content": [
        {
            "transfer_id": "59d96a63-1c41-43ab-af10-7bf961d73457",
            "state": "EXECUTED",
            "owner_address": "test.cloud.blueprint2",
            "sender_address": "trans_eur_receiver@test.cloud.blueprint2",
            "receiver_address": "conct_eur_sender@test.cloud.blueprint2",
            "amount": "1000000000.000000000",
            "internal_id": null,
            "end_to_end_id": null,
            "sending_fee": null,
            "created_at": "2021-02-16T08:20:59.842Z",
            "user_info": [],
            "labels": []
        },
        {
            "transfer_id": "2e6f1167-e91a-43dd-be81-716088884549",
            "state": "EXECUTED",
            "owner_address": "test.cloud.blueprint2",
            "sender_address": "conct_eur_sender@test.cloud.blueprint2",
            "receiver_address": "trans_eur_receiver@test.cloud.blueprint2",
            "amount": "900000.000000000",
            "internal_id": null,
            "end_to_end_id": null,
            "sending_fee": null,
            "created_at": "2021-02-16T07:29:35.036Z",
            "user_info": [],
            "labels": []
        },
        {
            "transfer_id": "89b6668c-7a81-485b-99b4-bc924f29140a",
            "state": "EXECUTED",
            "owner_address": "test.cloud.blueprint2",
            "sender_address": "conct_eur_sender@test.cloud.blueprint2",
            "receiver_address": "trans_eur_receiver@test.cloud.blueprint2",
            "amount": "99999000000.000000000",
            "internal_id": null,
            "end_to_end_id": null,
            "sending_fee": null,
            "created_at": "2021-02-16T07:29:07.781Z",
            "user_info": [],
            "labels": []
        },
        {
            "transfer_id": "594ceb52-4db1-4bbd-9956-141e7d3f032a",
            "state": "COMPLETED",
            "owner_address": "test.cloud.blueprint1",
            "sender_address": "trans_usd_sender@test.cloud.blueprint1",
            "receiver_address": "conct_usd_sender@test.cloud.blueprint1",
            "amount": "3.000000000",
            "internal_id": "456",
            "end_to_end_id": "123",
            "sending_fee": null,
            "created_at": "2021-01-04T09:33:35.105Z",
            "user_info": [
                {
                    "node_address": "test.cloud.blueprint1",
                    "state": "EXECUTED",
                    "json": {},
                    "created_at": "2021-01-04T09:33:35.106Z"
                }
            ],
            "labels": []
        },
        {
            "transfer_id": "827da53b-8086-4d1b-8d0f-d8d18ac46051",
            "state": "COMPLETED",
            "owner_address": "test.cloud.blueprint1",
            "sender_address": "trans_usd_sender@test.cloud.blueprint1",
            "receiver_address": "conct_usd_sender@test.cloud.blueprint1",
            "amount": "3.000000000",
            "internal_id": "456",
            "end_to_end_id": "123",
            "sending_fee": null,
            "created_at": "2020-12-02T13:52:40.909Z",
            "user_info": [
                {
                    "node_address": "test.cloud.blueprint1",
                    "state": "EXECUTED",
                    "json": {},
                    "created_at": "2020-12-02T13:52:40.909Z"
                }
            ],
            "labels": []
        }
    ]
}

The first fields of the response are important:

Copy
Copied!
{
    "first": true,
    "last": false,
    "number": "0",
    "numberOfElements": "5",
    "size": "5",
    "totalElements": "18",
    "totalPages": "4",
}
...

The above response indicates that 18 payments (totalElements) are available on 4 pages (totalPages).

last is a boolean indicating whether this is the last page of the response data.

Implement logic in your middleware to keep retrieving your data until last=true, and increment page by 1 for each call.

Logging checklist

Action Completed
The full request

Pseudocode

Action Completed
Receive the successful response
Read last boolean field
If last=false, use pagination
Else, process data

Example Java code for pagination of GET /v4/payments response:

Copy
Copied!
Payments payments = paymentsServices.getPaymentsFromAllPages(getPaymentsParams);

...

@Override
public Payments getPaymentsFromAllPages(GetPaymentsParams getPaymentsParams)
        throws RippleNetProblemException, RnAuthException {

    int page = 0;
    getPaymentsParams.setPage(page);

    // We retrieve the payments on page 0 first
    Payments payments = getPayments(getPaymentsParams);
    int numberOfElements = payments.getNumberOfElements();

    // If the page is not the last one
    while (!payments.isLast()) {
        // We increase the page by 1
        page++;
        getPaymentsParams.setPage(page);

        // We retrieve the payments on the incremented page
        Payments paymentsTemporary = getPayments(getPaymentsParams);

        // And for each payment on the new page
        for (Payment payment : paymentsTemporary.getContent()) {
            // we add it to the global list of Payments
            payments.addContentItem(payment);
            // we also update the total number of payments/elements in the payments object
            numberOfElements++;
            payments.setNumberOfElements(numberOfElements);
        }
        // Finally, at the end of the while loop, we update the first/last booleans
        // If page 1, for example, is the last, then the while loop will exit
        payments.setLast(paymentsTemporary.isLast());
        payments.setFirst(paymentsTemporary.isFirst());
    }

    LOGGER.info("Total amount of payments retrieved: {}", numberOfElements);
    LOGGER.info("Last page where payments were retrieved is page {}",
            getPaymentsParams.getPage());

    // Return the list of Payments
    return payments;
}

3. Process data

After retrieving all the data on all pages, process the list containing your data in the content response field.

Pseudocode

For example for payments.

Action Completed
If total number of payments > 0, then send them for processing
else do nothing, wait for next polling
Copy
Copied!
if (payments != null) {
    if (payments.getNumberOfElements() > 0) {
        paymentsProcessor.processPayments(payments, hasSubStates);
    } else {
        LOGGER.info("No payments in {} states retrieved", getPaymentsParams.getStates());
    }
}

Logging checklist

Action Completed
Total number of data to process
Last page used

HTTP errors

HTTP Status Code Error Message Description
400 invalid_request Invalid Request parameters
401 Error 401 Unauthorized Bad credentials, wrong authentication method OR the token can be expired
403 Forbidden No handler found for GET
500 Server not available Service is temporarily unavailable.

Handlers

Name Code Action
Unauthorized 401 Credentials could have been altered by an administrator or renew the token.
Server not available 500 Retry three times every two seconds. If it fails, raise an alert.

Handlers sample code

To handle retries with a 500 HTTP status code, the RippleNet Reference Implementation uses the Spring @Retryable annotation.

Copy
Copied!
@Retryable(value = {RippleNetProblemTemporaryException.class},
            maxAttempts = 3,
            backoff = @Backoff(maxDelay = 2000, random = true))
    Payments getPayments(GetPaymentsParams getPaymentsParams)
            throws RippleNetProblemException, RnAuthException;