# Pagination Some RippleNet API operations paginate their responses to make the result set easier to handle. For example, if you request a list of objects that is potentially too large to handle efficiently, the operation returns the first batch of results along with a marker that you use to access the next batch of results. In RippleNet this marker is a URL parameter, `page`, indicating the starting point. For example, if you call `GET /v4/transfers`, the default response includes 5 transfers (the default `size`) on `page=0` (the default `page`). If there are more transfers to fetch (indicated by `last=false` in the API response), fetch the next set of transfers by using the same endpoint with `page=1`. We recommend that you fetch all the results on all pages before processing them. If you paginate over a moving list, you may miss some results. ## Pagination flowchart ![Pagination flowchart](/assets/pagination-flowchart.1836b7a1da52ed43134968fc9391359788cd8978a381218efe5681b4e4ddbe7f.40f61ff2.svg) ## Step-by-step implementation 1. [Create the request](#1-create-the-request) 2. [Read the response](#2-read-the-response) 3. [Process data](#3-process-data) hr ### 1. Create the request Choose the data you need to retrieve (payments, transfers, etc.). The HTTP method for all the data types is `GET`. For example: ```json GET {ripplenet_base_url}/v4/transfers Authorization: Bearer eyJhbGciO...l0ZSIwIm ``` To use pagination, add the `page` parameter to the request. Its default value is 0. Your request becomes: ```json GET {ripplenet_base_url}/v4/transfers?page=X Authorization: Bearer eyJhbGciO...l0ZSIwIm ``` #### Logging checklist | Action | Completed | | --- | --- | | The full request | | #### Pseudocode | Action | Completed | | --- | --- | | Choose the data to retrieve (payments, transfers...) | | | Include the `page` parameter in the request | | hr ### 2. Read the response Before processing the data you received with `page=0`, read and analyze the response. This is a sample response from `GET /v4/transfers?page=0&size=5`: details summary Click to show complete response ```json{ "first": true, "last": false, "number": "0", "numberOfElements": "5", "size": "5", "totalElements": "18", "totalPages": "4", "sort": [ { "direction": "DESC", "property": "lastModifiedDate", "ignoreCase": false, "nullHandling": "NATIVE", "ascending": false, "descending": true } ], "content": [ { "transfer_id": "59d96a63-1c41-43ab-af10-7bf961d73457", "state": "EXECUTED", "owner_address": "test.cloud.blueprint2", "sender_address": "trans_eur_receiver@test.cloud.blueprint2", "receiver_address": "conct_eur_sender@test.cloud.blueprint2", "amount": "1000000000.000000000", "internal_id": null, "end_to_end_id": null, "sending_fee": null, "created_at": "2021-02-16T08:20:59.842Z", "user_info": [], "labels": [] }, { "transfer_id": "2e6f1167-e91a-43dd-be81-716088884549", "state": "EXECUTED", "owner_address": "test.cloud.blueprint2", "sender_address": "conct_eur_sender@test.cloud.blueprint2", "receiver_address": "trans_eur_receiver@test.cloud.blueprint2", "amount": "900000.000000000", "internal_id": null, "end_to_end_id": null, "sending_fee": null, "created_at": "2021-02-16T07:29:35.036Z", "user_info": [], "labels": [] }, { "transfer_id": "89b6668c-7a81-485b-99b4-bc924f29140a", "state": "EXECUTED", "owner_address": "test.cloud.blueprint2", "sender_address": "conct_eur_sender@test.cloud.blueprint2", "receiver_address": "trans_eur_receiver@test.cloud.blueprint2", "amount": "99999000000.000000000", "internal_id": null, "end_to_end_id": null, "sending_fee": null, "created_at": "2021-02-16T07:29:07.781Z", "user_info": [], "labels": [] }, { "transfer_id": "594ceb52-4db1-4bbd-9956-141e7d3f032a", "state": "COMPLETED", "owner_address": "test.cloud.blueprint1", "sender_address": "trans_usd_sender@test.cloud.blueprint1", "receiver_address": "conct_usd_sender@test.cloud.blueprint1", "amount": "3.000000000", "internal_id": "456", "end_to_end_id": "123", "sending_fee": null, "created_at": "2021-01-04T09:33:35.105Z", "user_info": [ { "node_address": "test.cloud.blueprint1", "state": "EXECUTED", "json": {}, "created_at": "2021-01-04T09:33:35.106Z" } ], "labels": [] }, { "transfer_id": "827da53b-8086-4d1b-8d0f-d8d18ac46051", "state": "COMPLETED", "owner_address": "test.cloud.blueprint1", "sender_address": "trans_usd_sender@test.cloud.blueprint1", "receiver_address": "conct_usd_sender@test.cloud.blueprint1", "amount": "3.000000000", "internal_id": "456", "end_to_end_id": "123", "sending_fee": null, "created_at": "2020-12-02T13:52:40.909Z", "user_info": [ { "node_address": "test.cloud.blueprint1", "state": "EXECUTED", "json": {}, "created_at": "2020-12-02T13:52:40.909Z" } ], "labels": [] } ] } ``` The first fields of the response are important: ```json { "first": true, "last": false, "number": "0", "numberOfElements": "5", "size": "5", "totalElements": "18", "totalPages": "4", } ... ``` The above response indicates that 18 transfers (`totalElements`) are available on 4 pages (`totalPages`). `last` is a boolean indicating whether this is the last page of the response data. Implement logic in your middleware to keep retrieving your data until `last=true`, and increment `page` by 1 for each call. #### Logging checklist | Action | Completed | | --- | --- | | The full request | | hr #### Pseudocode | Action | Completed | | --- | --- | | Receive the successful response | | | Read `last` boolean field | | | If `last=false`, use pagination | | | Else, process data | | Example Java code for pagination of `GET /v4/payments` response: ```java Payments payments = paymentsServices.getPaymentsFromAllPages(getPaymentsParams); ... @Override public Payments getPaymentsFromAllPages(GetPaymentsParams getPaymentsParams) throws RippleNetProblemException, RnAuthException { int page = 0; getPaymentsParams.setPage(page); // We retrieve the payments on page 0 first Payments payments = getPayments(getPaymentsParams); int numberOfElements = payments.getNumberOfElements(); // If the page is not the last one while (!payments.isLast()) { // We increase the page by 1 page++; getPaymentsParams.setPage(page); // We retrieve the payments on the incremented page Payments paymentsTemporary = getPayments(getPaymentsParams); // And for each payment on the new page for (Payment payment : paymentsTemporary.getContent()) { // we add it to the global list of Payments payments.addContentItem(payment); // we also update the total number of payments/elements in the payments object numberOfElements++; payments.setNumberOfElements(numberOfElements); } // Finally, at the end of the while loop, we update the first/last booleans // If page 1, for example, is the last, then the while loop will exit payments.setLast(paymentsTemporary.isLast()); payments.setFirst(paymentsTemporary.isFirst()); } LOGGER.info("Total amount of payments retrieved: {}", numberOfElements); LOGGER.info("Last page where payments were retrieved is page {}", getPaymentsParams.getPage()); // Return the list of Payments return payments; } ``` hr ### 3. Process data After retrieving all the data on all pages, process the list containing your data in the `content` response field. #### Pseudocode For example for payments. | Action | Completed | | --- | --- | | If total number of payments > 0, then send them for processing | | | else do nothing, wait for next polling | | ```java if (payments != null) { if (payments.getNumberOfElements() > 0) { paymentsProcessor.processPayments(payments, hasSubStates); } else { LOGGER.info("No payments in {} states retrieved", getPaymentsParams.getStates()); } } ``` #### Logging checklist | Action | Completed | | --- | --- | | Total number of data to process | | | Last `page` used | | hr ### HTTP errors | HTTP Status Code | Error Message | Description | | --- | --- | --- | | `400` | `invalid_request` | Invalid Request parameters | | `401` | Error 401 Unauthorized | Bad credentials, wrong authentication method OR the token can be expired | | `403` | Forbidden | No handler found for GET | | `500` | Server not available | Service is temporarily unavailable. | ### Handlers | Name | Code | Action | | --- | --- | --- | | Unauthorized | `401` | Credentials could have been altered by an administrator or renew the token. | | Server not available | `500` | Retry three times every two seconds. If it fails, raise an alert. | #### Handlers sample code To handle retries with a 500 HTTP status code, the RippleNet Reference Implementation uses the Spring [`@Retryable`](https://www.baeldung.com/spring-retry#using-spring-retry) annotation. ```java @Retryable(value = {RippleNetProblemTemporaryException.class}, maxAttempts = 3, backoff = @Backoff(maxDelay = 2000, random = true)) Payments getPayments(GetPaymentsParams getPaymentsParams) throws RippleNetProblemException, RnAuthException; ```