Step-by-step tutorials for you to get started with web scraping

Download Octoparse

How to handle pagination with page numbers?

Thursday, August 16, 2018

“Next” button for pagination is not always available and pagination on some websites may be done by page numbers.

(Check out the tutorial of extracting multiple pages by clicking “Next” )

 

So in this case, to extract multiple pages of data, we will need to modify the XPath of “Click to pagination” step and make it always locate the next number.

(e.g. You’re on the #1 page and you would have to locate the #2 page so that it can always click the next page for pagination purpose.)

(Check out the complete tutorial of extracting multiple pages by page number links )

 

After we clicked page #1 and created a pagination loop, create a new XPath for “Click to paginate” action. XPath syntax “following-sibling” is used most often for this case to select all the siblings after current node. (Learn more about locating elements with XPath )

Here’s an example XPath:

(example URL: http://www.enzolifesciences.com/product-listing/?product_type=Antibodies&application=&text= )

 

And next, replace the auto-generated XPath for the pagination loop with the new XPath.

Download Octoparse to start web scraping or contact us for any
question about web scraping!

Contact us Download
btn_sidebar_use.png
btn_sidebar_form.png