Step-by-step tutorials for you to get started with web scraping

Download Octoparse

Why does Octoparse stop after clicking “Next” ?

Thursday, August 16, 2018

Sometimes you may find Octoparse stop or take so long to move on after clicking "Next" button for pagination scraping. This may due to the JavaScript technique (particularly AJAX, short for Asynchronous JavaScript and XML) used on the "Next" button hence Octoparse cannot identify the page reloading without AJAX configuration.

While scraping data from the web, Octoparse takes the reloading as the signal to execute the action, such as "Click item" and "Click to paginate". For the web page using AJAX, it updates new contents without reloading. As there is no reloading, Octoparse doesn't receive the signal to act and would be stuck in the last step. As a result, we may get zero, or much fewer extracted data than we expect.

To fix this issue, you will need to set up “AJAX Load” timeout under “Advanced Options” of the “Click Item” action.

 

Tips!

AJAX timeout setting in Octoparse is based on the time it takes for web pages to load. You can also set up a higher value as long as there is enough time for the next page to load.

 

 

In fact, many websites update new data with AJAX configuration such as Amazon, eBay, Google etc. Sometimes, if Octoparse takes so long to scrape data after clicking an item, you should also configure AJAX timeout to make sure there’s enough time for detail pages to load.

 

If you do have questions about AJAX, read more details here 

 

 

Download Octoparse to start web scraping or contact us for any
question about web scraping!

Contact us Download
btn_sidebar_use.png
btn_sidebar_form.png