Q: How to scrape detail page data with pagination?

 

A: 

 

When the task executes, the action (the blue textbox) of the workflow are executed in order.

If only one action is in a loop/cycle, then all the items extracted in the loop/cycle will execute the action before going to the next action. Similarly, if there are two actions in a loop/cycle, Octoparse will execute these actions orderly for all the items extracted in the loop/cycle.

So when we drag the second "Loop Item List" box into the first "Cycle Pages" box and place it right above the "Click to paginate" action in the Workflow Designer, Octoparse will get all the data required from the current page, then click the "Click to paginate" action to go to the next page, and will scrape the data of all the items from that page.

 

 

Request Pro Trial Contact
us

Leave us a message

Your name*

Your email*

Subject*

Description*

Attachment(s)

Attach file
Attach file
Please enter details of your issue and we will get back to you ASAP.