Q: How to scrape data from multiple web pages/URLs?

 Solutions are available for a related question here. Go to have a check now!




Sometimes you will encounter web compatibility issues with Octoparse’s built-in browser. For example, the hyperlink on the web page cannot be triggered when the next page button is clicked.

Sometimes Octoparse stops executing the next step during the extraction if it takes a really long time to load the URL while the web content has loaded completely.

For some websites, we would suggest that you use the “URL list” loop to extract information out of multiple web-pages with a similar layout.

You can observe whether the website URLs you want to scrape has the same characters or parameters.

Sample URLs:




Octoparse can scrape data from multiple web pages that share similar layout or many website URLs that are organized as a logical sequence by using “URL list Loop”.

There are only 4 steps to scrape multiple URLs. See the picture below.

  1. Drag a Loop action to workflow
  2. Choose the “List of URLs” mode
  3. Enter/Paste a list of URLs you want to scrape into the text box
  4. Don’t forget to click OK and Save button



That’s done! The “Go to Web Page” action will be generated automatically.


We use cookies to enhance your browsing experience. Read about how we use cookies and how you can control them by clicking cookie settings. If you continue to use this site, you consent to our use of cookies.
Accept decline