Step-by-step tutorials for you to get started with web scraping

Download Octoparse

Set up wait time

Thursday, August 16, 2018

Set up a waiting time is to slow down the execution of the current step. Its ultimate purpose is to make sure the steps of the scraper process properly.

 

Why does it need to "wait before execution"?

In some cases, the previous step may require certain time to complete, like manual input, or the current page requires a longer time to load. So you would need to set up "Wait before execution" for steps like those. Otherwise, it may lead to some error like missing data.

 

How to set up "Wait before execution"

  • All steps created in the workflow are able to set up a waiting time, except "Go To Web Page". It’s easily found "Wait before execution" in the "Advanced Options".

 

 

  • Wait time options in Octoparse. So far, it’s not available to customize waiting time.

             

 

Tips!

  • Set up a waiting time will prolong the execution of the task. Please make sure that the step needs to "wait before execution", or it will impact the effectiveness of the crawler.
  • Please remember to click "Save" after changing any settings.

 

Related articles:

Extract data from a list of URLs 

Extract behind a login 

Deal with Ajax 

Learn more techniques for building tasks 

Case tutorial| scrape business information from Yelp 

Download Octoparse to start web scraping or contact us for any
question about web scraping!

Contact us Download
btn_sidebar_use.png
btn_sidebar_form.png