Step-by-step tutorials for you to get started with web scrapingDownload Octoparse
Set up wait timeWednesday, November 24, 2021
The latest version for this tutorial is available here. Go to have a check now!
Set up a waiting time is to slow down the execution of the current step. Its ultimate purpose is to make sure the steps of the scraper process properly.
Why does it need to "wait before execution"?
In some cases, the previous step may require certain time to complete, like manual input, or the current page requires a longer time to load. So you would need to set up "Wait before execution" for steps like those. Otherwise, it may lead to some error like missing data.
How to set up "Wait before execution"
- All steps created in the workflow are able to set up a waiting time, except "Go To Web Page". It’s easily found "Wait before execution" in the "Advanced Options".
- Wait time options in Octoparse. So far, it’s not available to customize waiting time.
- Most popular tutorials
- Scrape tweets from Twitter
- Extract data from a list of URLs
- Extract multiple pages through pagination
- Scrape data on Instagram
- How to download images from a list of URLs?