Step-by-step tutorials for you to get started with web scrapingDownload Octoparse
Set up wait timeThursday, August 16, 2018
Set up a waiting time is to slow down the execution of the current step. Its ultimate purpose is to make sure the steps of the scraper process properly.
Why does it need to "wait before execution"?
In some cases, the previous step may require certain time to complete, like manual input, or the current page requires a longer time to load. So you would need to set up "Wait before execution" for steps like those. Otherwise, it may lead to some error like missing data.
How to set up "Wait before execution"
- All steps created in the workflow are able to set up a waiting time, except "Go To Web Page". It’s easily found "Wait before execution" in the "Advanced Options".
- Wait time options in Octoparse. So far, it’s not available to customize waiting time.
- Most popular tutorials
- Extract multiple pages through pagination
- Scraping info from Craigslist
- Scraping search results from Google Scholar
- Scraping restaurant info from Grubhub
- Scrape product images from eBay