Step-by-step tutorials for you to get started with web scrapingDownload Octoparse
What are concurrent runs?Thursday, August 16, 2018
Tasks that are running concurrently means they are being executed simultaneously either in the cloud or on your local machine.
1. Local concurrent runs basically mean executing more than one task on the local machine. For this specific feature, the Free Plan is limited to two concurrent local runs while all the other plans allow for unlimited concurrent runs.
2. Cloud concurrent runs can be checked from the dashboard by filtering for “Running in the Cloud”.
3. Question: When a plan comes with 6 cloud servers, does it mean there should always be 6 concurrent runs in the cloud?
Answers: Not exactly. It is possible to have 6 concurrent runs when each executing task only takes up one cloud server to run. However, when a task is set to run in the cloud, Octoparse will first check to see if the task can be further split into smaller sub-tasks then assign each of the sub-task to run on a cloud server for speeding up the extraction. In this case, one task can take up more than one if not all of the cloud servers (Learn more about task splitting).
- Most popular tutorials
- Scrape product information from Amazon
- How to download images from a list of URLs?
- Extract multiple pages through pagination
- Scraping info from Craigslist
- Scraping search results from Google Scholar