Step-by-step tutorials for you to get started with web scrapingDownload Octoparse
Why cloud extraction does not run all the tasks I set up to execute?Friday, September 21, 2018
Cloud extraction is the most important feature in our premium plans. You can set up tasks running in the cloud, or even schedule the tasks running at a certain frequency and totally free your computer.
Many users may have noticed that not all the tasks would execute immediately after being set up to run in the cloud.
There are two possible reasons for this issue.
1) Cloud servers under your account are all taken up
2) The number of the tasks have exceeded the limit you set up
1）Cloud servers under your account are all taken up.
Octoparse Cloud service is supported by hundreds of cloud servers. One task would take up one server at least. Split-able tasks (built with URL list, Fixed list or Text list) would use several or all of the servers under one account to speed up the extraction process. If you have already executed several tasks which have taken up all the servers, you will not see other tasks extracting data.
Standard Plan can only run at most 6 tasks while Professional Plan can have at most 20 concurrent extractions.
If a task does not have the used time shown on the dashboard, it is actually not running but is waiting to run.
2) The number of the tasks has exceeded the maximum number of tasks in parallel
If your tasks do not take up all the servers but you still can not get other tasks running, you need to check the maximum number of tasks in parallel you set up. The default maximum number is 6, which means you can only have at most 6 tasks running at the same time even there are cloud servers not used.
Click the account setting button in the sidebar panel.
You can select a desired number from the drop-down options
- Most popular tutorials
- Extract multiple pages through pagination
- Scraping info from Craigslist
- Scraping search results from Google Scholar
- Scraping restaurant info from Grubhub
- Scrape product images from eBay