When Do You Need Cloud Servers
Thursday, July 21, 2016
When the amount of information you plan to collect is very huge, Octoparse's cloud servers would make your job easier.
For example, you want to grab comments, sales records, customer inquiries,product description like product name, the image and the price on an e-commerce site. You can configure a data extraction rule to grab all of this information. But the extraction process may be slowed down because of huge amount of data.
Cloud Extraction means data extraction tasks running in the cloud. You need to configure a rule and upload it to our cloud platform. Then your task will be reasonably assigned to one or several cloud servers to extract data simultaneously via central control commands. For example, you have configured a rule to extract data across pages ( 99 pages in total). Well, your tasks will be automatically divided into three sections and evenly assigned to three cloud servers to extract data at the same time. In this way, it will only take you one third of the original time to extract data from 99 websites.
Professional Edition can run a task by up to 10 cloud servers concurrently or run 10 tasks simultaneously. You can set the priority of the tasks as well.
By dividing the tasks reasonably, we have many cloud server to perform the extraction on one page. In this case we usually send 10 servers to help you grab the entire URLs with the fastest speed.
Author: The Octoparse Team
For more information about Octoparse, please click here.
Sign up today.
Author's Picks
A Comparison among Three Editions of Octoparse
Octoparse 6.0 is Now Available
What A Price Monitor Can Help you?
Most popular posts
Posts by topic
Download Octoparse to start web scraping or contact us for any
question about web scraping!
