When Do You Need Cloud Servers

2 min read

When the amount of information you plan to collect is very huge, Octoparse’s cloud servers would make your job easier.

For example, you want to grab comments, sales records, customer inquiries,product description like product name, the image and the price on an e-commerce site. You can configure a data extraction rule to grab all of this information. But the extraction process may be slowed down because of huge amount of data.

Cloud Extraction means data extraction tasks running in the cloud. You need to configure a rule and upload it to our cloud platform. Then your task will be reasonably assigned to one or several cloud servers to extract data simultaneously via central control commands. For example, you have configured a rule to extract data across pages ( 99 pages in total). Well, your tasks will be automatically divided into three sections and evenly assigned to three cloud servers to extract data at the same time. In this way, it will only take you one third of the original time to extract data from 99 websites.

Professional Edition can run a task by up to 10 cloud servers concurrently or run 10 tasks simultaneously. You can set the priority of the tasks as well.

By dividing the tasks reasonably, we have many cloud servers to perform the extraction on one page. In this case, we usually send 10 servers to help you grab the entire URLs with the fastest speed.

Hot posts

Explore topics

Get web automation tips right into your inbox
Subscribe to get Octoparse monthly newsletters about web scraping solutions, product updates, etc.

Get started with Octoparse today


Related Articles