undefined

Run tasks in the Cloud

Monday, March 13, 2017 8:01 AM

For the latest tutorials, visit our new self-service portal. Sharpen your skills and explore new ways to use Octoparse.

 

Octoparse offers a powerful Cloud platform for premium users (Standard & Professional ) to run your tasks 24/7. 

 

When you run a task with "Cloud Extraction", it runs in the cloud with multiple servers using Octoparse's IPs. You can shut down the APP or your computer while the task is running. No need to worry about hardware limitations. Data extracted will be saved in the cloud and can be accessed at any time.

 

Task scheduling is also supported by Octoparse Cloud extraction. To retrieve the most updated information, you can schedule your task to run as frequently as you need.

 

Features covered in this tutorial:

· Run your task with Cloud extraction

· Batch run tasks with Cloud extraction

· Settings of Cloud extraction

· Schedule a run in the Cloud

 

To run your task with cloud extraction:

When you finish configuring your task, click "Run" and select "Run task in the Cloud" to execute a run in the cloud.

 

Once a task is set to run in the cloud, its status will change to "Running" on the dashboard. 

 

To batch run tasks with cloud extraction:

Select any tasks that need to be run, click on 'Run (Cloud)' and the tasks will run together in the Cloud.

 

Settings of cloud extraction:

Octoparse cloud extraction allows for executing multiple tasks simultaneously. 

 

On the Standard Plan, you can run 6 concurrent tasks  in the cloud (6 cloud servers available), and on the Professional Plan, you can run 20 concurrent tasks  (20 cloud servers available).  To set the maximum number of tasks running in parallel, click and select a desired number from the drop-down options:

dropdown options

 

Tips!

1. How’s the performance of cloud extraction?

Getting data extracted in the Cloud can be a lot faster than running the tasks locally given the task is split-table (Learn about when a task is split-table).

A split-table task can be broken down into multiple subtasks which can be run on multiple servers simultaneously, thus making the extraction faster.

2. Can I run more tasks than the maximum number's allowing for?

Yes, you can. But some of the tasks will be queued until more cloud servers become available upon completion of the earlier tasks. 

 

To schedule a run in the cloud: 

When you finish configuring your task, click "Run" and select "Schedule(Local)".

 

Select how frequently you want to run it:  Once/Weekly/Monthly/Repeat. And customize the time and date according to your data requirements. Click "Save and Run" and the task will be run as scheduled.

 

The time for the next execution can be found on the dashboard on the "Next Run" column.

 

And if you wish to cancel a scheduled task, click "More", select "Schedule OFF" in the "Cloud runs".

 

Tips!

What's the default time zone for Octoparse Cloud platform?

The next execution time shown on the dashboard is in your local time zone (according to your operating system) by default. However, if you've built the task to extract "current date & time" in the Cloud, the extracted time & date will be in UTC±00:00 regardless of your actual location.

Currently, Octoparse does not support changing the timezone. 

 

To set a schedule for a group of tasks, switch to 'Task Group' mode, select a task group, then choose "Set a schedule for the task group".

 

Other advantages of Cloud Extraction:

Automatic IP rotation 

Extraction speed up 

Export data via API 

Octoparse Advanced API 

 

Happy Data Hunting!

Author: The Octoparse Team

Download Octoparse Today

 

For more information about Octoparse, please click here.

Sign up today. 

We use cookies to enhance your browsing experience. Read about how we use cookies and how you can control them by clicking cookie settings. If you continue to use this site, you consent to our use of cookies.
Accept decline