Access to The Data Extracted in the Cloud via API

4/21/2016 10:33:38 PM

Cloud Extraction is one of Octoparse's outstanding features.

Octoparse provides you with multiple cloud servers to extract web data simultaneously. You can access to the extracted data in the cloud via API.

(Note: Cloud extraction is not available in the Free Edition.)

You can access to the extracted data and download them via theURL,

“http://dataapi.octoparse.com/SkieerDataAPI/GetData?key=89051c84-1f82-428a-9db6-6f67a87fec35&from={BeginTime}&to={EndTime}”.

To get this URL, click the “Create an API” option, as shown below.

 

Then, you’ll see the URL will be displayed in the pop-up screen.

 

Click your rule file, and choose “Start Cloud Extract” option.

Immediately the cloud management system will read the rule and send out a command to assign cloud servers to extract data in accordance with the rule.

During the process, you can see the data extraction progress clearly.

If you are using Standard Edition, you can get four cloud servers to help you extract data at the same time.

If you are using Professional Edition, you can get 10 cloud servers.

(If you want more cloud servers, please feel free to contact our support team: support@octoparse.com

 

In the process of data extraction, you can choose “View Data” option to check out the data extracted.

And of course you can also access to them via API as described above.

 

An API URL includes three parameters, KEY, BeginTime, and EndTime.

API Key is the unique identifier of each task for verifying ID and authority. (Don’t leak or modify the key. )

The BeginTime and EndTime are set by yourself. But both time are using the the universal time, UTC-05:00.

The format of API is XML.

Before you use the API URL, you need to modify the content in the two brackets (BeginTime and EndTime) manually.

BeginTime: the starting datetime of Extraction. Format: yyyy-mm-dd hh:mm:ss

EndTime: the ending datatime of Extraction. Format: mm-dd-yyyy hh:mm:ss

 

 

 

 

 

 

 

 

 

Author: The Octoparse Team

 

 

 

Download Octoparse Today

 

 

For more information about Octoparse, please click here.

Sign up today.

 

 

Author's Picks

 

About Octoparse

A Comparison among Three Editions of Octoparse

Octoparse 6.0 is Now Available

What A Price Monitor Can Help you?

Collect Data from Amazon

Collect Data from eBay

Collect Data from LinkedIn

Collect Data from Gumtree.com

 

 

 

Recent Posts

Contact
us

Leave us a message

Your name*

Your email*

Subject*

Description*

Attachment(s)

Attach file
Attach file
Please enter details of your issue and we will get back to you ASAP.