undefined

Export Data Extracted into MySQL Database

Wednesday, July 20, 2016 9:56 PM

For the latest tutorials, visit our new self-service portal. Sharpen your skills and explore new ways to use Octoparse.

 

This guide will teach you how to export data to your MySQL database manually and automatically. It will also introduce Octoparse's auto-export tool for databases.

Here are the prerequisites before we start:

  • Tasks with data results (Auto-export to the database is for Cloud data only)
  • A table with data fields configured in your database to receive data 

 

Note: "Auto-export to database" only supports MySql and SQLServer.

If you need to connect with other databases or platforms (such as your own websites), please refer to Octoparse API: OpenAPI or use Zapier.

 

1. How to connect to your database

This tutorial will use the MySql database as an example.

View the Cloud data for the target task.

  • Click the "Export Data" button
  • Open "Export to Database" or "Auto-export" to database
  • Choose the type of your database
  • Press "OK"

extract-data-to-mysql

 

  • Fill out the required fields for the target database settings.

scrape-to-mysql-2

 

After filling out the information, click "Test Connection" to see if you can connect to your database.

mceclip2.png

If the settings are alright, you will receive a "Connected" notice as above. Then, you can click "Next" to move forward.

We need to configure the data mappings now.

  • Select an existing table from the Data Table drop-down options

extract-data-to-mysql-3

 

  • Match the source data fields (data fields extracted) to the target data fields from the designated table
  • Select any source data fields to use as identifiers (primary key)

extract-data-to-mysql-4

To export manually, you can go to the next step now. For the automatic export, you have to set up the schedule settings.

  • Enter the name of the schedule into the text box next to "Scheduled Export"
  • Select at what time interval you would like to export the data to the database, i.e. the export is scheduled to run every 1 hour in the screenshot.
  • Click "Next

extract-data-to-mysql-4

 

  • After set-up is completed, press the "Completed" button to exit.

 

Tips: Once a data field is selected as a primary key, subsequent data records with the same primary key will overwrite the existing ones in the database.

Save Settings:

Once the configuration is completed, it can be saved for later use. You can name the settings first and then press the "Save" button to save it.  

 extract-data-to-mysql-6

 

Next time, if you want to use your previous database export settings, you can just choose a saved setting to save time.

 extract-data-to-mysql-7

 

 

2. Introduction to database auto-export tool

After you set up the database settings, the Auto-export tool will pop up simultaneously. It will show you all the configuration details about auto-export to the database. You need to click "Start" to start the export. Octoparse will do the export at the interval you have set up after starting.

 extract-data-to-mysql-8

 

From the pop-up window, you can start/stop/delete the scheduled auto export to the database. You can also check the error report.

Please note that you have to keep the database auto-export tool open to keep exporting data. That means you cannot turn off your computer during the export. Otherwise, your data export will be stopped.

 

Tip: The auto-export tool is also accessible from the "Toolbox" from the side navigation. 

 extract-data-to-mysql-9 

 

Should you encounter any problems with auto-exporting to a database, feel free to leave your message.
 

Happy Data Hunting!

Author: The Octoparse Team

Download Octoparse Today

 

For more information about Octoparse, please click here.

Sign up today. 

We use cookies to enhance your browsing experience. Read about how we use cookies and how you can control them by clicking cookie settings. If you continue to use this site, you consent to our use of cookies.
Accept decline