undefined

Set up IP Rotation in Octoparse

Wednesday, July 20, 2016 11:43 PM

For the latest tutorials, visit our new self-service portal. Sharpen your skills and explore new ways to use Octoparse.

 

If you've run into any of the issues below, you may want to try setting up IP proxies manually in Octoparse:

  • Unable to log into Octotoparse because your company or institution uses intranet and restricts external requests
  • Target website is very sensitive to web scraping and implements anti-scraping strategies such as IP blocking to stop scraping activities

In this tutorial, we are going to show you how to set up proxies to resolve the issues mentioned above.

Check the video tutorial here >>

ip rotation

 

 

There are 2 ways to do so -

1. Use proxy to login Octoparse

2. Rotate external proxy servers to counteract IP blocking 

 

1. Use proxy to login Octoparse

  • Click mceclip0.png on the login panel
    • Enter proxy information as required

 

  • You can Test to see if the connection is good then Confirm the setup

 

2. Rotate external proxy servers to counteract IP blocking 

If a website blocks your IP or our Cloud, you can set up an IP proxy in the task settings. The IP proxy will work for both local extraction and Cloud extraction.

  • Open the task you would like to set up proxies for
  • Click mceclip0.png on the top left

 

  • Tick the Use IP Proxies box and go to Settings
  • Note that Octoparse only supports HTTP proxies - please separate the IP address of the proxy server and port number with a colon. e.g. 58.22.214.29:2318 then Confirm your setup

Tip! 

The quality of the IP proxy is essential when using this feature for your tasks. We have a recommendation for you-Bright Data, which provides the most advanced rotating residential proxy service offering the fastest and largest real-peer IP network in the world. Check out details at Octoparse Proxy Integration with Bright Data

 

Happy Data Hunting!

Author: The Octoparse Team

Download Octoparse Today

 

For more information about Octoparse, please click here.

Sign up today. 

We use cookies to enhance your browsing experience. Read about how we use cookies and how you can control them by clicking cookie settings. If you continue to use this site, you consent to our use of cookies.
Accept decline