Set up IP Rotation in OctoparseWednesday, July 20, 2016 11:43 PM
If you've run into any of the issues below, you may want to try setting up IP proxies manually in Octoparse:
- Unable to log into Octotoparse because your company or institution uses intranet and restricts external requests
- Target website is very sensitive to web scraping and implements anti-scraping strategies such as IP blocking to stop scraping activities
In this tutorial, we are going to show you how to set up proxies to resolve the issues mentioned above.
Check the video tutorial here >>
There are 2 ways to do so -
1. Use proxy to login Octoparse
- Click on the login panel
- Enter proxy information as required
- You can Test to see if the connection is good then Confirm the setup
2. Rotate external proxy servers to counteract IP blocking
If a website blocks your IP or our Cloud, you can set up an IP proxy in the task settings. The IP proxy will work for both local extraction and Cloud extraction.
- Open the task you would like to set up proxies for
- Click on the top left
- Tick the Use IP Proxies box and go to Settings
- Note that Octoparse only supports HTTP proxies - please separate the IP address of the proxy server and port number with a colon. e.g. 18.104.22.168:2318 then Confirm your setup
The quality of the IP proxy is essential when using this feature for your tasks. We have a recommendation for you-Bright Data, which provides the most advanced rotating residential proxy service offering the fastest and largest real-peer IP network in the world. Check out details at Octoparse Proxy Integration with Bright Data
Happy Data Hunting!
Author: The Octoparse Team
For more information about Octoparse, please click here.
Sign up today.