Set up IP Rotation in Octoparse
Wednesday, July 20, 2016 11:43 PMFor the latest tutorials, visit our new self-service portal. Sharpen your skills and explore new ways to use Octoparse.
If you've run into any of the issues below, you may want to try setting up IP proxies manually in Octoparse:
- Unable to log into Octotoparse because your company or institution uses intranet and restricts external requests
- Target website is very sensitive to web scraping and implements anti-scraping strategies such as IP blocking to stop scraping activities
In this tutorial, we are going to show you how to set up proxies to resolve the issues mentioned above.
Check the video tutorial here >>
There are 2 ways to do so -
1. Use proxy to login Octoparse
2. Rotate external proxy servers to counteract IP blocking
1. Use proxy to login Octoparse
- Click
on the login panel
- Enter proxy information as required
- You can Test to see if the connection is good then Confirm the setup
2. Rotate external proxy servers to counteract IP blocking
If a website blocks your IP or our Cloud, you can set up an IP proxy in the task settings. The IP proxy will work for both local extraction and Cloud extraction.
- Open the task you would like to set up proxies for
- Click
on the top left
- Tick the Use IP Proxies box and go to Settings
- Note that Octoparse only supports HTTP proxies - please separate the IP address of the proxy server and port number with a colon. e.g. 58.22.214.29:2318 then Confirm your setup
Tip! The quality of the IP proxy is essential when using this feature for your tasks. We have a recommendation for you-Bright Data, which provides the most advanced rotating residential proxy service offering the fastest and largest real-peer IP network in the world. Check out details at Octoparse Proxy Integration with Bright Data |
Happy Data Hunting!
Author: The Octoparse Team
For more information about Octoparse, please click here.
Sign up today.