Q: I’m having problem collecting data from LinkedIn. What should I do?
Tips for collecting data from LinkedIn:
- LinkedIn is a complicate website since they have anti-bot measures. So if you try to collect data from LinkedIn and keep opening many LinkedIn web pages, it may think that you're a robot and shut down your account.
- Octoparse is a software that mimics your operation on the web page like clicking etc. So if you try to collect data from LinkedIn, please make sure:
- 1. Different account types have different limit on search. It shouldn't be a problem to scrape data from LinkedIn within the commercial use limit on search.
- 2. The X Path of the information is correct. i.e. You should locate the right position of the data you plan to crawl.
- 3. Set a longer Ajax timeout like 30 seconds or even more.The result will take a longer time to come out after that.