Q: I’m having problem collecting data from LinkedIn. What should I do?
Tips for collecting data from LinkedIn:
LinkedIn is a complicate website since they have anti-bot measures. So if you try to collect data from LinkedIn and keep opening many LinkedIn web pages, it may think that you're a robot and shut down your account.
Octoparse is a software that mimics your operation on the web page like clicking etc.
So if you try to collect data from LinkedIn, please make sure:
1. Different account types have different limit on search. It shouldn't be a problem to scrape data from LinkedIn within the commercial use limit on search.
2. The X Path of the information is correct. i.e. You should locate the right position of the data you plan to crawl.
3. Set a longer Ajax timeout like 30 seconds or even more.The result will take a longer time to come out after that.
Check out the tutorial of how to extract public data from LinkedIn. It shows you how to set Ajax timeout when configuring your task of extracting data from LinkedIn.
If you have any question, we'd happy to help.