Octoparse Web Crawler Helps Automatically Collect Twitter Data for Academic Research

5/8/2017 4:06:24 AM

[Octoparase User Review] By Fizah Zainudin from London- Basic Plan User 

 

I am currently using Octoparse to obtain Twitter data for my EngD research. I use the functionality to automatically open and log into my twitter account and perform a specific search on twitter to get the data that I require.

 

How Octoparse works for my project?

I created a task using advanced mode to initially open the twitter page and then login using my credentials. The task will then search for a specific set of lexicons that I have defined and will then show the results of the search.

 

The task will then scroll down the twitter search results for predefined number of times in order to get as many tweets as possible and then the task will iterate through tweets to extract the username of the person tweeting and also the actually tweets and the timestamp. I then used the extracted data for further sentiment analysis using a separate tool.

 

Why I choose Octoparse?

I find that it is very intuitive to use Octoparse. There are also many resources available on its website, and a video specifically showing how to extract tweets which is very useful. Also, there appear to be an increasing number of users which means that I expect more discussions and tips to be posted online moving forward. I also find that for a resource hungry processes, it is very useful to have the ability to run those processes on cloud so that it will finish quicker.

 

I have only scratched the surface in terms of the functionality available on Octoparse and look forward to experiment with them and potentially use this tool as the strategic platform for data extraction for my research. I find other tools out there to be have very limited capabilities and are not as intuitive to use as Octoparse. Additionally, I also look forward to introducing Octoparse as the strategic platform for social media data extraction to my department at the university. As it stands, I also feel that Octoparse has the right price point which is not too prohibitive for user in a small organisation as myself.

 

Author's Picks

How to Extract Tweets from Twitter

Scrape Websites with Infinite Scrolling (Quora, Facebook,Twitter)

Octoparse Cloud Service 

 

Download Octoparse Today

For more information about Octoparse, please click here.

Sign up today!

  

 

 

Request Pro Trial Data
Collection
Service
Email
us

Leave us a message

Your name*

Your email*

Subject*

Description*

Attachment(s)

Attach file
Attach file
Please enter details of your issue and we will get back to you ASAP.
× get my coupon now No Thanks