Blog > Big Data > Post

Cryptocurrency Market Analysis with Web Scraping

Thursday, August 1, 2019


Since the introduction of cryptocurrency -- Bitcoin in 2009, the entire market of cryptocurrency is prospected and continues to flourish.  To date, there are over 4,000 altcoins (alternative cryptocurrency other than Bitcoins) have been created. Besides, cryptocurrency is notorious for its volatility. It could be very difficult to keep an eye on the market, especially for novice investors.


How Web Scraping matters in Cryptocurrency Trading?

Web scraping is predominantly used in e-commerce and sales for price monitoring and leads generation. Nowadays more investors start to leverage the tech in online financial. It automates the process of data extraction from multiple sources and stores data in a structured format for further analysis. 


In this article, I am going to break down the steps and show you how to overcome investment obstacles using web scraping:

  • Extract historical crypto market data for a comprehensive market analysis
  • Monitor the crypto pricing so you have a clear picture of the whole market cap like a pro trader.


There are many ways you can use to extract data. You can write scrip or choose a web scraping tool to automate the process. This is a list of 10 most popular web scraping tools as reference. The tool I will use is Octoparse. It has a scraping pipeline and integrates the data into a database or with cloud APIs for Enterprises of all sizes. For users who want a lighter volume of data, the basic version is completely free to use. 


I want to invest Bitcoin but don’t know if it is the right time. I am going to extract Bitcoin Market Data from Jan 2019 till now so I can see the market trend and decide whether I should invest or not.  In this case, go to CoinMarketCap page and set up a time range

Then, in Octoparse:

Step One: Start an extraction task with Advance Mode, and upload the URL.

Step Two: Click the date of the chart. Octoparse will find similar elements with the same attributes. Follow the Action Tip and select all the fields including Open, High, Low, Close, Volume and Market Cap.

Step Three: Start extraction and export the data into Excel. 

You should be able to get the whole chart like this: Bitcoin Market in 2019.



Now we have the data in Excel. We can graph it with Excel Waterfall chart

 bitcoin chart


As you can see that the market follows a clear pattern in which the price rises, drops off and stabilizes. A price pump will be followed by a consolidated period. And, a sharp rise will soon follow a sharp dropping off. It makes sense because when the market is too volatile it needs to cool down to a stabilized level. The market since June has higher volatility. The consolidated window gets shorter and shorter, price rises and drop rapidly comparing that at the beginning of the year. In most cases, the higher the volatility, the riskier the investment. At the end of the chart, it indicates the price fluctuates with an upward trend without any sign of getting stabilized. It clearly shows that it’s not the best time of the year to invest in Bitcoin. 


  • Price Monitoring

Since the market price fluctuates, for both spectators and investors, it is necessary to keep an eye on the market. Web scraping can monitor the price change and deliver to your database for later access. So whenever the prices hit a certain point, you can take action in time. 


Set up a new project with https://coinmarketcap.com/all/views/all/. I am going to select each coin name to extract the price data. Octoparse finds all similar elements by selecting all listing names. I click “TR” command from the Action Table to tell Octoparse to extract rows instead of columns.


Then I follow the Action Tip, click “Select All Sub-element” to define extraction fields from the same row. Octoparse will select all data fields in the same pattern. I then follow the guide on the Action Tip and click “Select All” command to confirm the selection. The selected field will turn green if it has been successfully selected.


Then Click “Extract Data in the loop” to extract the data. Now we have all the data including the Market Cap, Price, Circulating Supply, Volume and changes within 1h, 24h and 7days.  




From the Dashboard, I set the extraction schedule as 30mins interval. As a result, my database gets updated every 30 mins. I can set my hands off the table while still monitor the market. 

This is a step by step guide tutorial: 


What else web scraping can help with Cryptocurrency Analysis?

Sentiment Analysis:

Sentiment Analysis measures people’s opinion through natural language processing. The idea is to monitor the public mood about the market from social media since the network is the main portal for most cryptocurrency investors to express the market sentiment. Guus argued that “A change in the market sentiment would be associated with a change in the value of the market index.” (Guss, 2017) Moreover, Kaminski (2014) also pointed out that the data collected from Twitter shows a significant correlation between the sentiment and the closing price, trading volume and intra-day price. 


News Aggregator:

For managerial professionals, lookout the news media to catch up with the latest bitcoin trading information is the daily basis. Web scraping ables to collect the news information from various platforms and send to your email so you can save your time from searching.

Author: Ashley

Ashley is a data enthusiast and passionate blogger with hands-on experience in web scraping. She focuses on capturing web data and analyzing in a way that empowers companies and businesses with actionable insights. Read her blog here to discover practical tips and applications on web data extraction

Si desea ver el contenido en español, por favor haga clic en:  5 Razones por El Web Scraping Puede Beneficiar a Su Negocio




https: //coinmarketcap.com Kaminski, J. (2014). Nowcasting the bitcoin market with twitter signals. arXiv preprint arXiv:1406.7577.







We use cookies to enhance your browsing experience. Read about how we use cookies and how you can control them by clicking cookie settings. If you continue to use this site, you consent to our use of cookies.
Accept decline