In fact, you don’t need to know much about Ajax to extract data. All you need is just to figure out whether the site you want to scrape uses Ajax or not. Many websites use a lot of Ajax such as Google, Amazon and eBay. Usually the URL of the page will not have any change when updating part of the content. With Octoparse, you can easily extract data from web pages where data is loaded with Ajax.
Ajax Case: Gumtree.com
On this page, it has contact details that need us to click the Reveal button to get the complete number. When we click “Reveal”, the rest of the contact number comes out and look at the URL, it doesn't have any change.
So we know this page uses AJAX and we need to set "Load with Ajax" in Octoparse. If not, the result cannot be extracted.
First, open the page in the bulit-in browser. (I just take one page for example.)
Then click on "Reveal". Select “Click an item”.
This page uses Ajax, so we need to set "Load page with Ajax".
Choose “Load page with Ajax”. Set an Ajax timeout. Click “Save”.
Then extract information you want.
Extract brand: Click on the title. Select “extract text.”
Extract price: And extract contact details you just reveal.
Then you run the local extraction and the data you are looking for would be extracted.
Now you know how to extract data from web pages loaded with Ajax.
Happy data hunting.
Author: The Octoparse Team
For more information about Octoparse, please click here.
Sign up today.