Step-by-step tutorials for you to get started with web scraping

Download Octoparse

Scrape hotel data from Booking.com

Thursday, November 01, 2018

In this tutorial, we are going to show you how to scrape hotel information from Booking.com. 

To follow through, you may want to use this URL in the tutorial: 

https://www.booking.com/searchresults.en-gb.html?label=gen173nr-1FCAEoggJCAlhYSDNYBGgxiAEBmAEuwgEDYWJuyAEO2AEB6AEB-AELkgIBeagCBA&sid=1964712b1d001062109ae5baf669fd26&checkin_month=12&checkin_monthday=1&checkin_year=2018&checkout_month=12&checkout_monthday=2&checkout_year=2018&class_interval=1&dest_id=20088325&dest_type=city&dtdisc=0&from_sf=1&group_adults=1&group_children=0&inac=0&index_postcard=0&label_click=undef&no_rooms=1&offset=0&postcard=0&raw_dest_type=city&room1=A&sb_price_type=total&src=index&src_elem=sb&ss=New%20York&ss_all=0&ssb=empty&sshis=0&ssne=New%20York&ssne_untouched=New%20York&selected_currency=USD&changed_currency=1&top_currency=1&nflt= 

We will enter each hotel page and scrape the details including the hotel name, rating, address, one photo, and room facilities.

 

Tips!

1. We strongly recommend to set up the filters as you need in your own browser and enter the configured URL into Octoparse to start the task.

2. The structure and display of Booking.com might vary depending on your IP and preferred language.

 

Here are the main steps in this tutorial: [Download demo task file here ]

1) Go To Web Page - to open the targeted web page

2) Create a pagination loop - to scrape all the results from multiple pages

3) Create a "Loop Item" - to loop click into each item on each list

4) Extract data - to select the data for extraction

5) Start extraction - to run the task and get data

 

 

 

1) Go To Web Page - to open the targeted web page

   · Click "+ Task" to start a task using Advanced Mode
     Advanced Mode is a highly flexible and powerful web scraping mode. For people who want to scrape from websites with complex structures, like Booking.com, we strongly recommend Advanced Mode to start your data extraction project.

   · Paste the URL into the "Extraction URL" box and click "Save URL" to move on

 

 

 

 

 

 

2) Create a pagination loop - to scrape all the results from multiple pages

   · Turn on the “Workflow Mode” by switching the "Workflow" button in the top-right corner in Octoparse

     We strongly suggest you turn on the "Workflow Mode" to get a better picture of what you are doing with your task, just in case you mess up with the steps.

   · Scroll down the page and click the next page button ">"

   · Click "Loop click the selected link" from "Action Tips"

 

 

 

 

 

 

3) Create a "Loop Item" - to loop click into each item on each list

   · Click "Go To Web Page" to go to the first page

     When extracting data throughout multiple pages, you should always begin your task building on the first page.

   · Click the first hotel link on the page

   · Click "Select all" on "Action Tips"

     Octoparse will automatically select all the links to the detail pages on the current page. The selected links will be highlighted in green while other links to the detail pages will be highlighted in red.

   · Click "Loop click each element" to create a "Loop Item"

     Octoparse will click through each link captured in the "Loop Item", and open the detail page.

 

 

 

 

 

4) Extract data - to select the data for extraction

After you click "Loop click each element", Octoparse will open the detail page of the first hotel.

    · Click on the data you need on the page

    · Extract the selected data

      · For extracting the text, select "Extract text of the selected element" from "Action Tips"

      · For extracting the image URL, select "Extract the URL of the selected link" from "Action Tips"

 

   · Rename the fields by selecting from the pre-defined list or inputting on your own

 

 

 

 

 

 

5) Start extraction - to run the task and get data

   · Click “Start Extraction” on the upper left side

   · Select “Local Extraction” to run the task on your computer, or select “Cloud Extraction” to run the task in the Cloud (for premium users only)

 

Here is the sample output. You can see some blank fields in the column “Hotel_Image_1” and column "All_Rooms_Include".  This is because some detail pages do not contain any image of the hotel and/or the information of room facilities.

 

 

Tips!

By default, if Octoparse cannot find the element of the defined pattern on the page, the field will be left blank. However, Octoparse may fail to find the element of the defined pattern even if the element needed is shown on the website. If you encounter this problem, here are a related tutorial you might need:

· What to do with those blank fields I got in the extracted result? 

 

 

 

Was this article helpful? Feel free to let us know if you have any question or need our assistance.

Contact us here  !

Download Octoparse to start web scraping or contact us for any
question about web scraping!

Contact us Download
btn_sidebar_use.png
btn_sidebar_form.png