Step-by-step tutorials for you to get started with web scraping

How to click through options in a drop-down menu?

Monday, December 02, 2019

The latest version for this tutorial is available here. Go to have a check now!

This tutorial will show you how to select options in a drop-down menu in Octoparse.

1) Click on the drop-down menu

2) From the Action Panel, click "Loop through options in the dropdown"

3) Switch to the Workflow Mode by toggling the Workflow switch on the upper right corner. A Loop Item had been created and added to the workflow automatically to loop through options in the drop-down menu.

4) Click on the Loop Item for the dropdown then refer to the looped items in the list on the right side. Check if all the items added to the loop were desired; If not, refine the list by using XPath function: position().

5) Now we are done configuring for the drop-down menu. Move on to select other options or click on the confirmation button to complete the search.

Tips!

When a Drop-down menu is detected and created in Octoparse, all available options will be selected by default. However, if you want to modify the list by adding or removing items, it can be done via using XPath function position() to specify which options are to be kept in the list.

In this example, our XPath for the drop-down loop is "//SELECT[@id='selTown']/OPTION"，by adding [position()="x"] to the end of the XPath, we can specify if we want to include only options of certain positions, ie. position( )=1, position( )=2, or position( )>1, etc. So, assuming we want to remove the first item from the list since it is not a real option but a header, we will use XPath: //SELECT[@id='selTown']/OPTION[position()>1] to include every single option with position greater than 1.

web scraping with octoparse - drop-down menus

PREVIOUS POST
Deal with AJAX in web scraping <

NEXT POST
Text/keyword input for keyword-based web scraping >

Download Octoparse to start web scraping or contact us for any
question about web scraping!