You are browsing a tutorial guide for the latest Octoparse version. If you are running an older version of Octoparse, we strongly recommend you upgrade because it is faster, easier and more robust! Download and upgrade here if you haven't already done so!
A dropdown menu is a list of options that only appear when the user clicks on a piece of text or a button. It is often used for websites to offer a range of choices.
At times you may want to scrape the selected dropdown option as an individual data field, for example, when the drop-down options entail category information. In this article, we'll take a quick look at the different ways you can capture data on a drop-down list.
Tip: If you need guidance on how to switch between options in a drop-down menu and extract page data accordingly, check out this tutorial on how to click through options in a drop-down menu.
In Octoparse, we can either scrape the option values directly like any other data field on the webpage or scrape them from the loop item.
1. Scrape the option value directly like any other data field on the webpage
For this example, we are going to use a drown-drop list from MyCarInfo to show you how this is done.
Click the dropdown menu box
Select Loop through the options in the dropdown menu to set up a Switch Dropdown loop
Click the dropdown menu box again
Select Text
Untick Extract data in the loop and click Apply
You can iterate through the options in the loop list and click Extract data to see if you get the selected dropdown option in the data preview section.
2. Scrape the option value using a loop item
If in some cases the option value doesn't change even when they are selected, we'll need to scrape the option value using a loop item.
Click the dropdown menu box to set up a Switch Dropdown loop
Add an Extract Data action inside the loop
Tick Extract data in the loop and click Apply
Click on the dropdown menu and select the SELECT tag and then the first OPTION
Click Text
You can also switch between the different options from the loop to see if the data changes accordingly in the preview section.