Blog > Big Data > Post

How to Generate Sales Leads Using Web Scraping

Friday, January 15, 2021

Put it simply, leads are people who have shown interest in your products or service. Every sale starts with a lead. When businesses want to make more profits, chances are they need to keep generating more leads.

But, it's not all that easy!

Table of Contents

Web Scraping for Lead Generation

Enter Web Scraping

Lead Generation with Octoparse

Scrape Leads Across Multiple Pages

Scraping Qualified Leads at Scale

In fact, 61% of B2B marketers think generating high-quality leads is one of their biggest challenges (IDG) and 68% of businesses report struggling with lead generation. (CSO Insights) There are literally hundreds of posts about why lead generation is important and I'll bet you already know about some of the old school ways to get leads - content marketing, email, social media, PPC ads and more. Most of them work nicely (forget about buying a list of prospectives), but they do take a great deal of time and investment.

What if I tell you there's one trick that works wonders and can get you thousands of leads almost instantly?

how to generate sales leads using web scraping

Web Scraping for Lead Generation

The internet has changed the way we do business. In fact, people generate 2.5 quintillion bytes of data each day (IBM, 2016) and more than 150 zettabytes (150 trillion gigabytes) of data will need analysis by 2025. (Forbes, 2019)

While these numbers may sound astounding, it's not hard to believe how data can be leveraged for growing businesses.

Consider this scenario:

Jerry runs a startup company that sells auto parts to auto repair shops. In order to reach out to his prospective customers and makes more sales, he needs to develop a pipeline with sales qualify leads. He needs details, such as the name of the auto repair shop, the street address, the contact number, the emails, and other basic information. Jerry is smart and knows that he could get this information online. He googled "Auto repair shops in the US" and soon found out there are online directories like Yelp and Yellowpage which has exactly the data he needs.

But wait, how can Jerry grab all that information from the webpages? He surely does not want to do it manually! Web scraping is the trick.

Enter Web Scraping

Web scraping is the process of fetching web data and turning it into structured data that can be easily used/analyzed. There are many ways to approach web scraping - write a script, hire a scraping service or use a coding-free automatic web scraper.

While there are a handful of web scraping tools to choose from, I'll show you how to extract the business leads Jerry needs using my favorite web scraper, Octoparse.

Lead Generation with Octoparse

First thing first, download Octoparse and launch it.

Step 1: Click the plus sign under Advanced Mode to start a new project

Step 2: Find the webpage URL to scrape from

Enter "www.yellowpages.com" in your everyday browser and search for "Auto Repair Shop" near "New York".

Step 3: Copy and paste the URL of the search result page to Octoparse

The webpage should then be loaded in Octoparse's built-in browser.

Step 4: Click data you want to scrape

Click on the first auto shop in the list, it will turn green, indicating that's been selected.

click data you want to scrape from yellow page

Step 5: Select all data across the list

The rest of the business names in the list will turn red, which indicates they have been identified as similar elements. To get all the business names selected, simply click on "Select all" on the "Action Tips" panel. Upon selected, all business names will be highlighted in green.

Step 6: Click into detail pages

We want to click into each single business page to capture the data we need, so click "Loop click each URL" on the "Action Tips" panel.

Step 7: Select detail data

Octoparse will now open the detail page for the first business on the list. Let's go ahead and extract the data we need. Click the name of the business. Select "Extract of the selected element."

Step 8: Repeat these steps to get more data

Repeat the same steps for extracting address, website, and other desired information from the page. Octoparse will get the same information from each business page automatically.

Scrape Leads Across Multiple Pages

We obviously don't want to scrape just the first page but all pages.

Step 1: Go back to the list page by clicking on "Go to webpage" of the workflow

Step 2: Once the list page's loaded, scroll down to the bottom of the page and click on the next page button

Step 3: Select "Loop click next page" on the "Action Tips" panel

Step 4: Just one last tweak - drag the first loop item to the inside of the newly created pagination loop. This is to make sure we'll loop through all businesses listed on the page before moving on to the next page.

Step 5: Now we are ready to start getting some data. Click , select "Local extraction" and Octoparse will collect the information needed into a CSV/Excel spreadsheet.

extracting yellow page data with octoparse

Scraping Qualified Leads at Scale

Leads are important, but you know what's more important?

Qualified leads！

Using the same method, you can scrape social platforms like Twitter, Instagram, different kinds of directories like Yelp and Yellowpage, real estate portals like Zillow and Realtor.com, or recruitment portals like Indeed. The possibilities are endless and the best part is you get to decide where to get the leads from, which is much more likely to give you qualified leads - leads to become successful customers.

This is not the end...

Using Octoparse's task scheduling feature, you can repeat the scraping job every day/week/month to keep your lead database fresh and up-to-date.

Moreover, there are some other ways to use web scraper for lead generation

Track your competition
Create directories
Scrape event and conference websites
More...

Do you already have a website in mind to scrape?

Si desea ver este blog en español, por favor haga clic en:¿Cómo Generar Clientes Potenciales de Ventas Usando El Web Scraping?

PREVIOUS POST
Amazon Scraper: Monitor Your Business Anytime <

NEXT POST
How to Scrape Product Data from Amazon without Coding Skills? >

Most popular posts

Posts by topic

Download Octoparse to start web scraping or contact us for any
question about web scraping!