Indeed Job Scraper: 3 Methods and a 946-Job Test (2026)

Indeed is the most-visited job site in the world, with a job count reported in the tens of millions at any given time. What it does not give you is an export button. If you want that data in a spreadsheet, you are stuck copying fields one posting at a time, and the results page reloads or blocks you long before you finish.

Indeed search results flowing through a scraper into structured job data

An Indeed job scraper solves that. It reads Indeed search results and writes each posting into a structured file: job title, company, salary, location, post date, and the apply link. This guide covers three ways to do it, from a no-code template to a custom Python script, and shows you which one fits your project.

Is There a Public Indeed API for Downloading Job-Search Results?

No. Indeed’s documented APIs support approved partner integrations rather than public, self-service job-data exports. For an authorized data project, the three practical routes are a ready-made Octoparse template for no-code extraction, a Python workflow for custom logic, or Indeed’s partner APIs for approved ATS and job-posting integrations.

• Method 1 — Octoparse template: Enter the role and location, then inspect and export structured job data without coding.
• Method 2 — Python workflow: Build custom fields and processing logic while maintaining the browser, selectors, and access controls yourself.
• Method 3 — Indeed partner APIs: Apply through Indeed’s partner program when you need an approved ATS or job-posting integration.

https://www.octoparse.com/template/indeed-job-scraper-by-keywords

What data can you extract from Indeed?

An Indeed job scraper pulls the fields that appear on a search results page and inside each job posting. A typical run returns the columns below.

 

That structure lets you sort by salary, filter by post date, or group by company as soon as the export lands.

3 ways to scrape Indeed jobs, compared

 

Is it legal to scrape Indeed?

Scraping publicly visible job listings sits on firmer ground than scraping data behind a login. Two US rulings shaped that line:

• Van Buren v. United States (Supreme Court, June 2021, 6-3). The Court narrowed the Computer Fraud and Abuse Act: using access you already have for a purpose the site dislikes is not “exceeding authorized access.”
• hiQ Labs v. LinkedIn (Ninth Circuit). After the Supreme Court sent it back in light of Van Buren, the Ninth Circuit reaffirmed in April 2022 that scraping public profile data does not violate the CFAA. A separate 2022 ruling still found hiQ had breached LinkedIn’s user agreement, which is a contract question, not a criminal one.

The practical takeaways: public pages carry less legal risk than logged-in data, a site’s Terms of Service is a contract you can still breach, personal data pulls extra rules (GDPR, CCPA), and country law varies. Read Indeed’s current terms, collect only the fields you need, and see our overview of whether web crawling is legal. This is not legal advice.

Method 1: Scrape Indeed with Octoparse (no code)

Octoparse is a no-code web scraper used by more than 3 million people, per its about page. Its prebuilt “Indeed Job Scraper” template does the parsing and the anti-bot work for you, so you only supply the search you want.

When you need job data this afternoon, not next sprint

The template takes three inputs:

1. Country (required): pick the Indeed domain from the dropdown, for example United States or United Kingdom.
2. Search Term (required): the keywords you would type into Indeed, such as data analyst. You can enter up to 20 terms, one per line, to run them in a single job.
3. Location (optional): a city, state, or region. Leave it blank to include every location.

Then run these steps:

https://www.octoparse.com/template/indeed-job-scraper-by-keywords

1. Open the Indeed Job Scraper template and click Try it.
2. Sign in or create a free account.
3. Fill in Country, Search Term, and Location.
4. Choose Run in the Cloud so the task uses Octoparse servers and rotating IPs.
5. Wait for the run to finish, then export the results as CSV, Excel, or JSON.

 

 

When the same report is due every Monday

Set the task to a schedule (daily, weekly, or a custom time) and Octoparse re-runs it in the cloud without you opening the app. Each run appends fresh postings, so you get a rolling record of what a company or a role pays over time. Pair it with a webhook or the Octoparse API if you want the file pushed straight into a database.

Hands-on test: run Indeed through Octoparse MCP

On September 10, 2026, we connected Codex to the official Octoparse MCP server and followed the documented sequence: search_templates, execute_task, get_task_status, and export_data. Discovery returned the Cloud-capable indeed-job-scraper-by-keywords template (template ID 1107).

Follow the official Octoparse MCP setup guide

For a small, reproducible run, we submitted these inputs once:

• Country: United States
• Search term: data analyst
• Location: New York, NY
• Unique task name: Indeed MCP article evidence 2026-09-10 1029 CST

We reused the same task rather than starting a duplicate. A fresh get_task_status call returned completed with 946 collected rows.

We then called export_data for page 1 with a page size of 2. The response reported 946 total rows across 473 pages. The visible sample contained two populated New York records: “Inverto | Senior Consultant, Digital, Data & AI” at Boston Consulting Group, and “Programmatic Yield Analyst” at Vaco LLC. Both included a location and salary value.

The template schema included Job_Title, Company_Name, Location, Posted_Date, Salary, Job_Type, Job_URL, Apply_Link, and Full_Description. Job listings change over time, so these values document the September 10, 2026 run rather than promise a permanent row count. The MCP responses did not expose a trustworthy end-to-end runtime, so this is a verified completion-and-output test, not a speed benchmark.

https://www.octoparse.com/template/indeed-job-scraper-by-keywords

Cost

The free plan includes 50,000 rows of data export per month and 10 tasks, which is enough for most single-market job studies. The Standard plan is $69 per month for higher volume and more concurrent cloud runs, and Professional is $249 per month.

Method 2: Scrape Indeed with Python

A custom script makes sense when you need parsing logic the template does not offer, or when Indeed data feeds a larger pipeline you already run in Python.

The catch is Indeed’s defenses. The site runs Cloudflare bot protection, so a plain requests call usually returns a challenge page. The job data itself is not in the visible HTML but in an embedded JSON blob: a mosaic-provider-jobcards variable on search pages and an _initialData variable on individual job pages. Each search page holds about 15 results, and pagination moves in offsets of 10. Most working scrapers drive a real browser to get past Cloudflare, then pull that JSON.

from playwright.sync_api import sync_playwright

def scrape_indeed(query, location, pages=1):
    rows = []
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()
        for i in range(pages):
            url = f"https://www.indeed.com/jobs?q={query}&l={location}&start={i*10}"
            page.goto(url, wait_until="domcontentloaded")
            for card in page.query_selector_all("div.job_seen_beacon"):
                rows.append({
                    "title": card.query_selector("h2 span").inner_text(),
                    "company": card.query_selector("[data-testid='company-name']").inner_text(),
                    "location": card.query_selector("[data-testid='text-location']").inner_text(),
                })
        browser.close()
    return rows

This sketch works on a small scale, then breaks when Indeed rotates its markup or rate-limits your IP. To run it in production you also need:

• Residential or mobile proxies with rotation, so requests do not all come from one address. See our list of web scraping proxy providers.
• Randomized delays and headers to avoid a fixed request signature.
• A CAPTCHA fallback, because challenge pages will appear.
• Selector monitoring, since a layout change silently returns empty rows.

Budget for ongoing maintenance, not a one-time build.

Method 3: Indeed API Access and Alternatives

As of September 10, 2026, Indeed’s current Partner API catalog does not list a public, self-service Indeed API for searching or downloading public job listings. Its documented APIs instead support approved integrations for managing jobs, applications, candidates, and employer data.

The Job Sync API, for example, is a GraphQL API for applicant tracking system (ATS) partners to create, update, expire, and check the status of jobs they send to Indeed. It manages a partner’s postings; it is not a search endpoint for collecting Indeed results.

Official access follows Indeed’s partner process. Indeed says a company must sign a Developer Agreement and submit a formal integration request before access is enabled. Approved partners receive an app and OAuth credentials through Partner Console, as described in Indeed’s integration roadmap and authentication guide.

The Octoparse API and Model Context Protocol (MCP) are separate. Indeed’s APIs connect approved partner systems to Indeed services. Octoparse API/MCP runs a supported Octoparse template or task and returns that task’s output; it does not issue an Indeed API key or provide Indeed partner access. For an official ATS or job-posting integration, use Indeed’s partner route. For authorized research data, use a collection workflow that respects Indeed’s current terms and robots rules.

How to scrape Indeed without getting blocked

Indeed’s Cloudflare layer blocks automated traffic that looks nothing like a person. These practices keep a job on the safe side:

• Rotate IPs. Send requests from a pool of residential addresses, not one server IP.
• Throttle. Space requests several seconds apart and cap total volume per hour.
• Render like a browser. Use a headless browser with real headers so the page loads its dynamic content.
• Scrape during off-peak hours in the target country to blend into lower traffic.
• Stop on challenge pages. If you hit repeated CAPTCHAs, pause that IP rather than hammering through.

A cloud tool like the Octoparse template applies most of this for you, which is why the no-code route needs no proxy setup.

What to do with Indeed job data

Once the export lands, the structured fields open up work that is slow to do by hand. A few concrete uses:

• Salary benchmarking. Scrape every “data analyst” posting across three target metros, then compare the median of the Salary column before you set a band for a new hire or ask for a raise. Postings without a listed salary are easy to filter out first.
• Hiring-trend tracking. Scrape a competitor’s company page every Monday. A jump from 5 to 40 open engineering roles over a quarter is a visible signal that they are standing up a new team.
• Recruiting lead lists. Pull every company hiring for a role you place, keep the Company_Name, Company_URL, and Apply_Link columns, and you have a prospect list with the hiring need already attached.
• Market research. Join Indeed data with listings from other boards for a fuller picture. Our guide on scraping LinkedIn job data covers the companion source.

FAQs about Indeed Job Scraping

Does Indeed allow scraping?

Indeed’s current Site Rules prohibit using bots, scrapers, spiders, AI, or agentic AI to access or data-mine the site without express written permission, with conditional crawling allowed only as described in Indeed’s robots.txt file. A project’s legal position also depends on applicable law, contracts, and the data collected. Obtain permission where required, limit collection to necessary fields, and avoid personal data. This is not legal advice.

How does Indeed itself get its job postings?

Indeed’s job-source explanation identifies two routes: jobs employers post directly on Indeed and jobs taken from external sources such as employers’ websites. For approved integrations, the Indeed Partner Docs describe supported Job Sync and XML-feed routes.

Can you scrape Indeed with Python?

Python can automate a browser through Playwright’s official Python library, but technical capability is not authorization. Indeed’s Site Rules restrict automated access without express written permission and reserve the use of technical measures to prevent it. For an authorized project, test a small sample, validate the output, and stop when access controls appear rather than attempting to bypass them.

How do I scrape Indeed for free?

The current Octoparse pricing page lists a free plan with 10 tasks, local-device runs, and up to 50,000 rows of monthly export, capped at 10,000 rows per export. Cloud extraction and scheduling are paid-plan features. A preset template may have separate usage terms, so check its template page before starting a run.

How often can I scrape Indeed?

There is no universal safe frequency. Indeed’s Site Rules govern whether automated access is permitted and do not publish a request threshold. For an authorized workflow, the Octoparse scheduling guide documents minute, hourly, daily, weekly, and monthly schedules on eligible plans. Use the least frequent schedule that meets your freshness requirement, avoid unnecessary requests, and validate every run.

Can I export Indeed jobs to Excel?

Yes. Octoparse’s supported export formats include Excel, CSV, HTML, JSON, and XML for both local and cloud datasets. In Excel, you can then filter by posted date, compare available salary fields, or group results by company.

Why does Indeed block scrapers?

Indeed’s Site Rules reserve the right to use technical measures to detect, investigate, or prevent unauthorized automated access. Indeed does not publish exact blocking thresholds or signals, so a fixed request rate, IP type, or CAPTCHA trigger should not be presented as an official rule. If an access challenge appears, stop and confirm authorization and workflow settings rather than trying to bypass it.

Ready to pull Indeed data?

The no-code template is the fastest path for most people: open the Indeed Job Scraper, enter your search, and export the file. Browse the full template library for other job boards and directories.