# DuckDuckGo Scraper

Automatically scrape search result data from DuckDuckGo using just keywords - get structured output (title, URL, description, site domain) ready for export in minutes.

- Template page: [https://www.octoparse.com/template/duckduckgo-scraper](https://www.octoparse.com/template/duckduckgo-scraper)
- Categories: Search Engine
- Status: `PUBLISHED`
- Run mode: `BOTH`

## Pricing

Free.

## What is an Octoparse template?

An Octoparse template is a ready-to-use scraping workflow with predefined inputs and structured outputs. Use the exact parameter keys below; do not invent parameters.

## Overview

This ready-to-use template collects structured data from search-results pages.

## Output

The current workflow extracts 5 fields, including `Description`, `Keyword`, `MainSite`, `ResultURL`, `Title`.

## Use Cases

- Use structured results for research, comparison, and monitoring.
- Collect rows without building the extraction workflow from scratch.
- Combine results with other templates for downstream enrichment.
- Review a representative result before starting a large batch.

## Why Octoparse

- **No coding required:** complete Keywords (up to 10,000), Number of Pages and start the template in local or cloud mode.
- **Flexible export:** review and export results as CSV, Excel, or JSON.

## Input

### Keywords (up to 10,000)

Required. Up to 10,000 per run, separate by line breaks. For large datasets, consider splitting them into multiple runs. Constraints: minItems=1; maxItems=10000; items.minLength=1; items.maxLength=5000.

### Number of Pages

Optional. Choose how many DuckDuckGo result pages to extract for each keyword. Constraints: minimum=1, maximum=100.

## How to Use

# DuckDuckGo Search Results Scraper (Octoparse Public Template)

## What It Does

Scrapes DuckDuckGo organic search results for up to 10,000 keywords per run. Extracts for each result: search keyword, title, URL, main site/domain, publication date, description/snippet.

## Input Parameters

| Parameter | Required? | Description | Example |
|-----------|-----------|-------------|---------|
| Keywords | Required | Up to 10,000 per run, separate by line breaks. For large datasets, split into multiple runs. | `Data analyst`  
`Web scraping` |
| Number of Pages | Required | Number of DuckDuckGo result pages to extract per keyword. | `1` (first page only); `3` (first 3 pages) |

## How to Use

1. **Start the template** – Click "Try it!" or "Start".
2. **Enter parameters** – Fill in keywords and number of pages.
3. **Run the scraper** – Click "Start" and select a run mode (greyed-out modes are not supported).
4. **Monitor & handle interruptions** – If a CAPTCHA appears, pause the task, solve it manually, then resume.
5. **Export data** – Go to "Data Preview" or output section; export as CSV, Excel, etc.

## Proxy Options

- **Octoparse built‑in proxy pool** (automatic rotation, no setup) – Check "Access websites via proxies", choose "Use Octoparse proxies". IPs rotate every 5 minutes (customizable). Credit‑based pricing.
- **Your own proxies** – Select "Use my own proxies", click "Configure", add your proxy list (HTTP, HTTPS, SOCKS5). Full control over rotation.

> For large‑scale scraping (1,000+ keywords), always use proxies to avoid rate limiting.

## Important Notes

- Scraper only accesses publicly visible DuckDuckGo organic search results – paid ads excluded by default. Knowledge graph panels require custom scraper.
- Legal compliance depends on how you use the data. Respect DuckDuckGo's terms of service and applicable laws.
- For large tasks, split keywords or schedule incremental runs to avoid blocks.
- Some search results may be dynamic; ensure your workflow handles pagination correctly.
- Free trial: 100 keywords. Paid plans: 10,000 per run, unlimited runs. Proxy usage may incur additional credits.
- Use cloud scheduler to automate daily, weekly, or custom interval runs with continuous proxy rotation.

## Limitations

Results depend on what the source site exposes at run time. Fields may be empty when a source page does not provide a value. The template enforces the input limits shown above.

## Tips

Use specific, valid inputs and review a representative result before starting a large batch. Remove duplicate inputs when repeated records are not needed.

## FAQ

### What do I need to enter?

Use the exact inputs and limits listed above.

### What data does this template return?

See Output Schema below; availability can vary by source record.

### How is usage billed?

See the Pricing section above for current pricing and billing details.

## Related Templates

- [eBay Listing Page Scraper (by keyword)](https://www.octoparse.com/template/ebay-product-list-scraper-by-keyword)
- [eBay Scraper](https://www.octoparse.com/template/ebay-scraper)
- [Google Search Scraper](https://www.octoparse.com/template/google-search-scraper)
- [Google Maps Listings Scraper (by Keywords)](https://www.octoparse.com/template/google-maps-scraper-listing-page-by-keyword)

## Input Schema

```json
{
  "title": "DuckDuckGo Scraper input",
  "description": "Input parameters configured for the current published template version.",
  "type": "object",
  "schemaVersion": 1,
  "additionalProperties": false,
  "properties": {
    "keywords": {
      "title": "Keywords (up to 10,000)",
      "description": "Up to 10,000 per run, separate by line breaks.\nFor large datasets, consider splitting them into multiple runs.",
      "type": "array",
      "items": {
        "type": "string",
        "minLength": 1,
        "maxLength": 5000
      },
      "minItems": 1,
      "maxItems": 10000,
      "x-octoparse": {
        "apiEnabled": true
      }
    },
    "number_of_Pages": {
      "title": "Number of Pages",
      "description": "Choose how many DuckDuckGo result pages to extract for each keyword.",
      "type": "integer",
      "minimum": 1,
      "maximum": 100,
      "x-octoparse": {
        "apiEnabled": true
      }
    }
  },
  "required": [
    "keywords"
  ]
}
```

## Input JSON Example

Example 1

```json
{
  "keywords": [
    "Help \"Ukraine\" Organization Fundraiser New York -news",
    "Help \"Ukraine\" Organization Fundraiser Los Angeles -news",
    "Help \"Ukraine\" Organization Fundraiser Chicago -news",
    "Help \"Ukraine\" Organization Fundraiser Houston -news",
    "Help \"Ukraine\" Organization Fundraiser Phoenix -news"
  ]
}
```

Example 2

```json
{
  "keywords": [
    "Help \"Ukraine\" Organization Fundraiser New York -news",
    "Help \"Ukraine\" Organization Fundraiser Los Angeles -news",
    "Help \"Ukraine\" Organization Fundraiser Chicago -news",
    "Help \"Ukraine\" Organization Fundraiser Houston -news",
    "Help \"Ukraine\" Organization Fundraiser Phoenix -news"
  ]
}
```

Example 3

```json
{
  "keywords": [
    "laptop"
  ],
  "number_of_Pages": 1
}
```

Example 4

```json
{
  "keywords": [
    "Help \"Ukraine\" Organization Fundraiser New York -news",
    "Help \"Ukraine\" Organization Fundraiser Los Angeles -news",
    "Help \"Ukraine\" Organization Fundraiser Chicago -news",
    "Help \"Ukraine\" Organization Fundraiser Houston -news",
    "Help \"Ukraine\" Organization Fundraiser Phoenix -news"
  ]
}
```

Example 5

```json
{
  "keywords": [
    "Help \"Ukraine\" Organization Fundraiser New York -news",
    "Help \"Ukraine\" Organization Fundraiser Los Angeles -news",
    "Help \"Ukraine\" Organization Fundraiser Chicago -news",
    "Help \"Ukraine\" Organization Fundraiser Houston -news",
    "Help \"Ukraine\" Organization Fundraiser Phoenix -news"
  ]
}
```


## Output Schema

```json
{
  "schemaVersion": 1,
  "type": "object",
  "properties": {
    "Description": {
      "type": "string",
      "description": ""
    },
    "Keyword": {
      "type": "string",
      "description": ""
    },
    "MainSite": {
      "type": "string",
      "description": ""
    },
    "ResultURL": {
      "type": "string",
      "description": ""
    },
    "Title": {
      "type": "string",
      "description": ""
    }
  },
  "title": "DuckDuckGo Scraper output",
  "description": "Initial output fields derived from the current published implementation. Field descriptions are maintained by Data Center; production field names and types should be refreshed from recent successful runs.",
  "additionalProperties": false
}
```

## Output Example

Example 1

```json
{
  "Description": "Highly rated charities involved in humanitarian relief, recovery, and peace-building efforts",
  "Keyword": "Help \"Ukraine\" Organization Fundraiser Memphis -news",
  "MainSite": "Only include results for this site",
  "ResultURL": "https://www.charitynavigator.org/discover-charities/where-to-give/ukranian-crisis/",
  "Title": "Ukraine Relief & Recovery - Charity Navigator"
}
```

Example 2

```json
{
  "Description": "After Russia began a military invasion of Ukraine, nearly 5 million Ukrainians have fled. IILA and our local communities are standing by to help refugees rebuild their lives. Donate now.",
  "Keyword": "Help \"Ukraine\" Organization Fundraiser Los Angeles -news",
  "MainSite": "Only include results for this site",
  "ResultURL": "https://iilosangeles.org/actnowforukraine/",
  "Title": "Act Now for Ukraine - IILA"
}
```

Example 3

```json
{
  "Description": "Shop Best Buy for laptops. Let us help you find the best laptop for you with our selection of laptop computers for work and play.",
  "Keyword": "laptop",
  "MainSite": "Only include results for this site",
  "ResultURL": "https://www.bestbuy.com/site/computers-pcs/laptop-computers/abcat0502000.c?id=abcat0502000",
  "Title": "Laptops & Notebook Computers - Best Buy"
}
```

Example 4

```json
{
  "Description": "Find ways you can reach out to help the people of Ukraine. UAST is a Non Profit Organization of Ukrainians in Texas promoting Ukrainian Culture and Traditions.",
  "Keyword": "Help \"Ukraine\" Organization Fundraiser Houston -news",
  "MainSite": "Only include results for this site",
  "ResultURL": "https://www.uast.org/help-ukraine",
  "Title": "Help Ukraine | UAST"
}
```

Example 5

```json
{
  "Description": "Mar 6, 2026Help Ukraine Now, San Diego. 637 likes · 1 talking about this · 4 were here. Help Ukraine Now is a non-profit organization based in San Diego, CA,...",
  "Keyword": "Help \"Ukraine\" Organization Fundraiser San Diego -news",
  "MainSite": "Only include results for this site",
  "ResultURL": "https://www.facebook.com/helpukrainenowsandiego/",
  "Title": "Help Ukraine Now | San Diego CA - Facebook"
}
```

## API

`POST https://openapi.octoparse.com/api/agentTools/executeTask`

This is the OpenAPI Agent Tools `executeTask` endpoint. Authenticate with a Bearer token.

Request body:

`templateName` is the template slug; `parameters` is a JSON object string whose keys match Input Schema; `taskName`, `idempotencyKey`, and `maxCollectedRows` are optional.

Obtain a token:

```http
POST https://openapi.octoparse.com/token
Content-Type: application/json

{
  "username": "YOUR_USERNAME",
  "password": "YOUR_PASSWORD",
  "grant_type": "password"
}
```

Use `data.access_token` as `Authorization: Bearer <token>`. After starting, poll `getTaskStatus` and export with `exportData`.

```json
{
  "templateName": "duckduckgo-scraper",
  "taskName": "DuckDuckGo Scraper",
  "parameters": "{\"keywords\":[\"Help \\\"Ukraine\\\" Organization Fundraiser New York -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Los Angeles -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Chicago -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Houston -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Phoenix -news\"]}"
}
```

## Python Example

```python
import json
import requests

OPENAPI_BASE = "https://openapi.octoparse.com"
ACCESS_TOKEN = "YOUR_ACCESS_TOKEN"

parameters = json.loads("{\"keywords\":[\"Help \\\"Ukraine\\\" Organization Fundraiser New York -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Los Angeles -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Chicago -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Houston -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Phoenix -news\"]}")

response = requests.post(
    f"{OPENAPI_BASE}/api/agentTools/executeTask",
    headers={
        "Authorization": f"Bearer {ACCESS_TOKEN}",
        "Content-Type": "application/json",
    },
    json={
        "templateName": "duckduckgo-scraper",
        "taskName": "DuckDuckGo Scraper",
        "parameters": json.dumps(parameters, ensure_ascii=False),
    },
    timeout=60,
)
print(response.status_code)
print(response.json())
```

## HTTP/cURL Example

```bash
curl -X POST "https://openapi.octoparse.com/api/agentTools/executeTask" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @- <<'JSON'
{
  "templateName": "duckduckgo-scraper",
  "taskName": "DuckDuckGo Scraper",
  "parameters": "{\"keywords\":[\"Help \\\"Ukraine\\\" Organization Fundraiser New York -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Los Angeles -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Chicago -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Houston -news\",\"Help \\\"Ukraine\\\" Organization Fundraiser Phoenix -news\"]}"
}
JSON
```

## MCP Setup

Connect an MCP-compatible AI client to `https://mcp.octoparse.com` using browser OAuth or an Octoparse API key. Find `duckduckgo-scraper`, inspect its inputs, create a cloud task with the exact Input Schema keys, and export JSON results.

## Machine-readable Resources

- [Template contract](https://www.octoparse.com/templates/duckduckgo-scraper.json)
- [Manifest](https://www.octoparse.com/templates/duckduckgo-scraper-manifest.json)
