# Twitter (X) Comments Scraper

Scrape tweet content, author, tweet post time, reply content, reply sender, reply time, etc from Twitter by search result URL.

- Template page: [https://www.octoparse.com/template/tweets-comments-scraper-by-search-result-url](https://www.octoparse.com/template/tweets-comments-scraper-by-search-result-url)
- Categories: Social Media
- Status: `PUBLISHED`
- Run mode: `CLOUD`

## Pricing

Current price: **$0.3/1,000 lines**. Billing is based on output rows; an Octoparse subscription may be required.

## What is an Octoparse template?

An Octoparse template is a ready-to-use scraping workflow with predefined inputs and structured outputs. Use the exact parameter keys below; do not invent parameters.

## Overview

This ready-to-use template collects structured data from search-results pages, comment pages, profile pages.

## Output

The current workflow extracts 23 fields, including `Author_Name`, `Author_Web_Page_URL`, `Comment_Author_URL`, `Comment_Author_name`, `Comment_Content`, `Comment_Image_URL`, `Comment_Likes`, `Comment_Replies`, `Comment_Retweets`, `Comment_Timestamp`, `Comment_View`, `Comment_Website`, `Error_MSG`, `Is_Deleted`, `Tweet_Bookmarked`, `Tweet_Content`, `Tweet_ID`, `Tweet_Image_URL`, `Tweet_Likes`, `Tweet_Posted_Time_UTC`, `Tweet_Replies`, `Tweet_Retweets`, `Tweet_Website`.

## Use Cases

- Use structured results for research, comparison, and monitoring.
- Collect rows without building the extraction workflow from scratch.
- Combine results with other templates for downstream enrichment.
- Review a representative result before starting a large batch.

## Why Octoparse

- **No coding required:** complete Tweet URLs (up to 100,000), Number of replies (up to 100,000) and start the template in cloud mode.
- **Flexible export:** review and export results as CSV, Excel, or JSON.

## Input

### Tweet URLs (up to 100,000)

Required. MainKeys Constraints: minItems=1; maxItems=100000; items.minLength=1; items.maxLength=1000.

### Number of replies (up to 100,000)

Optional. Number_of_replies (up to 100,000) Constraints: minimum=0, maximum=100000.

## How to Use

# Twitter (X) Comments Scraper

Collects the original public X post and its replies from direct status URLs. It follows configured page loops on X (formerly Twitter) search-results and comments pages and returns structured rows. Data source: [X (formerly Twitter)](https://x.com/).

**Price:** **$0.3/1,000 lines**. Billing is based on the number of output lines produced by the task.

## Input

- **Tweet URLs** (Required) — Up to 100,000 entries per run.
- **Number of replies** (Optional) — Accepted range: 0 to 100000.

## How to use

1. Open Twitter (X) Comments Scraper and click **Try it!**.
2. Complete **Tweet URLs** and **Number of replies** using the formats and limits above.
3. Run it in the Octoparse cloud and start the task.
4. Review fields such as `Tweet_Website`, `Tweet_ID`, `Author_Name`, `Author_Web_Page_URL`, `Tweet_Posted_Time_UTC`, and `Comment_Website`, then export the rows in the format you need.

## Output

The dataset includes post and reply content, author details, timestamps, media, engagement metrics, and reply activity. Keep key fields such as `Author_Web_Page_URL` in exports so rows can be traced to their originating input or page.

## Tips

- Test one representative value in **Tweet URLs** before submitting a large batch.
- Use `Tweet_Content` and `Tweet_Image_URL` to study post topics and creative formats.
- Use `Comment_Content`, `Comment_Likes`, `Comment_Replies`, and `Comment_View` to study reply sentiment and response activity.
- Use `Tweet_Likes`, `Tweet_Retweets`, `Tweet_Replies`, and `Tweet_Bookmarked` to compare engagement.

## Limitations

Results depend on what the source site exposes at run time. Fields may be empty when a source page does not provide a value. The template enforces the input limits shown above.

## Tips

Use specific, valid inputs and review a representative result before starting a large batch. Remove duplicate inputs when repeated records are not needed.

## FAQ

### What do I need to enter?

Use the exact inputs and limits listed above.

### What data does this template return?

See Output Schema below; availability can vary by source record.

### How is usage billed?

See the Pricing section above for current pricing and billing details.

## Related Templates

- [Twitter Advanced Search Comments Scraper](https://www.octoparse.com/template/twitter-advanced-search-comments-scraper)
- [Twitter Advanced Search Scraper](https://www.octoparse.com/template/twitter-scraper-by-keywords)

## Input Schema

```json
{
  "title": "Twitter (X) Comments Scraper input",
  "description": "Input parameters configured for the current published template version.",
  "type": "object",
  "schemaVersion": 1,
  "additionalProperties": false,
  "properties": {
    "tweet_URLs": {
      "title": "Tweet URLs (up to 100,000)",
      "description": "MainKeys",
      "type": "array",
      "items": {
        "type": "string",
        "minLength": 1,
        "maxLength": 1000
      },
      "minItems": 1,
      "maxItems": 100000,
      "x-octoparse": {
        "apiEnabled": true
      }
    },
    "number_of_replies": {
      "title": "Number of replies (up to 100,000)",
      "description": "Number_of_replies (up to 100,000)",
      "type": "integer",
      "minimum": 0,
      "maximum": 100000,
      "x-octoparse": {
        "apiEnabled": true
      }
    }
  },
  "required": [
    "tweet_URLs"
  ]
}
```

## Input JSON Example

Example 1

```json
{
  "number_of_replies": 100.0,
  "tweet_URLs": [
    "https://x.com/SpaceX/status/1827663529543168152"
  ]
}
```

Example 2

```json
{
  "number_of_replies": 100.0,
  "tweet_URLs": [
    "https://x.com/SpaceX/status/1827663529543168152"
  ]
}
```

Example 3

```json
{
  "number_of_replies": 100,
  "tweet_URLs": [
    "https://x.com/DanScavino/status/1828615095586357466"
  ]
}
```

Example 4

```json
{
  "number_of_replies": 100,
  "tweet_URLs": [
    "https://x.com/SpaceX/status/1827663529543168152"
  ]
}
```

Example 5

```json
{
  "tweet_URLs": [
    "https://x.com/KaiCenat/status/1841654445257441415"
  ]
}
```


## Output Schema

```json
{
  "schemaVersion": 1,
  "type": "object",
  "properties": {
    "Author_Name": {
      "type": "string",
      "description": ""
    },
    "Author_Web_Page_URL": {
      "type": "string",
      "description": ""
    },
    "Comment_Author_URL": {
      "type": "string",
      "description": ""
    },
    "Comment_Author_name": {
      "type": "string",
      "description": ""
    },
    "Comment_Content": {
      "type": "string",
      "description": ""
    },
    "Comment_Image_URL": {
      "type": "string",
      "description": ""
    },
    "Comment_Likes": {
      "type": "string",
      "description": ""
    },
    "Comment_Replies": {
      "type": "string",
      "description": ""
    },
    "Comment_Retweets": {
      "type": "string",
      "description": ""
    },
    "Comment_Timestamp": {
      "type": "string",
      "description": ""
    },
    "Comment_View": {
      "type": "string",
      "description": ""
    },
    "Comment_Website": {
      "type": "string",
      "description": ""
    },
    "Error_MSG": {
      "type": "string",
      "description": ""
    },
    "Is_Deleted": {
      "type": "string",
      "description": ""
    },
    "Tweet_Bookmarked": {
      "type": "string",
      "description": ""
    },
    "Tweet_Content": {
      "type": "string",
      "description": ""
    },
    "Tweet_ID": {
      "type": "string",
      "description": ""
    },
    "Tweet_Image_URL": {
      "type": "string",
      "description": ""
    },
    "Tweet_Likes": {
      "type": "string",
      "description": ""
    },
    "Tweet_Posted_Time_UTC": {
      "type": "string",
      "description": ""
    },
    "Tweet_Replies": {
      "type": "string",
      "description": ""
    },
    "Tweet_Retweets": {
      "type": "string",
      "description": ""
    },
    "Tweet_Website": {
      "type": "string",
      "description": ""
    }
  },
  "title": "Twitter (X) Comments Scraper output",
  "description": "Initial output fields derived from the current published implementation. Field descriptions are maintained by Data Center; production field names and types should be refreshed from recent successful runs.",
  "additionalProperties": false
}
```

## Output Example

Example 1

```json
{
  "Author_Name": "SpaceX",
  "Author_Web_Page_URL": "https://x.com/SpaceX",
  "Comment_Author_URL": "https://x.com/gamecruzee",
  "Comment_Author_name": "gamecruzee",
  "Comment_Content": "@SpaceX @PolarisProgram I wish https://t.co/kXlTpoELUB",
  "Comment_Image_URL": "https://pbs.twimg.com/media/GWG9XDyXcAAW0Eh.jpg",
  "Comment_Likes": "0",
  "Comment_Replies": "0",
  "Comment_Retweets": "0",
  "Comment_Timestamp": "Thu Aug 29 00:21:44 +0000 2024",
  "Comment_View": "43",
  "Comment_Website": "https://x.com/gamecruzee/status/1828951129901699437",
  "Error_MSG": "",
  "Is_Deleted": "False",
  "Tweet_Bookmarked": "468",
  "Tweet_Content": "Polaris Dawn and SpaceX completed a full rehearsal of launch day activities ahead of liftoff on Tuesday https://t.co/qz2RD5bnVM",
  "Tweet_ID": "1827663529543168152",
  "Tweet_Image_URL": "https://pbs.twimg.com/media/GV0pkBWbsAAW6XI.jpg|https://pbs.twimg.com/media/GV0plqCa0AAZspR.jpg|https://pbs.twimg.com/media/GV0poHpbMAAYYjG.jpg|https://pbs.twimg.com/media/GV0ppysWEAAeRHK.jpg",
  "Tweet_Likes": "18751",
  "Tweet_Posted_Time_UTC": "Sun Aug 25 11:05:16 +0000 2024",
  "Tweet_Replies": "828",
  "Tweet_Retweets": "2636",
  "Tweet_Website": "https://x.com/SpaceX/status/1827663529543168152"
}
```

Example 2

```json
{
  "Author_Name": "SpaceX",
  "Author_Web_Page_URL": "https://x.com/SpaceX",
  "Comment_Author_URL": "https://x.com/gamecruzee",
  "Comment_Author_name": "gamecruzee",
  "Comment_Content": "@SpaceX @PolarisProgram I wish https://t.co/kXlTpoELUB",
  "Comment_Image_URL": "https://pbs.twimg.com/media/GWG9XDyXcAAW0Eh.jpg",
  "Comment_Likes": "0",
  "Comment_Replies": "0",
  "Comment_Retweets": "0",
  "Comment_Timestamp": "Thu Aug 29 00:21:44 +0000 2024",
  "Comment_View": "43",
  "Comment_Website": "https://x.com/gamecruzee/status/1828951129901699437",
  "Error_MSG": "",
  "Is_Deleted": "False",
  "Tweet_Bookmarked": "468",
  "Tweet_Content": "Polaris Dawn and SpaceX completed a full rehearsal of launch day activities ahead of liftoff on Tuesday https://t.co/qz2RD5bnVM",
  "Tweet_ID": "1827663529543168152",
  "Tweet_Image_URL": "https://pbs.twimg.com/media/GV0pkBWbsAAW6XI.jpg|https://pbs.twimg.com/media/GV0plqCa0AAZspR.jpg|https://pbs.twimg.com/media/GV0poHpbMAAYYjG.jpg|https://pbs.twimg.com/media/GV0ppysWEAAeRHK.jpg",
  "Tweet_Likes": "18747",
  "Tweet_Posted_Time_UTC": "Sun Aug 25 11:05:16 +0000 2024",
  "Tweet_Replies": "828",
  "Tweet_Retweets": "2636",
  "Tweet_Website": "https://x.com/SpaceX/status/1827663529543168152"
}
```

Example 3

```json
{
  "Author_Name": "Dan Scavino Jr.🇺🇸🦅",
  "Author_Web_Page_URL": "https://x.com/DanScavino",
  "Comment_Author_URL": "https://x.com/JuliaSp68139555",
  "Comment_Author_name": "JuliaSp68139555",
  "Comment_Content": "@DanScavino @realDonaldTrump @DrPhil #TrumpIsNotFitToBePresident  #MAGA24 is declining sharply. 📉 https://t.co/xLvCjhHVlO",
  "Comment_Image_URL": "https://pbs.twimg.com/media/GWMTN9gXgAAzXC_.jpg",
  "Comment_Likes": "0",
  "Comment_Replies": "0",
  "Comment_Retweets": "0",
  "Comment_Timestamp": "Fri Aug 30 01:15:20 +0000 2024",
  "Comment_View": "16",
  "Comment_Website": "https://x.com/JuliaSp68139555/status/1829327007622029596",
  "Error_MSG": "",
  "Is_Deleted": "False",
  "Tweet_Bookmarked": "1434",
  "Tweet_Content": "A MUST WATCH INTERVIEW, @realDonaldTrump and @DrPhil from Las Vegas, Nevada…https://t.co/OGRVbaeq9S",
  "Tweet_ID": "1828615095586357466",
  "Tweet_Image_URL": "https://pbs.twimg.com/media/GWBrPk7XsAAmq82.jpg",
  "Tweet_Likes": "7801",
  "Tweet_Posted_Time_UTC": "Wed Aug 28 02:06:27 +0000 2024",
  "Tweet_Replies": "233",
  "Tweet_Retweets": "3636",
  "Tweet_Website": "https://x.com/DanScavino/status/1828615095586357466"
}
```

Example 4

```json
{
  "Author_Name": "SpaceX",
  "Author_Web_Page_URL": "https://x.com/SpaceX",
  "Comment_Author_URL": "https://x.com/gamecruzee",
  "Comment_Author_name": "gamecruzee",
  "Comment_Content": "@SpaceX @PolarisProgram I wish https://t.co/kXlTpoELUB",
  "Comment_Image_URL": "https://pbs.twimg.com/media/GWG9XDyXcAAW0Eh.jpg",
  "Comment_Likes": "0",
  "Comment_Replies": "0",
  "Comment_Retweets": "0",
  "Comment_Timestamp": "Thu Aug 29 00:21:44 +0000 2024",
  "Comment_View": "43",
  "Comment_Website": "https://x.com/gamecruzee/status/1828951129901699437",
  "Error_MSG": "",
  "Is_Deleted": "False",
  "Tweet_Bookmarked": "468",
  "Tweet_Content": "Polaris Dawn and SpaceX completed a full rehearsal of launch day activities ahead of liftoff on Tuesday https://t.co/qz2RD5bnVM",
  "Tweet_ID": "1827663529543168152",
  "Tweet_Image_URL": "https://pbs.twimg.com/media/GV0pkBWbsAAW6XI.jpg|https://pbs.twimg.com/media/GV0plqCa0AAZspR.jpg|https://pbs.twimg.com/media/GV0poHpbMAAYYjG.jpg|https://pbs.twimg.com/media/GV0ppysWEAAeRHK.jpg",
  "Tweet_Likes": "18746",
  "Tweet_Posted_Time_UTC": "Sun Aug 25 11:05:16 +0000 2024",
  "Tweet_Replies": "828",
  "Tweet_Retweets": "2636",
  "Tweet_Website": "https://x.com/SpaceX/status/1827663529543168152"
}
```

Example 5

```json
{
  "Author_Name": "Kai Cenat",
  "Author_Web_Page_URL": "https://x.com/KaiCenat",
  "Comment_Author_URL": "https://x.com/CheezeGoinCrazy",
  "Comment_Author_name": "CheezeGoinCrazy",
  "Comment_Content": "@KaiCenat Nigga u ah tether stop copying our culture u cosplaying goofy",
  "Comment_Image_URL": "",
  "Comment_Likes": "0",
  "Comment_Replies": "0",
  "Comment_Retweets": "0",
  "Comment_Timestamp": "Sun Oct 27 05:31:36 +0000 2024",
  "Comment_View": "72",
  "Comment_Website": "https://x.com/CheezeGoinCrazy/status/1850409995168289242",
  "Error_MSG": "",
  "Is_Deleted": "False",
  "Tweet_Bookmarked": "2385",
  "Tweet_Content": "Back to streaming https://t.co/OO71tXcK9U",
  "Tweet_ID": "1841654445257441415",
  "Tweet_Image_URL": "https://pbs.twimg.com/media/GY7e9RHXUAA6JrP.jpg",
  "Tweet_Likes": "163378",
  "Tweet_Posted_Time_UTC": "Thu Oct 03 01:40:10 +0000 2024",
  "Tweet_Replies": "1015",
  "Tweet_Retweets": "4935",
  "Tweet_Website": "https://x.com/KaiCenat/status/1841654445257441415"
}
```

## API

`POST https://openapi.octoparse.com/api/agentTools/executeTask`

This is the OpenAPI Agent Tools `executeTask` endpoint. Authenticate with a Bearer token.

Request body:

`templateName` is the template slug; `parameters` is a JSON object string whose keys match Input Schema; `taskName`, `idempotencyKey`, and `maxCollectedRows` are optional.

Obtain a token:

```http
POST https://openapi.octoparse.com/token
Content-Type: application/json

{
  "username": "YOUR_USERNAME",
  "password": "YOUR_PASSWORD",
  "grant_type": "password"
}
```

Use `data.access_token` as `Authorization: Bearer <token>`. After starting, poll `getTaskStatus` and export with `exportData`.

```json
{
  "templateName": "tweets-comments-scraper-by-search-result-url",
  "taskName": "Twitter (X) Comments Scraper",
  "parameters": "{\"number_of_replies\":100.0,\"tweet_URLs\":[\"https://x.com/SpaceX/status/1827663529543168152\"]}"
}
```

## Python Example

```python
import json
import requests

OPENAPI_BASE = "https://openapi.octoparse.com"
ACCESS_TOKEN = "YOUR_ACCESS_TOKEN"

parameters = json.loads("{\"number_of_replies\":100.0,\"tweet_URLs\":[\"https://x.com/SpaceX/status/1827663529543168152\"]}")

response = requests.post(
    f"{OPENAPI_BASE}/api/agentTools/executeTask",
    headers={
        "Authorization": f"Bearer {ACCESS_TOKEN}",
        "Content-Type": "application/json",
    },
    json={
        "templateName": "tweets-comments-scraper-by-search-result-url",
        "taskName": "Twitter (X) Comments Scraper",
        "parameters": json.dumps(parameters, ensure_ascii=False),
    },
    timeout=60,
)
print(response.status_code)
print(response.json())
```

## HTTP/cURL Example

```bash
curl -X POST "https://openapi.octoparse.com/api/agentTools/executeTask" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @- <<'JSON'
{
  "templateName": "tweets-comments-scraper-by-search-result-url",
  "taskName": "Twitter (X) Comments Scraper",
  "parameters": "{\"number_of_replies\":100.0,\"tweet_URLs\":[\"https://x.com/SpaceX/status/1827663529543168152\"]}"
}
JSON
```

## MCP Setup

Connect an MCP-compatible AI client to `https://mcp.octoparse.com` using browser OAuth or an Octoparse API key. Find `tweets-comments-scraper-by-search-result-url`, inspect its inputs, create a cloud task with the exact Input Schema keys, and export JSON results.

## Machine-readable Resources

- [Template contract](https://www.octoparse.com/templates/tweets-comments-scraper-by-search-result-url.json)
- [Manifest](https://www.octoparse.com/templates/tweets-comments-scraper-by-search-result-url-manifest.json)
