> ## Documentation Index
> Fetch the complete documentation index at: https://www.octoparse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Data collection for review and sentiment analysis

> How to collect review data for sentiment analysis with web scraping: the data to gather, choosing sources, designing collection, extending to analysis, and copyright and personal-data considerations.

Reviews and ratings are data that reflect customer satisfaction, complaints, and the quality of demand. Web scraping is a way to collect reviews scattered across multiple sites, continuously, and shape them into a form usable for sentiment and reputation analysis.

This page is a technical playbook for the data design that takes you from collecting reviews to analyzing them. For how to collect reviews from a specific site, see the site-specific guides; for designing the analysis, use this page.

## Data you get from reviews

Review pages yield structured data you can use for analysis.

| Field                        | Use in analysis                                           |
| ---------------------------- | --------------------------------------------------------- |
| Review text                  | Sentiment classification, topic extraction, summarization |
| Rating score                 | Quantifying satisfaction, correlating score with text     |
| Post date                    | Rating trends over time, impact of campaigns or issues    |
| Subject (product / store)    | Comparison by product or store                            |
| Reviewer attributes (if any) | Trend analysis by segment                                 |
| Helpful count / replies      | Review influence, seller responsiveness                   |

Combining text with score surfaces patterns a score alone can't — for example, "high score but the text describes a complaint."

## Choosing data sources

Choose review sources based on what you're analyzing.

* **E-commerce and product reviews**: collect product reviews and buyer voices from Amazon and Rakuten. See [How to scrape Amazon](/docs/en/academy/scrape-amazon) and [How to scrape Rakuten Ichiba](/docs/en/academy/scrape-rakuten).
* **Store and service reviews**: read store ratings and area reputation from review sites. See [How to scrape Yelp](/docs/en/academy/scrape-yelp).
* **Map service reviews**: collect local-business reputation from Google Maps reviews. See [How to scrape Google Maps](/docs/en/academy/scrape-google-maps).
* **Social and forums**: collect reputation and topic volume for products and brands from posts.

Reading reviews across multiple sources reveals differences in rating by channel and the overall shape of reputation.

## Designing collection

To get review data that's usable for analysis, design at the collection stage.

1. Define the analysis target (product, store, brand, period).
2. Decide the fields to collect (text, score, and date at minimum).
3. Choose the collection method (bulk backfill of past reviews, or point-in-time monitoring of new ones).
4. Deduplicate (detect duplicate and re-posted reviews).
5. Normalize (align language, encoding, and rating scale).

<Tip>
  Always collect the rating score and post date alongside the review text. Score serves as training or validation data for sentiment, and post date drives time-series analysis. Text alone narrows what you can analyze substantially.
</Tip>

## Extending to analysis

Once collected and normalized, review data feeds a range of analyses.

* **Sentiment classification**: classify text as positive/negative/neutral and track the shifting mix.
* **Topic extraction**: extract recurring topics (price, quality, delivery, service) and pinpoint complaints.
* **Rating trends over time**: measure the impact of initiatives or issues from shifts in score or negative rate.
* **Product and store comparison**: compare competitors or your own products and stores by rating and review content.
* **Score-text correlation**: read the gap between score and text to see what a score alone hides.

Monitoring new reviews and defining events (such as a spike in negative rate) lets you feed alerts and automated reports.

## What to watch for

* **Copyright in review text**: review text is highly copyright-protected. Being able to collect it is separate from being free to republish, redistribute, or reuse it. Distinguish internal use for analysis from public reuse.
* **Personal data**: minimize personal data such as reviewer names — don't collect or store it if analysis doesn't need it.
* **robots.txt and terms of service**: check crawl permission and terms per target URL.
* **Server load**: keep frequency and volume modest on scheduled collection.

For the criteria that determine legality, see [Is web scraping legal?](/docs/en/academy/is-web-scraping-legal).

## Implementing with Octoparse

Octoparse renders pages in a headed, real browser, so reviews shown through incremental loading can be collected too. You can visually configure both a bulk backfill of past reviews and recurring collection of new ones, and accumulate them continuously with cloud runs.

Set up recurring collection with [scheduled runs](/docs/en/platform/schedules), and use [refine and clean extracted data](/docs/en/platform/refine-data) to shape the reviews you collect.

## Related resources

* [Data collection for market and competitive research](/docs/en/academy/market-research) — track the fuller competitive picture, reputation included
* [E-commerce data collection](/docs/en/academy/ecommerce-data) — collecting reviews alongside products and prices
* [How to scrape Amazon](/docs/en/academy/scrape-amazon) — a data source for product reviews
* [How to scrape Yelp](/docs/en/academy/scrape-yelp) — a data source for store reviews
* [Is web scraping legal?](/docs/en/academy/is-web-scraping-legal) — legality and scope of collection