> ## Documentation Index
> Fetch the complete documentation index at: https://www.integrate.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# ETL: Trustpilot Source
> How to configure the Trustpilot component to read service reviews and business categories in your Integrate.io ETL pipeline.
Use the Trustpilot source component to read your company's service reviews and business categories from the Trustpilot Business Units API and ingest them into your [Integrate.io](http://integrate.io/) ETL pipeline. The connector supports two authentication modes: an API key alone reads public review data, and adding your API secret unlocks private reviews with server-side incremental loading.
## Connection Setup
Create a Trustpilot connection from **Connections → New connection → Trustpilot**.
| Field | Description |
| :--------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Name | Display name for the connection inside Integrate.io |
| Business Unit ID | The identifier of your business profile on Trustpilot (a 24-character ID such as `46d95a2b0000640005003f6f`). Every source table is scoped to this business unit. See [Finding your Business Unit ID](#finding-your-business-unit-id) below. |
| Business User ID | Optional. Not used by the source component; reserved for future features such as sending review invitations. Leave blank unless instructed otherwise. |
| API Key | The API key (also called the Client ID) of your Trustpilot application. Created in the Trustpilot Business portal under **Integrations → Developers → APIs → Create Application**. |
| API Secret | Optional. The client secret of the same Trustpilot application. When set, the connector authenticates through Trustpilot's OAuth client\_credentials flow and the **Private Reviews** table becomes available, including incremental loading. When left blank, the connector uses the public API with the key alone. |
API access requires a Trustpilot Business account with the API module. If you do not see the Developers section in your Trustpilot portal, contact Trustpilot to enable API access on your plan.
After filling the form, click **Test connection**:
* With an API key only, the test reads your public business profile.
* With an API key and secret, the test obtains an OAuth token and reads the private reviews endpoint, so a passing test confirms the whole private data path works.
### Finding your Business Unit ID
The quickest way is Trustpilot's public lookup endpoint. Replace `YOUR_API_KEY` and your website domain:
```bash theme={null}
curl -H "apikey: YOUR_API_KEY" \
"https://api.trustpilot.com/v1/business-units/find?name=yourcompany.com"
```
The `id` field in the response is your Business Unit ID.
## Source Tables
The dropdown in Step 02 of the component editor lists the available tables. Which tables appear depends on the connection's authentication mode:
| Table | Requires API Secret | Incremental Load | Contents |
| :--------------- | :------------------ | :--------------- | :--------------------------------------------------------------------------------------------------------------------------------- |
| reviews | No | No | Public service reviews for your business unit: stars, title, text, language, consumer display info, timestamps |
| categories | No | No | The business categories your company is listed under, with ranking position |
| private\_reviews | Yes | Yes | The same service reviews through Trustpilot's private API, which adds consumer email, reference ID, and other business-only fields |
Public and private reviews cover the same underlying review stream. The private table exists because Trustpilot's public API neither returns business-only fields nor accepts date filters. If you have the API secret, prefer **private\_reviews** for scheduled pipelines.
## Load Type
Select how records are loaded on each pipeline run:
* **Full Load.** Fetches all records for the selected table on every run. This is the only mode for `reviews` and `categories`.
* **Incremental Load.** Available on `private_reviews` only. Fetches only reviews created after (or before) a reference date, using Trustpilot's server-side `startDateTime`/`endDateTime` filters.
### Why only private\_reviews supports incremental loading
This is a Trustpilot API constraint, not a connector limitation. The public reviews endpoint accepts no date-filter parameters at all (only star rating, language, page, and sort order), so there is nothing for an incremental filter to hook into. Offering the option would silently load the full table on every run. The private endpoint accepts `startDateTime` and `endDateTime`, so incremental loading is offered exactly where the API supports it.
### Incremental Load Settings
When **Incremental Load** is selected on `private_reviews`, three additional fields appear:
| Field | Description |
| :-------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Sync date field | `createdAt`, the time the review was written. This is the only field offered because Trustpilot's date filters operate on creation time. Filtering on other datetime fields is not supported by the API. |
| Load records | The filter direction: `newer than ( > )` or `older than ( < )` the reference date. |
| Reference date | `Last successful run` auto-fills the date from the previous successful job, so each scheduled run picks up only new reviews. This is the recommended option for scheduled pipelines. `Fixed Date` and `Variable` are useful for one-off backfills. |
Because the filter operates on `createdAt`, an edited review does not re-sync on incremental runs; only newly written reviews are picked up. If you need edits reflected, schedule a periodic full load alongside the incremental pipeline.
## The 100,000 Record Limit on Public Reviews
Trustpilot caps the public reviews endpoint at 100,000 retrievable records. This is a hard API ceiling: requests beyond it are rejected by Trustpilot regardless of the client.
* A full load of `reviews` on a business unit with more than 100,000 reviews stops at 100,000 records (the most recent ones, in the API's default ordering). The job completes successfully at that point, so if your business unit has more reviews than that, compare the job's output record count with your review total on Trustpilot to know whether you hit the ceiling.
* The remaining older reviews cannot be fetched through the public endpoint.
To load a complete history of more than 100,000 reviews, use `private_reviews` and split the history into two one-off loads around a midpoint date, since each run applies a single date bound:
1. One run with **Load records** set to `older than ( < )` and a `Fixed Date` at the midpoint
2. One run with **Load records** set to `newer than ( > )` and the same `Fixed Date`
Pick a midpoint that puts fewer than 100,000 reviews on each side. If your destination appends records, use the review `id` as the merge or dedupe key so a review written exactly on the boundary date is not loaded twice. After the backfill, switch to Incremental Load with `Last successful run` for ongoing syncs. For histories above 200,000 reviews, two segments are not enough; contact support to plan the backfill.
## Field Flattening
The Trustpilot API returns nested JSON objects on each review (the consumer who wrote it, the business unit, invitation metadata). The connector automatically flattens these into individual columns with underscore-separated names, so each value lands in its own column instead of a serialized JSON blob.
For example, the nested consumer object:
```json theme={null}
{
"consumer": {
"displayName": "Jane D.",
"numberOfReviews": 12
}
}
```
becomes the columns `consumer_displayName` and `consumer_numberOfReviews`.
## Schema
After configuring the source properties, the **Schema** section (Step 03) displays the discovered fields with detected data types. Typical fields on the reviews tables:
| Field | Type | Notes |
| :-------------------- | :------- | :----------------------------------------------------------------------------------- |
| id | string | Trustpilot review ID |
| stars | long | Star rating, 1 to 5 |
| title / text | string | Review title and body |
| language | string | Two-letter language code |
| createdAt | datetime | When the review was written; the incremental sync field on private\_reviews |
| experiencedAt | datetime | When the reviewed experience happened |
| isVerified | boolean | Whether the review is verified |
| consumer\_displayName | string | Flattened from the nested consumer object |
| businessUnit\_id | string | Flattened from the nested business unit object |
| referenceId | string | private\_reviews only: your order or reference identifier from the review invitation |
Use the field selector to choose columns, rename with aliases, and override detected types as needed.
## Best Practices
* **Set the API secret if you have it.** It unlocks incremental loading and the business-only fields, and scheduled pipelines become dramatically cheaper: an incremental run fetches only new reviews instead of re-reading the whole table.
* **Use incremental with Last successful run** on `private_reviews` for scheduled pipelines, after seeding the destination with one full load (or date-windowed backfill for histories over 100,000 reviews).
* **Full loads of public reviews re-read everything.** A business unit with 100,000 reviews takes roughly 1,000 API calls per run. This is fine for one-off or occasional pulls, but prefer the private table for recurring syncs.
* **Restart clusters after changing the connection.** Connection settings, including adding the API secret, are applied to a cluster when it starts. Jobs on an already-running cluster keep the old settings until the cluster is restarted.
## FAQ
**Q: Why do reviews and categories not support incremental loading?**
Trustpilot's public API endpoints accept no date-filtering parameters, so there is no way to ask the API for "reviews since date X". The private reviews endpoint does accept date filters, which is why `private_reviews` (available once the API secret is set) supports incremental loading. See [Why only private\_reviews supports incremental loading](#why-only-private_reviews-supports-incremental-loading).
**Q: I set the API secret but do not see the private\_reviews table. Why?**
Click the **Refresh** button next to the table dropdown to bypass the cached table list. If it still does not appear, run **Test connection** on the connection: in secret mode the test validates the OAuth token, so a failing test means the key and secret pair is not valid for token minting (check for typos, and confirm both values come from the same Trustpilot application).
**Q: How does authentication work in each mode?**
With an API key only, requests carry the key in Trustpilot's `apikey` header. With a key and secret, the connector obtains an OAuth access token through Trustpilot's client\_credentials grant and sends it as a bearer token. Tokens live for around 100 hours and are cached and reused across runs; obtaining a new token does not invalidate existing ones.
**Q: Is the API secret visible after I save it?**
No. The secret is write-only: it is stored encrypted and never returned to the browser. The connection form shows a masked placeholder with a change link, the same pattern as the API key.
**Q: How does pagination and rate limiting work?**
The connector pages through results 100 records at a time and throttles to 5 requests per second, retrying up to 3 times on transient errors with backoff. Public reviews stop at Trustpilot's 100,000-record ceiling; see [the section above](#the-100000-record-limit-on-public-reviews) for loading longer histories.
## Related