> ## Documentation Index
> Fetch the complete documentation index at: https://brightdata-ipv6-release.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# 触发数据收集 API

> 了解如何使用 Web Scraper API 触发数据收集，提供发现和 PDP 抓取器的选项。自定义请求，设置交付选项，并高效地检索数据。

## 正文

要供抓取器使用的输入。可以作为 JSON 或 CSV 文件提供：

<ParamField body="Content-Type" type="string">
  **`Content-Type: application/json`**

  输入的 JSON 数组

  > **示例**: `[{"url":"https://www.airbnb.com/rooms/50122531"}]`

  ***

  **`Content-Type: multipart/form-data`**

  一个 CSV 文件，字段名为 `data`

  > **示例** (curl): `data=@path/to/your/file.csv`
</ParamField>

## Web Scraper 类型

每种抓取器可能需要不同的输入。主要有两种类型的抓取器：

### 1. PDP

这些抓取器需要 URL 作为输入。PDP 抓取器从网页中提取详细的产品信息，如规格、定价和功能。

### 2. Discovery

Discovery 抓取器允许您通过搜索、类别、关键词等来探索和发现新实体/产品。

<Frame>
  <img src="https://mintcdn.com/brightdata-ipv6-release/yAA1dELwzqgxV65_/images/scraping-automation/web-scraper-api/ae.com.png?fit=max&auto=format&n=yAA1dELwzqgxV65_&q=85&s=dac882a084510e3e7d543525687ddea5" alt="ae.com.png" width="722" height="162" data-path="images/scraping-automation/web-scraper-api/ae.com.png" />
</Frame>

## 请求示例

### `PDP` 以 URL 输入

`PDP` 的输入格式始终是 URL，指向要抓取的页面。

```sh Sample Request theme={null}
curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '[{"url":"https://www.airbnb.com/rooms/50122531"},{"url":"https://www.airbnb.com/rooms/50127677"}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_ld7ll037kqy322v05&format=json&uncompressed_webhook=true"
```

### 基于 `discovery` 方法的 Discovery 输入

```sh Sample Request theme={null}
curl -H "Authorization: Bearer x2x3fdaaddrer" -H "Content-Type: application/json" -d '[{"keyword":"light bulb"},{"keyword":"dog toys"},{"keyword":"home decor"}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&endpoint=https://webhook-url.com&auth_header=QWxhZGRpbjpPcGVuU2VzYW1l&notify=https://notify-me.com/&format=ndjson&uncompressed_webhook=true&type=discover_new&discover_by=keyword&limit_per_input=10"
```

`discovery` 的输入格式可以根据特定的抓取器有所不同。输入可以是：

<CodeGroup>
  ```JSON keywords theme={null}
  [{"keyword": "light bulb"},{"keyword": "dog toys"},{"keyword": "home decor"}]
  ```

  ```JSON 搜索设置 theme={null}
  [{"url": "https://www.amazon.com/s?i=luggage-intl-ship", "sort_by": "Best Sellers"}]
  ```

  ```JSON 位置 theme={null}
  [{"location": "Europe"},{"location": "Greece"},{"location": "United States"}]
  ```
</CodeGroup>

以及更多。了解每个抓取器需要的输入，请参见 [这里](https://brightdata.com/cp/data_api).


## OpenAPI

````yaml dca-api POST /datasets/v3/trigger
openapi: 3.1.0
info:
  title: Brightdata API
  description: API for interaction with datasets marketplace
  version: 1.0.0
servers:
  - url: https://api.brightdata.com
security:
  - bearerAuth: []
paths:
  /datasets/v3/trigger:
    post:
      description: Scrape the target website based on the inputs provided in body
      parameters:
        - name: dataset_id
          description: Dataset ID for which data collection is triggered.
          in: query
          required: true
          schema:
            type: string
            example: gd_l1vikfnt1wgvvqz95w
        - name: type
          in: query
          schema:
            type: string
            enum:
              - discover_new
          description: >-
            Set it to "discover_new" to trigger a collection that includes a
            discovery phase.
        - name: discover_by
          in: query
          schema:
            type: string
          description: >-
            Specifies which discovery method to use. Available options:
            "keyword", "best_sellers_url", "category_url", "location" and more
            (according to the specific API). Relevant only for collections that
            include a discovery phase.
        - name: include_errors
          in: query
          schema:
            type: boolean
          description: Include errors report with the results.
        - name: limit_per_input
          in: query
          schema:
            type: number
            minimum: 1
          description: >-
            Limit the number of results per input. Relevant only for collections
            that include a discovery phase.
        - name: limit_multiple_results
          in: query
          schema:
            type: number
            minimum: 1
          description: Limit the total number of results.
        - name: notify
          in: query
          schema:
            type: string
          description: >-
            URL where the notification will be sent once the collection is
            finished. Notification will contain snapshot_id and status.
        - name: endpoint
          in: query
          schema:
            type: string
          description: Webhook URL where data will be delivered.
        - name: format
          in: query
          schema:
            type: string
            enum:
              - json
              - ndjson
              - jsonl
              - csv
          description: >-
            Specifies the format of the data to be delivered to the webhook
            endpoint.
        - name: auth_header
          in: query
          schema:
            type: string
          description: >-
            Authorization header to be used when sending notification to notify
            URL or delivering data via webhook endpoint.
        - name: uncompressed_webhook
          in: query
          schema:
            type: boolean
          description: >-
            By default, the data will be sent to the webhook compressed. Pass
            true to send it uncompressed.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              anyOf:
                - $ref: '#/components/schemas/TriggerInput'
                  title: Only inputs
                - $ref: '#/components/schemas/TriggerAndDeliverBody'
                  title: Deliver config and inputs
          multipart/form-data:
            schema:
              type: string
              description: A CSV file, in a field called data
      responses:
        '200':
          description: Collection job successfully started
          content:
            application/json:
              schema:
                type: object
                properties:
                  snapshot_id:
                    type: string
                    description: ID of your request that can be used in the next APIs
                    example: s_m4x7enmven8djfqak
components:
  schemas:
    TriggerInput:
      type: array
      example:
        - url: https://il.linkedin.com/company/bright-data
      items:
        type: object
        additionalProperties: {}
    TriggerAndDeliverBody:
      type: object
      example:
        deliver:
          type: webhook
          filename:
            extension: json
            template: '{[snapshot_timestamp]}_{[snapshot_id]}'
          endpoint: https://example.com/foo/bar
        input:
          - url: https://il.linkedin.com/company/bright-data
      properties:
        deliver:
          $ref: '#/components/schemas/DeliverConfig'
        input:
          $ref: '#/components/schemas/TriggerInput'
    DeliverConfig:
      description: Deliver configuration
      oneOf:
        - $ref: '#/components/schemas/DeliverConfigWebhook'
          title: Webhook
        - $ref: '#/components/schemas/DeliverConfigGCS'
          title: Google Cloud
        - $ref: '#/components/schemas/DeliverConfigGCSPubSub'
          title: Google Cloud PubSub
        - $ref: '#/components/schemas/DeliverConfigS3'
          title: Amazon S3
        - $ref: '#/components/schemas/DeliverConfigSnowflake'
          title: Snowflake
        - $ref: '#/components/schemas/DeliverConfigAliOSS'
          title: Aliyun Object Storage Service
        - $ref: '#/components/schemas/DeliverConfigSFTP'
          title: SFTP
        - $ref: '#/components/schemas/DeliverConfigAzure'
          title: Microsoft Azure
    DeliverConfigWebhook:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - webhook
            endpoint:
              type: string
              format: uri
              description: The endpoint URL for the webhook.
    DeliverConfigGCS:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - gcs
            bucket:
              type: string
              description: Name of the bucket.
            credentials:
              type: object
              additionalProperties: false
              description: Credentials for authentication
              properties:
                client_email:
                  type: string
                private_key:
                  type: string
              required:
                - client_email
                - private_key
            directory:
              type: string
              description: Target path
          required:
            - bucket
            - credentials
    DeliverConfigGCSPubSub:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - gcs_pubsub
            topic_id:
              type: string
            attributes:
              type: array
              items:
                type: object
            credentials:
              type: object
              additionalProperties: false
              properties:
                client_email:
                  type: string
                private_key:
                  type: string
              required:
                - client_email
                - private_key
          required:
            - topic_id
            - credentials
    DeliverConfigS3:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - s3
            bucket:
              type: string
            credentials:
              type: object
              additionalProperties: false
              minProperties: 2
              properties:
                aws-access-key:
                  type: string
                aws-secret-key:
                  type: string
                role_arn:
                  type: string
                external_id:
                  type: string
              oneOf:
                - title: Role ARN
                  required:
                    - role_arn
                    - external_id
                - title: Access Key
                  required:
                    - aws-access-key
                    - aws-secret-key
            region:
              type: string
            directory:
              type: string
          required:
            - bucket
            - credentials
    DeliverConfigSnowflake:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - snowflake
            database:
              type: string
            schema:
              type: string
            stage:
              type: string
            role:
              type: string
            warehouse:
              type: string
            credentials:
              type: object
              additionalProperties: false
              properties:
                account:
                  type: string
                user:
                  type: string
                password:
                  type: string
              required:
                - account
                - user
                - password
          required:
            - database
            - schema
            - stage
            - role
            - warehouse
            - credentials
    DeliverConfigAliOSS:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - ali_oss
            bucket:
              type: string
            credentials:
              type: object
              additionalProperties: false
              properties:
                access-key:
                  type: string
                secret-key:
                  type: string
              required:
                - access-key
                - secret-key
            region:
              type: string
            directory:
              type: string
          required:
            - bucket
            - credentials
            - region
    DeliverConfigSFTP:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - sftp
            path:
              type: string
              format: hostname
            port:
              type: integer
              minimum: 0
              maximum: 65535
            credentials:
              type: object
              additionalProperties: false
              properties:
                username:
                  type: string
                password:
                  type: string
                ssh_key:
                  type: string
                passphrase:
                  type: string
              required:
                - username
            directory:
              type: string
          required:
            - path
            - credentials
    DeliverConfigAzure:
      allOf:
        - $ref: '#/components/schemas/DeliverConfigBase'
        - type: object
          properties:
            type:
              enum:
                - azure
            container:
              type: string
              minLength: 3
              pattern: ^[a-z0-9](-?[a-z0-9])*$
            credentials:
              type: object
              additionalProperties: false
              properties:
                account:
                  type: string
                  pattern: ^[a-zA-Z0-9]+$
                key:
                  type: string
                  format: byte
                sas_token:
                  type: string
              required:
                - account
              oneOf:
                - required:
                    - key
                  title: Acess key
                - required:
                    - sas_token
                  title: Shared access token
            directory:
              type: string
          required:
            - container
            - credentials
    DeliverConfigBase:
      type: object
      additionalProperties: false
      properties:
        type:
          $ref: '#/components/schemas/DatasetDeliveryType'
        filename:
          type: object
          additionalProperties: false
          properties:
            template:
              type: string
              description: Template for the filename, including placeholders.
            extension:
              $ref: '#/components/schemas/DeliveredFileExt'
          required:
            - template
            - extension
      required:
        - type
        - filename
    DatasetDeliveryType:
      type: string
      description: Type of the delivery target
      enum:
        - azure
        - build
        - email
        - gcs
        - gcs_pubsub
        - s3
        - sftp
        - snowflake
        - webhook
        - ali_oss
    DeliveredFileExt:
      type: string
      enum:
        - json
        - jsonl
        - csv
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer

````