Enabling Data View from BigQuery
The following section describes a way to get access to traffic data as a view in BigQuery. For more information, please contact support or your CS representative.
Prerequisite
To gain access to the Malicious Traffic Data View, you would need the following:
- A Google Cloud Platform account with access to BigQuery
- A list of user emails with Google Cloud access
View Types
We currently support the following data types as views:
- Requests - All the requests that are intercepted to the customer’s website (including risk)
- PageViews - All requests made to the same page by the same user in a single browsing session count as one page view.
Requests Fields
| Field Name | Description |
|---|---|
| timestamp | Time of the request - UTC |
| app_id | HUMAN app ID assigned per application |
| account_name | Customer account name |
| application_name | HUMAN app name |
| ip | IP the request came from |
| is_mobile_sdk | If the request originated in mobile device = True |
| vid | HUMAN visitor unique identifier |
| human_client_uuid | Page view identifier designated by HUMAN |
| location_href | Full URL of the request (including domain, request params, etc.) |
| header_referer | The previous page the request came from (the page that led to this request) |
| domain | Parent domain for the request as derived from location href |
| path | Path of the request (where was the request to within the domain) |
| http_method | HTTP Method (POST, GET, HEAD, etc.) |
| asn_org_name | ISP provider for the request's original IP |
| asn_number | ISP provider number for the request's original IP |
| ua | User Agent string the request came from |
| os_version | Version of the operating system used in the request |
| os_family | Type of operating system used in the request |
| browser_version | Version of the browser used |
| browser_family | Type of browser used |
| country | Country the request came from |
| filter_type | Indicating if the request is classified as "always deny" or "always allow" |
| filter_origin_reason | The reason the origin was filtered (e.g., Spot.IM Crawler) |
| risk_score | Score given to the request estimating the likelihood of the request originating from bot traffic. Range 0 (most likely human) to 100 (most likely bot) |
| known_bot | If the request came from a known bot = True |
| socket_ip_classification | Any known classifications/ characteristics we might have for the original IP. e.g.: [{"name":"Google Cloud","class":"Cloud"}] |
| traffic_source | Returns web or mobile |
| incident_types | Incident types |
| blocked | True if the request was blocked |
| simulated_block | True if the request was blocked in monitor mode |
| custom_param1-9 | Custom parameters as defined by the customer |
Pageviews Fields
| Field Name | Description |
|---|---|
| timestamp | Time of the request - UTC |
| app_id | HUMAN app ID assigned per application |
| account_name | Customer account name |
| application_name | Customer application name |
| ip | IP the request came from |
| is_mobile_sdk | True if the request originated from a mobile device |
| human_client_uuid | Page view identifier designated by HUMAN |
| http_method | HTTP Method (POST, GET, HEAD, etc.) |
| location_href | Full URL of the request (including domain, request parameters, etc.) |
| domain | Parent domain for the request as derived from location href |
| path | Path of the request (where the request was directed within the domain) |
| ip_classification | Any known classifications/characteristics for the original IP |
| ua | User Agent string the request originated from |
| os_version | Version of the operating system used in the request |
| os_family | Type of operating system used in the request |
| browser_version | Version of the browser used |
| browser_family | Type of the browser used |
| country | Country the request came from |
| incident_types | Incident types |
| filter_type | Indicates if the request is classified as "always deny" or "always allow" |
| captchas_presented | Number of Captchas shown to the user |
| captchas_solved | Number of Captchas solved by the user |
| known_bot | True if the request came from a known bot |
| blocked | True if the request was blocked |
| score | Score estimating the likelihood of the request originating from bot traffic. Range 0 (most likely human) to 100 (most likely bot) |
| custom_param1-9 | Custom parameters as defined by the customer |
Data Retention
The data retrieved within the Big Query view is from the last 14 days
Setting up the Big Query View
Once HUMAN has created the view, and your Google account has the necessary permissions, you will be able to use it as follows:
- First time only
- Add our project to your view.
- Press ADD DATA (next to Resources)
- Click Pin a project and enter the project name.
- Click enter\pin to confirm.
- Once the project is pinned, you will be able to click on it and see the dataset (your company name) and inside the view (named requests or pageviews)
- Now click QUERY VIEW to open a base query.
We recommend limiting your search by adding a condition on the query (e.g. WHERE Date(timestamp) > 'XXXX-XX-XX').
If you receive a quota limit message for a user in your organization, try querying fewer days.
Updated 12 days ago