Third Party Integrations

Enabling Data View from BigQuery

The following section describes a way to get access to traffic data as a view in BigQuery. For more information, please contact support or your CS representative.

Prerequisite

To gain access to the Malicious Traffic Data View, you would need the following:

  • A Google Cloud Platform account with access to BigQuery
  • A list of user emails with Google Cloud access

View Types

We currently support the following data types as views:

  • Requests - All the requests that are intercepted to the customer’s website (including risk)
  • PageViews - All requests made to the same page by the same user in a single browsing session count as one page view.

Requests Fields

Field NameDescription
timestampTime of the request - UTC
app_idHUMAN app ID assigned per application
account_nameCustomer account name
application_nameHUMAN app name
ipIP the request came from
is_mobile_sdkIf the request originated in mobile device = True
vidHUMAN visitor unique identifier
human_client_uuidPage view identifier designated by HUMAN
location_hrefFull URL of the request (including domain, request params, etc.)
header_refererThe previous page the request came from (the page that led to this request)
domainParent domain for the request as derived from location href
pathPath of the request (where was the request to within the domain)
http_methodHTTP Method (POST, GET, HEAD, etc.)
asn_org_nameISP provider for the request’s original IP
asn_numberISP provider number for the request’s original IP
uaUser Agent string the request came from
os_versionVersion of the operating system used in the request
os_familyType of operating system used in the request
browser_versionVersion of the browser used
browser_familyType of browser used
countryCountry the request came from
filter_typeIndicating if the request is classified as “always deny” or “always allow”
filter_origin_reasonThe reason the origin was filtered (e.g., Spot.IM Crawler)
risk_scoreScore given to the request estimating the likelihood of the request originating from bot traffic. Range 0 (most likely human) to 100 (most likely bot)
known_botIf the request came from a known bot = True
socket_ip_classificationAny known classifications/ characteristics we might have for the original IP. e.g.: \[{"name":"Google Cloud","class":"Cloud"}\]
traffic_sourceReturns web or mobile
incident_typesIncident types
blockedTrue if the request was blocked
simulated_blockTrue if the request was blocked in monitor mode
custom_param1-9Custom parameters as defined by the customer

Pageviews Fields

Field NameDescription
timestampTime of the request - UTC
app_idHUMAN app ID assigned per application
account_nameCustomer account name
application_nameCustomer application name
ipIP the request came from
is_mobile_sdkTrue if the request originated from a mobile device
human_client_uuidPage view identifier designated by HUMAN
http_methodHTTP Method (POST, GET, HEAD, etc.)
location_hrefFull URL of the request (including domain, request parameters, etc.)
domainParent domain for the request as derived from location href
pathPath of the request (where the request was directed within the domain)
ip_classificationAny known classifications/characteristics for the original IP
uaUser Agent string the request originated from
os_versionVersion of the operating system used in the request
os_familyType of operating system used in the request
browser_versionVersion of the browser used
browser_familyType of the browser used
countryCountry the request came from
incident_typesIncident types
filter_typeIndicates if the request is classified as “always deny” or “always allow”
captchas_presentedNumber of Captchas shown to the user
captchas_solvedNumber of Captchas solved by the user
known_botTrue if the request came from a known bot
blockedTrue if the request was blocked
scoreScore estimating the likelihood of the request originating from bot traffic. Range 0 (most likely human) to 100 (most likely bot)
custom_param1-9Custom parameters as defined by the customer

Data Retention

The data retrieved within the Big Query view is from the last 14 days

Setting up the Big Query View

Once HUMAN has created the view, and your Google account has the necessary permissions, you will be able to use it as follows:

  1. First time only
    1. Add our project to your view.
    2. Press ADD DATA (next to Resources)
    3. Click Pin a project and enter the project name.
    4. Click enter\pin to confirm.
  2. Once the project is pinned, you will be able to click on it and see the dataset (your company name) and inside the view (named requests or pageviews)
  3. Now click QUERY VIEW to open a base query.
    We recommend limiting your search by adding a condition on the query (e.g. WHERE Date(timestamp) > ‘XXXX-XX-XX’).
    If you receive a quota limit message for a user in your organization, try querying fewer days.