> This page is for Applications Protection.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.humansecurity.com/llms.txt.

# Known Bots and Crawlers

## Viewing Known Bots and Crawlers

HUMAN maintains a list of known bots and crawlers (for example, Google crawlers).

**TO see the list of known bots and crawlers:**

1. Go to **Product Settings > Security Policy**.\
   ![Security\_policy](/_fern-img/5a82d3e06a6d8c4e1d5b1e798181701ec1af1a9095cf5949c594d5fe3e919f61.webp)
2. Under **Policy rules**, select **Known bots & Crawlers**.\
   ![Known\_bots\_and-crawlers](/_fern-img/2cb8b411d7f1d15eeb6df49322f71c02db706327e3d1d086a627104c60ddddb2.webp)

## Managing Known Bots and Crawlers

Depending on your organization's needs, you can apply one of the following policies to any bot or crawler:

* **Allow** ![Allow](/_fern-img/e4e0f88f9766d09d2a4f5e82de4fd73dd254f34a9ee3c889e21a1cc67bcbf4b2.webp) - All traffic from the selected bot or crawler will pass through.
* **Deny** ![Deny](/_fern-img/f50ee0b540d0f95229826ada95fe53695717e8867b0bce7d757b08c5f6b8cc0d.webp) - All traffic from the selected bot or crawler will be blocked.
* **No policy** - Turn off the toggle button ![Policy\_off\_on](/_fern-img/6dd8142c733358a952b0aa08672ac64c4a76f4b5687fd1a4184121a04e84caf1.webp) to deactivate the policy. All traffic from this bot or crawler will be treated as traffic from any other source. You can always switch the policy back on.

The recommended policy is marked with a star ![Recommended policy](/_fern-img/2fc51dbe02f80bbac2f3bef8d7180f60a5016b065df625b6c52da825401fea66.webp)

Bots or crawlers that can be [abused](/applications/known-bots-and-crawlers#abusable-policy-rules) by attackers are marked with an exclamation mark ![Abusable\_policy](/_fern-img/29f473c9f9c381456287c8f24c5173dcfe9fda4d1344cdc77a7d8d9fa8326c1d.webp). We don't advise allowing traffic from these bots or crawlers, unless absolutely necessary.

## What are Abusable Policy Rules

These are policy rules that allow traffic based on the User Agent header only. If these policies are applied to some bots or crawlers, attackers can easily spoof their User Agent to circumvent Bot Defender protections using these bots or crawlers. The way to prevent this is to include a condition category based on an IP.

In short, a policy rule is classified as abusable if it:

* Is an Allow rule
* Is based only the User Agent header only
* Doesn't contain a condition category based on an IP

## Traffic Volume

When the traffic detection on an application is below the detection threshold set by HUMAN, the application’s Volume will be N/A. Once the detection threshold is reached, the application’s Volume is displayed. When no traffic is detected, the Volume is displayed as 0.