← Back

Block Bad Bots on Your WordPress VIP Website

Are You Using Regular WordPress?
If your website is not hosted on WordPress VIP, use the regular WordPress integration instead.

WordPress VIP User Agent Restrictions can block requests containing specific user agent tokens at the edge.

Steps

  1. Generate the user agent tokens you want to block with the Known Agents User Agent Token Lists REST API.
  2. Construct a WordPress VIP User Agent Restriction rule group from those tokens using the contains operator.
  3. Submit the group list, including any existing User Agent Restriction groups, using the WordPress VIP updateUserAgentAccessRestrictions mutation.

Notes

Example: Using a GitHub Action

This reference workflow runs daily at 6:00 AM UTC and blocks AI Data Scrapers and AI Data Providers. Change agentTypes to select different agent types. Add KNOWN_AGENTS_ACCESS_TOKEN and WORDPRESS_VIP_ACCESS_TOKEN as repository secrets, and WORDPRESS_VIP_ENVIRONMENT_ID as a repository variable. 

name: Sync Known Agents User Agent Restrictions

on:
    schedule:
        - cron: "0 6 * * *"
    workflow_dispatch:

jobs:
    sync:
        runs-on: ubuntu-latest
        steps:
            - name: Sync Known Agents User Agent Restrictions
              uses: actions/github-script@v9
              env:
                  KNOWN_AGENTS_ACCESS_TOKEN: ${{ secrets.KNOWN_AGENTS_ACCESS_TOKEN }}
                  WORDPRESS_VIP_ACCESS_TOKEN: ${{ secrets.WORDPRESS_VIP_ACCESS_TOKEN }}
                  WORDPRESS_VIP_ENVIRONMENT_ID: ${{ vars.WORDPRESS_VIP_ENVIRONMENT_ID }}
              with:
                  script: |
                      const agentTypes = ["AI Data Scraper", "AI Data Provider"]
                      const environmentID = Number(process.env.WORDPRESS_VIP_ENVIRONMENT_ID)
                      const managedGroupNotes = "Managed by Known Agents"

                      const knownAgentsResponse = await fetch(
                          "https://api.knownagents.com/user-agent-token-lists",
                          {
                              method: "POST",
                              headers: {
                                  Authorization: "Bearer " + process.env.KNOWN_AGENTS_ACCESS_TOKEN,
                                  "Content-Type": "application/json",
                              },
                              body: JSON.stringify({
                                  agent_types: agentTypes,
                              }),
                          },
                      )

                      if (!knownAgentsResponse.ok) {
                          throw new Error("Known Agents request failed: " + knownAgentsResponse.status)
                      }

                      const userAgentValues = await knownAgentsResponse.json()

                      if (!Array.isArray(userAgentValues) || userAgentValues.length == 0) {
                          throw new Error("Known Agents returned no user agent values")
                      }

                      const existingGroupsQuery = `
                          query ($environmentID: Int!) {
                              environment(id: $environmentID) {
                                  edgeConfig {
                                      accessRestrictions {
                                          userAgent {
                                              groups {
                                                  id
                                                  notes
                                                  rules { operator value }
                                              }
                                          }
                                      }
                                  }
                              }
                          }
                      `

                      const existingGroupsData = await requestWordPressVIP(existingGroupsQuery, {
                          environmentID,
                      })
                      const groups =
                          existingGroupsData.environment?.edgeConfig?.accessRestrictions?.userAgent?.groups ?? []
                      const managedGroup = groups.find((group) => {
                          return group.notes == managedGroupNotes
                      })
                      const updatedGroups = groups.filter((group) => {
                          return group.notes != managedGroupNotes
                      })

                      updatedGroups.push({
                          ...(managedGroup ? { id: managedGroup.id } : {}),
                          notes: managedGroupNotes,
                          rules: userAgentValues.map((value) => {
                              return {
                                  operator: "contains",
                                  value,
                              }
                          }),
                      })

                      const updateGroupsMutation = `
                          mutation ($input: EdgeConfigUpdateUserAgentAccessRestrictionsInput) {
                              updateUserAgentAccessRestrictions(input: $input) {
                                  groups { id }
                              }
                          }
                      `

                      await requestWordPressVIP(updateGroupsMutation, {
                          input: {
                              environmentId: environmentID,
                              groups: updatedGroups,
                          },
                      })

                      async function requestWordPressVIP(query, variables) {
                          const response = await fetch("https://api.wpvip.com/graphql", {
                              method: "POST",
                              headers: {
                                  Authorization: "Bearer " + process.env.WORDPRESS_VIP_ACCESS_TOKEN,
                                  "Content-Type": "application/json",
                              },
                              body: JSON.stringify({ query, variables }),
                          })
                          const responseBody = await response.json()

                          if (!response.ok || responseBody.errors) {
                              throw new Error(JSON.stringify(responseBody))
                          }

                          return responseBody.data
                      }