{"id":23084,"date":"2025-07-11T14:18:05","date_gmt":"2025-07-11T14:18:05","guid":{"rendered":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/?lw_help_doc=security\/bot-management"},"modified":"2026-07-02T13:27:26","modified_gmt":"2026-07-02T17:27:26","slug":"bot-management","status":"publish","type":"lw_help_doc","link":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/","title":{"rendered":"Bot Management"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Bot management is the practice of identifying and controlling automated traffic to your website. It is used to distinguish between beneficial bots, such as search engine crawlers, and malicious bots that can negatively impact your site.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Malicious bots can be used for content scraping, credential-stuffing attacks, and other activities that compromise security, skew analytics data, and degrade performance. An effective bot management strategy allows beneficial automated traffic while blocking these malicious activities.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<div class=\"kb-row-layout-wrap kb-row-layout-id385276_b99c22-9c alignnone wp-block-kadence-rowlayout\"><div class=\"kt-row-column-wrap kt-has-1-columns kt-row-layout-equal kt-tab-layout-inherit kt-mobile-layout-row kt-row-valign-top kb-theme-content-width\">\n<p class=\"wp-block-paragraph\">Bot Management Articles<\/p>\n<div class=\"wp-block-query is-layout-flow wp-block-query-is-layout-flow\">\n<div class=\"help-docs__post-count-badge is-style-pill wp-block-tribe-query-results-count\">\n\t<p>4 results<\/p>\n<\/div>\n<\/div>\n<p class=\"wp-block-paragraph\">This collection of articles will guide you through the various facets of bot management, from foundational concepts to practical, hands-on techniques.<\/p>\n<div data-wp-context=\"{}\" data-wp-interactive=\"core\/query\" data-wp-key=\"0\" data-wp-router-region=\"query-0\" class=\"wp-block-query is-layout-flow wp-block-query-is-layout-flow\"><ul class=\"wp-block-post-template is-layout-flow wp-block-post-template-is-layout-flow\"><li data-wp-key=\"post-template-item-23588\" class=\"wp-block-post post-23588 lw_help_doc type-lw_help_doc status-publish hentry vertical-hosting lw_resource_category-security lw_resource_section-security\">\n<h2 class=\"wp-block-post-title\"><a href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/bots-and-their-impact\/\" target=\"_self\" >Bots and their Impact<\/a><\/h2>\n\n<div class=\"wp-block-post-excerpt\"><p class=\"wp-block-post-excerpt__excerpt\">Bots are automated programs that can help or harm your site. Learn what bots are, how they work, and how to tell the good from the bad. <\/p><\/div>\n\n<a class=\"wp-block-read-more\" href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/bots-and-their-impact\/\" target=\"_self\">Read more<span class=\"screen-reader-text\">: Bots and their Impact<\/span><\/a>\n<\/li><li data-wp-key=\"post-template-item-23589\" class=\"wp-block-post post-23589 lw_help_doc type-lw_help_doc status-publish hentry vertical-hosting lw_resource_category-security lw_resource_section-security\">\n<h2 class=\"wp-block-post-title\"><a href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/rate-limiting\/\" target=\"_self\" >Rate-Limiting<\/a><\/h2>\n\n<div class=\"wp-block-post-excerpt\"><p class=\"wp-block-post-excerpt__excerpt\">Control traffic and stop bad bots with rate-limiting. Learn how to limit request volume to protect your site from attacks and improve performance. <\/p><\/div>\n\n<a class=\"wp-block-read-more\" href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/rate-limiting\/\" target=\"_self\">Read more<span class=\"screen-reader-text\">: Rate-Limiting<\/span><\/a>\n<\/li><li data-wp-key=\"post-template-item-23590\" class=\"wp-block-post post-23590 lw_help_doc type-lw_help_doc status-publish hentry vertical-hosting lw_resource_category-security lw_resource_category-troubleshooting lw_resource_section-security\">\n<h2 class=\"wp-block-post-title\"><a href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/troubleshooting-bot-access-issues\/\" target=\"_self\" >Troubleshooting bot access issues<\/a><\/h2>\n\n<div class=\"wp-block-post-excerpt\"><p class=\"wp-block-post-excerpt__excerpt\">Search bots blocked? Troubleshoot `robots.txt` issues &#038; CSF IP blocks (ModSecurity). Guide: check perms, unblock IPs &#038; whitelist trusted bots. <\/p><\/div>\n\n<a class=\"wp-block-read-more\" href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/troubleshooting-bot-access-issues\/\" target=\"_self\">Read more<span class=\"screen-reader-text\">: Troubleshooting bot access issues<\/span><\/a>\n<\/li><li data-wp-key=\"post-template-item-23591\" class=\"wp-block-post post-23591 lw_help_doc type-lw_help_doc status-publish hentry vertical-hosting lw_resource_category-security lw_resource_section-security\">\n<h2 class=\"wp-block-post-title\"><a href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/what-are-webcrawlers-and-robots-txt\/\" target=\"_self\" >What Are Webcrawlers and robots.txt?<\/a><\/h2>\n\n<div class=\"wp-block-post-excerpt\"><p class=\"wp-block-post-excerpt__excerpt\">Web crawlers populate search engine databases, indexing websites for search results. They vary in behavior and pace. The robots.txt file guides them, indicating where not to scan and how quickly to crawl the site. <\/p><\/div>\n\n<a class=\"wp-block-read-more\" href=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/what-are-webcrawlers-and-robots-txt\/\" target=\"_self\">Read more<span class=\"screen-reader-text\">: What Are Webcrawlers and robots.txt?<\/span><\/a>\n<\/li><\/ul>\n\n\n\n<\/div>\n\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Learn to manage web bots. Our guides help you block bad bots, allow good ones, and secure your site. Control your website&#8217;s automated traffic.<\/p>\n","protected":false},"author":90,"featured_media":0,"parent":22933,"menu_order":0,"template":"","meta":{"footnotes":"","_nx_search_keywords":"","_nx_search_boost":0},"vertical":[256],"lw_resource_category":[2750],"lw_resource_topic":[],"lw_resource_section":[3561],"class_list":["post-23084","lw_help_doc","type-lw_help_doc","status-publish","hentry","vertical-hosting","lw_resource_category-security","lw_resource_section-security"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.9 (Yoast SEO v27.9) - https:\/\/round-lake.dustinice.workers.dev:443\/https\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Bot Management - Nexcess Documentation<\/title>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Bot Management\" \/>\n<meta property=\"og:description\" content=\"Learn to manage web bots. Our guides help you block bad bots, allow good ones, and secure your site. Control your website&#039;s automated traffic.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/\" \/>\n<meta property=\"og:site_name\" content=\"Nexcess Documentation\" \/>\n<meta property=\"article:modified_time\" content=\"2026-07-02T17:27:26+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/hosting\\\/security\\\/bot-management\\\/\",\"url\":\"https:\\\/\\\/docs.nexcess.com\\\/hosting\\\/security\\\/bot-management\\\/\",\"name\":\"Bot Management - Nexcess Documentation\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/#website\"},\"datePublished\":\"2025-07-11T14:18:05+00:00\",\"dateModified\":\"2026-07-02T17:27:26+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/hosting\\\/security\\\/bot-management\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/docs.nexcess.com\\\/hosting\\\/security\\\/bot-management\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/hosting\\\/security\\\/bot-management\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/docs.nexcess.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Security Overview\",\"item\":\"https:\\\/\\\/docs.nexcess.com\\\/hosting\\\/security\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Bot Management\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/#website\",\"url\":\"https:\\\/\\\/docs.nexcess.com\\\/\",\"name\":\"Nexcess Documentation\",\"description\":\"Search the docs, or browse by product, category, and topic across Hosting and WordPress.\",\"publisher\":{\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/docs.nexcess.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/#organization\",\"name\":\"Nexcess Documentation\",\"url\":\"https:\\\/\\\/docs.nexcess.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/docs.nexcess.com\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/favicon-1.webp\",\"contentUrl\":\"https:\\\/\\\/docs.nexcess.com\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/favicon-1.webp\",\"width\":512,\"height\":512,\"caption\":\"Nexcess Documentation\"},\"image\":{\"@id\":\"https:\\\/\\\/docs.nexcess.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Bot Management - Nexcess Documentation","robots":{"index":"noindex","follow":"follow"},"og_locale":"en_US","og_type":"article","og_title":"Bot Management","og_description":"Learn to manage web bots. Our guides help you block bad bots, allow good ones, and secure your site. Control your website's automated traffic.","og_url":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/","og_site_name":"Nexcess Documentation","article_modified_time":"2026-07-02T17:27:26+00:00","twitter_card":"summary_large_image","schema":{"@context":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/","url":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/","name":"Bot Management - Nexcess Documentation","isPartOf":{"@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/#website"},"datePublished":"2025-07-11T14:18:05+00:00","dateModified":"2026-07-02T17:27:26+00:00","breadcrumb":{"@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/bot-management\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/"},{"@type":"ListItem","position":2,"name":"Security Overview","item":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/hosting\/security\/"},{"@type":"ListItem","position":3,"name":"Bot Management"}]},{"@type":"WebSite","@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/#website","url":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/","name":"Nexcess Documentation","description":"Search the docs, or browse by product, category, and topic across Hosting and WordPress.","publisher":{"@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/#organization","name":"Nexcess Documentation","url":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/#\/schema\/logo\/image\/","url":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-content\/uploads\/2026\/07\/favicon-1.webp","contentUrl":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-content\/uploads\/2026\/07\/favicon-1.webp","width":512,"height":512,"caption":"Nexcess Documentation"},"image":{"@id":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_help_doc\/23084","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_help_doc"}],"about":[{"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/types\/lw_help_doc"}],"author":[{"embeddable":true,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/users\/90"}],"version-history":[{"count":4,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_help_doc\/23084\/revisions"}],"predecessor-version":[{"id":55708,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_help_doc\/23084\/revisions\/55708"}],"up":[{"embeddable":true,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_help_doc\/22933"}],"wp:attachment":[{"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/media?parent=23084"}],"wp:term":[{"taxonomy":"vertical","embeddable":true,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/vertical?post=23084"},{"taxonomy":"lw_resource_category","embeddable":true,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_resource_category?post=23084"},{"taxonomy":"lw_resource_topic","embeddable":true,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_resource_topic?post=23084"},{"taxonomy":"lw_resource_section","embeddable":true,"href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/docs.nexcess.com\/wp-json\/wp\/v2\/lw_resource_section?post=23084"}],"curies":[{"name":"wp","href":"https:\/\/round-lake.dustinice.workers.dev:443\/https\/api.w.org\/{rel}","templated":true}]}}