Korean Blog Platform AI Crawler Policies Compared 2026: Naver Blog, Tistory, and Brunch robots.txt

Post the same article to Naver Blog and to Tistory and the odds of an AI search engine citing it are not the same. What separates the two is not sentence craft or depth of reporting. It is a single text file: the robots.txt each platform keeps at its domain root. Open the robots.txt of the Korean blog platforms and three patterns show up: block everything, block training bots only, block nothing. This page works from the raw robots.txt of Naver Blog, Tistory, Brunch, and velog to show where an article stays eligible for AI search citation, which gate drops it, and what to publish where.

30-Second Definitions

• Training crawlers gather web documents for model training. GPTBot, ClaudeBot, CCBot, and the Google-Extended token belong to this group^[7][8][10].
• AI search crawlers gather the source candidates an engine attaches to an answer. OAI-SearchBot, Claude-SearchBot, and PerplexityBot fall into this group^[7][8][9].
• User-triggered fetchers open a page only at the moment someone asks a question or supplies a URL. ChatGPT-User, Claude-User, and Perplexity-User count as these^[7][8][9].
• A platform robots.txt is the text file at the domain root that allows or blocks those three groups by bot name. The platform operator writes it, not the author.

Blur the four together and everything collapses into the phrase "AI blocked." The outcome depends entirely on whether a platform stopped training alone or stopped search citation along with it.

Korean Blog Platform AI Crawler Policies Compared

"No specific rule" means the bot name never appears in the file, so the User-agent: * block applies to it. Under the robots exclusion standard, a bot that goes unnamed is allowed by default.

What the Raw robots.txt Files Say

Naver Blog's robots.txt opens with the comment "BOT ACCESS FOR THE PURPOSES OF AI TRAINING AND RETRIEVAL-AUGMENTED GENERATION (RAG) IS STRICTLY PROHIBITED." Below it, GPTBot, OAI-SearchBot, PerplexityBot, Google-Extended, ClaudeBot, Claude-SearchBot, meta-externalagent, Applebot-Extended, and CCBot each carry Disallow: /^[1]. The wording of that comment names training and RAG together, which is the part that matters. This is not a training opt-out; it closes the retrieval step that produces citations as well.

Naver Influencer blocks the same list of AI bots. The comment line quoted above and the Yeti block, however, are absent from its robots.txt^[6]. Naver Cafe goes a step further and shuts User-agent: * plus Googlebot and Bingbot with Disallow: /^[5]. Naver Blog's file also disallows Yeti, Naver's own crawler^[1]. Why Naver blocks its own crawler here is not explained in any public document. Disappearing from outside AI engines and showing up in Naver's own search and AI briefings are separate questions.

Brunch is built the opposite way. The first block of its robots.txt sits under an "AI training crawlers" comment and shuts GPTBot, ClaudeBot, anthropic-ai, Claude-Web, Google-Extended, meta-externalagent, FacebookBot, Bytespider, Amazonbot, CCBot, Applebot-Extended, cohere-ai, Diffbot, omgili, Timpibot, and ImagesiftBot with Disallow: /. A second block labeled "Other search engines and AI search assistants" covers bingbot, Applebot, DuckDuckBot, OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, and PerplexityBot, and blocks only admin and feed paths such as /write, /library, /me/, /feed, and /search, leaving article URLs open^[2]. The top of the file carries the line # Last updated: 2026-04-22 verbatim^[2].

The robots.txt served on a Tistory blog subdomain excludes /guestbook, /m/guestbook, /manage, /owner, /admin, /search, and /m/search from User-agent: * and gives bingbot a Crawl-delay: 20. Not one AI crawler name appears in it^[3]. velog keeps a bare User-agent: * line with no rules underneath, so again nothing singles out an AI crawler^[4].

Why Blocking a Bot Name Reaches the Answer Itself

Engine vendors separate their bots by job in their own documentation. OpenAI splits GPTBot for model training, OAI-SearchBot for surfacing in ChatGPT search, and ChatGPT-User for handling a live user request, then states that "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links."^[7] Naver Blog blocking OAI-SearchBot means its posts drop out of the source list under a ChatGPT search answer.

Anthropic draws the same line: ClaudeBot collects training data, Claude-SearchBot improves search result quality, Claude-User handles user requests^[8]. Perplexity describes PerplexityBot as the bot that will "surface and link websites in search results on Perplexity," and notes that Perplexity-User generally does not follow robots.txt rules because a user initiated the request^[9]. A robots.txt block severs the unprompted citation path; it does not sever the path where someone hands the engine a link.

Google is the piece people misread most often. Google's documentation states that "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search."^[10] Google-Extended is a robots.txt token controlling training and grounding use for the Gemini app and Vertex AI, and it has no separate HTTP user agent string of its own^[10]. Naver Blog blocking Google-Extended does not remove those posts from Google Search.

What Platform Choice Actually Changes

First, on the engines whose search bots the file blocks by name, Naver Blog posts do not enter the source pool. The confirmed set is OAI-SearchBot, PerplexityBot, and Claude-SearchBot^[1][7][8][9]. That is where a familiar pattern begins: ask ChatGPT or Perplexity about a Korean brand and the Korean-language documents they pull cluster in news outlets, communities, and company-owned sites.

Second, a Brunch post drops out of training data and survives as an answer source. For an operator who wants to refuse training and still be cited, that combination is the closest fit available.

Third, Tistory and velog clear the crawler gate and then have to win the next one. An open robots.txt buys nothing if the raw HTML carries no body copy. JavaScript rendering and the AI crawler blind spot takes that problem on separately.

Fourth, the value of Naver Blog lies elsewhere. Those robots.txt rules address external crawlers, and the file says nothing about how Naver's own search and AI briefings treat its blogs. Its worth as a domestic search asset is a separate judgment. Folding every channel into one is the mistake. Running a Naver Blog is not. Naver SEO and GEO integrated operation guide covers how to handle domestic search and AI answers together.

Action, What to Check and How

1. Open the platform robots.txt yourself. Pull the raw file with curl -s https://<domain>/robots.txt and search it for bot names. Articles and summaries are not evidence; the file is. Platform policy changes without notice.
2. Judge training bots and search bots as separate questions. Whether GPTBot, ClaudeBot, and CCBot are blocked is one question. Whether OAI-SearchBot, PerplexityBot, and Claude-SearchBot are blocked is another. When the second set is blocked, citation is effectively finished^[7][9].
3. Inspect the raw HTML. Run curl -s <post URL> | grep and check whether body sentences, FAQ text, and JSON-LD show up in the response. Anything drawn only by JavaScript falls out of the candidate pool.
4. Split the original from the reach copy. Keep the complete document on a domain where you control robots.txt, and use the platform for a post that drives people to it.
5. Measure citation repeatedly, engine by engine. A single query is not a sample. Run the same question across engines on a schedule and log the distribution of cited domains. To start without spending anything, run the self-scan tools in GEO free audit tools compared first.

Ways to Check Whether You Are Cited

The capability and limitation columns summarize what each vendor publishes about itself, with details on the linked company pages. Pricing follows each vendor's published rates and changes over time. BOIDA is a brand operated by Designovel, and the product is BVI. What should drive tool selection is not price but diagnostic accuracy and technical implementation: which engines get checked on what cycle, and whether the results turn into schema and structure fixes.

Summary

Korean blog platforms fall into three camps on AI crawlers. The Naver properties closed training and search citation together^[1][5][6], Brunch closed training and left citation open^[2], and Tistory and velog never address bot names at all^[3][4]. Work out which camp your platform sits in before working on the writing. To design your own allow and block combinations per crawler type, see the AI crawler management guide, and for the underlying concepts, what GEO is.