xoxys/geeklabor_legacy
Archived
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

disallow all taxonomy subdirs for web crawler

Browse Source
This commit is contained in:
Robert Kaussow 2019-02-05 15:28:00 +01:00
parent 60a197c008
commit 34652c19db
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
static/robots.txt
View File

@ -1,7 +1,7 @@
User-agent: * User-agent: *
Disallow: /feed.atom.xml Disallow: /feed.atom.xml
Disallow: /categories/ Disallow: /categories/*
Disallow: /authors/ Disallow: /authors/*
Disallow: /author/ Disallow: /author/*
Disallow: /tags/ Disallow: /tags/*
Sitemap: https://geeklabor.de/sitemap.xml Sitemap: https://geeklabor.de/sitemap.xml