maxjhandsome
/
seata
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

optimize: add robots.txt (#548) 

Signed-off-by: slievrly <slievrly@163.com>
This commit is contained in:
jimin 2023-02-01 14:10:29 +08:00 committed by GitHub
parent 2fefb0b316
commit da14b63952
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 43 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

43
robots.txt Normal file
View File

@ -0,0 +1,43 @@
User-agent: Googlebot
Disallow:
User-agent: googlebot-image
Disallow:
User-agent: googlebot-mobile
Disallow:
User-agent: MSNBot
Disallow:
User-agent: Slurp
Disallow:
User-agent: Teoma
Disallow:
User-agent: twiceler
Disallow:
User-agent: Gigabot
Disallow:
User-agent: Scrubby
Disallow:
User-agent: Robozilla
Disallow:
User-agent: Nutch
Disallow:
User-agent: ia_archiver
Disallow:
User-agent: baiduspider
Disallow:
User-agent: naverbot
Disallow:
User-agent: yeti
Disallow:
User-agent: yahoo-mmcrawler
Disallow:
User-agent: psbot
Disallow:
User-agent: asterias
Disallow:
User-agent: yahoo-blogs/v3.9
Disallow:
User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: /bin/
Sitemap: https://seata.io/sitemap.xml