One thing that is missing however is a
robots.txt file, instructing web robots such a the Googlebot and the Baidubot to index or not to index specific files or posts on your blog.
Typically you want everything to be publicly available, but you do not want robots hitting your administrative pages and filling up the error-logs.
Luckily, it is quite easy to configure a
robots.txt file. Remember however that this is not a security feature, malicious web indexers might still ignore whatever you put into that file.
Go to your Ghost installation directory (replace
installation_dir with the real location on your server. Move to your current theme installation directory (in the example the default
cd installation_dir cd content/themes/casper nano robots.txt
Put the following content in the
User-agent: * Disallow: /ghost/
This instructs web spiders to index everything, but not the administrative pages.
If you use a proxy such as
Nginx and use rewriting rules, you might want to change the
Disallow: /ghost/ to whatever your remapping rules dictate.
You can easily check if the installation was successful by surfing to http://YOUR.DOMAIN.NAME/robots.txt (change the
YOUR.DOMAIN.NAME to the URL of your blog).
That's it! Easy, wasn't it? Have fun!
All you never wanted to know about a
robots.txt file kan be found here.