Sunday, 5 March 2017

How to Add Custom Robots.txt file in Blogger

Here I will discuss about what robots.txt file is and how to add custom robots.txt file in Blogger. It is very Important For SEO. For a newbie blogger it may be difficult To enable custom robots.txt. So In This tutorial lets you how to add custom robots.txt file in blogger.

What is Robots.txt?

Robots.txt file interacts with all kind of crawlers or spiders like Googlebot, which is Google search engine's spider. In simple words, search engines always want to index the fresh content on the web so they send their spider or crawler to crawl the new pages on the web. If the spiders find new pages they will likely to index those pages, now robots.txt file comes to the picture on behalf of you, spiders or crawlers only index the pages you are allowing in your robots.txt file. Keep in your mind, the crawler or spider will firstly look at your robots.txt files to obey the rules you have instructed. If you have disallowed a page in your robots.txt file then spiders will follow the rule and they will not index that page into their search engine

Enable Custom Robots.Txt in Blogger

  • To add Robot.txt code In Your website Or blog, let Follow below Steps.
  • First of all, go to Blogger.com >> your site >> Settings >> Search engine Preference >> Crawlers and indexing.
    Now you will be able to see two options i.e. Custom robots.txt and Custom robots header tags.
  • These two options would provide you the flexibility to customize your robot.txt file
  • See Custom Robot.txt Section In below.
  • Save changes that's it

How to Check Your Robots.txt File?

Each website Have a robots.txt File. You Can verify Your Blog Robots.txt File By Adding /robots.txt After Your website.
See Below Link
http://www.mytechtalk.org/robots.txt
User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://www.mytechtalk.org/feeds/posts/default?orderby=UPDATED
 if you have more than 500 published posts, then you can
use these two sitemaps like below:
Sitemap: http://myblog.blogspot.com/atom.xml?
redirect=false&start-index=1&max-results=500
Sitemap:
http://myblog.blogspot.com/atom.xml?
redirect=false&start-index=500&max-results=1000
So That's How you can Add Custom Robots.txt File In Blogspot. We hope this Little tip Help You to Search Preference Visibility.

Conclusion: robots.txt file is beneficial for those who don't want to share specific things to search engines or other robots. You may resolve duplicate content issue by using it.

If you find this article informative Share This with Your Friends

Rishi yadav is the founder of MyTechTalk.A Popular tech blog dediceted for Geeks and blogger You Can Follow Me on Facebook Or You Can Connent With Me On Twitter @alltechpeople Email:mytechtalk99@gmail.com

4 comments

Very helpfull artical thanks bro...

thanks for share, this tamplate from arlinadzgn? thank you

Hello, this is really a good way to use robot.txt, yeah of course this is going to help a lot for many bloggers and for me, but I used wordpress CPM and cloud hosting, that's why even I am giving discount on Cloudways promo code.


EmoticonEmoticon