How to Add Custom Robots.txt File in Blogger Website

A robots.txt file is a standard used by websites to communicate with web crawlers and other automated agents. Let's learn how to add it to blogger.
Please wait 0 seconds...
Scroll Down and click on Go to Link for destination
Congrats! Link is Generated

Search engine optimization (SEO) is a crucial aspect of any website, including blogs. It helps to ensure that your content is easily discoverable by both search engines and users. In this blog post, we will discuss how to add custom robots.txt file in blogger website.

Adding Custom Robots.txt File on Blogger/Blogspot

Adding Custom Robots.txt File on Blogger/Blogspot

When it comes to optimizing your website for search engines, one important step you can take is adding a custom robots.txt file. This file tells search engine crawlers which pages and files they should and should not index on your website. If you're using Blogger as your website platform, here's how you can add a custom robots.txt file.

1. Create a robots.txt file

The first step is creating a robots.txt file. You can use a text editor like Notepad or TextEdit to create this file. Here's an example of what your robots.txt file might look like:

User-agent: *
Disallow:

This example robots.txt file allows all user agents (search engine crawlers) to access all parts of your website. However, you can customize this file to disallow certain pages or directories from being indexed.

2. Upload the robots.txt file to your website

Once you've created your robots.txt file, you'll need to upload it to your Blogger website. Here's how to do it:

- Go to your Blogger dashboard and click on the "Settings" tab.

- Click on "Search preferences".

- Under "Crawlers and indexing," click on "Custom robots.txt".

- Select "Yes" to enable custom robots.txt content.

- Copy and paste your robots.txt file's contents into the text box.

- Click "Save changes".

That's it! Your custom robots.txt file is now uploaded to your Blogger website and will help search engine crawlers index your website more efficiently. Remember to update your robots.txt file as needed if you add or remove pages or directories from your website.

Custom Robots.txt for Blogger/Blogspot

A robots.txt file is a standard used by websites to communicate with web crawlers and other automated agents. This file tells search engine crawlers which pages of your website they are allowed to crawl and index.

By creating and adding a custom robots.txt file to your blogger website, you can control which pages are indexed by search engines. This helps to prevent duplicate content from being indexed, which can negatively affect your website's ranking.

There are different types of custom robots.txt file for blogger website. You can use any of them according to your desire. If I talk about myself, I personally use the type 1. 

Robots.txt Type 1
User-agent: *
Disallow: /search
Allow: /

Sitemap: https://yourwebsite.blogspot.com/sitemap.xml?redirect=false&start-index=1&max-results=500

Robots.txt Type 2
User-agent: Mediapartners-Google
Disallow: 

User-agent: *
Disallow: /search
Disallow: /b
Allow: /
Sitemap: https://yourwebsite.blogspot.com/feeds/posts/default

Robots.txt Type 3
User-agent: Mediapartners-Google
Disallow: 

User-agent: *
Disallow: /search
Disallow: /b
Allow: /

Sitemap: https://yourwebsite.blogspot.com/atom.xml?redirect=false&start-index=1&max-results=500
Sitemap: https://yourwebsite.blogspot.com/atom.xml?redirect=false&start-index=501&max-results=500

Note: Remember to replace https://yourwebsite.blogspot.com with website URL. 

So this article was all about adding a custom robots.txt file to a blogger website. I hope you find the information mentioned in this article helpful. If this article helped you in any ways, feel free to share it with your friends so that they can also get the benefit.

About the Author

I'm a creative writer who loves to explore the mysteries of the world through puzzles and optical illusions. I'm passionate about writing articles that challenge and entertain readers of all ages.

Post a Comment

Cookie Consent
We serve cookies on this site to analyze traffic, remember your preferences, and optimize your experience.
Oops!
It seems there is something wrong with your internet connection. Please connect to the internet and start browsing again.
Site is Blocked
Sorry! This site is not available in your country.