Running a website is one way to tell Google and other search engines to crawl and index your pages.
The only problem is that Google and its search engine friends only send spiders to your website when they feel like it - except maybe you send a personal invitation to them.
To invite Google is to submit your website to Google. This helps Google find your website faster even if it has no backlinks.
In this article, I'll explain how you can submit your website to Google and other search engines without breaking a sweat.
The whole process is as simple as shown in the map below.
For you to be reading this, it means that you already have a website.
But do you have a sitemap?
How to generate your sitemap for different blogging platforms
Your sitemap is a powerful tool in your SEO journey. It tells Google and other search engines which pages are important and need to be crawled on your website.
To submit your website to Google and other search engines, you need your sitemap URL. The way you find or create this URL depends on your blogging platform.
How to find or create your sitemap for a Blogger blog
Google now automatically creates a sitemap for Blogger blogs, so there is really no need for a special sitemap generator.
To obtain your sitemap, all you need to do is to add /sitemap.xml after your blog URL as shown below.
[Your blog]/sitemap.xml
And for your pages, the sitemap takes this format:
[Your blog]/sitemap-pages.xml
Note that it doesn't matter if your blog has a custom URL or a default URL, the sitemap format remains the same, which means you can have:
myblog.blogspot.com/sitemap.xml
Or
myblog.com/sitemap.xml
Before we continue, please note that, although the focus is on XML, there are other sitemap formats like RSS (Really Simple Syndication) and atom.
Fortunately, Google accepts the three formats. However, for optimal crawling, it is advisable to use both XML and RSS/Atom feeds.
To obtain the atom feed for a Blogger blog, all you need to do is to add atom.xml?redirect=false&start-index=1&max-results=500 after your blog URL. See example below:
myblog.com/atom.xml?redirect=false&start-index=1&max-results=500
OR
you may locate your sitemap by visiting: [your site]/robots.txt
For my site, all I need to do is to enter akinyemiabdulmalik.com.ng/robots.txt and I'll get below result.
How to find or create your sitemap for a WordPress blog
For a WordPress blog/website, you need to install a free SEO plugin like Yoast, Rank Math, or The SEO Framework. All three of these create sitemaps for you. To find your sitemap upon installing these plugins, just enter: [Your blog]/sitemap_index.xml
How to find or create your sitemap for other platforms
If you use Wix, Shopify, or squarespace, your sitemap can be found here: [Your blog]/sitemap.xml
If you can't find your sitemap there, then you might want to check:
[your site]/robots.txt
It's as simple as that.
Now that you have your sitemap, let's discuss how to submit it, so that Google and other search engines can crawl it faster.
How to submit your website to Google
Basically, there are three ways through which you can submit your blog/website to Google.
1. Robots.txt
2. Webmaster tool
3. Ping Google
Robots.txt:
This particular method is peculiar to Blogger websites.
To submit your blogger website to Google using the robots.txt, follow these simple steps.
- Go to your blogger dashboard
- Click on Settings
- Look for Enable Custom robots.txt option under Crawlers and indexing and select it.
See below image.
After that, you copy and paste below line of text and save.
User-agent: *
Disallow: /None
Allow: /
Sitemap: https://www.blog.com/sitemap.xml
Sitemap: https://www.blog.com/sitemap-pages.xml
Sitemap: http://www.blog.com.ng/atom.xml?redirect=false&start-index=1&max-results=500
In the quoted text, replace the highlighted text with your blog URL, and you are good to go.
Also, if you have more sitemaps to add, you may create a new line and add the sitemaps the same way they have been added above.
The disadvantage of this method is that you can't tell whether or not Google has crawled your blog; no indication whatsoever. Moreover, you have to input your sitemaps manually whenever you go beyond 500 blog posts.
One advantage, however, is that this method doesn't just work for Google alone; it works for other search engines as well.
Webmaster tool:
This method works for different blogging platforms, so long you have your sitemap URL.
Google search console is a Google tool that helps website owners monitor how they are performing on Google and optimize search performance.
After completing the sign-up process and verifying ownership of your site, follow the steps below.
1. Select your blog
2. Click on sitemap
3. Enter the URL of the sitemap that you have generated
To submit your website to bing, visit the bing webmaster tool and follow the instructions. It's as simple as that.
Ping Google:
Google operates a “ping” service where you can request a fresh crawl of your sitemap. All you have to do is to type the following into your browser:
http://www.google.com/ping?sitemap=<your sitemap URL>
Don't forget to replace the end part with your own sitemap URL.
Also, you should only ping Google with new or updated sitemaps. Never ping unchanged sitemaps multiple times.
For a WordPress blog, this method is unnecessary. This is because plugins like Yoast, Rank Math and The SEO Framework ping Google automatically.
What next after submitting your website to Google
Upon submitting your website to Google, Google sends spiders or bots to crawl your website. Once crawled, your site is ready to be indexed. To fully understand how this works, check the link below.
Recommended reading: How Search Engines Work
You should know that Google does not crawl your website as soon as you submit your sitemap. The process is not instantaneous. Crawling can take anywhere from a few days to a few weeks.
Indexing is what happens when Google stores your website data in its database. It is from this database that Google selects websites or pages to display in search results.
Submitting your website to Google doesn't guarantee that it will be indexed (even though it has been crawled). This can happen for several reasons that I'll teach you how to fix.
But first, let's find a way to discover pages that have not been indexed. We can achieve this using Google search console.
How to discover unindexed pages on your website
Follow these steps to discover pages that have not been indexed on your website and for what reason(s).
1. Go to Google search console
2. Select your property (your website)
3. Click on "pages" under "Indexing" (This is labelled 1 in the image below)
4. Select your preference from the dropdown (This is labelled 2 in the image below)
In the above image, just beneath the dropdown labelled 2, you can see the number of pages that have not been indexed (grey colour) and those that have been indexed (green colour). If you scroll down, you will see the list of pages that have not been indexed and why that is so.
Recommended reading: How to create a badass 404 error page for your Blogger blog
How to ask Google to index your pages
You can request Google to index your pages using the URL inspection tool on Google search console. To do that, follow these steps.
1. Go to search console
2. Click on URL inspection (labelled 1 in image below)
3. Enter the URL of the page you want to request for (labelled 2 in image below)
4. Click on "Test Live URL" to check whether it can be indexed (labelled 3)
5. Request indexing (labelled 4)
In the above image, the part labelled 5 tells us that the page has been crawled but currently not indexed. This could be as a result of several factors. It could be an error on your website/page, or maybe Google has not just found time to index it.
"Testing live URL" helps to ascertain the reason your page is not on Google. If upon entering the URL of your page, you get a result saying, "URL is on Google", then you don't have to test live URL or request indexing, as this has already been done.
However, if you get the result in the above image that your URL is not on Google, then it is advisable to test live URL first, so that you can know whether or not the page you entered doesn't have any error and can be indexed by Google.
Upon testing live URL, I got the image below. Note that the message has changed to "URL is available to Google" This indicates that the page has no error and is available to be indexed by Google. Then you can go ahead to request indexing.
That's it. You have done your part.
Now, there is nothing stopping you from appearing on search engines.
But to be honest, just appearing on search engines isn't enough.
You need visibility.
You need to appear on the first page of Google and other search engines, which isn't too difficult if you play your cards right.
If you like this post, please share. If there's anything else you'd like to know, please use the comments section below.
