8 Actionable Steps So Your Website Is Crawled Faster Than Your Competitors(In 2022)

Hey peoples! Pablo Espinal here,

In this post, I will show you 8 highly actionable steps you can take now to ensure your website is Mobile ready.

After going through these steps, the mobile pages on your website will load at lightning speed, leading to a definite edge over your competitors.

Here we go…

Some Basics And Statistics To Get Us Started

Statista.com says advertisers spent over $327 billion in Mobile advertising in 2022.

This figure is expected to grow even more in 2023 as there are around 307 million people(in the US alone) using mobile devices.

People spend an average of more than 3 hours a day on their phones.

As of August 2022, over 61% of all web traffic is done from Mobile devices.

Mobile-First indexing is defined as Google using the mobile version of a website’s content to rank pages from that site, as opposed to the desktop version.

Tip 1. Your Meta Robots Should Match 

Meta robots are defined as meta tag that tells web crawlers whether they should index or not and follow the links on a page.

Make sure the meta robots for both the desktop and mobile versions of your website match (especially the noindex or nofollow tags)

Here is how to check meta tags on the desktop/mobile versions of your website:

1. Go to the website’s homepage in your browser

2. Right-click anywhere on the page and click “View Page Source” or “View Source”

3. A new window will open with lots of code. You can search for ‘meta robots’

4. Check if the meta robots are noindex,nofollow,noarchive (for desktop) and noindex,follow (for mobile version)

5. If they don’t match, make sure you get them updated so that both versions have the same tags applied. This will help ensure your website is properly indexed by Google’s Mobile-First indexing algorithm.

Tip 2. Ensure Google Can Scan Lazy Loaded Content

Lazy loading is defined as a technique in which the web content is loaded only when it appears on the screen, as opposed to all at once.

Since Google cannot scan lazy loaded content by default, you need to make sure that your mobile website can be properly indexed.

Here’s how: 

1. Check if there are any Ajax calls occurring in your links/images or JavaScript/jQuery code

2. If any of these are present, make sure they’re not blocking off Googlebot from displaying them on the page (you can use a tool like ‘Fetch As Google’ to check this)

3. Make sure the server response code is 200 OK and not 404 Not Found (this will ensure that Googlebot finds the lazy loaded content on your website)

4. If you’re using any CDN or a caching system, make sure that it’s not blocking the Googlebot from accessing lazy loaded content (you can use ‘Fetch As Google’ to check this)

5. If you have any advertisements on your website, make sure they’re not blocking off Googlebot either (you can use ‘Fetch As Google’ to check this)

Tip 3. Make Your Resources Available To Google Crawling

Google crawling is defined as Google’s process of discovering and indexing web pages so that they can be displayed in search results.

To make sure your URL is not blocking Google with the disallow directive, you need to ensure that there’s no robots.txt file in your root directory and no disallow directive in the document type declaration (DTD).

Here’s how to check if a URL is available for Google crawling: 

1. Simply navigate the webpage, right-click anywhere on it, and click “View Page Source” or “View Source.”

2. A new window will open with lots of code. You can perform a search for ‘disallow’ and check if the dtd has any disallow directives applied to it

3. If there are any disallow directives, remove them from the document type declaration (DTD) so that Google bots can crawl your website correctly.

Tip 4. Same Amount Of Content On Both Your Desktop and Mobile Site

This one is self-explanatory, but if you have a 2500-word blog post on your desktop site, make sure your mobile site has the exact word count for the same blog post.

The above goes for headings, images, tags, etc. as well.

A tool you can use to measure word count on any of your web pages is the Hemingway App.

Tip 5. Get Your Structured Data On Point

We covered a bit of structured data in our recent Black Friday blog post, but we’ll go into more details here.

Breadcrumb Data

This means having the name of your web page as a breadcrumb or subheading on your site.

Example: Home >> Products >> Product Detail Page 

Product Data

This means having the name, price, and image of your product in support of the search query that brought a user to your site.

While Google may not use this data for ranking purposes, it can help users better understand what they’re looking at on your website.

You can also add star ratings to product listings so that users can easily see how well a product is rated by other users.

Rating data can help you stand out in search results as it signals to Google that your content or product is useful and relevant to the query.

Example: 5/5 stars 

VideoObject Data

This means making sure that your video summary, title, and thumbnail have been filled out to help Google better understand the content of your video.

For example, suppose you have a tutorial video on how to use a certain product. 

In that case, it’s important to include the product’s name in the title field and in the description so that users can quickly search for the product in Google and get to your video quickly.

You can use tools like YouTube’s Video Manager or Genesis Robotics’ videoscribe to create these types of data-rich structures.

Correct URLs

This means ensuring that all URLs are correctly formatted, such as using HTTPS for secure sites and www. for non-secure sites.

It’s also important to ensure that you have canonical tags in place so that there’s no duplicate content on your website, as this can negatively affect how Google crawls your site.

All structured data between your mobile and desktop sites should be present.

Tip 6. Metadata Should Match On Mobile And Desktop

Metadata is defined as the bits of information that you can add to your site in order to help Google crawl, index, and understand the content on your website.

Specifically, the descriptive titles and meta descriptions should match your site’s mobile and desktop versions.

A straightforward way to ensure metadata is consistent across mobile and desktop sites is to use a tool like Yoast SEO.

Here is how to check metadata consistency using Yoast SEO:

1. Open your website in a web browser and go to the Yoast SEO dashboard

2. Next, click on the “Search Appearance” option from the left-hand panel

3. Scroll down and click on the “Content Types” tab

4. Check to make sure that all metadata for your mobile and desktop site match across all content types

5. If you notice any discrepancies, update or add missing metadata as needed to ensure that both versions are consistent

6. Click “Save Changes” when you’re finished making any necessary updates.

Tip 7. Your Images Need To Be On Point

Specifically, your images should be:

High quality

To ensure that your images are high quality, avoid using photos or screenshots from other websites, as these may have been cropped or resized to fit in a specific layout.

A great tool we use for high-quality images here at SAAI consulting is Envato elements.

Images are using a supported format.

For example, PNG and JPG formats are generally accepted by Google.

It can also help to use alternative text (ALT text) for images to describe what the image is showing, as this helps Google crawl and index your images better.

Here is how to check what format an image is before you upload it to your site:

1. Select the image you want to upload and right-click on it

2. Choose “Properties” or a similar option from your menu

3. Check what format the image is currently in, as well as its dimensions and file size

Make sure Image URLs are not changing with each image reload

This means ensuring you are linking to your images and videos with a consistent URL.

If you don’t do the above, Google will have difficulty processing these images.

“alt text” has to be the same for every image

alt text for images is defined as the text that will be displayed if the image cannot be loaded.

This is important because Google uses this text to understand what your images are showing and whether they’re relevant to a certain search query.

Here is how to check the alt text of an image before you upload it to your site:

1. Select the image you want to upload and right-click on it

2. Choose “Properties” or a similar option from your menu

3. Check what alt text is currently included for the image, then make sure to include this same alt text in any future images that you upload

Tip 8. Your Videos Need To Be On Point

Precisely, your videos should:

Always use the same URLs 

Just as with images, if your URLs are different for your videos across your mobile and desktop versions of your site, Google will have a hard time processing and indexing these videos correctly.

Use supported video formats

This means using formats like MP4 or AVI for your videos, as Google generally accepts these.

It can also help to include an accurate meta description and title tags such as <video>, <embed>, or <object> for your videos, as this helps Google better understand what the video is about.

Here is how to check the format of a video file before you upload it to your site:

1. Select the video you want to upload and right-click on it

2. Choose “Properties” or a similar option from your menu

3. Check what format the video is currently in, as well as its dimensions and file size

4. Make sure that all videos you upload have the same format and dimensions, then proceed with uploading your video to your site.

Match video structured data across Mobile and desktop

One way to do this is using a tool like Video SEO Yoast, which automatically allows you to create and edit videos in bulk.

Overall, there are many things you can do to ensure that Google crawls and indexes your

Be easy to find when viewed on a mobile device

For Google to crawl and index your website successfully, your site must be easy to find and view when viewed on a mobile device.

A quick way to do this is to ensure the visitor can only scroll a little on the landing page to find the video.

Common Issues And How To Fix Them

Structured Data is missing

To fix this, you need to ensure all your structured data is included in the code and formatted correctly.

You may want to use a tool like Google Structured Data Testing Tool to check for errors or missing data before publishing your content.

The tag “noindex” is coming up on pages

To fix this, you must check all your pages for the <noindex> tag and remove it from any affected pages.

You may also want to work with a web developer to make sure that this issue does not occur again in the future.

Image is missing from either the desktop or mobile version of the website

To fix this, you will need to ensure that all of your images are included in the code for both versions of your website.

You may want to use a tool like Google Search Console to help you identify and fix this issue quickly.

Mobile image is being blocked by robots.txt

Add a “noindex” tag to your robots.txt file, preventing all your images from indexing on Mobile by Google.

Once you’ve done this, check that the change has taken effect by using a tool like Mobile SERP Test or Search Console.

The mobile image is too small or has low resolution

To fix this, you’ll need to ensure that your images are sized and scaled properly for mobile devices.

You can also pay for a website like Envato mentioned earlier. They will provide you with copyright-free high-quality images that you can use for both your mobile and desktop sites.

The page title is missing

To fix this, you’ll need to make sure that each page has a descriptive and clear title tag for both the mobile and desktop versions of your site.

There are issues with hostload

To fix this, you may want to consider using a CDN for the hostload speeds or working with a web developer to improve your site’s overall performance.

Frequently Asked Questions About Mobile-First Indexing

How can I make sure my website is optimized for mobile-first indexing?

You can take several steps to ensure that your website is ready for mobile-first indexing.

This includes using supported video formats, matching structured data across mobile and desktop versions of your site, and ensuring your landing page does not require excessive scrolling on a mobile device.

Additionally, you should make sure all images on your site are correctly sized and scaled for mobile viewing and that your page titles and hostload speeds are optimized.

What are the benefits of mobile-first indexing?

Mobile-first indexing has several key benefits, including faster loading times for users, increased visibility in Google search results, and improved user experience overall.

Additionally, by prioritizing the mobile version of your website, you can ensure that it is accessible to a wider range of users as more and more people use their smartphones to browse the web.

Overall, improving your website’s performance through mobile-first indexing can help drive more traffic and engagement from current and potential customers.

How Do I Know If My Phone Is Mobile-First Indexing?

There are several signs that your website is using mobile-first indexing.

One of the most obvious is faster load times for your site on desktop and mobile devices.

Additionally, you may notice increased visibility in Google search results and improved rankings for relevant keywords or topics.

Does Indexing Increase Performance?

There is no definitive answer to this question, as the performance benefits of mobile-first indexing can vary depending on various factors, including your website’s content and coding.

However, in general, we know that improved site performance can lead to higher engagement levels and more traffic from current and potential customers.

Does Indexing Take Up Space?

It is important to note that mobile-first indexing does not require additional storage space on your device.

Instead, it simply improves how your website loads and displays content, which can help improve user experience and overall performance.

Suppose you are still concerned about whether or not indexing will take up too much space on your device.

In that case, consulting with a web developer or online resources is a good idea to learn more about the benefits of mobile-first indexing.

Lastly, please don’t forget to leave a comment below and share on your social media if you found this information useful!

Till next time!