Have you ever landed on a webpage only to see a big error message serving as a dead end to your reading? As a user, you’re left with little direction and close the tab. As the website owner, that has bad news written all over it. While these pages serve a purpose, they can be harmful to your website if misused.
An example of a “page not found” error used in the wrong context is a soft 404 error. These can wreak havoc on your search engine rankings, website traffic, and even credibility.
Below, I’ll define what a soft 404 error is and cover the difference between regular (or hard) 404 errors. Then I’ll explain how to find these errors and resolve them.
Soft 404 errors are pages that return a “not found” error code (404), but the content on the page is not actually missing. The page may be empty, or it may contain partial content from other pages on the site. Soft 404s can be caused by misspellings in URLs, by outdated links, or by problems with the server.
A soft 404 is not an official status; rather, it is one Google creates to alert you. This means Google and other search engines continue to crawl these web pages until you take action.
With a hard 404 error, the 404 error code is returned to both page visitors and search engines. This tells both that the page doesn’t exist. Here’s an example of a hard 404 error from the clothing site ModCloth:
With a soft 404 error, the 404 error code is displayed to visitors but not search engines.
Instead, they’ll see a 200 OK status which gives them the green light to crawl. So if you’re seeing soft 404 errors in your Google Search Console, it means that there are pages on your site that may be returning 404 errors even though they actually exist.
The short answer is yes; soft 404 errors can impact your website’s ranking on SERPs. To understand why, you need to understand how Google crawls a website.
To prevent strain on a website’s server and reduce the workload for the crawl bots, Google uses something called a crawl budget to determine how much and how often to crawl a site. A crawl budget is the number of pages that Googlebot will crawl on your site during each visit. The crawl budget is composed of crawl rate and crawl demand.
According to Google, the definitions are as follows:
When you have a large number of soft 404 errors on your website, you’re using a considerable amount of your crawl budget. This means fewer of your existing pages are crawled, leading to fewer and poorer rankings in SERPs.
Think of it this way: If your site has 100 pages, but Google determines your crawl budget is 70, then Google will only crawl 70 percent of your pages. If 10 pages are soft 404 errors, there’s a risk that some or all of those pages will fall into the crawl budget of 70. Thus, you lose out on the search engines crawling valuable pages.
Aside from lower rankings in SERPs, other problems are associated with soft 404 errors on your website.
For example, if Googlebot crawls a page with a soft 404 error and finds that the content on the page is not actually missing, it may assume that your site is serving up fake 404 errors. This could lead to your site being penalized by Google.
One of the biggest problems is a poor user experience. Since soft 404 URLs still appear in search results, users may be directed to non-existent pages.
If a user clicks on a link to a page that returns a soft 404 error, they may assume that the page doesn’t exist and leave your site. This can hurt your site’s bounce rate and cause users to spend less time on your site.
There can also be implications for the website’s performance and operations.
While 404 error pages don’t take up as much server space as pages with content, they do still take up bandwidth. If Google and other search engines direct traffic to a non-existent page, you’re still taking hits, which can slow website speed and performance.
Since soft 404 errors aren’t an official status, they aren’t as obvious to website owners.
One issue Search Console can help you to resolve is soft 404 errors.
Once connected and Search Console has had time to collect information, you’ll find soft 404 errors in Google Search Console under Coverage:
It can show in one of two ways: as an error or as excluded (as shown above).
The soft 404s that appear with an error status in Google Search Console are what you want to target first. Those with an excluded status should still be addressed, but Google has stopped crawling those pages for one reason or another.
If you don’t see any soft 404 errors in Google Search Console, you can also use a crawler like Screaming Frog to crawl your site and look for pages that return 404 errors. Just go to the “Response Codes” tab and look for any pages that are returning 404 errors.
Is there a difference between the 404 and 410 error codes when it comes to search engines?
According to Google’s John Mueller, not really. Both of these error codes tell Google to stop indexing the URL, which is the outcome you want in either case.
Most Content Management Systems (CMSs) have built-in 404 and 410 error pages. To make the most out of your 404 pages, I recommend adding a few related blog posts or providing a search feature or tool.
On my site, for example, users can analyze their website SEO from my 404 page:
Google is good at finding soft 404 errors, but it’s not perfect. It may incorrectly report a page as a soft 404, even if content exists on the page.
This generally occurs when the content is short or irrelevant.
When combining content pages or moving content around, it’s important to set up a redirect as soon as possible. Otherwise, your website visitors may be directed to a 404 page.
Once set up, the redirect is communicated to search engine crawlers so they crawl the correct pages.
Soft 404 errors are “page not found” errors that weren’t properly communicated to search engine crawl bots. The error shows correctly to website visitors but not to crawl bots, which means they continue to crawl those pages, wasting time and resources.
There are a few different things that can cause a soft 404 error. The most common reason is that the page has been moved to a new location and a redirect hasn’t been set up. Other common causes include poor content on the page and pages that return 404 errors even though they actually exist.
When soft 404 errors are resolved, the crawl bots can focus on a larger percentage of your active content pages. As a result, more content appears in SERPs. In theory, this can increase website traffic.
The difference between a regular 404 and a soft 404 error is what is communicated to search engine crawl bots. With a regular 404, the website server delivers the 404 error to the crawl bot. With a soft 404, the website server delivers a 200 OK status, which means the bots continue to crawl the page.
There’s no official penalty for soft 404 errors, though there are natural consequences if the errors aren’t resolved. These include lower visibility on search engine results pages and a poor user experience.
404 errors can hurt your site’s ranking if they’re not properly handled. If Googlebot crawls a page with a 404 error, it may assume that the page doesn’t exist and stop crawling your site. As a result, your most important pages may not be getting crawled as often as they should, which can hurt your site’s ranking.
Additionally, if users click on a link to a page that returns a 404 error, they may assume that the page doesn’t exist and leave your site. This can hurt your site’s bounce rate and cause users to spend less time on your site, both of which can negatively impact your site’s ranking.
To avoid these problems, be sure to set up a proper 404 or 410 error if a page doesn’t exist, and be sure to set up a 301 redirect if a page has been moved.
To set up a 404 or 410 error, you’ll need to add a “404.html” or “410.html” file to your site.
If the page does exist but is returning a soft 404 error, you’ll need to improve the content on the page and then reindex it. This can be done by adding more relevant content to the page or by improving the page’s title and meta tags.
If the page was moved to a new location, you’ll need to set up a 301 redirect. A 301 redirect tells users that the page has been permanently moved to a new location. To set up a 301 redirect, you’ll need to add a “301.html” file to your site.
Soft 404 errors are “page not found” errors that weren’t properly communicated to search engine crawl bots. The error shows correctly to website visitors but not to crawl bots, which means they continue to crawl those pages, wasting time and resources.
There are a few different things that can cause a soft 404 error. The most common reason is that the page has been moved to a new location and a redirect hasn’t been set up. Other common causes include poor content on the page and pages that return 404 errors even though they actually exist.
When soft 404 errors are resolved, the crawl bots can focus on a larger percentage of your active content pages. As a result, more content appears in SERPs. In theory, this can increase website traffic.
The difference between a regular 404 and a soft 404 error is what is communicated to search engine crawl bots. With a regular 404, the website server delivers the 404 error to the crawl bot. With a soft 404, the website server delivers a 200 OK status, which means the bots continue to crawl the page.
There’s no official penalty for soft 404 errors, though there are natural consequences if the errors aren’t resolved. These include lower visibility on search engine results pages and a poor user experience.
404 errors can hurt your site’s ranking if they’re not properly handled. If Googlebot crawls a page with a 404 error, it may assume that the page doesn’t exist and stop crawling your site. As a result, your most important pages may not be getting crawled as often as they should, which can hurt your site’s ranking.
Additionally, if users click on a link to a page that returns a 404 error, they may assume that the page doesn’t exist and leave your site. This can hurt your site’s bounce rate and cause users to spend less time on your site, both of which can negatively impact your site’s ranking.
To avoid these problems, be sure to set up a proper 404 or 410 error if a page doesn’t exist, and be sure to set up a 301 redirect if a page has been moved.
Additionally, make sure that all of your pages have good content that is relevant to what users are looking for. You can perform technical SEO audits throughout the year to ensure your pages are up to par. By taking these steps, you can help ensure that your site’s ranking isn’t hurt by soft 404 errors.
If the page doesn’t actually exist, creating a customized 404 page or 410 error will help. A 404 error tells users that the page they’re looking for doesn’t exist, while a 410 error tells users that the page has been permanently removed.
To set up a 404 or 410 error, you’ll need to add a “404.html” or “410.html” file to your site.
If the page does exist but is returning a soft 404 error, you’ll need to improve the content on the page and then reindex it. This can be done by adding more relevant content to the page or by improving the page’s title and meta tags.
If the page was moved to a new location, you’ll need to set up a 301 redirect. A 301 redirect tells users that the page has been permanently moved to a new location. To set up a 301 redirect, you’ll need to add a “301.html” file to your site.
When a page is removed from your website, there are proper ways to signal this removal to both website visitors and search engine crawl bots.
Depending on the circumstances, you might set up a permanent 404, set up a 301 redirect, or improve the page content and reindex it. These fixes take just a few minutes but can significantly impact website performance.
When was the last time you checked for soft 404 errors?
