A Few Words About What A 404 Error Is And How It Can Kill Your SXO

I think there is nothing worse than a 404 error. For example, when you're looking for some content on the internet, and you find an interesting site (or worse – a product), bah: "error 404 page not found". You surely know that feeling. Frustrating. And yet so common. The topic of 404 errors and how to handle them is prevalent in technical SEO. However, I believe this topic is so vital that it's worth devoting a few words to it on our blog. 

Navigation

1. Error 404 – what does it mean for your website?
2. 404 error happens when...
3. Common causes of 404 errors
4. Hunting for errors. How to find a 404 error on your site?
5. Soft Error 404 – sounds harmless but means trouble
6. How to fix error 404?
7. What if a 404 error can become your friend?

Error 404 – what does it mean for your website?

Error 404 (HTTP 404, 404 not found, 404, 404 error, page not found, or file not found) is a wrong response code. Refers to a subpage that wasn't found during the last visit of the crawler. It informs us that the user accessing the given subpage refers to a location or file that doesn't exist. The "page not found" error is often confused with the "server not found" error.

404 error happens when...

So, in summary, error code 404 happens when the client (e.g., a web browser) can communicate with the server, but the server cannot find the requested file, or it's configured not to fulfill the request and not display why.

Common causes of 404 errors

If you notice 404 errors on your site, the most common causes are:

• Change in the structure of website URLs – due to switching CMS systems. Then you should use the 301 redirect to redirect the old URLs to the new ones. If you don't do this, users may end up on subpages that don't exist.
• Modifying the URL, e.g., due to the product name change – in this case, you should also create a 301 redirect so that the user immediately finds a new subpage and not a 404 error.
• Removal of an article, product page, or any other subpage on the site. Remember that even if you delete a resource, it may still remain in the Google index for a while, and users will end up on non-existent pages until Google removes them from the index. However, users often save the addresses of the subpages they are interested in and find them directly, not from the search engine.
• Changes to the .htaccess file.
• Uncleaned cookies and cache files – this may result in displaying an outdated version of the page that returns an error. This could be the case if the page is not loading correctly on only one device.
• It happens that the 404 error is the result of a mistake of the user who made a typo in the website address. This often occurs when accessing a website by entering an address from memory.

Whether it's a user's error or yours – by not setting a redirect, the effect will be the same – the user won't see the resources they expected.

Hunting for errors. How to find a 404 error on your site?

There are some excellent tools out there to help you find 404 errors on your site. I personally use three of them on a daily basis:

• Google Search Console is a free treasury of knowledge about your site and indexation. With this tool, you can quickly check the health of your website, not only for 404 errors.
  source: Google Search Console

• Screaming Frog is undoubtedly one of the favorite tools of SEO specialists. The free version allows you to crawl up to 500 subpages of your website. After running the scan, you can see the server's response code in the "Status code" column and thus find pages with 404 errors. Another solution that can help is Sitebulb, which is a fully paid option. However, you can try it out thanks to the 14-day free trial.
  
  source: Screaming Frog

• Ahrefs – in my opinion, is a straightforward and helpful tool to analyze pages in terms of SEO, especially in the case of link building, but I'll write more about it someday. :) Ahrefs allows you to schedule regular website audits for various bugs, including 404 errors. 

Soft Error 404 – sounds harmless but means trouble

While analyzing your website, you may encounter a bug described in Google Search Console as "Submitted URL seems to be a Soft 404". What does it mean? The subpage doesn't exist, but the browser still receives a status code of 200. It also happens that the user is shown a page with little or almost empty content. Such a result will be indexed in the search results, which means that the Google robot - instead of indexing existing pages we want it to, will return to a non-existent subpage and try to index it again and again. Therefore, we lose the crawl budget on subpages that neither the user nor the robot should be able to access.

How to fix error 404?

Well, it depends (yeah, I know it's not funny anymore, but I can't hold myself back). ;) In general, Google's recommendations are clear: if we don't care about a given subpage in terms of position in search results, we can leave it as it is. Single pages with 404 errors are automatically found and removed from the Google index. 

However, if we don't want to lose the link juice of incoming links, we need to set a permanent redirect 301. It's only worth remembering that it should lead to the subpage that was thematically closest to the deleted one, consistent with the user's intention.

You can also properly configure the .htaccess file, automatically directing traffic from the error page to the home page of our website. But in my opinion, this is not the optimal solution.

What if a 404 error can become your friend?

As you know, there is no one correct answer to a problem in SEO. And sometimes, it turns out that what on the surface is a mistake may turn out to be your asset. A creative web admin may also use 404 errors to their advantage. How? After all, you can design a 404 page with great graphics that draw users' attention. Alternatively, you can place an internal search engine on your website where users can enter a phrase to find what they are looking for or direct them to other pages on the site. 

Don't forget about a sense of humor. And it may turn out that with the creative use of 404 pages, you'll improve user experience and increase your conversion rate. ;) Look at these examples of clever 404 pages.