Articles on: Health

Scan problems overview

You might experience scan issues with Morningscore. Here are the common reasons and possible solutions.

This guide and our proposed solutions requires technical knowledge.

Problem: 401 Unauthorised

When our web crawler visits your website and receives a 401 error code, it means that the web crawler is not authorized to access the requested resource. It typically happens when the web crawler fails to provide valid authentication credentials, such as a username and password.

In simple terms, it means that your website is protected by a password and the web crawler failed to provide the correct login information.

Solution: 401 Unauthorised

This might not be a problem. It's normal that some pages require a login and that you don't want crawlers to visit them. In that case you can ignore this problem.

To fix this, you can allow certain web crawler’s IP address or user-agent to access those parts of the website without the login or provide the web crawler with a special login credentials (username and password) that it can use to access the website. We currently only support custom implementations in this scenarios, please contact us through the support for possible offers.

Our user-agent is: "Morningscore Bot" (without "")

Example of how you allow our crawler in your robots.txt file:

User-agent: Morningscore Bot
Allow: /


For a list of our crawler IP's please reach out to us in the chat.


Problem: 403 Forbidden

A 403 response code means that the website is not allowing our web crawler to access its pages. The website may be blocking all web crawlers, or it may be blocking just certain ones. The website administrator may have set up rules to block web crawlers from accessing certain pages or sections of the website.

Solution: 403 Forbidden

You can fix a 403 error by allowing our web crawler to access your website. This can be done by:

adding the web crawler's IP address or user-agent to the website's whitelist which is a list of IP addresses or user-agents that are allowed to access the website.

If you are using a security plugin, you need to check if that plugin is blocking the web crawler or not and adjust the settings accordingly.

You might be behind a strong firewall. It could be via a DNS provider like Cloudflare.

Our user-agent is: "Morningscore Bot" (without "")

Example of how you allow our crawler in your robots.txt file:

User-agent: Morningscore Bot
Allow: /


For a list of our crawler IP's please reach out to us in the chat.


Problem: 451 Unavailable for legal reasons

When our web crawler visits your website and receives a 451 error code, it means that the web crawler is not able to access the requested resource due to legal reasons. The HTTP 451 error code is an HTTP status code specifically designed for situations when a website is blocked by legal authority.

In simple terms, it means that your website is blocked by the government or some legal authority and the web crawler is not able to access it.

Solution: 451 Unavailable for legal reasons

As a website owner, you will not be able to fix this problem by yourself, as the block is enforced by a legal authority. You may want to check with the legal authorities or consult with a legal expert to understand the reason for the block and what actions you can take to lift the block.

This is a rare problem.

Please contact us in the chat if you have this problem.


Problem: Robots.txt rules

When our web crawler visits your website and is blocked because of Robots.txt rules, it means that you have set up specific instructions for web crawlers in a file called "robots.txt" which is located on your website's root directory. This file contains rules that specify which pages or sections of your website should not be crawled or indexed by web crawlers.

In simple terms, it means that you have told web crawlers not to visit certain pages or sections of your website by adding them to the Robots.txt file.

Solution: Robots.txt rules

you can fix this problem by either removing the disallow directive from the Robots.txt file or by adding our web crawler's IP address or user-agent to the website's whitelist which is a list of IP addresses or user-agents that are allowed to access the website.

Our user-agent is: "Morningscore Bot" (without "")

Example of how you allow our crawler in your robots.txt file:

User-agent: Morningscore Bot
Allow: /


For a list of our crawler IP's please reach out to us in the chat.

Updated on: 18/01/2023