Headers Check

Every visitor to your website passes Header Fields that define the operating parameters of an HTTP transaction. All the nowadays Internet Browsers pass all the correct headers and this is one way to detect if the visitor is a bot or a human. Many bots and services visit your websites every day without you knowing it and without you needing them. Many of them doesn't pass all the headers and this is an easy way to detect them.

Missing Header Accept

One of the most missing header fields is the Header Accept. This header is used to specify certain media types which are acceptable for the response. If no Accept header field is present, then it is assumed that the client accepts all media types. If an Accept header field is present, and if the server cannot send a response which is acceptable according to the combined Accept field value, then the server SHOULD send a 406 (not acceptable) response.

Not sending Header Accept means that the visitor is some kind of bot and not a legitimate visitor as no browser will omit to send this header field. Which doesn't necessarily makes it a bad bot. Still the main purpose of AbyssGuard is to filter out the unwanted visitors and to save your website resources only for the legitimate ones.

Missing User-Agent

The User-Agent header containts information about the user agent originating the request. This headers is used for statistics and for identification. The User-Agent can contain multiple product tokens and versions. No browser will omit to pass this header so visitors without it are bots of some kind.

Note that the User-Agent can be easily manipulated in order to display false information, for example the user may present itself as Googlebot. AbyssGuard has builtin Search Engine Detection.

Invalid or missing Host

The Host request-header field specifies the Internet host and port number of the resource being requested, as obtained from the original URI given by the user or referring resource. A client MUST include a Host header field in all HTTP/1.1 request messages. If the requested URI does not include an Internet host name for the service being requested, then the Host header field MUST be given with an empty value. The Host field value MUST represent the naming authority of the origin server or gateway given by the original URL. Sometimes however badly configured bots pass incorrect or missing host or the host is in incorrect format which will result in ban. No real browser will omit to pass this header.

In attempt not to ban any real visitors AbyssGuard has builtin exempts for some legitimate services and older browsers. If however there are visits from Services or Robots that doesn't pass the correct headers, you can always Exempt them in your Client's Exempts page.

In order for the Headers Check to work you have to enable the Core Protection option in your Settings.