Frontend Learning HTTP Digest Authentication

Previous words

The basic certification described in the previous article is convenient and flexible, but extremely unsafe. The user name and password are transmitted in clear text, and no action is taken to prevent tampering with the message. The only way to safely use Basic authentication is to use it in conjunction with SSL

Digest authentication is compatible with basic authentication, but it is more secure. This article will introduce the principle and practical application of the summary authentication in detail.

Working principle

Abstract authentication is another HTTP authentication protocol that attempts to fix a serious flaw in the Basic authentication protocol. Specifically, the summary certification has been improved as follows: Never send passwords on the network in clear text, prevent malicious users from capturing and replay the authenticated handshake, selectively prevent tampering with the contents of the message, and guard against several other common ways of attacking

Abstract authentication is not the most secure protocol and does not meet the many requirements of secure HTTP transactions. For these requirements, it is more appropriate to use Transport Layer Security (Transport layer Safety, TLS) and secure HTTP (secure HTTP, HTTPS) protocols.

However, abstract authentication is much more powerful than the basic authentication it is to replace. Abstract authentication is also much more powerful than many of the common strategies that are recommended for use by other Internet services (such as CRAM-MD5 that have been recommended for LDAP, pop, and IMAP)

So far, abstract certification has not been widely used. However, due to the inherent security risks of basic authentication, the HTTP designer has suggested in RFC 2617: "As far as practicable, all services that are currently in use with Basic authentication should be converted as soon as possible to a Digest authentication method"

The maxim followed by Digest authentication is "Never send a password over the network." Instead of sending a password, the client sends a "thumbprint" or a "digest" of the password, which is the non-reversible scrambling code for the password. Both the client and the server know the password, so the server can verify that the provided digest matches the password. To get the summary, there is no other way to find out which password the digest is from except to try out all the passwords.

Learn how Digest authentication works

In Figure A, the client requests a protected document

In Figure B, the server refuses to provide the document until the client can prove that it knows the password to confirm its identity. The server initiates a challenge to the client asking for a password in the form of a user name and digest

In Figure C, the client passes a digest of the password, proving that it knows the password. The server knows the password for all users, so the summary provided by the customer can be compared to the summary computed by the server itself to verify that the user knows the password. The other party is hard to forge the correct digest without knowing the password

In Figure D, the server compares the summary provided by the client to the summary computed within the server. If it matches, it means the client knows the password. You can set a digest function so that it produces many numbers, making it impossible to be lucky to guess a digest. After the server has matched the validation, the document is made available to the client-the entire process does not send a password on the network

"One-way Summary"

Abstract is a "concentration of information subject", it is a one-way function, is mainly used to convert the infinite input value into a finite concentration output value. A common digest function, MD5, converts a sequence of bytes of any length into a 128-bit digest

For these summaries, the most important thing is that if you do not know the password, it will be very difficult to guess correctly the summary of the server to be sent. Similarly, if there is a summary, it is very difficult to determine which of the countless input values it produces.

A 128-bit digest of the MD5 output is usually written as 32 hexadecimal characters, each representing 4 bits. A few sample input MD5 summaries are given in the following table

Digest functions are sometimes referred to as cryptographic checksums, one-way hash functions, or fingerprint functions

"Prevent replay attacks with random numbers"

Using a one-way digest eliminates the need to send a password in clear text, you can send only a digest of the password, and you can be confident that no malicious user can easily decode the original password from the digest

However, simply hiding the password does not avoid danger, because even if you do not know the password, the person with ulterior motives can intercept the digest and replay it to the server over and over again. The digest is as good as the password

To prevent such replay attacks, the server can send a special token called a random number (nonce) to the client, which changes frequently (probably every millisecond, or every time the authentication is changed). The client should append this random number token to the password before calculating the digest.

Adding a random number to a password causes the digest to change with each change in the random number. The recorded password digest is valid only for a specific random value, and without a password, the attacker cannot calculate the correct digest, which prevents the replay attack from occurring.

Abstract authentication requires the use of random numbers, because this small replay weakness makes the non-randomized digest authentication as fragile as the basic authentication. The random number is transmitted from the server to the client in the Www-authenticate challenge.

"Handshake mechanism"

HTTP Digest authentication protocol is an upgrade version of the authentication method, the use of a header similar to the Basic authentication. It adds some new options to the traditional header and adds a new optional header Authorization-info

Describes a simplified three-step handshake mechanism for Digest authentication

In step (1), the server calculates a random number

In step (2), the server places this random number in the Www-authenticate Challenge message, which is sent to the client with the list of algorithms supported by the server

In step (3), the client chooses an algorithm to calculate a digest of the password and other data

In step (4), the summary is sent back to the server in a authorization message. If the client wants to authenticate the server, it can send the client random number

In step (5), the server receives the digest, the selected algorithm, and the support data, calculating the same digest as the client. The server then compares the locally generated digest with the digest sent over the network to verify that it matches. The client summary is created if the client in turn challenges the server with a random number of clients. The server can pre-calculate the next random number and pass it to the client in advance so that the next time the client can send the correct digest beforehand

Many of these messages are optional and have default values

The principles of Basic authentication and digest authentication are compared

Abstract algorithm

The core of abstract authentication is a one-way summary of the combination of public information, confidential information and time-limited random values.

The digest is based on a pair of functions consisting of a one-way hash function H (RF) and a digest kd (s,d), where s represents a password, d represents data, and a block of data containing security information, including a password, called A1, and a block of data that contains non-confidential attributes in the request message, known as the A2 three components. H and KD process two pieces of data A1 and A2, generating summaries

Summary authentication supports the selection of various digest algorithms. The two algorithms recommended by RFC 2617 are MD5 and md5-sess ("Sess" for the session), and if no other algorithm is specified, the default algorithm is MD5

Whether using MD5 or md5-sess, a function h is used to calculate the MD5 of the data, and a digest function kd is used to calculate the MD5 of the password and non-confidential data connected by colons.

H (<data>) = MD5 (<data>) KD (<secret>,<data>) = h (Concatenate (<secret>:< data>))

A block of data called A1 is the product of a password and protected information, which contains such things as user names, passwords, protected domains, and random numbers. A1 only involves security information and is independent of the underlying message itself. A1 will be used together with H, KD and A2 for summary calculations

RFC 2617 defines two ways to calculate A1 based on the algorithm selected

1, MD5

Run a one-way hash function for each request. A1 is the user name, domain, and password triples that are concatenated with colons.

2, Md5-sess

Run the hash function only once for the first www-authenticate handshake. A CPU-intensive hash of the user name, domain, and password is placed before the current random number and the client random number (cnonce)

Data Block A2 represents information about the message itself, such as the URL, the request method, and the body of the message entity. A2 helps prevent tampering of methods, resources, or messages. A2 will be used with H, KD and A1 for summary calculations

RFC 2617 defines two strategies for A2, based on the quality of protection chosen (QOP)

The first policy contains only HTTP request methods and URLs. This policy is used when qop= "auth", which is the default condition

The second strategy adds the body part of the message entity to provide a certain level of message integrity detection. qop= "Auth-int" when used

The Request-method is the HTTP request method. Uri-directive-value is the request URI in the request line. May be a "*", Absoluteurl, or abs_path, but it must be consistent with the request URI. If the request URI is Absoluteurl, it must be an absolute URL

RFC 2617 defines two ways to calculate a summary after the H, KD, A1, and A2 are given

The first approach is compatible with the old specification RFC 2069 and is used when no qop option is available. It calculates the digest using the hash value of the classified information and the random message data.

The second approach is now recommended-this approach includes support for random number calculations and symmetric authentication. As long as Qop is auth or auth-int, use this method. It adds random counts, Qop, and cnonce data to the digest.

When a client responds to a www-authenticate challenge to a protected space, it initiates an authentication session of this protection space (a domain that is combined with the standard root of the access server defines a "protected space")

The authentication session lasts until the client receives another Www-authenticate challenge from any server that protects the space. The client should remember the user name, password, random number, random count, and some cryptic values related to the authentication session so that it can be used in future to build the authorization header of the request in this protection space

When a random number expires, the server can choose to accept the information even if the old authorization header contains random numbers that are no longer fresh. The server can also return a 401 response with a new random number, allowing the client to retry the request, specifying that the response is stale=true, indicating that the server is telling the client to retry with a new random number, instead of re-prompting for a new user name and password.

"Pre-authorization"

In the normal authentication mode, before the end of the transaction, each request must have a loop of request/challenge, a

If the client knows in advance what the next random number is, it can cancel the request/Challenge loop so that the client can generate the correct authorization header before the server makes a request. If the client can compute it before the server requires it to compute the authorization header, it can pre-send the authorization header to the server without having to make a request/challenge. Figure B shows the effect of this approach on performance
 
Pre-authorization is not important for basic certification and is common. Browsers typically maintain some client databases to store user names and passwords. Once the user is authenticated with a site, the browser usually sends the correct authorization header for subsequent requests to that URL

Because the Digest authentication uses the random number technique to destroy the replay attack, the pre-authorization is slightly more complicated for the Digest authentication. The server generates arbitrary random numbers, so it is not always possible to determine what authorization header should be sent before the client receives the challenge.

Abstract authentication also provides several pre-authorization methods while preserving a lot of security features. Here are three optional ways in which the client can obtain the correct random number without waiting for a new www-authenticate challenge

1. The server pre-sends the next random number in the Authentication-info success header

The next random number can be pre-supplied to the client in the Authentication-info success header. This header is sent with the previous successful authentication

Authentication-info:nextnonce="<nonce-value>"

With the next random number, the client can pre-publish the authorization header.

Although this pre-authorization mechanism avoids the request/challenge loop and speeds up transaction processing, it also destroys the ability to pipe multiple requests to the same server, since it is important to receive the next random value before publishing the next request. Pipelining is a fundamental technique to avoid delays, so it can cause significant performance damage

2, the server allows the use of the same random number for a short period of time

The second method is to reuse the random number within a limited number of times. For example, a server might allow a random number to be reused 5 times, or 10 seconds to be reused

In this case, the client is free to publish the request with the authorization header, and because the random number is known beforehand, the request can also be piped. When a random number expires, the server sends a 401 unauthorized challenge to the client and sets the www-authenticate:stale=true directive

Www-authenticate:digest realm="<realm-value> " nonce="< nonce-value>" stale=True

Reusing random numbers makes it easier for attackers to successfully implement replay attacks. While this does reduce security, the lifetime of the reused random number is controllable, from strict prohibition of reuse to longer reuse, so you should be able to find a balance between security and performance

In addition, there are other features that make replay attacks more difficult, including incremental counters and IP address testing. But these technologies can only make the implementation of the attack more cumbersome, not to eliminate the resulting security risks

3. Client and server using synchronous, predictable random number generation algorithm

The third method is to use time-synchronized random number generation algorithm, the client and server can generate the same random number sequence (such as security ID card) which cannot be easily predicted by the third party according to the shared key.

"Random Number"

The content of the random number is opaque and relevant to the implementation. But the pros and cons of performance, security, and convenience depend on the smart choice.

RFC 2617 recommends the use of this hypothetical random number formula:

"" " : " private-key))

Where Time-stamp is the server-generated time or other value that is not duplicated, the etag is the value of the HTTP ETag header associated with the requested entity, and Private-key is the data that only the server knows

With this form of random number, the server can recalculate the hash after receiving the client's authentication header, rejecting the request if the result does not match the random number of the header, or if the timestamp value is not new. The server can limit the effective duration of random numbers in this way

Contains the etag to prevent replay requests to the updated resource version. In the random number contains the client's IP address, the server seems to be able to limit the original client to obtain this random number of reuse of this random number, but this will undermine the work of the proxy cluster. When using a proxy cluster, multiple requests from a single user are typically transferred through different proxies, and IP address spoofing is not difficult to implement.

Implementations can choose not to accept random numbers or summaries that were previously used to prevent replay attacks. Implementations can also choose to use a one-time random number or digest for a post or put request, using timestamps for GET requests

"Symmetric Authentication"

RFC 2617 extends the digest authentication mechanism to allow the client to authenticate the server. This is done by providing a random value for the client, and the server generates the correct response summary based on its correct knowledge of sharing confidential information. The server then returns this summary to the client in the Authorization-info header

This symmetric authentication method is standardized as RFC 2617. In order to be compatible with the original RFC 2069 standard, it is optional, but since it provides some important security promotion mechanisms, it is strongly recommended that all of today's clients and servers implement all RFC 2617 features. In particular, symmetric authentication is required as long as the qop instruction is provided, and symmetric authentication is not required when no QOP instruction is performed

The response summary is calculated similar to the request digest, but because there is no method in the response and the message Entity data is different, only the message body information A2
The following table is the definition of A2 in the request summary

The following table is the definition of A2 in the response summary

The Cnonce value and the NC value must be the corresponding value in the client request that this article responds to. If gop= "auth" or qop= "Auth-int" are specified, the response auth, cnonce, and nonce count instructions must be provided

Enhanced protection

Qop fields can be provided in three summary headers: www-authenticate, Authorization, and Authentication-info

With the Qop field, clients and servers can negotiate the protection of different types and challenges. For example, even if the transmission speed is severely reduced, some transactions may also have to check the integrity of the message body

The server first outputs a comma-delimited list of qop options in the Www-authenticate header. The client then chooses one of the options it supports and satisfies its needs and puts it back to the server in the Authorization Qop field

The Qop field is optional, but is optional only if the latter is compatible with the original RFC 2069 specification. All modern summary implementations should support the QOP option

RFC 2617 defines the initial values for two protection qualities: The auth that represents authentication, and the authentication auth-int with message integrity protection. Additional qop options may also appear in the future

If integrity protection is used (qop= "Auth-int"), H (the body part of the entity) is the hash of the entity body part, not the body part of the message. For the sender, it is calculated before any transfer encoding is applied, and for the receiver, it should be computed after all transfer encodings have been removed. Note that for any content type that contains multiple parts, the boundaries of the multipart and the headers embedded in each part are included

Both the basic authentication and the Digest authentication protocol contain the WWW-AUTHENTICATE header-hosted authorization challenge and the authorization header-hosted authorization response. The summary certification also adds an optional Authorization-info header, which is sent after successful authentication to implement the three-step handshake mechanism and transmits the next random number. The following table gives the primary certification and summary certification

Practical issues

Here are a few practical questions to consider when using Digest authentication

"Multi-Challenge"

The server can initiate multiple challenges on a resource. For example, if the server does not understand the capabilities of the client, it can provide both a Basic authentication challenge and a summary certification challenge. When a client faces multiple challenges, it must respond with the strongest challenge mechanism it supports.

The challenge itself may contain a comma-delimited list of authentication parameters. If the Www-authenticate or proxy-authenticate header contains multiple challenges, or multiple www-authenticate headers are provided, User Agent Agent should be very careful when parsing www-authenticate or proxy-authenticate header field values

[note] Many browsers only support Basic authentication, which is required to be the first authentication mechanism submitted to it.

With the range of authentication options available, there is a clear "weakest link" on security issues. The server should include it only if the basic authentication is the lowest acceptable authentication mode, and the administrator should also warn the user that even if different levels of security are running, there is a risk of using the same password between the systems.

"Error Handling"

In Digest authentication, if an instruction or its value is used improperly, or if a required instruction is missing, the response to the

If the requested digest does not match, a login failure should be logged. Multiple successive failures of a client may indicate that an attacker is guessing the password

Authentication server Be sure that the URI directive specifies the same resource as the resource specified in the request line. If different, the server should return a "Bad Request" error. This can be a sign of an attack, so the server designer might consider logging such errors. This field contains content that is duplicated in the request URL and is used to respond to changes that the intermediary agent might make to the client request. The summary of the request calculation that has been modified (but the estimated semantics are equivalent) may be different from the summary computed by the client

"Protect Space"

Domain value, combined with the standard root URL of the server being accessed, defines the protection space

Domains allow you to divide the protected resources on a server into a set of protected spaces, each with its own authentication mechanism and/or authorization database. The domain value is a string that is typically assigned by the original server and may have additional semantics specific to the authentication scheme

[note] There may be multiple authorization schemes that are the same, and the domain different challenges

Secure space determines the area where the certificate can be automatically applied. If a previous request has been authorized, for a period of time, all other requests in the protection space can be reused for the same certificate, and the length of time is determined by the authentication scheme, parameters, and/or user preferences. Unless the authentication scheme is otherwise defined, a single protected space cannot be extended beyond its server scope

The specific calculation of the protected space depends on the authentication mechanism

In Basic authentication, the client assumes that all paths in the request URI or under it are in the same protection space as the current challenge. Clients can pre-commit authentication of resources in this space without waiting for another challenge from the server

In the summary authentication, the Www-authenticate:domain field of the challenge defines the space of protection more precisely. The domain field is a list of URIs enclosed in quotation marks, separated by spaces. It is generally assumed that all URIs in the domain list and all URIs that are logically under these prefixes are in the same protected space. If there is no domain field, or if this field is empty, all URIs on the challenge server are in the protected space

"Overriding URI"

The proxy can override the URI by changing the URI syntax without changing the actual resource described

You can standardize the hostname or replace it with an IP address

You can replace embedded characters with a "%" escape form

If some of the attached properties of a type do not affect getting resources from a specific original server, you can attach or insert them into the URI

The proxy can modify the URI, and the Digest authentication will check the integrity of the URI value, so if any modification is made, the Digest authentication will be destroyed

Cache

When a shared cache receives a request that contains a authorization header and a response from that request, it must not use the response as an answer to any other request unless one of the following two Cache-control directives is provided in the response

1. If the original response contains the Cache-control directive must-revalidate, the cache can use the entity portion of the response to answer the subsequent request. But it first needs to authenticate with the original server again with the request header of the new request so that the original server can authenticate the new request

2. If the original response contains the Cache-control directive public, the entity part of the response can be returned in response to any subsequent request

Security Considerations

"First Tamper"

To provide a straightforward tamper-proof system, either end-to-end encryption or a digital signature of the header-preferably a combination of the two. Abstract the focus of certification is to provide a tamper-proof authentication mechanism, but it is not necessarily to extend this protection to the data. The header with a certain level of protection is only www-authenticate and authorization

"Replay Attack"

In the current context, replay attacks refer to the use of authentication certificates stolen from one transaction for another transaction. Although this is also a problem for get requests, it is necessary to provide a simple way for post and put requests to avoid replay attacks. While transferring form data, successful replay of the previously used certificate can lead to serious security issues

Therefore, in order for the server to accept "replayed" certificates, you must also send random numbers repeatedly. One way to alleviate this problem is to have the server generate a random number that contains a summary based on the client IP address, timestamp, resource ETag, and private server key. In this way, the combination of an IP address and a short time-out value can pose a significant barrier to an attacker.

But there is an important flaw in this solution. Creating a random number with a client IP address destroys the transport of the proxied cluster. In this type of transport, multiple requests from a single user may pass through different proxies. Also, IP spoofing is not difficult to achieve

One way to completely avoid replay attacks is to use a unique random number for each transaction. In this implementation, the server publishes a unique random number and a timeout value for each transaction. The published random number is valid only for the specified transaction and only within the duration of the timeout value. This approach increases the burden on the server, but the burden is negligible

"Multi-authentication mechanism"

When the server supports multiple authentication mechanisms (such as Basic authentication and Digest authentication), the option is usually provided at the Www-authenticate header. Because the client is not required to select the most powerful authentication mechanism, the resulting authentication is almost the same as the weakest authentication scheme.

The most straightforward way to avoid this problem is to let the client always choose the one that is most powerful in the available authentication scheme. If this is not possible, the only option is to use a proxy server that only maintains the strongest authentication scheme, but only in areas where all clients are known to support the selected authentication scheme

"Dictionary attack"

Dictionary attack is a typical password-guessing attack mode. A malicious user taps a transaction and uses a standard password-guessing program for random number/response pairs. If the user is using a relatively simple password, and the server is using a simple random number, it is likely to find a match. If there is no password expiration policy, as long as there is enough time and a single cost to crack the password, it is easy to collect enough passwords, causing substantial damage

In addition to using complex, relatively hard-to-decipher passwords and appropriate password expiration strategies, there is really no good way to solve this problem

"Malicious agent attack and intermediary attack"

Many internet traffic now flows through an agent in this or that place. With the advent of redirection technology and interception agents, users are not even aware that his requests have passed through an agent. If one of these proxies is malicious or vulnerable to intrusion, the client is placed under a man-in-the-middle attack

This attack can take the form of eavesdropping, or you can delete all the options provided, replace the existing authentication mechanism with the weakest authentication policy (such as Basic authentication) and modify it

One way to invade a trusted proxy is to use its extended interface. Sometimes the agent provides a complex programming interface that can write an extension (for example, plug-in) for such agents to intercept and modify the traffic. However, the security provided by the data center and the agent itself makes the possibility of a man-in-the-middle attack through malicious plug-in very slim.

There is no good way to solve the problem. A viable solution involves providing the client with visible clues about the authentication feature, configuring the client to always use the one that is most powerful in the available authentication policy, and so on. But even with the most powerful authentication strategy, the client is still vulnerable to eavesdropping. The only easy way to prevent these attacks is to use SSL

"Select Clear text Attack"

A client using Digest authentication generates a response with a random number provided by the server. However, if there is a compromised or malicious agent in the middle of intercepting traffic (or having a malicious original server), it is easy to provide random numbers for the client's response calculations. The password analysis process that uses a known key to calculate the response can simplify the response. This approach is known as the Select plaintext attack (chosen plaintext attack). There are several variants of the select plaintext attack

1. Pre-Computed dictionary attacks

This is a combination of dictionary attacks and the choice of plaintext attacks. First, the server that launched the attack generates a set of responses with a predetermined number of random and common password changes, creating a dictionary. Once a sizable dictionary is available, the attack server or proxy can complete the blocking of traffic and send a predetermined number of random numbers to the client. When an attacker obtains a response from the client, the generated dictionary is searched for a match. If there is a match, the attacker captures the user's password

2. Mass violent attacks

The difference between a batch brute-force attack is how the password is calculated. Instead of trying to match a pre-computed digest, it uses a set of machines to enumerate all possible passwords in the specified space. As machines run faster and faster, the possibility of violent attacks is becoming more and more powerful.

In short, the threats posed by these attacks are easy to deal with. One way to prevent these attacks is to configure the client to use the optional cnonce directive, which is generated based on the client's judgment, rather than the random number provided by the server (the random number may be compromised by the attacker). This method, combined with some policies that enforce the use of strong passwords, and a good password expiration policy, can completely eliminate the threat of the choice of plaintext attacks.

"Store Password"

The Digest authentication mechanism compares the user's response to what is stored inside the server-typically a user name and an H (A1) tuple pair, where H (A1) is derived from a digest of the user name, domain, and password.

Unlike traditional password files in Unix machines, if the Digest authentication password file is compromised, the attacker will be able to use all the files in the domain immediately, and no further decoding is required.

The way to eliminate this problem is to protect it as if it contained a plaintext password in a password file, and to ensure that the domain name is unique across all domains. Thus, if the password file is compromised, the damage is restricted to a specific domain. Full path domain name with host and domain to meet this requirement

Although Digest authentication provides a much stronger and more secure solution than basic authentication, it does not provide any guarantee of the security of the content-a truly secure transaction can only be achieved through SSL

Instruction description

The www-authenticate instructions are described in detail in the following table, as explained in RFC 2617

The following table provides a detailed description of the authentication directive.

The following table provides a detailed description of the AUTHENTICATION-INFO directive.

Frontend Learning HTTP Digest Authentication