Use OpenSSL APIs for Secure Programming, Part 1: Secure handshake (2)

Source: Internet
Author: User
Tags openssl api rfc ssl connection mitm attack

Prevent man-in-the-middle (mitm) Attacks

Document options

Level: elementary

Kenth Ballard (, free programmer

May 23, 2005

Secure handshake in a Secure Socket Layer (SSL) session is very important because all security in the connection is established during the handshake. This article describes how to enhance the security of the SSL handshake to prevent mitm attacks. At this time, the intruded party may pretend to be another source. This article also introduces the concept of digital certificates and how the OpenSSL API handles digital certificates.

A short time ago, the security handshake was a mark of the Business implementation of both parties. After all, handshaking is a face-to-face opportunity to evaluate potential partners. A secure and trusted handshake means that both Parties believe that what they are doing is beneficial to both parties. Insecure handshakes indicate that only one party has a correct understanding of the transaction.

Handshakes work in the same way as online transactions.

The previous article "using OpenSSL APIs for Secure Programming, Part 1: API overview" on developerworks describes how to use OpenSSL to create basic and simple secure connections. However, this article only shows the basic default settings. It does not introduce how to customize the content. However, I suggest you read this article to make your understanding of this article more complete, because the previous article introduced the concept of digital certificates, it also describes how to determine whether a certificate has successfully passed OpenSSL internal verification.

This article will introduce OpenSSL in depth and introduce how to enhance the security of handshaking and prevent the so-calledIntermediary(Mitm) attacks.

About digital certificates

After this article, we will introduce how to obtain and verify a digital certificate, so we will quickly discuss what a digital certificate is and how it works. Skip this section if you are familiar with data encryption and SSL. For more information about encryption and SSL, see the articles and tutorials listed in references later in this article.

The simplest form of digital certificates isAsypolicric cryptography key). Currently, the digital certificate standard contains some identification information, which is also contained in the key. A typical digital certificate contains the name of the owner (if the certificate is used on a Web server, the name is the complete domain name), the contact information, and a valid date range, and a security signature to verify that the certificate has not been tampered.

You can use the OpenSSL command line tool or other tools for this purpose to create a digital certificate. However, any digital certificate created by anyone has a trust issue. A digital certificate is not only an encryption key, but also an online certificate. The certificate will prove your identity to those who attempt to communicate with you. To display the trust relationship, a digital certificate can be signed by an authenticated authority (CA.

Certification Authority acts as a trusted third party in the digital security field. It is very difficult to prove the identity of an entity in the online field to take over this challenge. They provide proof of identity for users who have purchased or signed the certificate. Therefore, to trust a certificate, you only need to trust the Certificate Authority. You can use a ca-based trust certificate to demonstrate your trust in the authentication authority. Verisign and thawte are well-known certification authorities.

If the security of a certificate has been threatened, the certificate will be discarded-that is, it is declared invalid. When a certificate is declared invalid, the Ca cannot notify all persons who have copied the certificate. Instead, CA releasesCertificate Revocation List (Certificate Revocation List)(CRL ). Browsers and other programs that use digital certificates can verify that the certificate has been revoked by its owner or Ca.

You can also use the OCSP protocol to check the certificate revocation. OCSP representativeOnline Certificate Status Protocol (Online Certificate Status Protocol)It is defined in RFC 2560. OpenSSL provides both OCSP and CRL functions, but the introduction to these functions is beyond the scope of this article. Currently, the digital certificate standard is X.509, which is defined in RFC 3280.

Back to Top

Handshake before starting the business

As this article focuses on how to process the server digital certificate during the handshake process, let's take a closer look at how the handshake works. Skip this section if you are familiar with the SSL process.

Starting a handshake on a connection usually starts when the client says "hello" to the server. The helllo message (which is said in the Specification) contains the security parameters of the client:

  • SSL version number
  • Randomly generated data
  • Password settings
  • Other content required for communication

The server responds with its own Hello message, which contains the server's security parameters. The information is of the same type as the information provided by the client. The server also sends its own digital certificate. If the client needs to authenticate the connection, the server sends a request to obtain the client certificate.

After the client receives the Hello Message from the server, the digital certificate is verified. This includes checking the parameters of the certificate to ensure that the certificate has never been compromised and is used within its validity period.

Another step is to check the certificate name based on the host name used by the connection. Although this is not part of the SSL standard, this step is highly recommended to prevent man-in-the-middle attacks. This step verifies that the certificate is the entity from which you think it comes from. If the two entities do not match here, you can only suspect that the certificate is invalid.

The random data shared between the client and the server is used to createPremaster secretThis is a shared secret value that only the server and client will know and is only used for this session. This secret value encrypts the server's digital certificate and sends it to the server to verify the identity of the client.

If the server requests client authentication, the client will create a one-way hash value for the randomly generated data during the handshake (only the server and client know it. The client uses its own private key to sign the hash value and sends the signed data and digital certificate to the server. The server uses this information to authenticate the client.

If the authentication succeeds, the server and client will use the shared random data to createMaster secret. From the master secret, the client and server can be createdSession Keys)This is the symmetric key in the symmetric password used to encrypt session data.

The client sends a message to the server indicating that the handshake has been completed, and sends an encrypted one-way hash value to the server for verification. This ends the handshake process. The server also sends a similar message to the client. The data must be verified by the client and server before the handshake ends and communication starts.

Back to Top


Although this is a game played by children, it is also a very serious attack on the Public Key Infrastructure (PKI. When discussing digital certificates, we must consider Man-in-the-middle attacks, because man-in-the-middle attacks can make these preventive measures useless regardless of the security parameters after SSL connections.

Let's assume that Kathy and Samantha want to use SSL for communication. Isabel is a third party who intercepts the connection request and acts as a proxy before the two of them. When she notices that an SSL connection is being established, she pretends to pretend to Samantha as Kathy, and to disguise it as Samantha. Because she is in the middle, the contents of both parties can be intercepted. If this session contains account and personal information, she can use this information to steal others' identities.

In this case, the common names in the trust chain and certificate can prevent man-in-the-middle attacks. The certificate is exchanged during the handshake. When you analyze the certificate validity, the system also checks the trusted signature. If the common name in the server certificate is verified together with the rest of the certificate, this attack will not be cracked, right? In fact, this is not entirely true.

Let's assume that Isabel has a certificate with the name of Samantha, and this certificate is signed by a CA in the Kaspersky trust model. In this case, only checking the common name does not prevent the occurrence of mitm attacks. The certificate and its trust relationship are valid, and the name is also checked. Now we have a big problem.

However, if you consider certification authority, this issue is not important. Most certification authorities attempt to verify his identity before issuing a digital certificate with a personal name. For this reason, it is difficult for Isabel to obtain a digital certificate with the Samantha name from a well-known Certification Authority.

If you work in the CA, You can weaken the role of this security facility. For example, if Ca and Isabel both work in one company (in other words, they work internally "). Then, the signature key may be stolen by the internal staff of the CA, and then they can use the name of anyone to create any certificate. Although you need to use the private part of the certificate when creating a signature, you can also steal the password by using some engineering methods or similar technologies.

Mitm attack in useProxy ServerEspecially important. Because secure connections must be provided by the proxy server with a tunnel to reach the destination, malicious proxy servers can easily steal any session. Malicious proxies can pretend to exist as if a tunnel exists. Remember this when using the "anonymous proxy" on the Internet: you are sending user names and passwords through their systems. How can you trust them?

However, we must realize that such attacks do not only exist in the computer and digital security fields. A woman once used a technology similar to the mitm attack to rob many families of a lot of money (see references ).

Back to Top

OpenSSL and digital certificates

OpenSSL has a library dedicated to digital certificates. If you already have the OpenSSL source code, the source code of this library is located in the crypto/X509 and crypto/x509v3 directories. The source code defines several structures for processing digital certificates. These structures are listed in Table 1.

Table 1. OpenSSL structure related to X.509 certificates

Structure Function
X509 Contains all data about the digital certificate
X509_algor Provides algorithms for this certificate design.
X509_val Valid time span of the certificate
X509_pubkey The Public Key algorithm of the certificate, usually RSA or DSA.
X509_sig Hash signature of the certificate
X509_name_entry Items of the data contained in the certificate
X509_name Stack containing name items

These are only the structures involved. Most X.509 structures used in OpenSSL are rarely used in applications. In the structures listed above, this article only uses two: X509 and x509_name.

Above these structures, there are some functions used to process digital certificates. These functions are named after the structures they apply. For example, a function whose name starts with x509_name is usually applied to an x509_name structure. We will introduce some functions as needed later.

Back to Top

Provide your own trust certificate

Before verifying the trust of a digital certificate, you mustSSL_CTXThe object provides a default trust certificate, which can be provided in several ways, but the simplest way is to save the certificate as a pem file and useSSL_CTX_load_verify_locations(ctx, file, path);Load it into OpenSSL.fileIs the path to a file that contains one or more PEM certificates.pathIs the path to one or more PEM format files, but the file name must use a specific format. It is easy to save the trust certificate in a pem file, so that the path parameter can be blank, as shown below:SSL_CTX_load_verify_locations(ctx, "/path/to/trusted.pem", NULL);.

Although it is easier to add or update a trusted certificate when multiple individual files exist in a directory, you are unlikely to update the trusted certificate so frequently, so you don't have to worry about this issue.

Back to Top

Verify Certificate

UseSSL_get_verify_result()To determine the internal validation results of OpenSSL certificates. IfSSL_get_verify_result()The Returned Code is not x509_v_ OK. Does this mean that this certificate is invalid? It depends on the returned code.

Generally, if the returned code is not x509_v_ OK, the certificate may be faulty, or the certificate may have security risks. Always remember one thing: When OpenSSL verifies the certificate, some security checks are not executed, including the certificate failure check and validation of the common name in the certificate.

SSL_get_verify_result()The Returned Code is inverifyThis is listed under apps. Some codes are not used yet, meaning they will never be returned. Some code is very important, while others are not. For example, if the saved trust certificate is not loaded and the trusted certificate cannot be verified, it is up to the developer to determine whether to continue communication.

Regardless of the verification result, whether or not to continue using some potentially insecure parameters depends entirely on the developer. Because the certificate may be insecure, the error code is returned.

Back to Top

Retrieve Certificate

If you want to display the certificate content to the user or verify the certificate based on the host name or certificate authority, you need to retrieve the certificate content. To retrieve the certificate after verifying the test result, callSSL_get_peer_certificate(). It returns an X509 pointer to the certificate. If the certificate does not exist, null is returned (see Listing 1 ).

Listing 1. Retrieving certificates

                        X509 * peerCertificate;                        if(SSL_get_verify_result(ssl) == X509_V_OK)                        peerCertificate = SSL_get_peer_certificate(ssl);                        else                        {                        /* Handle verification error here */                        }                        

Back to Top

Verify Certificate

The server certificate provided during handshake must have a name that matches the Host Name of the server. If not, the certificate should be marked as suspect. The internal verification process has verified the trust and validity period of the certificate. If the certificate has expired or contains an untrusted signature, the certificate will be marked as invalid. Because this is not part of the SSL standard, OpenSSL does not need to check the certificate name based on the host name.

The "name" of the certificate is actually the common name field in the certificate. This field should be retrieved from the certificate and verified based on the host name. If the two cannot match, only the certificate is suspected to be invalid. Some companies (such as Yahoo) use the same certificate on different hosts, even if the common name in the certificate is only used for one host. To ensure that the certificate is from the same company, you can perform more in-depth checks, but this depends entirely on the security needs of the project.

Two steps are required to retrieve a common name from a certificate:

  • Retrieve from certificate structureX509_NAMEObject.
  • ThenX509_NAMEObject Search name.

UseX509_get_subject_name()Retrieve from CertificateX509_NAMEStructure. This returnsX509_NAME. UseX509_NAME_get_text_by_NID()And save it to a string (as shown in Listing 2 ).

Listing 2. Retrieving and verifying the common name

                        char commonName [512];                        X509_NAME * name = X509_get_subject_name(peerCertificate);                        X509_NAME_get_text_by_NID(name, NID_commonName, commonName, 512);                        /* More in-depth checks of the common name can be used if necessary */                        if(stricmp(commonName, hostname) != 0)                        {                        /* Handle a suspect certificate here */                        }                        

Use the Standard C string function or the string library you are used to compare the common name and host name. The processing of non-matching depends entirely on the needs of the project or users' decisions. For more in-depth checks, we recommend that you use a separate string library to reduce complexity.

Back to Top

Gain trust

In this article, we have introduced how to enhance the security of handshakes to prevent man-in-the-middle attacks (the attacking party disguised as another source). We also introduced the concept of digital certificates, and how the OpenSSL API handles digital certificates.

Remember, it is very important to enhance the session security during an SSL session because all the security in the connection is established during the handshake process. How important each step described in this article depends on the project requirements and the developer's decision.


  • For more information, see the original article on the developerworks global site.

  • Download the source code from the author's Web site.
  • "Using OpenSSL APIs for Secure Programming: API overview" (developerworks, November July 2004) shows how to use OpenSSL APIs to establish a simple connection. This is a good pre-article.
  • Internet X.509 Public Key Infrastructure certificate and Certificate Revocation List (CRL) profile is a good resource for X.509 standards.
  • Written by Bruce SchneierSecrets and lies: digital security in a networked world(Wiley Publishing, inc., published in January 2004) briefly introduces intermediary and other security attack methods.
  • You can also read about man-in-the-middle attacks through free Wikipedia.
  • DataSan Francisco ChronicleOn September 19, March 18, 2004, a woman snatched others using a method similar to man-in-the-middle attack.
  • "Linux socket programming, Part 1" (developerworks, October 2003) describes how to start socket programming.
  • Communications programming concepts: Sockets is a good article about socket and how to use socket.
  • The series "Introduction to cryptography, Part 1: Overview" (developerworks, September 1st) describes cryptography in summary and details.
  • Find more resources for Linux developers in the developerworks Linux zone.
  • Join the developerworks community by joining developerworks blogs.
  • Purchase discount books for Linux in the Linux area of developer bookstore.
  • Order a free SEK for Linux. These two dvds contain the latest IBM trial software on the Linux platform, including DB2, Lotus, rational, Tivoli, and websphere.
  • Use IBM trial software in your next Linux development project, which can be obtained from the download directory on developerworks.

About the author


Kenth Ballard holds a Bachelor's degree in computer science from Peru State College (in Peru, Nebraska), where he is a school newspaperThe Peru state times. He also has a associate degree in computer programming at Southwestern Community College (in Creston, Iowa), where he is a half-skilled PC technician. His research fields include Java, C ++, COBOL, Visual Basic, network, database management, and Internet programming. You can contact kenth via

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.