Use OpenSSL APIs for security programming, Part 1: API Overview

The APIs used to learn how to use OpenSSL, the most famous Open Library for secure communication, are difficult because their documentation is incomplete. You can use the tips in this article to add this knowledge and use this API. After establishing a basic connection, you can view how to use the BIO library of OpenSSL to establish a secure connection and a non-secure connection. At the same time, you will learn some knowledge about error detection.

The OpenSSL API documentation is vague. There are not many tutorials on OpenSSL, so it may be difficult for beginners to use it in applications. So how can we use OpenSSL to implement a basic secure connection? This tutorial will help you solve this problem.

The difficulty of learning how to implement OpenSSL lies in its incomplete documentation. Incomplete API documentation usually prevents developers from using this API, which usually means it is doomed to fail. However, OpenSSL is still very active and is becoming more and more powerful. Why?

OpenSSL is the most famous Open Library for secure communication. In the result returned by searching "SSL library" in Google, OpenSSL is at the top of the list. It was born in 1998 and originated from the ssleay library developed by Eric Young and Tim Hudson. Other SSL toolkit includes gnu tls, which complies with the GNU General Public License release, and Mozilla Network Security Services (NSS) (refer to references later in this article for additional information ).

So what makes OpenSSL superior to gnu tls, Mozilla NSS, or all other libraries? License is a factor (see references ). In addition, gns tls (so far) only supports TLS V1.0 and SSL V3.0.

The release of Mozilla NSS follows both Mozilla Public License and gnu gpl, which allows developers to choose from. However, Mozilla NSS is larger than OpenSSL and requires other external libraries to compile the libraries. OpenSSL is completely self-contained. Similar to OpenSSL, most NSS APIs do not have documentation. Mozilla NSS received support for PKCS #11, which can be used for encryption tags such as smart cards. OpenSSL does not support this feature.

Prerequisites

To fully understand and use this article, you should:

• Proficient in C programming.
• Familiar with Internet communication and programming of Internet-Supported Applications.

You are not absolutely required to be familiar with SSL, because a brief description of sll will be provided later. However, if you want a link to the article about SSL in detail, see references. It is good to have cryptographic knowledge, but this is not necessary.

 

 

What is SSL?

SSL stands for the Secure Sockets Layer. It is a standard that supports secure communication over the Internet and integrates data cryptography into the Protocol. Data is encrypted before it leaves your computer and decrypted only after it reaches its intended destination. Certificates and cryptographic algorithms support all these operations. With OpenSSL, you will have the opportunity to understand them.

Theoretically, if the encrypted data is intercepted or eavesdropped before it reaches the target, the data cannot be cracked. However, since the computer changes faster than a year ago and the password translation method has developed, the possibility of cracking the encryption protocol used in SSL is also increasing.

SSL and secure connections can be used for any type of protocol on the internet, whether HTTP, POP3, or FTP. You can also use SSL to protect Telnet sessions. Although SSL can be used to protect any connection, SSL is not required for each type of connection. If the connection transmits sensitive information, use SSL.

 

 

What is OpenSSL?

OpenSSL is not just SSL. It can implement message digest, file encryption and decryption, digital certificates, digital signatures and random numbers. There are a lot of content about the OpenSSL library, which is far from an article.

OpenSSL is not just an API, but also a command line tool. The command line tool can do the same work as the API, and further test the SSL server and client. It also gives developers an understanding of OpenSSL capabilities. For more information about how to use the OpenSSL command line tool, see references.

 

 

What do you need

The latest version of OpenSSL is required first. Refer to the references section to determine where to obtain the latest source code that can be compiled by yourself, or the latest binary file (if you do not want to spend time compiling ). However, for security reasons, we recommend that you download the latest source code and compile it yourself. Binary versions are usually compiled and released by third parties rather than OpenSSL developers.

Some Linux releases come with the binary version of OpenSSL, which is sufficient for learning how to use the OpenSSL library. However, if you want to do something practical, you must get the latest version and keep it updated.

We recommend that you obtain the RPM package from the release manufacturer to update your OpenSSL release version. For security reasons, we recommend that you use the latest release version. If your release version does not support the latest OpenSSL version, we recommend that you only overwrite the library file and do not overwrite the executable file. The FAQ document that comes with OpenSSL contains details about this.

Note that OpenSSL is not officially supported on all platforms. Although the manufacturer has tried its best to make it cross-platform compatible, there is still the possibility that OpenSSL cannot be used on your computer and/or operating system. See OpenSSL web sites (links in references) for information on which platforms are supported.

To use OpenSSL to generate certificate requests and digital certificates, you must create a configuration file. In the OpenSSL packageappsFolder namedopenssl.cnf. I will not discuss this file, because it is not within the scope of this article. However, this template file has some very good comments, and if you search on the Internet, you can find a lot of tutorials about modifying this file.

 

 

Header file and initialization

This tutorial uses only three header files: SSL. H, bio. H, and err. h. They are all located in the OpenSSL subdirectory and are required to develop your project. To initialize the OpenSSL library, you only need three lines of code. Listing 1 lists all content. Other header files and/or initialization functions may be necessary for other functions.

Listing 1. Required header files

/* OpenSSL headers */#include "openssl/bio.h"#include "openssl/ssl.h"#include "openssl/err.h"/* Initializing OpenSSL */SSL_load_error_strings();ERR_load_BIO_strings();OpenSSL_add_all_algorithms();

 

 

Establish a non-secure connection

Whether the connection is secure or insecure, OpenSSL uses an abstract library named bio to process various types of communication including files and sockets. You can also set OpenSSL as a filter, such as a filter for UU or base64 encoding.

It is a little troublesome to fully describe the bio database here, so I will introduce it at 1.1 points as needed. First, I will show you how to establish a standard socket connection. This operation requires fewer lines of code than the BSD socket library.

Before establishing a connection (whether secure or not), You must create a pointer to the bio object. This is similar to creating a file pointer for a file stream in Standard C.

List 2. pointer

BIO * bio;

 

Open connection

To create a new connection, you must callBIO_new_connect. You can specify both the host name and port number in the same call. You can also split it into two separate calls: one is to create a connection and set the Host NameBIO_new_connectCall, the other is to set the port numberBIO_set_conn_port(OrBIO_set_conn_int_port.

In any case, once the bio host name and port number are specified, the pointer will attempt to open the connection. Nothing can affect it. If a problem occurs during bio object creation, the pointer will be null. To ensure successful connection, you must executeBIO_do_connectCall.

Listing 3. Create and open a connection

bio = BIO_new_connect("hostname:port");if(bio == NULL){ /* Handle the failure */}if(BIO_do_connect(bio) <= 0){ /* Handle failed connection */}

 

Here, the first line of code uses the specified host name and port to create a new bio object and format the object in the style shown. For example, if you want to connect to port 80 of www.ibm.com, the string will bewww.ibm.com:80. CallBIO_do_connectCheck whether the connection is successful. If an error occurs, 0 or-1 is returned.

Communicate with the server

Whether the bio object is a socket or a file, the read and write operations on it are completed through the following two functions:BIO_readAndBIO_write. Very simple, right? The wonderful thing is that it is always so.

BIO_readWill try to read a certain number of bytes from the server. It returns the number of bytes read, 0, or-1. In the blocked connection, the function returns 0, indicating that the connection is closed, and-1 indicates that the connection has an error. In the case of non-blocking connections, 0 indicates no data is available, and-1 indicates a connection error. YesBIO_should_retryTo determine whether the error may be repeated.

Listing 4. Read from the connection

int x = BIO_read(bio, buf, len);if(x == 0){ /* Handle closed connection */}else if(x < 0){ if(! BIO_should_retry(bio)) { /* Handle failed read here */ } /* Do something to handle the retry */}

 

BIO_writeWill try to write the byte into the socket. It returns the number of bytes actually written, 0, or-1. SameBIO_read, 0 or-1 does not necessarily indicate an error.BIO_should_retryIs the way to find out the problem. If you need to retry the write operation, it must use the same parameters as the previous one.

Listing 5. Write to connection

if(BIO_write(bio, buf, len) <= 0){ if(! BIO_should_retry(bio)) { /* Handle failed write here */ } /* Do something to handle the retry */}

 

Close connection

Closing the connection is also very easy. You can disable a connection in either of the following ways:BIO_resetOrBIO_free_all. If you need to re-use the object, use the first method. If you no longer use it, you can use the second method.

BIO_resetClose the connection and reset the internal status of the Bio object so that you can use the connection again. If you want to use the same object in the entire application, such as using a secure chat client, it is helpful to do so. This function does not return values.

BIO_free_allAs it said: it releases the internal struct and releases all associated memory, including disabling the associated socket. If Bio is embedded in a class, you should use this call in the class destructor.

Listing 6. Close the connection

/* To reuse the connection, use this line */BIO_reset(bio);/* To free it from memory, use this line */BIO_free_all(bio);

 

 

Establish a secure connection

Now we need to provide what needs to be done to establish a secure connection. The only thing to change is to establish and connect. All other contents are the same.

A secure connection requires a handshake after the connection is established. During the handshake, the server sends a certificate to the client, and then the client verifies the certificate based on a set of trusted certificates. It will also check the certificate to make sure it has not expired. To verify that the certificate is trustworthy, you must load a trusted certificate library before the connection is established.

The client sends a certificate to the server only when the server sends a request. This process is called client authentication. Use the certificate to pass the password parameter between the client and the server to establish a secure connection. Although the handshake is performed only after the connection is established, the client or server can request a new handshake at any time.

The netscasp article and RFC 2246 listed in the references section describe in more detail about handshakes and other aspects of establishing secure connections.

Set security connection

Several lines of code are required to set the Security connection. At the same time, a pointer of the ssl_ctx type is required. This structure stores some SSL information. You can also use it to establish an SSL connection through the BIO library. You can use the SSL method function to call ssl_ctx_new to create this structure.SSLv23_client_method.

Another SSL pointer is needed to maintain the SSL connection structure (required for some connections that can be completed in a short time ). You can use this SSL pointer to check the connection information or set other SSL parameters.

Listing 7. Setting SSL pointers

SSL_CTX * ctx = SSL_CTX_new(SSLv23_client_method());SSL * ssl;

 

Load a trusted certificate library

A trusted certificate library must be loaded after the context structure is created. This is required to successfully verify each certificate. If you cannot confirm that the certificate is trustable, OpenSSL will mark the certificate as invalid (but the connection can continue ).

OpenSSL comes with a set of trusted certificates. They are located incertsDirectory. However, each certificate is an independent file-that is, each certificate needs to be loaded separately. IncertsDirectory, there is a subdirectory to store expired certificates. An error occurred while attempting to load these certificates.

If you want to, you can load each file separately. For convenience, the trusted certificates of the latest OpenSSL release version are usually stored in the source code file named "truststore. in a single file. If you already have a trusted credential library and intend to use it for a specific project, you only need to replace "truststore. PEM (or use separate function calls to load them all.

YesSSL_CTX_load_verify_locationsTo load the trusted certificate library file. Three parameters are used: context pointer, path and file name of the trusted library file, and path of the certificate directory. The Directory of the trusted library file or certificate must be specified. If the request is successful, 1 is returned. If any problem occurs, 0 is returned.

Listing 8. Load the trust Database

if(! SSL_CTX_load_verify_locations(ctx, "/path/to/TrustStore.pem", NULL)){ /* Handle failed load here */}

 

If you plan to use a directory to store trusted libraries, you must name the files in a specific way. The OpenSSL document clearly explains how to do this. However, OpenSSL comes withc_rehashWhich can be used to configure foldersSSL_CTX_load_verify_locations.

Listing 9. Configure the certificate folder and use it

/* Use this at the command line */c_rehash /path/to/certfolder/* then call this from within the application */if(! SSL_CTX_load_verify_locations(ctx, NULL, "/path/to/certfolder")){ /* Handle error here */}

 

To specify all the required authentication certificates, you can name any number of individual files or folders as needed. You can also specify files and folders.

Create a connection

The pointer pointing to the SSL context is used as the unique parameter.BIO_new_ssl_connectCreate a bio object. You also need to obtain a pointer to the SSL structure. In this article, only this pointer is usedSSL_set_modeFunction. This function is used to set the ssl_mode_auto_retry flag. Use this option to set up. If the server suddenly wants a new handshake, OpenSSL can process it in the background. If this option is not available, when the server wants to perform a new handshake, an error will be returned for read or write operations, and a retry flag will be set during this process.

Listing 10. Setting bio objects

bio = BIO_new_ssl_connect(ctx);BIO_get_ssl(bio, & ssl);SSL_set_mode(ssl, SSL_MODE_AUTO_RETRY);

 

After setting the SSL context structure, you can create a connection. Host Name is usedBIO_set_conn_hostnameSet the function. The specified host name and port are in the same format as the preceding one. This function can also enable the connection to the host. To confirm that the connection has been successfully opened, you mustBIO_do_connect. The call also performs a handshake to establish a secure connection.

Listing 11. Open a secure connection

/* Attempt to connect */BIO_set_conn_hostname(bio, "hostname:port");/* Verify the connection opened and perform the handshake */if(BIO_do_connect(bio) <= 0){ /* Handle failed connection */}

 

After the connection is established, you must check the certificate to determine whether it is valid. In fact, OpenSSL has completed this task for us. If the certificate has a fatal problem (for example, the hash value is invalid), a connection cannot be established. However, if the certificate problem is not fatal (when it has expired or is not legal), you can continue to use the connection.

You can use the SSL structure as a unique parameter and callSSL_get_verify_resultTo check whether the certificate has passed the OpenSSL check. If the certificate has passed the OpenSSL internal check, including the trust check, x509_v_ OK is returned. If something goes wrong, an error code is returned, which is recorded in the command line tool'sverifyOption.

It should be noted that the failed verification does not mean that the connection cannot be used. Whether connections should be used depends on the verification results and security considerations. For example, failed trust verification may only mean that there is no trusted certificate. The connection is still available, but the security awareness needs to be improved ideologically.

List 12. Check whether the certificate is valid

if(SSL_get_verify_result(ssl) != X509_V_OK){ /* Handle the failed verification */}

 

This is all the operations required. Generally, communication with the server requiresBIO_readAndBIO_write. And you only need to callBIO_free_allOrBIO_resetYou can close the connection. Which method to call depends on whether Bio is reused.

The SSL context structure must be released at some point before the application ends. YesSSL_CTX_freeTo release the structure.

Listing 13. Clearing the SSL Context

SSL_CTX_free(ctx);

 

 

Error Detection

Obviously, OpenSSL throws some type of error. What does this mean? First, you need to get the error code itself;ERR_get_errorYou can complete this task. Then, you need to convert the error code into an error string, which isSSL_load_error_stringsOrERR_load_BIO_stringsPointer to the permanent string loaded into the memory. This operation can be completed in a nested call.

Table 1 describes how to retrieve errors from the error stack. Listing 24 shows how to print the last error message in a text string.

Table 1. Retrieving errors from stacks

ERR_reason_error_string	Returns a pointer to a static string, which can be displayed on the screen, written to a file, or processed in any way you want.
ERR_lib_error_string	Specifies the database in which the error occurred.
ERR_func_error_string	Returns the OpenSSL function that causes the error.

 

Listing 14. Print the last error

printf("Error: %s/n", ERR_reason_error_string(ERR_get_error()));

 

You can also have the library provide pre-formatted error strings. YesERR_error_stringTo obtain the string. This function uses the error code and a pre-allocated buffer as parameters. The buffer must be 256 bytes long. If the parameter is null, OpenSSL writes the string to a 256-byte static buffer and returns a pointer to the buffer. Otherwise, it returns your pointer. If you select the static buffer optionERR_error_stringThe buffer is overwritten.

Listing 15. Obtaining pre-formatted error strings

printf("%s/n", ERR_error_string(ERR_get_error(), NULL));

 

You can also dump the entire error queue to a file or bio. You can useERR_print_errorsOrERR_print_errors_fpTo achieve this operation. The queue is dumped in readable format. The first function sends the queueBIO, The second function sends the queueFILE. The string format is as follows (derived from the OpenSSL document ):

[pid]:error:[error code]:[library name]:[function name]:[reason string]::[line]:[optional text message]

Where,[pid]Is the process ID,[error code]Is an 8-bit hexadecimal code,Is the source code file in the OpenSSL library,[line]Is the row number in the source file.

Listing 16. Dumping error queues

ERR_print_errors_fp(FILE *);ERR_print_errors(BIO *);