Algorithm of RSA encryption signature algorithm using OpenSSL

The process of encrypting (signing) is (M's e-square) mod N, where we assume the message m as a number, but in fact the message is generally a string, so there must be a rule to convert the string to a number, and to make the number equal to n (and not larger than N). The purpose of this is to make (M's e-th square) > N, if not so c= (M's e-square) mod n = (M's e-order), that is, mod n has no effect at all, the attacker can easily recover m by taking the e-root of C. So here's a discussion of the criteria for this conversion:
1, calculates the size required to format the encrypted grouping. If N is an L-bit number, then the encryption group will have L/8 bytes long (only).
2, the first high byte is always 0, and the second is a grouping type. For encryption, this is 2, which is 1 for the signature, which ensures that the number formed is slightly smaller than N.
3, the message is stored in the low byte of the encrypted packet, and placed before a 0
XX FF FF FF FF ... 00 Message data-This is how the signature is converted
00 02 Pseudo-Random non-0 byte 00 message data-This is how encryption is converted
As can be seen here, the signature result is the same when a message is repeatedly signed. When you encrypt a message repeatedly, the encryption grouping is different. When you restore an encrypted grouping, you only need to move from the third byte to the right until you get to a 0 byte, and this finds the starting position of the data.

However, it is important to note that the abstract algorithm identifier is added to the above procedure because:
In the Rsa_sign function in Rsa_eay_private_encrypt (i,s,sigret,rsa,rsa_pkcs1_padding); the function has a program before, the answer to the question is here, the summary upstairs is in M, and s= 3020300c06082a864886f70d020505000410+m (34-bit), this procedure is very well understood, is to add the digest algorithm identifier. So 3020300c06082a864886f70d020505000410 is the representative of MD5 this algorithm.
So we can get the conversion format of the signed data.
XX FF FF FF FF ... 00 algorithm identifies message data

So let's look at the process of certificate signing in OpenSSL:
int X509_sign (X509 *x, Evp_pkey *pkey, const EVP_MD *MD)
{
First, the Ret->cert_info->signature, and the ret->sig_alg settings;
Inl=i2d_x509_cinf (ret->cert_info,null);//Find out the length after the certificate is encoded
buf_in= (unsigned char *) openssl_malloc ((unsigned int) inl);//Application space
Outll=outl=evp_pkey_size (Pkey1);
buf_outl= (unsigned char *) openssl_malloc ((unsigned int) inl);
if ((buf_in = = null) ││ (buf_outl== null))
{
outl=0;
Goto err;
}
P=buf_in;//p and Buf-in share an address
I2d_x509_cinf (ret->cert_info,&p);//Deposit the certificate code buf-in
Evp_md_ctx_init (&CTXL);//initialization
Evp_signinit (&CTXL,DGST);//the digest algorithm that needs to be used is stored in CTXL
Evp_signupdate (&CTXL, (unsigned char *) BUF_IN,INL);//The encoded value of the deposit certificate
Evp_digestfinal (&ctxl,& (m[0]), &m_len);//The length of the encoding is M_len digest value is stored in M
Rsa_sign (CTXL->DIGEST->TYPE,M,M_LEN,BUF_OUT,OUTL,PKEY->PKEY.RSA)//To fetch the signature value of the digest value, and finally deposit the signature value of Outl to Buf-out.
Rsa_sign is mostly called rsa_eay_private_encrypt (int flen, unsigned char *from, unsigned char *to, RSA *rsa, int padding) functions. One thing to note here is that the p and Q factors in the RSA key file are called by default when Rsa_eay_private_encrypt encryption, using the Chinese remainder theorem algorithm to encrypt, which can improve efficiency rather than calling pkey->pkey.rsa-> directly D is encrypted, so sometimes the pkey->pkey.rsa->d is changed directly and the encrypted result will not change.
In this process, the encoding function uses the EME-PKCS1--5-ENCOD encoding function, the conversion function between the large number and the string is the OS2IP, i20sp format conversion function bn_bin2bn and Bn_bn2bin. Eme-pkcs1_5-encode function
Input: The string M, Emlen represents the length of the information encoding. (Note: The length of information m must not be greater than emlen-10 bytes)
Output: EM means the contents of M after encoding. The specific procedure for a function is to randomly produce emlen-len (M)-2 byte lengths of non-0-string PS, and Len (PS) >=8 then connect them em= 02││ps││00││m in the following manner.