Discussing the atomic problem of the output content of the error_log function in php _ PHP Tutorial

This article discusses the atomicity of the output content of the error_log function in php. Discussing the atomicity of the output content of the error_log function in php a few days ago, I discussed with my colleagues how to ensure that the log output by error_log has a unique identification header in a loginphp call, the conclusion is to explore the atomicity of the output content of the error_log function in php.

A few days ago, I discussed with my colleagues how to ensure that all the logs output by error_log have a unique identification header in a login php call. The conclusion is that the player account + current time + random number, the current user level should meet the requirements. of course, this is not the focus of this article.

After determining this simple solution, I am thinking about two questions, which are also the focus of this article:

1. Can the error_log call ensure that the output content is complete?

2. if it is complete, how can it be guaranteed?

Authentication started. at first, I thought that error_log would add a file lock to the target file in the call, and directly look at the source code (php5.6.12 ):

From: basic_functions.c

PHPAPI int _php_error_log_ex(int opt_err, char *message, int message_len, char *opt, char *headers TSRMLS_DC) /* {{{ */{php_stream *stream = NULL;switch (opt_err){case 1:/*send an email */if (!php_mail(opt, PHP error_log message, message, headers, NULL TSRMLS_CC)) {return FAILURE;}break;case 2:/*send to an address */php_error_docref(NULL TSRMLS_CC, E_WARNING, TCP/IP option not available!);return FAILURE;break;case 3:/*save to a file */stream = php_stream_open_wrapper(opt, a, IGNORE_URL_WIN | REPORT_ERRORS, NULL);if (!stream) {return FAILURE;}php_stream_write(stream, message, message_len);php_stream_close(stream);break;case 4: /* send to SAPI */if (sapi_module.log_message) {sapi_module.log_message(message TSRMLS_CC);} else {return FAILURE;}break;default:php_log_err(message TSRMLS_CC);break;}return SUCCESS;}/* }}} */

From: streams. c

/* Writes a buffer directly to a stream, using multiple of the chunk size */static size_t _php_stream_write_buffer(php_stream *stream, const char *buf, size_t count TSRMLS_DC){size_t didwrite = 0, towrite, justwrote; /* if we have a seekable stream we need to ensure that data is written at the  * current stream->position. This means invalidating the read buffer and then * performing a low-level seek */if (stream->ops->seek && (stream->flags & PHP_STREAM_FLAG_NO_SEEK) == 0 && stream->readpos != stream->writepos) {stream->readpos = stream->writepos = 0;stream->ops->seek(stream, stream->position, SEEK_SET, &stream->position TSRMLS_CC);}while (count > 0) {towrite = count;if (towrite > stream->chunk_size)towrite = stream->chunk_size;justwrote = stream->ops->write(stream, buf, towrite TSRMLS_CC);/* convert justwrote to an integer, since normally it is unsigned */if ((int)justwrote > 0) {buf += justwrote;count -= justwrote;didwrite += justwrote;/* Only screw with the buffer if we can seek, otherwise we lose data * buffered from fifos and sockets */if (stream->ops->seek && (stream->flags & PHP_STREAM_FLAG_NO_SEEK) == 0) {stream->position += justwrote;}} else {break;}}return didwrite;}

By looking at the source code file, error_log only opens the file in O_APPEND mode, and then writes the buf. well, there is no shadow of the file lock. The problem is also obvious. the write may be called multiple times, so it is basically determined that msg is not necessarily output by the atom. Multiple writes make this problem more serious.

Multiple write operations cannot guarantee atomic operations. what about a single write operation?

1. from: man write

 If the file was open(2)ed with O_APPEND, the file       offset is first set to the end of the file before writing.  The       adjustment of the file offset and the write operation are performed       as an atomic step.

2. from: man write

Atomic/non-atomic: A write is atomic if the whole amount written in one operation is not interleaved with data from any other process. This is useful when there are multiple writers sending data to a single reader. Applications need to know how large a write request can be expected to be performed atomically. This maximum is called {PIPE_BUF}. This volume of IEEE Std 1003.1-2001 does not say whether write requests for more than {PIPE_BUF} bytes are atomic, but requires that writes of {PIPE_BUF} or fewer bytes shall be atomic.

Man probably means that if it is in the O_APPEND mode, the position at the end of the file and the write call are atomic operations. there is no need to adjust the end of the file during the write operation, resulting in dislocation. At present, this atomicity only guarantees the first write after open and later, and the subsequent write is not guaranteed. Like error_log, if the buf is too long, resulting in multiple writes, it is certainly not guaranteed that the complete output of the buf is one time.

Continue to study, so is a single write atomic? Man also gave the answer, no. atomic operations are guaranteed only when the data to be written is smaller than or equal to PIPE_BUF.

The problem was theoretically answered. experiment verification is started below:

Test_error_log.php

Check_line.py

filename = ./append.txtfor line in open(filename):    print len(line)

Test. sh

#!/bin/bash rm -f append.txtfor ((counter=0; counter < 10; ++counter))do    php test_error_log.php $counter $1 $2 &donesleep 2#python check_line.py > a.txtpython check_line.py | sort | uniq -c

Verification logic: add a line break at the end of the output msg. if the output line is the same size as the initial buf, it indicates the complete output of error_log. after verification, the PIPE_BUF in kernel 3.5.0-23 is 8 K, the error_log output within 8 kB can be complete, otherwise there is a risk of confusion.

Workshop talked with colleagues a few days ago about how to ensure that all the logs output by error_log have a unique identification header in a login php call. The conclusion is play...