Mem, bio, one of redis code structures

Redis code structure 1 MEM, bio

1. redis code structure

Event Library	Type Library	Network Library	Persistence	Copy	Subscription	Transactions	Main	Client	Others
AE. c AE _epoll.c AE _kqueue.c AE _select.c Syncio. c	Adlist. c Intset. c Object. c SDS. c T_hash.c T_list.c T_set.c T_string.c T_zset.c Ziplist. c Zipmap. c	Anet. c Networking. c	Aof. c RDB. c Redis-check-aof.c Redis-check-dump.c	Replication. c	Pubsub. c	Multi. c	Config. c Redis. c Dict. c DB. c	Redis-cli.c	Debug. c Lzf_c.c Lzf_d.c Pqsort. c Redis-benchmark.c Release. c Sha1.c Slowlog. c Sort. c Util. c VM. c Zmalloc. c Bio. c Endian. c

2. zmalloc. c

This file is used to encapsulate memory splitters: the memory splitters provided by redis include malloc, jemalloc, and acmalloc of the c library. For the application scenarios of these splitters, see (). The main purpose of providing this layer encapsulation is to count the memory usage. Because redis is designed for full memory, it is necessary to observe the memory usage at all times.
Void * zmalloc (size_t size) is used to allocate memory. It first determines that the allocator is used with header information, that is, the size is saved to the top of the content. The default three splitters all have header information, so prefix_size = 0, then sizeof size (long) byte alignment, (the MS malloc allocated by default should be size alignment, the other two methods are unclear). If there is no alignment, the allocated size is extended and used_memory is accumulated.
Update_zmalloc_stat_alloc (_ n ,__ size)
If (_ N & (sizeof (long)-1) _ n + = sizeof (long)-(_ N & (sizeof (long)-1 ));
Used_memory + = _ N;
Void * zcalloc (size_t size) is similar to zmalloc, unlike C's calloc, because it only has one input parameter.
Void * zrealloc (void * PTR, size_t size) is the same as realloc in C, but only the size of used_memory is updated. Zfree is similar.
Char * zstrdup (const char * s) is similar to strdup for string copying. This function allocates memory and requires the function to be called to release the corresponding memory at the end, otherwise, the memory will leak.
Size_t zmalloc_used_memory (void) returns the amount of memory in use.
Void zmalloc_enable_thread_safeness (void) is used to design whether the Allocator supports multithreading. redis does not support it by default.
Size_t zmalloc_get_rss (void), which is used to obtain the RSS of the redis-server, that is, the number of pages currently residing in the physical memory. This function reads/proc/Pid/STAT to obtain this information. The preceding used_memory indicates that the amount of virtual memory currently used is different from that of RSS.

3. Bio. c

This file is used to provide functions related to background threads. Redis introduced two background threads from 2. # For the aof file close and fsync operations of the main thread. These two operations will cause the main thread to block. For detailed application scenarios, see the previous article "redis aof. Here we will directly introduce the functions in this file.
This file contains four static global variables and a bio_job structure.

Static pthread_mutex_t bio_mutex [redis_bio_num_ops]; // provides mutex protection for accessing the static global variable bio_jobs and bio_pending. Static pthread_cond_t bio_condvar [redis_bio_num_ops]; static list * bio_jobs [redis_bio_num_ops]; // save the two adliststatic unsigned long bio_pending [redis_bio_num_ops] of the job; // The number of jobs waiting for processing struct bio_job {// The struct of the job. arg1 is used to save FD time_t time;/* time at which the job was created. */void * arg1, * arg2, * arg3 ;};

Void bioinit (void), which is called by initserver to complete the initialization of the above global variables, and to create the Close job processing thread and the fsync processing thread (bioprocessbackgroundjobs ).
Void biocreatebackgroundjob (INT type, void * arg1, void * arg2, void * arg3). This function is used to create bio_job and add the job to the bio_jobs [type] list. This function is called in three places: 1. Create a redis_bio_close_file job. When the background rewrite aof sub-process ends and the main process wait receives the sigchld signal, it calls backgroundrewritedonehandler for corresponding signal processing: Close (oldaof) is required, then rename (tempfilename, oldaofname), if you follow this process directly, it will lead to blocking, so the method used here is to enable it if the oldaof is already disabled (not required if it is not disabled ), then rename. At this time, the oldaof has an open reference, so it will not be blocked without Unlink. Finally, it will be closed by the background redis_bio_close_file thread we just mentioned. This thread will be blocked, but this will not affect the main thread; second, create redis_bio_aof_fsync
Job, also in the backgroundrewritedonehandler, When you reset the FD of tempfile to server. if (server. appendfsync = appendfsync_everysec) Then aof_background_fsync (newfd); this function creates a redis_bio_aof_fsync job. Third, the flushappendonlyfile function, which we introduced in redis aof, calls this function in the beforesleep function before each event loop. Obviously, this function is used to refresh the aof file (note: is there a write operation that will cause the main thread to block? Why not add it to file?
Event ?), If redis_bio_aof_fsync job already exists, wait for biopendingjobsoftype (redis_bio_aof_fsync )! = 0, you do not need to add a fsync job to aof_background_fsync (because there is only such an aof FD). Otherwise, if it is server. appendfsync = appendfsync_everysec, aof_background_fsync will add
The job is operated by the fsync thread in the background.
The above is where the current version is used in the background close and fsync threads.
Void * bioprocessbackgroundjobs (void * Arg), which is the processing function of two threads. They are identified as different threads through the ARG parameter. This function is to obtain the corresponding bio_jobs list job through the type, and then call close (job-> arg1) or aof_fsync (job-> arg1) to complete the corresponding operation.
Unsigned long biopendingjobsoftype (INT type), returns the number of jobs waiting for the type thread to process. For fsync jobs, it is clear that the number of pending is a maximum of 1, as we have mentioned above, because it is only valid for the current aof FD, and the file is unique at the task time; there may be multiple pending numbers for close jobs, because the old aof file is closed, obviously this file is different each time.
The following figure shows the relationship between several processes and threads.

Figure 1 processes and threads in redis

NOTE: For the running status of bgsave and bgrewrite sub-processes, see the previous article.