Linux finds TLS information from core information

Background

When we look at core issues, we sometimes need to look at the value of a certain TLS variable. But GDB doesn't provide direct commands, or I don't know. The purpose of this text. is to find out from the core file that a thread holds the contents of the TLS variable.

Basis

Linux glibc Library when creating threads. Use mmap to create a piece of memory space as the stack space for this thread. And struct pthread put a data structure called the top of the stack (refer to the GLIBC code [email protected] ). and the data structure of TLS is in struct pthread :

struct pthread{    // ...    struct pthread_key_data    {        uintptr_t seq;        void *data;    } specific_1stblock[PTHREAD_KEY_2NDLEVEL_SIZE];    struct pthread_key_data *specific[PTHREAD_KEY_1STLEVEL_SIZE];    // ...};

The specific_1stblock array is the first layer of the TLS variable, which is PTHREAD_KEY_2NDLEVEL_SIZE a macro definition, and the size in glib2.20 is 32. Assuming that the TLS variable exceeds this value, it is used specific to store it. Can be seen from here. Just want us to find specific_1stblock the place. You can find the location of the TLS variable.

Based on the above analysis. We need to find struct pthread the place first. First look struct pthread at the position in the stack:

 /* Place the thread  Descriptor at the end of the stack. */  #if tls_tcb_at_tp  pd = (struct  Pthread *) ((char  *) mem + size-coloring)-1 ;  #elif tls_dtv_at_tp  pd = (struct  pthread *) ((((UI                   ntptr_t) mem + size-coloring-__static_tls_size) & ~__STATIC_TLS_ALIGN_M1) -Tls_pre_tcb_size);  #endif   

pdThe definition is struct pthread *pd; . In the code mem is the mmap first address of the memory created using. coloring COLORING_INCREMENT Determines whether a change is a value based on the macro definition. The code version number I looked at and the operating system used (Redhat 6.5) were installed in glibc, which is 0, meaning coloring a constant of 0. There are also two macros defined conditions, TLS_TCB_AT_TP and TLS_DTV_AT_TP , in glibc2.20. X86_64 is used on the TLS_TCB_AT_TP . So pd the relative mem offset is the fixed size sizeof(struct pthread) .

From the description above, assuming we can know the memory segment of a thread, we find the end of the memory segment and then offset sizeof(struct pthread) it forward to find struct pthread * the address and find specific_1stblock the specific location.

Another question, however, is how to determine sizeof(struct pthread) the value?

Although the size of a struct has been fixed after it has been compiled, there are so many macro definition restrictions to see the complex definitions in glibc. I just can just hehe. Just, my other trick is to determine the size directly from some of the programs that are currently running sizeof(struct pthread) .

The very many functions provided by GLIBC get TLS information, for example pthread_self .

This function is very short:

pthread_t__pthread_self (void){  return (pthread_t) THREAD_SELF;}

THREAD_SELFthe definition in the code is

# define THREAD_SELF \  struct pthread *__self;                                   asm ("mov %%fs:%c1,%0""=r" (__self)                         "i" (offsetof (struct pthread, header.self)));                 __self;})

This code simply gets the fs segment register plus a fixed offset value. In fact, I thought about using fs the value of the register directly, but the value is not visible to gdb in both the running program and the core file.

Well, it's been so much wasted.

Fortunately, GDB is able to run functions directly while debugging a running program. I pthread_self() take the return value of the function and then compare it with the memory in the segment of the thread, and I can know struct pthread * the offset from the bottom of the stack.

Dickens got it and looked sizeof(struct pthread) back. Half of the task was finished. You also need to know specific_1stblock struct pthread * the relative offset. Just OK, this is more easy to do, look at pthread_getspecific the assembly code at a glance:

Dump of assembler Code forfunction Pthread_getspecific:0x0000003bcd40c470<+0: CMP $X1F,%edi   0x0000003bcd40c473<+3:Push   %RBX   0x0000003bcd40c474<+4;: ja0x3bcd40c4ba<pthread_getspecific+ About>0x0000003bcd40c476<+6;: mov%edi,%eax   0x0000003bcd40c478<+8: SHL $X4,%rax   0x0000003bcd40c47c<+ A;: mov%fs:0x10,%RDX   0x0000003bcd40c485<+ +: Lea0x310(%RDX,%rax,1),%RDX   0x0000003bcd40c48d<+ in;: mov0x8(%RDX),%rax   0x0000003bcd40c491<+ -;: Test%rax,%rax   0x0000003bcd40c494<+ $: JE0X3BCD40C4AC<pthread_getspecific+ -> .....

Compare the code in GLIBC:

  struct pthread_key_data *data;  /* Special case access to the first 2nd-level block.  This is the     usual case.  */  if (__glibc_likely (key < PTHREAD_KEY_2NDLEVEL_SIZE))    data = &THREAD_SELF->specific_1stblock[key];  else

THREAD_SELFis the current thread struct pthread * .

C code with the assembly code to see, very easy to find specific_1stblock the offset. The register in the assembly edi is the number of incoming parameters pthread_key_t key .

mov %fs:0x10,%rdxThis line of code uses fs registers. As with the function seen above pthread_self , this determines the address to be obtained struct pthread * .
Then the next line lea 0x310(%rdx,%rax,1),%rdx is naturally the specific_1stblock value obtained. Registers are stored in this line rdx struct pthread* . raxStorage key * sizeof(struct pthread_key_data) , the last rdx + (rax * 1) + 0x310 value is put into the rdx , very obvious, 0x310 is specific_1stblock the offset (0x310).

Until now. You are ready to get all the conditions for the TLS variable, sizeof(struct pthread) and specific_1stblock the offset. The following starts with a hands-on test certificate.

Test

Write a test code that uses TLS
This code creates a thread variable and a thread, and the created thread sets the value of the thread variable.

#include <pthread.h>#include <unistd.h>pthread_key_t key;void*thread_func (void*arg) {pthread_setspecific (Key, (Const void*)0x12345678);//Set a special value for easy detection of test resultsSleep -);//sleep for a period of time to generate a core file    returnNULL;}intMainintargcChar**ARGV) {pthread_key_create (&key, NULL);    pthread_t Tid;    Pthread_create (&tid, NULL, THREAD_FUNC, NULL); Pthread_join (Tid, NULL);return 0;}

Compile

g++ -lpthread test.cpp

The default build a.out . Run directly, will pause for a period of sleep time in, with GDB attach up.
Runinfo thread

 (GDB) info thread  2  thread  0x7f6cc2d15710   (LWP 15000 ) 0x0000003bcd0a6a8d  in  nanosleep () from/lib64/ Libc So.6  *  1  thread  0x7f6cc2d17720  (LWP 14999 ) 0x0000003bcd40803d  in  pthread_join () from/lib64/libpthread So.0  (gdb)  

Let's look at thread 2, which is the thread that was created.
Run thread 2 switch to thread 2.
Run call pthread_self() . And the results are

(gdb) call pthread_self()$8 = -1026468080

Change to hexadecimal print

$8$90xc2d15710

Obviously still incorrect, rather silent, GDB's call instruction only printed 4 bytes. Just a little attention. The info thread result of the output is found, and there is one data that is the same as here:

20x7f6cc2d1571015000)  0x0000003bcd0a6a8dinfrom /lib64/libc.so.6

The number that follows the thread is pthread the address. Only this data is not printed when debugging the core file:

thread  2Thread14999  0x0000003bcd40803din pthread_join () from /lib64/libpthread.so.0*1Thread15000  0x0000003bcd0a6a8din nanosleep () from /lib64/libc.so.6

Although the results of the operation did not match the expected, but fortunately got the pthread address. Next, find the memory segment where the thread resides. is the stack interval. The data segment information for the process can be seen from the/proc/ pid /maps file. pidThis is the process number.

This is the memory information in the process that I tested:

7f6cc2315000-7f6cc2316000---P00000000 xx:xx 0 7f6cc2316000-7f6cc2d1d000 rw-p00000000 xx:xx 0 7fff4c321000-7fff4c337000 rw-p00000000 xx:xx 0[Stack]7fff4c35a000-7fff4c35b000 R-XP00000000 xx:xx 0[VDSO]

Very obvious. 0x7f6cc2d15710 belongs to this paragraph:

7f6cc2316000-7f0000000000:000

This is the stack space for thread 2. Because the stack is growing from the top down, the bottom of the stack is 7f6cc2d1d000. Its distance from the 0x7f6cc2d15710 is 0x78f0.

gcoregenerate a core file in gdb with a command. Use GDB to open the core file to verify the test and find out the TLS value.

gdb a.out core

Print out the program memory segment of the core file record

(GDB) Info filessymbols from"/data01/usergrp/wangyl11/a.out". Local Core Dumpfile: '/data01/usergrp/wangyl11/core. 14999‘,file typeelf64-x86-.        0x0000000000400000-0x0000000000400000  isLoad10x0000000000600000-0x0000000000601000  isLoad20x00000000006d1000-0x00000000006f2000  isLoad3 ......... .............0x0000003bcde83000-0x0000003bcde84000  isLoad240x00007f6cc2316000-0x00007f6cc2d1d000  isLoad250x00007fff4c321000-0x00007fff4c337000  isLoad260x00007fff4c35a000-0x00007fff4c35b000  isLoad270xffffffffff600000-0xffffffffff601000  isLoad28 .....

A whole bunch of memory segments. Which is the thread you're looking for?

The space in which a thread is located is a stack space, which simply finds variables or other information on the stack of a thread, and then, based on that information, can find the corresponding memory segment. There is a very easy view of the stack information is the stack register rsp .

Look at the stack register for the thread:

1to115000)]#0  0infrom /lib64/libc.so.6(gdb) info reg rsprsp            0x7f6cc2d14c90   0x7f6cc2d14c90

This will find the paragraph:

0x00007f6cc23160000x00007f6cc2d1d000is load25

This paragraph is also just see the line stacks space.

Take the bottom of the address is 0x00007f6cc2d1d000, minus pthread offset 0x78f0 is 0x?7f6cc2d15710?, plus specific_1stblock the offset 0x310, get 0x7f6cc2d15a20?

The last one, verify get address correctness:

(gdb) x/2xg 0x7F6CC2D15A200x7f6cc2d15a20: 0x0000000000000001      0x0000000012345678

Done. The result above, the first number is seq , the second one is data (the two are struct pthread_key_data members).

Although the verified core file is generated just by running the program, it is only as good to run it again to generate a new core file.

It's just that there's a restricted place. The most important reason is that the thread data is located at the bottom of the struct pthread stack, and the stack is a separate memory segment in the process space. Assume that this stack space is provided by the user when creating the thread. This approach may not be applicable. Hopefully a more general approach can be found later, and perhaps GDB will directly provide command access to thread variables.

Summarize

1. Find the struct pthread address first.

   The ability to trace the running program through GDB, find the process stack memory space, find the distance from the bottom of the stack;

2. By disassembly pthread_getspecific . Find specific_1stblock struct pthread * the offset relative to;
3. In the core file, through the address of the stack register RSP, find the memory segment of the thread, based on the information in the previous two steps, the computed specific_1stblock address, and then print out the value of the TLS variable.

Note: This method is limited by the memory stack space that GLIBC itself creates and the Linux x86_64 environment.

Linux finds TLS information from core information