Linux system fault checking artifact Sysdig installation Instructions

Sysdig is a source, the implementation of the Linux system monitoring and troubleshooting tools, this article we discuss Sysdig installation and how to use.

Sysdig listens at the operating system level and captures system activities such as system calls and system events, which makes it look like a system-oriented tcpdump or Wireshark. If you plan to troubleshoot the system, then Sysdig will be a handy tool to solve your problems.





on Linux, you can use the following command to install Sysdig:





Curl-s Https://s3.amazonaws.com/download.draios.com/stable/install-sysdig | sudo bash





This will install Sysdig into the RPM or Deb Linux system.





Capture system Activity





real-time capture, resulting in print to standard output:





Sysdig





saves the capture results to a file System.scap for later analysis:





sysdig-w System.scap





captures the specified number of events by 200 and saves to a file:





sysdig-n 200-w System.scap





read the captured file:





sysdig-r System.scap





Capture Result Interpretation





(1) (2) (3) (4) (5) (6) (7) (8)

1 10:54:50.462463956 0 sysdig (29043) > Sysdigevent event_type=1 event_data=0


2 10:54:50.462603110 0 sysdig (29043) > Sysdigevent event_type=1 event_data=0


3 10:54:50.462729565 0 Sysdig (29043) > Sysdigevent event_type=1 event_data=0


4 10:54:50.462859521 0 Sysdig (29043) > Sysdigevent event_type=1 event_data=0



5 10:54:50.463206317 0 sysdig (29043) > Switch next=0 pgft_maj=0 pgft_min=1790 vm_size=35748 vm_rss=7164 0


6 10:54:50.464246835 0 <NA> (0) > Switch next=7 pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_swap=0


7 10:54:50.464249707 2 <NA> (0) > Switch next=8374 pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_swap=0


8 10:54:50.464255940 0 <NA> (7) > Switch next=0 pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_swap=0


9 10:54:50.464264256 2 <NA> (8374) > Switch next=0 pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_swap=0


10:54:50.464358113 2 <NA> (0) > Switch next=854 (mlnet) pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_swap= 0


10:54:50.464370099 2 mlnet (854) < poll res=0 fds=


10:54:50.464378193 2 mlnet (854) > Poll fds= timeout=5


10:54:50.464385400 2 mlnet (854) > Switch next=0 pgft_maj=216 pgft_min=3386 vm_size=162608 vm_rss=12196 vm_swa p=2716


-10:54:50.464950541 0 <NA> (0) > Switch next=1105 (memcached) pgft_maj=0 pgft_min=0 vm_size=0 vm_rss=0 vm_ Swap=0


10:54:50.464954692 0 memcached (1105) < epoll_wait res=0


10:54:50.464976007 0 memcached (1105) > Epoll_wait maxevents=32


10:54:50.464984030 0 memcached (1105) > Switch next=0 pgft_maj=3 pgft_min=247 vm_size=327412 vm_rss=1860 vm_sw ap=468


10:54:50.465256687 2 <NA> (0) > Switch next=2181 (plugin-containe) pgft_maj=0 pgft_min=0 vm_size=0 Vm_rss =0 vm_swap=0


10:54:50.465261465 2 plugin-containe (2181) < poll res=0 fds=


10:54:50.465297692 2 plugin-containe (2181) > Getrlimit resource=3 (rlimit_stack)





the results captured by Sysdig as shown above, each column is meant to be:





Event Number


time Stamp


CPU Number


Process Name


Thread ID


Event Direction,> for entry event,< for Exit event


Event types, such as open, read, etc.


event argument list





Filter Capture Results





by default, Sysdig captures so much information that we need to find the information we are interested in, which requires a filter like grep.





Filter By field Category:





sysdig-r system.scap Proc.name=sysdig





This command filters out system events with process name Sysdig, and the result is:




1 10:54:50.462463956 0 sysdig (29043) > Sysdigevent event_type=1 event_data=0


2 10:54:50.462603110 0 sysdig (29043) > Sysdigevent event_type=1 event_data=0


3 10:54:50.462729565 0 Sysdig (29043) > Sysdigevent event_type=1 event_data=0


4 10:54:50.462859521 0 Sysdig (29043) > Sysdigevent event_type=1 event_data=0



5 10:54:50.463206317 0 sysdig (29043) > Switch next=0 pgft_maj=0 pgft_min=1790 vm_size=35748 vm_rss=7164 0





Sysdig provides field categories including FD, process, evt, user, group, and Syslog, which can be queried by sysdig-l.





except =, the Sysdig filter expression also supports comparison operators such as!=, <, <=, >, >=, and contains.





Also, you can use Boolean operators such as and, or, not. For example:





sysdig-r system.scap proc.name=sysdig and Evt.type=switch





chisels





in Sysdig, Chisels is a script written in Lua that can be used to extend the filtering capabilities of Sysdig.





For example we want to see the most frequently read and write disk files process, you can use Topprocs_file this chisels:





sysdig-c Topprocs_file





results are:





Bytes Process


------------------------------


448.36KB mozstorage


220.38KB Perl


1.69KB Tmux


1.62KB SH


1.59KB Xorg


1.30KB URXVTD





More chisels, can be understood through SYSDIG-CL. Of course, if you are familiar with Lua, you can also write your own chisels.