Tap into your Linux system with SYSTEMTAP

https://major.io/2010/12/07/tap-into-your-linux-system-with-systemtap/

December 7, by Major Hayden 4 Comments

One of the most interesting topics I's ve seen so far during my RHCA training at Rackspace this week Issystemtap. In short, SystemTap allows-dig out a bunch of details about your running system relatively easily. It takes scripts, converts them to C, builds a kernel module, and then runs the code within your script.

Hold it:the steps below is definitely not meant for those who is new to Linux. Utilizing SYSTEMTAP on a production system was a bad idea-it can chew up significant resources while it runs and it can a LSO cause a running system to kernel panic if you ' re not careful with the packages you install.

These instructions would work well with Fedora, CentOS and Red Hat Enterprise Linux. Luckily, the SystemTap folks put together some instructions for Debian and Ubuntu as well.

Before can start working with SystemTap on your rpm-based distribution, you'll need to get some prerequisites together :

12	yum Install Gcc Systemtap Systemtap-runtime Systemtap-testsuite Kernel-devel yum --enablerepo=*-debuginfo Install Kernel-debuginfo Kernel- Debuginfo-common

Whoa there:ensure that the kernel-devel and kernel-debuginfo* packages so you install via Yum match up with your runnin G kernel. If there ' s a newer kernel available from your Yum repo, Yum would pull the that one. If It's been a while since your updated, you'll either need to upgrade your current kernel to the latest and reboot or you ' ll need to hunt off the corresponding kernel-devel and kernel-debuginfo* packages from a repository. Installing the wrong package version can leads to kernel panics. Also, bear on Mind, the Debuginfo packages is quite large:almost 200MB in Red Hat/centos and almost 300MB in Fedora.

You can ' t write the script in just any language. SYSTEMTAP uses an odd syntax to get things going:

12	#! / usr/bin/env stap Probe begin { println("Hello World") exit () }

Just run the script with stap :

12345678

# Stap -v HelloWorld. STP Pass1: parsed user script and the Library script(s) using 94380virt< C13>/21988res/2628shr kb, in 140usr/30sys/167real Ms. Pass2:AnalyzedScript:1Probe (S) , 1 Function (S 0 Embed (S, 0 Global (S Using 94776virt/22516res/2692shr Kb, In 10usr/0sys/5real Ms Pass3: using cached /root/. Systemtap/cache/bc/stap_ bc368822da380b943d4e845ee15ed047_773. C Pass4: using cached /root/. Systemtap/cache/bc/stap_ bc368822da380b943d4e845ee15ed047_773. Ko Pass5: starting run. Hello WorldPass5: run completed in 0usr/20sys/285real Ms.

The package systemtap-testsuite gives you a tubload of extremely handy SystemTap scripts. For example:

1234567891011

# CD /usr/share/systemtap/testsuite/systemtap. Examples/io/ # Stap iotime. STP 151384706351 (httpd) access/ usr/share/cacti/index. php read: 0 Write: 0 151422436351 (httpd) access/ usr/share/cacti/include/auth. PHP Read : 0 Write: 0 151437806351 (httpd) access/ usr/share/cacti/include/global. PHP Read: 0 Write: 0 151440996351 (httpd) access/ etc/cacti/db. php read: 0 Write : 0 151876416351 (httpd) access/ usr/share/cacti/lib/adodb/adodb. Inc . PHP Read: 106486 write: 0 151876646351 (httpd) iotime/ usr/share/cacti/lib/adodb/adodb. Inc . PHP time: 218 151949656351 (httpd) access/ usr/share/cacti/lib/adodb/adodb-time . IncPHP read: 0 Write: 0 15195692 6351 (Httpd Access /usr/share/cacti/lib/adodb/adodb< Span class= "Crayon-o" >-iterator.inc.php Read: 0 Write: 0    .. . output continues : .

The iotime.stp script dumps out of the reads and writes occurring on the system in real time. After starting the script above, I accessed my cacti instance on the server and immediately started seeing some reads as a Pache began picking up PHP files to parse.

Consider a situation in which the need to decrease interrupts on a Linux machine. This was vital for laptops and systems, need to remain in low power states. Some might suggest powertop for, and why isn't give SystemTap a try?

1234567891011

# CD /usr/share/systemtap/testsuite/systemtap. Examples/interrupt/ # Stap interrupts-by-dev. STP OHCI_hcd: USB3 : 1 OHCI_hcd: usb4 : 1 hda_intel : 1 eth0 : 2 eth0 : 2 eth0 : 2 eth0 : 2 eth0 : 2 eth0 : 2

On this particular system, it's pretty obvious that the Ethernet interface are causing a lot of interrupts.

If you want more examples, keep hunting around in the Systemtap-testsuite package (remember rpm -ql systemtap-testsuite ) or review the giant LIS T of examples on SystemTap ' s site.

Thanks again to Phil Hopkins at Rackspace for giving us a detailed explanation of system profiling during training.

Tap into your Linux system with SYSTEMTAP