Simulation of mouse and keyboard actions in python

I was planning to develop a fun project last month, but I have never had time. This is a part of this project, and it is easy to handle this part, such as poor communication. First, simulate the mouse and keyboard press the release action. I use the X11 library, so I need to understand X11 programming. Second, I can use c or c ++ to implement it, however, because I am a py fan, I always want to move the code into python, so I need to implement the python module. The ctypes used in this article will be attached to the c extension module of python in the future. 1. X11 programming should first briefly introduce X11. I will not repeat it if I have an introduction on the Internet. We know that X provides services in the form of server and client. to use its functions, we need to communicate with the server. Use Display * XOpenDisplay (char * display_name) to obtain a Display-type handle pointer. Display_name can be the DISPLAY environment variable, and the output Using echo $ DISPLAY is: 0 (this is output by my linux mint ). If display_name is NULL, the environment variable is saved by default. Several common header files used for X11 programming: # include <X11/Xlib. h> # include <X11/Xutil. h> # include <X11/Xos. h> I am using # include <X11/Xlib. h> and # include <X11/extensions/XTest. h>. XTest. h: The interfaces required for simulating the mouse and keyboard are XTestFakeButtonEvent, XTestFakeMotionEvent, and XTestFakeKeyEvent. For more information, you only need to add a function name to man on the terminal. For example, XTestFakeMotionEvent interface: int XTestFakeMotionEvent (display, screen_number, x, y, delay); Display * display; // This value is obtained from XOpenDisplay int screen_number; // set it to-1 to indicate the int x, y of the current screen; // The unsigned long delay of the screen position; // The delay is millisecond, set it to CurrentTime to indicate no delay. Finally, we need to disable the Display handle: XCloseDisplay (Display * display ). Interface implementation: 1 # include <stdio. h> 2 # include <X11/extensions/XTest. h> 3 # include <X11/Xlib. h> 4 Display * dspopen () {5 6 Display * dsp = XOpenDisplay (NULL); 7 if (! Dsp) {8 printf ("open display failed \ n"); 9 return NULL; 10} 11 return dsp; 12} 13 int presskey (Display * dsp, int s) {// press 14 if (dsp = NULL) 15 return-1; 16 // KeySym keysym = XStringToKeysym (s); 17 KeyCode key = XKeysymToKeycode (dsp, s ); 18 if (key = NoSymbol) 19 return-1; 20 XTestFakeKeyEvent (dsp, key, 1, CurrentTime); 21 XFlush (dsp); 22 return 0; 23} 24 int move (Display * dsp, int x, int y) // move the mouse 25 {26 if (0 = XTestFake MotionEvent (dsp,-1, x, y, CurrentTime) 27 {28 printf ("Cannot move! \ N "); 29 return-1; 30} 31 return 0; 32} 33 int buttonpress (Display * dsp, int type) // press the mouse, type = 1 indicates the left button, 3 is right-click, 2 is middle Key 34 {35 if (0 = XTestFakeButtonEvent (dsp, type, 1, CurrentTime) 36 {37 printf ("press failed \ n "); 38 return-1; 39} 40 return 0; 41} 42 int buttonrelease (Display * dsp, int type) // mouse release 43 {44 if (0 = XTestFakeButtonEvent (dsp, type, 0, CurrentTime) 45 {46 printf ("release failed \ n"); 47 return-1; 48} 49 return 0; 5 0} 51 int releasekey (Display * dsp, int s) {// keyboard release52 if (dsp = NULL) 53 return-1; 54 // KeySym keysym = XStringToKeysym (s); 55 KeyCode key = XKeysymToKeycode (dsp, s); 56 if (key = NoSymbol) 57 return-1; 58 XTestFakeKeyEvent (dsp, key, 0, CurrentTime); 59 XFlush (dsp); 60 return 0; 61} 62 void dspclose (Display * dsp) {63 if (dsp! = NULL) {64 XCloseDisplay (dsp); 65 66} 67} 68 // int main () {// after the program ends, output c69 // Display * dsp = dspopen (); 70 // presskey (dsp, 'C'); 71 // releasekey (dsp, 'C') before the cursor '); 72 // dspclose (dsp); 73 // return 0; 74 //} The main function commented out above can be used for testing, we save the above Code as display. c is compiled into a shared library, which requires the X11 and Xtst libraries. Gcc-fPIC-shared-o libdisplay. so display. c-lX11-lXtst will be generated after compilation. libdisplay. so. Now we use this dynamic shared library in the ctypes module. 2. ctypes is a simple introduction and use. We know that the types in python are different from those in c. It should be said that there are not the same. For int, python regards it as the PyObject type for processing. Then we need to use the interface provided by ctype for the type of this link. There is a figure showing the interface corresponding to the type conversion. The following describes the specific operations. We load data into the database through the CDLL () interface: lc = CDLL (". /libdisplay. so ") Then you can use the interface provided in the library, but the returned value of the dspopen () interface above is a Display pointer, so we need to use c_void_p () to convert it: d = c_void_p (lc. after dspopen (), you can use d for processing. The Code is as follows: from ctypes import * import timeclass MOUSE: LEFT = 1 MiDDLE = 2 RIGHT = 3lc = CDLL (". /libdisplay. so ") d = c_void_p (lc. dspopen () time. sleep (5); lc. buttonpress (d, c_int (MOUSE. RIGHT) lc. buttonrelease (d, c_int (MOUSE. RIGHT) lc. the code above dspclose (d) will open the right-click menu at the pointer five seconds later. So many libraries written in c using ctypes. In the future, I will write part of the c code as the c extension of python. You can use the above interface to simulate the keyboard and mouse to do some interesting things ....