Python simulates mouse keyboard action Ghostmouse tinytask call an external script or program autopy right-click Save As

1. Reference

• Autopy (see the last chapter in practice)

Using Python to make a game plug-in (top)

Autopy Introduction and Tutorial

Autopy.mouse.smooth_move (1, 1) allows for smooth movement

Autopy-api Reference

• Pip Install Pyuserinput

Savinaroja/pyuserinput

[python3.5] [Pyuserinput] simulating mouse and keyboard simulations

python-Simulating mouse Keyboard action

• AutoIt

Selenium upload (download) detailed with AutoIt

Selenium Webdriver Right-click Save as download file (combined with robot and autoIt)

• Win32API

Python's approach to simulating keystrokes and mouse clicks under Windows

Simulate keyboard input with Python

• Pyautowin

https://pyautogui.readthedocs.io/en/latest/

• Other Pyautogui etc...

Python analog Keyboard Mouse input

Pyautogui-python version of Autoit/ahk

2.

http://www.ghost-mouse.com/

Https://en.uptodown.com/windows/automatization >> Https://tinytask.en.uptodown.com/windows

https://www.autoitscript.com/site/autoit/downloads/

3. Introduction to Tools

(1) The Ghostmouse export file is readable and can be scripted to extract mouse tracks and keyboard input content. However, running the exported file from the command line only opens the program automatically, and you need to run playback through the shortcut key.

(2) Tinytask export file is not readable, can export compiled EXE, through the command line can be directly without interface playback.

Python Module Learning: subprocess Creating child processes

Python executes system commands, Os.system && os.popen && subprocess. Popen

4.python calling an external script or program (such as Tinytask)

ImportsubprocessdefRun_tinytask (rec_file_compiled):#run an external script, pass in parameters, and accept print content    #p = subprocess. Popen ([' Python ', ' xxx.py ', '-n ', sth_to_pass],stdout=subprocess. PIPE)    #result = P.stdout.read ()        #Reference pytesseract.pyCommand =[rec_file_compiled]#proc = subprocess. Popen (command, stderr=subprocess. PIPE)Proc =subprocess. Popen (command) status=proc.wait ()#error_string = Proc.stderr.read ()    #proc.stderr.close ()    #print status, Error_string    returnStatusdefMain (): Rec_file_compiled='G:/pydata/install/test.exe' # Tinytask exported compiled files      PrintRun_tinytask (rec_file_compiled)Print 'finished'    if __name__=='__main__': Main ()

5. Script to extract mouse tracks and keyboard entries for Ghostmouse records

(1) Reference

Swipe verification code hack: Python Selenium 2.0 App

(2) Code implementation

#Coding:utf-8#Ghostmouse the exported RMS file#{Delay 2.13}#{Move (1225,349)}#{Delay 0.23}#{Move (729,657)}#{Delay 1.65}#{lmouse down (727,658)} #鼠标按下开始拖动#{Delay 0.99}#{Move (727,658)}#{Delay 0.11}#{Move (790,659)}#{Delay 0.91}#{lmouse up (790,659)} #鼠标释放结束拖动ImportOS#xyd_typical = (1,0,0.04)defread_rms (file_rms): with open (file_rms) as FP:#Os.path.sepLmouse_down =False forLineinchfp:#{lmouse down (727,658)}            #{Delay 1.65}            if 'Lmouse Down' inchLineor(Lmouse_down = = True and 'Move' inchLine ):if 'Lmouse Down' inchLine:lmouse_down=True xyd_records=[] x_last, Y_last= 0, 0#ensure that the first offset is the actual start positionXy_pos = Line.split ('(') [1].split (')') [0].split (',') X_pos, Y_pos= [Int (i) forIinchXy_pos]Continue            #{Move (729,657)}            #{Delay 1.65}            ifLmouse_down = = True and 'Delay' inchLine:x_delta, Y_delta= X_pos-x_last, y_pos-Y_lastifX_delta = = 0 andY_delta = =0 and Len (xyd_records)! = 0: #len Maybe the starting point is 0,0                       .Continue                Else: Delay= Float (Line.split (' ') [1].split ('}') [0]) Xyd_records.append ((X_delta, Y_delta, delay)) X_last, Y_last=X_pos, Y_posContinue                            #{lmouse up (790,659)}            ifLmouse_down = = True and 'Lmouse up' inchLine :#x_init y_init x_change y_change                #x y d offset per second                #x y dWith open (File_txt,'a') as Fh_txt:#x_change = SUM (x for x,y,d in xyd_records)X_init =Xyd_records[0][0] Y_init= Xyd_records[0][1] X_change= SUM (x forX,y,dinchXyd_records[1:]) Y_change= SUM (y forX,y,dinchXyd_records[1:]) Fh_txt.write ('{ } {} {} {}\n'. Format (X_init, Y_init, X_change, Y_change))#plus os.linesep is ' \ r \ n '                     forX,y,dinchXyd_records[1:]:#The first record is the starting position, and the value records every offset afterFh_txt.write ('{} {} {}\n'. Format (x, y, D)) Lmouse_down=False xyd_records= []                                    defRead_txt (file_txt): With open (File_txt,'R') as Fp:result= {}#(X_init, Y_init, X_change, Y_chang): [(X0,Y0,D0), (X1,Y1,D1) ...]                 forLineinchFp:line=Line.strip (). Split ()ifLen (line) = = 4: Key= Tuple ([Int (i) forIinchLine ]) Result[key]= []            elifLen (line) = = 3: X,y,d=Line x,y,d=int (x), int (y), float (d) result[key].append ((int (x), int (y), float (d)))returnresultif __name__=='__main__': File_rms= Os.path.join (Os.path.abspath ('.'),'mouse.rms') File_txt= Os.path.join (Os.path.abspath ('.'),'Mouse.txt') read_rms (file_rms) result=read_txt (file_txt) forKvinchResult.items ():Printk,v

6.selenium+autopy Implement right-click Save As

#!/usr/bin/env python#-*-coding:utf-8-*Import Time fromSeleniumImportWebdriver fromSelenium.webdriver.common.action_chainsImportActionchains#Import Autopy fromAutopyImportKey, Mousedriver=Webdriver. Chrome ()#Driver = Webdriver. Firefox ()Driver.get ('http://www.baidu.com')#<a class= "Mnav" href= "http://news.baidu.com" name= "Tj_trnews" > News </a>E = Driver.find_element_by_partial_link_text (u'News')#The page displays the link text, not the specific link address, so ' news ' doesn't work!!! #e = driver.find_element_by_name (' tj_trnews ')#fifefox geckodriver Context_click anomalies#https://stackoverflow.com/questions/6927229/context-click-in-selenium-2-2#http://bbs.csdn.net/topics/392058306#Https://stackoverflow.com/questions/40360223/webdriverexception-moveto-did-not-match-a-known-commandactionchains (Driver). Context_click (E). Perform ()#actionchains (Driver). Move_to_element (E). Context_click (E). Perform () #也行#Mouse.click (mouse. Right_button)Time.sleep (1.5) key.type_string ('k') Time.sleep (1.5) key.type_string (Time.strftime ('%h%m%s') ) Key.tap (key. K_return)

Python simulates mouse keyboard action Ghostmouse tinytask call an external script or program autopy right-click Save As