Python's dynamic re-encapsulation tutorial

Let's describe the plot of this article: Suppose you want to run a process on the local machine, and some of the program logic is somewhere else. Let us especially assume that this program logic will be updated from time to times, and that you want to use the latest program logic when you run the process. There are many ways to meet the requirements just mentioned, and this article will show you several methods.

As the "lovely Python" column continues, the ongoing enhancements to my public domain utility txt2html have been discussed. The utility converts the "smart ASCII" text file to HTML. The previous article discussed the WEB Proxy version of the utility and the curses interface of the utility program. Again, I occasionally notice that some ASCII tokens can be converted in a more efficient way, or that an error in handling a particular markup structure is resolved.

In fact, the articles in this column are written in ASCII and then converted into HTML formats that you can read during the editing process. Before I published the draft article, I ran a program similar to the following:
Command-line HTML for an article

txt2html charming_python_7.txt > charming_python_7.html

If you want, I can specify some flags to modify the operation, but in any case, the latest version of the converter is in my local drive and path. If you work on another machine, or if you want to use the utility, the process is cumbersome: please visit my website, pay attention to compare the version number and the file date (sometimes the change is too small, I will not change the version number), download the current version, copy the current version to the correct directory, and then run the command line converter. (See resources later in this article.) ）

The above process includes several steps that require manual operation and are relatively time-consuming. It should be simpler and can do this.
Command-line Web access

Most people think of the Web as a way to interactively browse pages in a GUI environment. It's good to do that, but there are many features in the command line. A system with a text-mode Web browser Lynx can completely view the entire Web as another set of files used by command-line tools. For example, I find that some commands are useful:
Using Lynx for command-line Web browsing

Lynx-dump http://gnosis.cx/publish/.lynx-dump http://ibm.com/developerworks/. > Ibm_developer.txtlynx-dump http://gnosis.cx/publish | WC | Sed "s/(*[0-9]* *\) \ ([0-9]*\) \ (. *\)/\2/g"

The first line says, "show David Mertz's home page (in ASCII text) to the console. The second line says, "Save the ASCII version of IBM's current DeveloperWorks home page to a file." The third line example says, "Displays the number of words on the David home page. (Don't worry about the details, it shows only the command-line tools that are combined with the pipeline.) ）

With regard to lynx, it is important to note that it (when using the-dump option) performs almost the exact opposite of txt2html: the previous tool transforms HTML into text, and the latter is converted to other formats. But there is no reason not to use txt2html, which is as popular as lynx. You can do this with a short Python script:
' fetch_txt2html.py ' command line converters

Import sys from urllib import urlopen, UrlEncode if Len (sys.argv) = = 2:  cgi = ' http://gnosis.cx/cgi/txt2html.cgi '  opts = UrlEncode ({' source ': sys.argv[1], ' proxy ': ' NONE '})  print Urlopen (CGI, opts). Read ()       else:  print " Please specify the URL for txt2html conversion "

To run this script, simply do the following:

Python fetch_txt2html.py http://gnosis.cx/publish/programming/charming_python_7.txt

This does not provide you with all the switches that the local txt2html handles, but it is easy to add them if necessary. You can transport and redirect output as you would with any command-line tool. However, in these versions, you can only process data files that URLs can reach, not local files.

In fact, fetch_txt2html.py can accomplish tasks that Lynx cannot accomplish (txt2html itself cannot): it not only obtains the data source from the URL, but also obtains the program logic remotely. If you use fetch_txt2html.py, you do not have to install txt2html on the local machine, call processing remotely (with the latest version), and send the results back as if you were running a local process. Isn't it great? The local version of txt2html can access the remote URL, just like accessing a local file, but it does not guarantee that it is up to date ...!

Dynamic initialization

Using fetch_txt2html.py ensures that the most up-to-date program logic is always used in the conversion. However, the other thing that this method can do is to transfer the processor (and memory) requirements to the Gnosis.cx Web server. The load for this particular process is not particularly high, but it is likely that other types of processes processed on the client are more efficient and satisfying.

The way organizations txt2html-that is, the way most programs are organized-is a core flow control function provided by a variety of utility functions. In particular, these utility functions are frequently updated functions, and the core functions (main () and some other functions) change only when a major rewrite is made. In summary, the utility function is the one that is effectively updated when each program runs. In fact, most of the functions in the main txt2html module dmtxt2html are sufficient.
' d2h_textfuncs.py ' Dynamic txt2html update

"" "Hot-pluggable replacement functions for txt2html" ""      #--functions to massage blocks by type #def      titleify (bloc k):    #def authorify (block):    # ... [more block massaging functions] ... #--Utility functions for text transformation #def adjustcaps (TXT):    #def capwords (TXT):    #def urlify (TXT):    def typographify     (TXT):      # [Module] names   r = re.compile (R    "' (\ \ S '/">]|^) \[(. *?) \] ([<\s\.\),:; ' "?! /-]) "" ", Re. K: Re. S)  txt = r.sub (    ' \\1 \\2 \\3 ', txt)      # *strongly emphasize* words   r = Re.compile (r    "" ' ([\ (\s '/"]|^) \* (. *?) \* ([\s\.\),:; ' "?! /-]) "" ", Re. K: Re. S)  txt = r.sub (    ' \\1\\2\\3 ', txt)      # ... [More text massaging] ...           return           txt    # ... [More text transformation functions] .....

To use the latest and most specific support modules, you need some preparation steps. First, download the main txt2html module to the local system (this is a one-time step). Next, create a Python script similar to the following example on the local system:
' dyn_txt2html.py ' command line converters

From      dmtxt2html     Import      *       # import the body of ' txt2html ' code     from      urllib     import      Urlopen    Import      sys    # Check for updated functions (fail gracefully if not fetchable)     try     :  Updates = Urlopen (    ' http://gnosis.cx/download/t2h_textfuncs.py '). Read ()  fh = open (    ' T2h_ textfuncs.py ',     ' W ')  fh.write (Updates)  fh.close ()    except     :  sys.stderr.write (    ' cannot currently download txt2html updates ')    # import the updated functions (if available)     try     :      from      t2h_textfuncs     Import      *    Except     :  sys.stderr.write (    ' cannot import the updated txt2html functions ')    # Set options based on RU Nmode (Shell vs. CGI)     if      len (SYS.ARGV) >= 2:  cfg_dict = Parseargs (sys.argv[1:])  Main (cfg_dict)    else     :      print "Specify URL (and options) for txt2html conversion"

In the dyn_txt2html.py script, note that when you execute the FROM T2H_TEXTFUNCS import * statement, all functions previously defined in dmtxt2html (such as typographify ()) are made by the T2h_textfuncs version Replace the function with the same name. Of course, if the T2h_textfuncs function is commented out, it will not be replaced.

One small problem to note is that different systems handle write STDERR in different ways. In a UNIX-like system, STDERR can be redirected when the script is run, but in the current OS/2 shell and Windows/dos, the STDERR message is appended to the console output. You may want to write the above error/warning to a log file, or just be accustomed to directing STDOUT to a file (which may be more useful). For example:
The command line of ' dyn_txt2html '

g:\txt2html> python dyn_txt2html.py test.txt > test.htmlcannot currently download txt2html updates

The error goes to the console, and the converted output goes to the file.

One more interesting thing is dyn_txt2html.py why not download the entire dmtxt2html module and download only the support module. Of course there is a reason for that. The T2h_textfuncs support module is much smaller than the main dmtxt2html module, especially since most functions have been truncated/commented out. It is significantly faster on a modem connection. But the download size is not the main reason.

For txt2html, it doesn't matter if the user automatically downloads the entire latest module. But what happens when the program logic is a distributed system (especially if the maintenance responsibility is also distributed)? You might have Alice, Bob, and Charlie separately responsible for modules Funcs_a, Funcs_b, and Funcs_c. Each of them makes regular (and independent) changes to the functions they are responsible for, and uploads the latest and best versions to their own sites (such as http://alice.com/Funcs_A.py). In this case, it is not feasible to have three programmers change the same master module. However, you can directly extend scripts similar to dyn_txt2html.py to attempt to import funcs_a, Funcs_b, and Funcs_c at startup (if you cannot get those resources, you will fall back to the Mainprog version).

Long-running dynamic processes

So far, the tools we have researched have been given dynamic program logic by downloading update resources at initialization time. This makes sense for command-line processing or batching, but what happens to long-running applications. This long-running application is most likely a server process that constantly responds to client requests. In this case, however, we will use the curses_txt2html.py developed for the previous article to illustrate the Python reload () function. The program curses_txt2html is the wrapper for the dmtxt2html local copy. This is not the second time to mention curses programming, and it is enough to talk about curses_txt2html providing a set of interactive menus to configure and run multiple successive txt2html transformations.

Curses_txt2html can run in the background all the time, and when we switch to its session and run the transformation, we want it to be able to use the latest program logic. For this particular simple example, shutting down and restarting the application is not difficult and does not cause any special damage. But this can easily be reminiscent of other processes that have been running (perhaps a process that describes the state of operations performed in a session).

In this article, a new File/update submenu is added. When it is activated, only the new function update_txt2html () is called. We have seen these steps in other examples in this article, in addition to the curses calls that are related to providing confirmation of the occurrence:
' curses_txt2html.py ' Dynamic update function

def update_txt2html     ():      # Check for updated functions (fail gracefully if not fetchable)   s = Curses.newwin (6, 4, 5) S.box ()  s.addstr (1, 2,     "* Press any KEY to CONTINUE *", curses. A_bold) s.addstr (3,2,     "... downloading ...") S.refresh ()      try     : from      urllib     Import      Urlopen    updates = Urlopen (    ' http://gnosis.cx/download/dmTxt2Html.py '). Read ()    fh = open (    ' dmtxt2html.py ',     ' W ')    fh.write (Updates)    fh.close () s.addstr (3,2,     "Module [dmtxt2html] Downloaded to current directory ")      except     : S.addstr (3,2,     " Download of updated [dmtxt2html] module failed !" )  Reload (dmtxt2html)  s.addstr (4, 2,     "Module [dmtxt2html] Reloaded from current directory") S.refresh () c = S.get CH ()   s.erase ()

There are two important differences between the dyn_txthtml.py and update_txt2html () functions. One of the differences is to continue the operation and import the main dmtxt2html module instead of just importing the support functions. This is primarily simplified for import. The problem here is that we use the import dmtxt2html to access the module, not the from dmtxt2html import *. In many ways, this is a more secure process, but the result is that it becomes more difficult to overwrite the functions in the dmtxt2html (either unintentionally or intentionally). If we are attaching a function from D2h_textfuncs, we must execute Dir () on the imported support module and append the member to the "dmtxt2html" namespace as an attribute. Performing this style of overlay is reserved for the reader's practice.

The main difference with the update_txt2html () function is the use of Python's built-in reload () function. Performing a completely new import dmtxt2html will not overwrite previously imported functions. Please pay close attention to this point! Many beginners think that re-importing the module will update the in-memory version. This is wrong. In fact, the method of updating the memory image of a function in a module is the reload () module.

Another tip has been performed in the example above. The download location for the Update dmtxt2html module is the local working directory, which may or may not be the original directory that was loaded into the dmtxt2html. In fact, if it is in the Python Library directory, then you might not be using it in that directory (perhaps without user permission). However, the reload () call attempts to load from the current directory before attempting the remainder of the Python path. So, regardless of whether the download is successful, reload () should be a safe operation (although it may load new modules or not mount).

