Perl with rich functions: Use Perl for application configuration

CPAN appconfig module and database-oriented Configuration

Level: elementary Teodorzlatanov (tzz@iglou.com), programmer, gold Software Systems October 09, 2000 If you use a manual build method, the file-based configuration will soon crash. Teodorzlatanov demonstrates how the appconfig module handles local configuration storage for Perl programs and stores these configurations in the database so that they can be accessed from any machine on the network. One of the primary needs of a program (from a low-level column directory program to a web browser) is that it should be configurable. It turns out that the combination of file-based configurability and command line options is a long-term and flexible solution for configurability needs. Perl programs usually use this method, although they often include a configuration file and a command line parsing routine. The command line parsing we will use in this article is a little complicated. Therefore, to avoid further confusion, if you are performing a resolution level higher than a simple parameter, we recommend that you use parse: recdescent (or an equivalent parsing module ). For more information about complex command line parsing, see my previous article on English-speaking Perl programs. Before starting, make sure that you have installed Perl 5.005 (or a later version) and the CPAN appconfig module on the system. You also need persistent: MySQL or the persistent class for your specific database. These can be obtained in CPAN (refer to references later in this article ). Simple Method: DIY) Theoretically (and when appropriate tools are available !) Anyone can build a configuration parser, right? For example,Perl cookbookDemonstrate a quick implementation that provides a good start. How hard is it to compile a configuration file parser if you start with this class implementation? It is actually quite difficult because such projects involve the following complex issues: Blank lines and comments in the configuration file Incorrect rows (such as misspelled keywords) and the content that is indispensable and negligible You must write your own parser because you may need multiple different data structures (such as Boolean, scalar, array, and hash) Multiple configuration files Variable Default Value Integrate command line options with file configurations and control their interaction Methods Use another DIY configuration file format to train users (this is usually similar to: "This is valid if only the '=' sign is in another line. Oh, there are comments starting with '#', but they must be independent of other rows. Do not forget to use uppercase for keywords and lowercase for values. Come back! Come back! I have not told you about mandatory keywords yet !") Re-compile or copy the configuration code that may have errors instead of reusing the module. Make the configuration a DIY random hash object with consistent interfaces instead of Common keywords Are you scared? This is why we use appconfig. It can handle all these problems. It is clear that you should not use DIY. Back to Top Turning to appconfig Although the appconfig CPAN module compiled by Andy wardley helps solve all the problems listed above, it is not omnipotent. It is impossible to magically improve your program. Sometimes you need to rewrite it to use appconfig. (There is also a learning curve. This article attempts to help you reduce the learning difficulty .) Obviously, if you are not sure whether to use DIY or appconfig, you should make a decision based on your experience and the content you are writing. However, I believe that appconfig cannot be as good or better as DIY. The following describes what appconfig can do for you (according to the problem list in the previous section ): Blank lines and comments in the configuration file: appconfig can recognize blank lines and comments and ignore them. Which content of the wrong row (like the spelling keyword) is indispensable and which content can be ignored: You can set the sensitivity of appconfig to ignore the error setting or abort the program. If other spelling rules can be used, the keywords can have aliases (for example, in international settings ). Write your own parser because you need different data structures (Boolean, scalar, array) and hash: appconfig processes all these data structures, but it does not need to be nested in this way. If you need nested hash or array, You need to manually or help appconfig. Multiple configuration files: appconfig processes any number of configuration files as needed and installs the settings from each file in sequence. You can also help appconfig reset the array and hash so that the values inserted to the bottom of the stack do not have to appear at the top of the stack. Default Value of the variable: appconfig provides the default value of the variable. The-variable syntax resets the variables in the configuration file to their default state. Control command line options and integrate them with the file configuration: appconfig provides support for parsing the getopt: STD and getopt: long command line options. Resolution can be performed before or after the configuration file is read. Use another DIY configuration file format to train users: appconfig uses a standard and flexible format. "Keyword value" and/or "keyword = value" are acceptable for scalar. Because the array is composed of elements, "arr = 1" followed by "arr = 2" will generate an arr array with elements 1 and 2. You can also specify Boolean options as "bool", "nobool", and "! Bool, bool on, bool off, bool Yes (but bool no produces an error), bool = 1, and bool = 0 ". (Apparently, Dr. George Boole, inventor of the "Notorious" symbolic logic-see references-to find these options very friendly .) The hash option is specified as "keyword parameter = value", and the value will become a hash item with the keyword parameter. Re-compile or copy the configuration code that may have errors, rather than reusing the module: appconfig is quite stable for this, and the interface to this module is unlikely to be changed. It has also passed tests by thousands of other programmers, so why not use it? Make the configuration a consistent interface object, instead of using the usual DIY random hash: the consistent API extracts the configuration handler from the main program, and simplify the connection with the processing program (in this example, the processing program is appconfig ). This method introduces fewer errors because it only uses methods instead of data structures.   As we have seen many good reasons for using appconfig, let's take a look at the complete example of appconfig usage with annotations. At present, we will omit many more advanced features (which will be discussed in the next section ). You can use "-varname value" on the command line to set scalar, Boolean, and array variables, and use "-varname key = value" to set hash variables. The configuration file config. pl is used here. The following is an example: # blank lines are ignored # set a booleandebug # set a scalarname=E.T. # set an arrayhosts = dbhosthosts = backuphost # reset the hosts array-hosts # add new values to hostshosts = firewallhosts = farewell # set a hashphone joe = 222-333-4444phone marge = 555-666-7777 Back to Top Appconfig advanced usage Appconfig supports variable extension at several levels, depending on expand settings. For more information, see the appconfig document. # expand all variables, globallymy $config = AppConfig->new({ GLOBAL => { EXPAND => EXPAND_ALL } }); # expand just HOME_DIR as UID, so "~username" will work as in the shell$config->define('HOME_DIR => { ARGCOUNT => ARGCOUNT_ONE, EXPAND => EXPAND_UID }); The INI style section is another feature of appconfig and you will find it useful. By using the [section] (which occupies one row) in the configuration file, you can list all keywords used before the end of the file, you can also use the underscore '_' to list all the keywords in this section and the next section. For example: location = /tmptype = txtname = accounts.txt[database]host = wyrmuser = slayerpassword = amethyst It is equivalent: file_location = /tmpfile_type = txtfile_name = accounts.txtdatabase_host = wyrmdatabase_user = slayerdatabase_password = amethyst You can use the varlist () function to check the appconfig configuration object. The following code prints the content of each variable in the appconfig object. Note: varlist () may be a bit troublesome because it must use regular expressions (null strings are definitely invalid ). use Data::Dumper; # for hash and array references my %varlist = $config->varlist('.*');foreach my $varname (keys %varlist){ print "Variable name $varname, value = ", Dumper $config->get($varname), "/n";} Appconfig has a getopt: Long interface, which allows access to all functions of the getopt: Long module. The following code defines the variable parameter getopt: Long. Call this code to parse parameters from the command line. Invalid values may cause errors. $config->define("help|h|!"); # define a boolean$config->define("code|c|=i"); # define a scalar integer$config->define("list|l|=f@"); # define a array of floating point values only$config->define("uids|u|=f%"); # define a hash of floating point values only $config->getopt(); # instead of args(), to use the Getopt::Long options You can also perform variable verification in appconfig. This means that by referencing a regular expression (or even a piece of code), the variable rejects attempts to set its value to some malicious value or completely meaningless value. # the username validation succeeds only when it is exactly "joe"# the password validation succeeds when it contains "joe" or "joE"$config->define( 'USERNAME' => { ARGCOUNT => ARGCOUNT_ONE, VALIDATE => sub # subroutine validation { my $varname = shift @_; my $value = shift @_; print "$varname = $value/n"; return ($value eq "joe"); } }, 'PASSWORD' => { ARGCOUNT => ARGCOUNT_ONE,VALIDATE => "jo[Ee]" # regex validation } ); Appconfig makes it possible to trigger the operation automatically. Therefore, this operation is executed every time the value of the variable is changed. Note: The reference to appconfig is also transferred to the subroutine, so a single change triggers other variable changes. $config->define( 'USERNAME' => { ARGCOUNT => ARGCOUNT_ONE, ACTION => sub # autoaction { my $config = shift @_; my $varname = shift @_; my $value = shift @_; print "$varname = $value/n"; } } ); Back to Top Appconfig restrictions Appconfig does not process the code embedded in variables. In my opinion, no code should exist in the configuration file, and it is a bad idea to allow users to execute arbitrary code. However, appconfig does not provide automatic evaluation of the variable, although it can indeed coordinate the verification and automatic operation subroutines associated with the variable to execute this task. If you do feel that you have a strong demand for this, pick out the variables in question and run eval () on them yourself (the method described below ). Needless to say, unless you want to grant this level of control over your program to the userNoIn this way. foreach my $varname ('username', 'password'){ $config->set($varname, eval $config->get($varname));} The INI style section in appconfig is not very important. They define code sections, but they must be known before use. It is better to design the sections so that they can create new appconfig objects nested in the parent object, but this is a secondary issue. In a simple test, using appconfig does not seem to affect the loading and execution speed. It is a relatively small module, and its size/speed cost is usually negligible. Of course, if your program is sensitive to time, You Should timing it separately for use and without appconfig, and decide whether the module is worth using. The complexity and learning curve of appconfig are much lower than you expected, largely because of excellent APIs. This is confusing (especially for new programmers), but in general it is not very important for anyone with Perl experience. The resolution limit of appconfig lies in its availability. If you need advanced parsing of command line options, see the previous article about English-speaking Perl programs. (This restriction has a special relationship with appconfig's inability to perform context-sensitive parsing .) Back to Top Use appconfig and persistent: DBI to upload the configuration to the database. Before reading this section, I suggest reading my article about saving data with the persistent module. You should also understand Perl references and SQL databases. My specific code example uses the MySQL database and the corresponding persistent module. If you are using another database (such as S or Oracle), you should look for other persistent modules. Before you configure a database environment, you must design the database mode. In other words, when writing code to save and restore dataBefore, You need to determine what data you want to save. This example stores Boolean, scalar, array, and hash values in different tables. This is not necessarily the best way. You can also use a table for all data types or divide tables by purpose. My example is only one of the many ways to implement persistent configuration. It is certainly not the only method. The model I provide here may be sufficient for most purposes. It does have some limits on the value and key length, but you can easily adjust those limits in the code. However, the method for creating an array and a hash element identity may cause problems. In these cases, there is no perfect solution, but there are only different solutions to the problem. It is always troublesome to store any structured data to a relational database. We will use the _ argcount () method in appconfig: state. For more information about this method, see the appconfig: State manual page. Simply put, if we know the variable name, this method can tell us what type of variable is being processed. In my code example, I used the MySQL database and the corresponding persistent module: persistent-config.pl. Back to Top Conclusion As long as a small amount of work is done, appconfig and persistent classes can work well together. The persistent configuration script shown in the previous section can process most configurations with short keys and variable names, but there is still room for improvement. After writing scripts according to your preferences, you canAnywhereStart it and let it load the current configuration from the central host. At least, improvements will teach you about Database Configuration and help you study applications in a new network-centric approach. Code reuse is usually the only biggest benefit of a module, especially for appconfig. Appconfig provides an effective single solution that may meet most configuration requirements when DIY (hands-on) methods produce errors and latencies. The "appconfig restrictions" section lists very few restrictions. Of course, before using appconfig, you should determine what is best for your project. It is best to remember how to apply the information provided here to your specific project and study the appconfig manual page. What are you doing now? You should use only what you need from appconfig. Instead of trying to make it do all the work for you. It seems interesting to put half of the programs in a configuration file, but if you do so, users will soon roar into your office. Make your configuration file logical and simple. Write a detailed description of the configuration syntax accepted by the program, including the thoughtful command line options provided by appconfig.