Thursday, January 10, 2008

Spot the warning signs in configuration file design

Sean McGrath, ITworld.com

Say you have identified a set of parameters for your application and you are now looking at how to store them, edit them, read them in and so on. You could start with a simple ini or you might be inclined to XML-ize. I like to use Python syntax for parameterization from the get-go. If I need to, I will write a parser for whatever subset of Python my application ends up using in the real world. But I wait for real-world experience using the application to tell me what that subset is. I don't try to second-guess it. ...continue reading 'Spot the warning signs in configuration file design'

4 comments:

Anonymous said...

It is easier to pare back from an overly powerful system than it is to expand an overly restricted system.

Easier perhaps, but also less safe, no? How do you know that you haven't inadvertently left in exploitable capabilities in your overly powerful system? In an overly restricted system I start from a position of safety.

Anonymous said...

Embedded DSLs are great for configuration. Another exercise would be to compare Rake to Ant. I have a thing for build configuration files (yeah - I'm kind of weird). I've studied make, scons, ant and rake - rake was a revelation. So much so I had to write something like it in perl (see link). I've been flirting with embedded-DSL-posing-as-config-files since I began using Emacs. They're my favourite waste of time.

Anonymous said...

High level languages lend themseves to their own configuration files. But be aware that you are cutting out a fat layer of abstraction when you do it. Most of the languages that lend themselves to this kind of thing (python, perl, ruby etc) make parsing simple config files or ini files trivially easy anyway. Something like YAML gives you the next step up. So it's only when things get complicated that this comes into play, and at that point it will be *hard* for someone else using another language to every parse your config files properly.

So while the technique is useful, I think it's useful in a slightly narrow range of situations

Jared said...

I did something very much like this several years ago.

One difference was it was done in Perl.

Anyway, I also wrote a Perl module to locate and load the config file.

Using config files based on the language in this way does present some opportunity for problems, but the benefits have so outweighed any problems that I would never consider doing config files any other way.

For instance in perl you can check for syntax correctness on the command line:

perl -cw "config.file"

Python and others can likely do the same.

One of the biggest advantages IMO is creation of parameters in data structures understood natively by the script that loads the parameter file.

For instance I use a perl script as a password server for retrieving passwords in scripts.

The data structure in the config file looks similar to this:

%pwd = {
all => {
db1 => {
sys => 'sys_password',
system => 'system_password'
},
db1 => {
sys => 'sys_password',
system => 'system_password'
},
}
}

I can check the syntax with "perl -cw pwd.conf"

The data structure is natively understood by the password server script, so there is no processing of parameter into variables inside the server script, it just uses the data.