Experimenting with Webware

This weekend I was looking at creating a web frontend to some information I've captured about my finances (don't ask). Having heard good things, I thought I'd try out Webware, a web-based-application development system. Summary: Really easy to start working with, and doesn't impose much pattern to the way you work.

Besides Webware, I'm using Ian Bicking's FunFormKit and SQLObject.

FunFormKit's a really well-thought out form designer, offering easy to use form validation, input conversions, and layout. It's incredibly easy to use.

SQLObject's an object-relational mapper focussed on mapping RDBMS tables to Python objects, and not attempting to map arbitrary Python objects. It's also incredibly simple to use - basically create one class, and set the __connection__ variable, and you're away...

The really cute thing about SQLObject is its SQL Builder, which uses Python objects and functions to generate the WHERE clause of a SELECT statement without breaking out into writing it in a string. For example:

for a in Transactions.select(Transactions.q.vendor.contains('rose')):
    print a

This prints every record that matches where the data in the vendor column of the record contains the characters rose.

So, here's my program, with some interspersed comments. (Sorry if that hurts copy-and-paste - it's not like you have the SQL data that makes it useful anyway...)

from WebKit.Page import Page
from WebUtils.Funcs import htmlEncode


from SQLObject import *

__connection__ = MySQLConnection(
    host='localhost', db='sqlobject_test',
    user='sqlobject', passwd='sqlobject')

SQLObject doesn't export junk, so I do the evil import * import. Here I set the __connection__ variable to my local database. That's all you need to do for most common uses of SQLObject, it seems.

class Transactions(SQLObject):
    _conn = __connection__
    date = DateTimeCol()
    vendor = StringCol(length=250)
    type = StringCol(length=25)
    ccdebit = DecimalCol(size = 10, precision = 2)
    cccredit = DecimalCol(size = 10, precision = 2)
    cadebit = DecimalCol(size = 10, precision = 2)
    cacredit = DecimalCol(size = 10, precision = 2)
    interaccount = DecimalCol(size = 10, precision = 2)

Here I describe the table I already have with all my financial transactions over the past few months (Yay eBucks!).

SQLObject uses a name mapper between (by default) mixedCase for the Python object attributes and underscore_separated for the SQL columns. Useful default, since that's what most people I know use. In this case, it's not necessary though. It's also easily customisable on a class-by-class basis, and also on a connection basis - so you can use underscore_separated in both, or use CamelCase instead of mixedCase if you wish. Or you can probably write your own converter...

Another useful default is having 'id' being the primary key column in each table - that's what I use. Easily accessible, however, is the transaction_id option, available via the same method as the converters above. I like libraries that easily support the way I think.

def buildQuery(searches):
    criteria = []

    for search in searches:
        field = search['field']
        operation = search['operation']
        value = search['value']
        item = getattr(Transactions.q, field)
        criterion = {
            'contains': lambda : item.contains(value),
            'is': lambda : item == value,
            'nonzero': lambda : item != zero,
            'gt': lambda : item > value,
            'lt': lambda : item < value,
        }[operation]()
        criteria.append(criterion)

    return SQLBuilder.AND(*criteria)

Here, buildQuery takes the query I build in the web interface, and converts it into a SQLObject query using Python objects. See, not a string in site there!

FunFormKit passes in the data in the usable form of a list of dicts, and I've conveniently used a select box that sends back the name of the column/field/attribute in my SQLObject-based class. So, getattr gets me the field I'm interested in.

Then, I use a lambda-switch-statement to confuse matters, but basically depending on the operation (contains, is, notzero, &c.;), I perform a standard Python comparison on the field (well, .contains isn't quite standard).

Finally, I take all my criteria, and use SQLBuilder's AND function to make my query require all the criteria to be fulfilled.

### FunFormKit

from FunFormKit.Form import *
from FunFormKit.Field import *
from FunFormKit.Validator import FormValidator

Again, FunFormKit doesn't export junk, so it's convenient to be naughty.

class SearchValidator(FormValidator):
    def __init__(self, operation, value, **kw):
        self._operation = operation
        self._value = value
        FormValidator.__init__(self, **kw)

    def validatePartialForm(self):
        return True

    def validatePartial(self, fieldDict):
        if not fieldDict.get(self._operation, None) \
           or not fieldDict.get(self._value, None):
            return None
        return self.validate(fieldDict)


    def validate(self, fieldDict):
        print fieldDict
        if fieldDict['operation'] not in ('none', 'nonzero'):
            if not fieldDict['value']:
                return {'value': 'Must have a value for operation %s' % fieldDict['operation']}

My criteria are a triple of field (column name), operation (contains, is, greater than, ...) and value. That is put together as one super-widget in FunFormKit, and has its own validator. This validator ensure that each search triple has appropriately enough information.

All methods but validate are boiler-plate for this sort of validation. validate gets called with a dictionary of the values of the sub-widgets. My validation is that the value field must contain something if the operation is "is", "greater than", and so forth; all but "none" and "not zero".

def searchOperations(name):
    fields = [
        ('vendor', 'Vendor'),
        ('type', 'Type'),
        ('cadebit', 'Cheque Account Debit'),
        ('cacredit', 'Cheque Account Credit'),
        ('ccdebit', 'Credit Account Debit'),
        ('cccredit', 'Credit Account Credit'),
        ('interaccount', 'InterAccount'),
    ]
    f = SelectField('field', fields)
    operations = [
        ('none', 'None'),
        ('is', 'Is'),
        ('nonzero', 'Not Zero'),
        ('gt', 'Greater Than'),
        ('lt', 'Less Than'),
        ('contains', 'Contains'),
    ]
    o = SelectField('operation', operations)
    t = TextField('value', size=25, maxLength=250)
    c = CompoundField(name, [f, o, t],
        options={'repeat': 3},
        formValidators = [SearchValidator('operation', 'value')])
    return c

searchOperations creates the super-widget for my search triple. It's pretty self-explanatory, I think.

fields is a list of tuples, the first value in the tuple is the value of the HTML Select widget, and the second value is the displayed text in the widget.

Same with operations, and you can see the TextField with built-in validators to check the maximum length of the text. The CompoundField bundles them together, and creates three separate copies of the super-widget, and sets up its validator.

fields = [
    searchOperations('searches'),
    SubmitButton('refine',
        description='Refine'),
    SubmitButton('show',
        description='Show'),
]

formDef = FormDefinition('TransactionsView',  # the form action
    fields, name='form')

The Form Definition is the central focus of FunFormKit - it basically describes the form in question in terms of Field objects, and can't change between invocations. If it did, what was shown in the form would not be what it's expecting to validate when that form is submitted.

class TransactionsView(Page, FormServlet):
    def __init__(self):
        FormServlet.__init__(self, [formDef])
        Page.__init__(self)

    def writeContent(self):
        #print self.request().fields()
        submitted, data = self.processForm()
        if str(data) == data:
            data = {}
        filterData(data)
        fields = self.request().fields()
        options = {'searches': {'repeat': len(data.get('searches', []))+2}}
        rf = self.renderableForm(defaults = data, optionSet = options)
        self.write(rf.htFormLayout([
            ':searches',
            ('refine', 'show')
            ],
            spacing=3))

        wr = self.write
        if fields.has_key('show') and data.has_key('searches'):
            q = buildQuery(data['searches'])
            for a in Transactions.select(q):
                wr(htmlEncode(str(a)))
                wr('<br />\n')

Here's where Webware comes in. It looks for an object of the same name as the file - in this case TransactionsView. The Page base class basically takes care of most of the requirements for an object to be published in Webware. Such an object should receive messages about the object no longer being used, being woken up, and about Webware transactions. But that's overkill for most work, and Page provides all you need except your content itself.

FunFormKit, being developed with the understanding that many (but not all) of its users will use Webware, provides a simple mix-in (FormServlet) that simplifies the use of FunFormKit with Webware.

writeContent is where you generate your content. This is where the hacks are...

First, I get the FormServlet to process the form and figure out whether I was successfully submitted. data will contain the data in a dictionary - as I said earlier, repeated widgets and super-widgets are handled as lists and dictionaries respectively.

First hack - if data's empty, I create an empty dictionary. This allows me to always pass a dictionary to filterData, which removes all of the search criteria triples where the operation is None. And to pass it back to the form when it displays, as this is an interactive form.

optionSet allows you to override some stuff about your formDef without changing it - here I make sure I always have two extra set of criterion super-widgets around to add more criteria to my search. defaults pre-populates the form with data, as described above.

So, the form object is made with self.renderableForm, and then htFormLayout allows me to describe how to lay out the form on the screen. Nothing fancy in this layout - the search widgets are prefixed with a colon to prevent a label from being printed, and the refine and show buttons are layed out side-by-side.

Another hack. I have two buttons, refine and show. When I refine, I don't perform the query that's described by the form, I just allow for more criteria. show does the query and displays the result. I use buildQuery to build my SQLObject query, and I apply it to the Transactions table, and write the resulting objects to the screen. No templating yet, my Monday night was eaten up by database work at work...

def filterData(data):
    searches = data.get('searches', [])
    newsearches = []
    for search in searches:
        if search["operation"] != 'none':
            newsearches.append(search)
    data['searches'] = newsearches

filterData is described above - removes unneeded criteria.

Next on the list - the Cheetah Template Engine. Having used Smarty extensively at work, I'm enjoying the freedom to separate templating/presentation from code completely.

This software stack - Python, Webware, FunFormKit, SQLObject, and Cheetach seems to be incredibly well-suited for rapid application development, without sacrificing anything. Like Python itself, the rest are easy-to-use, intuitive, and follow the Policy Of Least Astonishment (POLA).

Of course, maybe I just haven't hit the shortcomings yet... Night!