Thursday, February 08, 2007

Wicket BPM Update

I got the sourceforge project running...

https://sourceforge.net/projects/wicketbpm

I'm going to try to add info on the project to the forum page for now. I have absolutely no time for documentation right now. I'm involved in about 20 different things.

I did go back and start writing tests for the code. See my previous blog post. I recently spent a day with a shop that actually does unit testing 100% of the time, and I got to see how it actually works in a real system. I've been an arrogant fool, what with all of my untested code. I was wrong.

Back to my big JSF/Facelets project till, probably, sometime next week. Then back to Wicket land.

I Finally Get It

I paid lip service to test driven development for a while now. Did the same for "agile" techniques and pair programming, etc. From my last job I had a very cynical view of the way this type of environment would actually work.

Well, I recently spent a day at a little consulting shop here in the city. It was a full day interview. I went in in the morning (very tough if you've been waking up without an alarm clock for 6 months), and worked with them all day. At first I thought the way they were developing was pretty anal. I still think SOME of it is, which I'll get to in a minute. However, later in the day, I switched seats and worked with someone else. The difference in people didn't matter so much. Its just that this second task was very different. It was modifying a data structure that went across several systems and screens, and was modeled in the database, as a class data structure in 2 languages, and as xml between systems (plus, you know, screens and whatnot). This guy hadn't ever seen this code. Obviously, I hadn't either. So we start banging away, and the tests are bombing. It took a little while, but by the time we were done, all the changes were made and it really looked like the system was going to be ok. We had stuff shake out that we'd NEVER have found by manual searching. It was awesome (you know, for a huge nerd such as myself).

I admit, I was totally wrong. Great system.

I do, of course, have some things I disagreed with. They were using Velocity as their front end technology. They did this because its simple and allows testing isolation. This is where I think a dogmatic test-all approach is a little wasteful. The first task that I was referring to above was to change the display of one report screen. What they would do is build a 'gold master' screen of html from test data. Then their unit test would periodically compare that and current output to see if they match. Then, if you change the screen, write out the new 'gold master'.

I asked him, you know, what's the point? Yeah, there are possible situations where you might trip something, but I have to imagine there are ways to get a warm fuzzy from your code without all this explicit work. I was thinking if you used an IDE with some code completion for JSTL, it should be able to recognize that your accessor string no longer matches up to a property of the object you're using. I know that, ideally, the tests should be able to run on their own, but this is a situation where you're painting yourself into a corner with regards to tool choice.

However, that's not the big take away. The big point is I saw test driven development in a real project setting, and it was as sweet as the hype claims.

On top of that, I saw immediately the benefits of pair programming. As a manager, I implicitly thought it was like paying double for the same work. I now realize there are many things pair programming saves. Nobody is browsing CNN or IM friends from the office. That right there probably pays for it. You also get the obvious stuff. Less bugs. All that jazz.

When we sort out my employment status there, I'll post the name of the company and maybe I can get them some business or whatever. All I can tell you is as a manager who worked at a high stress financial company, I always thought every place was about the same with regards to real world vs. ideal. I couldn't be more wrong. This was like the Shangri-La of programming. They were very productive, but not stressed. Awesome. The biggest golf clap ever.

Sunday, February 04, 2007

Wicket and JBPM

* Update * Since this post, I've published the initial project at...

http://www.kgalligan.com/wicketbpm

Now back to the original post...

Settle in. This will be a long one. I know its Superbowl Sunday. Maybe its my HS football knee injury, or my shocking inability to remember names, but I just don't care. What we should do is put it on a Tuesday, then make it a national holiday. This would result in a day off of work, and maybe that would induce me to care. I can dream.

Anyway, I've been working on a drop-in web app client for JBPM for a while now. I started it a long time ago, but didn't take it very far. Its hard to explain exactly what it does or why its useful. The first thing to understand about this implementation is that its essentially a wrapper around the concepts I've learned over the past several years implementing a BPM solution inside an organization. Its an attempt to whittle out the stuff that we found commonly popped up. The end result? You should be able to include this application in YOUR web application, dress it up to make it look native, and seamlessly provide workflow/BPM functionality.

Lets have some background first.

What is BPM?

BPM, or Business Process Managment, describes a class of tools, platforms, frameworks, standards, etc. for modeling business processes with software systems. Its what you might call "workflow".

I'll have to come back and flush this out a bit later. For now, please read the following, as its a much better explanation than I could give here.

The major points to take away for this project are that with BPM you can graphically model a business process. Once that is done, certain steps in that process can be set up as tasks to be performed by a human user. When you start an actual process instance, and a task is assigned to a particular user, that user can see the task in a list on their screen, and do whatever is needed with it.

What is Wicket?

Wicket is a web application framework, built for the Java/J2EE platform. See the wicket project for more information.

Why Wicket?

There are a few reasons here. Probably the biggest driver is that I have recently started looking at Wicket and see many good concepts. I'm finishing a rather large project built on top of JSF and Facelets right now, and have done tons of Struts stuff in the past, and as such I've been exposed to a few web application framework concepts. Originally I was anti-Wicket, under the idea that we have too many frameworks already, and time would be better spent on augmenting an existing one (Tapestry, JSF/Myfaces, etc). However, there are some very compelling ideas that, even if Wicket doesn't stand the test of time, should be incorporated in other frameworks. I wanted a chance to test Wicket out. For that I needed a small, relatively isolated project. This was that.

Wicket is configured and extended through the use of standard OOP concepts, rather than relying on XML configurations and complex reflection practices. This is the first thing I really liked. Extensive XML configuration always got me a little upset. I never understood why it was better to do stuff outside of the code. In code, the IDE will tell you when you've got problems. You can extend and refactor easily, etc (this is one of my issues with the "dynamic" languages popular of late, but that's a whole different discussion). My recent foray into JSF showed me what extensive XML config means. Just my JSF config files on my recent project come out to about 2500 lines. That doesn't include hibernate, web.xml, or the various other config files I have *.

I like the way Wicket manages its page state. The initial problem I had with JSF was that if you kept the bean object in the request object you have to manually hold onto all of your state between requests. If you put it in the session you've got all sorts of problems with state coherence, back button, etc. Its a mess. The myfaces project includes t:saveState, which is awesome and saved my life, but I found it to be clunky at times, and in general odd to require a third party extension to make the platform functional. Wicket keeps a copy of the page object in a special Queue of pages. If you have a page with specific state in it, the back button keeps itself straight. This makes life a LOT easier. If used properly, of course.

One of the best things about Wicket? You can enforce an interface to your page. If you have data requirements for your page, just make a constructor with them. No default constructor. I can't tell you how awesome that is. I was mentally designing a way to do that in JSF, as I'd run into this problem on a number of occasions. JSF just gives you a way to define your bean in an xml file. You can't say, "the application needs to provide x, y, and z to make this bean valid". You could code that yourself somewhere, but again, this theme of avoiding the OOP platform we're building the application on. It doesn't make sense. I think Wicket hit this one dead on with a simple, and after seeing it, obvious solution.

By the way, related to that, I'd like to address the old "you can change the paths without recompiling argument" that went way back to the days of home grown control Servlets. This is just about the stupidest thing I've ever heard, and I've actually heard it. I have what I call the "Darwin test" for co-workers. Essentially, bring up a horrible idea in a meeting. I mean so bad you'd have to be totally incompetent to think it was a good idea. So bad, the people who aren't total nuts will know you're kidding. Then, whoever says something positive about it, and isn't themselves kidding, should be immediately fired. The first horrible idea I had? Telling everybody that we've "pre-debugged" everything, so we didn't need any QA. The "changing paths without recompiling" argument is now the second horrible idea.

Why is the "change paths" thing related? In JSF, which should be a modern example of a web app framework, the links between pages are stuffed into an XML file. The first problem? Well, the idea is that you're supposed to be able to change the target of a link. Could you? Yes. What's the problem? Very few pages can be shown "raw", as in without context arguments. You almost always need an ID, or a value, or SOMETHING. Right? Well, lets say you coded the application a year ago, or somebody else did it, and you go around changing those links later on. You might run into data trouble, but you won't know it till you start mucking around with the app. Similarly, if you change the target page around a bit and add new required fields, all calling pages will need updates. You won't know about problems till somebody clicks on the source page and it bombs. Wicket's method of linking essentially fixes this problem.

What's the other JSF problem? Not only does the config file not ensure data integrity, you really CAN'T pass data back and forth. You actually have to wire a link to the new bean into your current page's bean, stuff data into it in a method call, then ALSO put the link mapping into the config file. Of course, you could use myfaces t:updateActionListener (also saved my life), but once again we need something third party just to make the platform functional. Also, by way of full disclosure, I totally abused the t:updateActionListener thing. After a while you just get tired of doing things "properly".

Its complicated. I could go on, but that's not what this post is about. Its about my JBPM client. In truth, there's a lot I like about JSF too. Its just that there are some things that seem really poorly designed. I would still suggest JSF, especially with facelets, for most larger applications. Its got huge industry momentum. Also, the components that have already been committed by the industry, open source and commercial, are very useful. I do like Wicket better, but I'm still new to Wicket and perhaps haven't yet seen all of the downside.

Oh yeah. One other thing about Wicket that's pretty sweet. The detachable models. I've run into all sorts of trouble with Hibernate and detached sessions. You really have to know what you're doing to code around these problems and not get into trouble. Wicket allows you to wrap model objects and detach them when your request is done. I've used this for the TaskInstance object in this client (see kg.bpm.jbpm.web.wicket.TaskInstanceModel). Also see databinder.net for a great hibernate based toolkit for use with Wicket.

Anyway, there's some of "Why Wicket?". If you use the client app, you'll see that extending and customizing it should be relatively easy. Originally this app was built with a servlet/JSP task list, xml config file(s), and standard URL forwards so it could be used with any standard web framework technology. While this might have been a little more useful to the average user, I think this version is cleaner, easier to extend and use, and I just plain old want to support Wicket on this one.

What I don't like about Wicket? The name. It needs something cooler. Like "Wicked". Ok, that's lame, but you get the idea.

Why This Client?

In my experience, BPM platforms provide the server functionality, and many connectors and whatnot, but only provide minimal client tools for user task lists. They may provide an example application for inclusion inside your application, but you still wind up doing much of the coding yourself. I've concluded the reason for this is that many clients will want something that is simply too customized. Attempting to write an app to make everybody happy will fail, and the resulting application will be too complicated to give to somebody as an example of using the framework. This client will not make everybody happy, but should provide a quick implementation for your application.

BPM functionality is very powerful. There are many domains that would benefit from a little BPM, but they don't get it. Why? Well, until recently, BPM engines were ridiculously expensive. They still are, except for JBPM and other open source versions. Also, including this functionality in your app is not a trivial exercise. Add to that the learning curve, and you have some problems. Besides the standard technical learning curve, you really have to think differently when implementing on BPM. Why bother? Once you get it, you really get it. There's just a lot more you can do with minimal effort once you get BPM set up and you understand how it works.

You CAN hardcode the same stuff into your application. There's no magic with BPM. You can do the same thing with straight code. The problem, of course, is that you'll be doing all sorts of extra work.

So, again, why this client? You could use the client tools provided by the platform. At least internally you could. Log in, see your tasks, do them and mark them complete. No so hard, right? The problem is that many screens require custom input or display. It is an application, remember. Maybe the BPM process has an id value to an Account. If you used the standard tool, you'll see a number. The id value. What you WANT to see is the account data. You could, of course, stuff some other account identifier into the BPM process and get the whole thing to work, but your clients will get tired of this kind of thing pretty quickly.

This client provides for a few things. Each task can be mapped to a standard or customized task screen (or set of screens, as in a wizard). So, lets say you need to input a final price for something that is only known by you. When you get the task, it shows up in your list. Click the task, and there will be an input box, complete with validation specific to the task at hand. This is very important, and is the primary reason this project exists.

The second major feature of the client is process reporting. This is a simple concept, although the implementation is a little difficult to explain. Although you can fully customize the reporting (or completely rip it out and provide a totally proprietary solution), a standard templating system has been included. This will allow you to define templates to display the data associated with a particular process instance. Currently there is a simple implementation based on Velocity.

This has changed into a rant about different types of web frameworks rather than about the project, so I'll cut this off here. If you want to read just pure instructions, see my website...

* Yes, I realize that my anti-config-file stance would support Rails and Ruby. However, I would counter that its the way Rails configures itself rather than anything inherent in the Ruby the language that removes the config files. The extensive config files I mentioned above are due to implementation choice. Hibernate could use reflection to map its tables, stuff like that.