Party on the patio

Its 70 degrees in NY today, and sunny. I'm inside doing this. That's ok, though. I have a handful of things to get done in the near future, but that should leave the late Spring and Summer to be not doing this stuff.

I've been plugging away on the Groovy/Wicket implementation. There's still a ways to go with regards to making everything work as it should, but I've made significant progress.

• Dynamically generated classes are cached rather than re-generated each time.
• Models, and soon behaviors and validators, can be added by the builder in the same way as components, including overridden methods.
• Closure representation has been significantly cleaned up. Initially local references were banned, but there are certain situations which require them, so the wrappers attempt to handle that properly.

There's been a lot of other tweaks and additions. Right now I'm focused on getting the major stuff that you'd expect to work to actually work. I'm also hoping to build a reasonable example application and put that in the code base with everything else.

In the meantime, to better explain how the whole thing works, I recorded myself building a simple page. The quality isn't so hot, and my narration isn't the most exciting in the world, but I've got a few other things to get to today:

http://kgalligan.com/apache2-default/bigone.htm

At some point in the near future I'll try doing the same thing with a more complicated page.

Wicket and Groovy

I attempted to explain the concept of what I'm working on in a previous post, but its a little convoluted, and gets into implementation details, etc. Not very clear.

I'm getting ready to announce the project on a few mailing lists, so in preparation, I'm going to attempt to explain the idea again. Please bear with me...

Groovy, Grails, Rails, recent interviews and feeling a little like Old Gill...

I love Java. I've loved Java for a long time. I started coding Java in college, when it was version 1.0. Try this out. Its an asteroids clone I wrote. If you can dig up a Pentium 75, it will actually work.

For the past several years I've been running a small technology group building fairly heavyweight enterprise applications for a financial company. A relatively sheltered existence, to be sure. They move very slow on to new technologies. For my own part, I read a lot, but I'll admit I stuck pretty much to the Java realm. Maven, Hibernate, JSF? All over it. Ruby and Rails? Didn't even look.

So, I've been interviewing lately. It turns out that not everybody thinks Rails is mostly hype (as I did). Specifically, I spent a day at a company called Cyrus Innovation (cyrusinnovation.com). Its a small group of very intelligent people who actually believe in agile techniques, pair programming, do a lot of Java, and absolutely get into Ruby and Rails.

At 31, my interviewing experience left me feeling a little like Old Gill (simpsons reference). The prevailing attitude, at least in some areas, is that Java is old and Rails is the next big thing. That would be fine, except I really can't fathom tossing away such huge piles of quality, tested code that exist in the Java space. Also, Java is a great language for certain classes of applications. Part of the value of Rails comes from the language dynamics of Ruby, but a lot of the advertised productivity improvement comes simply from the framework. In short, there's no magic going on. Good ideas, but ideas are transferable.

It didn't take long to stumble onto Groovy and Grails. Its like an intellectual slap in the face for a long time Java programmer. Groovy is a dynamic language, syntactically similar to Java, which can seamlessly inter-operate with Java. Grails is a Rails-like framework built on top of Groovy. Assuming some time for stabilization and optimization, it levels the playing field in my mind. You can use the platform you're currently on, and extend it with a significantly more expressive language where needed. Not throwing the baby out with the bathwater.

Groovy and Grails are behind in terms of IDE support and general "smoothness" of development. I personally think Groovy is a fantastic technology, and my biggest concern is it simply won't get the time and effort needed to make it a first class option for the Java platform. It should, though. Java and Groovy together is a fantastic combination.

Wicket?

Right before I started with the dynamic languages, I had first tried out Wicket. For the past year or so I've been working with JSF. First on my own, then with my soon to be released major project. JSF is more productive than simpler options like Struts, and has some major industry support, but many things with JSF feel unnatural.

I had read about Wicket here and there, but had mentally stuffed it into the, "Oh man, another web framework" category (ie. the circular file). I finally gave it a shot, though. Yep, I was totally wrong. Wicket does some things in a way that seems, in retrospect, obvious. I think it has the freedom of not trying to be everything to everybody, and not having large committees making decisions, as JSF has to deal with. Its a great technology for the 98% of web applications that aren't be hit by thousands of users concurrently. Its a component framework, like JSF, which is nice for creating more complex user input constructs. Its page creation and linking features, however, are much better than Struts and JSF. By "much better", I mean respectively: Struts, JSF, Wicket -> walking, skateboard, ferrari. The way models can detach and reattach data objects is just awesome. I'll stop there. In summary, I like it.

So, now I'm all big on Groovy. What I disliked about Rails, and what I like about Groovy and Grails, is that you can use Java's existing code base in your new apps. Along with that, as Wicket really is great technology, I want to use it from Groovy code. Can you use Wicket and Groovy together? Yes, but its clunky. Why? Well, if you compile your Groovy into classes, it'll just work. That's good. However, Wicket relies on anonymous inner classes all over the place. You can create regular classes and override methods in them, but anonymous inner classes make that process much easier. Works great in Java. Groovy, however, doesn't do anonymous inner classes. Groovy uses closures, which are sweet as well, but you can't use closures with Wicket (obviously. If that wasn't obvious, you should probably stop now because you're going to be really lost soon). My goal with this project is to make Wicket programming with Groovy at least as easy as it is with Java, and hopefully easier. Besides, of course, the inherent expressiveness of the language itself.

Wicket Groovy Builder

Groovy has a concept called a builder. See here...

http://www-128.ibm.com/developerworks/java/library/j-pg04125/

The idea came to me to use a builder when I saw the SwingBuilder. It allows you to build a Swing app in groovy, with significantly simpler code. The Swing component structure is roughly tree shaped. Well, that's the first thing I noticed about Wicket code. It looked a lot like Swing code.

Simple concept. Write a builder implementation that build a Wicket component tree. Sounds easy, but its fairly complex. I won't go in to it, but lets say I do some things with closures that they weren't meant to do. Over the next couple weeks I'm going to be working on the project as well as the documentation, so hopefully I'll have some more detail on the implementation soon.

Originally I started doing this as a seperate project, but Eelco Hillenius from wicketstuff contacted me about the project. He had done some work on the 'wicket-groovy' project from wicketstuff, and proposed I move the builder code in there. Here's the website...

http://wicketstuff.org/confluence/display/STUFFWIKI/wicket-groovy

Right now my code is in the 1.3 branch only. You can see the latest subversion tree at...

https://wicket-stuff.svn.sourceforge.net/svnroot/wicket-stuff/branches/wicket-1.3/wicket-contrib-groovy

Does it help? Here are two screen shots of roughly equivalent code:

Java Version


Groovy Version


The larger the form, the more obvious the benefit.

Future Plans

The biggest thing is just using it in some applications, and writing some better test cases. I was banging my head against a wall today and yesterday over some pretty major stuff, so I'm sure there are a few other issues in there somewhere.

There are certain places where optimization would help significantly. Specifically on the dynamic class generation. Its generating a new class on each run. This isn't good. I have to sort out a problem with the Closure's owner, and then I can re-enable class caching. Today, though, a production implementation would probably run into trouble not too far down the road.

I haven't looked at the wicket-extensions at all.

I'm going to add GORM support from the Grails package. That is a really cool technology. Alternatively, we might be able to use Wicket as a render technology inside a grails app. Anybody know if this is possible? I have very little Grails experience.

Simplify the IDE and build procedure. This is one of those things that quickly turns off new users. Frustrating build experiences are bad.

Add some template applications. Set it up so getting off the ground takes little effort.

If anybody would like to pitch in, let me know.

Backwards Magic

Just spent a long, long time on this puppy. I couldn't get the overridden methods on a Form object to see a variable reference at the Page level from inside a closure.

class CompClosuresPage extends WebPage {

SomeData someData = new SomeData(name:"heyo", value:"someval")

public CompClosuresPage()
{
Closure testClosure = {
println someData.name
}

Class newFormClass = CompClosuresPageTestForm.generateClassInJava(testClosure)
Form form = newFormClass.getConstructor((Class [])[String.class]).newInstance((Object[])['baseForm'] )
form.setModel(new CompoundPropertyModel(someData))
form.onTestEvent()
}
}

Pretend 'onTestEvent' is like 'onSubmit'.

This fails. 'CompClosuresPageTestForm' is a test class I came up with that extends a Wicket Component of some type. If I remove that base class, the code above works.

Anyway, very long story short, If I changed the base class to a non-Wicket one, it worked. A Wicket one, it didn't. After a long walk all over the class hierarchy, I sorted it out. Inside wicket.Component:

/**
* Gets the component at the given path.
*
* @param path
* Path to component
* @return The component at the path
*/
Component get(final String path)
{
// Path to this component is an empty path
if (path.equals(""))
{
return this;
}
throw new IllegalArgumentException(
exceptionMessage("Component is not a container and so does not contain the path "
+ path));
}

It looks like Groovy is calling this all the time. I'm not sure what to do about it. At 'wicket.MarkupContainer', its final, so I can't override it and try to handle it gracefully. I can't really try to change the guts of groovy either. For today, if you want to access a class level property from an overridden closure, you'll have to call the getter method (implicitly or explicitly defined). The above closure would have to be written as follows:

Closure testClosure = {
println getSomeData().name
}

Ok, I spent a little more time on this. It is in fact the issue. The MetaClass picks up the 'get(String)' method as the general property accessor method. What I tried to do was put a delegating metaclass into the registry for that class type after its dynamically generated. Then, if the access fails, bubble that up and it should check the Page class. Its not working for some reason, though, and I've spent too much time on it today. I'll leave the MetaClass class in the code and get back to it later. I think without altering either groovy or wicket, that would be the way to go.

Wicket Groovy Builder finds a home

After a chat with Eelco Hillenius from wicketstuff.org, my new wicket groovy project is now integrating with wicket-groovy over at wicketstuff.org. The wiki page is at...

http://wicketstuff.org/confluence/display/STUFFWIKI/wicket-groovy

Putting it there makes a whole lot more sense than starting a new project.

I've converted the package name and committed the first version of the builder project to...

https://wicket-stuff.svn.sourceforge.net/svnroot/wicket-stuff/branches/wicket-1.3/wicket-contrib-groovy

The trunk has not been updated yet, as it works with wicket 2.0, and I haven't even looked at that code yet.

The original wicket-groovy project had one maven project with an example app mixed in. I've moved that into two maven projects:

1. jar: This is the java-only base package
2. webapp-example: This is an example webapp (as of today, I haven't added anything here)

Setting up the project and IDE can be a bit complex. I have a basic description on my site at:

http://www.kgalligan.com/wicketgroovy-setup

Wiki updates and mailing list announcements coming soon...

Roadblocks

Sometimes all the clever stuff in the world can't get you past something seemingly simple. I'm still plugging away on my groovy/wicket builder implementation. In order to emulate anonymous inner classes I pass a closure to the builder, with the same name as the overridden method. An example:

Java:

add(new Form("someForm"){
protected void onSubmit()
{
//Do something
}
}


Groovy:

form("someForm", onSubmit:
{
//Do something
}
)

Under the hood, I generate a new class that overrides the base class (in this case, wicket.markup.html.form.Form). The method is overridden. The overridden method calls the Closure. Complicated? Yeah, but functional.

Well, almost functional. There's one little thing (so far) that doesn't work. It looks like the following:

form("someForm", onSubmit:
{
super.onSubmit()
}
)

Since the class under the hood overrides 'onSubmit', you can't use 'this'. From what I can gather, there's no way to call 'super' through reflection. I tried several different things, but in the interest of getting on with life, I made a shortcut. Each method that is overridden will now have another method that looks as follows:

[access] [return] super_[method name]([params])
{
super.[method name]([params])
}

An example:

protected void super_onSubmit()
{
super.onSubmit()
}

Hey. It works.

I've been out for a few days. Checking with sourceforge again...

Groovy Wicket Continued

Ok. I wrote a setup guide for trying out the Groovy Wicket stuff. Take a look:

http://www.kgalligan.com/wicketgroovy-setup

Also added a simple example page tutorial:

http://www.kgalligan.com/wicketgroovy-simplepage

I'm still waiting for sourceforge approval. More to come...

Groovy Wicket

Intro

This is going to be a long post. The first part is a ramble and background info on dynamic languages and other stuff. If you want to read about the wicket/groovy thing, skip ahead. Also, I'm currently interviewing in NY. Shameless plug at the end...

Its been a strange time for me, technology wise. Well, all over the place, but certainly on the technical front. I'm about to go live with my big project of months past. Its based on JSF/Facelets and Hibernate. I've been a huge Java fan for quite some time, so new front end technologies, along with the solid ORM tool we've come to love seemed like a natural fit. I'd heard a little about these new fangled dynamic technologies, but it all sounded like a lot of hype to me.

About a month ago I spent a day interviewing at a company call Cyrus Innovation. This was a huge eye opener. They actual follow agile techniques, including actual pair programming all the time, which is something I've never seen. I've heard a lot about it, but in most cases, management didn't have the will to actually do it. I, as a manager, thought it was kind of a crazy concept, but having seen it, I think it works. I also learned that unit testing is actually important. Seeing full unit testing in practice makes that clear. I also saw a group of people who actually thought Ruby and Rails is a revolutionary platform. That, I just didn't get.

Its easy to call it hype, but the people at Cyrus are very intelligent, and they are very, very efficient, so hype would burn out pretty quick if it didn't deliver. So why were they so big on it?

See, here's the thing. I actually don't dislike Ruby or Rails. What I don't like is the concept of dumping all of your existing code (and all of the world's open source code, and the non-open source stuff, if you like that sort of thing). It seems a little drastic to just toss everything out. Right? I've been reading lots of blogs and opinions, and the pro-dynamic camp have the opinion that languages age and die out, and new ones come and take their place. This is logical, as its what's happened in the past. However, as you hear in the omnipresent financial services commercials, "past performance does not predict future results". I just think we have too much value in existing platforms to simply toss it out and start over. On top of that, I don't think dynamic languages serve all purposes, and I certainly don't agree with the productivity improvement claims in all cases.

So, with all that in mind, there are a few options:

JRuby would be one, but there's some work to get it functional with Rails (as far as I know). It also seems a little reactionary for my taste. Not that there's anything wrong with it. Jython is another entry into this category. Both have the issue of a foreign syntax, at least for a Java person. I also just get the impression that neither is really going to pull ahead.

You know where I'm going. Groovy rules. At least for the Java platform, it seems like the obvious choice. You could argue that it doesn't have all of the features of Ruby, or maybe isn't quite so "dynamic", but you'd really have to wow me with something to get me to think it would make a difference. Groovy is well suited for the JVM, and it only took a little while to love it. If you add on top of that Grails support for Rails-like ORM, but on top of, and compatible with, standard Hibernate, and I'm sold.

In fact, I think Java would be best served to be largely left alone, and have Java and Groovy co-exist. Java is a solid platform and well suited to more system level stuff. Groovy isn't as good on performance, but is easier to code and test with. You can call classes back and forth, seamlessly, and use either flavor of language as the situation dictated. I think Groovy is a fantastic technology and all I'm worried about is that it'll get buried industry momentum alone. (I also think the name should be changed. I was on an interview yesterday and mentioned it. Doesn't sound professional, and the reaction reflected that. Just my $.02)

Anyway, yeah, Groovy is good. Right before I got into Groovy, I started looking at a technology called Wicket. After working with JSF for almost a year, trying Wicket was like that movie scene where the clouds part and this big ray of light hits you in the face. I just had this feeling while JSF'ing that certain things were harder than they needed to be. Well, I was right, and the Wicket people figured it out.

Wicket and Groovy

Technically, you can use Wicket and Groovy with no changes. You can just compile your groovy classes into java classes, and that's it. There is, in fact, an existing library for wicket/groovy integration, but from what I can tell its just providing a classloader for groovy classes. If you're compiling, you don't really need it.

There is a problem, though. You *can* code Wicket pages with Groovy, but its painful. Groovy doesn't allow for inner classes, and certainly not the anonymous kind. Wicket can get by with name subclasses, but you'd be doing a whole lot of extra coding. Groovy is supposed to be cutting that down. So, I started looking for a way to smooth this out a bit. Since Groovy is dynamic, I figured there must be a way to do better.

I first tried to look at enhancing Wicket's library, like Groovy does with the GDK and standard Java library. I could add closure support to the standard extension points, and this would solve the problem for Groovy. Sounds nice, but there are a few problems:

1. The first critical issue; I don't think you can use the MetaClass functionality when calling from outside Groovy. At least not directly. This will take a Groovy expert to verify, but here's what I think is going on. If you call an object that is 100% Groovy from Java, it makes the call through the MetaClass proxy. If you call a Java class from Groovy, it'll use the global Registry MetaClass for that Java type. If you extend a Java class in Groovy, and call it from Groovy, that object will get its own MetaClass. However, if you extend a Java class in Groovy, and call it again from Java, you're calling the methods directly again. No MetaClass proxy. If that is not correct, let me know, but that's what it looked like to me. The rest of the app is coming from Wicket, which is coming from Java, and wouldn't have access to the dynamic features until you've completely crossed the blood/brain barrier (and you wouldn't have).
2. Even if you got around that, some Wicket classes are abstract. Simply intercepting calls and handed them to closures would be pointless. You'd have to implement a complex scheme to handle that.
3. Another issue that I had to deal with anyway, Closures aren't Serializable. Hanging on to a reference in your objects doesn't work with Wicket.

To put it lightly, the start was painful. So, I kept digging away, and pretty soon I stumbled on the answer. Builders.

It was really the SwingBuilder that tipped me off. One of the first things you notice when starting with Wicket, assuming you've had some Swing, is the similar object model relationship. Its tree-like. Lots of 'add(component)' calls. So, we could just build the tree, add closures where needed, and we'd be off. Sounds nice.

Pretty quickly I ran into problems #2 and #3 above, compounded by a frustrating attempt to use the MetaClass to do a lot of the work for me. In the end, here's what I came up with.

1. The name of each builder "function" node will correspond to a Wicket component. The builder will look through various packages for a component matching that name. A string is provided for the key. Pretty much every component needs one, so that was a no-brainer.
2. Most components correspond to the [String] and [String, IModel] constructors. Some, however, do not. It would've been easier to just figure out the whole model and implement what was needed for each component type that strayed from the standard, but we want to play nice and support sub-classes and all that. Plus, there are special circumstances where you can use shortcuts to save you time. So, I built a facility to register custom component builder factories. You register a factory, and that class and its sub-classes will attempt to use it. You can override it for a sub-class, so you're not locked in. Also, if you have a custom component library, you can add it to the configuration in your application start up.
3. The builder naturally follows the tree model, so in simpler cases the code cleans up significantly. As an example, the following Java:

item.add(new Label("id"));
item.add(new Label("name"));
item.add(new Link("setItem"){

@Override
public void onClick()
{
form.setPersistentObject(item.getModelObject());
}

});

turned into...

label("id")
label("name")
link("setItem", onClick:{ form.setPersistentObject(item.getModelObject()) })

4. The subclassing issue was ultimately handled with closures. How that works is a bit complicated, and to be perfectly honest, I'm not sure if the solution is elegant and ugly. But it works.

In a nutshell, that's it. Keep your builder names the same as the component class names, and keep the component reference page (not yet created) handy, and you should be ok. This project is still FAR from prime time, but I think if the kinks can be worked out, it'll make Wicket development on Groovy much more compelling. At least I'm going to use it ;)

Something very important to keep in mind. The syntax looks like I'm substituting Java for a whole different language, and would be rigid and cause problems as you go forward. However, inside the builder you can use regular Java/Groovy code, and directly create Components with regular constructors, and add them manually. All of this while still in the builder tree. Because of this, you get the 80%-90% easy case with the builder, and can still seamlessly do the painful 10% that doesn't fit right.

An Example Page

Let's take a quick look. I made a demo application with a very simple data object, using databinder and hibernate. This should give you a quick visual overview on what the wicket/groovy builder can do. I'm using screen shots because posting code in the blog was difficult. Plus, this gives you a good idea of what it'll look like in the editor (which is where you'll see it anyway, right?)

Java Version


Groovy Version


If you take a look you'll notice the Groovy file looks similar, but just has less clutter. That, in a nutshell, is a lot of what the dynamic language stuff is about. I didn't "get it" for a long time, but I do now. The great thing about Groovy is that its type optional, so if you want your types in there, you can have them. Now all we need is some better IDE support, and I think Groovy will be (groovy).

Details

If you know a lot about Wicket and/or Groovy, and you have some time, I'd like to lay out the details of my implementation. Or, at least start. Hopefully I can get some feedback as I'm a beginner in both technologies. Also, I started this on Saturday, and I had been out pretty late the night before. The code is a little on the messy side right now, but I think the general design is functional.

Here's how things work. We match up components based on name. Right now the matching facility is pretty flat. It looks through the configured packages for the name, and uses the first one found. After that, the name is cached. It will also look in the same package as the page. This allows you to override a class right in your groovy file and use it. The configuration needs to be more robust. Maybe you only want certain parts of the app to use certain figurations? For today, there are bigger problems, but this is on the list.

WicketBuilder builder = new WicketBuilder(page)
builder.label("pageTitleHeader", model:new PropertyModel(this, "name"))

The first line creates the builder. You supply a context argument. The second line is the root level call, to 'label' in this case. It does a simple match to the Wicket 'Label' component. Next is the list of arguments to 'label'. The first is the main one, which is the Wicket key value. After that, you supply [name:value] pairs. In this case, 'model' is the name, and a new 'PropertyModel' object is the value. Anyway...

So, you've found the component you want. Now you need to look in the configuration for a ComponentBuilder factory for that particular component class. The structure is in a class called 'ClassHierarchyTree'. I'm sure there are plenty of similar implementations, but I wanted something quick and dirty. It works like a tree map. You can add Class definitions as key values, and associate them with data objects. When you go to look something up, you pass it a Class object. It'll pull the data associated with that Class, or the closest superclass. Pretty simple.

Ok, so you have a factory class. This factory will get the attributes you set for the object and use them to initialize an instance. If you are missing something, it should throw an exception and bomb out with a reasonable message (cleaning the messages up is a todo). In the example above, the factory will call the standard constructor for Label with the String key, then call 'setModel' with the model argument.

Things get interesting when you supply a closure as an argument:

link("setItem", onClick:{ form.setPersistentObject(item.getModelObject()) })

Here, 'link' matches with the Link component. 'setItem' is the key, similar to the Label above. However, 'onClick' has a closure as an argument. What happens now is the factory class builds a list of all closure arguments, then actually generates a new sub-class of the component. This sub-class will override all relevant methods, and delegate to the closure at runtime. If a method isn't found for the named closure, it will throw an exception and the page will fail (say, if we'd typed 'inClick' instead of 'onClick').

Rigth now there is no caching going on, so each time this happens a new class is generated dynamically. It works, but we could cache classes with the same signature. That's on the todo list.

My big concern is how I'm handling closures. Closure instances aren't Serializable, so keeping them in Components doesn't work. However, Closures correspond to Classes under the hood. So, I actually stuff a reference to the Class representing the closure into the Component instance. It seems to work ok. There are some issues I had to deal with. First, local variable references would be worthless. Luckily, the Closure has a different constructor in these situations. If local references are found when the Closure class is set, it will fail with a useful message.

So, anyway, sub-classes. The beauty is that abstract classes get concrete implementations as a result. It all sort of works. I'm sure there's a big issue in there somewhere, but tonight I feel pretty good about the whole thing.

I'm going to upload a zip of the code for now in case you'd like to take a peek. Hopefully the sourceforge project will be up soon, but in the meantime, this should do.

Wrap Up

If you're still with me here, congrats. Here's my immediate plan. I've posted a sourceforge project request. When that comes back, I'll add everything there. I'm working on a big project right now that I've been mildly neglecting to get this functional, but will start using this on that project. Just the back end pages, of course. I'd like to find somebody with serious Groovy and/or Wicket understanding who would be interested in taking a look and seeing if there's anything glaringly horrible going on. If anybody wants to help, obviously let me know.

Off the top of my head the todo's are as follows:

1. Stabilize configuration and come up with a decent demo app
2. Do some real unit tests (there are some, but they're pretty light)
3. Explore integrating GORM from Grails, or the other way, using this as an alternate view layer in grails. I have very little grails understanding, although the GORM stuff is pretty sweet, and I'd love to hook it into this for an easy win
4. Make sure there aren't big issues with the way Closures and dynamic classes are being generated. I'm not sure how a clustered environment would react
5. Implement the extension stuff. Right now its just the standards and a little of databinder
6. Cache generated classes
7. Build some simple CRUD generation tools. Similar to Grails scaffolding, but far less robust. Just point at a domain class and build a list display and edit form

That's it for now. I'm wiped.

Employment

If you're still with me, I'd like to put out a call for employment opportunity. I've been taking my time finding my next career step. Its been a couple months of interviewing now and I'd like to make a decision soon. See my page for resume, other work, etc.

More to come...