XHAB: Xavier Hanin's Blog

Play! framework Flex module development

2010-10-16T22:14:00.005+02:00

Playing around

I've been recently starrting to evaluate Play! to use as a basis for future developments at 4SH. What I've seen so far is really pleasant, I like the productivity you get, and especially the hot reload experience: I'm used to develop in Java with pretty good hot reload capabilities, but with Play! I almost never have to restart my server, even when I touch my JPA entities to add or remove properties. It's really a pleasure. Moreover I really like the architecture, the concepts of plugins and class enhancers opens a set of interesting possibilities without relying on a complex build mechanism which causes problems in your IDE (I've experienced that with AspectJ): in playframework the complex build is part of the runtime, so you can just rely on the framework whatever you use for development.

My use case: calling methods in my Play app classes from a Flex client

While conducting my evaluation I had to test how good I could integrate a Flex client with the Play backend, partly because some customers request a shiny Flex UI, and partly because I really like the productivity we have with Flex. It's also a good way to see how difficult it is to go outside the Play! box...
My intention in this module is to allow to perform remote calls from the flex application to public static methods on my Play classes (something similar to action methods). Indeed I have like the domain driven approach you can take with Play!, and thus I want to be able to perform remote calls directly on that static methods we get for free on the domain classes.

First steps

So my first step was to investigate how Flex performs its remote invocation, having a look at BlazeDS classes really helped for that. Integrating this in Play is not too difficult, thanks to the very high level hook plugins provide: rawInvocation is called before the HTTP request is route, so you can do whatever you want here without going through the whole action routing. It's nice to have this kind of very low level hook. With this approach I had something working in less than a day, not too bad considering that I was just beginning with Play. I even quickly hacked something to make the serialization understand the JPA lazy initialization so that it doesn't serialize collections which have not been fetched, so you can easily control what get serialized to the client, without requiring copies to DTOs.
From a Play! perspective, what I was doing was just a plugin, that I was putting in the classpath of my application (with a mere project dependency in eclipse). So my contract was almost fulfilled: calling methods was possible, but still I needed to write all the client side code, including kind of duplicating the code of my domain classes to convert them in ActionScript3...

Improving productivity: generating client side code

Writing the client side code manually when you have such a productive framework on the server side is just too painful. Moreover FlashBuilder 4, the tool Adobe offers for Flex development, has very good code generation features when you use BlazeDS on the server side, so it was not acceptable to drop this productivity boost on the client just for better productivity on the server... So I first tried to see how FlashBuilder gets meta information from BlazeDS to generate the code, but unfortunately it relies on the server side on the configuration of a Servlet (flex.rds.server.servlet.FrontEndServlet) which is provided in BlazeDS binaries but not in BlazeDS sources :(
Therefore I considered using the code generation tool that GraniteDS provides: Gas3. But while commuting to work friday morning on my bike, I thought that Play! already offers a pretty easy way to generate source code: its regular template mechanism to serve HTTP requests. The advantage is that it's fully integrated, and I thought it was going to be very fast to develop: a controller with 3 actions: one to give the list of classes from which generation is possible, one to generate the actionscript to represent a value object (based on my entities on java side) and one to generate the service (on client side I prefer to distinguish classes to represent the remote objects from the value objects). With that you can simply copy / paste the generated code (that you get in the browser).
When I started to further read the document on how to do that from py plugin, I realized that what I needed was a Play! module rather than a mere plugin. Indeed Guillaume Bort is clear about the difference:

A plugin is a java class that extends PlayPlugin. A module is an application fragment that can contain anything like any application. And of course it can contain some plugin too.

Enter module development

So I first searched for documentation on how to write a module, and the main source of information is this page in the documentation. Ok, so here the documentation is mixing information on what you can do inside a module and what you need to do to install a module in your app. I tihnk for some people it may be confusing, most users only need to use a module, and actually most of that page describes this use case.
Still based on that doc I could quickly start up with the play new-module command, which gives you the structure to get started, nice! But quickly I had trouble figuring out where my classes should go, and I think the documentation is not clear on that subject. From what I've understood, it's important to understand that in the module you need to distinguish between different kind of classes:

Plugin and utility classes

src

play build-module

ant

lib

Play classes and views of your module

app

documentation on the subject

Once I had properly understood that, I was ready to actually develop my module controller to generate my source code. I still had some minor issues due to my inexperience with Play (like an OutOfMemory error after an exctract method in my controller class, because I moved a recursive public static void to my controller, which was considered by Play as an action chaining... making it private was enough to fix the problem). Still the code generation pages were ready pretty fast.

Still not enough productivity: enter commands

Once I got my code generation working, I quickly felt that the copy/paste from browser to sources was going to be annoying. So I decided to go further in the automation and try to provide commands to generate the code directly in the proper files. After all Play! provides a skeleton of commands.py in the module skeleton provided by play new-module, so adding a command which more or less performs a wget shouldn't be too complicated.
It shouldn't, but I underestimated the time I had to spend. First of all, it was my very first time that I wrote Python. With my 15 years of development I'm not afraid by this kind of challenge, still my productivity in this case is not good, and sometimes the spaces dependent interpretation of python was making me remember my very old days of COBOL in 1994. Don't get me wrong, python seems very productive when you're used to it, it's just far enough from my mostly Java background to get me fall in every possible trap.
Moreover I had some troubles to get my commands integrate in play. When I tried my first command with confidence I got that:

sample1-client $ play flex:gen
~        _            _ 
~  _ __ | | __ _ _  _| |
~ | '_ \| |/ _' | || |_|
~ |  __/|_|\____|\__ (_)
~ |_|            |__/   
~
~ play! 1.1RC1-76-g2e8d571, http://www.playframework.org
~
~ Usage: play cmd [app_path] [--options]
~ 
~ with,  new      Create a new application
~        run      Run the application in the current shell
~        help     Show play help
~
~ Invalid command: flex:gen
~

Invalid command, mmm, it seems my module is not properly recognized...

sample1 $ play modules
~        _            _ 
~  _ __ | | __ _ _  _| |
~ | '_ \| |/ _' | || |_|
~ |  __/|_|\____|\__ (_)
~ |_|            |__/   
~
~ play! 1.1RC1-76-g2e8d571, http://www.playframework.org
~
~ Application modules are:
~ 
~ /Users/xavierhanin/dev/tools/play/play-trunk/modules/flex
~

Mmm, it's there, so what's going wrong? I double checked my command declaration in my commands.py, googled for this kind of problem, even started to investigate on how the play command line mechanism is working, but with my poor knowledge of python I didn't even know how to enter in step by step debug mode. I decided to revert back to the initially generated commands.py, and then:

sample1 $ play flex:hello
~        _            _ 
~  _ __ | | __ _ _  _| |
~ | '_ \| |/ _' | || |_|
~ |  __/|_|\____|\__ (_)
~ |_|            |__/   
~
~ play! 1.1RC1-76-g2e8d571, http://www.playframework.org
~
~ hello
~

Ok, so the problem is actually in my commands.py when I add my new command. Fortunately at a point in time I just wanted to generate my eclipse settings and then did something like this:

sample1 $ play eclipsify
~        _            _ 
~  _ __ | | __ _ _  _| |
~ | '_ \| |/ _' | || |_|
~ |  __/|_|\____|\__ (_)
~ |_|            |__/   
~
~ play! 1.1RC1-76-g2e8d571, http://www.playframework.org
~
Traceback (most recent call last):
  File "/Users/xavierhanin/dev/tools/play/play-trunk/play", line 111, in 
    cmdloader.commands[play_command].execute(command=play_command, app=play_app, args=remaining_args, env=play_env, cmdloader=cmdloader)
  File "/Users/xavierhanin/dev/tools/play/play-trunk/framework/pym/play/commands/eclipse.py", line 121, in execute
    execfile(commands)
  File "/Users/xavierhanin/dev/tools/play/play-trunk/modules/flex/commands.py", line 46
    for c in entities.splitlines():
                                  ^
IndentationError: unindent does not match any outer indentation level

OK, now I get enough details on the compilation error in my commands.py! With that in my mind, every time I got a Invalid command message I return to play eclipsify to get the details, and it's good enough to fix the problem. I still had to fight a couple of times against my habits to get the code working, but now I have my code generation working like a charm, being able to generate the classes I want, with patterns to select which one like this:

sample1-client $ play flex:gen --class models.**
~        _            _ 
~  _ __ | | __ _ _  _| |
~ | '_ \| |/ _' | || |_|
~ |  __/|_|\____|\__ (_)
~ |_|            |__/   
~
~ play! 1.1RC1-76-g2e8d571, http://www.playframework.org
~
~ models.Comment entity generated in src/models/Comment.as
~ models.User entity generated in src/models/User.as
~ models.details.UserDetails entity generated in src/models/details/UserDetails.as
~ models.details.UserDetails entity delegate generated in src/models/details/UserDetailsDelegate.as
~ models.Post entity generated in src/models/Post.as
~ models.User service generated in src/services/UserService.as
~ models.details.UserDetails service generated in src/services/details/UserDetailsService.as
~ models.Post service generated in src/services/PostService.as

I'm pretty happy with that, even though I still have a lot of work to do on the doc, error management, security and yet improve productivity (to avoid a security black hole I force to add @RemotingInclude on every method to expose, but then it's difficult to expose methods provided by Play like findAll and sisters). But my conclusion is that my overall feeling with Play! is very good, the module development part is not very well documented but that's why I wrote this blog entry, hoping it could help others.
Have fun!

ForEach / Generics java puzzler

2009-11-18T12:37:00.005+01:00

While assisting at Mark Reinhold talk on JDK 7 updates at devoxx, it reminded me a problem we had very recently in a project.

Here is a small piece of java code:

import java.util.ArrayList;
import java.util.List;


public class TestForEachLoop {
 public static void main(String[] args) {
  List<String> listOfStrings = new ArrayList<String>();
  
  doSomethingWithListInPreJava5(listOfStrings);
  for (Object o : listOfStrings) {
   System.out.println(o);
  }
 }

 private static void doSomethingWithListInPreJava5(List list) {
  list.add(new Integer(12));
 }
}

So, do you have an idea of what will happen if I compile and run this piece of code? I'll let you comment if you have an idea, and I promise the final answer is not obvious!

Strange NPE in jboss 4.2.2

2009-11-18T10:51:00.002+01:00

A customer for which I work currently recently reported an error when deploying a new version of the application we just delivered to them. A jboss MBean (annotated with @Service) wasn't starting successfully, with an NPE in the logs in jboss code:

java.lang.NullPointerException
at org.jboss.ejb3.service.ServiceContainer.start(ServiceContainer.java:179)

Googling for this, I didn't get much clue on what was going wrong. It seems nobody reported such problem before, that's why I thought it may be worth sharing the conclusion. Digging into the logs and in the jboss configuration setup by my customer, I found this:

TIMER SERVICE IS NOT INSTALLED

looking at their ejb-deployer.xml, I realized they had commented out the timer mbean (org.jboss.ejb.txtimer.EJBTimerServiceImpl). What's really strange is that enabling this again fixed the NPE problem, at least in our case. So in case you met such an issue yourself, check that first, it may save you a couple of hours.

Maven leadership in question

2009-04-23T04:05:00.003+02:00

While reading my feeds tonight I stumbled upon a post by Nicolas Deloof: OPA hostile sur Maven, that I would translate with the help of google "Hostile takeover on Maven".

The content of this post is interesting, I've already heard people complaining about the way the maven project is lead which doesn't seem to follow the "Apache way". I'm surprised that it comes to an extremity where a Maven committer ends up a blog post with a sentence like "At this rate I'll end up back to the good old Ant and his buddy Ivy.". Maybe it's a good time to switch to Maven alternatives like EasyAnt or Gradle :-)

Apache Ivy 2.0 final is out!

2009-01-25T19:17:00.004+01:00

I'm very pleased to relay the announcement made on the Ivy mailing list: Ivy 2.0 is finally out! After a little more than 2 years since the last official stable release (1.4.1), Ivy has gone a long journey toward this 2.0 release: it started incubation at the Apache Software foundation in late 2006, it graduated as a sub project of Ant in october 2007. In the mean time, we produced 2 alpha releases, 2 beta releases, and 2 release candidates before finally releasing 2.0.0.

This is definitely a major release for Ivy, for several reasons:

This is the first final stable release of Ivy as an Apache project
The code base has been fully reworked to make it more modular, easier to understand and to maintain
Maven 2 compatibility has never been as good, Ivy is used by many projects to leverage public maven 2 repositories
As often requested by users, cache management is now much more flexible, allowing to address complex development and integration environments
More flexible dependency management, with module wide exclude, transitive dependency version overriding, dynamic resolve mode, ...
A new packager resolver, which allow to use a repository declaring how to package modules instead of storing them, used in Ivy roundup repository
Ivy core has still no dependency at all, now even for command line use. So using Ivy can be as simple as java -jar ivy.jar. Ivy is now also an osgi bundle, which adds one more way to use it.
IvyDE, the Ivy eclipse plugin, has also been improved a lot. A Netbeans plugin and an IDEA plugin are also available
A growing community, with 4 committers, 70 contributors and active mailing lists
As always, documentation has been fully updated with all new features and improvements (the printer friendly version counts now more than 180 pages).
220 bug fixes, 84 improvements and 24 new features since 1.4.1!

Here are the stats of number of issues per release:

So if you haven't tried Ivy yet, go ahead, it's hot! And if you already use it, upgrade now, this release is mostly backward compatible with previous ones, and has been extensively tested!

Strange RuntimeException with Seam on JBoss AS

2009-01-22T22:18:00.003+01:00

I'm currently setting up a seam application, and I ran into a strange problem recently:

--- MBEANS THAT ARE THE ROOT CAUSE OF THE PROBLEM ---
ObjectName: jboss.j2ee:service=EJB3,module=jboss-seam.jar
State: FAILED
Reason: java.lang.RuntimeException: You did not specify a @Resource.mappedName() on javax.ejb.TimerService org.jboss.seam.async.TimerServiceDispatcher.timerService and there is no binding for enc name env/org.jboss.seam.async.TimerServiceDispatcher/timerService in XML

ObjectName: jboss.j2ee:service=EJB3,module=app.jar
State: FAILED
Reason: java.lang.RuntimeException: @javax.annotation.PostConstruct annotated method has the wrong signature - public void org.jboss.seam.intercept.SessionBeanInterceptor.postConstruct(javax.interceptor.InvocationContext)

ObjectName: jboss.j2ee:module=jboss-seam.jar,uid=33039820,service=EJB3
State: FAILED
Reason: java.lang.RuntimeException: You did not specify a @Resource.mappedName() on javax.ejb.TimerService org.jboss.seam.async.TimerServiceDispatcher.timerService and there is no binding for enc name env/org.jboss.seam.async.TimerServiceDispatcher/timerService in XML

Googling around didn't really help, so I had to figure out myself what I did wrong. Actually I messed up my ear lib directory, and had the ejb-api.jar there. Obviously this jar shouldn't go in the ear lib directory, since it's provided by the server.
Unfortunately the error message doesn't really help to understand what can be wrong, so I thought posting this may help other people facing a similar situation.

Apache Ivy 2.0.0-RC2

2008-11-06T08:32:00.003+01:00

Maarten has just posted the announcement for Ivy 2.0-rc2. We are getting very close to 2.0.0 final, should be a matter of a couple of weeks now!

Here is the announcement :

Nov 4, 2008 - The Apache Ivy project is pleased to announce the
release of Ivy 2.0.0-rc2, the second release candidate for Ivy 2.0.0.

Ivy is a tool for managing (recording, tracking, resolving and
reporting) project dependencies, characterized by flexibility,
configurability, and tight integration with Apache Ant.

This is a release candidate for 2.0.0 final, meaning that no changes
except bug fixes will occur between this release candidate and 2.0.0 final.
Problems found at this phase can be fixed in the final release, so now
is a good time to download and use it. If no outstanding bugs are reported
in the coming weeks, this release candidate will be promoted as the 2.0.0
final release.

Key changes in this 2.0.0-rc2 version are
* enhanced Maven2 compatibility, with several bug fixes
* 20+ bug fixes as documented in Jira and in the release notes

Issues should be reported to:
https://issues.apache.org/jira/browse/IVY

Download the 2.0.0-rc2 release files at:
http://ant.apache.org/ivy/download.cgi

More information can be found on the Ivy website:
http://ant.apache.org/ivy/

Regards,
Maarten Coene (2.0.0-rc2 release manager)

A gasless week

2008-06-24T06:25:00.002+02:00

Last tuesday morning after a lengthy and opinionated discussion with friends about global warming, I decided to go to work by bike. I mostly work at less than 10kms away from home, so biking there takes less than half an hour.

But why stop with one day... So the day after I used my bike again, to go to work, then to go climbing, then go visit a friend, and then come back home at 1am. About 40 kms in a day, not that hard after all.

Then I had two days working at home, avoiding to use my scooter was easy :-)

Friday night I went to a party with friends from tennis club. 8kms each way, by bike again. Saturday I went to a friend's birthday party, only 4kms, but I had to bring charcoal for the BBQ and paper tabblecloth... I used my bike again, keeping balance more or less easily. Going back home was pretty funny, a storm started just when I left, so I was fully soaking wet when I get back home. Fortunately human body is waterproof :-)

Sunday I went to a volley ball tournament, and it was only 1km away from my home, so I enjoyed using my bike to go there. And yesterday I worked at home, fulfilling my gasless week. Today I plan to go to work with my bike again, it's a small contribution to decreasing the carbon dioxyd emission, but it's better than nothing. And you, what do you do for the earth?

SpringSource Repository

2008-05-07T10:05:00.002+02:00

A new public java module repository has just come into the dance: the spring source repository. This repository is both a Maven repository and an Ivy repository. It means that it contains metadata not only in maven 2 pom format (which BTW Ivy is able to recognize) but also in Ivy native format: Ivy files.

The modules contained are not plain open source jars, most of them have been repackaged in OSGi bundles (all that were not already). I'm not sure about what this will mean for the future of public repositories, but it's great to see that spring source continue its commitment to support both Maven and Ivy.

Gradle, a new Groovy based build system

2008-04-24T20:02:00.002+02:00

Hans Dockter recently announced the availability of first public version of Gradle, a new build system leveraging Grovvy, Ant and ... Ivy!

I won't comment too much on the tool yet, I've only read a few pages out of the 50+ pages the user guide counts.

What really pleased me while reading the comments on the server side announcement, is the praise made to Ivy:

And I must say I'm absolutely thrilled with Gradle's choice for Ivy integration, which is a gem in its own right.

I was solely using Maven2 for years. I just knew that Ivy existed and I thought it plays more or less in the same league as Maven's dependency management. I couldn't have been more wrong.

I completely agree that Ivy is a true gem. Starting from the quality of the code base to the fantastic solution of the problem space of dependency management. And its getting better and better. If I had to choose now between Ant & Ivy or Maven, my decision would be very clear. Well, but now there is Gradle :) It adds one missing piece to Ivy. A build system that offers a build-by-convention integration for it.

It's nice to hear this kind of thing, it's one of the best reward of the time and energy involved in this project.

IvyBeans wins Netbeans innovator grant!

2008-04-17T19:50:00.004+02:00

Winners of Netbeans innovator grants have just been announced, and IvyBeans, a project submitted by Laurent Foret with my collaboration is amoung the 10 large projects selected!

This is a very good news both for the Ivy community and the Netbeans community. I can't tell too much about the content of this project yet, but you already have good hints of what it is about.

Update: The project is now started, hosted at google code: http://code.google.com/p/ivybeans/

Ivy ramblings

2008-04-17T16:41:00.003+02:00

Today I spent some time setting up and developing a project for a customer, which involved some Ivy tricks for dependency management. I thought sharing this might interest some Ivy users or contemplators ;-)

First my Ivy settings:

<ivysettings>
 <settings defaultResolver="default" />
 <include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
 <include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
 <resolvers>
           <chain name="public">
         <!-- official maven2 repo -->
  <ibiblio name="maven2" m2compatible="true" />
         <!-- apache incubating maven2 repo, used for CXF -->
  <ibiblio name="apache-incubating" m2compatible="true" 
            root="http://people.apache.org/repo/m2-incubating-repository" />
         <!-- jboss maven2 repo, used for jbpm -->
  <ibiblio name="jboss" m2compatible="true" 
            root="http://repository.jboss.org/maven2/" />
         <!-- some jbpm modules seem to be missing from jboss repo, here is one where we can get what we need -->
  <ibiblio name="elca-services" m2compatible="true" 
            root="http://el4.elca-services.ch/el4j/maven2repository/"/>
  <!-- java.net repo, containing useful java apis -->
  <ibiblio name="java.net" root="http://download.java.net/maven/1/" 
                            pattern="[organisation]/[type]s/[artifact]-[revision].[ext]"/>
  <!-- restlet maven 2 repo -->   
  <ibiblio name="restlet" m2compatible="true" root="http://maven.restlet.org/" />
    </chain>
    <!-- eventbus is not available on any repo, we access it directly from its web site -->
    <url name="eventbus">
   <artifact pattern="https://eventbus.dev.java.net/files/documents/4303/75132/EventBus-[revision].jar" />
    </url>   
 </resolvers>
 <include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
 <include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
 <modules>
   <!-- use eventbus repo for eventbus only -->
   <module organisation="net.java.eventbus" name="eventbus" resolver="eventbus"/>
   <!-- use shared resolver for all modules developed internally, to improve perf -->
   <module organisation="com.foobar" name="*" resolver="shared"/>
 </modules>
</ivysettings>

These settings are based on default Ivy settings, as such I follow the recommendations given here.

Besides the chain of public maven repos I use for the set of tools used in this project (ranging from hibernate to spring, jbpm, groovy, restlet, flex...), there is one very specific trick used for eventbus. Eventbus doesn't seem to be available on any public repo, but its jar is available on its java.net home page. Thanks to Ivy flexibility, I can use this location as a "repository". But this repository will "find" almost any module, because there is no [module] or [artifact] tokens in the pattern. So I must be careful and put this resolver outside of the default chain resolver. Hence this resolver will never be used, except when I instruct Ivy to do so. That's exactly what I do with the module line, telling to use this resolver for modules whose name is "eventbus" and org is "java.net.eventbus".

Then in my Ivy file I can declare my dependency as if it where available like any other module:

<dependency org="net.java.eventbus" name="eventbus" rev="1.1" />

The other trick I use is to tell Ivy to use the shared resolver for all modules from "com.foobar". This trick is very useful to improve performance, it tells Ivy to look for the modules I develop for my customer in the shared resolver only, avoiding the latency of the public repo for these modules.

Ok, that's it for the settings. Now here is a very specific conflict management issue I had to deal with. As I said earlier I use both groovy and hibernate in my project. So here's my pretty simple first attempt to get the two dependencies in my module:

<dependency org="org.codehaus.groovy" name="groovy" rev="1.5.4" />
<dependency org="org.hibernate" name="hibernate" rev="3.2.6.ga" />

The problem is that groovy 1.5.4 depends on asm 2.2, and hibernate 3.2.6.ga depends on cglib 2.1_3, which in turn depends on asm 1.5.3. Ivy resolves the conflict automatically according to my settings, and use the latest version between the two: 2.2. The problem is that I then have a classloader issue at runtime: java.lang.NoClassDefFoundError: org/objectweb/asm/CodeVisitor

A quick search on the net and I find this solution:

The easiest solution I’ve found is to remove the CGLib and ASM (1.5.3), which come with Hibernate and instead replace them with a copy of cglib-nodep.

Allright, let's convert this in instructions for Ivy. First, we need to exclude the dependency on cglib that you have in hibernate metadata:

<dependency org="org.hibernate" name="hibernate" rev="3.2.6.ga">
    <exclude module="cglib"/> <!-- we use cglib-nodep instead, to fix a conflict with groovy asm usage -->
</dependency>

Then I add the dependency on cglib nodep instead:

<!-- this is to replace the regular dependency of hibernate on regular cglib -->
<dependency org="cglib" name="cglib-nodep" rev="2.1_3" />

I'm not really happy with this, because I have to add a dependency cglib-nodep, while my module doesn't depend on cglib-nodep. But Ivy conflict management engine can't really help since I want to replace one module (cglib) by another (cglib-nodep). But I'll come back to this later.

Let's run my test again... allright, it seems to work properly! Let's have a look at the dependencies and the version of asm I now get in my classpath:

asm-2.2.jar
asm-util-2.2.jar
asm-tree-2.2.jar
asm-analysis-2.2.jar
asm-attrs-1.5.3.jar

Mmm, I don't like this: I have a mix of versions of asm modules. Checking Ivy report, I clearly see why: hibernate has a direct dependency on asm-attrs 1.5.3, while groovy doesn't depend at all on asm-attrs. Thus there is no conflict on this module, and Ivy selects the only revision of this module that was requested. Too bad in this particular case, I'd prefer to have all asm modules aligned in version.
With the override mechanism introduced recently, it's easy:

<override org="asm" rev="2.2" />

This is enough to tell Ivy to use revision 2.2 of all modules from asm organization, whatever is actually requested. Since I don't use Ivy trunk version in this project, I had to circumvent the problem this way:

<dependency org="org.hibernate" name="hibernate" rev="3.2.6.ga">
 <exclude module="cglib"/> <!-- we use cglib-nodep instead, to fix a conflict with groovy asm usage -->
 <exclude module="asm.*" matcher="regexp" /> <!-- we want to take care of asm dependency ourself -->
</dependency>
<!-- this is to replace the regular dependency of hibernate on regular cglib -->
<dependency org="cglib" name="cglib-nodep" rev="2.1_3" />
<!-- this would better be handled by new override mechanism -->
<dependency org="asm" name="asm-attrs" rev="2.2" />

Ok, so now it works properly and I have aligned versions for all asm jars. That's nice, but it was a bit cumbersome, and my metadata is cluttered with bad information: I have declared a dependency on cglib-nodep that I don't really have (same for asm, but with trunk version this would be avoided). Maybe we could find a way to improve the metadata or conflict management to be able deal with this kind of case... But the real problem is that these "modules" don't match the concept of module in Ivy:

A module in ivy is a piece of software that is reusable, and that follow a unique cycle of revision.

In this case, all asm artifacts (attrs, tree, ...) would better fit in a single module in Ivy, with a configuration for each artifact (if it makes sense, I don't know asm enough). The same applies for cglib and cglib-nodep: it's actually a single module, it's just different packaging and usage, hence different configurations in Ivy.
So with actual Ivy metadata, getting asm-attrs in one version and asm-tree in another would almost be impossible (well, you'd need to use some Ivy tricks to get them if you really want to). And the problem with cglib could be handled with the conflict management or override system. Metadata would be kept consistent, and easier to maintain. But unfortunately I have to use maven 2 repos for this customer, and there is no such clean Ivy metadata... yet.

A day for Ivy development

2008-04-03T20:59:00.002+02:00

Yesterday my main customer cancelled a meeting which was planned for today, so I decided to take advantage of this free day to fix some of the open issues targeted for Ivy 2.0 final.

I started with the final touch to the implementation of transitive override support (IVY-784). This will allow users to override the dependency constraint declared by a transitive dependency. This complement the already powerful support for transitive excludes, and fine grain conflict manager configuration. Together, those features are a very nice way to deal with bad metadata over which you don't have control. Actually, this implementation was required due to a change in dependency management handling introduced in maven 2.0.6: now the dependencyManagement section override transitive dependencies versions, so we had to implement this as well as part of our maven compatibility feature.

Then I worked on IVY-297, improving name consistency and readibility, deprecating allownomd="true | false" in favour of descriptor="required | optional" on resolvers. I also updated buildlist to replace skipbuildwithoutivy="true | false" by onMissingDescriptor="head | tail | skip | warn | fail", which is both more readable and give more control about what to do when a module is missing a module descriptor.

The next job consisted in fixing an issue with properties and included files: properties used to be local to the settings file in which they were defined. Therefore it was not possible to define properties in an included file and use them in the including file.

Later I fixed IVY-777, which required to introduce a new interface: Validatable. Now any Ivy plugin can be validated at the end of settings parsing, simply by implementing this interface. It's the case for dependency resolvers, which now checks that the configured cache manager, latest strategy and name space actually exist.

The next issue was a maven 2 compatibility issue: supporting SNAPSHOT versions used to be rather limited in Ivy: with no specific handling, Ivy didn't know how to handle SNAPSHOT versions with a unique revision (where a timestamp and build number is used) and didn't try to update the revision when you don't use unique revision. This is now implemented, relying on maven-metadata.xml to get the required information.

To end up my day, I resolved some easy to fix issues, like IVY-579, IVY-592, IVY-437, IVY-722 and IVY-774.

So it's been a pretty busy day, but we now have 12 less issues on our road map toward Ivy 2.0!

New Maven 2 repository search site

2008-03-12T17:45:00.003+01:00

mvnrepository.com has been unavailable for a few days, so I finally decided to quickly develop a new site, providing only one very basic but very useful feature: maven 2 repository search.

The site is available here:
http://javarepo.xoocode.org/

Nothing terrible here, just a very simple web interface. The advantage over mvnrepository.com:

it is available now

I'm just kidding, it may be down in a couple of minute, and I guess mvnrepository will be back pretty soon

More Ivy friendly

JavaRepo uses Ivy terminology instead of maven one, and also provides snippets to add a dependency in Ivy as well as in a Maven pom on each module revision page like this one.

More searchs

JavaRepo provides several kind of searches: the classic general search, but also more constrained searches, like the list of organizations, or the list of modules in an organization, or the list of revisions per module, and in any case you can restrict the result to names starting with a given prefix.

RESTful interface

I've developed the site using restlet, to keep control over the URLs. With no way to modify the data, being RESTful is pretty easy... But I've tried to follow some of the principles I've learned in "Restful Web services" book to design the URLs pattern, and the returned XHTML, which should be usable by human and computers.

These two last things are actually my main motivation behind this site. In the future I'd like to provide better code completion and easy dependency adding by search in IvyDE when using maven 2 repository, and we first need a service like this one to implement that.

Will I improve it in the future? Will it say at this location? Will I maintain it alone? I really don't know, there's no big plan, just a simple solution to a problem of my own.

Ant + Ivy team growing!

2008-02-22T14:42:00.002+01:00

Yesterday I published the end result of the vote I started last week to accept a new committer on the Apache Ant project, for his contribution to Ivy and IvyDE: Nicolas Lalevée has joined the team, congratulations Nicolas, keep up your good work, and continue to help make IvyDE a kick ass tool :-)

EasyAnt: Ant based pre packaged build system for java projects

2008-01-10T15:29:00.000+01:00

Here is a copy of an e-mail I've just posted on dev@ant.apache.org mailing list. I thought it might interest some of this blog readers too.

It's been a long time since I'm thinking about this, and thought it might be interesting to share with you and see where the idea can go.

I see many developers adopt Maven because they want a build system able to provide common features with no effort. Most of them don't want to spend much time writing an Ant script, or have seen or heard that maintaining Ant build scripts is troublesome. So they choose to use Maven only because it's easy to use for common use cases: install, write a simple pom of a few lines or generate it using an archetype, and you're ready to compile, test and package your new project following the Maven standard structure. They also get dependency management for free, and with only a few more effort they have multi module builds, and some nice features like code analysis, coverage, and a set of report gathered in a web site. That's really nice and that's what I like about Maven.

But Maven suffers from a lack of flexibility and robustness IMHO. And later the same people who first adopted Maven because of its perceived ease of use become frustrated when they need to tweek the system to their own needs or don't understand how the release plugin work. Then some of them go back to Ant, first having to go through a sometimes painful road to describe their whole build system in xml, especially if they aren't Ant experts. Others try to use new build tools like raven, buildr or others.

I really like Ant, and think it is a very good basis for robust and flexible build systems. People with enough knowledge of Ant can write very good build systems, testable, maintainable and adaptable. But you need to get your hands dirty, and you need to get a good knowledge of some of the mechanisms which can make an Ant based build system manageable: import, scripts and scriptdef, macrodef, presetdef, and so on.

Hence I'm wondering if it wouldn't be a good idea to package a set of Ant build files, providing all the basic features of a build system for java projects: dependency management, compilation, testing and packaging, plus maybe some more advanced features like code coverage and code auditing. Multi module build support would be nice to have too. Then someone needing only those features could simply have a build file per project mostly consisting of a single import of the common build file provided. Some needing more could provide plugins to the build system itself. Some needing to tweak the system could simply override some target definitions or properties. Others with very specific needs could simply use the build scripts as examples or basis.

I guess most people on this list know the benefit of having such a build system and how well it scales, and most of us already have developed such a set of build files. But providing the basis of such a good build system well packaged and documented could improve the Ant community IMO. With some efforts from our community we could end up with something interesting pretty easily. Most of us don't have much time, but we probably already have a good basis from the build files we work with around, and if this can be done in a community effort it could remain affordable in terms of time required.

So, what do you think? Do you think this would be useful? Would you be interested in contributing? Do you think a new Ant sub project would be a good fit?

Wicket and Databinder new releases

2007-11-11T19:08:00.000+01:00

This week-end Apache Wicket has released its first release candidate for the 1.3 version. That's very good news, I'm using 1.3 beta versions for a while now, and getting closer to the 1.3 final release is exciting! Kudos to the Wicket developer team for their work!

A couple of hours after this release, Nathan released a new beta version of databinder, upgraded to the new wicket version. Nathan you're so fast!

An interesting detail about this databinder beta: it now includes the Query Panel I developed some months ago, and make it even easier to use. Indeed, Nathan had the neat idea to provide a Data Browser page, which is mounted by default on the /dbrowser URL only in development mode. So if you have a databinder application, you benefit from this page for free! Good job Nathan!

Ivy has graduated!

2007-10-16T16:46:00.000+02:00

After almost one year of incubation, two incubating releases, 1800+ e-mails on the user list, 1400+ e-mails on the developer list, with the help of 4 mentors, 3 committers, 38 contributors and many more users, Ivy has officially graduated as a sub project of Apache Ant!

This means that Ivy is now an official Apache project, and I hope this will help us attract more contributors and committers in our community. Being a subproject of Ant, all Ant committers now have right access to Ivy svn repository, so this is already an important increase in the number of committers! And since we are about to decide to use the same development mailing list, hopefully the development community behind Ant+Ivy will soon be only a single and stronger community.

As a direct consequence of graduation, Ivy's web site is now reachable at http://ant.apache.org/ivy/. Update your bookmarks!

Upcoming talks

2007-08-21T17:30:00.000+02:00

Last week I've been contacted by Eelco Hillenius to replace him to give a talk on Wicket at JavaZone. After some questions about my skils to prepare and give this talk, I accepted. This will be my first trip to Norway, and my first talk on Wicket.

I'm not a Wicket expert, but I hope the lessons I've learned since I'm using it for some of my open source projects will help me give an interesting talk and answer questions.

What's pretty funny is that almost the same day I've been contacted by some Javapolis folks to give short talk on Ivy. And if ever I'm accepted to give a talk on Ivy at the Asia OS Summit in Hong Kong, the end of the year will be booked with quite a lot of conferences and traveling!

Now it's time for me to prepare the Wicket talk, and return to Xooctory and Ivy development, we have a beta to release pretty soon on both projects. In october I'll have much less time for open source with my courses at ENSEIRB starting again, and probably a consultancy for Thales Avionics, and some training sessions.

Top 10 reasons why you should try Wicket

2007-07-04T11:49:00.000+02:00

If you haven't already tried Wicket so far, here are my top ten reasons why you should:

Wicket is now Apache Wicket

graduated as a top level Apache project

Wicket 1.3 beta 2 is available

released a second beta version of their 1.3 stream

their JIRA

Very active community

receiving 1300+ mail a month

Proven web framework

difficult to know exactly the number of wicket application running today

Neat component model

Creating components is really easy

Large set of existing components

provides a large set of components

dojo

databinder

No XML

Separation of concerns

ever

Clean rendered web application

My first german article

2007-07-02T09:16:00.000+02:00

For those of you speaking german, Jan Materne and I have written an article on Ivy in JavaMagazin, which is already online and should be published in the next printed version.

It's an introduction article, so don't expect to learn much if you're already familiar with Ivy :-)

For those who might wonder, I don't speak german; so I'm not even able to understand the article :-) Jan did the translation from the english version on which we worked. It's pretty funny for a french guy like me to write an article in english to work with a german guy finally translating the article! Good old Europe and our so many languages!

Xooctory 0.2

2007-06-22T16:10:00.000+02:00

I've just published and announce the 0.2 version of Xooctory, the open source continuous integration server on which I'm currently spending most of time (along with Ivy, and providing trainings and java consulting).

I'm happy that we respect the road map we have set up so far, if we continue to work as expected the 0.3 version should come in a few weeks, and the 1.0 beta 1 in september. We have upgraded our own installation, so you can see it in action here:
http://builds.xoocode.org/

This version is still very limited and far from providing all the features we intend to have for 1.0, but it begins to give an idea of some of the concepts. The flexibility has been greatly improved with the introduction of some plugins, and the architecture has been improved after some discussions with people interested in the project.

One of the most interesting new feature in this version is the result publishing architecture. It makes it pretty easy to collect any kind of result from the builds. In this release only junit is implemented this way, but more are coming (compilation, javadoc, checkstyle, ...). This result publication system is interesting because it fully adhere to our instant feedback feature: you get the results as soon as they are produced by your build, instead of waiting for the end of the build. The result format is flexible, but still need to be improved. After a discussion with Arnaud Bailly, we may try to use RDF as format for publishing results. We'll see how we can evolve the system in future versions to take advantage of this.

Now it's time for me to investigate more the distribution problem, I'm currently considering Jini very seriously as a way to discover new agents in the system and easily upload code for build execution (with very simple administration). We will see how the 0.3 will implement this, but the distribution problem is very interesting and the discussions I've had so far on xooctory makes me enjoy its development very much!

wicket hibernate query panel

2007-06-19T11:38:00.000+02:00

It's so easy to create components in wicket that I almost create one reusable component a day! After the JQuery tab panel and a JQuery accordion panel, I've now setup a more complete component: a panel where you can enter an hibernate query and execute it. This is very simple, but very useful for debug and administration needs.

Here is how it looks like:

Once again the source code is not too complex and only requires a session factory (obtained from the spring container in my case). And since this is ASL v2 licensed code which depends on hibernate, wicket and databinder only, you can pretty easily reuse it for your own needs.

The development of this component was very easy, the longer part is to setup the columns, depending on the request. For instance, you can type "select job.name as name, job.id as id from JobModel job" and it will display two columns in the result, 'name' and 'id'. But you don't specify a select clause (eg "from JobModel") you will get all the properties of JobModel in the result table.

This is still far from what the Grails Ext user interface provides, but it's pretty useful already.

If I have time, I plan to extend it to provide the list of available entities for an easy access to corresponding queries. If I have even more time it could be interesting to push the idea even further and create some kind of phpMyAdmin for hibernate applications.

Wicket JQuery tab component

2007-06-04T19:04:00.000+02:00

I'm currently using wicket for the open source project on which I spend most of my time, and I was needing a client side tabbed panel component to submit a form using tabs in one POST.

So I've decided to implement a simple one using JQuery and the tabs plugin by Klaus Hartl.

The implementation was pretty fast, and it can be used as drop in replacement of wicket TabbedPanel component. Therefore I just have to replace:
new WicketPanel(id, tabs)
by
new JQWicketPanel(id, tabs)

Very easy, and I can also customize the tabs with the available options from the tabs plugin. For instance new JQWicketPanel(id, tabs, "{ fxFade: true, fxSpeed: 'fast' }") will create a tabbed panel with a fast fading between each tab.

For those interested, the source code is available in xooctory, the license is an ASLv2 license, so feel free to copy and adapt it to your needs.

Wicket makes very easy to develop such components, and that's one of the thing I like the most about wicket. It remembers me a session at last javapolis where experts explained during one hour how easy it is to create custom JSF components in eleven points. I luckily won their book while attending to javapolis, it's a 400 pages book just to explain how to create ajax components for JSF. This is not what I call easy. My wicket implementation is about 140 lines of Java (well, 50 lines of Java and the rest of comments and imports) and 40 lines of html (most of which is used for testing only). That's all, and you don't need to be an expert in wicket to write such a component. I really like wicket!

My road to the open source world

2007-05-28T11:17:00.000+02:00

During the past months my professional life has drastically changed and I think I've finally found my way...

Beginning of my trip

Since my diploma in 1999, I've tried may different forms of working. I spent a few months as an employee working for a bug consulting company, then a few other months for a Parisian startup; I later decided to become a freelancer, which worked quite well for a few years, but when I decided to come back to Bordeaux in early 2002, it was much more difficult to find missions, then I decided to join Lectra as an employee. The challenges of the research and development department of this french "software CAD/CAM equipment for textile leather and soft material maker" were interesting, but the way to manage employees was not my cup of tea, and I decided to leave with friends of mine and start Jayasoft.

The very first thing I did for this company was setup the development environment and the build system. After a quick evaluation of our needs, I wrote a small plugin for Apache Ant to manage the dependencies between our projects. A few months later, we decided to open source this tool: Ivy was born!

Ivy, first milestone on the road

Open sourcing Ivy was not really thought as a contribution to the open source world from which we already leveraged a lot of libraries and tools, but as a way to try to make our company better known, with a tool which would never have become commercial anyway.

Then the first posts appeared on the forum, and people actually began to download, install and use this small tool. I was very excited, it was the very first time I was in direct contact with people from all over the world using my work.

Excited by this new form of contact with people, I spent a lot of time improving the tool, answering to people, and trying to communicate on it. Some people began to talk about it on their blogs, and after a few months we reached the 1.0 version.

Then my excitation and involvement in this tool grew, but it was still an open source software developed in a closed source way. I slowly learned how important it was to have a public issue tracking system, and a public source repository. Later some people get more involved, and we decided to welcome Maarten Coene as a committer on the project. This was our very first step in the direction of an open development.

At that time we were contacted by a californian university to help them improve their build system by reviewing their Ant builds and installing Ivy. I left the old continent for 3 months, which was a very interesting time during which I had time to get even more involved on Ivy.

On the other hand, this period was pretty difficult for our company, a commercial project on which we invested a lot was not going as well as we hoped, and we had to face the truth: we were not able to make enough money with our current business plan.

September 2006, the big turn

When I cam back to France in September 2006, we decided to split the company. Two of us left to go back to the employee world, another one decided to become a freelancer acting for jayasoft and providing services to local big companies. On my side, I decided to come back as a freelancer, with a simple idea: I don't need much money to live, and I prefer to be free to do what I want most of the time. So I decided to try to do only to short missions, giving me enough time to spend in developing ideas for the open source community. Indeed at the same time Ivy entered incubation in the Apache Software Foundation, and I was getting more and more excited about the open source world and community.

During my first month as a freelancer I spent quite a lot of time in missions, and my time for open source was mostly dedicated to the migration of Ivy to the ASF: migration of documentation and web site, migration to the mailing lists and the Apache way, refactoring of code to make it cleaner, and so on.

And now it's been a few months since I've started a new project I was dreaming about for a while: the xoocode.org project.

Next stop: the xoocode.org project

This project is actually a crazy idea: I'm not really satisfied with the services offered by open source hosting providers like sourceforge, java.net, google code or javaforge. I would like an hosting solution as easy to use as google code, but with at least as many services as sourceforge. I'd also like to have continuous integration for all hosted projects.

Thus I've started xoocode.org as a small hosting solution which is not currently accepting projects, because there isn't much added value right now.

To provide the services I want, I've started two open source projects:
XooSent, a very simple issue tracking system, developed in less than a week. It is still very rudimentary right now, but I hope it will later become the tool I imagine.
Xooctory, yet another continuous integration server, for which we announced the 0.1 version today.

Many will wonder why creating new projects when you can already find pretty good open source tools out there. The answer is based on two main things:

features
interest

So I've setup a server with:

subversion
subversion browsing with websvn
subetha based mailing lists also usable as a forum
a very simple site, based on a project I start for Ivy web site: xooki
very rudimentary issue tracking, based on xoosent
basic continuous integration, based on xooctory

Where will I go next?

In the near future xooctory development has my priority. Later I hope to find enough time or community interest to continue XooSent development. Ivy is still pretty high in my priorities, but it already has a community, which is not the case for the xoocode project. I would also like to develop xooki to introduce some server side features to make it possible to edit pages online, while still keeping the offline edition feature. And I hope I'll be able to contribute to the wicket-contrib-push project on which I was recently promoted to the committer rank.

Then when the platform will begin to be stable and feature rich, I hope I'll find enough money and community interest to accept external projects hosting, to provide the services I was dreaming about to any open source project.

Where all of this will really lead me I don't know, maybe one day I'll wake up wondering I spent so much time of my life just for the pleasure of developing crazy ideas. But right now the pleasure to develop and discuss challenging things with smart people is worth much more than all the money I could make by being a more conventional developer.

JavaMail, Java6 and Classloader issue

2007-04-04T17:17:00.001+02:00

I'm currently on a small application in which I need to send e-mails. Using the JavaMail API from the jakarta commons-email API make it pretty simple, and I got my app sending e-mail in a matter of minutes in development.

But when I tried to deploy my application I ran into a strange issue:

javax.activation.UnsupportedDataTypeException: no object DCH for MIME type plain/text

Uh, what's the problem. Googling on the subject I quickly found an entry on jGuru's FAQ. According to this entry, it may be due either to badly registered mime type (pretty strange for text/plain), or to some classloading or version issues between javax.mail and the java beans activation framework (JAF)... But what's the difference between my development environment and my deployed environment: mainly a classloader (since I use Ivy to launch my app, it is actually loaded in a ClassLoader created by Ivy). But what is strange is that I ahve only one version of javax.mail and of activation.jar, and they are both loaded by the same classloader (from Ivy).

After some research, I found a small entry on the new home of javax.mail stating that JAF is part of glassfish... and of Java SE 6. And since I'm using Java SE 6, it means that JAF is loaded from the main ClassLoader, while javax.mail is loaded by Ivy ClassLoader, thus leading to this strange exception.

Fortunately I have enough control over my classpath and classloader, and asa workaround I've now added javax.mail jar to my jvm classpath, instead of loading it from Ivy classloader, and removed activation.jar from my deployment configuration in Ivy.

A new logging experience!

2007-03-30T10:58:00.000+02:00

I've been using logback for a few months now, and I'm impressed!

With excellent documentation and support, neat logging features, blazing performance and an innovating eclipse plugin, I've finally found a good replacement for the good old log4j.

The first thing I really appreciate compared to log4j or the java logging api is the documentation. The guide is well written, they have a nice demo, and you are up to speed in a few minutes.

Neat logging features

There are some very simple features which makes the life so much easier, such as the intelligent logger name reduction when it's too long: instead of simply truncating the name, it put the first letter of each package:

09:59:04.203 [main] INFO o.x.x.web.XoosentApplication - starting XooSent

The Mapped Diagnostic Context (MDC) is also a killing feature. It allows to associate metadata to the current thread, to correlate messages to their context. For instance in a web application with authentication, you can associate the user name to the the thread and add this user name to all log messages, without any change to your log calls. For instance, you add this at user authentication time:

MDC.put("user", username);

Then in your logback configuration you can use %Xuser in your pattern, and you will see the authenticated user name. It's that simple, and in multithreaded environment where multiple traces overlap, it really helps.
Updated: As Jorg pointed out, this isn't a feature introduced by logback, as log4j already supports NDC and MDC.

Performance

The parametrized logging is a key feature of logback, which improves performance by avoiding a toString() call when your message is not logged:

logger.debug("Hello, my name is {}, I am {} years old", username, age);

Note that obviously the performance gain applies only if you don't enclose your logging statements with if (logger.isDebugEnabled()) statements). But if you look at a benchmark run by Sebastien Pennec, one the developers of logback, it's really impressive:

Log4j direct debug call: 442
Log4j tested (isDebugEnabled) debug call: 19
Logback direct debug call: 435
Logback tested (isDebugEnabled) debug call: 10
Logback parametrized debug call: 15

OK, we all know how we should consider benchmarks, especially when written by someone biased as Sebastien obviously is, but these numbers can't be completly wrong, and what's interesting is that a logback parametrized call takes approximately the same time as a log4j isDebugEnabled call. Really cool!

Still on the performance area, logback introduces TurboFilters, which allows to filter logging before the logging event is actually constructed, saving a lot of unnecessary time.

Beyond console and files

Beyond classical ways to track and configure your logs, you have very interesting features with logback, such as a JMX configuration, and a new Eclipse plugin which is really neat.

One of the thing I like the most with this plugin is the option to go to the source which is at the origin of the log. Double click on the log, and it will open your source editor at the line where the log call is performed! Awesome! How many times did I use the search tool to find where a particular log call is performed in a big application, and waste time because the message was the result of a concatenation and thus my search failed...

Another interesting thing is the option to change the pattern and apply it to all the logs, including previous one.

And you have also a good filter option, where you can apply any logback filter expressions. This still need to be improved IMO to be able to apply the filter in real time to previous logs, but hey, it's only the first version of this plugin!

Excellent support

Last but not least the support is amazing. There isn't much traffic on the user mailing list for the moment (their documentation is so good :-)), but very often developers take time to answer your questions with a lot of details, trying to reproduce your environment, and providing whole configuration snippets. Great work, kudos to Sebastien, Jean-Noel and Ceki!

Adding license header to a whole code base

2007-03-27T19:55:00.001+02:00

I'm currently working to prepare the first release of Apache Ivy in the Apache Incubator, and there is a lot of administrative work to do to comply to Apache requirements.

I think it will worth it, joining Apache has already increased the feedback from the community even if we haven't released anything at Apache for the moment. But for the moment I have to update Ivy to comply to Apache rules, and one of them is to have an Apache License header on all distributed files, java, html, xml, properties, ...

So I wrote a simple ruby script to help me add these headers:

exts = ['html', 'xml', 'properties', 'java']

# check license in files, add it if not present
require 'find'

updated = 0
conform = 0
exts.each() do |ext|
 puts "--- processing #{ext} files"
 license = IO.readlines('license.'+ext).join()
 Find.find('./') do |f|
  if File.file?(f) && File.extname(f) == '.'+ext
   content = ''
   header = ''
   IO.readlines(f).each() do |line|
    if content.length == 0 && (ext == 'xml' || ext == 'html') && (line.index('<?') == 0 || (line.index('<!') == 0 && line.index('<!--') != 0))
     header += line
    else
     content += line
    end
   end
   if content.index(license) != 0
    File.open(f, 'w') do |f2|
     f2.puts header + license + content
    end
    puts "#{f} updated"
    updated += 1
   else
    conform += 1
   end
  end
 end
end
puts "------------------------------------------------"
puts "#{updated} files updated, #{conform} already compliant"

The main problem I faced is that xml files do not allow to have comments before the processing instructions (if any), so I had to add the ugly test to isolate the header in the files. But that's only a one shot script, so pragmatism was my main focus.

To be useful this script requires a license file for each kind of files to which you want to add a license header. In my case I used four files (for html, xml, java and properties) with Apache License.

If you have similar needs, you can download the script and the license files I used here.

IE7 support for XmlHttpRequest

2006-11-25T19:39:00.000+01:00

I'm currently working on a client side documentation engine, and had some compatibility issues with IE7.

Indeed I use XMLHttpRequest to load a file (a template), this file can be either a remote one or a local one, using the file: protocol for its URL.

The problem I had is that what was working with IE6 doesn't work with IE7: IE7 has a direct support for XMLHttpRequest object, as have Firefox. But the problem is that this object does not allow to send a synchronous request on a file: URL, in this case you get an "access refused" exception.

The solution is to use the ActiveX XMLHttpRequest even in IE7, which properly supports sending a synchronous request on a file: URL!

So my code snippet to obtain the XMLHttpRequest object is the following one:

JSR 277 again: Stanley and Andreas interviewed by InfoQ

2006-11-17T19:37:00.000+01:00

I've just stumble upon this article on infoQ:
http://www.infoq.com/news/2006/11/jsr77_jsr294_qa

Even if the comments say there is nothing to read, I found it quite interesting, with some answers to questions raised by the community.

JSR 277 feedback from the community

2006-11-07T19:44:00.000+01:00

As I said in my previous post JSR 277 has released its first early draft for review by the community, and fortunately the community has reacted.

Here are the links I've collected so far in no particular order:
http://robilad.livejournal.com/1672.html
http://www.javalobby.org/java/forums/t82769.html
http://www.osgi.org/blog/2006/10/jsr-277-review.html
http://milinkovich.blogspot.com/2006/10/java-component-war.html
http://www.jroller.com/page/murphee?entry=jsr_277_vs_osgi_here
http://www.eclipsezone.com/eclipse/forums/t83134.rhtml
http://weblogs.java.net/blog/stanleyh/archive/2006/10/jsr277_early_dr.html
http://www.infoq.com/news/jsr277-early-draft
http://www.mernst.org/blog/archives/10-01-2006_10-31-2006.html#120
http://epirsch.blogspot.com/2006/10/jsr-277-at-last-module-system-for-java.html
http://underlap.blogspot.com/2006/11/public-discussion-of-jsr-277.html
http://www.java.net/pub/pq/125
http://blogs.sun.com/andreas/entry/first_public_specification_of_the
http://weblogs.java.net/blog/patrikbeno/archive/2006/10/jsr_277_review_1.html
http://weblogs.java.net/blog/stanleyh/archive/2006/11/more_jsr277_ear.html

Java Module System: early draft available for review

2006-10-12T10:23:00.000+02:00

I have the chance to be a member of the JSR 277 expert group, and thus participate to the discussions which give shape to this important JSR.
The JSR has just published an early draft available for community review, so I would like to give some insight on its content to raise your curiosity, so that you read the document and provide feedback to the JSR.

Here is the link to the JSR homepage where you will find the early draft:
http://jcp.org/en/jsr/detail?id=277

But first, why do I consider this JSR important?

In a few words, the goal of this JSR is to introduce a new packaging and deployment mechanism for the Java platform. It's targeted for inclusion in Java7, and may thus change our life in a near future. Change our life? Yes, I think that this JSR is something that will really change how we develop application or libraries. We will now have a true module system, and thus be able to package applications or libraries in modules.

So what's the difference compared to JARs?

The file format

First, a module is packaged in a JAM file, which can contain classes and resources as a JAR, but also contain module metadata, and may even contain legacy jars. It will thus be possible to package a whole application in one JAM, in a way similar to how JavaEE developers where used to package applications in EAR or WAR.

But the aim is not to provide an application deployment format (as for EAR or WAR), but a module one. Thus a very important part of this new module system is how it deals with interactions between modules.

New encapsulation features

First, the module system defines a way to expose only part of the classes and interfaces of a module to the external world. What does it mean? This means that where you were used to be able to access to all public classes and interfaces available in the classpath whatever how they were deployed, you will now be restricted to only those the module has chosen to expose. This means that as a module developer this will give you more control over your module encapsulation, and thus increase reusability. As a module user, this means that you will have a narrower view on the module you depend on, and thus ease to focus only on what is useful... if the module developer has taken the right choices, or at least the choices that fit your needs.

So I think there will be a time of adaptation required to think in module and make good choice to expose only the relevant part of the API of our modules. Fortunately, it should always be possible to access an hidden part using the reflection API, providing that you have enough security rights in the JVM your code is running.

Dealing with dependencies

Another important part of the Module System is the way it deals with dependencies between modules. Until now you were used to deploy an application with all its dependencies, in complete isolation from other java applications deployed on the system. The problem is that this often lead to have the same JAR deployed a lot of times on the system (try to find log4j.jar for example on your machine, and you may be surprised). This lead to problems of maintenance, when a JAR needs to be fixed with a security update, you have to search and replace it, which can be very cumbersome.

So the new module system attempts to solve this problem by providing the concept of repository and declared dependencies between modules.

The module repository

A repository in JSR277 is a location where modules are deployed and are accessible by the system. In the specification, two main repositories are available: a local repository and an url repository. I want to underline that the URL based repository is not aimed to deal with a large scale public repository, like ibiblio. Its purpose is closer to what people used to JNLP where used to: deploy a small set of modules required for one application and give access to it through the network.

Version Policy and Versioning

Having a repository is a good thing, but then you need to view your application runtime in a completly different point of view. You no longer have a static classpath, but your application need to determine which modules, and which versions of these modules, should be used, considering what is deployed in the repository.

To do so, the system has to know which modules are available in the repository, and especially which versions. Then each module has to determine its dependencies, and select a version for each dependency. This is what is done by the VersionPolicy.

It also has to know which version is greater than another one (i.e. more recent), when a module choose to select the latest version of a dependency. Thus a versionning mechanism is defined by the specification, and all modules will have to adhere to this versionning system.

A new kind of problem

This part is very important, especially to my biased eyes :-)
I think it's important because it can introduce problems we were not used to. Why? Because if it happens that two modules used in the same application loads two different versions of the same module, then the classes from the two versions will be considered different by the JVM, and thus will be the objets.
This may lead to things very strange that JavaEE developers sometimes see. Imagine the following:

A class Baz in module baz, identical in both version 1.0.0 and 1.0.1:

A class Foo in module foo which depends on module baz 1.0.0.

A class Bar in module bar which depends on module baz 1.0.1.

Your application which depends on module foo and bar:

So what is the result of calling addBaz in MyApp? You guess? a ClassCastException!

I'm not sure that the average Joe will be able to understand the ClassCastException saying that the JVM cannot cast baz.Baz in baz.Baz... So first I think there will be an effort to give better error messages in such cases, and I hope we will see something like "cannot cast baz.Baz(1.0.0) in baz.Baz(1.0.1)". This would be a huge improvement, ClassCastException messages are already very poor, in the Module System case they will be even more important.

But we can imagine even worse scenarios! Imagine module bar depends on baz 1.0+ instead of 1.0.1. This would mean that MyApp could work perfectly on a system where only version 1.0.0 of baz is available in the repository, but fail when both version 1.0.0 and 1.0.1 of baz are available!

Moreover, do I really care to be able to understand the ClassCastException if I'm not able to fix it? No, that's why I think it's very important to consider the dependencies on versions carefully. The early draft does not address this particular point which is very difficult to address cleanly, but your feedback is already welcome on the subject.

Comparing versions

Another point I'm concerned with is the versioning chosen in the early draft. We didn't agree completly in the Expert Group on this subject, that's why I would like to have your opinion.

As you can see in chapter 5 on page 40 of the early draft, when versions are compared, the qualifier is compared using plain string comparison. What does it mean? It means that if you choose to name your versions like that:
1.0-b1
1.0-b2

Then you will have the surprise if you reach 1.0-b10, that it will be considered lower than 1.0-b1:
1.0-b1 < 1.0-b10 < 1.0-b2

This is stated clearly in the specification on page 42, but I'm not sure everybody will be aware of that and I'm afraid it may lead to a lot of problems. My opinion is that we should consider a sequence of digits as a number in the qualifier, and compare them as numbers. But again, what matter is what you think, so your feedback is welcome!

OK, I think I already lost you in much details with this entry on the subject, but the community review will only last until november 13th, so we need your input now!

Open Source documentation dilemna

2006-10-06T21:52:00.000+02:00

As the creator of Ivy, I have been confronted to a dilemna many open source developers have faced: how to write documentation for the project.

I usually see two ways to support the documentation:

generated from structured text files, like what is done in maven
edited online with an ad hoc engine, a wiki for example, like confluence

For Ivy we decided more than a year ago to use drupal for documentation writing. We are pretty happy with this solution, even if we had to write our own module to have a clean offline version of the documentation.

So, what are the pros and cons of each of these solutions? Documentation generated from structured text files is interesting because it allows to edit the documentation offline, and also it let users contribute to documentation through patches. The downside is that such writing is usually more painful, the generation step before actually seeing your changes is not helpful, and needing to upload your page after the generation after a simple typo correction is not a very straightforward task. Moreover you need your whole toolkit to be able to make any documentation update, instead of needing only a web access as with online documentation engines.

On the other hand, the online documentation engine is usually easier to maintain, especially if the tool is good, and pages reorganisation is easier. The syntax is often easy to learn and quite powerful. As a matter of fact, I've often seen projects which have chosen such documentation engine to have a better documentation. Look at Stripes or Mule for instance and I think you'll understand what I mean. The downside is that you cannot edit your documentation offline (and many open source developers like to work offline in airplane for example), and it's simply not possible for users without writing rights to your site to contribute documentation update.

So, what is the best choice between the two? I'm still puzzled by this question, but the more I wonder the more I think there could be a tool with both offline and online edition features, joining the best of both worlds. Maybe I'd need to start a new open source project... What do you think?

Code Search: the giant comes into the dance

2006-10-05T10:51:00.000+02:00

In my last post I was talking about a great resource I found for searching developer related information, like code, but also articles or projects.

Now I've just found that the search giant google offers a code search facility:
google code search

Nothing revolutionary here, except that it's google, and that they search also in code located in zip files, for example.

As developers, we now have a good set of tools to search for existing answers to our problems. Here is the short list of those I know:

Krugle has my preference because it does much more than just code search, and has a great user interface. But it's always good to have more than one tool in your toolbox.

Searching and browsing source code

2006-09-20T17:33:00.000+02:00

While browsing the web today I found two very interesting tools for developers.

The first one is not dedicated to one particular language, it's a search site focused on developer centric searches, called krugle. And it's really awesome. Try it yourself, have a look at their very well realised demo, these guys have a great product. Not only the search seems to be very accurate (at least for the simple tests I've done so far), but it's very responsive, and the UI and the navigation is simply excellent!

For search accuracy, for example, I've tried to search for Ivy. It's pretty difficult to find our website home with google, because Ivy is a very common word. With google you will find pages related to the plant or to the band. If you try to give better keywords you will add java or ant and find pages related to Ivy, but we still don't manage to get our home page ranked well in google. So you often have to go through an old javalobby announcement to find our web site if you just remember the name.

With kruggle it's really simple, type Ivy in the search box and you will get Ivy home page at third position. This is good. They do not index our source code, but we do not use standard open source hosting, so I can't blame them for that! I will see if I can submit a svn url for their indexing.

Another neat tool I found also today is an open source java source code cross referencing tool: Sorcerer. And it looks really nice! It's only in version 0.1 but it seems to work already pretty well, at least what I've seen on the sample report is very promising.

Another tool which I didn't found today but which has a set of features in the same order of idea is World Of Java. The idea here is different, it focuses on IDE integration (eclipse, netbeans, IDEA) to offer java source code and javadoc directly in your IDE as if you had spent time configuring all your dependencies.

It seems that there are more and more great tools for accessing code related information, this may influence the way code reuse is leveraged in our developments. I really like that!

Ant+Ivy vs Maven: my biased opinion

2006-09-19T15:01:00.000+02:00

I recently read a post on Carlos Sanchez's Blog about The black magic inside Maven. He himself reacted to a post on Sylvain Wallez's blog: Spring switching to Maven? Oh no, think twice!.

These entries are interesting, and as Ivy creator and project Leader, I would like to share my biased opinion on the subject.

Obviously, what I strongly agree on with Carlos is that Sylvain, Carlos and I, and anybody giving an opinion, is biased. I often see people choosing a tool or another one simply because they've seen on a blog someone saying "Maven is great!" or "Ant rocks!".

I don't think that choosing a tool or a set of tools for the build management, or software configuration management, should be done like that. These kind of tools are very important in the way you build your software, and building the software is the ultimate step to transform source code which only interest developers and IDEs, into an actual software which will be useful (at least we hope so).

These tools are very important because they are both used day to day by developers, but also used once a while in a very important step in the software development process: when you release your software. And if building for developers is important and should be straightforward, I think that the release part is even more important. Why? Because it occurs less often, and so you will often forget about how it works between two releases. Because it must ensure that what you do will be absolutely reproducible in the future. Because it must be predictable. Simply because it's the ultimate separation between a bunch of source code and the software which will be used.

And that's because the build and release management system is so important that I strongly share the opinion of Sylvain and Leo Simons in his comment on Carlos blog:

I don't want automagically or out of the box, I want safe, controlled, consistent, predictable, repeatable, documented, with no surprises.

This is an important point, even if being easily usable out of the box is an advantage, at least it can help to evaluate easily, what is most important is to be able to keep control over your build and release and truly understand how it works.

So if you are very confortable with Maven, if you have experts in your team, I think it's definitly the good choice. There are some concepts in Maven that I really like. Describing your project at a high level and then applying rules to perform your build tasks, this is a good idea. What I don't like with Maven is that its lack of flexibility. Maven guys seem to know better than anyone else on the world how building a software should be done. How you should layout your projects. How you should work in a team. And maybe they are right, maybe they know that.

But what I believe is that each case is different, the environment and the constraints are always different, and there is no single solution to this problem. So that's what I like with a tool like ant: it's really flexible. So yes it's often more painful when you start from scratch to have a first build. But then you can truly understand and control what it does.

And that's also Ivy philosophy, to be flexible, to adapt to your environment and your constraint. It's Ivy philosophy to be safe, predictable and repeatable (even if there are unknown bugs as in any software, more and more are discovered and fixed by the community). That's also why we try to have a good reference documentation, even if we still lacks of examples, guides and tutorials, the reference documentation is the ultimate place to get knowledge about a tool, and we try to keep it up to date and accurate.

So, whatever you think after reading this post or other truly interesting ones, take time to get your own opinion on the subject considering your own environment and your constraints. This will later save you a lot of headaches!

Interesting discussions about Maven vs Ant+Ivy comparison

2006-09-19T11:51:00.001+02:00

This post is a copy from a news originally posted on jayasoft.org site

At the beginning there was a news from the SpringFramework contemplating about migrating to maven for their build.

Then there was the reaction of Sylvain Wallez on his blog.

And this post triggered interesting discussions, both in the comments, and on several other blogs. Carlos Sanchez, Steve Loughran, Stephane Baillez gave their interesting point of view on what goes much further than Ivy, Ant or Maven discussion, and more about the build and release systems requirements.

Follow up: I've later blogged in a more opinionated way here

Intense end of my californian experience...

2006-09-19T02:42:00.000+02:00

My stay in Northern California is about to end up in less than a week, and I've seldom been so busy!

I'm finishing a full time contract where I helped introduce Ivy and review the build and release system of a well known university. This contract was really interesting, with many challenges, a lot of work with Ant and Ivy, and finally a customer who really seem to be happy, and I like that. I think it's the most rewarding part of my job, with customers or with community users, when you see that you can help and improve things.

They now have a brand new build and release system, with easy workspace management (like checkout or update a project with all its dependencies), and a release system which will change the way they see it. With a couple of commands they can release an application and all the modules actually requiring to be released, i.e. modules which have changed since the last release. They can identify easily which modules are part of each release, in which version, with the associated tag. They can also query the repository to know for a single module version with which application it has been released.

The installation is also fully automatic, they can install any application build easily (be it a release or an integration build). The system I set up also keeps track of each installation, so that they can know for each release where and when and in which configuration it has been installed.

This was quite a lot of work, but it's only my day job... then can some the night, and since I spent my last 3 months most of the time alone I found time to make many improvements and implement new features in Ivy. I've just released the 1.4-RC1 version yesterday, and with the help of the community and other commiters we managed to ship more than 90 new features, improvements and bug fixes in this version. That's a lot of stuff, I won't detail things here, I already wrote about this on the new and noteworthy page of the upcoming 1.4 release.

With this release my last week-end was very intense, but that's not all. I worked with yet another customer, a belgian company, which contracted us to introduce Ivy to better manage dependencies in what they call the soft split of their software. These guys are building very complex software, and used to have only a few units of work. But as complexity grows they decided to break out units in modules. To avoid changing everything, they decided to do a soft split: split the unit in modules but keep the modules together in the unit, so that you can still work on the whole unit but ensure a good isolation between modules, by compiling them separately for example. This is a very good way to deal with this kind of problem, and can ease a lot the migration to an even more modular system.

To deal with that system, they already set up a build system relying on properties files to describe dependencies. The problem they face is that whenever dependencies change in one module they have to modify dependencies of all modules depending on it. Needless to say that they lose a lot of time dealing with that, especially when they have to merge several modifications of this type!

So they called us to ask if it was possible to introduce Ivy to automatically deal with dependencies inside the unit transitively. With a four day contract I am on the way to finish, they will now have automatic transitive dependencies management, with as a bonus a graph of all dependencies in their modules generated automatically, and the build order for these modules calculated automatically. And with no migration cost: my work integrates completly with their former build system and the storage of dependencies in properties file with which they were already confortable. This will save a lot of time to all the developers and they will have better insight on their dependencies. I think they will be pretty happy with their choice, and I'm glad to contribute to such an improvement.

So my last days here were very intense, I think that once I'll get back to France I'll need some vacations, and spend time with my wife after a quarter year without seeing her! But before I still have to finish my work with both my belgian and californian customers, publish a new version of IvyDE, and share some bordeaux wine with the so nice people I met here! I think I'll be pretty busy til the end of the week :-)

IE7 RC1: first impressions

2006-08-30T05:34:00.000+02:00

I've just downloaded the Release Candidate 1 of IE7, and I should say I was expecting a better quality for a release candidate. With my configuration (windows XP SP2 running on a dell laptop), I have problems with images transparency (both with GIF and PNG): I also have trouble with microsoft community site itself! Notice especially the truncated popup menu (OK, the image is very small, but if you have good eyes you will see it in the middle). To finish on a good point, I like the quick preview of all tabs, easily accessible (ctrl+Q), fast (but no special effect à la Exposé), and useful: So I will stick with firefox for the moment, and wait til they improve the quality of their rendering engine, especially for image transparency, before even considering a switch.

Constructing ant path via contribution

2006-08-20T00:01:00.000+02:00

I already post this recently on my former jroller blog, but I would like to see how easy (or difficult) it is to post code snippets on blogger.

I'm currently setting up a build system, and I try to make it modular. I try to separate each feature of the build in a separate build file imported from a common one. This eases maintenance a lot.

The problem I faced today was to introduce emma code coverage tool in the build. As a separate feature, I created a separate ant build file dealing with all emma aspects.

But my problem is that emma needs to interact with the junit task in several ways:

add system property specifying the output file and some parameters
replace regular classes by instrumented ones
add emma in the junit classpath

All of this should be done only when emma is enabled...

Adding system property was pretty easy with the propertyset feature of ant. Replacing regular classes by instrumented one can easily be done, either by actually replacing them on the filesystem, or by replacing the reference to the path corresponding to the classes.

But adding emma in the junit classpath is not an easy task: the junit path is defined elsewhere, so to keep things modular I don't want to redefine it completly with the emma lib and replace the reference. Neither do I want to put some if/unless targets to deal with my cases at the moment I define the classpath for junit: it introduces a high dependency between the two. So I thought that it would be nice to be able to modify an existing path to add some other path elements, either at the beginning or at the end.

I googled to find a solution with no result. So I decided to write a custom ant task, which is actually very basic but now I can do things like this:
or To put the instrumented classes before anything else.

This is quite useful for my modular build, if some of you I've checked it in Ivy sources, but feel free to copy and use it outside Ivy (even if using Ivy could be a good idea to improve your ant based build system :-)).

You can view the code here

Damned, I found no ease way to post xml code snippet! I add to replace manually the tags to make them appear on the page. On this point jroller was better... I'll post a feature request somewhere on blogger, and hope google smart people will be able to fix that soon.

Updated one day later: I finally found a pretty satisfying way to post code: I use the textarea tag, I set the cols attribute, and I disable the auto line break option of blogger. This is not amazing but it works.

Blogger beta

2006-08-19T21:00:00.000+02:00

I just realizeed that I'm actually using a brand new version of blogger, in beta since only a few days. I used my google account to login, and didn't notice that I was using a beta version (and there are so many things in beta with google :-). It seems that I've been lucky to fall on the beta version, since at least the rich html editor and the tag system are new with the beta. So are some templates. So I'm pretty happy to now be a beta tester of this new version, but I already had a first problem with it: I tried to use the blog this button in picasa, and I didn't managed to log in using my google account. It seems that the account are not the same between the beta and the current release, and I assume picasa only works with the release. So I'll have to wait to easily add my photo on the profile. Sorry for you, you won't see my beautiful face until some time :-) Oh, but it seems that I can add an image directly on the post. So here is one: I believed you guesses this is not me! This is a picture I take during my trip to yosemite last week-end. I may tell more about this later, right now I'm hungry, it's time to unplug from my internet addiction!

Moving to Blogger

2006-08-19T20:39:00.000+02:00

I've been using jroller for a while now, and even if I do not blog very often, I must say that I'm a bit fed up with the user interface for creating and editing blogs.

The remember me option rarely work, the UI for posting an item is not very nice, well, some little things which let me thought that it may be interesting trying another blog web site...

And here I am! It's still difficult to say if this one will really fit my needs, but right now the user interface seems clean and quite easy to use.

So maybe here will come my next random thoughts...

About

2005-06-26T19:43:00.002+02:00

Welcome to my blog!

I use this blog to dump random thoughts about Java, Ivy, Xooctory, and software development in general.

For those interested by these details, I'm an independent Java consultant providing services to people seeking for talented Java developer or architect.

I regularly provide trainings on Hibernate, Spring, EJB3, Ant and Ivy. I'm also often involved in software architecture review or expert consulting on dependency management.

I spend a large amount of my time in the open source community, I'm the creator of Ivy, Xooki and Xooctory, and a committer on WicketStuff.

For those who are not afraid by french, my resume is available on my french blog.

I'm also involved in a commercial public web site project, started in 2010, based on google app engine. The project, called NextAutoz and available at http://www.nextautoz.com/ is a french portal delivering announces of car selling by professionals.

You can reach me at xavier DOT hanin AT gmail DOT com.