Maven Release with Flat Project Structures Revisited

26 01 2010

You may recall my previous blog on Maven Release Woes with Flat Project Structures where I described in detail how the supposed 2.0-beta-10-SNAPSHOT flat structure release support did not perform as expected. After some back and forth with one of the Apache developers, it has come to light that this conclusion is partially correct. The term “flat structure” is contextual and in this particular context, the Apache group’s idea of context was SCM. SCM structure is what needs to be in one of a few known structures for the release plugin to function appropriately. In the end, another JIRA ticket was created for a different use case, albeit very similar to the previously closed ticket. I’ll review what flat structure does work and some variations that don’t.

SCM Structures
First, let’s understand the structures the release plugin does support. Here is your typical nested multi-module project structure that’s supported and recommended (remember SCM structure):

ExpectedSCMNestedStructure.vyHqvDoV95Ju.jpg

Now let’s take a look at the SCM flat structure that the release plugin will support, provided you’re using at least 2.0-beta-10-SNAPSHOT:

ExpectedSCMFlatStructure.aeYlwUeJjVvs.jpg

You can see that all the projects of the multi-module project are flat, behind trunk. If you have multiple projects in your SCM, I’d recommend breaking them up with project names at the SCM root, similar to what I have done here with flat-structure-test. If not, when the release plugin goes to tag your code, it will tag everything behind trunk, not just the folders your multi-module project uses. Another thing recommended to me by the developer was to use a particular configuration, -DcommitByProject=true. For example:

emiles-macbook:release-parent emiles$ mvn release:prepare -DcommitByProject=true

With this diagramed, you must still be mindful that projects must be structured completely flat behind trunk. For testing purposes, I tried a slightly altered flat structure where release-module2 was under a random, non-project folder under trunk named somesubfolder.

WackySCMBefore.wnRv2UyuruE4.jpg

This structure passed the prepare goal just fine, but when it came time to actually perform the release, it failed with an SCM checkout failure.

INFO] [INFO] Scanning for projects...
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [ERROR] FATAL ERROR
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Error building POM (may not be this project's POM).
[INFO] 
[INFO] 
[INFO] Project ID: unknown
[INFO] 
[INFO] Reason: Could not find the model file '/Users/emiles/Projects/release-workspace/release-parent/target/checkout/release-module2'. for project unknown
[INFO] 
[INFO] 
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Trace
[INFO] org.apache.maven.reactor.MavenExecutionException: Could not find the model file '/Users/emiles/Projects/release-workspace/release-parent/target/checkout/release-module2'. for project unknown

Summary
While a specific flat project structure use case is supported, you must be mindful of the SCM structure as slight variations may not work with the release plugin. As I recommended in my previous blog, use the nested structure for multi-module projects if you can. If your project structures already exist in SCM, if possible, restructure them to model the nested structure.

Be on the lookout for my next blog article providing some suggestions on how to tackle releases if you’re stuck in an SCM structure that the release plugin does not support.





Maven Release Woes with Flat Project Structures

22 12 2009

Introduction
The last two clients I’ve been on, I was asked to determine how Maven could be leveraged (better) at their site. One client was not using Maven at all and the second client was already using Maven for two years. Both clients were using Subversion and both had their own custom repository structure. The first client had a truly unique setup of how trunk/branches/tags were used and was nothing like I had seen before. There was only a single trunk, a single branches, and a single tags folder, and ALL projects were in numerous folders based on system functionality under these. The second client followed a more traditional model, with each project on the root of the Subversion repository with each project having a trunk, branches, and tags subfolders. However, both employed a “flat” project structure to make lives easier with their chosen IDEs.

Both clients had challenges that were were both related and un-related to Maven, but I was able to work through them until I got to the release process. As most of you know, the release plugin does a lot of work that would normally be done manually: incrementing versions, creating tags, updating SCM locations, checking into SCM, etc. The release plugin is particularly handy when dealing with multi-module projects. After spinning my wheels on the first client for a day or two, I came across the following Jira. There wasn’t much else on the net regarding this issue, nor did any of the Maven books I had read cover a flat structured maven project. Most described multi-module projects in your typical nested structure.

This stumped me and proved to be the ultimate decision to scrap the use of Maven at the first client (their odd SCM structure also played a part). The second client was nearly 8 months after the first; I was extremely pleased to see that a patch had been applied to trunk of the release plugin that would support flat project structures. Numerous test cases were added to cover these so I had high hopes. I pulled down the latest snapshot version, 2.0-beta-10-SNAPSHOT, and started to perform some releases on a sample project I threw together. I quickly found the issue is not yet fixed After some minor debugging, I thought maybe it was the SVN SCM provider so I tried several others. The following is an account of my findings.

Project Setup
This is the typical flat project structure, where the parent pom specifices each module like ../release-module1 and the modules specify the parent pom with a relative path ../release-parent/pom.xml.

Subversion
SVN was the first provider I tried. Here is a snippet of the SCM information used by the release plugin.

<scm>
    <connection>scm:svn:http://localhost/dev/release-parent/trunk</connection>
    <developerConnection>scm:svn:http://localhost/dev/release-parent/trunk</developerConnection>
</scm>

While running release:prepare, we received an error regarding directories not being working copies. Here is a snippet of the error received while running release:prepare

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Unable to commit files
Provider message:
The svn command failed.
Command output:
svn: '/Users/emiles/Projects/release-workspace' is not a working copy

CVS
CVS is the next SCM provider I tried. Here is a snippet of the SCM info from the parent POM:

<scm>
    <connection>scm:cvs:local:/Users/emiles/Projects/cvsrepo:release-parent</connection>
    <developerConnection>scm:cvs:local:/Users/emiles/Projects/cvsrepo:release-parent</developerConnection>
</scm>

Again, while running release:prepare with this provider, we receive an error similar to the one we received using SVN. Here is a snippet of the error received while running release:prepare:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] An error is occurred in the checkin process: /Users/emiles/Projects/release-workspace/release-module1/pom.xml was not contained in /Users/emiles/Projects/release-workspace/release-parent

GIT
I hadn’t used GIT before, but read the SVN to GIT crash course and tried this SCM provider. Here is a snippet of the SCM info from the parent POM:

<scm>
    <connection>scm:git:file://localhost/Users/emiles/Projects/gitrepo/release-parent.git</connection>
    <developerConnection>scm:git:file://localhost/Users/emiles/Projects/gitrepo/release-parent.git</developerConnection>
</scm>

As with the previous two providers, an error occurred during release:prepare. While the error message is not exactly like the one received for SVN nor CVS, it seems as though there are some similarities with being “outside a working repository”.:

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Unable to commit files
Provider message:
The git-add command failed.
Command output:
fatal: '/Users/emiles/Projects/release-workspace/release-module1/pom.xml' is outside repository

Conclusion
Flat project structures are still not supported by the release plugin. The good news is that the parties responsible for the release plugin have reopened the previously closed defect, however it took them 2+ years to get this first patch together. It seems the issue is with trying to commit all the updated POMs of a flat structured multi-module project in a single transaction. If each of these were committed one at a time, there would probably be no problem. However, as with any transaction, we want all these POMs committed at once or none at all.

I love Maven, however I’m perplexed as to why this defect has been open this long without resolve. It seems it would be a major issue for numerous Maven “switchers” from other build frameworks. My advice is if you’re starting from scratch, setup your multi-module project in the expected nested structure. A lot of other tools such as Hudson and Continuum expect multi-module projects to be in a nested structure and by doing so will save yourself a lot of headache. If your project already exists and it is not in a nested structure, if it’s a small project, you might be able to live with the manual release process. If you can’t live with the manual process or if your project is a large one, you should think about re-structuring your SCM repository to provide for the expected nested project structure.

Attached are the sample projects (currently configured for SVN) and output from all three SCMs used.