Connecting The Dots with Android SyncAdapter

22 09 2010

I recently had an opportunity to work with the Android SDK. This was my first foray into Android development and I found it interesting and rewarding. One of my tasks was to look into creating a sync adapter, responsible for syncing our local device storage with a RESTful service in the cloud. While a sync example is provided in the SDK and the Android javadocs are fruitful, there was still a certain lack of prescription that I find necessary when looking into a new technology. This blog will provide the necessary steps in a prescriptive format, hoping to help any others struggling with this topic.

Before getting started, I absolutely must recommend viewing the 2010 Google I/O presentation on Android REST Client Applications, presented by Virgil Dobjanschi. It’s a fantastic introduction into some of the concepts and recommended patterns for writing well design syncing applications. Along with this presentation and the sync adapter sample provided with the SDK, the following list should provide you ample ammunition to get a sync adapter up and running fairly quickly.

1. ContentProvider
There are two important Android components that are required for your sync adapter, the basic being a ContentProvider. The content provider should contain data you want to sync. Content providers are generally backed by one of the storage mechanisms in Android: Shared Preferences, Internal Storage, External Storage, SQLite Database or network storage. Writing a content provider is outside the scope of this entry. However, the main thing to note when implementing a content provider is the authority you assign in the Android manifest. It will be used in another step to tie some of our components together. In this example, it’s com.captechventures.unanet seen in this snippet:

<provider android:name=".content.UnanetProvider"
        android:authorities="com.captechventures.unanet" android:enabled="true"></provider>

2. Account
The other important component is that you must have an account registered with the Android OS that you intend to link to your sync adapter. The bad news is that there is no “stock” functionality to give you an easy way to provide an Account to the system. However, in the same Sync Adapter Example that comes with the SDK there is a lot of code you can borrow to give you Account functionality. Unless you desire a custom credentials screen, you can heist all the code in the package with only a few minor changes. You’ll also need to snag the authenticator.xml from the resource directory. There is a reference to Constants.ACCOUNT_TYPE in the Authenticator class. Be sure to replace this value with your own account type value, such as com.captechventures.unanet.account which is what we used for our example. You’ll also need to change the accountType attribute in the authenticator.xml to match the value you just used for replacement.

3. Sync Adapter Descriptor
This is the file that ties your sync adapter to the content provider and the account. In the res/xml folder, you’ll need to place an xml file that describes your sync adapter. It doesn’t matter the name, the contents are what’s important (that was foreign to me). Here is my example, found in the res/xml folder and named syncadapter.xml:

<sync-adapter xmlns:android=""

Recognize the contentAuthority and accountType attributes and their values. This is how we tie our content provider and our account to our sync adapter.

4. Abstract Threaded Sync Adapter Implementation
You’ll need to implement an AbstractThreadedSyncAdapter. Here is where the meat of your sync logic will live/start. There’s no one right way for sync logic and it will be different from implementation to implementation. However, I highly recommend following one of the patterns Dobjanschi outlines in his presentation for managing the sync itself. Once you watch the presentation and absorb the information, it just “makes sense” and doesn’t seem like there could be a better way.

Be sure to use the SyncResult in your overridden onPerformSync method appropriately. Using the object in the appropriate manner will allow the sync manager to be more effective.

5. Sync Adapter Service
The syncing mechanism will run in the background on a different thread from the UI. To do this, we’ll need to setup a service that filters on sync intents. The service implementation is simple and only a few lines long. Follow the example in the Sync Adapter Sample, changing the types to match yours. Here is the declaration in the Android Manifest for my sample service, notice the addition of the intent filter and the android:resource reference to the Sync Adapter Descriptor we created in step 3:

<service android:name=".sync.SyncService"
                <action android:name="android.content.SyncAdapter" />
        <meta-data android:name="android.content.SyncAdapter"
                android:resource="@xml/syncadapter" />

6. Android Manifest
The final piece is to pull everything together in your Android Manifest. We’ve already identified you’ll need your content provider as well as the Service responsible for reacting to sync manager requests declared. You’ll also need to provide a few permissions to allow several aspects of your application to function appropriately: ability to read/write accounts, ability to interact with the network, and ability to read/write sync settings. Here is a snippet from our sync adapter:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.GET_ACCOUNTS" />
<uses-permission android:name="android.permission.USE_CREDENTIALS" />
<uses-permission android:name="android.permission.MANAGE_ACCOUNTS" />
<uses-permission android:name="android.permission.AUTHENTICATE_ACCOUNTS" />
<uses-permission android:name="android.permission.READ_SYNC_STATS" />
<uses-permission android:name="android.permission.READ_SYNC_SETTINGS" />
<uses-permission android:name="android.permission.WRITE_SYNC_SETTINGS" />

While this roadmap is aimed at kickstarting your sync adapter development, it by no means replaces reading and understanding the Android documentation on sync adapters and all the related classes and components. Be sure to read the documentation linked throughout this entry.


Maven Shell Features Explored

26 03 2010

In my previous blog, I gave a brief Introduction to Maven Shell and what to expect from the environment. I really like what Maven Shell has to offer so I wanted to follow up with a detailed review of some of the more interesting features of Maven Shell. Leveraging these features should make for a better Maven and Maven Shell experience and possibly allow you to work more efficiently.

ANSI Color
Maven Shell has bash like color coding to the shell, providing easy identification of the different diagnostic messages provided by Maven and Maven Shell. The colors Maven Shell produce are dependent on your shell customizations, however, with a standard terminal of black background and white text color, here are some colors used:

RED — Warnings
BOLD RED — Build errors
GREEN — Reactor summary listing of modules
GREEN — Currently executing plugin name (inside of standard white informational message)
BOLD WHITE — Executing goal of current plugin
BOLD GREEN –Successful build!
LIGHT TEAL — During plugin execution in a multi-module build, identifies which module is currently being worked on.
GOLD — Debug tag

A sample screen shot:

Notice, it’s really easy to determine if there is an error or warning during builds now or if it completed successfully. ANSI color is a nice feature.

Growl Notifications
If you have have Growl installed, you will receive build completion notifications via the Growl system. The notification will include whether the build failed or not. At first, I didn’t think this was highly usable as I spend most of my time watching the build perform….and there’s the kicker. If you have a long running build, you can go do other stuff while Maven cranks away and you won’t waste time flipping between windows to check on build status. Growl will popup a notification and you can go back to your mvnsh window. Higher efficiency achieved again!

Note: I’m assuming if you have Growl for Windows installed on a Windows based machine, it will work. I do not have a Windows VM to test this out on.

If you find yourself typing long winded commands over and over, try using the alias feature. It’s very similar to the alias feature available in Linux. Rather than typing the long winded Maven build command: mvn -Pint-test,oracle,reports -Djdbc.driver=someoraclestring -Dmyprop=val1 clean install, you could only have to type runmybuild instead by using an alias:

mvnsh(/):~/Projects/fyb_maven_workspace/multi-example> alias runmybuild mvn -Pint-test,oracle,reports -Djdbc.driver=someoraclestring -Dmyprop=val1 clean install

Once your alias is set, issue it to perform a build:

mvnsh(/):~/Projects/fyb_maven_workspace/multi-example> runmybuild
[INFO] Scanning for projects...
[WARNING] Some problems were encountered while building the effective model for com.captechventures.playground:multi-example:pom:0.0.3-SNAPSHOT

Rather than specifying aliases every time you load up a new Maven shell, put aliases you tend to repeat into the $HOME/.m2/mvnsh.rc file, which is loaded at Maven Shell startup.

The Variable feature allows the user to set system properties or variables that persist during the life of the Maven Shell session. System properties are JVM system properties and key value pairs you would supply to a Maven execution with -D. For example, if you were the type of person who hated to run unit tests (I hope you aren’t), you could set a system property to skip tests:

mvnsh(/):~/Projects/fyb_maven_workspace/multi-example> set -m property skipTests true

Now for every time you execute Maven during this shell session, tests will always be skipped. Be sure you’re not in the /pref command group, as there is a set command available in that command group and as you would find out soon enough, the syntax is not the same: you’ll receive a command line error. The -m property of the command identifies that you would like to set a property. You must then provide a key and then a value. If the value contains spaces, put the value in quotes. To set a variable, you would issue -m variable. To see a listing of all currently set properties or variables, issue the set command without specifying a key and value:

mvnsh(/):~/Projects/fyb_maven_workspace/multi-example> set -m property'Java(TM) SE Runtime Environment'

I’ll be honest, I’m unsure what variables are for. By listing the currently defined variables in my shell, it looks as though they are related to how the shell and user interact with each other. I’d love to hear from some of the developers as to what the distinction is between variables and properties.

The Preferences facility allows you to customize your environment in a way that is persisted across mvnsh restarts. While the mechanism to set/get/unset preferences is generic to support functionality in the future, the items you currently need to be concerned with are located using the list -r /mvnsh command.

mvnsh(/pref):/Developer/Applications/mvnsh-0.10/bin> /pref/list -r /mvnsh

The /mvnsh tree contains the possible nodes where preferences can be set against commands in the mvnsh environment; currently, the most important to us are mvn and nexus. By setting preferences on the mvn command, we can how maven will behave when we issue that command. For instance, we can make maven executions quiet no matter what we’re building:

mvnsh(/pref):/Developer/Applications/mvnsh-0.10/bin> /pref/set /mvnsh/commands/mvn quiet true

If you were to issue the mvn clean install command on a big multi-module project, it would be eerily void of most console output unless it were to error out. Again, this preference is persisted across mvnsh restarts, so this behavior would be exhibited the next time mvnsh is loaded. To unset this preference, merely issue the unset command:

mvnsh(/pref):~/Projects/fyb_maven_workspace/multi-example> /pref/unset /mvnsh/commands/mvn quiet

The big question is, how do I know what preferences I can set? As far as I can tell, there is no documentation for this yet. The only way to pull this up open up source code for these commands and look for attributes annotated with org.sonatype.gshell.util.pref.Preference. The preference key name is the getter/setter convention for the attribute. For example, the quiet preference we just referenced can be found in the class:

    @Option(name="q", longName="quiet")
    private Boolean quiet;

I’d love to see some documentation or examples from the developers for this feature as I’m sure it much more robust that the trivial example here.

Like in a bash shell, there is a buffer/history mechanism. It works very similar to its bash shell counterparts. To see all the commands you’ve executed in mvnsh, execute the history command to retrieve an indexed list:

mvnsh(/):~> history
  311 ls
  312 cd wcf_workspace/wcf/
  313 mvn -DskipTests=true clean install
  314 mvn -e -DskipTests=true clean install
  315 mvn -DskipTests=true clean install
  316 mvn -DskipTests=true -Pint-test clean install
  317 mvn clean install

You can either use copy and paste within the shell, or you can use the recall command to re-execute a history item with the appropriate index:

mvnsh(/):~/Projects/fyb_maven_workspace/multi-example> recall 315
[INFO] Scanning for projects...
[WARNING] Some problems were encountered while building the effective model for com.captechventures.playground:multi-example:pom:0.0.3-SNAPSHOT
[WARNING] 'reporting.plugins.plugin.version' is missing for org.apache.maven.plugins:maven-pmd-plugin @ com.captechventures.playground:multi-example:0.0.3-SNAPSHOT, /Users/emiles/Projects/fyb_maven_workspace/multi-example/pom.xml

Completely ignore my warnings there, those are due to some very bad practices in the POM under execution, not with the command itself.

BlackBerry APIs Need Some Work

18 03 2010

While unstaffed the last few weeks, a manager tapped me to help put together a POC BlackBerry application for one of our clients. I have had experience working on a BlackBerry project from a previous employer, however I did not have hands on experience programming the device myself. I was looking forward to this as I had been wanting to do some dabbling with mobile devices, I just had not had the time.

I knew programming for BlackBerries were a bit tricky due to the numerous versions of the OS and devices, each with their own hardware specs (with or without GPS, supported resolution, touchpad, trackball, etc.). We tailored our POC for the 8800 series due to clent request and for the GPS API support. I downloaded the latest java development software, version 5.0 beta. With 11+ years of Java experience, jumping in and immediately putting together an application was easier than anticipated. However, I quickly became frustrated at the APIs (or lack thereof) that has me seriously wondering if RIM doesn’t need to do an overhaul on their APIs and development model for applications.

For example, I had RichTextFields that required a text color other than black. Nothing existed on the class that resembled a way to change the text color so I Googled. I came across several message board posts, but all seemed to point to the same resolution: override the paint(Grahpics) method of the class to change the color on the Graphics object. Say wuh? Just to change the color of the text, I had to override the entire paint method? Here’s an example of what was required:

RichFieldText rft = new RichFieldText("Here's my text") {
        public void paint(Graphics g) {

Of course, you could provide a base class responsible for these sort of things and make it available to your entire project, but this is a horrible model given the triviality of changing the text color of a text field! I found another example of this poor development model while working with the MapField class. I have to think one of the most common use cases for this component is to “mark” something on the map, once the desired location is within the view. Again, there are no APIs to provide this and to implement this fucntionality, you have to override/extend and draw the bitmap yourself:

MapField mf = new MapField() {
        protected void paint(Graphics graphics) {
                XYPoint center = new XYPoint();
                convertWorldToField(getCoordinates(), center);
                Bitmap bitmap = Bitmap.getBitmapResource("house.png");
                graphics.drawBitmap(center.x, center.y, bitmap.getWidth(), bitmap.getHeight(), bitmap, 0, 0);

I also found the lack of useable layout managers another source of frustration. It seems unless you want things lined up left to right or up to down, you really don’t have a lot of options unless you want to write something yourself. Keep in mind, I’m using the latest version of their APIs. Hard to believe.

Given the numerous devices (each with their own specifications), the various flavors of the OS, and the seemingly poor development model set forth by RIM, I can’t imagine having to develop and maintain so many code bases for a product. If the architecture and opportunity presented itself, I think I’d rather develop a web application to use the much improved BlackBerry browser or a third-party browser and tailor the view if necessary.

Maven Release Non-supported SCM Structures

8 02 2010

As I’ve mentioned in previous blogs (Maven Release Woes with Flat Project Structures and Maven Release with Flat Structures Revisited), the support for custom SCM structures is spotty in the release plugin. It is highly desirable to leverage the release plugin as it provides a lot of functionality we do not want to have to duplicate. By using profiles and activations, we can still leverage the release plugin with only marginal manual intervention.

First let’s review what functionality the release plugin provides:

  1. Check that there are no uncommitted changes in the sources
  2. Check that there are no SNAPSHOT dependencies
  3. Change the version in the poms from x-SNAPSHOT to a new version (you will be prompted for the versions to use)
  4. Transform the SCM information in the POM to include the final destination of the tag
  5. Run the project tests against the modified POMs to confirm everything is in working order
  6. Commit the modified POMs
  7. Tag the code in the SCM with a version name (this will be prompted for)
  8. Bump the version in the POMs to a new value y-SNAPSHOT (these values will also be prompted for)
  9. Commit the modified POMs
  10. Checkout from an SCM URL with optional tag
  11. Run the predefined Maven goals to release the project (by default, deploy site-deploy)

That’s a lot of functionality we don’t want to reproduce manually. Provided your aggregator and modules are setup in the “flat structure” I’ve used for demonstration purposes in the previous two blogs, you’re in luck.

We’re in a bit of catch-22 here when the aggregator is also used for inheritence. Our modules need the aggregator/parent to be deployed before they can be released. However, our aggregator/parent identifies all the modules to be released which ultimately fail because the release plugin doesn’t support our SCM structures! I recommend placing the modules declaration into a profile that can be activated by your CI server or developers when necessary. This will allow the user to release each module of the multi-module build individually. This will require the releaser to know the order of release to perform this correctly.


Here I use a property for activation named BUILD_NUMBER as this is a property Hudson supplies to all Maven builds.

There are some minor things that must be done manually to get all this to work, but this solution is better than the alternative (nothing). Once the modules declaration is located inside a non-activated profile, here are the steps to perform a release for your mutli-module build.

  1. Release aggregator/parent project.
  2. Open all modules of the multi-module build and change the parent version from *-SNAPSHOT to the released version and check in those poms. (easy with an IDE or sed/awk).
  3. Release each module individually, in the order the reactor would perform a release.
  4. Open all modules of the multi-module build and change the parent version from the released version to the new -SNAPSHOT version.

There is also a small byproduct of the release process that might be a bug with the release plugin, I’m not sure. If you’ve structured your projects like mine, the module of a multi-module build do not specify versions allowing the inheritance mechanism to supply that metadata. Once released, the release plugin will update your POM with a version element, something that was not present before. You’ll need to also manually remove these.

For example, before release:


After release:


I’m sure a small global replace, similar to the one we used to change the parent pom version, would work to remove this unnecessary entry.

Sample Project

Maven Release with Flat Project Structures Revisited

26 01 2010

You may recall my previous blog on Maven Release Woes with Flat Project Structures where I described in detail how the supposed 2.0-beta-10-SNAPSHOT flat structure release support did not perform as expected. After some back and forth with one of the Apache developers, it has come to light that this conclusion is partially correct. The term “flat structure” is contextual and in this particular context, the Apache group’s idea of context was SCM. SCM structure is what needs to be in one of a few known structures for the release plugin to function appropriately. In the end, another JIRA ticket was created for a different use case, albeit very similar to the previously closed ticket. I’ll review what flat structure does work and some variations that don’t.

SCM Structures
First, let’s understand the structures the release plugin does support. Here is your typical nested multi-module project structure that’s supported and recommended (remember SCM structure):


Now let’s take a look at the SCM flat structure that the release plugin will support, provided you’re using at least 2.0-beta-10-SNAPSHOT:


You can see that all the projects of the multi-module project are flat, behind trunk. If you have multiple projects in your SCM, I’d recommend breaking them up with project names at the SCM root, similar to what I have done here with flat-structure-test. If not, when the release plugin goes to tag your code, it will tag everything behind trunk, not just the folders your multi-module project uses. Another thing recommended to me by the developer was to use a particular configuration, -DcommitByProject=true. For example:

emiles-macbook:release-parent emiles$ mvn release:prepare -DcommitByProject=true

With this diagramed, you must still be mindful that projects must be structured completely flat behind trunk. For testing purposes, I tried a slightly altered flat structure where release-module2 was under a random, non-project folder under trunk named somesubfolder.


This structure passed the prepare goal just fine, but when it came time to actually perform the release, it failed with an SCM checkout failure.

INFO] [INFO] Scanning for projects...
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Error building POM (may not be this project's POM).
[INFO] Project ID: unknown
[INFO] Reason: Could not find the model file '/Users/emiles/Projects/release-workspace/release-parent/target/checkout/release-module2'. for project unknown
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Trace
[INFO] org.apache.maven.reactor.MavenExecutionException: Could not find the model file '/Users/emiles/Projects/release-workspace/release-parent/target/checkout/release-module2'. for project unknown

While a specific flat project structure use case is supported, you must be mindful of the SCM structure as slight variations may not work with the release plugin. As I recommended in my previous blog, use the nested structure for multi-module projects if you can. If your project structures already exist in SCM, if possible, restructure them to model the nested structure.

Be on the lookout for my next blog article providing some suggestions on how to tackle releases if you’re stuck in an SCM structure that the release plugin does not support.

Maven Release Woes with Flat Project Structures

22 12 2009

The last two clients I’ve been on, I was asked to determine how Maven could be leveraged (better) at their site. One client was not using Maven at all and the second client was already using Maven for two years. Both clients were using Subversion and both had their own custom repository structure. The first client had a truly unique setup of how trunk/branches/tags were used and was nothing like I had seen before. There was only a single trunk, a single branches, and a single tags folder, and ALL projects were in numerous folders based on system functionality under these. The second client followed a more traditional model, with each project on the root of the Subversion repository with each project having a trunk, branches, and tags subfolders. However, both employed a “flat” project structure to make lives easier with their chosen IDEs.

Both clients had challenges that were were both related and un-related to Maven, but I was able to work through them until I got to the release process. As most of you know, the release plugin does a lot of work that would normally be done manually: incrementing versions, creating tags, updating SCM locations, checking into SCM, etc. The release plugin is particularly handy when dealing with multi-module projects. After spinning my wheels on the first client for a day or two, I came across the following Jira. There wasn’t much else on the net regarding this issue, nor did any of the Maven books I had read cover a flat structured maven project. Most described multi-module projects in your typical nested structure.

This stumped me and proved to be the ultimate decision to scrap the use of Maven at the first client (their odd SCM structure also played a part). The second client was nearly 8 months after the first; I was extremely pleased to see that a patch had been applied to trunk of the release plugin that would support flat project structures. Numerous test cases were added to cover these so I had high hopes. I pulled down the latest snapshot version, 2.0-beta-10-SNAPSHOT, and started to perform some releases on a sample project I threw together. I quickly found the issue is not yet fixed After some minor debugging, I thought maybe it was the SVN SCM provider so I tried several others. The following is an account of my findings.

Project Setup
This is the typical flat project structure, where the parent pom specifices each module like ../release-module1 and the modules specify the parent pom with a relative path ../release-parent/pom.xml.

SVN was the first provider I tried. Here is a snippet of the SCM information used by the release plugin.


While running release:prepare, we received an error regarding directories not being working copies. Here is a snippet of the error received while running release:prepare

[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Unable to commit files
Provider message:
The svn command failed.
Command output:
svn: '/Users/emiles/Projects/release-workspace' is not a working copy

CVS is the next SCM provider I tried. Here is a snippet of the SCM info from the parent POM:


Again, while running release:prepare with this provider, we receive an error similar to the one we received using SVN. Here is a snippet of the error received while running release:prepare:

[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] An error is occurred in the checkin process: /Users/emiles/Projects/release-workspace/release-module1/pom.xml was not contained in /Users/emiles/Projects/release-workspace/release-parent

I hadn’t used GIT before, but read the SVN to GIT crash course and tried this SCM provider. Here is a snippet of the SCM info from the parent POM:


As with the previous two providers, an error occurred during release:prepare. While the error message is not exactly like the one received for SVN nor CVS, it seems as though there are some similarities with being “outside a working repository”.:

[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Unable to commit files
Provider message:
The git-add command failed.
Command output:
fatal: '/Users/emiles/Projects/release-workspace/release-module1/pom.xml' is outside repository

Flat project structures are still not supported by the release plugin. The good news is that the parties responsible for the release plugin have reopened the previously closed defect, however it took them 2+ years to get this first patch together. It seems the issue is with trying to commit all the updated POMs of a flat structured multi-module project in a single transaction. If each of these were committed one at a time, there would probably be no problem. However, as with any transaction, we want all these POMs committed at once or none at all.

I love Maven, however I’m perplexed as to why this defect has been open this long without resolve. It seems it would be a major issue for numerous Maven “switchers” from other build frameworks. My advice is if you’re starting from scratch, setup your multi-module project in the expected nested structure. A lot of other tools such as Hudson and Continuum expect multi-module projects to be in a nested structure and by doing so will save yourself a lot of headache. If your project already exists and it is not in a nested structure, if it’s a small project, you might be able to live with the manual release process. If you can’t live with the manual process or if your project is a large one, you should think about re-structuring your SCM repository to provide for the expected nested project structure.

Attached are the sample projects (currently configured for SVN) and output from all three SCMs used.