The Big Move II: Java Deployment Automation with javafx-gradle

Before cracking on to step 2 in the master plan, where we rewrite the app in javafx instead of swing, it might be possible to tidy up how I build releases for the users. Currently I take the jar (previously created by eclipse and now by gradle) and then run the javabuilder command from java 1.8 with a whole bunch of parameters to create the install package for each platform, e.g. for mac it’s this:

#!/bin/bash
export JAVA_HOME=`/usr/libexec/java_home`
export JP=$JAVA_HOME/bin/javapackager
CMD="$JP -deploy -srcfiles ./DrumScoreEditor-2.00.jar -outdir ./outdir -outfile DrumScoreEditor -native installer -appclass org.whiteware.DrumScoreEditor -name Drum\ Score\ Editor -Bicon=Drum\ Score\ Editor.icns"
eval $CMD

Run it and you get:

➜ builder.test ./build.sh
No base JDK. Package will use system JRE.
Building DMG package for Drum Score Editor
Result DMG installer for Drum Score Editor: /Users/alanwhite/Development/export/builder.test/./outdir/bundles/Drum Score Editor-1.0.dmg
Building PKG package for Drum Score Editor
Building Mac App Store Bundle for Drum Score Editor

This results in a great dmg in outdir/bundles that has the usual drag to install experience for mac. It’s also signed, and for the life of me I have no recollection how it gets hold of my developer certificates from the keychain, probably a smart default/convention (on the ToDo list to investigate).

What I’d really like is to have gradle do all this for me, without me trying to remember each time what I need to do for Mac and Windows. This is what the plugin at https://bitbucket.org/shemnon/javafx-gradle/overview promises, I think! This is what we’re discovering here, don’t want to gripe as it may be my lack of context but documentation seems very sparse.

First step was to download the 32MB repository and investigate the contents. Then following the readme and building a couple of the samples, it all seemed to make sense. What is really useful is the FullyExpressed sample, which shows most of the configuration you can specify if the built in defaults (called convention these days) aren’t suitable.

Then, when adding the plugin to my test gradle project for Drum Score Editor, it got messy. So the only way I could see to integrate is was to copy the file javafx.plugin into the gradle project directory and reference it in the build.gradle using plugin from: syntax. Once through this, I was simply not understanding the errors encountered when trying a gradle build, it seemed to be a clash between jar files / directives in the build.gradle. To save you the same pain, this is the point I realised that if I add the javafx plugin, I had to remove the java plugin from the build.gradle. From there, once I’d specified as a minimum the mainClass in the javafx closure in build.gradle, I could at least get a complete successful compile and deploy cycle, i.e. it produced the mac app, dmg, pkg etc. Short lived success though – running the app gave:

LSOpenURLsWithRole() failed with error -10810 for the file /Users/alanwhite/Development/gradletest/build/distributions/DrumScoreEditor.app.

I remember in the dim and distant past dealing with such obscure errors when trying to figure out in the bad old days, prior to javapackager, how to create the mac app. This one appears to say it can’t find a java runtime, but I learned a while back that there’s lots of misleading clues in this space. Turns out I’d mistyped the mainClass in the build.gradle – only telling you here in case anyone googles the error and maybe avoids a wasted hour tracking it down!

Time to move the last pieces of functionality explicitly stated in the old build scripts above, the name of the app is fairly simple, that’s a directive you can see in the final build.gradle below, however icons require some explanation, all became clear when I found this blogpost from the author of the javafx-plugin http://speling.shemnon.com/blog/2013/01/28/adding-icons-with-the-gradle-javafx-plugin/. Don’t be put off by the age of some of these posts btw (it’s 2015 I’m writing this), I’ve seen comments about javafx-plugin not being maintained. This will be a problem when it stops working, right now, I’m getting close to liking it by this point!

Having found my original icons that made the iconset in use on the Mac builds, after copying them into src/deploy/package and prefixing them with shortcut_, a gradle build generated the icns needed and buried them in the app.

The last piece of customisation is the developer id for signing the app. Not sure how the prior method works, but I think now it works by providing my name (as specified in the developer cert), by concatenating with the well known strings Apple use, it pulled it out of the keychain. Very happy with the progress here, next – integrating the Windows build.

Windows

First the good news, taking all the work performed on the Mac, adding to a git repo, pushing and pulling down to the windows machine and “it just worked”. gradlew did what is was supposed to in loading the correct version of gradle etc – really useful feature.

The problem was my build was failing when trying to build the installer. I learnt a lesson here after an hour or so poking around the web and trying different things. Important lesson: if something doesn’t work use “gradlew build –debug”. The output is verbose but all I had to work on before this was error 2 on exec of iscc.exe. Nice, huh? With the debug output I could see that iscc.exe was complaining the icon being used for setup was too large.

A bit more digging and it looks like javafx-gradle combines all the different sized pngs in src/deploy/package into a multi-layered ico file so windows can choose the best fit resolution. Useful except if you’ve got the full complement of sizes required for a full Mac iconset, the generated ico is bigger than the 100Kb that Inno Setup supports (taken from the source code on github).

By trial and many errors, I thought the best solution I could come up with for maintaining the conventional directory structures and minimal customisation to the build.gradle file was to use the icon configuration statement and put it inside the windows platform specific section, which overrides the convention if invoked. That way there’s no need to complicate the Mac build with additional statements, nor compromise on file locations.

This however wasn’t actually why it worked, I’d clumsily also deleted a 512×512 png at the same time as it turns out from checking the javafx-plugin source code that the icons statement doesn’t apply within the platform specific scope statements. Grr, controlled testing needed.

apply plugin: 'eclipse'
apply from: 'javafx.plugin'

dependencies {
    compile fileTree(dir: 'libs', include: ['*.jar'])
}

sourceSets {
    main {
        resources {
            srcDir 'src/main/java'
        }
    }
}

javafx {
    appName 'Drum Score Editor'
    mainClass 'org.whiteware.DrumScoreEditor'

    profiles {
        macosx {
            bundleArguments = [
                'mac.signing-key-user-name' : 'Alan White'
            ]
        } 
    }
}
Advertisements

The Big Move

I’d like to have Android and iOS versions of Drum Score Editor, I’ve been asked many times. To do this, the recommended way according to each of the vendors of the platforms (Google, Apple, Microsoft) is to buy and learn their proprietary implementation languages to get the best seamless experience, completely rewrite your app and have it feel like it belongs on their platform, naturally. Drum Score Editor is 25,000 lines of Java, I’m not rewriting that several times!

  • Mac OS X: learn a language called Swift, the Xcode IDE, the Mac OS APIs, the Mac App Store, proprietary licensing and in-app purchase technology
  • iOS: as above, but with a bunch of different underlying iOS APIs ……
  • Microsoft Windows: a different language, C++, different Windows APIs, a different App Store, licensing and purchasing technologies
  • Android: well it’s written in Java, but not as we know it Jim. A different build environment, yet another set of app store and purchasing technologies

Unfortunately that means lots of different skills, and lot’s of code rewriting multiple times; skills I can’t afford to employ or learn, and time I’d lose for little functional progress as Drum Score Editor is my hobby, not some multi-million dollar IT conglomerate.

However, there’s new technologies becoming available from the open source world that might make the concept of one piece of software for all platforms possible. Drum Score Editor is written in a language called Java and it’s user interface built using the Swing libraries. There’s a new way of writing desktop apps in Java called JavaFX. Once written in JavaFX, you can use various technologies to make your app installable, and runnable on Mac OS X, Windows, iPads and Android tablets. Some work will need to be done for each platform to make the app feel natural, but the bulk of the effort remains common.

Sidebar: I don’t think I’ll ever get away from the differing app stores and technologies, I do like the reach the app store concept gives, e.g. the free 1.97 version of Drum Score packaged for the Mac App Store had 1,077 downloads. Neat, but to replicate on all platforms, and provide an in-app purchase for the studio workflows is a lot of work.

Looking into the technologies needed to get closer to having the bulk of Drum Score’s source code being the same across all platforms and to reduce the amount of effort involved in packaging and releasing it, the first thing I need is a cross-platform build system. Gradle is touted as the way to go with the best integrations for Android and iOS, as well as being designed with Java in mind in the first place. Currently I use Eclipse’s built in build mechanism to produce an executable jar, and then use the javabuilder tool to create installers for Mac and Windows. Gradle appears to consume all of that, and integrate with Eclipse.

So ….. the plan!
Step 1: convert to Gradle, ensuring I can retain the Eclipse IDE, source code control in Git, and build Mac and Windows packages as before.
Step 2: convert Drum Score Editor to JavaFX, leveraging the development and packaging system in step 1, producing Mac and Windows packages
Step 3: produce iOS package, and see how usable it is & introduce platform specific code in Drum Score Editor for the iOS platform
Step 4: repeat step 3 for Android

Step 1: Convert to gradle
A little tricky figuring it out. There’s lot’s of documentation but it doesn’t seem to be complete, it’s hard to pick through it, and in some cases various stack exchange answers were the only source. Nevertheless if you’re used to working open source you’ll be used to picking around for the ultimate truths.

I wanted to leverage the convention over configuration paradigm as much as possible, without changing actual program source code.

My settings.gradle file looks like this:

rootProject.name = 'DrumScoreEditor'

This ensures the produced jar file with the application in has the same name, rather than the name of the directory you’ve put the code in to build it.

The step 1 build.grade looks like this so far:

apply plugin: 'java'
apply plugin: 'eclipse'

dependencies {
     compile fileTree(dir: 'libs', include: ['*.jar'])
}

sourceSets {
     main {
          resources {
               srcDir 'src/main/java'
          }
     }
}

jar {
     from configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }
     manifest {
          attributes 'Main-Class': 'org.whiteware.DrumScoreEditor'
     }
}

The dependencies statement is needed because I always copy locally the exact versions of 3rd party libraries I’m using. We can argue about best practises but this is to reduce my overheads. I do a lot of development disconnected from the network so need to be self-contained and as few distractions due to changing versions of these libraries. This effectively tells the compiler where to look for the 3rd party libraries. This is one area where I had to change the project structure.

The sourcesets resources directive is because the resources, i.e. images, that the app loads are currently in a well known location in the source tree, and this ensure they’re copied over into the equivalent location in the class tree that’s bundled into the final jar. I could have tried moving them, so the convention would work but that means a source code change – could be simple as I used a public static constant for the path, not taking the distraction on just yet.

The jar statement is all about how to configure the produced jar file. Here we say bundle all 3rd party libraries in the jar, and then specify the entry point to the program.

At this stage ensure you can run your jar, after ‘gradle build’ it’s in build/libs. A ‘java -jar DrumScoreEditor.jar’ and all was eventually well, once I’d got the build.gradle file looking as above.

A quick ‘gradle eclipse’ and the necessary files were created to allow a relatively drama free import into eclipse using File->Import, Existing Project Into Workspace. Ran first time, due to having resolved all the runtime libraries first as per the build.gradle.