Java Desktop App Packaging Automation

Tonight I have 2 hours free. It’s time to pull together the various snippets of info and manual tasks I’ve been following to build my app that runs on Mac OS X and Windows desktops and engineer a script I can just run each time.

The tools: first javafxpackager. https://docs.oracle.com/javafx/2/deployment/packager.htm

I’ve used this partially to build the windows executable but am using manually built ant script for Mac OS and manually signing everything in the bundle messy. Let’s sort that first.

Mac OS X

My compile / edit / test cycle is in Eclipse, and when I’m happy I export a runnable jar with the required Apache libs embedded, so my starting point for packaging is the jar. Might be nice one day to have a task in Eclipse that just does the packaging, anyway no distractions tonight.

I’m using jdk 7 still on the Mac and javafxpackager is nowhere to be found. First hurdle. The hunt begins, firstly Oracle tell me it’s included in the version of the jdk I’m running, 7.0.6. Aha it’s been renamed javapackager – nope that’s not there either. Sod it, let’s update to the latest jdk 7 – not ready to go to 8 yet. Hmm, still no packager. Pokes about a bit and sees in /Library/Java/JavaVirtualMachines there’s a few versions of the sdk now including 7u71, the most recent. Checks it out and the 7u6 I think I’m running and lo and behold there’s javafxpackager in both. OK so where is Mac OS X picking the SDK up from, or more importantly where _should_ it be picking it up from.

And voila more learnings! Short answer is ensure JAVA_HOME is set preferably using /usr/libexec/java_home to choose. Now javac reports it’s using the jdk I just installed. OK but still no javafxpackager in the default path. Henceforth it shall be known as $JAVA_HOME/bin/javafxpackager ….

Back to the job in hand, how do I go from a runnable jar to a signed .app with an embedded jre and a suitably signed installer pkg? javafxpackager -deploy appears to be the way, or it’s equivalent using ant, which might be more extensible and usable going forward. $JAVA_HOME/lib/ant-javafx.jar is your friend at this point. 30 minutes reading and researching later I think the 2 hours tonight isn’t going to be enough …..

Prototyping using the command line, the following generated a usable .app

javafxpackager -deploy -srcfiles ./DrumScoreEditor-Community-1.00.jar -outdir ./outdir -outfile DrumScoreEditor -native image -appclass org.whiteware.DrumScoreEditor -name Drum\ Score\ Editor

Seemed to strip out the Quicktime libraries that Apple reject your app store submission for, but can’t find a way to get it to sign everything for me yet, nor produce a .pkg (signed or otherwise). Using -native installer produced a well put together dmg with a typical ‘drag to /Applications’ image – nice but doesn’t help for the app store.

More when I have another 2 hours …..

Advertisements

Home Lab – Ruby on Rails Web Server on Ubuntu 14.04.1

Objective is to end up with a reusable template of a VM that can be cloned for testing ruby on rails app deployment, upgrades etc. This post builds Ubuntu 14.04 LTS, nginx as the web server, passenger as the app server and mysql for the database. This is more of a memo for me, hence not explaining every single command.

Linux

Download server image from Ubuntu, http://www.ubuntu.com/download/server and set up your VM according to cloud providers process, or local vmware / fusion etc.

Get yourself ssh access https://help.ubuntu.com/community/SSH/OpenSSH/Configuring to install the ssh daemon, then copy your public key file to authorized_keys on the new server. Much easier than console access only from this point!

sudo apt-get update
sudo apt-get install openssh-server
sudo apt-get install avahi-daemon avahi-discover

Name Services

If you’re using Fusion or some such local virtual machine technology that allocates IP addresses using DHCP, get some naming going to make it easier to connect to your VM. Enable bonjour protocols using sudo apt-get install avahi-daemon avahi-discover. Change the hostname, sudo vi /etc/hostname and sudo vi /etc/hosts to change 127.0.1.1 entry then reboot, eg use a single hostname such as labweb1

BROKEN – cannot set up multiple local virtual hosts using avahi, mimicking how this is done in regular DNS to allow nginx server blocks (or apache virtual hosts) to work. Airtonix has scripts but couldn’t get them to work

SSH Access

With name services now set up,  Get yourself ssh access  – copy your public key file to .ssh/authorized_keys on the new server. Much easier than console access only from this point!

Web Server

Install nginx https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-14-04-lts

sudo apt-get install nginx

Ruby

sudo apt-get install git
git clone https://github.com/sstephenson/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(rbenv init -)"' >> ~/.bash_profile
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc

restart your shell

git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
sudo apt-get install build-essential libssl-dev
rbenv install 2.1.3
rbenv global 2.1.3
echo "gem: --no-document" >> ~/.gemrc

App Server

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
sudo apt-get install apt-transport-https ca-certificates

create file /etc/apt/sources.list.d/passenger.list and insert deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main

sudo chown root /etc/apt/sources.list.d/passenger.list
sudo chmod 600 /etc/apt/sources.list.d/passenger.list
sudo apt-get update
sudo apt-get install nginx-extras passenger

Edit /etc/nginx/nginx.conf and uncomment passenger_root and passenger_ruby

Note this will mean passenger launches with the default ruby installed by ubuntu in /usr/bin/ruby – this is OK!

Edit /etc/nginx/nginx.conf to look similar to this:

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    root /var/www/<INSERT YOUR RAILS APP NAME>/current/public;
    passenger_ruby /Users/alanwhite/.rbenv/versions/2.1.3/bin/ruby
    passenger_enabled on;
    passenger_set_cgi_param SECRET_KEY_BASE "blah whatever it was in your ruby on rails app";
}

This will ensure passenger uses your installed ruby version and not the system installed one. There’s still a flaw here in that to change ruby version you have to edit this file as root.

Database

Going to implement a mysql database here too, it’s really a web server, but one of the application use cases I need to experiment with relies on a mysql database local to the web server, rather than on a separate (virtual or otherwise) machine.

sudo apt-get install mysql-server

I use the password labsql1 usually for the mysql root user. This is a transient lab that gets torn down.

sudo mysql_install_db

Might be a transient lab but let’s get rid of the defaults.

sudo mysql_secure_installation

Now we have nginx, passenger and mysql running but haven’t deployed an app to it. Probably a good time to stop and use your virtualisation managers clone or snapshot facility, whatever works best, to create a known starting point for testing apps.

One of the areas I want to investigate further – needing all the dev libraries to install gems – seems wrong

sudo apt-get install libmysqlclient-dev

Prepare For Deployment

I’d like to deploy my test app to /var/www/my_app_name. Unfortunately Ubuntu 14 LTS doesn’t allow package deployers to point anything there so the installed nginx points to /usr/share/nginx/html

sudo mkdir /var/www
sudo chgrp www-data /var/www
sudo chmod 775 /var/www

Don’t forget to put you git/bitbucket credentials for deploy on the new server if you’re using a capistrano recipe that pulls down from one of those repo hosts.

Ruby on Rails Installation on Ubuntu 12.04.5

Objective is to have a server where multiple ruby or rails projects could be hosted, and the app could be simply installed from github without any faffing about.

What do I mean by faffing about? Not needing root access to the server in any way in order to deploy and run your app. Sure, for some baseline system packages like apache and plugins, and to define the virtual host to apache you may need sudo but once configured, deploying your app, updates and even changing ruby and rails versions should require no system privileges.

Prepping the server

It’s been about a while, a few aborted attempts to install various things linger. First step is to tidy up. My preference these days is rbenv for ruby version management, and bundler for gem mgmt, with gems local to each app, not system installed.

RVM was already installed from a previous adventure, time to remove it

Removing rvm

The process was to run rvm implode, and remove references in the startup scripts and make sure the gem was gone. Please note I only removed the .z* files as all they had in them were the rvm references. Check yours before doing that. Also removed all references to rvm in .bashrc and .bash_profile.

Install Ruby Environment Manager

Next install rbenv from github, avoiding the linux distro’s package manager. NB If you’re behind an enterprise firewall that spoofs certificates so ssh/ssl doesn’t work, you’ll have to use one of the other documented methods for cloning from github, e.g. http and plain text user/passwords – frustrating when experts do this in the name of security and end up with plain text credentials traversing the internet.

installrbenv

Exit the shell and start a new one so the rbenv command is available, then install the ruby-build plugin which allows you to build new ruby versions when needed.

installbuild

Install Ruby

Then use rbenv to install ruby!

installruby

Set the default ruby version for this user to the newly installed one, e.g. rbenv global 2.1.3

This process is directly from the rbenv github readme, which contains all the commands in a ready to cut’n’paste format.

Install Rails

Installing rails now becomes as simple as gem install rails but you may wish to tell gem to not bother installing all the documentation if you’re intending this install to be a server only, in which case enter echo "gem: --no-document" >> ~/.gemrc beforehand.

Web & Application Server

Next we need a web server hosting an application server that knows how to run ruby on rails apps. I’d usually go for nginx as a web server, but as this linux VM is already running apache2 with a bunch of other web sites we’ll stick with that. Still need to add the application server – for this we’re installing Phusion Passenger following their instructions.

First we establish the authentication of the Phusion repos.

preprepo

Then we add them to the list of repos Ubuntu uses.

addphusion

Then a quick sudo apt-get update and ubuntu has the latest phusion passenger software ready to install.

installpass

OK, now it’s all installed, the software needs enabled in apache. Works as per the Passenger install guide using sudo a2enmod passenger and then restart apache sudo service apache2 restart

Apache Virtual Host

You’ll need a virtual host for your web site.

<VirtualHost *:80>
    ServerName blog.blahblah.org
    DocumentRoot /var/www/blog/current/public
    PassengerRuby /home/arw/.rbenv/shims/ruby
    SetEnv SECRET_KEY_BASE 1389....
    <Directory /var/www/blog/current/public>
        Allow from All
        Options -MultiViews
    </Directory>
 </VirtualHost>

Note the use of the PassengerRuby directive or it will try to use the system installed ruby, bundler etc and it will all go horribly wrong!

Next Steps

If you’ve developed your ruby on rails app, followed all the capistrano guides and deployed it, in this case to /var/www/blog, and remembered to set up your production database of choice, and ran the rake db:migrate on production ….. it just works! Honest.

Distributing an app on Mac OS X without Xcode, outside the Mac App Store

I have a java app, built with eclipse, packaged in a DMG. As it’s not “signed” in any way so Mac users have to set their Gatekeeper to allow software downloaded from anywhere to run. I kinda get Apple’s desire to control how apps are sourced to their platforms, for stability, quality etc, but unless you want to have your apps users exposed (according to Apple) you have to pay some money and add more complexity to your application packaging process.

Naturally there’s no great documentation for exactly what I want to do here, not that I can find anyway, happy to be corrected! As always, I’ll credit any articles I find as I work through this.

Apple have a documented process for taking an app built with Xcode to the point of being signed. At a high level it looks like this:

  1. Get an apple developer id (costs money btw)
  2. Request developer ID certificates
  3. Sign your application using the Developer ID Application certificate
  4. Sign the installer using the Developer ID Installer certificate

Sounds easy at this level, huh? When you look into the detail, it’s all geared toward Xcode. What follows is my adventures in trying to do this without Xcode.

Step 1 – Get an Apple Developer ID.

No problem here, you just have to pay money at https://developer.apple.com/register/index.action

Step 2 – Request Developer ID Certificates

On the developer portal there’s a Member Center page, upon which you select the Certificates, Identifiers & Profiles link.

Following the Certificates link you get a page where, after fair warning that the easiest way to get certificates is via Xcode, you find a link to upload a Certificate Signing Request from your Mac.

Now we get asked what type of certificate do we need. Apple split this into 2 categories, development and production. Looking through the brief description of all the options, I chose the Production Developer ID certificates.

There’s some verbiage at the bottom of the page about needing some Intermediate Certificates that Xcode automagically installs. Further reading says these are certificates that need to be installed in your keychain as part of the trust chain for the certs you’re about to receive. Clicking them results in a downloaded file which if you click open Keychain Access and prompts to install. Do this for each of them.

At the next screen, you get another warning that Xcode is the preferred method for requesting and installing these certificates on a Mac. Onwards we go boldly ignoring what has now escalated from being an “easier”, to the “preferred” way of doing this.

First thing to note is you can’t request both the Application and the Installer certificates at the same time.

So what this does, when pressing continue, is take you to screen that tells you how to use Keychain Access tool that comes with Mac OS X. The instructions are complete and after this step we have a Certificate Signing Request (CSR) on the Desktop, and a couple of new entries in the keychain as shown below.

Back on the developer website, selecting continue on the Next step is to upload that CSR and ask Apple to generate the certificate. Fairly painlessly this happens and the certificate is downloaded in the browser. Double-clicking prompts Keychain Access to ask which keychain to install the certificate in. Bad move, I have no idea. Options seem to be login (default), iCloud or System. Kudos to this page, as it goes through the exact process I’ve just been through but for iOS development, looks very similar and shows the cert installed in the login keychain. So let’s accept the default and crack on.

This works fine and examining the login keychain you can see the certificate is there with a 5 year lifetime.

At this stage I choose Add Another, from the options at the foot of the page the cert was downloaded from, as we still need the Installer cert.

The process is identical to the above, but choosing Installer instead of Developer cert when asked. Notably, once this is in the keychain you can see it’s only got a 1 year lifetime. Guess this means you have to revisit your installer package once a year at a minimum – not an unreasonable position to take.

OK so we now have some certs for signing our app installed in the keychain on the Mac. A quick backup now invoked on Time Machine to ensure we have a copy somewhere.

Step 3 – Sign Your Application

Now what? All the docs appear to talk about Xcode packages, archives and other terminology, then I found the Code Signing Guide in the Apple developer library.

Cutting through all the words, if you’re not looking to sandbox your app (although that’s also best practise these days), signing the app appears to work with:

$ export SIGNER="Developer ID Application: Alan White (NMZ5U4C57W)"
$ codesign -v -f -s $SIGNER Drum\ Score\ Editor.app
Drum Score Editor.app: signed bundle with Mach-O thin (x86_64) [org.whiteware.DrumScoreEditor]

Point to note: if you’ve embedded a jre inside the app, you’ll need to sign that before signing the app as a whole.

$ codesign -v -f -s $SIGNER /Users/alanwhite/Development/export/builder/Drum\ Score\ Editor.app/Contents/PlugIns/jdk1.7.0_60.jdk
/Users/alanwhite/Development/export/builder/Drum Score Editor.app/Contents/PlugIns/jdk1.7.0_60.jdk: signed bundle [com.oracle.java.7u60.jdk]
$ codesign -v -f -s $SIGNER Drum\ Score\ Editor.app
Drum Score Editor.app: signed bundle with Mach-O thin (x86_64) [org.whiteware.DrumScoreEditor]

Step 4 – Sign Your Installer

So how about building an installer and signing the install package?

$ export INSTALLSIG="3rd Party Mac Developer Installer: Alan White (NMZ5U4C57W)"
$ productbuild --component Drum\ Score\ Editor.app /Applications --sign $INSTALLSIG Drum\ Score\ Editor.pkg
productbuild: Adding component at /Users/alanwhite/Development/Drum Score/Drum Score Editor.app
productbuild: Signing product with identity "3rd Party Mac Developer Installer: Alan White (NMZ5U4C57W)" from keychain /Users/alanwhite/Library/Keychains/login.keychain
productbuild: Adding certificate "Apple Worldwide Developer Relations Certification Authority"
productbuild: Adding certificate "Apple Root CA"
productbuild: Wrote product to Drum Score Editor.pkg

Last step after this is to create the DMG to hold the installer package, as most people are used to getting software in a dmg.

hdiutil create -volname DSEditorInstaller -srcfolder Drum\ Score -ov -format UDZO DrumScore.dmg

Success

Took my prior DMG containing the unsigned app to a vanilla installed Mac running 10.8, sure enough it wouldn’t run, citing preferences by default only allowing apps signed by a validated Apple Developer ID. Took my new DMG across there, double-clicked the installer package, ran straight through, installed the app fine. Double-clicked the app in /Applications and it burst into life …. after I downloaded and installed a java runtime, it still had the old Apple 1.6 version, but at least it gave me a nice dialog box telling me it couldn’t find a runtime!

 

 

 

Java Mac OS X App Bundle Machinations

Lost 6 hours of my life tracking down this issue, and of course in the end it turns out to be my user error, although as always as a developer you challenge whether it needed to be this hard to debug.

Objective

What was I trying to achieve was to create mac app bundle to put in a dmg to distribute my java desktop app. In Eclipse, generated a Mac app bundle using the export task.

9

Into Finder, double clicked the app and nothing happens. A quick search around Console, shows:

87

 

A quick google makes suggestions like there’s no main entry point defined in the jar file

5

This however is a rat-hole, there is indeed no main entry point defined in the manifest but a few more searches reveal the package isn’t invoked using java -jar xx.jar, it’s invoked using java -cp xx.jar mainEntryPoint. So we check the Info.plist in the app bundle, and find the main class is defined. So what’s going wrong?

This article http://docs.oracle.com/javase/7/docs/technotes/guides/jweb/packagingAppsForMac.html seems to imply  a tool is needed at Java 7, so yep, great but why wouldn’t that be in eclipse already, i.e. why would eclipse claim to produce an app bundle if it did it the wrong way? Is it?  Time to go build one manually and see.

Manually creating an App Bundle, Java 7 (1.7.0_45-b18), Mac OS X 10.9.2

Downloaded appbundler-1.0.jar from https://java.net/projects/appbundler/downloads, as described in the Oracle article above and put it in a test directory.4

Set up the build.xml file from the various entries on the java.net and Oracle articles referenced above and running ant causes a no such file exception. It’s looking for an Info.plist file to be in the build directory. Odd, this isn’t mentioned anywhere!

3

Helpful: http://supunmali-myexperience.blogspot.co.uk/2012/10/jar-bundler-mac-os-x.html, resolved by setting JAVA_HOME but then I hit another problem:

1

If you’re not an experience Ant user (and I am not!), what this means is I forgot a target statement in the build.xml file, here’s the file I used at this stage which worked ….. or so I thought, it certainly allowed Ant to build the app bundle.

<?xml version="1.0" encoding="UTF-8"?>
<project name="DrumScore" default="default" basedir=".">

  <property environment="env"/>

  <taskdef name="bundleapp"
      classname="com.oracle.appbundler.AppBundlerTask"
      classpath="appBundler-1.0.jar"/>

  <target name="default">
    <bundleapp outputdirectory="."
        name="Drum Score Editor"
        displayname="Drum Score Editor"
        identifier="org.whiteware.DrumScoreEditor"
        shortversion="1.96"
        mainclassname="org/whiteware/DrumScoreEditor/main">
      <runtime dir="${env.JAVA_HOME}"/>
      <classpath
          file="${user.home}/Development/export/DrumScoreEditor-beta1.96.jar"/>
    </bundleapp>
   </target>
 </project>

And this gave me:

10

Frustratingly, trying to launch the generated .app bundle, I got exactly the same errors. What was going on here? I checked the Mac OS X Gatekeeper security level, it’s set to allow apps downloaded from anywhere so this should work!

Solution

There are many articles on various things which cause the 10810 error, from process table full to permissions and missing libraries.

What a huge diversion all these ratholes are you can go down. This is all so sketchy, returning errors meaning “unknown error”, meaningless “return code 1” in the syslog.

Bottom line is: do not try and follow the examples, work stuff out. Mine was failing because ……. I had assumed the main method name needed specific in the mainclassname statement, as is shown in the Oracle example. As soon as I made it the class name, and not the method it worked. i.e.

<?xml version="1.0" encoding="UTF-8"?>
<project name="DrumScore" default="default" basedir=".">

  <property environment="env"/>

  <taskdef name="bundleapp"
      classname="com.oracle.appbundler.AppBundlerTask"
      classpath="appBundler-1.0.jar"/>

  <target name="default">
    <bundleapp outputdirectory="."
        name="Drum Score Editor"
        displayname="Drum Score Editor"
        identifier="org.whiteware.DrumScoreEditor"
        shortversion="1.96"
        mainclassname="org/whiteware/DrumScoreEditor">
      <classpath
          file="${user.home}/Development/export/DrumScoreEditor-beta1.96.jar"/>
    </bundleapp>
   </target>
 </project>

Should I post this? After all it was my error in specifying it wrongly. Yes, because if anyone else is searching for answers in this murky, poorly documented space, made messy by Apple divorcing Java without a properly focused handing of the baton to Oracle….. this might just help!

I’m now scared … Mac, Sandbox, App Store: And the world just got infinitely more complicated …. http://www.intransitione.com/blog/take-java-to-app-store/

Java Drag and Drop: Part 3, visualising it

Introduction

In the prior posts in this series we looked at creating an app using the AWT drag and drop functionality and a functionally identical app which uses the Swing TransferHandler approach.

We saw that leveraging the Swing approach reduces the amount of code involved, however patterns emerge in the code that show how Swing is encapsulating the underlying AWT routines, i.e. we see similar code moving from a generic controller to the TransferHandler. By using Swing containers rather than our own Model/View combination, we also reduce code.

Both the applications have limited functionality, what this article does is demonstrates how to add a visual icon of the widget being transferred, so the user can see exactly what the canvas would look like where they drop the widget.

Custom Component

When using the custom component (see DragginCustom2 in the github repo), the changes are all in the controller. When we’ve decided we’re starting a drag operation we create an image of the widget.

The next challenge is the user could grab the widget anywhere within it, so we need to work out the offset of the mouse pointer from the top of the widget, so the image can be displayed correctly without it jerking to the mouse pointer location.

This offset is also used to place the widget in the right place when the user releases the mouse and drops it in place. If we didn’t take the offset into account the widget would be added with it’s top left corner where the mouse pointer is, not where it is visually on the screen.

CanvasDragController

Point to note, in the dragGestureRecognized method, once we’ve identified the widget over which the mouse drag was started, we create a BufferedImage and ask the widget to paint itself in that buffer.

Notice how the bounds location of the widget are set to 0,0 temporarily. This is because we want it to be drawn starting at the top of the buffer, not the same distance in as it is from the canvas.

Also we need to store locally the dragImageOffset when the drag is initiated. That value is needed later in the drop method but the AWT system doesn’t store it anywhere, you have to take care of that.

	class CanvasDragController implements DragGestureListener, DragSourceListener, DropTargetListener {
		
		private Point dragImageOffset = new Point();
		
		public CanvasDragController(Canvas canvas) {
			
			// to allow drags to be initiated on the canvas
			DragSource source = DragSource.getDefaultDragSource();
			source.createDefaultDragGestureRecognizer(canvas, DnDConstants.ACTION_MOVE, this);
			
			// to allow drops to happen on the canvas
			DropTarget target = new DropTarget(canvas,this);
		}

		// DragGestureListener
		public void dragGestureRecognized(DragGestureEvent dge) {
			DragSource source = DragSource.getDefaultDragSource();
			Canvas canvas = (Canvas) dge.getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();

			CanvasWidget widget = canvasModel.getWidgetAt(dge.getDragOrigin());
			if ( widget != null ) {
				Rectangle widgetBounds = widget.getBounds();
				BufferedImage dragImage = new BufferedImage(widgetBounds.width,widgetBounds.height,BufferedImage.TYPE_INT_ARGB);
				Graphics g = dragImage.getGraphics();
				Point widgetSavedLocation = widget.getBounds().getLocation();
				widgetBounds.setLocation(0, 0);
				widget.paint(g);
				widgetBounds.setLocation(widgetSavedLocation);
				dragImageOffset = new Point(widget.getBounds().x - dge.getDragOrigin().x, widget.getBounds().y - dge.getDragOrigin().y);
				widget.setMoving(true);
				canvasModel.notifyModelChanged();
				CanvasWidgetTransferable transferablePackage = new CanvasWidgetTransferable(widget);
				source.startDrag(dge, DragSource.DefaultMoveDrop, 
						dragImage,
						dragImageOffset,
						transferablePackage, this);
			}

		}

		// DragSourceListener
		public void dragEnter(DragSourceDragEvent dsde) {}
		public void dragOver(DragSourceDragEvent dsde) {}
		public void dropActionChanged(DragSourceDragEvent dsde) {}
		public void dragExit(DragSourceEvent dse) {}

		public void dragDropEnd(DragSourceDropEvent dsde) {
			Canvas canvas = (Canvas) dsde.getDragSourceContext().getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();
			
			Rectangle bounds = null;
			try {
				bounds = (Rectangle) dsde.getDragSourceContext().getTransferable().getTransferData(widgetFlavor);
			} catch (UnsupportedFlavorException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}

			if ( dsde.getDropSuccess() ) {
				if ( dsde.getDropAction() == DnDConstants.ACTION_MOVE) {
					// we need to remove the source element now
					Point p = new Point(bounds.x,bounds.y);
					CanvasWidget widget = null;
					for ( CanvasWidget searchWidget : canvasModel.widgetList ) {
						if ( searchWidget.getBounds().getLocation().equals(p) )
							widget = searchWidget;
					}
					if ( widget != null)
						canvasModel.remove(widget);
				}
			} else {
				// we need to mark it as no longer moving
				Point p = new Point(bounds.x,bounds.y);
				CanvasWidget widget = canvasModel.getWidgetAt(p);
				widget.setMoving(false);
				canvasModel.notifyModelChanged();
			}
		}

		// DropTargetListener
		public void dragEnter(DropTargetDragEvent dtde) {}
		public void dragOver(DropTargetDragEvent dtde) {}
		public void dropActionChanged(DropTargetDragEvent dtde) {}
		public void dragExit(DropTargetEvent dte) {}

		public void drop(DropTargetDropEvent dtde) {
			if (!dtde.isDataFlavorSupported(widgetFlavor)) {
				dtde.rejectDrop();
			}
			
			dtde.acceptDrop(DnDConstants.ACTION_MOVE);
			
			Canvas canvas = (Canvas) dtde.getDropTargetContext().getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();
			
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) dtde.getTransferable().getTransferData(widgetFlavor);
			} catch (UnsupportedFlavorException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}
			
			CanvasWidget widget = new CanvasWidget();
			
			Point mouseLocation = dtde.getLocation();
			Point realDropLocation = new Point(mouseLocation.x + dragImageOffset.x,mouseLocation.y + dragImageOffset.y);
			bounds.setLocation(realDropLocation);
			widget.setBounds(bounds);
			canvasModel.add(widget);
			dtde.dropComplete(true);
		}
	}

Swing Component

The changes happen in the TransferHandler, full source is available in the DragSwing2 app in the github repo. Pretty much identical logic as the custom example above but just in different places.

The same issues apply about ensuring we work out the offset from where the mouse was dragged to the top left of the component, and using that when dropping the widget on the canvas.

CanvasTransferHandler

There are some nuances to using a drag image with this approach, particularly where you can call the setDragImage method and accompanying setDragImageOffset. We call it to set the image when the transferable is created, the only other option is when Swing asks for the drag options supported by the canvas, this didn’t seem a logical choice.

Although the TransferHandler approach allows the dragImageOffset to be retrieved when the drop is taking place, i.e. in the importData method, it bizarrely has no ability to tell us where the mouse was when get a chance to specify the drag offset! So we override the exportAsDrag method to squirrel away the mouse location at the time, so it can be used later.

	class CanvasTransferHandler extends TransferHandler {	
		private Point dragStart = new Point();
		
		public int getSourceActions(JComponent c) {
			return TransferHandler.MOVE;
		}

		protected Transferable createTransferable(JComponent c) {
			for ( Component comp : c.getComponents() ) {
				if ( comp instanceof CanvasWidget ) {
					CanvasWidget canvasWidget = (CanvasWidget) comp;
					if ( canvasWidget.isMoving() ) {
						canvasWidget.setMoving(false);
						BufferedImage widgetImage = new BufferedImage(canvasWidget.getWidth(),
								canvasWidget.getHeight(),BufferedImage.TYPE_INT_ARGB);
						Graphics g = widgetImage.getGraphics();
						canvasWidget.paintAll(g);
						canvasWidget.setMoving(true);
						setDragImage(widgetImage);
						setDragImageOffset(new Point(canvasWidget.getX() - dragStart.x, canvasWidget.getY() - dragStart.y));
						return new CanvasWidgetTransferable(canvasWidget);
					}
				}
			}
			return null;
		}

		protected void exportDone(JComponent source, Transferable data,
				int action) {
			if ( action == TransferHandler.MOVE ) {
				Canvas canvas = (Canvas) source;
				for ( Component comp : source.getComponents() ) {
					if ( comp instanceof CanvasWidget ) {
						CanvasWidget canvasWidget = (CanvasWidget) comp;
						if ( canvasWidget.isMoving() ) {
							canvas.remove(canvasWidget);
							canvas.repaint();
						}
					}
				}
			}
		}

		public boolean importData(TransferSupport support) {
			if ( !canImport(support) )
				return false;

			Canvas canvas = (Canvas) support.getComponent();
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) support.getTransferable().getTransferData(widgetFlavor);
			} catch (Exception e) {
				e.printStackTrace();
				return false;
			}
			
			CanvasWidget widget = new CanvasWidget();
			Point mouseLocation = support.getDropLocation().getDropPoint();
			Point realDropLocation = new Point(mouseLocation.x + getDragImageOffset().x,mouseLocation.y + getDragImageOffset().y);
			bounds.setLocation(realDropLocation);
			widget.setBounds(bounds);
			canvas.add(widget);
			canvas.repaint();
			return true;
		}

		public boolean canImport(TransferSupport support) {
			if ( support.isDataFlavorSupported(widgetFlavor) )
				return true;
			return false;
		}

		public void exportAsDrag(JComponent comp, InputEvent e, int action) {
			dragStart = ((MouseEvent) e).getPoint();
			super.exportAsDrag(comp, e, action);
		}

	}

Resources

Full source to all example applications available at http://github.com/alanwhite/drag-artist

Congratulations

You deserve it for reading this far, thank you for that. If this has been any use or if you think it could be made better or clearer, please either comment or tweet and let me know!

Java Drag and Drop: Part 2, Swing widgets

Introduction

This article focuses on using the TransferHandler approach to drag and drop in a Java Swing based app. Many developers avoid the Swing TransferHandler as it’s not particularly well understood, however used appropriately it can save a lot of work. In order to compare and contrast the differences between using AWT drag and drop and Swing TransferHandler, we’re using exactly the same app as the first post in this series, and with reference to the previous point, we’re nearly 100 lines of code shorter overall – with considerably more functionality available to use which we won’t cover until a future post – keeping it working the same as the DragginCustom example at this stage.

The Example Application

The use case for the application here is starting very simply as being able to drag and drop boxes around in a window. These boxes are derived from Swing JPanel, to demonstrate fully how to do this if you’re using pure Swing components.

We will build this into a complete document layout application, for example think of having a sheet of paper and you want to manually (using drag and drop) move the various areas on the page around, areas could be widgets containing text, or images or musical symbols as in the main app I maintain (drumscore.whiteware.org).

Class Structure

DragginSwing contains the main entry point for the app, scheduling creation of all graphical resources on the Swing event dispatch thread (EDT, don’t worry about it if you haven’t heard of it before, just trust me at this stage the technique here is how to make sure your app remains thread safe and responsive).

In the constructor for DragginSwing, which is derived from a Swing JFrame, the main window container for the app, we create the resources needed i.e. the main Canvas which draws the CanvasWidgets, and the CanvasDragController which controls all actions on the CanvasWidgets on the Canvas.

In MVC terms the Canvas is the View and the Model in this case, and CanvasDragController is not surprisingly the Controller.

Once these objects are created the constructor exits and hands over execution to the Swing event driven model, which calls methods in the CanvasDragController as needed.


public class DragginCustom extends JFrame {
	private DataFlavor widgetFlavor = new DataFlavor(CanvasWidget.class,&quot;Draggin canvas widget&quot;);

	public DragginCustom() {
		setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
		this.setPreferredSize(new Dimension(600,400));
		
		CanvasModel canvasModel = new CanvasModel();
		Canvas canvas = new Canvas(canvasModel);
		add(canvas, BorderLayout.CENTER);
		
		new CanvasDragController(canvas);
		
		CanvasWidget widget = new CanvasWidget();
		widget.setBounds(new Rectangle(10,10,50,50));
		canvasModel.add(widget);
		
		pack();
		setVisible(true);
	}
	
	public static void main(String[] args) {
		SwingUtilities.invokeLater(new Runnable() {
			public void run() {
				new DragginCustom();
			}
		});
	}
}

Don’t be concerned about the declaration of the DataFlavor object at the top of the DragginCustom object at this stage, it’s just so there’s a globally available definition of a data type throughout the file, in a production application the various classes involved would be in separate files for ease of maintenance so a public static could be declared in the right place.

We’ll move to that in a later post in the series, right now all the classes here are private classes within the DragginSwing class.

Canvas and CanvasWidget

These are the 2 classes which make up the main body of the application. In the version of this app based on custom components we had a Model to hold the widgets, however we’re leveraging the Swing component model here where the component stores the data like a model and knows how to render it like a View.

The CanvasWidget is derived from the Swing JPanel component also, in order to demonstrate how to store components within components, the standard containment hierarchy in Swing, and leverage the TransferHandler for drag and drop.

We’ll cover the controller class once we’ve taken a deeper look at the view and the widgets the model stores.

Canvas

The Canvas object serves as the view and the model, storing all the widgets as well as being responsible for the overall rendering of those widgets. One major difference with the design compared to the custom component described in the DragginCustom app in the prior article, is that Swing doesn’t always tell it’s view to update when changes happen to the components it’s storing, i.e. you can delete a component from the JPanel but you also have to explicitly tell it the data has changed and it should repaint.

	class Canvas extends JPanel {
		public Canvas() {
			setLayout(null);
		}
	}

Very simply put, as we’re leveraging more of the Swing goodness, the Canvas is a lot simpler than when dealing with customer components. All we are doing is setting a null layout manager in the Canvas class, as we’re explicitly controlling where our CanvasWidgets are placed.

CanvasWidget

This is a very simple object whose only job is to draw a rectangle when and where it’s told to, and in one of two different colours, depending on whether it’s moving property is set.

	class CanvasWidget extends JPanel {
		private boolean moving = false;
		
		public CanvasWidget() {
			setBackground(Color.DARK_GRAY);
		}
		
		public boolean isMoving() {
			return moving;
		}
		
		public void setMoving(boolean moving) {
			if ( moving )
				setBackground(Color.LIGHT_GRAY);
			else
				setBackground(Color.DARK_GRAY);
			this.moving = moving;
		}
	}

CanvasDragController, the CanvasTransferHandler and the CanvasWidgetTransferable

The CanvasDragController is responsible for handling all user input directed at the view it’s registered on; interpreting that user input; and eventually manipulating the contents of the model appropriately. It leverages the CanvasTransferHandler derived from the Swing TransferHandler for much of this, making the CanvasTransferHandler an extension of the controller functionality, but provided mostly by Swing instead of coding it ourselves.

A CanvasWidgetTransferable is an object created by the CanvasTransferHandler to store all the details of the CanvasWidget it has identified to be dragged around. We’ll see why it’s important to take a copy of the widgets we want to drag and drop as we examine the way the TransferHandler approach works.

CanvasWidgetTransferable

This class can be regarded as a suitcase into which all the data that defines a particular CanvasWidget gets stuffed when it’s time for it to be moved.

In the constructor of the CanvasWidgetTransferable we take the bounds of the supplied CanvasWidget and store that rectangle, in a production app we’ll probably want to store more than just the bounds, if it’s a container in it’s own right, i.e. if it’s derived from JPanel it’s probably because you’ve got other components within it that you’ll want to preserve and have transferred with the object.

This data can be retrieved in a number of different formats, these are identified by what AWT refers to as DataFlavors. We allow the stored bounds to be retrieved in two different formats, either as a Rectangle object, or as a string representation of the rectangle object.

	class CanvasWidgetTransferable implements Transferable {
		private DataFlavor[] flavorArray = { widgetFlavor, DataFlavor.stringFlavor };
		private Rectangle bounds = null;
		
		public CanvasWidgetTransferable(CanvasWidget canvasWidget) {
			bounds = canvasWidget.getBounds();
		}

		public DataFlavor[] getTransferDataFlavors() {
			return flavorArray;
		}

		public boolean isDataFlavorSupported(DataFlavor flavor) {
			return flavor == widgetFlavor || 
				flavor == DataFlavor.stringFlavor;
		}

		public Object getTransferData(DataFlavor flavor)
				throws UnsupportedFlavorException, IOException {
			if ( flavor == DataFlavor.stringFlavor ) 
				return this.bounds.toString();
			if ( flavor == widgetFlavor )
				return new Rectangle(this.bounds);
			return null;
		}		
	}

As per the prior post, the reason for allowing the data to be obtained as a DataFlavor.stringFlavor as well as the Rectangle we want, is that it means we can drop any CanvasWidgetTransferable onto any application (including non-Java apps) that can have text dragged to them – useful as your objects stored in the Transferable become more complicated.

The class is line for line identical between this Swing TransferHandler based app and the AWT based app – reason for this if it’s not clear is that Swing TransferHandler abstracts the underlying AWT architecture and simplifies it’s use, when used appropriately.

CanvasDragController

The controller is where all the users inputs are interpreted, and actions taken as a result, usually against the model. In our case all we’re doing is having the mouse gestures which happen while the cursor is over the Canvas directed to us, so we can initiate a Swing TransferHandler based drag and drop when appropriate.

Our app starts by having a single CanvasWidget in the top left of the window. If a user grabs it with the mouse and drags it elsewhere, our app immediately changes the colour of the grabbed CanvasWidget, so it appears in a lighter colour in it’s original location.

The thinking here is this allows the user to know where they’ve dragged it from until they’ve decided where to put it. Once we’ve told the TransferHandler to start the drag operation all interaction is controlled through our CanvasTransferHandler until the user releases the mouse.

Should the drag operation be cancelled, or fail for whatever reason the CanvasTransferHandler ensures the original CanvasWidget has it’s colour reset to show it’s staying there.

As described in the first post in the series the transferable is put onto the system clipboard so other apps or objects controllers will get told if the user drags the mouse over their components. With the transferable being formed the way it is, with DataFlavors, the other components controllers can examine if they understand the format and can use it to create a new dropped object.

The CanvasDragController starts by declaring it wants all the mouse gestures that happen over the Canvas, and that the Canvas should have the CanvasTransferHandler associated with it for all drag and drop actions.

	class CanvasDragController extends MouseAdapter {
		public CanvasDragController(Canvas canvas) {
			canvas.addMouseMotionListener(this);
			canvas.setTransferHandler(new CanvasTransferHandler());
		}

		public void mouseDragged(MouseEvent e) {
			Canvas canvas = (Canvas) e.getComponent();
			Component comp = canvas.getComponentAt(e.getPoint());
			if ( comp != null ) {
				if ( comp instanceof CanvasWidget ) {
					CanvasWidget canvasWidget = (CanvasWidget) comp;
					canvasWidget.setMoving(true);
					TransferHandler th = canvas.getTransferHandler();
					th.exportAsDrag(canvas, e, TransferHandler.MOVE);
				}
			}
		}

	}

As the CanvasTransferHandler is used, the CanvaseDragController itself is much simpler, with some of the functionality migrating to the transfer handler, and some being handled for us, by the underlying Swing classes.

CanvasTransferHandler

This is the centre-piece of the drag and drop operation, when using the Swing approach. The way to think about this is this is a two-part operation, and the methods in the TransferHandler are dedicated to each part. The first part is to export the data that’s to be transferred into a package that the system can drag around. The second part is to import that data if the user dropped it on this component.

Part 1 – Export

If a drag operation is requested on the component this is associated with, the TransferHandler is consulted to see if the drag operation is a valid one, this is determined in the getSourceActions method.

Swing uses the createTransferable method to have the contents of the object packaged for transfer. Once the drag and drop operation completes, Swing invokes the exportDone method to say everything’s done and to allow the source of the drag to do any tidy up operation which might be needed now there’s no further need for the package.

Part 2 – Import

While the mouse is being moved over the component this is associated with, the TransferHandler repeatedly calls canImport to see if we’ll accept the transfer if the user releases the mouse.

If they do drop the package here, Swing call the importData method so you can interpret the contents of the package and add it as appropriate.

	class CanvasTransferHandler extends TransferHandler {	
		public int getSourceActions(JComponent c) {
			return TransferHandler.MOVE;
		}

		protected Transferable createTransferable(JComponent c) {
			for ( Component comp : c.getComponents() ) {
				if ( comp instanceof CanvasWidget ) {
					CanvasWidget canvasWidget = (CanvasWidget) comp;
					if ( canvasWidget.isMoving() ) {
						return new CanvasWidgetTransferable(canvasWidget);
					}
				}
			}
			return null;
		}

		protected void exportDone(JComponent source, Transferable data,
				int action) {
			if ( action == TransferHandler.MOVE ) {
				Canvas canvas = (Canvas) source;
				for ( Component comp : source.getComponents() ) {
					if ( comp instanceof CanvasWidget ) {
						CanvasWidget canvasWidget = (CanvasWidget) comp;
						if ( canvasWidget.isMoving() ) {
							canvas.remove(canvasWidget);
							canvas.repaint();
						}
					}
				}
			}
		}

		public boolean importData(TransferSupport support) {
			if ( !canImport(support) )
				return false;

			Canvas canvas = (Canvas) support.getComponent();
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) support.getTransferable().getTransferData(widgetFlavor);
			} catch (Exception e) {
				e.printStackTrace();
				return false;
			}
			
			CanvasWidget widget = new CanvasWidget();
			bounds.setLocation(support.getDropLocation().getDropPoint());
			widget.setBounds(bounds);
			canvas.add(widget);
			canvas.repaint();
			return true;
		}

		public boolean canImport(TransferSupport support) {
			if ( support.isDataFlavorSupported(widgetFlavor) )
				return true;
			return false;
		}
	}

Conclusion

We now have a working app, and like the custom component based one it has a few shortcomings – try running this app on Mac OS X, then on Microsoft Windows, then Linux. The behaviour should be the same imho, i.e. an image of the object being dragged appears under the mouse pointer. The API allows a way to fix this behaviour and we’ll look at that.

Also in the real world, we’ll need to cope with the user scaling the Canvas, i.e. zooming in or out. The standard approach to that in Java is an AffineTransform but we’ll need to figure out how to apply that transform to the TransferHandler, as it only knows the actual sizes of the components in the panel – will it tell us truly when over a zoomed component? This is problem that currently there’s no answer to and a true shortcoming in the TransferHandler design.

Another challenge is those CanvasWidgets aren’t always the same size, in fact the user may want to be able to change their size using the mouse, expectation in a regular user interface today is that you grab the corner of an object and drag it and we’ll show how to integrate that with the potential to launch a Swing drag and drop from the controller.

What if the app design has to allow the user to select one or more CanvasWidgets and then apply any action to the subset of CanvasWidgets that are selected? Maybe drag them all, or change all their sizes, while zoomed. How would they use the mouse to select the CanvasWidgets they wanted, yep by dragging the mouse around them, once again complicating the use of the drag gesture on the Canvas.

You’ve got the bones of a fairly complete drawing app if you have all that.

Resources

Full source to all example applications available at http://github.com/alanwhite/drag-artist

Congratulations

You deserve it for reading this far, thank you for that. If this has been any use or if you think it could be made better or clearer, please either comment or tweet and let me know!