Introduction
Java Swing drag and drop isn’t trivial, and descriptions of it to be found out on the net are sometimes contradictory or incomplete. Even the official Java tutorial starts by saying avoid using the underlying AWT drag and drop functionality and use the Swing TransferHandler approach, and immediately goes on to show examples using AWT.
The reason for all this confusion is there is no one size fits all with the current Swing TransferHandler functionality.
So this is the first in a series of blog posts about different use cases and approaches for drag and drop. It confused me horribly trying to learn the different ways of doing things when writing a very complete desktop specialist music manuscript editing program. This is mostly what I’ve learned, and I’m hoping publishing this will help others out and also help me learn more should those who know better chime in!
The Example Application
The use case for the application here is starting very simply as being able to drag and drop boxes around in a window. These boxes are not derived from Swing components, to show absolute clarity of how to do this if you’re using a potentially large number of graphical constructs that don’t need the Swing overheads and function. This is often the case where you need the graphical items to be moved around (and resized, as shown in a later post) and you need to interact with them in a multitude of ways, but it’s still recommended at an overall app level to leverage the Swing framework to get the app up and running and all the other goodness Swing brings, e.g. menus, actions, multi-threading model and so on.
We will build this into a complete document layout application, for example think of having a sheet of paper and you want to manually (using drag and drop) move the various areas on the page around, areas could be widgets containing text, or images or musical symbols as in the main app I maintain (drumscore.whiteware.org).
This is a reasonably complete example of using java drag and drop where you’re using custom widgets on your drawing canvas and not other Swing components.
Class Structure
DragginCustom contains the main entry point for the app, scheduling creation of all graphical resources on the Swing event dispatch thread (EDT, don’t worry about it if you haven’t heard of it before, just trust me at this stage the technique here is how to make sure your app remains thread safe and responsive).
In the constructor for DragginCustom, which is derived from a Swing JFrame, the main window container for the app, we create the resources needed i.e. the main Canvas which draws the CanvasWidgets stored in the CanvasModel, and the CanvasDragController which controls all actions on the data in the CanvasModel.
In MVC terms the Canvas is the View, the CanvasModel is the Model, and CanvasDragController is not surprisingly the Controller.
Once these objects are created the constructor exits and hands over execution to the Swing event driven model, which calls methods in the CanvasDragController as needed.
public class DragginCustom extends JFrame {
private DataFlavor widgetFlavor = new DataFlavor(CanvasWidget.class,"Draggin canvas widget");
public DragginCustom() {
setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
this.setPreferredSize(new Dimension(600,400));
CanvasModel canvasModel = new CanvasModel();
Canvas canvas = new Canvas(canvasModel);
add(canvas, BorderLayout.CENTER);
CanvasWidget widget = new CanvasWidget();
widget.setBounds(new Rectangle(10,10,50,50));
canvasModel.add(widget);
pack();
setVisible(true);
}
public static void main(String[] args) {
SwingUtilities.invokeLater(new Runnable() {
public void run() {
new DragginCustom();
}
});
}
}
Don’t be concerned about the declaration of the DataFlavor object at the top of the DragginCustom object at this stage, it’s just so there’s a globally available definition of a data type throughout the file, in a production application the various classes involved would be in separate files for ease of maintenance so a public static could be declared in the right place.
We’ll move to that in a later post in the series, right now all the classes here are private classes within the DragginCustom class.
Canvas, CanvasModel and CanvasWidget
These are the 3 classes which make up the main body of the application. As described above Canvas is the view and is responsible for rendering the data on the screen. CanvasModel is the model and is responsible for storing all the data for the app and is used by the view.
Of course the CanvasWidgets themselves could be more complex combinations of classes which make up an MVC model themselves but we’re not doing that here, at this stage, as there’s enough to take in already.
We’ll cover the controller class once we’ve taken a deeper look at the view, the model and the widgets the model stores.
Canvas
The Canvas object serves as the view and implements the Observer interface so that it can be notified by the model when data has changed and it needs to refresh the on screen display.
The update() method is invoked when the model notifies the view of data changes, and as we’re only observing the CanvasModel we were given in the Canvas constructor, we don’t need to go checking who invoked us, it’s safe to assume all we need to do is tell Swing to schedule painting the screen again, by calling JPanel.repaint().
class Canvas extends JPanel implements Observer {
CanvasModel canvasModel;
public Canvas(CanvasModel canvasModel) {
this.canvasModel = canvasModel;
canvasModel.addObserver(this);
}
protected void paintChildren(Graphics g) {
for ( CanvasWidget widget : canvasModel.getWidgetList() )
widget.paint(g);
}
public void update(Observable o, Object arg) {
repaint();
}
public CanvasModel getCanvasModel() {
return canvasModel;
}
}
For Swing aficionados note we’re not setting a null layout manager in the Canvas class, even though we’re explicitly controlling where our CanvasWidgets are stored. This is because we’re not even telling Swing about our graphical content for this pane, we do this by not using the JPanel.add method, and storing them in our separate model.
So if we’re not using Swing to do the rendering of our canvas why are we bothering to tell Swing about it? The Canvas is derived from JPanel, this is where in the class hierarchy we break from pure Swing to using our own graphical items, but there are some things we have to respect in the underlying Swing architecture.
Calling repaint() means that at the right point in the processing of events in the framework, Swing will call the JPanel standard routines for updating the display, these include paint(), paintComponent(), paintChildren(), paintBorder() and a number of other required methods to maintain the logical structure. We override the paintChildren() method and do our drawing there. By not calling super.paintChildren() in our override we effectively prevent Swing from managing the display contents and it’s all ours but within the framework which still allows things like borders should we want them.
CanvasModel
A very simple model implementation, which wraps a list of CanvasWidgets. It provides add and remove convenience methods and the getWidgetList method so the contents can be iterated, e.g. in the view for updating the screen.
The getWidgetAt method is provided to identify which of the CanvasWidgets in the list sit under a given point. This method is used exclusively at the moment by the controller to figure out if a mouse gesture was made over one of our widgets.
We provide the notifyModelChanged() method to standardise how we tell other objects who’ve registered an interest in the contents that a change to the contents has been made.
class CanvasModel extends Observable {
private List widgetList = new ArrayList();
public void add(CanvasWidget widget) {
widgetList.add(widget);
notifyModelChanged();
}
public void remove(CanvasWidget widget) {
widgetList.remove(widget);
notifyModelChanged();
}
public List getWidgetList() {
return widgetList;
}
public CanvasWidget getWidgetAt(Point p) {
for ( CanvasWidget widget : widgetList ) {
if ( widget.getBounds().contains(p) )
return widget;
}
return null;
}
public void notifyModelChanged() {
setChanged();
notifyObservers();
}
}
CanvasWidget
This is a very simple object whose only job is to draw a rectangle when and where it’s told to, and in one of two different colours, depending on whether it’s moving property is set.
class CanvasWidget {
private Rectangle bounds = new Rectangle(0,0,10,10);
private boolean moving = false;
public CanvasWidget() {
}
public Rectangle getBounds() {
return bounds;
}
public void setBounds(Rectangle bounds) {
this.bounds = bounds;
}
protected void paint(Graphics g) {
if ( isMoving() )
g.setColor(Color.LIGHT_GRAY);
else
g.setColor(Color.DARK_GRAY);
g.fillRect(bounds.x, bounds.y, bounds.width, bounds.height);
}
public boolean isMoving() {
return moving;
}
public void setMoving(boolean moving) {
this.moving = moving;
}
}
CanvasDragController and the CanvasWidgetTransferable
So that’s all pretty standard stuff up till now, this is where the magic happens, as they say at Disney World! CanvasDragController is the controller and is responsible for handling all user input directed at the view it’s registered on; interpreting that user input; and eventually manipulating the contents of the model appropriately.
A CanvasWidgetTransferable is an object created by the CanvasDragController to store all the details of the CanvasWidget it has identified to be dragged around. We’ll see why it’s important to take a copy of the widgets we want to drag and drop as we examine the way CanvasDragController works.
CanvasWidgetTransferable
This class can be regarded as a suitcase into which all the data that defines a particular CanvasWidget gets stuffed when it’s time for it to be moved.
In the constructor of the CanvasWidgetTransferable we take the bounds of the supplied CanvasWidget and store that rectangle as the bounds are effectively the only unique piece of data in every CanvasWidget.
This data can be retrieved in a number of different formats, these are identified by what AWT refers to as DataFlavors. We allow the stored bounds to be retrieved in two different formats, either as a Rectangle object, or as a string representation of the rectangle object.
class CanvasWidgetTransferable implements Transferable {
private DataFlavor[] flavorArray = { widgetFlavor, DataFlavor.stringFlavor };
private Rectangle bounds = null;
public CanvasWidgetTransferable(CanvasWidget canvasWidget) {
bounds = canvasWidget.getBounds();
}
public DataFlavor[] getTransferDataFlavors() {
return flavorArray;
}
public boolean isDataFlavorSupported(DataFlavor flavor) {
return flavor == widgetFlavor ||
flavor == DataFlavor.stringFlavor;
}
public Object getTransferData(DataFlavor flavor)
throws UnsupportedFlavorException, IOException {
if ( flavor == DataFlavor.stringFlavor )
return this.bounds.toString();
if ( flavor == widgetFlavor )
return new Rectangle(this.bounds);
return null;
}
}
So why do we allow two different ways of retrieving the data? Surely only the Rectangle object is of any use as that’s what Java will understand.
The reason for allowing the data to be obtained as a DataFlavor.stringFlavor is that it means we can drop any CanvasWidgetTransferable onto any application (yes even non-Java ones) that can have text dragged to them.
This is really useful for testing when your transferable becomes much more complicated. If things aren’t working out as you expect you can drag and drop into a text editor which will request the string format, and then you can examine exactly what’s been created in the transferable. This helps isolate if your issues are in the creation of the transferable or the retrieval.
If you’re now asking why bother, as Eclipse has a great debugger that could examine this, go ahead and try using that debugger while dragging an object around and let me know how you get on!!
CanvasDragController
The controller is where all the users inputs are interpreted, and actions taken as a result, usually against the model. This, as you can imagine, can lead to fairly lengthy logic in controllers. No exception to that rule here, the CanvasDragController is the lengthiest of the classes. As we grow the app in later posts, you’ll see how we structure the controller so it doesn’t grow to be a single monolithic, unmaintainable chunk of code.
So what does ours do, and how? Our app starts by having a single CanvasWidget in the top left of the window. If a user grabs it with the mouse and drags it elsewhere, our app immediately changes the colour of the grabbed CanvasWidget, so it appears in a lighter colour in it’s original location.
The thinking here is this allows the user to know where they’ve dragged it from until they’ve decided where to put it. Once the drag operation is started we’re in the frameworks hands until the user releases the mouse to drop the CanvasWidget at the new location. Our controller is then notified and it creates a new CanvasWidget at the dropped location and deletes the old one.
Should the drag operation be cancelled, or fail for whatever reason the controller ensures the original CanvasWidget has it’s colour reset to show it’s staying there.
Questions? Yes, like why on earth doesn’t it just change the x,y co-ordinates of the original CanvasWidget instead of all this complexity? Answer: what if the user is dropping it on a different Canvas from the one it came from? That’s one of the big plus points in using the provided drag and drop framework, it allows the objects to be dropped not only on other instances of a canvas, but also in other non-Java apps.
When the framework takes the transferable is puts it into a system clipboard and other apps or objects controllers will get told if the user releases the mouse over their components. With the transferable being formed the way it is, with DataFlavors, the other components controllers can examine if they understand the format and can use it to create a new dropped object.
The CanvasDragController starts by declaring the Canvas it’s been passed as being a component that should be watched for mouse drag gestures, and should such a gesture be detected, the CanvasDragController is to be told about it.
In the constructor it also declares the same Canvas as potentially having objects dragged to it (i.e. dropped on it), and should that ever happen, yep, the CanvasDragController wants to know about it.
The DragGestureListener interface defines methods which will be invoked if the framework detects a drag gesture has happened while over a component it’s been told to watch out for.
The DropTargetListener interface defines methods which will be invoked if the framework detects a drop gesture over a component it’s been told could have objects dropped on it.
We also implement a 3rd interface, the DragSourceListener. This defines methods which are called, at different stages in any ongoing drag operation, for the benefit of the component that the drag originated from, in case it wants to take any action. The only reason we do this in our app is to tidy up once we know a drop operation has completed elsewhere, i.e. delete the source CanvasWidget as we’ve created a new one elsewhere – because the drag operation in this example is a ‘move’, not a ‘copy’. We’ll show how to add that in the next posts.
class CanvasDragController implements DragGestureListener, DragSourceListener, DropTargetListener {
public CanvasDragController(Canvas canvas) {
// to allow drags to be initiated on the canvas
DragSource source = DragSource.getDefaultDragSource();
source.createDefaultDragGestureRecognizer(canvas, DnDConstants.ACTION_MOVE, this);
// to allow drops to happen on the canvas
DropTarget target = new DropTarget(canvas,this);
}
// DragGestureListener
public void dragGestureRecognized(DragGestureEvent dge) {
DragSource source = DragSource.getDefaultDragSource();
Canvas canvas = (Canvas) dge.getComponent();
CanvasModel canvasModel = canvas.getCanvasModel();
CanvasWidget widget = canvasModel.getWidgetAt(dge.getDragOrigin());
if ( widget != null ) {
widget.setMoving(true);
canvasModel.notifyModelChanged();
CanvasWidgetTransferable transferablePackage = new CanvasWidgetTransferable(widget);
source.startDrag(dge, DragSource.DefaultMoveDrop, transferablePackage, this);
}
}
// DragSourceListener
public void dragEnter(DragSourceDragEvent dsde) {}
public void dragOver(DragSourceDragEvent dsde) {}
public void dropActionChanged(DragSourceDragEvent dsde) {}
public void dragExit(DragSourceEvent dse) {}
public void dragDropEnd(DragSourceDropEvent dsde) {
Canvas canvas = (Canvas) dsde.getDragSourceContext().getComponent();
CanvasModel canvasModel = canvas.getCanvasModel();
Rectangle bounds = null;
try {
bounds = (Rectangle) dsde.getDragSourceContext().getTransferable().getTransferData(widgetFlavor);
} catch (UnsupportedFlavorException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
if ( dsde.getDropSuccess() ) {
if ( dsde.getDropAction() == DnDConstants.ACTION_MOVE) {
// we need to remove the source element now
Point p = new Point(bounds.x,bounds.y);
CanvasWidget widget = null;
for ( CanvasWidget searchWidget : canvasModel.widgetList ) {
if ( searchWidget.getBounds().getLocation().equals(p) )
widget = searchWidget;
}
if ( widget != null)
canvasModel.remove(widget);
}
} else {
// we need to mark it as no longer moving
Point p = new Point(bounds.x,bounds.y);
CanvasWidget widget = canvasModel.getWidgetAt(p);
widget.setMoving(false);
canvasModel.notifyModelChanged();
}
}
// DropTargetListener
public void dragEnter(DropTargetDragEvent dtde) {}
public void dragOver(DropTargetDragEvent dtde) {}
public void dropActionChanged(DropTargetDragEvent dtde) {}
public void dragExit(DropTargetEvent dte) {}
public void drop(DropTargetDropEvent dtde) {
if (!dtde.isDataFlavorSupported(widgetFlavor)) {
dtde.rejectDrop();
}
dtde.acceptDrop(DnDConstants.ACTION_MOVE);
Canvas canvas = (Canvas) dtde.getDropTargetContext().getComponent();
CanvasModel canvasModel = canvas.getCanvasModel();
Rectangle bounds = new Rectangle(0,0,10,10);
try {
bounds = (Rectangle) dtde.getTransferable().getTransferData(widgetFlavor);
} catch (UnsupportedFlavorException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
CanvasWidget widget = new CanvasWidget();
bounds.setLocation(dtde.getLocation());
widget.setBounds(bounds);
canvasModel.add(widget);
dtde.dropComplete(true);
}
}
Eagle-eyed Java coders will note that the CanvasDragController could have 4 or 5 fewer lines of code. The constructor could store the Canvas and it’s CanvasModel in variable accessible by all the methods in the class, instead of extracting them from the event parameter passed in each method.
I don’t do that in order to show that you can examine the components the events come from using the getComponent() method on each event parameter on the various drag and drop methods. I also prefer that as it localises the variables and you can see where their values originate from.
Conclusion
We now have a working app, it has a few shortcomings, try running this app on Mac OS X, then on Microsoft Windows, then Linux. The behaviour should be the same imho, i.e. an image of the object being dragged appears under the mouse pointer. The API allows a way to fix this behaviour and we’ll look at that.
Also in the real world, we’ll need to cope with the user scaling the Canvas, i.e. zooming in or out. The standard approach to that in Java is an AffineTransform but as you’ll notice above, there’s no MouseAdapter involved so we have to compensate and scale the values the DragGestureListener receives.
Another challenge is those CanvasWidgets aren’t always the same size, in fact the user may want to be able to change their size using the mouse, expectation in a regular user interface today is that you grab the corner of an object and drag it but wait … aren’t we already having Java interpret drags as being to move the object, how do we make it do both?
I’ve not finished yet, what if the user wanted to be able to select one or more CanvasWidgets and then apply any action to the subset of CanvasWidgets that are selected? Maybe drag them all, or change all their sizes, while zoomed – maybe. How would they use the mouse to select the CanvasWidgets they wanted, yep by dragging the mouse around them, once again complicating the use of the drag gesture on the Canvas.
You’ve got the bones of a fairly complete drawing app if you have all that.
What if in your app, you absolutely must have the CanvasWidgets as JPanels in their own right, and they have to be part of the component hierarchy and so they have to be added to the JPanel the Canvas is derived from, and therefore you can’t override paintChildren() either. We’ll take a look at that next, and then look at the compromises each approach brings.
Resources
Full source to all example applications available at http://github.com/alanwhite/drag-artist
Congratulations
You deserve it for reading this far, thank you for that. If this has been any use or if you think it could be made better or clearer, please either comment or tweet and let me know!
devreboot 