Java Drag and Drop: Part 3, visualising it

Introduction

In the prior posts in this series we looked at creating an app using the AWT drag and drop functionality and a functionally identical app which uses the Swing TransferHandler approach.

We saw that leveraging the Swing approach reduces the amount of code involved, however patterns emerge in the code that show how Swing is encapsulating the underlying AWT routines, i.e. we see similar code moving from a generic controller to the TransferHandler. By using Swing containers rather than our own Model/View combination, we also reduce code.

Both the applications have limited functionality, what this article does is demonstrates how to add a visual icon of the widget being transferred, so the user can see exactly what the canvas would look like where they drop the widget.

Custom Component

When using the custom component (see DragginCustom2 in the github repo), the changes are all in the controller. When we’ve decided we’re starting a drag operation we create an image of the widget.

The next challenge is the user could grab the widget anywhere within it, so we need to work out the offset of the mouse pointer from the top of the widget, so the image can be displayed correctly without it jerking to the mouse pointer location.

This offset is also used to place the widget in the right place when the user releases the mouse and drops it in place. If we didn’t take the offset into account the widget would be added with it’s top left corner where the mouse pointer is, not where it is visually on the screen.

CanvasDragController

Point to note, in the dragGestureRecognized method, once we’ve identified the widget over which the mouse drag was started, we create a BufferedImage and ask the widget to paint itself in that buffer.

Notice how the bounds location of the widget are set to 0,0 temporarily. This is because we want it to be drawn starting at the top of the buffer, not the same distance in as it is from the canvas.

Also we need to store locally the dragImageOffset when the drag is initiated. That value is needed later in the drop method but the AWT system doesn’t store it anywhere, you have to take care of that.

	class CanvasDragController implements DragGestureListener, DragSourceListener, DropTargetListener {
		
		private Point dragImageOffset = new Point();
		
		public CanvasDragController(Canvas canvas) {
			
			// to allow drags to be initiated on the canvas
			DragSource source = DragSource.getDefaultDragSource();
			source.createDefaultDragGestureRecognizer(canvas, DnDConstants.ACTION_MOVE, this);
			
			// to allow drops to happen on the canvas
			DropTarget target = new DropTarget(canvas,this);
		}

		// DragGestureListener
		public void dragGestureRecognized(DragGestureEvent dge) {
			DragSource source = DragSource.getDefaultDragSource();
			Canvas canvas = (Canvas) dge.getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();

			CanvasWidget widget = canvasModel.getWidgetAt(dge.getDragOrigin());
			if ( widget != null ) {
				Rectangle widgetBounds = widget.getBounds();
				BufferedImage dragImage = new BufferedImage(widgetBounds.width,widgetBounds.height,BufferedImage.TYPE_INT_ARGB);
				Graphics g = dragImage.getGraphics();
				Point widgetSavedLocation = widget.getBounds().getLocation();
				widgetBounds.setLocation(0, 0);
				widget.paint(g);
				widgetBounds.setLocation(widgetSavedLocation);
				dragImageOffset = new Point(widget.getBounds().x - dge.getDragOrigin().x, widget.getBounds().y - dge.getDragOrigin().y);
				widget.setMoving(true);
				canvasModel.notifyModelChanged();
				CanvasWidgetTransferable transferablePackage = new CanvasWidgetTransferable(widget);
				source.startDrag(dge, DragSource.DefaultMoveDrop, 
						dragImage,
						dragImageOffset,
						transferablePackage, this);
			}

		}

		// DragSourceListener
		public void dragEnter(DragSourceDragEvent dsde) {}
		public void dragOver(DragSourceDragEvent dsde) {}
		public void dropActionChanged(DragSourceDragEvent dsde) {}
		public void dragExit(DragSourceEvent dse) {}

		public void dragDropEnd(DragSourceDropEvent dsde) {
			Canvas canvas = (Canvas) dsde.getDragSourceContext().getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();
			
			Rectangle bounds = null;
			try {
				bounds = (Rectangle) dsde.getDragSourceContext().getTransferable().getTransferData(widgetFlavor);
			} catch (UnsupportedFlavorException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}

			if ( dsde.getDropSuccess() ) {
				if ( dsde.getDropAction() == DnDConstants.ACTION_MOVE) {
					// we need to remove the source element now
					Point p = new Point(bounds.x,bounds.y);
					CanvasWidget widget = null;
					for ( CanvasWidget searchWidget : canvasModel.widgetList ) {
						if ( searchWidget.getBounds().getLocation().equals(p) )
							widget = searchWidget;
					}
					if ( widget != null)
						canvasModel.remove(widget);
				}
			} else {
				// we need to mark it as no longer moving
				Point p = new Point(bounds.x,bounds.y);
				CanvasWidget widget = canvasModel.getWidgetAt(p);
				widget.setMoving(false);
				canvasModel.notifyModelChanged();
			}
		}

		// DropTargetListener
		public void dragEnter(DropTargetDragEvent dtde) {}
		public void dragOver(DropTargetDragEvent dtde) {}
		public void dropActionChanged(DropTargetDragEvent dtde) {}
		public void dragExit(DropTargetEvent dte) {}

		public void drop(DropTargetDropEvent dtde) {
			if (!dtde.isDataFlavorSupported(widgetFlavor)) {
				dtde.rejectDrop();
			}
			
			dtde.acceptDrop(DnDConstants.ACTION_MOVE);
			
			Canvas canvas = (Canvas) dtde.getDropTargetContext().getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();
			
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) dtde.getTransferable().getTransferData(widgetFlavor);
			} catch (UnsupportedFlavorException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}
			
			CanvasWidget widget = new CanvasWidget();
			
			Point mouseLocation = dtde.getLocation();
			Point realDropLocation = new Point(mouseLocation.x + dragImageOffset.x,mouseLocation.y + dragImageOffset.y);
			bounds.setLocation(realDropLocation);
			widget.setBounds(bounds);
			canvasModel.add(widget);
			dtde.dropComplete(true);
		}
	}

Swing Component

The changes happen in the TransferHandler, full source is available in the DragSwing2 app in the github repo. Pretty much identical logic as the custom example above but just in different places.

The same issues apply about ensuring we work out the offset from where the mouse was dragged to the top left of the component, and using that when dropping the widget on the canvas.

CanvasTransferHandler

There are some nuances to using a drag image with this approach, particularly where you can call the setDragImage method and accompanying setDragImageOffset. We call it to set the image when the transferable is created, the only other option is when Swing asks for the drag options supported by the canvas, this didn’t seem a logical choice.

Although the TransferHandler approach allows the dragImageOffset to be retrieved when the drop is taking place, i.e. in the importData method, it bizarrely has no ability to tell us where the mouse was when get a chance to specify the drag offset! So we override the exportAsDrag method to squirrel away the mouse location at the time, so it can be used later.

	class CanvasTransferHandler extends TransferHandler {	
		private Point dragStart = new Point();
		
		public int getSourceActions(JComponent c) {
			return TransferHandler.MOVE;
		}

		protected Transferable createTransferable(JComponent c) {
			for ( Component comp : c.getComponents() ) {
				if ( comp instanceof CanvasWidget ) {
					CanvasWidget canvasWidget = (CanvasWidget) comp;
					if ( canvasWidget.isMoving() ) {
						canvasWidget.setMoving(false);
						BufferedImage widgetImage = new BufferedImage(canvasWidget.getWidth(),
								canvasWidget.getHeight(),BufferedImage.TYPE_INT_ARGB);
						Graphics g = widgetImage.getGraphics();
						canvasWidget.paintAll(g);
						canvasWidget.setMoving(true);
						setDragImage(widgetImage);
						setDragImageOffset(new Point(canvasWidget.getX() - dragStart.x, canvasWidget.getY() - dragStart.y));
						return new CanvasWidgetTransferable(canvasWidget);
					}
				}
			}
			return null;
		}

		protected void exportDone(JComponent source, Transferable data,
				int action) {
			if ( action == TransferHandler.MOVE ) {
				Canvas canvas = (Canvas) source;
				for ( Component comp : source.getComponents() ) {
					if ( comp instanceof CanvasWidget ) {
						CanvasWidget canvasWidget = (CanvasWidget) comp;
						if ( canvasWidget.isMoving() ) {
							canvas.remove(canvasWidget);
							canvas.repaint();
						}
					}
				}
			}
		}

		public boolean importData(TransferSupport support) {
			if ( !canImport(support) )
				return false;

			Canvas canvas = (Canvas) support.getComponent();
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) support.getTransferable().getTransferData(widgetFlavor);
			} catch (Exception e) {
				e.printStackTrace();
				return false;
			}
			
			CanvasWidget widget = new CanvasWidget();
			Point mouseLocation = support.getDropLocation().getDropPoint();
			Point realDropLocation = new Point(mouseLocation.x + getDragImageOffset().x,mouseLocation.y + getDragImageOffset().y);
			bounds.setLocation(realDropLocation);
			widget.setBounds(bounds);
			canvas.add(widget);
			canvas.repaint();
			return true;
		}

		public boolean canImport(TransferSupport support) {
			if ( support.isDataFlavorSupported(widgetFlavor) )
				return true;
			return false;
		}

		public void exportAsDrag(JComponent comp, InputEvent e, int action) {
			dragStart = ((MouseEvent) e).getPoint();
			super.exportAsDrag(comp, e, action);
		}

	}

Resources

Full source to all example applications available at http://github.com/alanwhite/drag-artist

Congratulations

You deserve it for reading this far, thank you for that. If this has been any use or if you think it could be made better or clearer, please either comment or tweet and let me know!

Advertisements

Java Drag and Drop: Part 2, Swing widgets

Introduction

This article focuses on using the TransferHandler approach to drag and drop in a Java Swing based app. Many developers avoid the Swing TransferHandler as it’s not particularly well understood, however used appropriately it can save a lot of work. In order to compare and contrast the differences between using AWT drag and drop and Swing TransferHandler, we’re using exactly the same app as the first post in this series, and with reference to the previous point, we’re nearly 100 lines of code shorter overall – with considerably more functionality available to use which we won’t cover until a future post – keeping it working the same as the DragginCustom example at this stage.

The Example Application

The use case for the application here is starting very simply as being able to drag and drop boxes around in a window. These boxes are derived from Swing JPanel, to demonstrate fully how to do this if you’re using pure Swing components.

We will build this into a complete document layout application, for example think of having a sheet of paper and you want to manually (using drag and drop) move the various areas on the page around, areas could be widgets containing text, or images or musical symbols as in the main app I maintain (drumscore.whiteware.org).

Class Structure

DragginSwing contains the main entry point for the app, scheduling creation of all graphical resources on the Swing event dispatch thread (EDT, don’t worry about it if you haven’t heard of it before, just trust me at this stage the technique here is how to make sure your app remains thread safe and responsive).

In the constructor for DragginSwing, which is derived from a Swing JFrame, the main window container for the app, we create the resources needed i.e. the main Canvas which draws the CanvasWidgets, and the CanvasDragController which controls all actions on the CanvasWidgets on the Canvas.

In MVC terms the Canvas is the View and the Model in this case, and CanvasDragController is not surprisingly the Controller.

Once these objects are created the constructor exits and hands over execution to the Swing event driven model, which calls methods in the CanvasDragController as needed.


public class DragginCustom extends JFrame {
	private DataFlavor widgetFlavor = new DataFlavor(CanvasWidget.class,"Draggin canvas widget");

	public DragginCustom() {
		setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
		this.setPreferredSize(new Dimension(600,400));
		
		CanvasModel canvasModel = new CanvasModel();
		Canvas canvas = new Canvas(canvasModel);
		add(canvas, BorderLayout.CENTER);
		
		new CanvasDragController(canvas);
		
		CanvasWidget widget = new CanvasWidget();
		widget.setBounds(new Rectangle(10,10,50,50));
		canvasModel.add(widget);
		
		pack();
		setVisible(true);
	}
	
	public static void main(String[] args) {
		SwingUtilities.invokeLater(new Runnable() {
			public void run() {
				new DragginCustom();
			}
		});
	}
}

Don’t be concerned about the declaration of the DataFlavor object at the top of the DragginCustom object at this stage, it’s just so there’s a globally available definition of a data type throughout the file, in a production application the various classes involved would be in separate files for ease of maintenance so a public static could be declared in the right place.

We’ll move to that in a later post in the series, right now all the classes here are private classes within the DragginSwing class.

Canvas and CanvasWidget

These are the 2 classes which make up the main body of the application. In the version of this app based on custom components we had a Model to hold the widgets, however we’re leveraging the Swing component model here where the component stores the data like a model and knows how to render it like a View.

The CanvasWidget is derived from the Swing JPanel component also, in order to demonstrate how to store components within components, the standard containment hierarchy in Swing, and leverage the TransferHandler for drag and drop.

We’ll cover the controller class once we’ve taken a deeper look at the view and the widgets the model stores.

Canvas

The Canvas object serves as the view and the model, storing all the widgets as well as being responsible for the overall rendering of those widgets. One major difference with the design compared to the custom component described in the DragginCustom app in the prior article, is that Swing doesn’t always tell it’s view to update when changes happen to the components it’s storing, i.e. you can delete a component from the JPanel but you also have to explicitly tell it the data has changed and it should repaint.

	class Canvas extends JPanel {
		public Canvas() {
			setLayout(null);
		}
	}

Very simply put, as we’re leveraging more of the Swing goodness, the Canvas is a lot simpler than when dealing with customer components. All we are doing is setting a null layout manager in the Canvas class, as we’re explicitly controlling where our CanvasWidgets are placed.

CanvasWidget

This is a very simple object whose only job is to draw a rectangle when and where it’s told to, and in one of two different colours, depending on whether it’s moving property is set.

	class CanvasWidget extends JPanel {
		private boolean moving = false;
		
		public CanvasWidget() {
			setBackground(Color.DARK_GRAY);
		}
		
		public boolean isMoving() {
			return moving;
		}
		
		public void setMoving(boolean moving) {
			if ( moving )
				setBackground(Color.LIGHT_GRAY);
			else
				setBackground(Color.DARK_GRAY);
			this.moving = moving;
		}
	}

CanvasDragController, the CanvasTransferHandler and the CanvasWidgetTransferable

The CanvasDragController is responsible for handling all user input directed at the view it’s registered on; interpreting that user input; and eventually manipulating the contents of the model appropriately. It leverages the CanvasTransferHandler derived from the Swing TransferHandler for much of this, making the CanvasTransferHandler an extension of the controller functionality, but provided mostly by Swing instead of coding it ourselves.

A CanvasWidgetTransferable is an object created by the CanvasTransferHandler to store all the details of the CanvasWidget it has identified to be dragged around. We’ll see why it’s important to take a copy of the widgets we want to drag and drop as we examine the way the TransferHandler approach works.

CanvasWidgetTransferable

This class can be regarded as a suitcase into which all the data that defines a particular CanvasWidget gets stuffed when it’s time for it to be moved.

In the constructor of the CanvasWidgetTransferable we take the bounds of the supplied CanvasWidget and store that rectangle, in a production app we’ll probably want to store more than just the bounds, if it’s a container in it’s own right, i.e. if it’s derived from JPanel it’s probably because you’ve got other components within it that you’ll want to preserve and have transferred with the object.

This data can be retrieved in a number of different formats, these are identified by what AWT refers to as DataFlavors. We allow the stored bounds to be retrieved in two different formats, either as a Rectangle object, or as a string representation of the rectangle object.

	class CanvasWidgetTransferable implements Transferable {
		private DataFlavor[] flavorArray = { widgetFlavor, DataFlavor.stringFlavor };
		private Rectangle bounds = null;
		
		public CanvasWidgetTransferable(CanvasWidget canvasWidget) {
			bounds = canvasWidget.getBounds();
		}

		public DataFlavor[] getTransferDataFlavors() {
			return flavorArray;
		}

		public boolean isDataFlavorSupported(DataFlavor flavor) {
			return flavor == widgetFlavor || 
				flavor == DataFlavor.stringFlavor;
		}

		public Object getTransferData(DataFlavor flavor)
				throws UnsupportedFlavorException, IOException {
			if ( flavor == DataFlavor.stringFlavor ) 
				return this.bounds.toString();
			if ( flavor == widgetFlavor )
				return new Rectangle(this.bounds);
			return null;
		}		
	}

As per the prior post, the reason for allowing the data to be obtained as a DataFlavor.stringFlavor as well as the Rectangle we want, is that it means we can drop any CanvasWidgetTransferable onto any application (including non-Java apps) that can have text dragged to them – useful as your objects stored in the Transferable become more complicated.

The class is line for line identical between this Swing TransferHandler based app and the AWT based app – reason for this if it’s not clear is that Swing TransferHandler abstracts the underlying AWT architecture and simplifies it’s use, when used appropriately.

CanvasDragController

The controller is where all the users inputs are interpreted, and actions taken as a result, usually against the model. In our case all we’re doing is having the mouse gestures which happen while the cursor is over the Canvas directed to us, so we can initiate a Swing TransferHandler based drag and drop when appropriate.

Our app starts by having a single CanvasWidget in the top left of the window. If a user grabs it with the mouse and drags it elsewhere, our app immediately changes the colour of the grabbed CanvasWidget, so it appears in a lighter colour in it’s original location.

The thinking here is this allows the user to know where they’ve dragged it from until they’ve decided where to put it. Once we’ve told the TransferHandler to start the drag operation all interaction is controlled through our CanvasTransferHandler until the user releases the mouse.

Should the drag operation be cancelled, or fail for whatever reason the CanvasTransferHandler ensures the original CanvasWidget has it’s colour reset to show it’s staying there.

As described in the first post in the series the transferable is put onto the system clipboard so other apps or objects controllers will get told if the user drags the mouse over their components. With the transferable being formed the way it is, with DataFlavors, the other components controllers can examine if they understand the format and can use it to create a new dropped object.

The CanvasDragController starts by declaring it wants all the mouse gestures that happen over the Canvas, and that the Canvas should have the CanvasTransferHandler associated with it for all drag and drop actions.

	class CanvasDragController extends MouseAdapter {
		public CanvasDragController(Canvas canvas) {
			canvas.addMouseMotionListener(this);
			canvas.setTransferHandler(new CanvasTransferHandler());
		}

		public void mouseDragged(MouseEvent e) {
			Canvas canvas = (Canvas) e.getComponent();
			Component comp = canvas.getComponentAt(e.getPoint());
			if ( comp != null ) {
				if ( comp instanceof CanvasWidget ) {
					CanvasWidget canvasWidget = (CanvasWidget) comp;
					canvasWidget.setMoving(true);
					TransferHandler th = canvas.getTransferHandler();
					th.exportAsDrag(canvas, e, TransferHandler.MOVE);
				}
			}
		}

	}

As the CanvasTransferHandler is used, the CanvaseDragController itself is much simpler, with some of the functionality migrating to the transfer handler, and some being handled for us, by the underlying Swing classes.

CanvasTransferHandler

This is the centre-piece of the drag and drop operation, when using the Swing approach. The way to think about this is this is a two-part operation, and the methods in the TransferHandler are dedicated to each part. The first part is to export the data that’s to be transferred into a package that the system can drag around. The second part is to import that data if the user dropped it on this component.

Part 1 – Export

If a drag operation is requested on the component this is associated with, the TransferHandler is consulted to see if the drag operation is a valid one, this is determined in the getSourceActions method.

Swing uses the createTransferable method to have the contents of the object packaged for transfer. Once the drag and drop operation completes, Swing invokes the exportDone method to say everything’s done and to allow the source of the drag to do any tidy up operation which might be needed now there’s no further need for the package.

Part 2 – Import

While the mouse is being moved over the component this is associated with, the TransferHandler repeatedly calls canImport to see if we’ll accept the transfer if the user releases the mouse.

If they do drop the package here, Swing call the importData method so you can interpret the contents of the package and add it as appropriate.

	class CanvasTransferHandler extends TransferHandler {	
		public int getSourceActions(JComponent c) {
			return TransferHandler.MOVE;
		}

		protected Transferable createTransferable(JComponent c) {
			for ( Component comp : c.getComponents() ) {
				if ( comp instanceof CanvasWidget ) {
					CanvasWidget canvasWidget = (CanvasWidget) comp;
					if ( canvasWidget.isMoving() ) {
						return new CanvasWidgetTransferable(canvasWidget);
					}
				}
			}
			return null;
		}

		protected void exportDone(JComponent source, Transferable data,
				int action) {
			if ( action == TransferHandler.MOVE ) {
				Canvas canvas = (Canvas) source;
				for ( Component comp : source.getComponents() ) {
					if ( comp instanceof CanvasWidget ) {
						CanvasWidget canvasWidget = (CanvasWidget) comp;
						if ( canvasWidget.isMoving() ) {
							canvas.remove(canvasWidget);
							canvas.repaint();
						}
					}
				}
			}
		}

		public boolean importData(TransferSupport support) {
			if ( !canImport(support) )
				return false;

			Canvas canvas = (Canvas) support.getComponent();
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) support.getTransferable().getTransferData(widgetFlavor);
			} catch (Exception e) {
				e.printStackTrace();
				return false;
			}
			
			CanvasWidget widget = new CanvasWidget();
			bounds.setLocation(support.getDropLocation().getDropPoint());
			widget.setBounds(bounds);
			canvas.add(widget);
			canvas.repaint();
			return true;
		}

		public boolean canImport(TransferSupport support) {
			if ( support.isDataFlavorSupported(widgetFlavor) )
				return true;
			return false;
		}
	}

Conclusion

We now have a working app, and like the custom component based one it has a few shortcomings – try running this app on Mac OS X, then on Microsoft Windows, then Linux. The behaviour should be the same imho, i.e. an image of the object being dragged appears under the mouse pointer. The API allows a way to fix this behaviour and we’ll look at that.

Also in the real world, we’ll need to cope with the user scaling the Canvas, i.e. zooming in or out. The standard approach to that in Java is an AffineTransform but we’ll need to figure out how to apply that transform to the TransferHandler, as it only knows the actual sizes of the components in the panel – will it tell us truly when over a zoomed component? This is problem that currently there’s no answer to and a true shortcoming in the TransferHandler design.

Another challenge is those CanvasWidgets aren’t always the same size, in fact the user may want to be able to change their size using the mouse, expectation in a regular user interface today is that you grab the corner of an object and drag it and we’ll show how to integrate that with the potential to launch a Swing drag and drop from the controller.

What if the app design has to allow the user to select one or more CanvasWidgets and then apply any action to the subset of CanvasWidgets that are selected? Maybe drag them all, or change all their sizes, while zoomed. How would they use the mouse to select the CanvasWidgets they wanted, yep by dragging the mouse around them, once again complicating the use of the drag gesture on the Canvas.

You’ve got the bones of a fairly complete drawing app if you have all that.

Resources

Full source to all example applications available at http://github.com/alanwhite/drag-artist

Congratulations

You deserve it for reading this far, thank you for that. If this has been any use or if you think it could be made better or clearer, please either comment or tweet and let me know!

Java Drag and Drop: Part 1, custom widgets

Introduction

Java Swing drag and drop isn’t trivial, and descriptions of it to be found out on the net are sometimes contradictory or incomplete. Even the official Java tutorial starts by saying avoid using the underlying AWT drag and drop functionality and use the Swing TransferHandler approach, and immediately goes on to show examples using AWT.

The reason for all this confusion is there is no one size fits all with the current Swing TransferHandler functionality.

So this is the first in a series of blog posts about different use cases and approaches for drag and drop. It confused me horribly trying to learn the different ways of doing things when writing a very complete desktop specialist music manuscript editing program. This is mostly what I’ve learned, and I’m hoping publishing this will help others out and also help me learn more should those who know better chime in!

The Example Application

The use case for the application here is starting very simply as being able to drag and drop boxes around in a window. These boxes are not derived from Swing components, to show absolute clarity of how to do this if you’re using a potentially large number of graphical constructs that don’t need the Swing overheads and function. This is often the case where you need the graphical items to be moved around (and resized, as shown in a later post) and you need to interact with them in a multitude of ways, but it’s still recommended at an overall app level to leverage the Swing framework to get the app up and running and all the other goodness Swing brings, e.g. menus, actions, multi-threading model and so on.

We will build this into a complete document layout application, for example think of having a sheet of paper and you want to manually (using drag and drop) move the various areas on the page around, areas could be widgets containing text, or images or musical symbols as in the main app I maintain (drumscore.whiteware.org).

This is a reasonably complete example of using java drag and drop where you’re using custom widgets on your drawing canvas and not other Swing components.

Class Structure

DragginCustom contains the main entry point for the app, scheduling creation of all graphical resources on the Swing event dispatch thread (EDT, don’t worry about it if you haven’t heard of it before, just trust me at this stage the technique here is how to make sure your app remains thread safe and responsive).

In the constructor for DragginCustom, which is derived from a Swing JFrame, the main window container for the app, we create the resources needed i.e. the main Canvas which draws the CanvasWidgets stored in the CanvasModel, and the CanvasDragController which controls all actions on the data in the CanvasModel.

In MVC terms the Canvas is the View, the CanvasModel is the Model, and CanvasDragController is not surprisingly the Controller.

Once these objects are created the constructor exits and hands over execution to the Swing event driven model, which calls methods in the CanvasDragController as needed.


public class DragginCustom extends JFrame {
	private DataFlavor widgetFlavor = new DataFlavor(CanvasWidget.class,"Draggin canvas widget");

	public DragginCustom() {
		setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
		this.setPreferredSize(new Dimension(600,400));

		CanvasModel canvasModel = new CanvasModel();
		Canvas canvas = new Canvas(canvasModel);
		add(canvas, BorderLayout.CENTER);

		CanvasWidget widget = new CanvasWidget();
		widget.setBounds(new Rectangle(10,10,50,50));
		canvasModel.add(widget);

		pack();
		setVisible(true);
	}

	public static void main(String[] args) {
		SwingUtilities.invokeLater(new Runnable() {
			public void run() {
				new DragginCustom();
			}
		});
	}
}

Don’t be concerned about the declaration of the DataFlavor object at the top of the DragginCustom object at this stage, it’s just so there’s a globally available definition of a data type throughout the file, in a production application the various classes involved would be in separate files for ease of maintenance so a public static could be declared in the right place.

We’ll move to that in a later post in the series, right now all the classes here are private classes within the DragginCustom class.

Canvas, CanvasModel and CanvasWidget

These are the 3 classes which make up the main body of the application. As described above Canvas is the view and is responsible for rendering the data on the screen. CanvasModel is the model and is responsible for storing all the data for the app and is used by the view.

Of course the CanvasWidgets themselves could be more complex combinations of classes which make up an MVC model themselves but we’re not doing that here, at this stage, as there’s enough to take in already.

We’ll cover the controller class once we’ve taken a deeper look at the view, the model and the widgets the model stores.

Canvas

The Canvas object serves as the view and implements the Observer interface so that it can be notified by the model when data has changed and it needs to refresh the on screen display.

The update() method is invoked when the model notifies the view of data changes, and as we’re only observing the CanvasModel we were given in the Canvas constructor, we don’t need to go checking who invoked us, it’s safe to assume all we need to do is tell Swing to schedule painting the screen again, by calling JPanel.repaint().

	class Canvas extends JPanel implements Observer {
		CanvasModel canvasModel;

		public Canvas(CanvasModel canvasModel) {
			this.canvasModel = canvasModel;
			canvasModel.addObserver(this);
		}

		protected void paintChildren(Graphics g) {
			for ( CanvasWidget widget : canvasModel.getWidgetList() )
				widget.paint(g);
		}

		public void update(Observable o, Object arg) {
			repaint();
		}

		public CanvasModel getCanvasModel() {
			return canvasModel;
		}

	}

For Swing aficionados note we’re not setting a null layout manager in the Canvas class, even though we’re explicitly controlling where our CanvasWidgets are stored. This is because we’re not even telling Swing about our graphical content for this pane, we do this by not using the JPanel.add method, and storing them in our separate model.

So if we’re not using Swing to do the rendering of our canvas why are we bothering to tell Swing about it? The Canvas is derived from JPanel, this is where in the class hierarchy we break from pure Swing to using our own graphical items, but there are some things we have to respect in the underlying Swing architecture.

Calling repaint() means that at the right point in the processing of events in the framework, Swing will call the JPanel standard routines for updating the display, these include paint(), paintComponent(), paintChildren(), paintBorder() and a number of other required methods to maintain the logical structure. We override the paintChildren() method and do our drawing there. By not calling super.paintChildren() in our override we effectively prevent Swing from managing the display contents and it’s all ours but within the framework which still allows things like borders should we want them.

CanvasModel

A very simple model implementation, which wraps a list of CanvasWidgets. It provides add and remove convenience methods and the getWidgetList method so the contents can be iterated, e.g. in the view for updating the screen.

The getWidgetAt method is provided to identify which of the CanvasWidgets in the list sit under a given point. This method is used exclusively at the moment by the controller to figure out if a mouse gesture was made over one of our widgets.

We provide the notifyModelChanged() method to standardise how we tell other objects who’ve registered an interest in the contents that a change to the contents has been made.


	class CanvasModel extends Observable {
		private List widgetList = new ArrayList();

		public void add(CanvasWidget widget) {
			widgetList.add(widget);
			notifyModelChanged();
		}

		public void remove(CanvasWidget widget) {
			widgetList.remove(widget);
			notifyModelChanged();
		}

		public List getWidgetList() {
			return widgetList;
		}

		public CanvasWidget getWidgetAt(Point p) {
			for ( CanvasWidget widget : widgetList ) {
				if ( widget.getBounds().contains(p) )
					return widget;
			}
			return null;
		}

		public void notifyModelChanged() {
			setChanged();
			notifyObservers();
		}
	}

CanvasWidget

This is a very simple object whose only job is to draw a rectangle when and where it’s told to, and in one of two different colours, depending on whether it’s moving property is set.

	class CanvasWidget {
	
		private Rectangle bounds = new Rectangle(0,0,10,10);
		private boolean moving = false;
		
		public CanvasWidget() {

		}

		public Rectangle getBounds() {
			return bounds;
		}

		public void setBounds(Rectangle bounds) {
			this.bounds = bounds;
		}
		
		protected void paint(Graphics g) {
			if ( isMoving() )
				g.setColor(Color.LIGHT_GRAY);
			else
				g.setColor(Color.DARK_GRAY);
			g.fillRect(bounds.x, bounds.y, bounds.width, bounds.height);
		}

		public boolean isMoving() {
			return moving;
		}

		public void setMoving(boolean moving) {
			this.moving = moving;
		}
	}

CanvasDragController and the CanvasWidgetTransferable

So that’s all pretty standard stuff up till now, this is where the magic happens, as they say at Disney World! CanvasDragController is the controller and is responsible for handling all user input directed at the view it’s registered on; interpreting that user input; and eventually manipulating the contents of the model appropriately.

A CanvasWidgetTransferable is an object created by the CanvasDragController to store all the details of the CanvasWidget it has identified to be dragged around. We’ll see why it’s important to take a copy of the widgets we want to drag and drop as we examine the way CanvasDragController works.

CanvasWidgetTransferable

This class can be regarded as a suitcase into which all the data that defines a particular CanvasWidget gets stuffed when it’s time for it to be moved.

In the constructor of the CanvasWidgetTransferable we take the bounds of the supplied CanvasWidget and store that rectangle as the bounds are effectively the only unique piece of data in every CanvasWidget.

This data can be retrieved in a number of different formats, these are identified by what AWT refers to as DataFlavors. We allow the stored bounds to be retrieved in two different formats, either as a Rectangle object, or as a string representation of the rectangle object.


	class CanvasWidgetTransferable implements Transferable {
		
		private DataFlavor[] flavorArray = { widgetFlavor, DataFlavor.stringFlavor };
		private Rectangle bounds = null;
		
		public CanvasWidgetTransferable(CanvasWidget canvasWidget) {
			bounds = canvasWidget.getBounds();
		}

		public DataFlavor[] getTransferDataFlavors() {
			return flavorArray;
		}

		public boolean isDataFlavorSupported(DataFlavor flavor) {
			return flavor == widgetFlavor || 
				flavor == DataFlavor.stringFlavor;
		}

		public Object getTransferData(DataFlavor flavor)
				throws UnsupportedFlavorException, IOException {
			if ( flavor == DataFlavor.stringFlavor ) 
				return this.bounds.toString();
			if ( flavor == widgetFlavor )
				return new Rectangle(this.bounds);
			return null;
		}
		
	}

So why do we allow two different ways of retrieving the data? Surely only the Rectangle object is of any use as that’s what Java will understand.

The reason for allowing the data to be obtained as a DataFlavor.stringFlavor is that it means we can drop any CanvasWidgetTransferable onto any application (yes even non-Java ones) that can have text dragged to them.

This is really useful for testing when your transferable becomes much more complicated. If things aren’t working out as you expect you can drag and drop into a text editor which will request the string format, and then you can examine exactly what’s been created in the transferable. This helps isolate if your issues are in the creation of the transferable or the retrieval.

If you’re now asking why bother, as Eclipse has a great debugger that could examine this, go ahead and try using that debugger while dragging an object around and let me know how you get on!!

CanvasDragController

The controller is where all the users inputs are interpreted, and actions taken as a result, usually against the model. This, as you can imagine, can lead to fairly lengthy logic in controllers. No exception to that rule here, the CanvasDragController is the lengthiest of the classes. As we grow the app in later posts, you’ll see how we structure the controller so it doesn’t grow to be a single monolithic, unmaintainable chunk of code.

So what does ours do, and how? Our app starts by having a single CanvasWidget in the top left of the window. If a user grabs it with the mouse and drags it elsewhere, our app immediately changes the colour of the grabbed CanvasWidget, so it appears in a lighter colour in it’s original location.

The thinking here is this allows the user to know where they’ve dragged it from until they’ve decided where to put it. Once the drag operation is started we’re in the frameworks hands until the user releases the mouse to drop the CanvasWidget at the new location. Our controller is then notified and it creates a new CanvasWidget at the dropped location and deletes the old one.

Should the drag operation be cancelled, or fail for whatever reason the controller ensures the original CanvasWidget has it’s colour reset to show it’s staying there.

Questions? Yes, like why on earth doesn’t it just change the x,y co-ordinates of the original CanvasWidget instead of all this complexity? Answer: what if the user is dropping it on a different Canvas from the one it came from? That’s one of the big plus points in using the provided drag and drop framework, it allows the objects to be dropped not only on other instances of a canvas, but also in other non-Java apps.

When the framework takes the transferable is puts it into a system clipboard and other apps or objects controllers will get told if the user releases the mouse over their components. With the transferable being formed the way it is, with DataFlavors, the other components controllers can examine if they understand the format and can use it to create a new dropped object.

The CanvasDragController starts by declaring the Canvas it’s been passed as being a component that should be watched for mouse drag gestures, and should such a gesture be detected, the CanvasDragController is to be told about it.

In the constructor it also declares the same Canvas as potentially having objects dragged to it (i.e. dropped on it), and should that ever happen, yep, the CanvasDragController wants to know about it.

The DragGestureListener interface defines methods which will be invoked if the framework detects a drag gesture has happened while over a component it’s been told to watch out for.

The DropTargetListener interface defines methods which will be invoked if the framework detects a drop gesture over a component it’s been told could have objects dropped on it.

We also implement a 3rd interface, the DragSourceListener. This defines methods which are called, at different stages in any ongoing drag operation, for the benefit of the component that the drag originated from, in case it wants to take any action. The only reason we do this in our app is to tidy up once we know a drop operation has completed elsewhere, i.e. delete the source CanvasWidget as we’ve created a new one elsewhere – because the drag operation in this example is a ‘move’, not a ‘copy’. We’ll show how to add that in the next posts.

	class CanvasDragController implements DragGestureListener, DragSourceListener, DropTargetListener {
		public CanvasDragController(Canvas canvas) {
			
			// to allow drags to be initiated on the canvas
			DragSource source = DragSource.getDefaultDragSource();
			source.createDefaultDragGestureRecognizer(canvas, DnDConstants.ACTION_MOVE, this);
			
			// to allow drops to happen on the canvas
			DropTarget target = new DropTarget(canvas,this);
		}

		// DragGestureListener
		public void dragGestureRecognized(DragGestureEvent dge) {
			DragSource source = DragSource.getDefaultDragSource();
			Canvas canvas = (Canvas) dge.getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();

			CanvasWidget widget = canvasModel.getWidgetAt(dge.getDragOrigin());
			if ( widget != null ) {
				widget.setMoving(true);
				canvasModel.notifyModelChanged();
				CanvasWidgetTransferable transferablePackage = new CanvasWidgetTransferable(widget);
				source.startDrag(dge, DragSource.DefaultMoveDrop, transferablePackage, this);
			}

		}

		// DragSourceListener
		public void dragEnter(DragSourceDragEvent dsde) {}
		public void dragOver(DragSourceDragEvent dsde) {}
		public void dropActionChanged(DragSourceDragEvent dsde) {}
		public void dragExit(DragSourceEvent dse) {}

		public void dragDropEnd(DragSourceDropEvent dsde) {
			Canvas canvas = (Canvas) dsde.getDragSourceContext().getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();
			
			Rectangle bounds = null;
			try {
				bounds = (Rectangle) dsde.getDragSourceContext().getTransferable().getTransferData(widgetFlavor);
			} catch (UnsupportedFlavorException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}

			if ( dsde.getDropSuccess() ) {
				if ( dsde.getDropAction() == DnDConstants.ACTION_MOVE) {
					// we need to remove the source element now
					Point p = new Point(bounds.x,bounds.y);
					CanvasWidget widget = null;
					for ( CanvasWidget searchWidget : canvasModel.widgetList ) {
						if ( searchWidget.getBounds().getLocation().equals(p) )
							widget = searchWidget;
					}
					if ( widget != null)
						canvasModel.remove(widget);
				}
			} else {
				// we need to mark it as no longer moving
				Point p = new Point(bounds.x,bounds.y);
				CanvasWidget widget = canvasModel.getWidgetAt(p);
				widget.setMoving(false);
				canvasModel.notifyModelChanged();
			}
		}

		// DropTargetListener
		public void dragEnter(DropTargetDragEvent dtde) {}
		public void dragOver(DropTargetDragEvent dtde) {}
		public void dropActionChanged(DropTargetDragEvent dtde) {}
		public void dragExit(DropTargetEvent dte) {}

		public void drop(DropTargetDropEvent dtde) {
			if (!dtde.isDataFlavorSupported(widgetFlavor)) {
				dtde.rejectDrop();
			}
			
			dtde.acceptDrop(DnDConstants.ACTION_MOVE);
			
			Canvas canvas = (Canvas) dtde.getDropTargetContext().getComponent();
			CanvasModel canvasModel = canvas.getCanvasModel();
			
			Rectangle bounds = new Rectangle(0,0,10,10);
			try {
				bounds = (Rectangle) dtde.getTransferable().getTransferData(widgetFlavor);
			} catch (UnsupportedFlavorException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}
			
			CanvasWidget widget = new CanvasWidget();
			bounds.setLocation(dtde.getLocation());
			widget.setBounds(bounds);
			canvasModel.add(widget);
			dtde.dropComplete(true);
		}
	}

Eagle-eyed Java coders will note that the CanvasDragController could have 4 or 5 fewer lines of code. The constructor could store the Canvas and it’s CanvasModel in variable accessible by all the methods in the class, instead of extracting them from the event parameter passed in each method.

I don’t do that in order to show that you can examine the components the events come from using the getComponent() method on each event parameter on the various drag and drop methods. I also prefer that as it localises the variables and you can see where their values originate from.

Conclusion

We now have a working app, it has a few shortcomings, try running this app on Mac OS X, then on Microsoft Windows, then Linux. The behaviour should be the same imho, i.e. an image of the object being dragged appears under the mouse pointer. The API allows a way to fix this behaviour and we’ll look at that.

Also in the real world, we’ll need to cope with the user scaling the Canvas, i.e. zooming in or out. The standard approach to that in Java is an AffineTransform but as you’ll notice above, there’s no MouseAdapter involved so we have to compensate and scale the values the DragGestureListener receives.

Another challenge is those CanvasWidgets aren’t always the same size, in fact the user may want to be able to change their size using the mouse, expectation in a regular user interface today is that you grab the corner of an object and drag it but wait … aren’t we already having Java interpret drags as being to move the object, how do we make it do both?

I’ve not finished yet, what if the user wanted to be able to select one or more CanvasWidgets and then apply any action to the subset of CanvasWidgets that are selected? Maybe drag them all, or change all their sizes, while zoomed – maybe. How would they use the mouse to select the CanvasWidgets they wanted, yep by dragging the mouse around them, once again complicating the use of the drag gesture on the Canvas.

You’ve got the bones of a fairly complete drawing app if you have all that.

What if in your app, you absolutely must have the CanvasWidgets as JPanels in their own right, and they have to be part of the component hierarchy and so they have to be added to the JPanel the Canvas is derived from, and therefore you can’t override paintChildren() either. We’ll take a look at that next, and then look at the compromises each approach brings.

Resources

Full source to all example applications available at http://github.com/alanwhite/drag-artist

Congratulations

You deserve it for reading this far, thank you for that. If this has been any use or if you think it could be made better or clearer, please either comment or tweet and let me know!