Processing 3 and SWT adventures

As a technology test for my attempts to build modern meteorological display system, I have investigated coupling of Eclipse 4 RCP platform with Processing 3 rendering engine and Unfoling Maps library. I'd like to share the results, although still partial, but I hope this will help someone like me :)

First, some results - a picture of my half-baked inteface, to prove this actually does anything :)

The map here is fully functional, with pan and zoom by mouse; it has a static bezier curve overlay from bezierVertex() Processing method. The resizing also works, though it is required to call the UnfoldingMap#resize method explicitly, since Processing does not provide the resizing and consequently, does not propagate the setSize events.


While there are some leads that use SWT_AWT bridge, I've decided to make the integration more seamless, using the SWT GLCanvas directly. After lurking inside Processing code, I've reached the conclusion, that main blocker is heavy coupling of PApplet with PSurface implementations, specifically the current implementation based on Newt GLWindow; and the true way is to use PGraphicsOpenGL directly. It could be more conventional to create a new PSurface implementation, I'll look into it in the future.
 
To initialize the GL system and bind together Processing component, I've created subclasses that provide initialization from GLContext directly and then provide setup that PApplet requires to run a sketch.

First, the modification of getGL method. I did not found GLAutoDrawable instance accessible in SWT context. The getGL(GLContext) here replaces getGL(GLAutoDrawable) used by PSurfaceJOGL#DrawListener:

import com.jogamp.opengl.GLContext;

import processing.opengl.PGraphicsOpenGL;
import processing.opengl.PJOGL;

public class PSWTJOGL extends PJOGL
{

 public PSWTJOGL( PGraphicsOpenGL pg )
 {
  super(pg);
 }
 public void getGL(GLContext context)
 {
  this.context = context;
  glContext = context.hashCode();
  this.setThread(Thread.currentThread());

  gl = context.getGL();
  gl2 = gl.getGL2ES2();
  try {
    gl2x = gl.getGL2();
  } catch (com.jogamp.opengl.GLException e) {
    gl2x = null;
  }
  try {
    gl3 = gl.getGL2GL3();
  } catch (com.jogamp.opengl.GLException e) {
    gl3 = null;
  }
  try {
    gl3es3 = gl.getGL3ES3();
  } catch (com.jogamp.opengl.GLException e) {
    gl3es3 = null;
  }
 }
 @Override
 public void beginRender() { super.beginRender(); }
 @Override
 public void endRender(int color) { super.endRender(color); }

}

The exposure of begin/endRender methods is due to need to recreate DrawListener functionality in my separate rendering method.

Second, copied the initGL method from the PSurfaceOpenGL and provided proper resize method:

import com.jogamp.opengl.GLCapabilities;
import com.jogamp.opengl.GLException;
import com.jogamp.opengl.GLProfile;

import processing.core.PGraphics;
import processing.opengl.PGL;
import processing.opengl.PGraphicsOpenGL;
import processing.opengl.PJOGL;

public class PGraphicsSWTOpenGL extends PGraphicsOpenGL
{

 GLProfile profile;

 public PGraphicsSWTOpenGL()
 {
  super();

  this.initGL();
 }
   // Factory method
 @Override
 protected PGL createPGL(PGraphicsOpenGL pg) {
     return new PSWTJOGL(pg);
 }

 @Override
 public void setSize(int iwidth, int iheight)
 {
  super.setSize(iwidth, iheight);
  this.resetMatrix();
  drawFramebuffer = null;
  readFramebuffer = null;
 }

 public void completeFinishedPixelTransfer()
 {
  PGraphicsOpenGL.completeFinishedPixelTransfers();
 }

 public void completeAllPixelTransfer()
 {
  PGraphicsOpenGL.completeAllPixelTransfers();
 }

 protected void initGL() {
  //  System.out.println("*******************************");
     if (profile == null) {
       if (PJOGL.profile == 1) {
         try {
           profile = GLProfile.getGL2ES1();
         } catch (GLException ex) {
           profile = GLProfile.getMaxFixedFunc(true);
         }
       } else if (PJOGL.profile == 2) {
         try {
           profile = GLProfile.getGL2ES2();
         } catch (GLException ex) {
           profile = GLProfile.getMaxProgrammable(true);
         }
       } else if (PJOGL.profile == 3) {
         try {
           profile = GLProfile.getGL2GL3();
         } catch (GLException ex) {
           profile = GLProfile.getMaxProgrammable(true);
         }
         if (!profile.isGL3()) {
           PGraphics.showWarning("Requested profile GL3 but is not available, got: " + profile);
         }
       } else if (PJOGL.profile == 4) {
         try {
           profile = GLProfile.getGL4ES3();
         } catch (GLException ex) {
           profile = GLProfile.getMaxProgrammable(true);
         }
         if (!profile.isGL4()) {
           PGraphics.showWarning("Requested profile GL4 but is not available, got: " + profile);
         }
       } else throw new RuntimeException(PGL.UNSUPPORTED_GLPROF_ERROR);
     }

     // Setting up the desired capabilities;
     GLCapabilities caps = new GLCapabilities(profile);
     caps.setAlphaBits(PGL.REQUESTED_ALPHA_BITS);
     caps.setDepthBits(PGL.REQUESTED_DEPTH_BITS);
     caps.setStencilBits(PGL.REQUESTED_STENCIL_BITS);

 //  caps.setPBuffer(false);
 //  caps.setFBO(false);

 //  pgl.reqNumSamples = PGL.smoothToSamples(graphics.smooth);
     caps.setSampleBuffers(true);
     caps.setNumSamples(PGL.smoothToSamples(this.smooth));
     caps.setBackgroundOpaque(true);
     caps.setOnscreen(true);
     ((PJOGL)pgl).setCaps(caps);
 }

}

 
Once again, complete...PixelTransfer() methods are for the main rendering method.

Input event handler. I've only provided mouse handler; I did not finish bridging the whole event system due to lack of time;

See here - http://stackoverflow.com/questions/6629160/swt-on-windows-scroll-control-at-cursor-not-focused-one - for the implementation of  Win32MouseWheelFilter

import org.eclipse.swt.SWT;
import org.eclipse.swt.opengl.GLCanvas;
import org.eclipse.swt.widgets.Event;
import org.eclipse.swt.widgets.Listener;

import com.jogamp.newt.event.InputEvent;

import processing.core.PApplet;
import processing.core.PConstants;
import processing.event.MouseEvent;

public class MouseHandler implements Listener
{

 private boolean mouseDown = false;
 private int mouseButton = 0;

 ProcessingCanvasPart part;

 public MouseHandler( ProcessingCanvasPart part )
 {
  this.part = part;

 }

 public static void registerHandler( ProcessingCanvasPart part )
 {
  MouseHandler handler = new MouseHandler( part );
  GLCanvas glcanvas = part.getCanvas();
  // TODO: move to app init: fixes mouse wheel event over unfocused widgets;
  new Win32MouseWheelFilter( glcanvas.getParent().getDisplay() );


        glcanvas.addListener( SWT.MouseDoubleClick,  handler);
        glcanvas.addListener( SWT.KeyDown,           handler);
        glcanvas.addListener( SWT.MouseDown,         handler);
        glcanvas.addListener( SWT.MouseEnter,        handler);
        glcanvas.addListener( SWT.MouseExit,         handler);
        glcanvas.addListener( SWT.MouseWheel,        handler);
        glcanvas.addListener( SWT.MouseHover,        handler);
        glcanvas.addListener( SWT.MouseMove,         handler);
        glcanvas.addListener( SWT.MouseUp,           handler);
 }

 @Override
 public void handleEvent( Event event )
 {
  int peAction = -1;
     switch(event.type)
     {
     case SWT.MouseDoubleClick: peAction = MouseEvent.CLICK; break;
     case SWT.MouseDown:
      if(mouseDown == false)
       mouseButton = event.button;
      mouseDown = true;
      peAction = MouseEvent.PRESS;
     break;
     case SWT.MouseEnter:
       peAction = MouseEvent.ENTER; break;

     case SWT.MouseExit:
      peAction = MouseEvent.EXIT;
      mouseDown = false;
      break;
     case SWT.MouseHorizontalWheel:
     case SWT.MouseVerticalWheel: peAction = MouseEvent.WHEEL; break;
     case SWT.MouseHover: break;
     case SWT.MouseMove:
      if( mouseDown )

       peAction = MouseEvent.DRAG;
      else
          peAction = MouseEvent.MOVE;
     break;
     case SWT.MouseUp:
      peAction = MouseEvent.RELEASE;
      mouseDown = false;
     break;

        }


        int modifiers = event.stateMask;
     int peModifiers = modifiers &
                       (InputEvent.SHIFT_MASK |
                        InputEvent.CTRL_MASK |
                        InputEvent.META_MASK |
                        InputEvent.ALT_MASK);

     int peButton = 0;

     if (peAction != MouseEvent.DRAG)
      peButton = event.button;
     else
      peButton = mouseButton;

     if(peButton == 1 ) peButton = PConstants.LEFT;
     else if(peButton == 2 ) peButton = PConstants.CENTER;
     else if(peButton == 3 ) peButton = PConstants.RIGHT;

    // System.out.println("button: " + peButton + " pevent: " + peAction);

     if (PApplet.platform == PConstants.MACOSX) {
       //if (nativeEvent.isPopupTrigger()) {
       if ((modifiers & InputEvent.CTRL_MASK) != 0) {
         peButton = PConstants.RIGHT;
       }
     }

     int peCount = 0;
     if (peAction == MouseEvent.WHEEL) {
       // Invert wheel rotation count so it matches JAVA2D's
       // https://github.com/processing/processing/issues/3840
       peCount = -event.count;
     } else {
       peCount = event.count;;
     }

     int sx = (event.x);
     int sy = (event.y);
     int mx = sx;
     int my = sy;


     MouseEvent me = new MouseEvent(event, event.time,
                                    peAction, peModifiers,
                                    mx, my,
                                    peButton,
                                    peCount);

     part.postEvent(me);
 }
}

And lastly, a implementation of Eclipse 4 view part, that is also a PApplet. Many thanks to Wade Walker with his excellent tutorial about integrating JOGL into SWT - https://wadeawalker.wordpress.com/2010/10/09/tutorial-a-cross-platform-workbench-program-using-java-opengl-and-eclipse/

import javax.annotation.PostConstruct;
import javax.inject.Inject;

import org.eclipse.e4.ui.di.UISynchronize;
import org.eclipse.swt.SWT;
import org.eclipse.swt.graphics.Rectangle;
import org.eclipse.swt.layout.FillLayout;
import org.eclipse.swt.opengl.GLCanvas;
import org.eclipse.swt.opengl.GLData;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Event;
import org.eclipse.swt.widgets.Listener;

import com.jogamp.opengl.GL;
import com.jogamp.opengl.GL2;
import com.jogamp.opengl.GL2ES1;
import com.jogamp.opengl.GLContext;
import com.jogamp.opengl.GLDrawableFactory;
import com.jogamp.opengl.GLProfile;

import de.fhpotsdam.unfolding.UnfoldingMap;
import de.fhpotsdam.unfolding.geo.Location;
import de.fhpotsdam.unfolding.providers.Google;
import de.fhpotsdam.unfolding.utils.MapUtils;
import processing.core.PApplet;

public class ProcessingCanvasPart extends PApplet implements Runnable, Listener
{
    /** Holds the OpenGL canvas. */
    private Composite composite;

    /** Widget that displays OpenGL content. */
    private GLCanvas glcanvas;

    /** Used to get OpenGL object that we need to access OpenGL functions. */
    private GLContext glcontext;

    @Inject UISynchronize synchronizer;

    private PGraphicsSWTOpenGL graphics;

 private PSWTJOGL pgl;

 private UnfoldingMap map;

 @PostConstruct
 public void createComposite(Composite parent)
 {
  graphics = new PGraphicsSWTOpenGL();
  graphics.setParent( this );
  graphics.setPrimary( true );
  graphics.setPath( this.sketchPath() );

  this.g = graphics;

  pgl = (PSWTJOGL) graphics.pgl;

        GLProfile glprofile = GLProfile.get( GLProfile.GL2 );

        composite = new Composite( parent, SWT.NONE );
        composite.setLayout( new FillLayout() );

        GLData gldata = new GLData();
        gldata.doubleBuffer = true;
        glcanvas = new GLCanvas( composite, SWT.NO_BACKGROUND, gldata );

        glcanvas.setCurrent();
        glcontext = GLDrawableFactory.getFactory( glprofile ).createExternalGLContext();
        glcanvas.addListener( SWT.Resize,  this);

        MouseHandler.registerHandler( this );

        glcontext.makeCurrent();
        GL2 gl2 = glcontext.getGL().getGL2();
        gl2.setSwapInterval( 1 );
        gl2.glClearColor( 1.0f, 1.0f, 1.0f, 1.0f );
        gl2.glColor3f( 1.0f, 0.0f, 0.0f );
        gl2.glHint( GL2ES1.GL_PERSPECTIVE_CORRECTION_HINT, GL.GL_NICEST );
        gl2.glClearDepth( 1.0 );
        gl2.glLineWidth( 2 );
        gl2.glEnable( GL.GL_DEPTH_TEST );
        glcontext.release();

        Thread thread = new Thread( this );
        thread.start();
    }

 @Override
 public void settings() {
  this.setSize(1900, 1600);
 }

 @Override
 public void setup()
 {
    map = new UnfoldingMap(this, 0, 0, 1900, 1600,new Google.GoogleTerrainProvider());
    map.zoomAndPanTo(new Location(51.507222, -0.1275), 10);
    MapUtils.createDefaultEventDispatcher(this, map);
 }

 @Override
 public void draw() {
   this.background(0.1f, 1,1);

   map.draw();

   this.fill(0, 10, 45, 80);
   this.strokeWeight(3);
   this.stroke(0, 10, 45, 255);
   this.beginShape();
   this.vertex(200, 400);
   this.bezierVertex( 400, 200,  600, 200, 600, 400);
   this.bezierVertex( 600, 600,  400, 600, 200, 400);
   this.endShape();
 }


 @Override
 public void run() {

  Runnable renderingMethod = new Runnable() {
            @Override
   public void run()
            {
                if( (glcanvas != null) && !glcanvas.isDisposed())
                {
                    glcanvas.setCurrent();
                    glcontext.makeCurrent();
                    GL2 gl = glcontext.getGL().getGL2();
                    gl.glClearColor(.3f, .5f, .8f, 1.0f);
                    gl.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT);

                    pgl.getGL(glcontext);
                    PApplet sketch = ProcessingCanvasPart.this;
                    int pframeCount = sketch.frameCount;
                    sketch.handleDraw();
                    if (pframeCount == sketch.frameCount) {
                      // This hack allows the FBO layer to be swapped normally even if
                      // the sketch is no looping, otherwise background artifacts will occur.
                      pgl.beginRender();
                      pgl.endRender(sketch.sketchWindowColor());
                    }

                    graphics.completeFinishedPixelTransfer();

                    if (sketch.exitCalled()) {
                     graphics.completeAllPixelTransfer();

                        sketch.dispose(); // calls stopThread(), which stops the animator.
                        sketch.exitActual();
                    }

                    glcanvas.swapBuffers();
                    glcontext.release();
                }
            }
        };

        while( (glcanvas != null) && !glcanvas.isDisposed() )
        {
         synchronizer.syncExec( renderingMethod );
            try {
                // don't make loop too tight, or not enough time
                // to process window messages properly
                Thread.sleep( 10 );
            } catch( InterruptedException interruptedexception ) {
                // we just quit on interrupt, so nothing required here
            }
        }
    }

    @Override
 public void handleEvent( Event event )
    {
        switch(event.type)
     {
     case SWT.Resize:
            glcanvas.setCurrent();
            glcontext.makeCurrent();

            Rectangle rectangle = glcanvas.getClientArea();
            int iWidth = rectangle.width;
            int iHeight = Math.max( rectangle.height, 1 );

            this.setSize(iWidth, iHeight);
            graphics.setSize(iWidth, iHeight);
            if( map != null )
             map.mapDisplay.resize(width, height);
           glcontext.release();

     break;
     }
    }

    GLCanvas getCanvas() { return glcanvas; }
}

I hope someone finds this useful, and any suggestions are very welcome.
See you in 2 month, Mongolia here I come :)