Sensor-based Augmented Reality made simple

I did some content-based augmented reality for Android and my former student developed a sensor-based Augmented Reality App. Thus, I thought I should be able to do the sensor-based stuff as well. I fiddled around a lot to make it work with the canvas but finally I realized that I’m just not able to do it with the Canvas and switched to OpenGL. I attached an Eclipse project with the source code.

Even though I couldn’t find a good example or tutorial it was pretty easy and definitely much easier than going the Canvas way. Basically you have to use the SensorManager to register for accelerometer and magnetometer (that’s the compass) events. You find the code in the class PhoneOrientation. Accelerometer data and compass data can be combined to create a matrix using the code below. I also had to “remap the coordinate system” because by the example uses a portrait mode.

SensorManager.getRotationMatrix(newMat, null, acceleration, orientation);
SensorManager.remapCoordinateSystem(newMat,
		SensorManager.AXIS_Y, SensorManager.AXIS_MINUS_X,
		newMat);

The newMat is a 4×4 matrix as a float array. This matrix must be passed to the OpenGL rendering pipeline and loaded by simply using:

gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadMatrixf(floatMat, 0);

That’s it basically. As I never learned how to use OpenGL, in particular how to load textures, the project is based on an earlier example that renders the camera image on a cube. The project also uses an Android 2.2 API and reflection to access camera images in a fast way (that’s why it works on Android 2.1). Check out the Eclipse project if you are interested or install the demo on you Android 2.1 device (on cyrket/in the market).

6 thoughts on “Sensor-based Augmented Reality made simple

  1. Pingback: Tweets that mention Sensor-based Augmented Reality made simple « Things about stuff -- Topsy.com

  2. Luis

    Dear Niels,

    Thanks for this and the previous source codes.

    I am having troubles with this one, though, in my 2.1 Galaxy S phone: both the compiled app and the market one stop working when I innitiate them on the phone. I am compiling with Android 2.1-update1, should it work?

  3. Luis

    I am facing this problem: on my Samsung Galaxy S (android 2.1) the application (both from market or eclipse compiled) crash when launched. I am compiling with Android2.1-update1. Am I doing something wrong or is it definitely not going to work on this phone no matter what I may do?

  4. Peter Todd

    Thanks for this, I look forward to trying it when my phone arrives… should be the missing piece of the puzzle to do something interesting, but the proof is in the pudding.

    btw, I notice that there’s a spam comment above and also your wp theme is printing out some errors in the footer of ‘tag’ pages like http://nhenze.net/?tag=accelerometer which is the one I landed on from Google.

Leave a Reply

Your email address will not be published. Required fields are marked *