I played a bit with the camera viewfinder on my G1 which is usually displayed directly by the camera driver. I hoped that I could synchronize the driver’s camera frame rendering with my own processing and visualization. After an hour or so later I now assume that this is not possible at the moment. However, while playing around I extended the example below as you can see in these screenshots.
An OpenGL cube textured with the camera frame is rendered on top of the standard camera viewfinder. Thus, the standard camera image in the background is colored while the the cube is only grayscale. I worry that I have to make the OpenGL texture colored as well soon. I also cleaned up the source code a bit by extending GLSurfaceView instead of doing most of the OpenGL stuff myself and using a SurfaceView. I uploaded an updated version to the android market (direct link to the android market). You find the sourceode here.
Thanks! This is pretty cool!
Really interesting project, thank you for the source code, I gave it a fast look, and it doesn’t looks to be really difficult to implement something useful… anyway, what’s your project about? 😛
Thank you and… good job!
THANKS!!! You saved me!!
Only one question. In this post (http://groups.google.com/group/android-developers/browse_thread/thread/08e7df2e96a7973d) Dianne Hackborn, a Google employee, says that “right now you simply can not have multiple overlapping surface views in a window”…So, did you try your code also on different devices other than ADP1?
I’m not sure if I understand the thread but I’m pretty sure that you can draw a GLSurfaceView on top the camera’s SurfaceView on other devices. We tested it on Android 1.5 and 1.6. I assume that augmented reality apps, such as Layar and Wikitude, use the same approach and they work on all kinds of devices.
However, the way I put the View’s on the screen seems a bit strange to me. I first call “setContentView(glView);” and then “addContentView(mPreview, …);”. I would expect that these calls put the glView below the other one but it’s the other way around…
Thank you for a good story, I really enjoyed your blog. Be sure to give a link to your friends!
Thank you very much…. this has helped me a great deal in implementing Augmented Reality.
Thank you very much, your sharing has helped me to understand more about android programming 😀
Hi, It’s cool, but I get carbage collector running constantly. Is it possible to make it so that new memory wouldn’t get allocated while program is running? How can I check where the allocations are made?
I’m quite sure that you can’t avoid the garbage collector at the moment. It might help if you use smaller preview images. A solution to completely avoid the garbage collector is patching the firmware. I wrote a post about patching the firmware to avoid the garbage collector but it would not work on a vanilla firmware.
Cool, but what is vanilla firmware?
Since I just started to learn developing for mobile devices: Might it be more reasonable to develop augmented reality (3D stuff on video) apps on JavaME or WindowsMobile rather than on Android? Or is it wiser to stick to Android?
Why isn’t the carbage collector constantly running when I run the apidemos Graphics/CameraPrewiew?
does NOT work on droid motorola nor other 2.1 android. works fine on G1, adp. See market app “bauble” for a similar situation (code in resources) that had the same problem.
Just comment this line on GLLayer :
this.setEGLConfigChooser(5, 6, 5, 8, 16, 0);
works well on htc hero with villain 10.3 (android 2.1).
thx for that
It worked on my HTC Desire with Android 2.2 but I found an issue I don´t know how to solve.
With your app open I press Home button (which only pauses the app) and if I enter the app again, the GL won´t draw anything. With your app open again, I have to press the Back button (which destroy the app) to make GL work again.
Do you know any solution for this?
Thanks and best regards,
Pingback: Sensor-based Augmented Reality made simple « Things about stuff
I wondered how to set the background of the opengl view with the camera preview.. it was just a simple function. Thanks a lot for that.
Wow! I definitely like this side better…
It isn’t working properly on my HTC Desire with Android 2.2.
After comment line:
this.setEGLConfigChooser(5, 6, 5, 8, 16, 0);
I get symmetrically, twice rendered the same cube with strange colors texture on. Commenting another line:
or changing pixel format to Opaque is giving result with one cube rendered with gray scale texture captured from camera but with black background. Do you know any solution for this issue?
I really need your help.
I want to draw the preview buffer on a simple 2D surface (not a cube..) And mirror the image. make it look like a twisted mirror.
Can anyone throw in a tip? I know it’s suppose to be easy.
Thanks for helping 🙂
I could fix problem with use this code
“this.setEGLConfigChooser(8, 8, 8, 8, 16, 0);”
“this.setEGLConfigChooser(5, 6, 5, 8, 16, 0);”
Hope this helps. Try it! 🙂
Crashed on my Samsung Nexus S with Android 2.3.1 even when commenting:
I upgraded Android to 2.3.3 and now it works, but only if commenting that line, otherwise it still crashes. However, the camera is only projected at the 3d object. The background is black.
I tried to have a GL Surface View on top of camera surface, so that I can draw GLobjects with camera preview as background. I tried but didn’t work.
I gave it a fast look, and it doesn’t looks to be really difficult to implement something useful!Anyway, what’s your project about?Anyway, thanks for sharing.
It might be more reasonable to develop augmented reality (3D stuff on video) apps on JavaME or WindowsMobile rather than on Android? Or is it wiser to stick to Android?
Now my question is will all devices have this behavior? Can I use this behavior in my app to get openGL layered on top of camera preview? Please answer this query, its so cool to have openGL over the camera preview! It opens up for some amazing stuff!
Also works on LG-V900 Tablet (Android 3.0) , but just so far that it shows the camera image. And some strange behaviour appeared: The camera image is shown with at good FPS, but delayed by 0.5 seconds. Like if there is an array that stores the images in between.
This works on the google nexus-s as well ,.Thanks a ton ! .Its really helpful as I cudnt find any other code for this purpose till I stumbled upon this .
I’m looking at this now and it seems you can take a camera stream into a SurfaceTexture object and then render that.
It fails on a US T-Mobile HTC G2 running 2.3.4. It throws exceptions, once a RuntimeException on startPreview() but sometimes an IOException on setPreviewDisplay().
I also get a lot of repetitions of the following line in logcat, which makes it even harder to debug:(
W/CameraService( 1230): Overlay create failed – retrying
The only change I made to the code is to add comments.
Nice Work. But i want moew help regarding it.
will u please help me for inner camera preview ?
I want to give some vartex Shader effect in Inner camera preview. Will you please help me for it.
I will be realy thankful to you if its done.
I still had errors that were not mentioned yet.
One: requestFeature() must be called before adding content …
Fix: cut and paste the line
directly under the call of the superclass in onResume()- method of the main activity
Two: Method called after release() …
Fix: insert the line
before the call of stopPreview() in the OnSurfaceDestroyed()-Method of the CameraLayer-Class
Thanks for the example!
I am trying to capture these video preview but not able capture video it will generate video file but show error message when i will open it like “Cannot play video”. Do you have any solution about it?
Anyone figure out how to get the image in color? I tried using that decodeYUV420SP function that’s floating around the net & then doing
this.bmp = Bitmap.createBitmap(this.rgbBuf, 512, 512, Config.ARGB_8888);
GLUtils.texImage2D(GL10.GL_TEXTURE_2D, 0, this.bmp, 0);
To use the bitmap as a texture, but no luck.
Anyone figure out how to render it in color?
I figured out that I need to convert from yuv to rgb565, but all the functions I’ve found don’t work :(. There’s two or three “toRGB565” floating around the net, but I get garbage with all of them :(.
To get the colors, draw the image on a canvas (constructed from a blank bitmap) and it’ll be on your new bitmap in argb 🙂
Not tested with this code though, as it’s not working for the moment