Android frameAvailable RGBA format send image with green lines

Help & Questions
1

Hi,

Description: I am working on a project that receive a ImageProxy from camera library in RGBA format and send the image buffer to other calling library that accepts RGBA. The image by other library is rendered correctly.

As I want to integrate DeepAR, I changed cameraX format to YUV and pass image to DeepAR and render it on surfaceView. It works awesome :slight_smile :slight_smile:

The next step is to send image from DeepAR to calling library. Thus I changed

// deepAR.setRenderSurface(surface, 1280, 720,)

-> to

deepAR.setOffscreenRendering(1280, 720, DeepARPixelFormat.RGBA_8888)

 override fun frameAvailable(image: Image?)

Issue:

As the frameAvailable receive frame in RGBA the expectation is it should work fine when passed to calling library.

However, it appears with green lines.

Diagnostics: As if I use surfaceView, the image is rendered correctly on View and sending RGBA directly from cameraX works fine, it seems somehow the calling library is not good with RGBA received from frameAvailable.

In addition, I converted RGBA from frameAvailable to bitmap and saved to images on device, it is also distorted.
Screen Shot 2022-09-09 at 6.14.18 AM

I have very limited knowledge of image rendering + conversion. Any feedback will help me.

2

HI @aulakh.inderpal, that you for reporting. You seem to have set the output format correctly, we will test this out to diagnose the issue and get back to you

1 Like
3

Thanks @jelena . As I am working on hackathon project and need to present next week, any quick help will be appreciated :slight_smile:

If DeepAR team can save the image to gallary on frameAvailable in correct format then I can pick up from there.

4

I fixed issue for me by experimenting around changing the setTargetResolution for cameraAPI.

I think DeepAR issue occurs when width > height. Not sure, why 1280/720 size caused issue for frameAvailable.

imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(400, 450)) → works fine

imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(1260, 720)) → issue occurs

deepAR.setOffscreenRendering(1280, 720, DeepARPixelFormat.RGBA_8888) → not changed

5

Sorry you had that issue, the width being greater than the height shouldn’t cause this to happen, could you compare your frameAvailable code to one of our example apps to check if everything is correct? From the look of the issue, it might be related to the stride.

@Override
    public void frameAvailable(Image frame) {
        if (frame != null) {
            final Image.Plane[] planes = frame.getPlanes();
            final Buffer buffer = planes[0].getBuffer().rewind();
            int pixelStride = planes[0].getPixelStride();
            int rowStride = planes[0].getRowStride();
            int rowPadding = rowStride - pixelStride * width;
            Bitmap bitmap = Bitmap.createBitmap(width + rowPadding / pixelStride, height, Bitmap.Config.ARGB_8888);
            bitmap.copyPixelsFromBuffer(buffer);
            offscreenView.setImageBitmap(bitmap);
        }
    }