Debugging touch events on filter

Help & Questions
,
1

I’m following the realtime face painting tutorial from Realtime AR Face Painting | DeepAR Help Center in my React Native/Expo iOS app using react-native-deepar and I’m having issues with touch events.

My setup works fine for many other filters, but for some reason I’m not able to get anything to draw onscreen as a result of the touch. The effect works fine in DeepAR Studio live preview, and I’ve confirmed that touch events are being passed through the React Native wrapper to the underlying DeepAR SDK, but then nothing happens.

I recognize the React Native library I’m using is community supported and detailed help may not be available, so I’m interested more in tips about how I might debug this issue rather than an actual solution. If touch events are being sent to the SDK and the effect is being loaded correctly, what else can I check? Are there debug options within the effect script that I could use to verify behavior somehow? I see I can output debug logs in DeepAR Studio, but I’m not sure that’s intended to work when deployed in my app as well.

Thanks in advance,
Andy

2

You can try using scripting to trigger an event on touch DeepAR Scripting API: Home, but my best guess is that the touches aren’t being sent to the SDK in a way it can read them

https://docs.deepar.ai/deepar-sdk/deep-ar-sdk-for-ios/api-reference/documentation/deepar/touchtype

3

Thanks for the ideas. Can you elaborate on “touches aren’t being sent to the SDK in a way it can read them”? How would I debug that? The code for sending touches looks fine to me at a glance but I’m not an expert in what the SDK expects

4

Any other thoughts you can offer on this? Looking at the DeepAR tutorial repo for the face paint demo the code looks identical to the react-native-deepar library I’m using (see previous link):

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
    if(event.type == UIEventTypeTouches) {
        UITouch * touch = [touches allObjects][0];
        CGPoint point = [self getPoint:touch];
        TouchInfo info = {point.x, point.y, START};
        [self.deepar touchOccurred:info];
    } else {
        [super touchesBegan:touches withEvent:event];
    }
}

    // Called every time a change in the previously started touch is detected
- (void)touchesMoved:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event {
    if(event.type == UIEventTypeTouches) {
        UITouch * touch = [touches allObjects][0];
        CGPoint point = [self getPoint:touch];
        TouchInfo info = {point.x, point.y, MOVE};
        [self.deepar touchOccurred:info];
    } else {
        [super touchesMoved:touches withEvent:event];
    }
}

    // Called every time a previously started touch is ended
- (void)touchesEnded:(NSSet<UITouch *> *)touches withEvent:(UIEvent *)event {
    if(event.type == UIEventTypeTouches) {
        UITouch * touch = [touches allObjects][0];
        CGPoint point = [self getPoint:touch];
        TouchInfo info = {point.x, point.y, END};
        [self.deepar touchOccurred:info];
    } else {
        [super touchesEnded:touches withEvent:event];
    }
}
5

Then I’m not sure, Have you tested if the touches are read correctly with a regular view instead of deepar?

6

Yep, touches seem to be coming through into the view just fine. I added logging in the react-native-deepar lib around the touchOccurred DeepAR SDK call and I do see it’s logging touches with plausibly correct coordinates:

Touch started: 0.537744, 0.408210, 0
Touch moved: 0.524173, 0.409730, 1
Touch moved: 0.496183, 0.641201, 1
Touch moved: 0.481764, 0.646902, 1
Touch ended: 0.477523, 0.649183, 2

Is there any way I can get visibility into what’s happening in DeepAR between this touchOccurred call and the onTouchOccurred function in my effect script? As far as I can tell, DeepAR is being called with the touch using code that follows the previously linked example, but onTouchOccurred isn’t firing.

Full logging code for reference (same pattern applies to MOVE and END):

  TouchInfo info = {point.x, point.y, START};
  [self.deepar touchOccurred:info];
  NSLog(@"Touch started: %f, %f, %d", info.x, info.y, info.type);
7

This is a different code, for touch support in the swift example, the only differences seem to be:

TouchType.START instead of START
and converting the point to a location in the arview

I don’t think there’s any more logs that can help you

override func touchesBegan(_ touches: Set, with event: UIEvent?) {
super.touchesBegan(touches, with: event)
let touch = touches.first
let point = touch?.location(in: self.arView)
let info = TouchInfo(x: point!.x, y: point!.y, type: TouchType.START);
self.deepAR.touchOccurred(info)