Why sometimes canvas is black while video is playing?

SDK
1

From time to time DeepAR doesn’t render video after setVideoElement (most frequently when effect is set).

It looks like this:

image
image704×690 11.5 KB

Here is a DOM for this component.

image
image816×220 34 KB

DeepAR is set to take video from this <video> element and draw it on canvas.
The canvas is located directly above video. If I hide canvas, I can see that video is working.

Judging by the logs DeepAR successfully aquired this video element

image
image892×878 132 KB

What could be the reason?

2

Could you share your source code perhaps?

3

It’s deeply integrated into our logic. But the main parts are:

  • Setting local stream to video:
useEffect(() => {
    if (videoRef.current && localStream) {
      videoRef.current.srcObject = localStream
    }
  }, [localStream, videoRef.current])
  • DeepAR initialization and effects setup
  const [initialized, setInitialized] = useState(false)
  useEffect(() => {
    if (!(features.effects && canvasRef?.current)) { return }

    dispatch(setLocalStreamWithEffects(canvasRef.current.captureStream()))

    setInitialized(false)
    ARService.initialize(canvasRef.current).then(() => setInitialized(true))

    return () => ARService.shutdown()
  }, [features.effects, canvasRef?.current])


  const [ready, setReady] = useState(false)
  useEffect(() => {
    if (!(features.effects && initialized && videoRef.current)) { return }

    setReady(false)
    ARService.setVideoElement(videoRef.current)
    setReady(true)
  }, [
    features.effects,
    initialized,
    videoRef?.current
  ])

  const { appliedEffect } = useAppState("effects")
  useEffect(() => {
    if (!(features.effects && ready)) { return }

    ARService.setEffect(appliedEffect)

    return () => ARService.clearEffect()
  }, [
    features.effects,
    ready,
    appliedEffect
  ])
4

Looks like this is related to hooks being called multiple times in dev mode of React.

My guess is:
There is a race condition between first and second hooks. initialize resolves faster than second hook detects change of initialized

To solve this, I’ve merged those two hooks into one

5

Thank you very muh for sharing the solution