Problems building v4 into an existing JS library

[chadbailey]

Hello! I work for Daily, and I’m experimenting with integrating the v4 SDK into our JS library. I have the quickstart-web-js-npm example running locally just fine, and I’m even serving the WASM, segmentation models, and effects from a simple server app I’m running locally. So I think I have the common source of WASM errors handled correctly.

But now I’m trying to get it working in the same place where we already have some background blur and replacement features. When I try code that’s essentially identical to the quickstart app, I get some errors. When I added an onError callback, the error message I get is Process frame called but DeepAR not initialized! Right after that, I get the classic deepar.esm.js?cc95:1 Uncaught RuntimeError: memory access out of bounds. But the first error I get is:

deepar.esm.js?cc95:1 Assertion failed: console.assert

With a stack trace that looks like this toward the top:

	hc	@	deepar.esm.js?cc95:1
$func6451	@	deepar.wasm:0x1a5fda
$func6865	@	deepar.wasm:0x1c8e10
$og	@	deepar.wasm:0x1595e4
t._processFrameVideo	@	deepar.esm.js?cc95:1
eval	@	deepar.esm.js?cc95:1
eval	@	deepar.esm.js?cc95:1
requestAnimationFrame (async)		
_e290	@	deepar.esm.js?cc95:1
eval	@	deepar.esm.js?cc95:1
requestAnimationFrame (async)		
_e290	@	deepar.esm.js?cc95:1
eval	@	deepar.esm.js?cc95:1
requestAnimationFrame (async)		
_e290	@	deepar.esm.js?cc95:1
eval	@	deepar.esm.js?cc95:1
requestAnimationFrame (async)		
_e290	@	deepar.esm.js?cc95:1
start	@	deepar.esm.js?cc95:1
t.startLoop	@	deepar.esm.js?cc95:1
e.setVideoElement	@	deepar.esm.js?cc95:1
onInitialize	@	DeepARCam.js?b664:55
eval	@	deepar.esm.js?cc95:1
Promise.then (async)		
e	@	deepar.esm.js?cc95:1
_initialize	@	DeepARCam.js?b664:37
start	@	DeepARCam.js?b664:20
eval	@	CamVideoProcessor.js?a357:504
asyncGeneratorStep	@	asyncToGenerator.js?c973:3
_next	@	asyncToGenerator.js?c973:25
Promise.then (async)		
asyncGeneratorStep	@	asyncToGenerator.js?c973:13
_next	@	asyncToGenerator.js?c973:25
eval	@	asyncToGenerator.js?c973:32
eval	@	asyncToGenerator.js?c973:21
_startProcess	@	CamVideoProcessor.js:618
postMessage (async)		
value	@	daily-iframe-esm.js?ec3a:1

Do I need to quadruple-check how I’m serving the various WASM and model files, even though they work in the example? Or am I somehow doing something bad with multithreading that may be causing this problem? Thanks in advance!

[jelena]

HI @chadbailey,

The troubleshooting steps described in this thread

Web SDK Memory Out Of Bounds exception

If you get this error while using DeepAR on your domain, you should check the metadata header settings (Content-Encoding and Content-Type) for .js, .wasm, model and effect files on your server. These are explained in the getting started guide You can also Inspect your page and compare the metadata headers to one of our demo examples. This should make it any mismatches easier to spot.

Refer specifically to errors that happen when serving DeepAR on a domain, that don’t show up when serving locally. I would recommend going through those steps first, especially comparing your content type and content encoding to our examples.

[chadbailey]

Thanks, @jelena. I’ve worked through a lot of those issues the hard way already. I’m running a dedicated server on localhost:7979 on my computer that’s serves WASM, model .bin files, and effects for two different apps I’m running locally. One is a modified version of quickstart-web-js-npm to load everything from my server, such as deeparWasmPath: "http://localhost:3939/deepar.wasm",. This version works fine; I can put myself into ‘creepy snail mode’ while serving all of the assets from that server app listening on 7979.

I have another test app, also running locally, that’s attempting to load the DeepAR SDK through our client library using the same asset paths, etc. It runs on localhost:3000; the quickstart demo app runs on localhost:8888. All of the response headers for the deepar.wasm file look identical between the two apps, including Content-Type: application/wasm, so I’m fairly confident I’m serving the files correctly. But I get the aforementioned errors only when I’m running DeepAR through our client lib.

Are there any JS SDK methods I can use to test things earlier in the load process? I have a bunch of console.log statements scattered throughout everything, and they all suggest everything’s working; I get console logs after new DeepAR({}), from the onInitialize callback, and finally from the deepAR.downloadFaceTrackingModel success callback. After that is when I get the errors.

[jelena]

I think your wasm content type is probably correct then, have you checked .js, model and effect files as well?

The way we check for those issues is to compare the content types and encoding to the demos

[chadbailey]

Those all look the same-ish (application/octet-stream for me vs. binary/octet-stream for you), but they still load fine in the quickstart example for me. I think I can rule out both segmentation models and effects, though. I commented them all out in both the quickstart app and my app. In the quickstart app, the WASM model loads just fine, and I get the DeepAR.ai watermark on an otherwise unmodified video in the <canvas> tag. But I still get that same error in my app.

[jelena]

I believe application/octet-stream is okay for the models.bin file, but effect files should be binary/octet-stream

• lib/deepar.js - application/javascript
• lib/deepar.wasm - application/wasm
• models/models-69-extreme.bin - binary/octet-stream or application/octet-stream
• effect files - binary/octet-stream

[chadbailey]

I’ll address that shortly, but I’m still seeing the errors in my app even if I’m not loading an effect at all. The same code (loading no effects at all) works fine in my modified quickstart example.

[jelena]

If the issue isn’t reproducible with the quickstart example and isn’t related to the content type there is probably some sort of integration issue on your end. I’m not sure if we can help much more unless you share your code, sorry.

[chadbailey]

Thanks for working on this with me, @jelena! I think I’m getting in touch with some support folks through other channels, so as not to spam the forum with my problems.

[Disho]

Hi @chadbailey!

We are digging into your issue. It would really help us if you could share code snippets of how you integrate and interact with DeepAR SDK. Also, if you can make a minimal reproducible example that we could run to reproduce this issue.

[Disho]

@chadbailey All the issues that you are having could be caused if you are calling DeepAR methods when DeepAR is not fully initialized. To make sure DeepAR is properly initialized you need to wait for the onInitialize callback and only then you can call all other DeepAR methods.

const deepAR = new DeepAR({
  // other parameters ...
  callbacks: {
    onInitialize: () => {
      // You can only call DeepAR methods once you reach this point.
      // Except for downloadFaceTrackingModel method.
    }
  }
});