Audio capture on webSDK

Help & Questions
1

Hi guys! This is my first time using DeepAR platform, I’ve been trying to capture video + audio but only the video is captured. I added the audio parameter on the mediaDevices settings (so my webApp asks for Mic permission at start), but even so the audio is not being recorded:

const stream = await navigator.mediaDevices.getUserMedia({
audio: true,
video: {
facingMode: mode
}
});

Am I forgetting something or doing something wrong? I’d presume the audio capture is enabled by default (as it is on other platforms like 8th Wall and Blippar)

Thanks!

2

Hi @Alessandro_Straccia , our current web SDK doesn’t currently have a sound capture feature it only records the canvas output, if you need sound capture you would need to implement it yourself or use a 3rd party solution.

4

Hi @Alessandro_Straccia, I was able to accomplish this by modifying the deepar.js code. Invasive, but it worked. Are you still looking for a solution?

1 Like
5

@KevinFlynn I’d love to get information on how you’ve done that?

6

Unfortunately I had to deliver the project and made the client aware that it’s a platform limitation, but I’m very interested to know about your approach on modifying the platform core!

7

Ahh, I found a really simple solution – record from the canvas element that DeepAR uses, rather than using DeepAR’s record functionality.

        const audioStream = await navigator.mediaDevices.getUserMedia({
            audio: { deviceId: { exact: this.state.selectedAudioInputDevice.deviceId } },
            video: false
        });

        let canvasStream = this.state.deepARCanvas.current.captureStream();

        var finalStream = new MediaStream();
        audioStream.getTracks().forEach(track => {
            finalStream.addTrack(track);
        });
        canvasStream.getTracks().forEach(track => {
            finalStream.addTrack(track);
        });

        recorder = RecordRTC(finalStream, {
            mimeType: "video/webm;codecs=h264",
            recorderType: MediaStreamRecorder,
            ....