Probably - DeepAR SDK Bug Report: Intermittent Black Frame Video Recording

Help & Questions
1

Probably - DeepAR SDK Bug Report: Intermittent Black Frame Video Recording

Environment

  • DeepAR SDK version: 5.6.21
  • Platform: iOS 17+
  • Devices: iPhone 14 Pro, iPhone 15 Pro (reproduced on multiple devices)
  • Xcode: 16.x
  • Recording mode: Multi-segment video recording with warmup enabled

Summary

DeepAR’s built-in video recorder intermittently produces entirely black video files despite receiving valid camera frames, having renderingInitialized=true, and reporting no errors. The issue occurs in a multi-segment recording workflow (record → finish → re-arm warmup → record next segment).

How We Initialize DeepAR

deepAR = DeepAR()
deepAR?.delegate = self
deepAR?.setLicenseKey(licenseKey)
deepAR?.changeLiveMode(true)

// After didInitialize callback:
deepAR?.videoRecordingWarmupEnabled = true

// Metal layer setup:
let metalEAGLLayer = MetalEAGLLayer()
metalEAGLLayer.device = MTLCreateSystemDefaultDevice()
metalEAGLLayer.pixelFormat = .bgra8Unorm
metalEAGLLayer.framebufferOnly = true
metalEAGLLayer.drawableSize = CGSize(width: 1080, height: 1920)
deepAR?.initialize(withWidth: 1080, height: 1920, window: metalEAGLLayer)

Recording Lifecycle (Multi-Segment Mode)

Each segment follows this cycle:

  1. Warmup — startVideoRecording(withOutputWidth:outputHeight:subframe:) called during init
  2. User taps record — resumeVideoRecording() called (warmup is enabled)
  3. User taps stop — finishVideoRecording() called
  4. DeepAR callback — didFinishVideoRecording(_:) fires with output path
  5. Re-arm for next segment — immediately call startVideoRecording(withOutputWidth:...) again to warm up the encoder for the next segment
// Re-arm after segment completes:
DispatchQueue.main.async { [weak self] in
    self?.startVideoRecordingWithOptions()  // re-arm warmup
}

Camera Frame Delivery

// In AVCaptureVideoDataOutputSampleBufferDelegate:
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, ...) {
    guard !isDeepARPaused else { return }
    deepAR?.enqueueCameraFrame(sampleBuffer)
}

Diagnostic Methodology

We added comprehensive logging at every boundary:

  1. Frame health — Validated every camera frame: 1080x1920 BGRA, ~30fps
  2. Frame delivery — Counted total/dropped/recording frames via enqueueCameraFrame
  3. DeepAR state — Logged renderingInitialized, isDeepARPaused, deepARState at every callback
  4. Metal layer — Monitored sublayer count for resource leaks
  5. Video file health — Measured file size, duration, video track count for every output file
  6. Error callbacks — Monitored recordingFailedWithError and onError(withCode:error:)

Evidence: 15 Recorded Segments

Seg File Size Duration KB/frame (@30fps) Status
1 17.8 MB 6.86s 86 KB OK
2 11.9 MB 4.60s 86 KB OK
3 16.4 MB 6.27s 87 KB OK
4 29.6 MB 11.39s 87 KB OK
5 142 KB 9.08s 0.5 KB BLACK
6 39.3 MB 15.13s 87 KB OK (self-recovered)
7 13.6 MB 5.20s 88 KB OK
8 12.8 MB 4.92s 87 KB OK
9 9.1 MB 3.52s 102 KB OK
10 7.7 MB 2.97s 89 KB OK
11 45 KB 2.94s 0.5 KB BLACK
12 77 KB 4.83s 0.5 KB BLACK
13 9.8 MB 3.78s 87 KB OK (self-recovered)
14 94 KB 5.96s 0.5 KB BLACK
15 185 KB 11.76s 0.5 KB BLACK

5 out of 15 segments (33%) produced black video.

All External Indicators Are Healthy During Black Segments

For every segment, including the black ones:

  • Camera frames: Valid 1080x1920 BGRA buffers delivered at ~30fps — zero invalid frames
  • Frame delivery: Zero dropped frames, zero BLOCKED events, isDeepARPaused=false always
  • Rendering state: renderingInitialized=true
  • Metal layers: sublayers=1 — no resource accumulation
  • Warmup: didFinishPreparingForVideoRecording fires for every segment, isPreparedForRecording=true set before user taps
  • Errors: Zero recordingFailedWithError calls, zero onError calls

Analysis

The black segments have:

  • Valid .mov container — correct duration, video track present
  • ~0.5 KB/frame vs normal ~87 KB/frame — the H.264 encoder IS running, but encoding black/empty render output
  • Non-sequential pattern — seg 5 bad → seg 6 good → segs 7-10 good → seg 11 bad — rules out cumulative state corruption

Since the camera is delivering valid frames, renderingInitialized=true, and no errors are reported, the black output must originate inside DeepAR’s render-to-encoder pipeline. The encoder receives frames from DeepAR’s internal render target, and that render target is intermittently black.

Suspected Root Cause

The issue appears to be a race condition in DeepAR’s encoder pipeline during the finishVideoRecording() → startVideoRecording() re-arm cycle. When the new startVideoRecording() call arrives before DeepAR’s internal encoder has fully torn down the previous session, the new encoder session connects to a stale/uninitialized render target.

Workaround Applied

We have implemented a client-side mitigation:

  1. Detect black segments via file-size-to-duration ratio (< 500 KB/s = black)
  2. Discard bad files and revert UI
  3. Pause/resume DeepAR before re-arming warmup, with a 100ms delay

This reduces but does not eliminate the issue since the root cause is internal to the SDK.

Request

Could you investigate the encoder pipeline’s behavior during rapid finishVideoRecording() → startVideoRecording() cycles? Specifically:

  1. Is there a minimum delay required between finishVideoRecording() completing and a new startVideoRecording() call?
  2. Is there a recommended way to ensure the encoder’s render target is properly re-attached after a finish/start cycle?
  3. Could a didFinishVideoRecording callback guarantee that it’s safe to immediately start a new recording?

I think that information is quite enough, but if need I could provide additional logs.