Trimmed direct stream copy larger than source. Why?

[johnmeyer]

I have several dozen MJPEG AVI 24fps progressive video files that I trimmed in VD and then saved using the the "direct stream copy" mode. However, instead of the resulting file being much smaller than the sum of the source files, I end up with a file that is almost twice as large.

I tried the same operation in Sony Vegas and it "smart rendered" the result (same idea as direct stream copy) and it too created a trimmed, combined file that was much larger than the source files.

Why is this happening?

[raffriff42]

Could you post a MediaInfo report for a typical source file, and for the same file after trimming?

[johnmeyer]

Could you post a MediaInfo report for a typical source file, and for the same file after trimming?

The Mediainfo report is copied below. The first section is for the original, un-trimmed file, and the second is for the same file after about 1/2 second was trimmed.

The obvious difference is the "Overall bit rate" in the "General" section. These do not match. However the bitrate shown in the "Video" section match exactly. Also, when I do the direct stream copy, the operation is virtually instantaneous, even on large files. This indicates that no rendering is being done.

I'll be interested if someone has some ideas about this.

I did a quick test on another computer and it seemed to behave correctly (i.e., smaller file sizes for the trimmed files). Therefore I think this may have something to do with the MJPEG codec setup. I'm using the MainConcept MJPEG codec.

General
Complete name : E:\Trail Camera Tests\MFDC0541.AVI
Format : AVI
Format/Info : Audio Video Interleave
File size : 8.54 MiB
Duration : 9s 958ms
Overall bit rate : 7 194 Kbps

Video
ID : 0
Format : JPEG
Codec ID : MJPG
Duration : 9s 958ms
Bit rate : 11.2 Mbps
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.505
Stream size : 13.3 MiB

Audio
ID : 1
Format : PCM
Format settings, Endianness : Little
Format settings, Sign : Signed
Codec ID : 1
Codec ID/Hint : Microsoft
Duration : 9s 942ms
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 1 channel
Sampling rate : 8 000 Hz
Bit depth : 16 bits
Stream size : 155 KiB (2%)
Interleave, duration : 42 ms (1.00 video frame)
=================================================================================
General
Complete name : E:\Trail Camera Tests\MFDC0541 - Trimmed.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 12.4 MiB
Duration : 9s 167ms
Overall bit rate : 11.3 Mbps
Writing library : VirtualDub build 30009/release

Video
ID : 0
Format : JPEG
Codec ID : MJPG
Duration : 9s 167ms
Bit rate : 11.2 Mbps
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.505
Stream size : 12.2 MiB (99%)

Audio
ID : 1
Format : PCM
Format settings, Endianness : Little
Format settings, Sign : Signed
Codec ID : 1
Codec ID/Hint : Microsoft
Duration : 9s 167ms
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 1 channel
Sampling rate : 8 000 Hz
Bit depth : 16 bits
Stream size : 143 KiB (1%)
Interleave, duration : 44 ms (1.05 video frame)
Interleave, preload duration : 500 ms

[jmartinr]

Maybe it has duplicate frames and is not really 24 fps.

[johnmeyer]

Here are the VD File Information dialogs for both the original and trimmed files (different files than the Mediainfo report, but same exact test: trimming 1/2 second from beginning of file). There is definitely something going on with the audio -- look at the "frame size" entry.

I did check, and in the Audio menu in VD, it is set to Direct Stream Copy, just like the Video menu.

Original:
http://i177.photobucket.com/albums/w208/johnmeyer/MJPEGoriginal_zps403c3f19.png

Trimmed:
http://i177.photobucket.com/albums/w208/johnmeyer/MJPEGtrimmed_zps3742c962.png

[johnmeyer]

Maybe it has duplicate frames and is not really 24 fps.Well ... that is a very, very interesting thought because I have complained to the manufacturer (this is from a Moultrie trail camera) that their "24 fps" video contains a lot of duplicate frames. I guess it is time to post a sample video.

Here is an original 10-second clip from the trail camera. As you go through, frame-by-frame, you will see a lot of duplicate frames.

Moultrie Trail Camera 10 Second Video (http://www.mediafire.com/watch/wj4pra2f1newp0j/Deer_05.AVI)

[raffriff42]

Well, I trimmed a very similar MJPEG video (created by AfterBurner) and the size decreased as one would expect. I have no idea why yours is increasing. Maybe Virtualdub rebuilt the AVI index? Maybe the original was incomplete, or Vdub converted it to the 'new' OpenDML type?

EDIT duplicate frames sounds likely - except I thought 'direct stream copy' meant just that. Downloading your sample...

EDIT In your AVI Info boxes above, note "frames" = "keyframes", meaning there are no null or dropped frames.

[raffriff42]

Confirmed size increase after trimming.
Most individual RIFF chunks are a bit larger in the VirtualDub version. Dunno why.
RIFF tree - before (https://www.dropbox.com/s/yl6msv7cspai49h/RIFF-chunk-Deer_05.png?dl=0)
RIFF tree - after (https://www.dropbox.com/s/lui60lbdiyhf4nh/RIFF-chunk-Deer_05-trim%2Bscopy.png?dl=0)

[johnmeyer]

Confirmed size increase after trimming.Thanks for confirming. It is still a puzzle ...

[raffriff42]

Gah, I was reading it wrong :o

Yes there *are* lots of missing frames - zero-byte video frames, replaced in the Vdub version by a duplicate of the previous frame.

https://www.dropbox.com/s/bbyg6mc5k0n1tel/RIFF-chunk-Deer_05-hilite.png?raw=1

[johnmeyer]

Does this mean that "Direct Stream Copy" is not really a direct stream copy ??

[raffriff42]

Huh, ffmpeg does the same: ffmpeg.exe -i "Deer_05.avi" -c:v copy -c:a copy -t 0:00:08 "Deer_05 ffmpeg-trim+copy.avi"
pause

[johnmeyer]

As I mentioned in an earlier post, this video is from a "trail camera" that is designed to be attached to a tree in a wilderness area and take photos and/or video for several months on a single set of batteries. It takes photos or videos when motion is detected and saves the results on a standard SD card. The "24 fps" spec is obviously a lie and, based on what you've discovered, it looks like they figured out a way to save space on the card by duplicating frames and saving nothing in those duplicate frame locations. However, what I can't understand is why VirtualDub, Sony Vegas Pro, and now ffpmpeg all change this admittedly odd structure when doing something that should be nothing more than bit-by-bit copy. If it were just VirtualDub, I'd make some statement that it has a flaw or bug, but the fact that all three do this makes me think there is something in some video spec that forces this behavior.