Trimmed direct stream copy larger than source. Why?

johnmeyer · 23rd December 2013, 22:45

I have several dozen MJPEG AVI 24fps progressive video files that I trimmed in VD and then saved using the the "direct stream copy" mode. However, instead of the resulting file being much smaller than the sum of the source files, I end up with a file that is almost twice as large.

I tried the same operation in Sony Vegas and it "smart rendered" the result (same idea as direct stream copy) and it too created a trimmed, combined file that was much larger than the source files.

Why is this happening?

raffriff42 · 24th December 2013, 18:15

Could you post a MediaInfo report for a typical source file, and for the same file after trimming?

johnmeyer · 24th December 2013, 18:36

Quote:

Originally Posted by raffriff42

Could you post a MediaInfo report for a typical source file, and for the same file after trimming?

The Mediainfo report is copied below. The first section is for the original, un-trimmed file, and the second is for the same file after about 1/2 second was trimmed.

The obvious difference is the "Overall bit rate" in the "General" section. These do not match. However the bitrate shown in the "Video" section match exactly. Also, when I do the direct stream copy, the operation is virtually instantaneous, even on large files. This indicates that no rendering is being done.

I'll be interested if someone has some ideas about this.

I did a quick test on another computer and it seemed to behave correctly (i.e., smaller file sizes for the trimmed files). Therefore I think this may have something to do with the MJPEG codec setup. I'm using the MainConcept MJPEG codec.

Code:

General
Complete name                    : E:\Trail Camera Tests\MFDC0541.AVI
Format                           : AVI
Format/Info                      : Audio Video Interleave
File size                        : 8.54 MiB
Duration                         : 9s 958ms
Overall bit rate                 : 7 194 Kbps

Video
ID                               : 0
Format                           : JPEG
Codec ID                         : MJPG
Duration                         : 9s 958ms
Bit rate                         : 11.2 Mbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 24.000 fps
Color space                      : YUV
Chroma subsampling               : 4:2:2
Bit depth                        : 8 bits
Scan type                        : Progressive
Compression mode                 : Lossy
Bits/(Pixel*Frame)               : 0.505
Stream size                      : 13.3 MiB

Audio
ID                               : 1
Format                           : PCM
Format settings, Endianness      : Little
Format settings, Sign            : Signed
Codec ID                         : 1
Codec ID/Hint                    : Microsoft
Duration                         : 9s 942ms
Bit rate mode                    : Constant
Bit rate                         : 128 Kbps
Channel(s)                       : 1 channel
Sampling rate                    : 8 000 Hz
Bit depth                        : 16 bits
Stream size                      : 155 KiB (2%)
Interleave, duration             : 42 ms (1.00 video frame)
=================================================================================
General
Complete name                    : E:\Trail Camera Tests\MFDC0541 - Trimmed.avi
Format                           : AVI
Format/Info                      : Audio Video Interleave
File size                        : 12.4 MiB
Duration                         : 9s 167ms
Overall bit rate                 : 11.3 Mbps
Writing library                  : VirtualDub build 30009/release

Video
ID                               : 0
Format                           : JPEG
Codec ID                         : MJPG
Duration                         : 9s 167ms
Bit rate                         : 11.2 Mbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 24.000 fps
Color space                      : YUV
Chroma subsampling               : 4:2:2
Bit depth                        : 8 bits
Scan type                        : Progressive
Compression mode                 : Lossy
Bits/(Pixel*Frame)               : 0.505
Stream size                      : 12.2 MiB (99%)

Audio
ID                               : 1
Format                           : PCM
Format settings, Endianness      : Little
Format settings, Sign            : Signed
Codec ID                         : 1
Codec ID/Hint                    : Microsoft
Duration                         : 9s 167ms
Bit rate mode                    : Constant
Bit rate                         : 128 Kbps
Channel(s)                       : 1 channel
Sampling rate                    : 8 000 Hz
Bit depth                        : 16 bits
Stream size                      : 143 KiB (1%)
Interleave, duration             : 44 ms (1.05 video frame)
Interleave, preload duration     : 500 ms

jmartinr · 24th December 2013, 18:58

Maybe it has duplicate frames and is not really 24 fps.

johnmeyer · 24th December 2013, 19:02

Here are the VD File Information dialogs for both the original and trimmed files (different files than the Mediainfo report, but same exact test: trimming 1/2 second from beginning of file). There is definitely something going on with the audio -- look at the "frame size" entry.

I did check, and in the Audio menu in VD, it is set to Direct Stream Copy, just like the Video menu.

Original:

Trimmed:

johnmeyer · 24th December 2013, 19:12

Quote:

Originally Posted by jmartinr

Maybe it has duplicate frames and is not really 24 fps.

Well ... that is a very, very interesting thought because I have complained to the manufacturer (this is from a Moultrie trail camera) that their "24 fps" video contains a lot of duplicate frames. I guess it is time to post a sample video.

Here is an original 10-second clip from the trail camera. As you go through, frame-by-frame, you will see a lot of duplicate frames.

Moultrie Trail Camera 10 Second Video

raffriff42 · 24th December 2013, 19:19

Well, I trimmed a very similar MJPEG video (created by AfterBurner) and the size decreased as one would expect. I have no idea why yours is increasing. Maybe Virtualdub rebuilt the AVI index? Maybe the original was incomplete, or Vdub converted it to the 'new' OpenDML type?

EDIT duplicate frames sounds likely - except I thought 'direct stream copy' meant just that. Downloading your sample...

EDIT In your AVI Info boxes above, note "frames" = "keyframes", meaning there are no null or dropped frames.

raffriff42 · 24th December 2013, 19:42

Confirmed size increase after trimming.
Most individual RIFF chunks are a bit larger in the VirtualDub version. Dunno why.

RIFF tree - before
RIFF tree - after

johnmeyer · 24th December 2013, 20:23

Quote:

Originally Posted by raffriff42

Confirmed size increase after trimming.

Thanks for confirming. It is still a puzzle ...

raffriff42 · 25th December 2013, 00:28

Gah, I was reading it wrong

Yes there *are* lots of missing frames - zero-byte video frames, replaced in the Vdub version by a duplicate of the previous frame.

johnmeyer · 25th December 2013, 00:34

Does this mean that "Direct Stream Copy" is not really a direct stream copy ??

raffriff42 · 25th December 2013, 01:53

Huh, ffmpeg does the same:

Code:

ffmpeg.exe -i "Deer_05.avi" -c:v copy -c:a copy -t 0:00:08 "Deer_05 ffmpeg-trim+copy.avi"
pause

johnmeyer · 25th December 2013, 02:05

As I mentioned in an earlier post, this video is from a "trail camera" that is designed to be attached to a tree in a wilderness area and take photos and/or video for several months on a single set of batteries. It takes photos or videos when motion is detected and saves the results on a standard SD card. The "24 fps" spec is obviously a lie and, based on what you've discovered, it looks like they figured out a way to save space on the card by duplicating frames and saving nothing in those duplicate frame locations. However, what I can't understand is why VirtualDub, Sony Vegas Pro, and now ffpmpeg all change this admittedly odd structure when doing something that should be nothing more than bit-by-bit copy. If it were just VirtualDub, I'd make some statement that it has a flaw or bug, but the fact that all three do this makes me think there is something in some video spec that forces this behavior.

23rd December 2013, 22:45	#1 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	Trimmed direct stream copy larger than source. Why? I have several dozen MJPEG AVI 24fps progressive video files that I trimmed in VD and then saved using the the "direct stream copy" mode. However, instead of the resulting file being much smaller than the sum of the source files, I end up with a file that is almost twice as large. I tried the same operation in Sony Vegas and it "smart rendered" the result (same idea as direct stream copy) and it too created a trimmed, combined file that was much larger than the source files. Why is this happening?

24th December 2013, 18:58	#4 \| Link
jmartinr Registered User Join Date: Dec 2007 Location: Enschede, NL Posts: 301	Maybe it has duplicate frames and is not really 24 fps. __________________ Roelofs Coaching

24th December 2013, 19:19	#7 \| Link
raffriff42 Retried Guesser Join Date: Jun 2012 Posts: 1,373	Well, I trimmed a very similar MJPEG video (created by AfterBurner) and the size decreased as one would expect. I have no idea why yours is increasing. Maybe Virtualdub rebuilt the AVI index? Maybe the original was incomplete, or Vdub converted it to the 'new' OpenDML type? EDIT duplicate frames sounds likely - except I thought 'direct stream copy' meant just that. Downloading your sample... EDIT In your AVI Info boxes above, note "frames" = "keyframes", meaning there are no null or dropped frames. Last edited by raffriff42; 24th December 2013 at 19:42.

24th December 2013, 19:42	#8 \| Link
raffriff42 Retried Guesser Join Date: Jun 2012 Posts: 1,373	Confirmed size increase after trimming. Most individual RIFF chunks are a bit larger in the VirtualDub version. Dunno why. RIFF tree - before RIFF tree - after Last edited by raffriff42; 18th March 2017 at 01:16. Reason: (fixed image links)

25th December 2013, 00:28	#10 \| Link
raffriff42 Retried Guesser Join Date: Jun 2012 Posts: 1,373	Gah, I was reading it wrong Yes there are lots of missing frames - zero-byte video frames, replaced in the Vdub version by a duplicate of the previous frame. Last edited by raffriff42; 18th March 2017 at 01:13. Reason: (fixed image link)

24th December 2013, 18:15	#2 \| Link
raffriff42 Retried Guesser Join Date: Jun 2012 Posts: 1,373	Could you post a MediaInfo report for a typical source file, and for the same file after trimming?

24th December 2013, 19:02	#5 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	Here are the VD File Information dialogs for both the original and trimmed files (different files than the Mediainfo report, but same exact test: trimming 1/2 second from beginning of file). There is definitely something going on with the audio -- look at the "frame size" entry. I did check, and in the Audio menu in VD, it is set to Direct Stream Copy, just like the Video menu. Original: Trimmed:

25th December 2013, 00:34	#11 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	Does this mean that "Direct Stream Copy" is not really a direct stream copy ??

25th December 2013, 01:53	#12 \| Link
raffriff42 Retried Guesser Join Date: Jun 2012 Posts: 1,373	Huh, ffmpeg does the same: Code: ffmpeg.exe -i "Deer_05.avi" -c:v copy -c:a copy -t 0:00:08 "Deer_05 ffmpeg-trim+copy.avi" pause

25th December 2013, 02:05	#13 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	As I mentioned in an earlier post, this video is from a "trail camera" that is designed to be attached to a tree in a wilderness area and take photos and/or video for several months on a single set of batteries. It takes photos or videos when motion is detected and saves the results on a standard SD card. The "24 fps" spec is obviously a lie and, based on what you've discovered, it looks like they figured out a way to save space on the card by duplicating frames and saving nothing in those duplicate frame locations. However, what I can't understand is why VirtualDub, Sony Vegas Pro, and now ffpmpeg all change this admittedly odd structure when doing something that should be nothing more than bit-by-bit copy. If it were just VirtualDub, I'd make some statement that it has a flaw or bug, but the fact that all three do this makes me think there is something in some video spec that forces this behavior.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode