X264 1080P master encode (25mbps) decoding speed issue

[magnatique]

Hey guys,

I am seeing some slow playback or simply crashing when loading my encoded files in Sony Vegas 9 (which uses mainconcept decoder plugin)


Right now, I have some projects with many pro-res files we edited, and I frameserving using debug mode to megui and get it encoded there with x264. I used the blue ray setting, but changed to quality export of 13 or so, to get roughly 25mbps output at 1920x1080. I export with fast decode option.

Probnlem is, when loading back the mp4 in Sony vegas 9, Smaller files will play fine, but with 20mins+ files, some play perfectly 30fps while some of them crash the program.

If I encode using the default settings, they pretty much all play, but lag at 10fps or so. ( I have to increase quality to 18 from 13 for similar size as well)

What I am wondering is if anyone has an idea what setting might be causing these random crashes, whereas what other settings might help with the decoding to build from the default settings?

My goal is to obtain a high quality file, roughly 25mbps, that will be of similar visual quality as original and will play back smoothly... so not all the bells and whistles to compress the size as I can go up to 25mbps, but still have the options that allow for a smooth playback while not triggering vegas/mainconcept crashes.



-------------

I've been trying to find a good solution to keep backup masters in high quality of my footage for future use, but in a format that can be imported on macs, or on PC.

We currently use Sony Vegas 9, and our partners final cut on mac. Since x264 play in QT with no prob, x264 sounds like the best solution as we are moving towards AVC editing more and more instead of HD on dv tapes.

[TheImperial2004]

If the problem is that you can't open these "High Quality" files in Vegas , it would be probably that the MainConcept decoder can't handle that profile , or it simply can't decode them quickly (Not Multithreaded) but I don't think that would lead to a crash ?

Just my two cents :)

[poisondeathray]

Vegas Pro 9 x64 ? Apparently not specific to x264, as many users report crashes occurs with h.264 derived from other encoders as well

IMO, not a good idea to use a lossy 25Mbps 2nd generation encode as a "master". It's intended as an end delivery format, not a digital intermediate suitable for editing


Since x264 play in QT with no prob...

You need specific settings for it to be compatible with the crappy QT Player

[Revgen]

Vegas is horrible when it comes to decoding video.

It's better to just frameserve through avisynth and use MakeAVI's to fake an uncompressed avi for Vegas to decode.

[shon3i]

What you think about GPU assisted decoder for Vegas http://www.divideframe.com/?p=gpudecoder You maybe need one graphic card with CUDA support (Nvidia) but decoding will incrase much, and you maybe newer have problems with various H264 streams because CUDA is capable to decode everything.

[magnatique]

Vegas Pro 9 x64 ? Apparently not specific to x264, as many users report crashes occurs with h.264 derived from other encoders as well

IMO, not a good idea to use a lossy 25Mbps 2nd generation encode as a "master". It's intended as an end delivery format, not a digital intermediate suitable for editing


You need specific settings for it to be compatible with the crappy QT Player


yeah I am fairly certain it`s vegas crappy decoder... but having to deal with it, I am trying to figure out what can be done heh.

As for the master, my main delivery is for web, 2mbps max. Using the x264 as source file to encode for web is not going to impact on quality, so the lossy version there is not a problem.

The other potential use I might have of re-editing these 2nd generation files is to re-export for dvd or at the most, blu-ray. Since BluRay has a max of 25mbps roughly, I am thinking I could either use the files directly (hence trying to tweak my profile to fit the BR profile), or re-encode in the program that would press the BR discs... the need to re-edit might arise for compilations, best off, promo material and such only, but these so-called masters are not intended as intermediates, just really the finalized edited product.

Since we`re moving away from HDV too, the x264 lossy compression at 25mbps FAR surpasses that of the 25mbps HDV, or even HDV at 50mbps since it is mpeg2. Most AVC cams will encode to 17 mbps and 25mbps in AVC, so I figured at 25mbps I would be safe?

I am open to other good master backup solutions to fit my needs... but so far, all of the vegas default codecs seem to blow...

I am also trying to avoid a 150-200mbps output such as Avid's DNxHD codec, or lossless formats such as lagarith that will run me 3GB per minute as it is a bit large to keep as backup due to the amount of footage we need to keep around on our raid5 boxes.


also,for Crappy-Qt, I totally agree ;) I haven't searched yet for threads explaining what needs to be on/off for QT, but do you happen to have a link to a thread or the information as to what needs to be disabled?

[magnatique]

If the problem is that you can't open these "High Quality" files in Vegas , it would be probably that the MainConcept decoder can't handle that profile , or it simply can't decode them quickly (Not Multithreaded) but I don't think that would lead to a crash ?

Just my two cents :)

yeah I did read on the board a post about benchmarks of decoders, and it mentionned mainconcept had probs with High profile... this is what I would try today.

[magnatique]

What you think about GPU assisted decoder for Vegas http://www.divideframe.com/?p=gpudecoder You maybe need one graphic card with CUDA support (Nvidia) but decoding will incrase much, and you maybe newer have problems with various H264 streams because CUDA is capable to decode everything.



yes!

I actually sent in an email or two to the developper, it does a perfect job playing it at 29.97 without a problem with just a 8600 gts... problem is, when I was doing some tests and seeking at different places, it'd bug out and send a lot of error (in debug mode) not being able to find the frame or something of the sort... waiting for an answer , but that would be the best solution to bypass vegas' decoder.

[Chengbin]

Since BluRay has a max of 25mbps roughly

No, Blu-ray has a max of 40mbps.

[shon3i]

No, Blu-ray has a max of 40mbps.
to be more precise 30mbps for video only, other 10mbps are for other streams whole BD can't exceed 40mbps

[magnatique]

to be more precise 30mbps for video only, other 10mbps are for other streams whole BD can't exceed 40mbps

my bad, I had kept the 25mbps on toip of my head as I use quality based encoding to stay in between 25-30 mbps, so I guess I am still safe ;)


could anyone suggest some prefered settings to try to use no matter what as they directly impact quality, and some other settings to disable for good QT compatibility and easier playback?

[Chengbin]

to be more precise 30mbps for video only, other 10mbps are for other streams whole BD can't exceed 40mbps

From Wikipedia (I know)

BD Video movies have a maximum data transfer rate of 54 Mbit/s, a maximum AV bitrate of 48 Mbit/s (for both audio and video data), and a maximum video bitrate of 40 Mbit/s

[poisondeathray]

also,for Crappy-Qt, I totally agree ;) I haven't searched yet for threads explaining what needs to be on/off for QT, but do you happen to have a link to a thread or the information as to what needs to be disabled?

Search engine here isn't the greatest, but there was a thread on that topic

Some things off the top of my head: AVC Main (no I8x8, no CQM) , no 8x8dct, no reference b, no b-pyramid, consec b-frames 1

[HWK]

No, Blu-ray has a max of 40mbps.

to be more precise 30mbps for video only, other 10mbps are for other streams whole BD can't exceed 40mbps

No, It is not. Max is 54 MB/S. Of 40 is located to Video and 12 is to audio.

It is under FAQ on
http://www.blu-ray.com/faq/#bluray_speed under section 1 question 7. Also take a look at 2.4 What is the difference between Blu-ray and DVD? It is on same site.

[magnatique]

Search engine here isn't the greatest, but there was a thread on that topic

Some things off the top of my head: AVC Main (no I8x8, no CQM) , no 8x8dct, no reference b, no b-pyramid, consec b-frames 1

ok, let me try these up ;)

quite long to do these tests as doing a small sample works fine in vegas heh. (I guess I should create a whole bunch of profiles and just batch encode them for 36 hours heh)

but so far, these caused vegas to crash :

--profile main --tune fastdecode --crf 13.0 --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --no-weightb --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 25000 --qcomp 0.5 --merange 12 --subme 2 --partitions p8x8,b8x8,i4x4,p4x4 --no-8x8dct --trellis 0 --no-mixed-refs --mvrange 511 --sar 1:1

--profile high --level 4 --tune fastdecode --crf 13.0 --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 25000 --qcomp 0.5 --merange 12 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --sar 1:1

[poisondeathray]

I think problem is Vegas x64 - everything crashes. I don't think any "magical" setting will help you... Other uses report the same thing, and not only with x264, but with Mainconcept and Sony encoded files as well

I have experienced no crashes on 32-bit version on ~1hr sequences. Maybe you have to wait for a bug fix from Sony

[magnatique]

I am using the 32-bit version as well :/

[poisondeathray]

I am using the 32-bit version as well :/

Hmm, there goes my theory :confused:

Maybe your system is unstable? overheating? ram?

What kind of crashes and error messages are you getting?

[magnatique]

Hmm, there goes my theory :confused:

Maybe your system is unstable? overheating? ram?

What kind of crashes and error messages are you getting?

tested on 3 different boxes, to be certain it wasn't a hardware issue.

E6600 boxes, 4GB ram, WIn XP

they do have similar setup all of them I tested on, so I might give a shot at testing on one of our Q6600 or even on my laptop to see if this happens there as well, but I am fairly confident it will be the same.

I fed about 10 different enciode profiles to run overnight on 2 boxes, I will update the thread with my findings ;)

[magnatique]

ok, after encoding over 30 20 min samples, I think I have a Vegas-Friendly encode now.

No More crashes, playback full 29.97 " 75% CPU on my E6600 CPU with 2 GB Ram.

Findings :

I couldn`t quite put the finger on what caused the H264 to crash in Vegas.

- The Fast Decode tuning gave some help for certain.
- Profiles / Levels had little or no impact on the speeds.
- Seeking was TERRIBLE with high GOP. the default of 250/25 was taking easily 3-8 seconds to seek at the proper area of the video. I used 30/2 Following the faq that mentionned to use a minimum size = framerate, but 24/2 worked as good.
- MV prediction / Trellis seemed to have minimal impact only.
- using a ME range of 12, Hexagon and subme2 greatly helped with smoother playback in Vegas
- b-pyramid had little to no impact
- adding a -vbv maxrate seemed to help playback (probably lower bitrate result = easier to playback) (40k max I used)
- CRF of 16 seemed to be the optimal level (matched with 40k max vbv) for me with visual quality and good playability. allowed me to stay around 75-80% CPU whereas by comparison CRF 14 would be between 90-100% cpu and cause stutters.


I tried staying within the bluray guidelines... here is my final config. If anyone spots something that would make it not BluRay complient without re-encoding, can you please let me know?

program --profile high --level 4.1 --preset fast --tune fastdecode --crf 16.0 --thread-input --deblock -1:-1 --keyint 30 --min-keyint 2 --b-adapt 2 --direct auto --ipratio 1.1 --pbratio 1.1 --vbv-maxrate 40000 --no-mbtree --merange 12 --subme 2 --partitions all --trellis 0 --mvrange 511 --nal-hrd --sar 1:1 --output "output" "input"

[shon3i]

For full blu-ray compilancy you need to add these swiches to your cmd.

--vbv-bufsize 30000 --aud --slices 4

[Chengbin]

- using a ME range of 12, Hexagon and subme2 greatly helped with smoother playback in Vegas


That can't be true. merange, motion estimation (me), and subme has nothing to do with decoding.

[magnatique]

That can't be true. merange, motion estimation (me), and subme has nothing to do with decoding.

unless doing so turned on/off something else that Vegas didn't like?

I can do a new test to compare my current config with a different range/subme and see what happens, maybe I mistakenly assumed it was these changes, when in fact it was something else.

[magnatique]

For full blu-ray compilancy you need to add these swiches to your cmd.

--vbv-bufsize 30000 --aud --slices 4

than you very much :)