Selam! (XviD-1.0-Beta3-26122003)

[Chainmax]

Blue_MiSfit: I don't think my script is that heavy. KernelDeint is there in place of Telecide's default postprocessing, and Undot+Smoother seems to be a widely used combination on scripts around here. Only VagueDenoiser could be considered an extra load and even there the number of nsteps is reduced so as to keep CPU usage as low as possible.
Maybe the low speed I sense comes from the fact that I used to have 256MB of PC133 ram and now I have 160MB of PC100 ram (long story). In any case, I do want to upgrade my PC, but newegg.com doesn't accept international CCs and stuff is way too expensive here in uruguay (an Athlon XP 2500+ goes for ~140 bucks) :(.

[Blue_MiSfit]

it's not too bad... try an encode with just resize and undot... if its way faster then we'll know...

Anyways abot not being able to get parts yeah tahts a real bitch. a real bitch. I would say scour on resellerratings.com and try to find a reseller that will ship internationally...

best of luck

~misfit

[Arcon]

Originally posted by Koepi
can you retest the 2nd pass with setting iframe boost to 5 (or 10) percent and report back?

now i used an iframe boost of 7%, target size was 1116616k, result is 1108874k, so again ~8mb undersized.

[Koepi]

arcon:

do you have any zones defined with fixed quantizers (in the first pass)?

Regards,
Koepi

[Arcon]

Originally posted by Koepi
do you have any zones defined with fixed quantizers (in the first pass)?

i've got 2 zones, but both have a weight (1.0 and 0.4). the second one is grayscale. so no fixed quantizers.

[Koepi]

Weight zones still are buggy - don't use them, they destroy size predictability.

like i wrote in the _very first post_ of this thread: Known bugs (do not report them):
- Weight zones don't work properly - if you need zones, use quant zones instead.

Regards
Koepi

[cretin]

I made some tests with 2 and 3 consecutive B frames with the following settings:
Everything was the same except the B frame settings which were
2/1.50/0.75
3/1.50/0.75

The resulting avi-s were identical, both had the same amount of B frames, i even made a "compare by content" with windows commander and the two files were 100% identical. I would like to know if is this normal, or is there any way to force to codec to make more B frames ?(maybe the BVOP sensitivity in Zone controll ??) Its just allowing 3 bframes should have some advantage over 2 B frames by default or not ?

[mikeson]

@cretin:
is there any way to force to codec to make more B frames ?(maybe the BVOP sensitivity in Zone controll ??)
Yes, increase BVOP sensitivity to get more b-frames. Anyway setting Max consecutive BVOPs works for me. What does stats file say?


When you set BVOP sensitivity to 90, you will always get amount of b-frames specified in Max consecutive BVOPs

[sysKin]

Originally posted by cretin
maybe the BVOP sensitivity in Zone controll ??Exactly. I answered similar question not a long time ago.Its just allowing 3 bframes should have some advantage over 2 B frames by default or not ? [/B]When I tuned the b-frame decision, it turned out that the average PSNR of all frames is higher with one b-frame, then almost identical with the second (at its best thresholds I could find) and then again identical with the third - again at its best thresholds, which turned out to be so low, that 3rd b-frame is quite rare.

B-frames have some other advantages, mostly making the stream more robust to data corruption, DCT mismatch, and so on. In most cases you should use either 1 or 2, I can't really tell the difference. If you're going to write to XCD, or play at standalones (iDCT mismatch possible) you might want to use 3 with higher sensitivity (5..10). Some lossy radio channels would probably work even better at higher settings ;)

Default decision aims at maximum PSNR and as a result, inserts 3rd b-frame very rarely.

Radek

[symonjfox]

And what appends if I set 15 or more consecutive b frames?
In theory the I P B decision should never use 10 consecutive b frames, and the final stream will have the right amount of b frames that are needed for this GOP? Am I right?

PS: The very first time I used xvid (some time ago) I set max bframes at 10, I made a small encode, but it worked well and it was played back perfectly. The quality was good. But then I read lots of guides, forums, Doom9's Forum :D and I always set it to 2 or 3.

PS2: I'm making a small test ... gimme 30 minutes and I'll back

I set 15 consecutive B frames.

[Koepi]

you can set more bframes, but with defaults, you'll get max 3 bframes in a row anyways. so it should b safe to use higher values.

Unfortunately i discovered that some encodes _do_ look better when only using 1 bframe max. (i.e. matrix).

Regards
Koepi

[mikeson]

@Koepi:
Unfortunately i discovered that some encodes _do_ look better when only using 1 bframe max. (i.e. matrix).
Have you discovered any 'rule' that describes on what sources (dark/noisy as Matrix is) is better to use 1 bframe instead of 2?

[Koepi]

I think it depends on the noise in the source. But i didn't make extensive tests yet.

Regards
Koepi

[symonjfox]

Results:
AVS:LoadPlugin("C:\Programmi\Avisynth 2.5\plugins\mpeg2dec3.dll")
Mpeg2Source("VTS__01_P01.16~9_1.d2v",idct=7,ipp=false, CPU=5)
Crop(8,16,-8,-16)
Lanczosresize(544,544)
Lengh 6:40

Xvid: 2 passes, 700 kbs average (to test b frames with medium-low bitrate).
All defaults except 15 max B frames, No packet bitstream, AR 16:9, VHQ 1.

Results: 33 MB, Visual quality is good, but in my opinion not so much (maybe the 544*544 res?), there are no defects or problems while playing (tried Nero decoder and FFDshow).
I opened pass.stat using StatsReader. It surprised me: for example there was a very dark scene and in this scene there was 1 GOP composed by 1 I, 15 B, 1 P, 15 B, 1 P and 15 B! Bitrate in this scene was very low.
In the rest of the clip, the encode is quite normal: 1 or 2 max consecutive bframes. Sometimes I found 3 or 4.

Conclusion: I think that setting 15 or 20 Max B frames can be a good way to improve low bitrate encodes, or a good way to store more than 120 mins in 1 CD with decent quality. Maybe playing with the BVOP sensivity would help (for example -5 -10 for high quality encodes and 5 10 15 for low bitrate encodes).

[Arcon]

Originally posted by Koepi
Weight zones still are buggy - don't use them, they destroy size predictability.
sorry, i didn't remember that.

[MajinMarc]

I just did a test using Adaptive Quantization and I have to say that I am quite impressed. I used it on Rurouni Kenshin OVA ACT 1 and it decreased the size dramatically without losing quality. However I did notice that if I use it in conjunction with Warpsharp it produces funny results. Oftentimes a wierd gray block would apear in the middle of a completely back part. I'm not exactly sure why this is yet but I'm going to continue testing different settings to see if I can isolate the problem.
By the way jsut because I'm curious and have yet to try, in the old version of Xvid there was a little note about B-frames having a quality lowering effect. Is this still true for beta-3?

[mikeson]

@MajinMarc:
in the old version of Xvid there was a little note about B-frames having a quality lowering effect. Is this still true for beta-3?
Not exactly. B-frames are frames with higher (mostly) quantizer than I and P frames, so quality of B-frames is a little bit worse than I and P frames, but overall movie quality is better because more bits are spared for problematic scenes. Do a Search on B-frames, there are many threads explaining them in depth.

[sh0dan]

There IS problems with the Xvid Dshow decoder and colorspaces.

1) (very obvoius) The force-list is upside down. Forcing YV12 forces it to RGB24, selecting RGB24 actually forces YV12. YUY2 and RGB32 is also swapped.

2) YV12 also selects IYUV.

3) Flipped video. This is NOT a hardware issue.

If you have the time, you should have another look at it, and try making it a bit more reliable. Many problems emerge from using ffdshow in "Raw Mode", but it _is_ working on a bunch of other dshow filters. The video is flipped in both YUY2 and YV12, which suggests the XviD decoder is doing something funky that other filters aren't.

[happy_harry]

@sh0dan: Are you sure you're not using dvobsub?
dvobsub causes some problems on my winxp with xvid1.0b3.

[sh0dan]

@happy_harry: Yes - it is not loaded into the filter graph. And the problem persists, if I construct the graph manually in graphedit.

[Leak]

Originally posted by sh0dan
@happy_harry: Yes - it is not loaded into the filter graph. And the problem persists, if I construct the graph manually in graphedit.

Strange - I just tested it in Graphedit, and the only time I'll get an upside down picture is when DirectVobSub is loaded. If I disable it, XviD will be connected directly to the VMR9 renderer and all is peachy, be it RGB, YUY2 or YV12. Oh, and you're right, the list in the combobox is totally upside down as well. :)

Of course, just checking the "flip picture vertically" checkbox in DVobSub fixes this, but it's still strange, as it never happened with ffdshow using any output colorspace.

np: Static/Justine Electra - Inside Your Heaven (Flavour Has No Name)

[Nicholi]

And now for something completely different.

Ran across a very strange error during the 2nd pass of my happy anime encode.

XviD settings: (otherwise default besides these specified)
BVOPs: Off completely.
Chroma Optimizer: On
VHQ Mode: 0 - off
Max I-frame interval: 230

The encode itself is also using VFR if that matters at all (was planning to plug it into Matroska all nice and happy like), though I wouldn't think so. The first pass finishes quite nicely in under 30 min from the huffy avi with a size of 169,216,000 bytes. Seems to play normally and has no problems.
As I've read the 1st pass (in 2pass mode) of beta3 is no longer the same full quality as it used to be and a 2nd pass should be followed. So I did so entering 180,000 kbytes as my desired size and let it whir on. Vdub (1.5.10 build 18160) died at frame 1868. I tried a 2nd pass again only to fail. My good friend Gizmo suggested it may be the stats file so I ran another 1st pass and it died again at the same place on the 2nd pass.

Here is VDub's lovely crash message which has literally no meaning to me.

VirtualDub crash report -- build 18160 (release)
--------------------------------------

Disassembly:
01701460: 48 dec eax
01701461: 0cc7 or al, c7
01701463: 40 inc eax
01701464: 1000 adc [eax], al
01701466: 0000 add [eax], al
01701468: 00ba01000000 add [edx+01], bh
0170146e: 8955f0 mov [ebp-10], edx
01701471: 8d93e8000000 lea edx, [ebx+e8]
01701477: 8b4df8 mov ecx, [ebp-08]
0170147a: 52 push edx
0170147b: 51 push ecx
0170147c: e867c6ffff call 016fdae8
01701481: 83c408 add esp, 08
01701484: 8b55f8 mov edx, [ebp-08]
01701487: 8b4df4 mov ecx, [ebp-0c]
0170148a: 52 push edx
0170148b: 51 push ecx
0170148c: e857c6ffff call 016fdae8
01701491: 83c408 add esp, 08
01701494: 8b8b14450100 mov ecx, [ebx+14514]
0170149a: 8b9318450100 mov edx, [ebx+14518]
017014a0: 899314450100 mov [ebx+14514], edx
017014a6: 8b55d0 mov edx, [ebp-30]
017014a9: 898b18450100 mov [ebx+14518], ecx
017014af: 899320450100 mov [ebx+14520], edx
017014b5: 89b31c450100 mov [ebx+1451c], esi
017014bb: be01000000 mov esi, 00000001
017014c0: ff8324450100 inc dword ptr [ebx+14524]
017014c6: 8975f8 mov [ebp-08], esi
017014c9: 8b4d90 mov ecx, [ebp-70]
017014cc: 8bd1 mov edx, ecx
017014ce: 83e207 and edx, 07
017014d1: 743c jz 0170150f
017014d3: 2bca sub ecx, edx
017014d5: 83c108 add ecx, 08
017014d8: 83f920 cmp ecx, 20
017014db: 894d90 mov [ebp-70], ecx
017014de: 722f jc 0170150f
017014e0: 8b5588 mov edx, [ebp-78]
017014e3: 8b4d94 mov ecx, [ebp-6c]
017014e6: 895584 mov [ebp-7c], edx
017014e9: 8b7108 mov esi, [ecx+08] <-- FAULT
017014ec: 8975d8 mov [ebp-28], esi
017014ef: 8b45d8 mov eax, [ebp-28]
017014f2: 0fc8 bswap eax
017014f4: 8945d8 mov [ebp-28], eax
017014f7: 8b55d8 mov edx, [ebp-28]
017014fa: 8b4d94 mov ecx, [ebp-6c]
017014fd: 83c104 add ecx, 04
01701500: 895588 mov [ebp-78], edx
01701503: 8b7590 mov esi, [ebp-70]
01701506: 83c6e0 add esi, e0
01701509: 894d94 mov [ebp-6c], ecx
0170150c: 897590 mov [ebp-70], esi
0170150f: 8b936c450100 mov edx, [ebx+1456c]
01701515: 85d2 test edx, edx
01701517: 0f8434030000 jz 01701851
0170151d: 83bb2845010000 cmp dword ptr [ebx+14528], 00
01701524: 0f8400060000 jz 01701b2a
0170152a: 8b4df0 mov ecx, [ebp-10]
0170152d: 85c9 test ecx, ecx
0170152f: 0f85ea050000 jnz 01701b1f
01701535: 85ff test edi, edi
01701537: 0f85c9010000 jnz 01701706
0170153d: bf01000000 mov edi, 00000001
01701542: e95bfbffff jmp 017010a2
01701547: 8d4412e8 lea eax, [edx+edx-18]
0170154b: 8b7d84 mov edi, [ebp-7c]
0170154e: baffffffff mov edx, ffffffff
01701553: d3ea shr edx, cl
01701555: 8955e8 mov [ebp-18], edx
01701558: 85c0 test eax, eax
0170155a: 7e79 jle 017015d5
0170155c: 237de8 and edi, [ebp-18]
0170155f: 8b db 8b

Windows 5.1 (Windows XP build 2600) [Service Pack 1]

EAX = 01a00e50
EBX = 01a00d80
ECX = 00eebff8
EDX = 00000000
EBP = 0306f080
DS:ESI = 0023:00000001
ES:EDI = 0023:00000000
SS:ESP = 0023:0306f004
CS:EIP = 001b:017014e9
FS = 003b
GS = 0000
EFLAGS = 00010246
FPUCW = ffff027f
FPUTW = ffffaaaa

MM0 = cfcfcfcfcfcfcfcf
MM1 = cfcfcfcfcfcfcfcf
MM2 = cfcecbc8c8cbcdcd
MM3 = 00cf00ce00cb00c8
MM4 = cf81cf7bcf81cf7b
MM5 = cf81cf7bcf81cf7b
MM6 = cf81cf7bcf81cf7b
MM7 = cf81cf7bcf81cf7b

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'xvid'...

...while decompressing video frame 1868 with "XviD MPEG-4 Codec" [biCompression=44495658] (VideoSource.cpp:1567)...

...while running thread "Processing" (thread.cpp:120).

Thread traces:

Thread 00000bcc (Main thread)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1786)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1768)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1786)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1768)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1786)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1768)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1786)
C:\p4root\dev_stable\VirtualDub\source\FilterSystem.cpp(569)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(618)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(648)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1768)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1786)
C:\p4root\dev_stable\VirtualDub\source\FilterSystem.cpp(429)
C:\p4root\dev_stable\VirtualDub\source\FilterSystem.cpp(569)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(618)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(648)
Thread 00000f28 (FastWriteStream)
Thread 00000308 (Processing)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1598)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(1946)
C:\p4root\dev_stable\VirtualDub\source\VideoSequenceCompressor.cpp(389)
C:\p4root\dev_stable\VirtualDub\source\VideoSequenceCompressor.cpp(406)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(2103)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(2143)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(1941)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1563)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1598)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(1946)
C:\p4root\dev_stable\VirtualDub\source\VideoSequenceCompressor.cpp(389)
C:\p4root\dev_stable\VirtualDub\source\VideoSequenceCompressor.cpp(406)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(2103)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(2143)
C:\p4root\dev_stable\VirtualDub\source\Dub.cpp(1941)
C:\p4root\dev_stable\VirtualDub\source\VideoSource.cpp(1563)
Thread 00000e50 (Dub-I/O)

Thread call stack:017014e9: xvid!xvid_decore [016f0000+9f68+7581]
016f9ff3: xvid!xvid_decore [016f0000+9f68+8b]
016f1d03: xvid!00001d03
016f9f99: xvid!xvid_decore [016f0000+9f68+31]
016f231e: xvid!0000231e
016f8977: xvid!DriverProc [016f0000+873c+23b]
77f7e358: ntdll!RtlInvertRangeList [77f50000+2e26c+ec]
77f7e358: ntdll!RtlInvertRangeList [77f50000+2e26c+ec]
77e7b063: kernel32!GetModuleFileNameA [77e60000+1ada9+2ba]
77e7b085: kernel32!GetModuleFileNameA [77e60000+1ada9+2dc]
77e7aeb7: kernel32!GetModuleFileNameA [77e60000+1ada9+10e]
77f944a8: ntdll!RtlRemoteCall [77f50000+442ea+1be]
77f944a8: ntdll!RtlRemoteCall [77f50000+442ea+1be]
77f57f98: ntdll!RtlAllocateHeap [77f50000+7bae+3ea]
77f58a3a: ntdll!RtlAllocateHeap [77f50000+7bae+e8c]
77f944a8: ntdll!RtlRemoteCall [77f50000+442ea+1be]
77f58497: ntdll!RtlAllocateHeap [77f50000+7bae+8e9]
77f57f98: ntdll!RtlAllocateHeap [77f50000+7bae+3ea]
77f58a3a: ntdll!RtlAllocateHeap [77f50000+7bae+e8c]
77d44d8f: USER32!ClientThreadSetup [77d40000+4d57+38]
77f5b554: ntdll!NtAllocateVirtualMemory [77f50000+b548+c]
77f834de: ntdll!RtlSizeHeap [77f50000+33316+1c8]
77f596da: ntdll!RtlFreeHeap [77f50000+8a3e+c9c]
77f576f1: ntdll!LdrGetDllHandle [77f50000+718e+563]
77fb172e: ntdll!RtlConvertUlongToLargeInteger [77f50000+616c0+6e]
77f58497: ntdll!RtlAllocateHeap [77f50000+7bae+8e9]
77f57f98: ntdll!RtlAllocateHeap [77f50000+7bae+3ea]
77f58a3a: ntdll!RtlAllocateHeap [77f50000+7bae+e8c]
77f9790d: ntdll!RtlUnhandledExceptionFilter [77f50000+47865+a8]
771256e2: OLEAUT32!SafeArrayCopyData [77120000+53c7+31b]
77f5b584: ntdll!NtCallbackReturn [77f50000+b578+c]
77d44e9e: USER32!ClientThreadSetup [77d40000+4d57+147]
77f75dba: ntdll!KiUserExceptionDispatcher [77f50000+25dac+e]
77f5b644: ntdll!NtContinue [77f50000+b638+c]
77f75dc8: ntdll!KiUserExceptionDispatcher [77f50000+25dac+1c]
77e73887: kernel32!RaiseException [77e60000+13837+50]
77f944a8: ntdll!RtlRemoteCall [77f50000+442ea+1be]
77f58497: ntdll!RtlAllocateHeap [77f50000+7bae+8e9]
77e73887: kernel32!RaiseException [77e60000+13837+50]
77f5c244: ntdll!NtSetInformationFile [77f50000+c238+c]
77e7f0ce: kernel32!SetFilePointer [77e60000+1f02e+a0]
73bd181d: MSVFW32!ICSendMessage [73bd0000+17f4+29]
73bd181d: MSVFW32!ICSendMessage [73bd0000+17f4+29]
73bd47c6: MSVFW32!ICDecompress [73bd0000+478b+3b]
004a0712: VideoSourceAVI::streamGetFrame()
00492eac: AVIOutputFile::writeIndexedChunk()
00463870: Dubber::WriteVideoFrame()
0045b68f: AVIPipe::getReadBuffer()
0046411e: Dubber::ThreadRun()
77e73887: kernel32!RaiseException [77e60000+13837+50]
77f5b884: ntdll!NtDuplicateObject [77f50000+b878+c]
77e7f01b: kernel32!DuplicateHandle [77e60000+1efb6+65]
004aed3e: VDThread::StaticThreadStart()
004c6cac: _threadstartex@4()
77e7d33b: kernel32!RegisterWaitForInputIdle [77e60000+1d2f8+43]

-- End of report

Lotta spam I understand if you all hate me now. I'm guessing its some strange problem on my machine, though I'm not sure only recently formatted and haven't encoded a thing yet so I'm just going to go back to beta2 and see what happens. Best of luck on beta4 :) looking forward to it.

[Soulhunter]

Originally posted by symonjfox
Maybe playing with the BVOP sensivity would help (for example -5 -10 for high quality encodes and 5 10 15 for low bitrate encodes). Would this be better than using the 2/3 but with lower quantizers... ??? :confused:

[seewen]

@Radek & Doom9
Sorry, I thought it was easy to do it. (it was in dev-api-3, so I thought... )

But why did you remove the DXN profiles ?
At least for the few encodes I made with Beta 1 & 2, it worked well.
---

@Cretin (Ca c'est un pseudo qui te convient à merveille)
DP-500 = DP-450 + NC
Both have the same chipset/problems.

Companies have to pay if they want the little "DXN logo" on their products... And I bet that's why DP-450, 508, 1000, 1500, 1504, etc.. doesn't have the little logo.

But till today I thought that everybody would understand that a "DXN certified" player remain certified even if you take the NetworkCard off, or if you add a HardDrive or you change the Box...

[symonjfox]

Originally posted by Soulhunter
Would this be better than using the 2/3 but with lower quantizers... ??? :confused:
I don't know ... I have many tests to do to understand if this is good or bad.

The only thing I surely know, is that in 2 pass encodes, B frames help a lot (expecially in 1 CD rips), and IMO using up to 10 or more consecutive B frames shouldn't be bad, because in the few tests I made, this only appends in very very still and dark scenes (so there's no visual difference with the original). The I P B decision algorithm is very good, if you set 10 consecutive B frames, it surely use 2 or 3 consecutive B in normal encodes, so there's no matter.

My next tests will show if I'm right or not.

I just launched some ideas that I have never heard before. AFAIK nobody used 10, 15, 20 consecutive B frames, just because Dev3 algo wasn't smart enough and also developers sayd to set it to 3 or 4 at max. Now as far as I can see, there sholud be no problem, maybe some standalone compatibility problem?

[sysKin]

Originally posted by Nicholi
Here is VDub's lovely crash message which has literally no meaning to me.The stack trace is a very weird combination of xvid's decoder (!) and nvidia's video driver (!!). Very very strange...
Does anyone else have had any crash with beta3?

Originally posted by seewen
@Radek & Doom9
Sorry, I thought it was easy to do it. (it was in dev-api-3, so I thought... )No it wasn't there either... XviD's decoder never respected maximum bitrate setting, even if GUI pretended otherwise.
But why did you remove the DXN profiles ?
At least for the few encodes I made with Beta 1 & 2, it worked well.It's one thing to 'borrow' profiles from DXN - they probably wouldn't mind - but another thing to borrow profiles and *not respect them*. I don't think DXN would forgive us doing so.
Or users, btw.

Radek

[bugsan]

(edit: i am using a cvs version compiled yesterday)
this psnr test shows that bframes are bad with high motion scenes, and good with low motion scenes.
with a low motion scene, psnr (overall) is increased by 0.78
with a medium motion scene, psnr (overall) is increased by 0.28
with a hi motion scene, psnr (overall) is decreased by 0.16

i think ibp decision algorythm is bad.


Low Motion: 500kbps
Med Motion: 800kbps
Hi Motion: 1600kbps

|---------------|---------------|---------------|
| Low Motion | Med Motion | Hi Motion |
|-------------------|------------ --|---------------|---------------|
| Options | PSNR | PSNR | PSNR | PSNR | PSNR | PSNR |
| | |overall| |overall| |overall|
|-------------------|-------|-------|-------|-------|-------|-------|
| minimum |44.1442|43.8022|44.7109|43.9511|43.2975|43.0578|
| bf2 sens-20 |44.7117|44.4438|44.8077|44.1885|43.3478|42.9984|
| bf2 |44.8131|44.5867|44.8473|44.2396|43.3023|42.8922|
| bf2 sens+20 |44.8171|44.5887|44.8759|44.2354|43.2732|42.8460|
|-------------------|-------|-------|-------|-------|-------|-------|

[DevilsChild]

How does your data show that b-frames are bad with high-motion scenes? Didn't you use different bitrates for the different scenes? Maybe they increase quality at low bitrates and slightly decrease quality at high bitrates?

[bugsan]

How does your data show that b-frames are bad with high-motion scenes? Didn't you use different bitrates for the different scenes?

bitrates are different because compressibility for each scene is different. High motion scenes will get always more bits than low motion scenes ;)


Maybe they increase quality at low bitrates and slightly decrease quality at high bitrates?

sorry to say that, but in a movie there are high bitrate scenes (~high motion scenes) and we could increase quality of these scenes by removing bframes from them.

[symonjfox]

I agree with you (B frames in high motion).

But this is only "mathematically" speaking.
Yes, high motion scenes need higher bitrate to get the same quality as a slow scene, BUT our eyes have a different perception of quality. If you have a blocky slow scene you surely see it's bad; if you have an high motion blocky scene, maybe you won't notice it since it's too fast (if you watch it frame by frame ... off course you see it).

[mikeson]

Originally posted by bugsan
i think ibp decision algorythm is bad.
Why should it be bad? You hardly notice worse b-frame in high-motion, but when in low-motion it would be more noticable IMHO.

[bugsan]

Why should it be bad
I think bframes should be totaly droped above 5% of key-blocks/frame (kblk).
kblocks are more quantized in bframes because bframes have a higher quant. and those blocks give a lower psnr...

maybe i am wrong...

[Nicholi]

Returning from before, beta2 finished the encode just fine 1st pass. And I tried a 2nd pass just to see what happened and no crash.

Perhaps I'll try clearing all my nvidia drivers and reinstalling them again and try beta3 some other time.

[Tommy Carrot]

I'm totally satisfied with the current b-frame decision algorithm, i don't think it should be changed. The older xvid versions had too restrictive algorithm, and the quality gain was much less noticable with them than right now with b-frames. Bugsan, you know PSNR is not everything, you shouldn't base your opinion on it.

[sh0dan]

Originally posted by sysKin
The stack trace is a very weird combination of xvid's decoder (!) and nvidia's video driver (!!). Very very strange...
Does anyone else have had any crash with beta3?


I don't see any references to the nVidia drivers - where are you seeing this?

A strange thing is however that it happends in the DEcompressor, which shouldn't be used for the operation you describe (unless I'm reading it wrongs).

MSVFW32!ICDecompress is the decompressor called from Vdub to fetch frames, but if you are encoding from a HuffYUV file it doesen't really make much sense that the decompressor is invoked. Weird!

[EthanoliX]

Talking about b-frames I have a question:

When doing a 2-pass encode, is the ipb decision algorythm used to affect bitrate distribution? Or is it only a matter of setting the quantizers to get the desired rate during a specific scene?

[sysKin]

Originally posted by bugsan
I think bframes should be totaly droped above 5% of key-blocks/frame (kblk).
kblocks are more quantized in bframes because bframes have a higher quant. and those blocks give a lower psnr...B-frames don't have kblocks.
I never said my p/b decision is the best possible, but it is the best so far, mostly because there is no competitors :D. You're free to join the 'race' for best decision code ;). The rule is that if someone's code is better, it gets used in cvs :)

Lack of competition isn't good.

Radek

[mikeson]

@Nicholi:

Please check if VirtualDub.subset.AddRange(x,x) in VirtualDub[Mod].jobs does not exceed amount of frames you're encoding (it may happen when you insert encoding into queue (jobs) and then later change trim in AviSynth script).

[mikeX]

i'd say about 7 out of 10 of my encodes with vdubmod & avisynth don't make it through the second pass since xvid 1 beta 1 :(

the latest crash went like this:

i made a basic avisynth script with a d2v source & fed it to virualdubmod through job control (2 passes):
avisynth 2.53 build nov 11 2003
virtualdubmod 1.5.10.1 build 2047
xvid 1 beta 3 (koepi's)

i got a non fatal (for vdubmod) error at about 80% of the second pass.
i couldn't start the pass again cause i got an avisynth read error as soon i tried to.
so i closed and reopened vdubmod and started the second pass again this time not through job control.
very close to the end of the movie (less than 1 min till the end of the pass, didn't look at anything else, like frame number etc, at the time) i got the following error:

Avisynth read error: access violation at 0x12101010 attempting to read from 0x12101010

doing as mikeson advised nicholi i found that vdubmod sellects a range of 170264 while gordian knot gives 170263 as the last frame when i open my d2v project.

i aslo found this at the job file, regarding the failed second pass i made through job control:
// $error "Avisynth read error:
Avisynth: caught an access violation at 0x015c36e7,
attempting to write to 0x77767b84"

i guess the second crash could be explained by the fact that the first pass worked on 170264 frames whereas that last pass was dealing with 170263 frames
looking at the stats file from the first pass i found that these were the last 3 frames

b 3 0 480 0 146 138
b 3 0 480 0 139 129
b 3 0 480 0 139 127

the stream should end with a p or I frame right?

but what about the first crash (the one through job control)?

why didn't it crash at the end as well???

all of my older crashes don't seem to occur at the end of the file either but i don't have any more info about them (just a couple of 'crashinfo.txt' files from vdubmod)

should i post those crashinfo files or is it really not an xvid bug?

[mikeson]

I've got a little bugreport here. It seems (at least that is what I've discovered) that XviD 1.0 Beta3 doesn't encode last frame.

Hardware:
P4 2.6GHz
1GB RAM
GeForce4 4400Ti

Software:
AviSynth 2.5.3
Koepi's XviD 1.0 Beta3
VirtualDubMod 1.5.10.1 (build 2389)
WinXP SP1

In XviD I've tried default settings and also my personal settings:
- MSP6
- Chroma motion
- VHQ4
- Trellis quantization
- MPEG quantization type
- Adaptive quantization
- GMC
- Qpel
- BVOPs(2,1.50,1.00)
- Packed bitstream
- Chroma optimizer

AviSynth script:
LoadPlugin("D:\Program Files\XviD\AviSynth_filters\yv12\MPEG2Dec3dg.dll")
mpeg2source("i:\Bug's Life.d2v",cpu=0,idct=7)
trim(0,1000)
crop(0,72,720,432)
LanczosResize(720,304)
Limiter(16,235,16,240)

...and encoded this short clip via XviD 1.0 Beta3, DivX 5.1.1 and CorePNG.

Number of frames reported by VirtualDubMod:
- AviSynth script itself - 1001 frames
- DivX - 1001 frames
- CorePNG - 1001 frames
- XviD - 1000.

In Java XviD Stats Viewer number of frames was 1000.
In Koepi's StatsReader 2.1 number of frames was 1003 (maybe it counts first three 'non-video' lines).

[EDIT]
When I do not use Packed bitstream, number of frames in stats file is:
- by Java XviD Stats Viewer - 999 frames
- by Koepi's StatsReader - 1002 frames
[EDIT]

[crusty]

Hi all, happy new year and just some remarks.

Sorry if any of this has been answered before and just hit me if you no like < Crusty straps on crash helmet> :D
It's been a while since my last encode and my last readup on this forum.

Syskin:
B-frames have some other advantages, mostly making the stream more robust to data corruption
Exactly why does it make it more robust? Because the information in a B-frame is less important?

Koepi:
I think it depends on the noise in the source. But i didn't make extensive tests yet.
Well that would make sense...after all B-frames carry over less accurate information than I/P-frames, so more noise in source would be more detrimental to the result if there are more B-frames....theoretically.....I think....perhaps not. :D

A question about making the IDCT decision automatic:

What kind of differences would an automatic algorithm have to sense to be able to switch automatically?
We've seen the weird discoloring effects when using differing IDCT implementations in the clip and the decoder, but how would these be translated to values in the actual frame.
I mean, how would you be able to see from the values in a macroblock that it's the 'wrong' IDCT without actually looking at the clip and seeing purple skies etc. ?
Also, would the problem lie in Keyframes, or would it also be in all other frames?

Like the developers said, it would be nice to have this done automatically, but they're not even sure if it's theoretically possible,let alone practical.

I'm no programmer and my math sucks bigtime but my first guess would be to look for the mathematical differences between the two IDCT implementations, and then to look for instances where you get different end values for the same start value.
Then, since we're talking about the decoding end here, you would have to make the decoder aware of those values that are 'known' to be 'possibly' created differently by two IDCT's.
If that would not be enough, the decoder would then have to look for more complex differences, like patterns of values.
So far it doesn't sound impossible, but it does sound very CPU intensive, and would take a lot of programming and testing.

It sounds like the manual method is so far the easiest way of implementing it, but it requires user intervention.
Since it's virtually impossible to have two versions of the decoder on a windows system at the same time, you'd be left with only a few options:
-Supply every project with it's own player, with the right codec built-in.
-Somehow create information about the correct IDCT-choice inside the videofile itself which could be detected by all newer codecs. This would be like setting a kind of 'encoder version info' inside the film. If this is impossible to do in the header of the file, it would have to be done in the content.
BTW: This sounds more intrusive than it would be. Version info could for instance be encoded in several ways. I'm talking about 'fingerprinting' the version info into a macroblock.
Please bear in mind that encrypting information (like plain text) into graphical information (like a bmp or a jpg) is very possible.
Some examples:
Take the first frame of the movie, then double it. Alter one macroblock, let's say the first one from the top left, in a predefined mathematical way to give it the version info and let all decoders from that moment on look for that 'fingerprint'. If it's not there, it's made by an older codec, and the decoder could adjust accordingly.
The altered macroblock would look like nothing more than a 'glitch' in the first frame, if noticable at all.
On second thought, you could let the decoder adjust the values in the macroblock to reflect the original values (before it was fingerprinted) and you wouldn't notice any difference at all!!

You could even make a tool that would adjust avi's made by older encoders that lack this feature. Since all new decoders would look for this feature, it would not be impossible for older encodes to be correctly displayed with newer decoders.
Sure there would be practical problems, like having to alter stuff that you already burned to a CD, but then again nothing in life is perfect.

You would btw not break any mpeg-4 compatability, since for other decoders it would be nothing more than a glitch that might make one macroblock, in one frame, look bad. Only 'fingerprint'-aware decoders would know that it is information and not noise.

It might sound complex, but on the whole I think it would be much, MUCH easier to implement than trying to have the decoder make a 'best guess' about which IDCT was used.
You could use this for the IDCT problem, but you could also use this to have the decoder automatically detect other known problems based on the version info, like for instance Q-pel smearing and modulated QM. It would make the decoder bigger, for every workaround takes up space, but not necessarily slower, since version detection would be a breeze.

Any thoughts on this?

[Wuntvor]

If im not mistaken , the reason why bframes make a file more robust is that no other frames reference a b-frame. A broken bframe will only be showed for an instant, but a broken i/p fram will then influence all the following frames that depends on it.

regards
/wuntvor

[mikeson]

I've found temporary fix to bug I've submited few posts ago (last frame not encoded in XviD). The point is in increasing frames of video by one in script (exactly copying last frame after real last frame, so last frame will be there twice).

I warn you, this fix is very ugly written:
whole = mpeg2source("i:\AntZ.d2v",cpu=0,idct=7)
part1 = trim(whole,0,111849)
part2 = trim(whole,111849,111849)
AlignedSplice(part1,part2)

Hope this works.

[bugsan]

@mikeson
this bug is caused by xvid itself. last frames are missing in the stats file AND in the encoded video.
the number of missed frames depends on max bframes value.

[RadicalEd]

Originally posted by mikeson
I warn you, this fix is very ugly written:
whole = mpeg2source("i:\AntZ.d2v",cpu=0,idct=7)
part1 = trim(whole,0,111849)
part2 = trim(whole,111849,111849)
AlignedSplice(part1,part2)

Hope this works.

You could just do DuplicateFrame(111849) :\

[mikeX]

hmmm, i made the first pass again now without job control

there is no error message but the pass is definately not complete since the xvid status window only shows 170244 frames prossesed
looking at the stats file (which is identical to the previous one) more carefully i notice this at the end:
...
p 2 115 140 225 7659 803
b 3 0 480 0 823 360
b 3 0 480 0 444 360
p 2 105 192 183 1388 880
b 3 0 480 0 164 155
b 3 0 480 0 164 149
b 3 0 480 0 200 167
b 3 0 480 0 190 158
b 3 0 480 0 156 146
b 3 0 480 0 141 136
b 3 0 480 0 146 138
b 3 0 480 0 139 129
b 3 0 480 0 139 127 (9 b-frames)

well i guess i shouldn't have set max b-frames to 20 :(
i based that on what symonjfox reported about his tests and thought i'd give it a try
the codec had no problem using 20 b-frames at the beginning of the movie though:
i 2 480 0 0 1372 1372
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
b 3 0 480 0 8 8
p 2 0 0 480 67 67

gonna try it again with max b-frames 2

rest of the settings:
ME 6 | Chroma Motion | VHQ 4 | max I frames 250 | H.263 matrix | Adaptive Quant | Q-Pel | Quant ratio 1.5 | Quant offset 0.75 | Closed GOV | Chroma Optimizer
rest @ default

[sysKin]

Originally posted by crusty
[Exactly why does it make it more robust? Because the information in a B-frame is less important?Exactly like Wuntvor said - a broken b-frame is just one broken frame. It might even be skipped completely an you won't see a difference. A broken p-frame makes everything broken until next keyframe.

Then, since we're talking about the decoding end here, you would have to make the decoder aware of those values that are 'known' to be 'possibly' created differently by two IDCT's.If decoder knew how the picture should look like, it wouldn't even require any bitstream (think about the compression ratio lol). Since it doesn't know that, it can't detect if the picture is wrong.
-Somehow create information about the correct IDCT-choice inside the videofile itself which could be detected by all newer codecs.Of course. We already know that all simple-idct encodes have XviD009 identification, but unfortunately there are also some walken-idct encodes which also have XviD009. It was a huge mistake not to change version number, or add any more info, to the changed builds.
If this is impossible to do in the header of the file, it would have to be done in the content.LOL definitely possible, all mpeg-4 encoders I know of add this signature to the file :)
You could even make a tool that would adjust avi's made by older encoders that lack this feature. Since all new decoders would look for this feature, it would not be impossible for older encodes to be correctly displayed with newer decoders.
Just like fixing aspect ratio, removing b-frame hacks etc etc it requires an mpeg-4 parser which we don't have.
Once someone would write such parser, I'd be happy to add more features to it - including this.

Originally posted by bugsan
this bug is caused by xvid itself. last frames are missing in the stats file AND in the encoded video.
the number of missed frames depends on max bframes value.It's not a bug - xvid is perfectly able to flush all b-frames from the queue once encoding is finished. xvid_encraw shows (and documents) how to do it - you set input picture to NULL and call encoder as long as it gives any bitstream output.

The only thing is that it can't be done in VfW. If you think it can be done, please add it.

Radek

[temporance]

Originally posted by sysKin
It's not a bug - xvid is perfectly able to flush all b-frames from the queue once encoding is finished. xvid_encraw shows (and documents) how to do it - you set input picture to NULL and call encoder as long as it gives any bitstream output.

The only thing is that it can't be done in VfW. If you think it can be done, please add it.It can be done in VfW because DivX5 does it. I believe a slight VfW hack was necessary to achieve this so it only works with some encoding tools like recent versions of VirtualDub. HTH.

[Koepi]

Use max bframes=1 and packed bitstream and see if a frame is missing.

(My guess is: don't play around with advanced settings if you don't know or like what they do ;) )

Regards
Koepi

[sysKin]

Originally posted by temporance
It can be done in VfW because DivX5 does it.Are you sure? Since divx5 produces delay 0x7f frame (it was their idea in the first place) and this frame gets ignored by vdub, you have exactly one frame ignored and missing. I see no way how the avi is supposed to have the same number of frames.

Does anyone know the trick used?

Radek