PDA

View Full Version : RMVB Encoding in Linux


Razorblade2000
2nd April 2005, 16:44
Any news on Linux-RMVB encoding?
Last time I checked (>a year ago) there wasn't any possibilty of getting the file-input to the encoder (piping or the abilty to read compressed input)

any news or is the situation still that sad :( ?

Sirber
2nd April 2005, 17:33
Don't think it changed. Not even a new build for windows... :(

Razorblade2000
3rd April 2005, 23:07
well, it's a pity

karl_lillevold
4th April 2005, 23:04
yes, it's a pity there are no Linux programmers that have volunteered to add this. Anyone?

Sirber
4th April 2005, 23:49
Is producer cmdline opensource?

karl_lillevold
4th April 2005, 23:51
yes, most of it. only the RM codecs and the file format are not open source. If anyone wanted to improve input format support on Linux, they have all they need in open source.

Sirber
5th April 2005, 00:09
Is the libavcodec project still alive?

http://forum.doom9.org/showthread.php?s=&threadid=77018&highlight=producer

xxthink
17th June 2009, 02:43
yes, most of it. only the RM codecs and the file format are not open source. If anyone wanted to improve input format support on Linux, they have all they need in open source.

Is there some opensource rmvb encoder on linux now?

benwaggoner
18th June 2009, 16:44
What are folks using RealVideo for these days?

The main places I hear about are .rmvb libraries in Chinese internet cafes, and a few enterprises still using it internally.

roozhou
19th June 2009, 13:01
The main places I hear about are .rmvb libraries in Chinese internet cafes, and a few enterprises still using it internally.

That's right. RV40 still dominates online videos in China. Many portable players and SAPs also have hardware decoding support for RV40 while only a few of them support WMV3/VC1.

P.S. I highly doubt VC1 gives better quality than RV40 at the same bitrate.

tetsuo55
19th June 2009, 13:46
not really an answer to your question, but i strongly recommend not encoding RMVB, no point in keeping this format alive any longer than needed.

That said libavcodec can now decode rv10,20,30 and 40

benwaggoner
19th June 2009, 17:54
That's right. RV40 still dominates online videos in China. Many portable players and SAPs also have hardware decoding support for RV40 while only a few of them support WMV3/VC1.
Online video? It seems to be more P2P or sneakernet distribution from what I saw.

Web video proper (played in a browser) that I saw was mainly WMP/Silverlight/Flash.

P.S. I highly doubt VC1 gives better quality than RV40 at the same bitrate.
Emperical question :). Do you have a scenario in mind?

xxthink
19th June 2009, 22:50
What are folks using RealVideo for these days?

The main places I hear about are .rmvb libraries in Chinese internet cafes, and a few enterprises still using it internally.

Then what another format should be prefered?

benwaggoner
19th June 2009, 23:15
Then what another format should be prefered?
I don't really understand why they're doing it that way now, so I can't really say would would do better. RV9/10 is one of the better codecs for quality@perf when DXVA isn't available. So maybe it's about low-end machine decode perf?

tetsuo55
20th June 2009, 09:22
That can't be it, RV30/40 are almost equally as difficult as h264.

At the same resolution the the rmvb will stutter more often on my xbox with xbmc when compared to the h264 in mkv.

I read once that realproducer was very easy to use for newcomers and that no other application has come along that matches its noob-user friendlyness.
Also all those chinese users have installed realplayer and there are even standalone devices (extremely high priced) to play RMVB.

Now with HD i have been trying to convince some of my chinese friends to use x264, i got some home videos in 720p that where encoded with rmvb, and my system that can happily play h264 was stuttering like crazy with the rmvb.

Dark Shikari
20th June 2009, 11:03
I don't really understand why they're doing it that way now, so I can't really say would would do better. RV9/10 is one of the better codecs for quality@perf when DXVA isn't available.Is this a joke?

No proprietary video format is ever decent in terms of performance because there's only a single decoder implementation which is generally horrifically slow. Even if you're lucky enough to get it reverse-engineered into libavcodec, they rarely care enough about your pet format to bend their brain trying to optimize it.

This is the primary reason why WMV9 was so godawful performance-wise: it isn't merely an issue of Microsoft being unable to write a fast decoder if their life depended on it, it's that without competition, you just won't get good decoders.

Nevermind the fact that RV's subpel is just patently insane and the inverse transform much more complicated than H.264's (it's almost as bad as VC-1's).

xxthink
20th June 2009, 14:20
Then x264, perhaps the most suitable choice now for HD video?
Only perhaps, before my word done :).

xxthink
20th June 2009, 14:22
That can't be it, RV30/40 are almost equally as difficult as h264.

At the same resolution the the rmvb will stutter more often on my xbox with xbmc when compared to the h264 in mkv.

I read once that realproducer was very easy to use for newcomers and that no other application has come along that matches its noob-user friendlyness.
Also all those chinese users have installed realplayer and there are even standalone devices (extremely high priced) to play RMVB.

Now with HD i have been trying to convince some of my chinese friends to use x264, i got some home videos in 720p that where encoded with rmvb, and my system that can happily play h264 was stuttering like crazy with the rmvb.

Would you like to post the configration of your machine?
Where you find the 720p video sequences?
Do you watch 1080p video?

tetsuo55
20th June 2009, 16:45
Would you like to post the configration of your machine? Xbox 1 unit (733mhz celeron PIII)
Where you find the 720p video sequences?Filmed with digital camera's, weddings and stuff.
Do you watch 1080p video?Yes but only on my real HTPC (C2D e8400)

benwaggoner
20th June 2009, 18:50
No proprietary video format is ever decent in terms of performance because there's only a single decoder implementation which is generally horrifically slow. Even if you're lucky enough to get it reverse-engineered into libavcodec, they rarely care enough about your pet format to bend their brain trying to optimize it.
Well, RealVideo was the first streaming codec that could do 720p24 on relatively low-end hardware. NAB 2001-2002? Before WM 9 Series launch.

And it had that super-aggressive filtering that kept it from getting too blocky. Sort of a prepreprocessed "in-loop" deblocking that did some kind of QP-feedback driven low-pass filter?

This is the primary reason why WMV9 was so godawful performance-wise: it isn't merely an issue of Microsoft being unable to write a fast decoder if their life depended on it, it's that without competition, you just won't get good decoders.
I'd say it has been more an issue of the slow pass of Windows and thus WMP releases. There are some great multithreaded VC-1 decoders created since WMP 11 shipped, but they're in things like the Xbox The Silverlight 2+ VC-1 software decoder is also quite a bit faster than the WMP11 one.

Plus stability/security gets a lot of attention; there may be more bounds checking etcetera in our decoders due to all the fuzz testing they go through.

Nevermind the fact that RV's subpel is just patently insane and the inverse transform much more complicated than H.264's (it's almost as bad as VC-1's).
I've never looked at the guts of that codec, just my recollections as a compressionist back in the era when RealVdieo and Windows Media were the main competing codecs. RV offered better decode complexity, and WIndows Media (probably V8 at that point) retained more detail.

roozhou
22nd June 2009, 13:30
Emperical question :). Do you have a scenario in mind?

I can do a RV40 vs VC1 test. Samples will be SD anime.
Encoders are Easy RealMedia Producer for RV40 and MediaCoder for VC1.

One problem is that with default settings RV40 encoder is much faster than VC1 encoder so I cannot consider it a fair contest. Any suggestion on how to tweak WMV PowerToy to speed up VC1 encoding?

benwaggoner
22nd June 2009, 17:40
I can do a RV40 vs VC1 test. Samples will be SD anime.
Encoders are Easy RealMedia Producer for RV40 and MediaCoder for VC1.
What VC-1 implementation does MediaCoder use?

For a meaningful test, you'd want to use a VC-1 Encoder SDK based product like Expression Encoder. If you can provide source, I can do that encode.

But shouldn't we pick something that's film/video? Anime's not very representative test content for anything but anime.

Dark Shikari
22nd June 2009, 17:45
What VC-1 implementation does MediaCoder use?

For a meaningful test, you'd want to use a VC-1 Encoder SDK based product like Expression Encoder. If you can provide source, I can do that encode.

But shouldn't we pick something that's film/video? Anime's not very representative test content for anything but anime.Roozhou is choosing anime because he knows RV will win, since RV uses a 4x4 transform and VC-1 uses an 8x8 transform (for intra at least) ;)

Tricksy hobbitses.

benwaggoner
22nd June 2009, 17:53
Roozhou is choosing anime because he knows RV will win, since RV uses a 4x4 transform and VC-1 uses an 8x8 transform (for intra at least) ;)

Tricksy hobbitses.
Spatial transform? VC-1 can switch between 8x8,4x4, 4x8, and 8x4 intra per 8x8 block. That shouldn't make a difference.

RealVideo 10's QP adaptive smoothing (at least that what it looked to me like what it was doing a few years ago) probably does a great job of sucking detail out of flat regions, which in Anime is mainly going to be noise.

Hmmm. I think I've got the uncompressed (albeit SD) source for the "Sinbad" trailer around here somewhere. Want to offer up some specs?

Dark Shikari
22nd June 2009, 17:55
Spatial transform? VC-1 can switch between 8x8,4x4, 4x8, and 8x4 intra per 8x8 block. That shouldn't make a difference.No, that's for inter coding. Intra is 8x8 only, from what I know.

From libavcodec (I-frame decoding, relevant stuff highlighted):

for(s->mb_y = 0; s->mb_y < s->mb_height; s->mb_y++) {
s->mb_x = 0;
ff_init_block_index(s);
for(; s->mb_x < s->mb_width; s->mb_x++) {
ff_update_block_index(s);
s->dsp.clear_blocks(s->block[0]);
mb_pos = s->mb_x + s->mb_y * s->mb_width;
s->current_picture.mb_type[mb_pos] = MB_TYPE_INTRA;
s->current_picture.qscale_table[mb_pos] = v->pq;
s->current_picture.motion_val[1][s->block_index[0]][0] = 0;
s->current_picture.motion_val[1][s->block_index[0]][1] = 0;

// do actual MB decoding and displaying
cbp = get_vlc2(&v->s.gb, ff_msmp4_mb_i_vlc.table, MB_INTRA_VLC_BITS, 2);
v->s.ac_pred = get_bits1(&v->s.gb);

for(k = 0; k < 6; k++) {
val = ((cbp >> (5 - k)) & 1);

if (k < 4) {
int pred = vc1_coded_block_pred(&v->s, k, &coded_val);
val = val ^ pred;
*coded_val = val;
}
cbp |= val << (5 - k);

vc1_decode_i_block(v, s->block[k], k, val, (k<4)? v->codingset : v->codingset2);

s->dsp.vc1_inv_trans_8x8(s->block[k]);
if(v->pq >= 9 && v->overlap) {
for(j = 0; j < 64; j++) s->block[k][j] += 128;
}
}

vc1_put_block(v, s->block);
if(v->pq >= 9 && v->overlap) {
if(s->mb_x) {
s->dsp.vc1_h_overlap(s->dest[0], s->linesize);
s->dsp.vc1_h_overlap(s->dest[0] + 8 * s->linesize, s->linesize);
if(!(s->flags & CODEC_FLAG_GRAY)) {
s->dsp.vc1_h_overlap(s->dest[1], s->uvlinesize);
s->dsp.vc1_h_overlap(s->dest[2], s->uvlinesize);
}
}
s->dsp.vc1_h_overlap(s->dest[0] + 8, s->linesize);
s->dsp.vc1_h_overlap(s->dest[0] + 8 * s->linesize + 8, s->linesize);
if(!s->first_slice_line) {
s->dsp.vc1_v_overlap(s->dest[0], s->linesize);
s->dsp.vc1_v_overlap(s->dest[0] + 8, s->linesize);
if(!(s->flags & CODEC_FLAG_GRAY)) {
s->dsp.vc1_v_overlap(s->dest[1], s->uvlinesize);
s->dsp.vc1_v_overlap(s->dest[2], s->uvlinesize);
}
}
s->dsp.vc1_v_overlap(s->dest[0] + 8 * s->linesize, s->linesize);
s->dsp.vc1_v_overlap(s->dest[0] + 8 * s->linesize + 8, s->linesize);
}
if(v->s.loop_filter) vc1_loop_filter_iblk(s, v->pq);

if(get_bits_count(&s->gb) > v->bits) {
ff_er_add_slice(s, 0, 0, s->mb_x, s->mb_y, (AC_END|DC_END|MV_END));
av_log(s->avctx, AV_LOG_ERROR, "Bits overconsumption: %i > %i\n", get_bits_count(&s->gb), v->bits);
return;
}
}
ff_draw_horiz_band(s, s->mb_y * 16, 16);
s->first_slice_line = 0;
}

roozhou
22nd June 2009, 19:03
What VC-1 implementation does MediaCoder use?

For a meaningful test, you'd want to use a VC-1 Encoder SDK based product like Expression Encoder. If you can provide source, I can do that encode.
MediaCoder uses WMF SDK? Can you provide a free VC-1 Encoder SDK based encoder that accepts raw YUV from stdin?

Tests must be taken on the same machine. I need same bitrates and same encoding speed.

But shouldn't we pick something that's film/video? Anime's not very representative test content for anything but anime.
Sure.

benwaggoner
22nd June 2009, 19:16
No, that's for inter coding. Intra is 8x8 only, from what I know.
Yes, of course. I was saying intra to discrimate from partition, when I meant to say predicted :).

Anyway, I found that Sinbad source, and it looks like Dreamworks actually gave me permission to distrubute the source for compression testing back in 2004, so I'm going to post it up to a share.

It's an interesting clip, with a mixture of cel and CGI animation, plus a few shots of actual video source. And there's some interesting/annoying banding in some frames that could produce some interesting.

I'm uploading it to SkyDrive right now (in .zip segments due to the darn 50 MB limitation).

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/Sinbad%7C_Source

It has six segments; maybe another hour or so in the upload. My ADSL is very "A"...

benwaggoner
22nd June 2009, 19:41
MediaCoder uses WMF SDK?
I don't have any idea what it uses, honestly.

Can you provide a free VC-1 Encoder SDK based encoder that accepts raw YUV from stdin?
Expression Encoder has a full-featured 30-day trial.

There's also this:
http://forum.doom9.org/showpost.php?p=1284628&postcount=173

Tests must be taken on the same machine. I need same bitrates and same encoding speed.
Are we doing a quality test or a quality@perf test? Two different things.

And what's the scenario for bitrates and rate control model?

And comparing with postprocessing on or off? On would be more realistic as to real-world experiences, while Off would be a better test of the codecs themselves.

Dark Shikari
22nd June 2009, 19:43
Use my blackpearl sample, it's standard enough ;)

benwaggoner
22nd June 2009, 19:50
Use my blackpearl sample, it's standard enough ;)
Link?

Dark Shikari
22nd June 2009, 19:59
Link?linkage (http://www.mediafire.com/?mymhmje0iki)

benwaggoner
22nd June 2009, 21:13
Got it. Preprocess to 704x288? What's the bitrate used for those internet cafes? 1 Mbps?

Dark Shikari
22nd June 2009, 21:17
Got it. Preprocess to 704x288?I just encode it at 720x352 (black cropped off) with no resizing.What's the bitrate used for those internet cafes? 1 Mbps?I usually use 500kbps for low bitrate and 1000kbps for medium-ish bitrate on that sample.

benwaggoner
22nd June 2009, 22:52
I just encode it at 720x352 (black cropped off) with no resizing.
Well, there's a little horizontal blanking left and right. I think you'd want to crop to at least 712.

If you don't want to do any scaling and want to be Mod16, I'd probably do:

MPEG2Source("D:\Dark Shikari\Black.Pearl.Sample.d2v",cpu=6)
tdecimate(mode=2)
CROP(8,64,704,352)

I usually use 500kbps for low bitrate and 1000kbps for medium-ish bitrate on that sample.
Sounds reasonable. Your usual unconstrained 2-pass VBR with 10 sec GOP?

Dark Shikari
22nd June 2009, 22:59
Well, there's a little horizontal blanking left and right. I think you'd want to crop to at least 712.I guess, I never bothered with that.Sounds reasonable. Your usual unconstrained 2-pass VBR with 10 sec GOP?Yup.

roozhou
23rd June 2009, 04:54
@ben
I am tesing DS's Black Pearl sample and still downloading your clip. Here are two questions concerning Expression Encoder 2

1) How can I set SAR? I cropped the video to 720x360 and set Video Aspect to 64:27, but it had no effect.
2) Why did the encoder complain about invalid video profile when I enabled dquant?



tdecimate(mode=2)

Why are you using tdecimate? The source is progressive.

benwaggoner
23rd June 2009, 06:28
@ben
I am tesing DS's Black Pearl sample and still downloading your clip. Here are two questions concerning Expression Encoder 2

1) How can I set SAR? I cropped the video to 720x360 and set Video Aspect to 64:27, but it had no effect.
First, make sure you have SP1 installed.

Set Resize Mode > Stretch
Video Aspect > Custom > 47:20 or whatever you like


2) Why did the encoder complain about invalid video profile when I enabled dquant?
Were you doing Advenced Profile?

Also, for that clip you only want to use I-Frame DQuant, and probably not even that for the 500 Kbps.

A good starting point would be:

Adaptive Dead Zone: Conservative
DQuant: I-Frame Only
Filters: In-Loop and Overlap On
Closed GOP: Off

For typical settings, set Search Range to Adaptive

For a HQ encode, also set:
Complexity:4
Chroma Search: Adaptive True Chroma
Match Method: Adaptive

And for an Insane encode:
Complexity: 5
Chroma Search: Full True Chroma
Threads Used: 1
Match

Why are you using tdecimate? The source is progressive.
I was getting some interlaced frames coming through for some reason, and it was left over from troubleshooting. Go ahead and delete.

Anyway, I threw up a couple of my own tests here:

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/DS%20Tests

roozhou
23rd June 2009, 09:58
Were you doing Advenced Profile?


I have SP1 installed. Tested with more clips, but it always gave me "Error: Bad video profile: Invalid video profile for VC-1 Advanced profile"

All my settings are as follows:

Framerate: Source
Key frame interval: 8
Profile: VC-1 Advanced Profile
Mode: Quality VBR

Width: 640
Height: 480
Quality: 65

Video Compexity: Good (2)
Adaptive Dead Zone: Off
DQuant: I and P frames

Filters: Only In-loop enabled

B-Frame Number: 2
Scene Change Detection: On
Adaptive GOP: On
Closed GOP: Off

Chroma Search: Luma Only
Match Method: SAD
Search Range: Adaptive

benwaggoner
23rd June 2009, 17:51
I have SP1 installed. Tested with more clips, but it always gave me "Error: Bad video profile: Invalid video profile for VC-1 Advanced profile"
Why are you using those settings? They're completely off scenario.

I don't see why you'd get an "invalid profile" error, though. Admittedly I've never seen that in SP1 before.

Can you PM me your settings file?

Have you compared my enodes to RMVB yet?


All my settings are as follows:

Framerate: Source
Key frame interval: 8
Profile: VC-1 Advanced Profile
Mode: Quality VBR

Width: 640
Height: 480
Quality: 65

Video Compexity: Good (2)
Adaptive Dead Zone: Off
DQuant: I and P frames

Filters: Only In-loop enabled

B-Frame Number: 2
Scene Change Detection: On
Adaptive GOP: On
Closed GOP: Off

Chroma Search: Luma Only
Match Method: SAD
Search Range: Adaptive

roozhou
24th June 2009, 11:05
@Ben
What is the length of the Black Pearl sample? Should it be 2 min 14 s?
Why does your encoded video has only 2 min?

benwaggoner
24th June 2009, 15:56
@Ben
What is the length of the Black Pearl sample? Should it be 2 min 14 s?
Why does your encoded video has only 2 min?
I have no idea. Is it missing frames, or is it running too fast?

I'm not at that machine right now to double check. I was having some weirdness in decoding the .m2v as I recall, so I might have messed something up there.

roozhou
25th June 2009, 08:50
OK, here is my encodes of DS's Black Pearl sample @ 500kbps
http://files.filefront.com/13928181

I don't have patient to tweak both of them to be perfect 500kbps. ~1% difference in size is tolerable.

RV40 settings:
2pass VBR
Video Mode: Sharpest image
Max Bitrate: 4000 kbps
Key Frame Interval: 10
Dropdup: Off
1st Pass Complexity: 50
2nd Pass Compexity: 85
B Frames: 1
CutOffQuant: 9
All other settings use default

VC1 settings:
Video Complexity: Good (2)
Key frame interval: 10
Profile: VC-1 Advanced Profile
Mode: VBR peak unconstrained
Adaptive Dead Zone: Conservative
DQuant: I and P Frames
In-Loop: On
Overlap, Denoise, Noise Edge Removal: Off
B-Frame Number: 1
Scene Change Detection: On
Adaptive GOP: On
Closed GOP: Off
Chroma Search: Luma Only
Match Methed: SAD
Search Range: Adaptive

Encoded size is 7.9M for wmv and 8.0M for rmvb.
Encoding time is ~140s for EE2 and ~130s for ERMP1.94, running on a E8400 @ 3.0GHz.
I think it is fair enough.

benwaggoner
25th June 2009, 09:18
Are you comparing fixed quality encodes? Shouldn't we be testing with rate control?

Also, your VC-1 encodes shouldn't be using P-frame DQuant; just I-frame at 1000 Kbps, and probably off entirely at 500 Kbps. The signalling overhead of DQuant is going to be a net loss at 500 Kbps.

Closed GOP should be OFF as well.


OK, here is my encodes of DS's VC1 settings:
Video Complexity: Good (2)
Key frame interval: 10
Profile: VC-1 Advanced Profile
Mode: VBR peak unconstrained
Adaptive Dead Zone: Conservative
DQuant: I and P Frames
In-Loop: On
Overlap, Denoise, Noise Edge Removal: Off
B-Frame Number: 1
Scene Change Detection: On
Adaptive GOP: On
Closed GOP: On
Chroma Search: Luma Only
Match Methed: SAD
Search Range: Adaptive

roozhou
25th June 2009, 09:53
Are you comparing fixed quality encodes? Shouldn't we be testing with rate control?

I am comparing quality under fixed bitrate. Both were encoded with 2-pass VBR rate control.

Also, your VC-1 encodes shouldn't be using P-frame DQuant; just I-frame at 1000 Kbps, and probably off entirely at 500 Kbps. The signalling overhead of DQuant is going to be a net loss at 500 Kbps.
WMV Powertoy says "Dquant applied to I and P frames only usually produces the best results" and I just followed that. I thought it was something similar to AQ in x264 so I turned it on.
Closed GOP should be OFF as well.
My bad. I did turn that off. Fixed.

BTW. I cannot find an expert on Real Producer. There should be some quality improvement in RV40 clip if I get good advice on RV40 settings.

benwaggoner
25th June 2009, 19:12
I am comparing quality under fixed bitrate. Both were encoded with 2-pass VBR rate control.
Cool.

WMV Powertoy says "Dquant applied to I and P frames only usually produces the best results" and I just followed that. I thought it was something similar to AQ in x264 so I turned it on.
It's akin, but the implementation in EEv3 is really tuned for HD bitrates. PowerToy refers to the Windows Format SDK .dll implementation, which has a better low bitate DQuant, but the other advantages of the new SDK leave it better on net.

Recent quality tuning has more focused on dynamic motion vector cost to address similar issues, which has the advantage of being compatible with all VC-1 profiles. Most of the time I don't use DQuant at all, but it's helpful with this particular clip on I-frames, as it reduces QP in the smoother areas of the backbround on the reference frame.

BTW. I cannot find an expert on Real Producer. There should be some quality improvement in RV40 clip if I get good advice on RV40 settings.
Indeed. I don't know if there are any expert compressionists focused on RealVideo anymore.

I had to do quite a bit of my own testing for the RealVideo chapter in my book.

roozhou
26th June 2009, 04:23
So what do you think about the two encoded clips?

IMHO The wmv seems to have more details and more grains but look more blocky as well.
Generally it doesn't look any better than rmvb.

benwaggoner
26th June 2009, 05:56
So what do you think about the two encoded clips?

IMHO The wmv seems to have more details and more grains but look more blocky as well.
Generally it doesn't look any better than rmvb.
Your encode or my encode?

Anyway, I an believe that would be the general distinction. Low bitrate VC-1 is tuned more for detail preservation assuming that deblocking and deringing postprocessing would be available to reduce blockiness (post-processing, although optional, is defined in the SMPTE VC-1 spec).

roozhou
26th June 2009, 07:07
Your encode or my encode?

Anyway, I an believe that would be the general distinction. Low bitrate VC-1 is tuned more for detail preservation assuming that deblocking and deringing postprocessing would be available to reduce blockiness (post-processing, although optional, is defined in the SMPTE VC-1 spec).

Deblocking is on, but still bad. The edges looks bad.

benwaggoner
26th June 2009, 07:11
Deblocking is on, but still bad. The edges looks bad.
Is that comparison with your encodes or my encodes?

Dark Shikari
26th June 2009, 07:45
the encode has more detail, but the edges are worse!You're comparing a 4x4-transform-based encoder to an 8x8-transform-based encoder and this is supposed to be new information? ;)

roozhou
26th June 2009, 08:25
Is that comparison with your encodes or my encodes?

Of course my encodes. Yours is of different resolution and wrong length.

roozhou
26th June 2009, 09:24
Roozhou is choosing anime because he knows RV will win, since RV uses a 4x4 transform and VC-1 uses an 8x8 transform (for intra at least) ;)

Tricksy hobbitses.
OK, I chose film.
I usually use 500kbps for low bitrate
OK, I used 500kbps.
You're comparing a 4x4-transform-based encoder to an 8x8-transform-based encoder and this is supposed to be new information?
So what do you want this time? I draw a conclusion "RV40 is bad, don't use it any more" to make you happy? You are trying to force others to accept what you think is right.

benwaggoner
26th June 2009, 16:35
Of course my encodes. Yours is of different resolution and wrong length.
Can you share your preprocessing settings then? I'd like to try with a *ahem* more recent implementation to see what kind of difference that would make.

Since you're doing a fixed-time encode, I'd still very much like to get preprocessing out of the equation. It could matter for quality as well.

roozhou
28th June 2009, 17:18
Can you share your preprocessing settings then? I'd like to try with a *ahem* more recent implementation to see what kind of difference that would make.

Since you're doing a fixed-time encode, I'd still very much like to get preprocessing out of the equation. It could matter for quality as well.

I didn't use any preprocessing. I only turned on in-loop, which is inside the VC1 spec.

benwaggoner
28th June 2009, 18:43
I didn't use any preprocessing. I only turned on in-loop, which is inside the VC1 spec.
I mean cropping, scaling, and deinterlacing.

Unless we're using the exact same YV12 bitmaps into the codecs. we're comparing apples and kumquats.

Hence using AVISynth to dump to a YV12 AVI, that both encoders can take natively, and so we're handing the same 4:2:0 samples off to the respective encoders.

How did you preprocess the RMVB?

roozhou
29th June 2009, 04:45
Only cropping was applied. I cropped 58 lines from top and 62 lines from bottom. Both EEv2 and ERMP have built-in croppers.

No scaling, no deinterlacing, no drops, no duplications and of course no other preprocessing for RMVB.

YV12 avi is too large (~1G).

benwaggoner
29th June 2009, 05:36
Only cropping was applied. I cropped 58 lines from top and 62 lines from bottom. Both EEv2 and ERMP have built-in croppers.

No scaling, no deinterlacing, no drops, no duplications and of course no other preprocessing for RMVB.
But they'll have different decoders, which matters if we're trying to time codecs. What MPEG-2 decoder does ERMP use? Or did you use AVISynth? If so, just post your AVISynth script and we'll be good.

YV12 avi is too large (~1G).
Lagarith YV12 and 7z :)?

roozhou
29th June 2009, 07:11
But they'll have different decoders, which matters if we're trying to time codecs. What MPEG-2 decoder does ERMP use? Or did you use AVISynth? If so, just post your AVISynth script and we'll be good.

I said I don't use avisynth at all. I muxed m2v into a mkv and fed it to ERMP and EEv2.

Both ERMP and EEv2 uses directshow. I saw ffdshow's trayicon during encoding so obviously they were using the same decoder.

BTW, if there is any software MPEG2 decoder faster than the latest ffdshow, let me know.

benwaggoner
29th June 2009, 22:01
I said I don't use avisynth at all. I muxed m2v into a mkv and fed it to ERMP and EEv2.

Both ERMP and EEv2 uses directshow. I saw ffdshow's trayicon during encoding so obviously they were using the same decoder.
Ah, I get it. Normaly EEv2 uses its own MPEG-2 decoder, but that's not included in the trial version. I'm sure it hasn't been tested wth EE.

I get that you don't want to upload a huge file, but I really think we're better off not conflating decode and preprocesing with the codec test. If we can agree on a .avs file.

The source is also pretty compressed, so cpu=6 in DGDecode would be appropriate to reduce the degree we're looking at how well the codecs deal with reencoding DCT artifacts versus encoding real image.

BTW, if there is any software MPEG2 decoder faster than the latest ffdshow, let me know.
Do you mean software only? EE can use hardware accelerated source decode via DXVA when appropriate hardware is available.

roozhou
30th June 2009, 03:11
Do you mean software only? EE can use hardware accelerated source decode via DXVA when appropriate hardware is available.

Are you joking? Decode to video memory and dump to main memory? I have never seen such implementation so far, though clsid said it's possible. Do you really believe this would be faster than software decoding?

The source is also pretty compressed, so cpu=6 in DGDecode would be appropriate to reduce the degree we're looking at how well the codecs deal with reencoding DCT artifacts versus encoding real image.
Will DGDecode decodes anything different from ffdshow? There must be only one CORRECT output.

benwaggoner
30th June 2009, 03:59
Are you joking? Decode to video memory and dump to main memory? I have never seen such implementation so far, though clsid said it's possible. Do you really believe this would be faster than software decoding?
Sure, for 1080i Long GOP MPEG-2 and AVCHD. It's not that much worse to take it back into memory than to display it. Plus that's memory bandwidth, not CPU power, so what you save on decode you get back.

Better yet, it makes scrubbing a lot faster. Doing frame-by-frame chapter markers and captioning with 1080i AVCHD software decode can be painful. Go back one frame to the last GOP, and have to decode all the reference frames in the previous GOP.

I agree it's not going to be that important a speedup for DVDs.

Will DGDecode decodes anything different from ffdshow? There must be only one CORRECT output.
DGDecode has deblocking and deringing in the decode, resulting in a more accurate file. There's enough artifacting in that source for it to help some.

Exactly. That's why I'm suggesting we define an AVS or a preprocessed AVI or whatever to keep any decoder implementations, color space conversions, etcetera out of the mix.

roozhou
30th June 2009, 08:39
DGDecode has deblocking and deringing in the decode, resulting in a more accurate file. There's enough artifacting in that source for it to help some.

Exactly. That's why I'm suggesting we define an AVS or a preprocessed AVI or whatever to keep any decoder implementations, color space conversions, etcetera out of the mix.

There is no deblocking and deringing in MPEG2 spec. Additional post-processing can be considered as cheating. And no color space conversion is required for mpeg2 -> YV12.

So I suggest you download ERMP and do both encodes yourself. You can use avs script, preprocessed lagarith or whatever you like.

IgorC
30th June 2009, 10:24
It's not avisynth script. It's just RV10 does better jobs for DVD source for 500-1000 kbps. It's easy to see.

benwagonner, stop avoiding to admit that. VC-1 never got out of ASPy look.

Dark Shikari
30th June 2009, 10:30
It's not avisynth script. It's just RV10 does better jobs for DVD source for 500-1000 kbps. It's easy to see.

benwagonner, stop avoiding to admit that. VC-1 never got out of ASPy look.The real question is whether his justification will be claiming that RV40 is good... or admitting that VC-1 is bad ;)

benwaggoner
30th June 2009, 18:41
There is no deblocking and deringing in MPEG2 spec. Additional post-processing can be considered as cheating. And no color space conversion is required for mpeg2 -> YV12.
Cheating? Using a deblocking decoder for distribution-rate MPEG-2 is industry best practice, no? Doesn't everyone use it around here coming from DVD/ATSC/DVB sources?

I doubt it'd could obsucre or reverse the value of the codecs. If anything by improving the SNR of the source it'd make codec difference more apparent.

So I suggest you download ERMP and do both encodes yourself. You can use avs script, preprocessed lagarith or whatever you like.
Fair enough. I just want to make sure we're using the exact same YV12 pixels, and that they're appropriately optimized for the test at hand.

It would be cheating if I used postprocessing on the source and you didn't.

roozhou
30th June 2009, 19:07
Fair enough. I just want to make sure we're using the exact same YV12 pixels, and that they're appropriately optimized for the test at hand.

It would be cheating if I used postprocessing on the source and you didn't.
Then you make your tests with postprocessing. You don't have to compare your encodes with my encodes.

And what about your Simbad sample?

benwaggoner
30th June 2009, 19:08
It's not avisynth script. It's just RV10 does better jobs for DVD source for 500-1000 kbps. It's easy to see.
Are we comparing with player postprocessing on or off? That has a huge impact at these bitrates.

Current implementations of VC-1's low bitrate behavior is tuned assuming post processing is available (one could argue that's an assumption of the bitstream design itself; postprocessing is included in the SMPTE VC-1 spec).

The EEv2 implementation doesn't have a differental quantization tuned well for those rates, which would be a big help in reducing low bitrate blocking. There's a better adaptive DQuant mode in PEP 2.1, but that hasn't migrated into the other shipping implementations at this point.

benwagonner, stop avoiding to admit that. VC-1 never got out of ASPy look.
Again, depends a lot on if postprocessing is being used. VC-1 can do 1000 Kbps okay, but no argument it can get pretty blocky without postprocessing at 500 Kbps SD. Our quality tuning focus hasn't been at those low bits/pixel ratios for a while now.

In-loop deblocking and overlap transform help quite a bit compared to ASP up maybe QP 16 or so, but much past that and the blocks will still pop through. The in-loop deblocking filter of H.264 is very nicely designed to handle the entire QP range, probably the top reason why H.264 High Profile is by far the best codec at these bits/pixel. And x264 has had a ton of tuning for these rates as well.

The key to making VC-1 work well at low rates is to manage bits/pixel so that QP stays in a reasonable range.

And that is a hint about something I'll share more about next week...

I'll take a whack at fixing my AVS once my current render job is done. Or to make things clearly neutral, I'd be happy to use one that DS provides us both.

roozhou
30th June 2009, 19:51
Are we comparing with player postprocessing on or off? That has a huge impact at these bitrates.


I am confused. Do you mean VC-1 needs additional postprocessing after decoding or EEv2 needs additional postprocessing before encoding?

The encoded VC-1 should look exactly the same with all decoders. So postprocessing outside the spec. should never be applied during our test.

benwaggoner
30th June 2009, 21:56
Then you make your tests with postprocessing. You don't have to compare your encodes with my encodes.
Well, that won't be really comparing the codecs at that point, but the end-to-end toolchain.

And what about your Simbad sample?
Right. We can use that. I actually did some encodes for that at 500 and 1000 Kbps I'll get uploaded later today.

roozhou
1st July 2009, 03:21
Well, that won't be really comparing the codecs at that point, but the end-to-end toolchain.
So we cannot use any GUI during test? I mentioned in previous posts that the only preprocessing I used is cropping.


Right. We can use that. I actually did some encodes for that at 500 and 1000 Kbps I'll get uploaded later today.
I did encodes at 500kbps too.

http://www.filefront.com/13949269/Simbad.zip

Settings different from Black Pearl sample:
[ERMP]
B-Frames: 1 -> 3
Dropdup: Off -> 2

[VC1]
B-Frames: 1 -> 2
DQuant: I + P -> I only

benwaggoner
1st July 2009, 03:44
So we cannot use any GUI during test? I mentioned in previous posts that the only preprocessing I used is cropping.
I understand. These should match in theory, but I have more confidence in practice; "Same as Source" is a safer choice :).


I did encodes at 500kbps too.

http://www.filefront.com/13949269/Simbad.zip

[VC1]
B-Frames: 1 -> 2
DQuant: I + P -> I only
I don't think either of those would help. You're probably better off with 1 B and no DQuant, particuarly at 500 Kbps.

roozhou
1st July 2009, 05:31
You're probably better off with 1 B and no DQuant, particuarly at 500 Kbps.

There is huge difference between VC1 and RV40. These settings won't help much either. The quality improvement will be trivial.

benwaggoner
1st July 2009, 19:05
There is huge difference between VC1 and RV40. These settings won't help much either. The quality improvement will be trivial.
I don't know if I'd say trivial :).

But certainly, at 500 Kbps with the Sinbad content, the VC-1 artifacts are a lot more annoying than the RV10 ones, as DS predicted.

I was able to get somewhat higher quality than yours, though, which I've uploaded to SkyDrive, along with a fixed LAGS version of DS's source (cpu=6 decode, cropped to active image area, and scaled to 640x272).

I'm just now uploading my version of those as well.

roozhou
2nd July 2009, 02:53
I don't know if I'd say trivial :).

But certainly, at 500 Kbps with the Sinbad content, the VC-1 artifacts are a lot more annoying than the RV10 ones, as DS predicted.

I was able to get somewhat higher quality than yours, though, which I've uploaded to SkyDrive, along with a fixed LAGS version of DS's source (cpu=6 decode, cropped to active image area, and scaled to 640x272).

I'm just now uploading my version of those as well.

Why do you resize it to 640x272? Scaling always brings quality loss.

benwaggoner
2nd July 2009, 05:17
Why do you resize it to 640x272? Scaling always brings quality loss.
640x squre pixel? Habit I guess :). I thought yours was 640x352.

Given the edge blanking, one could do 704x288 Mod16 square pixel as easily. IIRC RealVideo as a format doesn't support anamorphic pixels. But if I have that wrong, we could do 704x352 as the biggest Mod16 size that woudln't be scaling up on either axis.

Anyway, whether scaling hurts quality depends what you mean by quality loss. It needn't reduce the per pixel quality (and can improve SNR per pixel if the fine detail in the source is mainly noise). It will reduce total signal as well, but not linearly to the reducion in pixels. Heck, DVD normally are low-pass filtered on the vertical anyway to reduce flicker on interlaced displays. So up to a 30% vertical shrink probably won't eliminate any real visual information for DVD sourced content.

roozhou
2nd July 2009, 05:48
Given the edge blanking, one could do 704x288 Mod16 square pixel as easily. IIRC RealVideo as a format doesn't support anamorphic pixels. But if I have that wrong, we could do 704x352 as the biggest Mod16 size that woudln't be scaling up on either axis.

Mine was 720x360 without scaling.

RMVB does support anamorphic. Look at my encoded sample. It is encoded at 720x360 but will be displayed at 856x360.

mgh
2nd July 2009, 06:41
rmvb supports different playback resolution from encoded resolution ( recommended is upscaling only), not only what is generally understood as anamorphic i.e. width being changed as per PAR while height is maintained.
e.g 720x360 in above example could also be displayed at 864x432 or a x b where a is greater than or equal to 720 and b is greater than or equal to 360:)

roozhou
2nd July 2009, 07:07
rmvb supports different playback resolution from encoded resolution ( recommended is upscaling only), not only what is generally understood as anamorphic i.e. width being changed as per PAR while height is maintained.
e.g 720x360 in above example could also be displayed at 864x432 or a x b where a is greater than or equal to 720 and b is greater than or equal to 360:)

It seems rmvb requires WxH to be mod 32. Both 856x360 and 854x352 are OK but 854x360 will crash the decoder.

And libavcodec's ffrv40 does not support different PAR. It just pads expanded area with black bars.