Xvid suggestion for anime

[lipton]

I was talking a friend of mine on IRC about the various settings for xvid and I have 1 suggestion and a few questions :)
My Suggestion:
- Allow the user to select a specific quantize level for a specific frame range.

Usually the opening scenes in an episode have high motion. This throws off xvid when you try to encode to small file sizes. Usually: High Motion OP / low - medium motion episode / medium to high motion Ending. For example: Hikaru no Go (HnG). At the end of the OP I have an overflow of about 7 megs (target size 115). Since my max P-quantize level is 7, it takes a while for the over flow to reach zero (I'm using Debug view's terms here). I also noticed that the P-Quantize level remains constant until the overflow reaches zero. I dunno if that is normal. I also experienced problems it the contrast of the episode. The re-encoded file was lighter then the source file (no filters added).

P.S. Sorry for making little to no sense at all. I kinda wrote this up in a hurry.

P.S.S: System Info:
- P4 1.7@1.7 gigs
- Nic XviD binary (22-04-02)
- Vdub 1.4.10
- Nvidia GForce 2 MX

[-h]

If you want to force a specific quantizer for a certain range, I'd just use the credits function (with I-frame/P-frame quant). Sure you're not encoding credits, but it lets you force 2 sections of the encode to a specific quantizer.

If you constrain the codec so much with the bitrate you're aiming for (with max P-frame quant = 7) I think you're always going to see such overflow behaviour.

-h

[gnoshi]

On that thought, I know you hate gui programming etc, but would it be possible to rather than using start and end credits, to write into xvid a 'regional control' kind of setup, where you define a region and then the fixed quant, or qual % etc for that region and then add it to a list?

That way you could have start credits, end credits, and any other bits in between you want. It may seem a bit overkill (and probably is) but it is just a thought.

If I'm not making much sense then just let me know and I'll clarify (and/or i'll shut up =)

gnoshi

edit: fixed stupid grammatical errors.

[-h]

On that thought, I know you hate gui programming etc, but would it be possible to rather than using start and end credits, to write into xvid a 'regional control' kind of setup, where you define a region and then the fixed quant, or qual % etc for that region and then add it to a list?

Yep it's possible indeed, it'd just mean modifying the 2-pass curve generation to loop through credits values instead of fixed start/end calculations, and a slight mod to codec_is_in_credits().

But I might leave it to someone else ;)

-h

[Koepi]

Damn, HOW ugly shall the setup become?

We should make it EASIER, not more complex!

Regional control can be done via start/end credits, so why bother? I really vote against including everything the people ask for if it can be achieved anyways with the current state of development.

So, how's the "easy" GUI coming along?

;P

[kastro68]

hey, are you Lipton_te from the Elite-Fansubs group?

If you are, could you convince your team to use the ogg container instead of avi containers. Also, can you use either Xvid or DivX5 to encode your anime rather than divX3?

[kastro68]

@lipton
I don't really understand what you are going on about...but here is a hint that may solve your problem.

I believe that your oversized file is not because of Xvid's poor size estimation...INSTEAD I believe it is due to the mp3 size being inaccurate.

If you are not using gordian knot as a bitrate calculator i suggest you do so since it also calculates the mp3 overhead for you.

AND, always encode the audio first. Then work out what size the Xvid video should be after you know the size of the audio. If you use vorbis audio and an ogg container, then assuming an overhead of zero gives a pretty accurate target file size.

[OUTPinged_]

-h, Instead of ugly interface range quantizer restriction (which would turn out to be impractical anyway), maybe implement ecf-like control file system(a-la nandub). You have already mentioned it can be moved into a to-do list of yours some time ago :-)

That involves very little GUI coding, too =)

[-h]

-h, Instead of ugly interface range quantizer restriction (which would turn out to be impractical anyway), maybe implement ecf-like control file system(a-la nandub). You have already mentioned it can be moved into a to-do list of yours some time ago :-)

Hm that doesn't sound like something I'd feel like doing ;)

Such interface improvements will probably wait until XviD has its own encoding app (or fork of VDub). Such functions really don't belong in a VfW codec, it's already far more complicated than it should be.

-h

[TripleA]

Well, actually, now that someone mentions it, ECF-like control would be most welcome. Or at least a way to force key-frames (for bookmarks in the middle of a fade-in(out) and the like). Maybe just a text file with a list of frames to be forced to KFs...

I tried using GKnot's force keyframe thingy on the *.stats but it didn't work. Of course, I never used that function of GKnot before (by the time GKnot was that far ahead (or by the time I knew it was that far ahead, anyway), I had already moved on to DivX 4...), so it might be that I'm doing something wrong...

[Edit: Grrr, spelling *really* isn't my forté...]

[Koepi]

It definatly should work to enforce keyframes via gknot.

The second pass uses the keyframes from the first pass, so inserting the keyframe-flag should help it.

[lipton]

@kastro68, Yes I am he and I figured out what the problem was from reading the alt curve thread. As for divx5... we will not be using it since it is to precessor intensive for post-precessing. As soon as Xvid is out of the alpha stages, we will begin doing pratical encodes on the series we do. For OGG - it's possible that we will be using the container for our XviD releases. We will most likely not switch to that format until XviD is in atleast the beta stage.
P.S. I hope the pic shows up this time. It's an example of the contrast error that I got when encoding Hikaru no Go.

[Neo Neko]

@Kastro68
Well whadya know Kastro you didn't have to track em down after all. The just fell in you lap.

@Lipton
Nice to see you here lipton_te. You guys keep up the good work. I just loved what you guys did with vandread second stage. One of your first series released in divx4 was it not?

@Kastro68 again
No worries Kastro. If any ripping group picks up on this stuff EF will. Unlike lots of ripping groups out there they seem to actually take the time to look at new stuff comming down the pipeline. While the rest do it the way they do it because it has been the way they always do it.

@everybody
Sorry for being so OT. Back to the serious discussion............

[Neo Neko]

Originally posted by lipton
For OGG - it's possible that we will be using the container for our XviD releases. We will most likely not switch to that format until XviD is in atleast the beta stage.

OGG and Xvid are not related. They are massively different projects. Just about anything that can go in an AVI will go in OGG and more. Divx3 no problem. Divx4 no problem. Divx5 no problem. Divx5.01 some issues. Xvid no problem. If you wanted to do sometes ting you could simply take one of your old series you have done and move the contents from the AVI into an OGG. No re-encoding nescessary. For releases on IRC the net and general file sharring programs OGG has many advantages over AVI. Not to mention that you could use Vorbis audio so that even more bitrate could be pumped to the video. Anyway if you would like to continue this line of discussion visit the new formats section. We would be more than happy to answer any questions you might have.

[lipton]

@Neo Neko

All of out Vandread releases were Divx 3, we haven't released any divx 4 encodes yet. The only group that I know who do that are Ishin.

Time for me to print out the alt CC thread so I can hammer the details into my brain ^_^

[morello12]

lipton:

If you switch to ogm, you can make your stuff for mode 2, thus meaning an extra 100 megs per disc, thus most likely enabling an extra episode a disc :) That Alt CC setting at the end of the thread works very nicely, I'd suggest starting there.

[kastro68]

Just a suggestion.

Is it possible to make your encodes fit perfectly onto a 700/650 meg cdr.

eg. making each episode 175 megs means we can fit exactly 4 episodes per cd without wasting any space. Xvid is great for achieving accurate file size targets. I can usually get within -+500Kb of desired target using Gknot and encoding teh audio first. If you encode the audio first, then the only uncertainty would be the overhead.

But then again, you can always encode the first 3 episodes first...then on the 4th episode, you work out what size it should be to exactly make up the remaining space of 700/650 megs.


Sorry for being too picky,
but I love your Hikaru no go encodes.

[Neo Neko]

Originally posted by lipton
@Neo Neko

All of out Vandread releases were Divx 3, we haven't released any divx 4 encodes yet. The only group that I know who do that are Ishin.

Time for me to print out the alt CC thread so I can hammer the details into my brain ^_^

Whoops! My bad. I was sure some of em were. But I trust what you say. ;)

[Belgabor]

Gosh, not looking for one day at this forum and most of the things I had wanted to ask about/request for for some time get answerded/asked for ;)

@-h: thx for the notion of using credits to force quants on certain frames (doh!)

@Koepi: thx for the hint about gknot. didnt think of that!

@Lipton: Arigatou & ganbatte kudasai :D
btw, bakamx did some divx4 eps and hnk does (or did?), too.

Regards
Belgabor

P.S.: Kanon was evil!

[lipton]

@kastro68

I'm playing around trying to get low motion anime to look nice at 140 megs. That would mean 5 episodes per CD. Fast motion based anime such as Najica or Vandread would have to be encoded at 4 episodes per CD. I just started encoding with xvid this week, so gimme some time to learn it :)

[TripleA]

Originally posted by Koepi
It definatly should work to enforce keyframes via gknot.

The second pass uses the keyframes from the first pass, so inserting the keyframe-flag should help it.

Weird... It still doesn't work!

I guess that means I'm doing something wrong, then ;-)

Let's check:

Using GKnot v0.23, Nandub Files -> Open the *.stats from the XviD 1st pass -> Stats File Editor -> Add the KFs from the Stats -> Manual Toggle KF for the specific frames I want -> Calculate the curve -> Save new stats.

Then in VDub, set XviD for ext. 2nd Pass using the new stats I just saved.

Anything else/different I should be doing?

P.S. Actually, I now get the impression GKnot just generates the ECF required...
P.P.S. And reloading the stats generated by GKnot in GKnot doesn't show frames I supposedly toggled to KFs as KFs...

[primitive]

Originally posted by lipton
@kastro68

I'm playing around trying to get low motion anime to look nice at 140 megs. That would mean 5 episodes per CD. Fast motion based anime such as Najica or Vandread would have to be encoded at 4 episodes per CD. I just started encoding with xvid this week, so gimme some time to learn it :)

Hi.

This is inquis from NLA speaking.

Abismo and I have been playing around with xvid and ogg for quite some time now, and here's what we've been finding:

1. xvid is much more tolerant of low bpp than div3. With postprocessing disabled the video looks good at .17-.18; with postprocessing enabled, we've found that even with high motion clips that we get acceptible output at .15. I've been taking Mini-Goddess episodes down in the .12-.13 range and not having them look terribly bad, though I think that ~.14 is a reasonable floor w/postprocessing. If I were encoding Mahoromatic with xvid, I would probably want to stay in the .16-.17 bpp range, if that gives you any impression of what the codec is capable of.

2. xvid is more tolerant of noisy, nasty raws than div3. Did a test encode of abeno 640x360 at 85% quality CBR. The output looked exactly like the raw (with the subtitles I added, of course), and came out to be 65MB (!). The abeno raw did NOT look pretty, yet xvid pulled some magic out of somewhere and managed to compress it well.

3. xvid + vorbis + ogg is ABOVEGOD for any longer pieces, especially if you have a r2 disc to rip from. Ogg just added chapter support to the directshow filter, and chapters are easy to extract if you have the DVD raw. As far as vorbis audio goes, q=3 is just about the floor as far as speech goes; lower than that and it gets offensive to my ears. I do go lower than that on occasion (r1 dvd rips where there is more than one audio track), and it doesn't sound too bad. In any case, it sounds as good if not better than mp3 at an equivalent bitrate to my ears.

[OUTPinged_]

Such interface improvements will probably wait until XviD has its own encoding app (or fork of VDub). Such functions really don't belong in a VfW codec, it's already far more complicated than it should be.

I may be wrong, but the only interface change for this would be a path to a ecf file.

As for implementation, it looks pretty simple comparing to MV hintfiles or that ME algo tweaking.

Anyway -h, you are the person who will decide what to put in or not, so we just have to bug you about that one :-)

ECF system isnt needed for typical movie encodes but it is very helpfull at times for animation rppers.

[kastro68]

I must agree with you.

However, you did leave out the fact that ogg contained movies are better for downloading. According to Neo Neko, ogm files can play around crc errors, or corrupt downloads; that is, no repairing is required. You can also play partially downloaded ogm files without having to rederive the keyframes.

And, if you are using the ogg container, you can keep the subtitles separate...so that you can turn it on and off as needed.


I have been going around to Anime Fansubbing forums to try and increase the awareness of the benefits of Xvid+Vorbis+Ogg. I'm sure if they knew what I was going on about in those forums, no one would still be using divX3+mp3.

[primitive]

Originally posted by kastro68
I must agree with you.

However, you did leave out the fact that ogg contained movies are better for downloading. According to Neo Neko, ogm files can play around crc errors, or corrupt downloads; that is, no repairing is required. You can also play partially downloaded ogm files without having to rederive the keyframes.

And, if you are using the ogg container, you can keep the subtitles separate...so that you can turn it on and off as needed.


I have been going around to Anime Fansubbing forums to try and increase the awareness of the benefits of Xvid+Vorbis+Ogg. I'm sure if they knew what I was going on about in those forums, no one would still be using divX3+mp3.

The ogg stream format is nice and solid, which I like. I took an episode of Mini-Goddess and cut it in half nastily using dd on my LUNIX machine; mplayer played it perfectly anyway (up until the cut, of course :) )

The biggest benefit I think will be that we won't have people camped in our channel, spamming "NEED RESUME PLS".

About the subtitle issue, we had a discussion and decided in the end that we don't want to distribute softsubs for three reasons:

1. The subtds filter doesn't support ssa format subtitle files. Srt files are fine if you don't want font colors, font styles, etc; however, if you've downloaded an episode lately you will see that having a karaoke op is the norm, and you absolutely can't do anything much more complex than basic text formatting (bold, italic, underline) with the current filter.

2. The srt files are recoverable from the ogg file. Call us crazy, but we aren't too hot on the idea of distributing the original timed, translated script with every episode.

3. Subtds can do some nasty things if you have dvobsub installed on your machine, and frankly we do anime, not tech support.

On the other hand, I am a big fan of xvid + vorbis + srt + chapters --> ogg for r1 DVD rips, and xvid + vorbis + chapters --> ogg for fansub releases is definently on the horizon. It's just so superior to the old style div3 + mp3 --> avi that I don't know why people haven't dumped avi yet.

[yokem55]

mplayer supports ogm's? The last time I tried mplayer, it simply wouldn't load the file....do you have to build some speciall libogg support into it?

[primitive]

Originally posted by yokem55
mplayer supports ogm's? The last time I tried mplayer, it simply wouldn't load the file....do you have to build some speciall libogg support into it?

There's a new version of mplayer, it works right if you have libogg and libvorbis.

The new mplayer is sexellent.