PDA

View Full Version : RV9 Animation DropDupe Pre-filter


karl_lillevold
28th June 2003, 21:55
If you would like to try a new pre-filter for RV9 that drops duplicated frames before encoding, please download videodupframedropper.dll (http://www.lillevold.com/files/videodupframedropper.dll) and place this DLL in the tools folder in the producer directory. Helix Producer 9.2 M4 is recommended, but it should work in earlier versions as well. If you would rather wait until it is more stable, please do. Parameters etc are likely to change, to make it work for noisier material. This pre-filter is open source, and contributions are welcome, both in the form of suggestions, parameter tuning, and code modifications. See below for more info.

DropDupe filter works by detecting duplicated frames and drop them before the encoder sees them. Since the RM file format supports variable framerate, playback will still appear just like if the frames were encoded.

The DropDupe pre-filter improves compression efficiency with RV9 for two reasons:
1) There is no need to encode the duplicated frames. These frames normally require few bits though, perhaps 5-10% overall.
2) more importantly, the adaptivity in the RV9 codec can do a better job when the duplicated frames are removed, so the overall improvement may in many cases be higher than these 5-10%.

This filter is very useful with RV9-EHQ, because encoding without the duplicated frames is much faster. If every 2nd frame is removed, it will be 2X faster. The DropDupe filter takes very little time by itself, being MMX optimized and it exits early whenever thresholds are exceeded. EHQ at level 80 is strongly recommended in addition to this pre-filter, for the best possible animation encodes.

Recommended (idea from midiguy): Use Donald's Graft's Dup as front-end for DropDupe. See section below.

How to enable the Animation DropDupe filter

GUI Front-ends:
Sirber's RealAnime (http://forum.doom9.org/showthread.php?s=&threadid=56576)

Producer Job file:
Insert this code in the prefilters section of the Producer job file (after making sure videodupframedropper.dll is in the tools folder). see post below for complete job file example.

<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">3</maxDroppedFrames>
<maxAvgSSD type="uint">700</maxAvgSSD>
<maxAreaSSD type="uint">6000</maxAreaSSD>
<maxAvgChromaSSD type="uint">200</maxAvgChromaSSD>
<maxAreaChromaSSD type="uint">1500</maxAreaChromaSSD>
<earlyExit type="bool">true</earlyExit>
<enableDetailedLogInfo type="bool">false</enableDetailedLogInfo>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>

Parameters
maxDroppedFrames : max number of frames to drop before letting a frame pass through.
maxAvgSSD : max SSD per 16x16 area averaged for the luma plane of the image, when comparing current with previous image, before considering the frame to be a new frame.
maxAreaSSD : max SSD in any 16x16 area, before considering the frame to be a new frame.
maxAvgChromaSSD : same as maxAvgSSD, but for the chroma planes (Cb and Cr).
maxAreaChromaSSD : same as maxAreaSSD, but for the chroma planes (Cb and Cr).
earlyExit : exit early from detection function when one of the thresholds is exceeded. Disable for accurate numbers in log.
enableDetailedLogInfo: log information for every frame : DROP/keep and all the measured SSD numbers. Format: (framenum) max: maxSSDluma maxSSDCr maxSSDcb avg: avgSSDluma avgSSDCr avgSSDCb. When SSD is 999999 or 2*threshold for avgluma it means early exit prevented exact calculation. Disable earlyExit to obtain exact numbers.

Note: SSD = Sum of Squared Differences
If a paramater is not included in the job file, the default value will be set as given in the example above. The minimal amount of code in the job file to enable the filter with all parameters set to their default value is

<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>

Source code
Along with the other Producer pre-filters, this is open source and contributions are welcome! The source code is available through the Helix Community via CVS checkout from producersdk/plugins/transform/videodupframedropper, or simply browse the CVS repository online (https://helixcommunity.org/viewcvs/cgi/viewcvs.cgi/producersdk/plugins/transform/videodupframedropper/). Please follow instructions on the HC website (https://common.helixcommunity.org/2004/devdocs/quickstart) for CVS access and build setup info. Maybe you have an idea for your own pre-filter? This is a nice and simple example to start with.

How do I know it's working?
Set enableDetailedLogInfo to 'true' and look in producer.log, or when playing back the RM files after encode, go to Tools->Playback Statistics. Click the Streams tab, and select the RealVideo stream in the dropdown box. The Frame rate will show the resulting frame rate after the duplicated frames were dropped. For instance, for a 23.976 source, with every 2nd frame dropped (on average) it will show around 12 fps. If it does not appear like any frames have been dropped, and the clip does indeed have duplicated frames, you have to enableDetailedLogInfo and look in producer.log for the SSD values found, and adjust thresholds.

Use Dup (AviSynth filter) as front-end
Donald Graft's Dup Avisynth filter (http://shelob.mordor.net/dgraft/dup/dupnew.html) does a better job with noisy clips, and has more advanced options for look-ahead, blend, use-last frame in sequence. If you add this in your Avisynth script before feeding it to Producer with DropDupe set to low thresholds (since Dup creates true duplicated frames), you will get very good results:
In your AviSynth script, include
LoadPlugin("<path_to_plugin>\dup.dll")
Video=Dup(Video,show=false)
Then set show=true, and load avs in VirtualDub to make sure you tune the Dup thresholds to your clip. See Dup documentation. Defaults have worked well, or maybe a little too aggressive, for my limited testing.

Then set the all thresholds for DropDupe to l (one), before using the .avs script with Dup as your input to producer. Remember to set show=false, otherwise DropDupe will detect changed frames, and not drop them. Besides you don't want to encode the debug info from the Dup filter.


Thanks
Thanks to Kaiousama for the original idea, testing and feedback, iwod for bringing it back up and experimenting with Avisynth filters, Sirber for example source material and testing, and to Yusaku for posting this little challenge, that I could not resist: "I think that this would be a great feature for anime encodes, but you should file this as a request to XviD (or other video format) developers, as it cannot be done on the container level. (and there is not much of a chance of it getting into encoding applications for RM, IMO)"

todo (will be edited when done)
- take history into account for more reliable and stable detection with noisy clips.
- Assumes MMX is available. This will be fixed shortly as well, but who has a PC without MMX these days?
- You tell me, after trying it out...

notes
this filter was not used for Sirber and Ramirez' anime codec comparison. Even though RV9+EHQ is possibly considered the best result at 350 kbps, pretty close to XviD, but much better than the other codecs, with RV9+EHQ+DropDupe, a new 250 kbps RV9 version (http://www.lillevold.com/files/T03_RV9_EHQ_DD_2pass_250kbps.rmvb) with this new filter looks virtually identical to the 350 kbps version (keep in mind, the compression artifacts you may see, are already in the high bitrate DivX source. See this thread (http://forum.doom9.org/showthread.php?s=&threadid=56392)).

The next post contains a complete job file example for Helix Producer 9.2. This will not work in the GUI Helix Producer, but if anyone is interested, I will post the code to use for GUI Helix Producer job files later.

Sirber
28th June 2003, 21:59
Great!

I'll add it to my software and test it. Thanks Karl!!! Is it me or your team is working quite fast? 1 week for EHQ and 1 week for DD :D

karl_lillevold
28th June 2003, 22:00
Here is a complete job file to use with Helix Producer 9.2 that enables the Animation drop duplicate frames pre-filter. Edit this with your favorite text or XML editor, save to myjobfile.rpjf (or any extension). Let me know if you have an XML editor that works well

Run with producer -j myjobfile.rpjf -lc w,i,d,e

The -lc (logging category) options are w=warning, i=information, d=diagnostics, and e=error messages. Producer's output is also placed in the producer.log file. Sometimes very handy. Even more information (too much) can be logged by having an empty file called debug.txt in the same folder as producer.

<?xml version="1.0" encoding="UTF-8"?>
<job xmlns="http://ns.real.com/tools/job.2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ns.real.com/tools/job.2.0
http://ns.real.com/tools/job.2.0.xsd">
<enableTwoPass type="bool">true</enableTwoPass>
<parInputs>
<input xsi:type="avFileInput">
<filename type="string">e:\avis\t03.avi</filename>
<prefilters>
<!-- -->
<!-- Animation DropDupe pre-filter settings -->
<!-- -->
<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">3</maxDroppedFrames>
<maxAvgSSD type="uint">700</maxAvgSSD>
<maxAreaSSD type="uint">6000</maxAreaSSD>
<maxAvgChromaSSD type="uint">200</maxAvgChromaSSD>
<maxAreaChromaSSD type="uint">1500</maxAreaChromaSSD>
<earlyExit type="bool">true</earlyExit>
<enableDetailedLogInfo type="bool">false</enableDetailedLogInfo>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>
<!-- -->
</prefilters>
</input>
</parInputs>

<parOutputs>
<output>
<destinations>
<destination xsi:type="fileDestination">
<filename type="string">d:\anime.rmvb</filename>
</destination>
</destinations>
<mediaProfile>
<audioMode type="string">music</audioMode>
<disableAudio type="bool">false</disableAudio>
<disableVideo type="bool">false</disableVideo>
<resizeQuality type="string">high</resizeQuality>
<videoMode type="string">normal</videoMode>
<audienceRefs>
<audienceRef>My Audience</audienceRef>
</audienceRefs>
</mediaProfile>
</output>
</parOutputs>
<audiences>
<audience>
<avgBitrate type="uint">600000</avgBitrate>
<maxBitrate type="uint">1300000</maxBitrate>
<name type="string">My Audience</name>

<streams>
<stream xsi:type="videoStream">
<pluginName type="string">rn-videocodec-realvideo</pluginName>
<codecName type="string">rv9</codecName>
<!-- EHQ settings for RV9 -->
<codecProperties type="bag">
<encoderComplexity type="uint">80</encoderComplexity>
<customPacketSize type="uint">16000</customPacketSize>
</codecProperties>
<encodingType type="string">vbrBitrate</encodingType>
<quality type="uint">1</quality>
<maxStartupLatency type="double">25</maxStartupLatency>
<maxFrameRate type="double">30</maxFrameRate>
<maxKeyFrameInterval type="double">10</maxKeyFrameInterval>
<enableLossProtection type="bool">false</enableLossProtection>
</stream>
<stream xsi:type="audioStream">
<!-- Audio codec flavor determines bitrate and
stereo vs mono. See
docs/AudienceFile.htm (Audio_Codec_Tables) -->
<codecFlavor type="uint">28</codecFlavor>
<codecName type="string">cook</codecName>
<pluginName type="string">rn-audiocodec-realaudio</pluginName>
<streamContext type="bag">
<audioMode type="string">music</audioMode>
<presentationType type="string">audio-video</presentationType>
</streamContext>
</stream>

</streams>
</audience>

</audiences>
</job>

Sirber
28th June 2003, 22:00
Hum...

What if I don't use job files? :confused: Can I use it from the Audience file? Could I get the Codec version?

karl_lillevold
28th June 2003, 22:07
Originally posted by Sirber
Great!

I'll add it to my software and test it. Thanks Karl!!! Is it me or your team is working quite fast? 1 week for EHQ and 1 week for DD :D
Thanks! The team in this case has been me by myself. EHQ took more than 1 week though, more like 3-4 :) I have been so lucky to have enough time to work on RV9 improvements lately. My video codec co-worker, has not been so lucky, having had to struggle with ports to various devices, but he is eager to get going too now, with various potential 2-pass VBR improvements.


What if I don't use job files? Can I use it from the Audience file?
Unfortunately, no. Job files are so much more flexible, I would strongly recommend moving to using those instead. It should be possible to write out a job file with the extra code necessary for job files, and call producer with -j myjobfile.rpjf. Everything (and more) which you can specify on the command line, can relatively easily be added to the job file. Let me know if you have any specific problems.

This filter is one reason. Another reason is that for multi-channel encoding, you have to use separate input files for audio (ac3) and video (avs), and the only way to specify this is via job files.

Sirber
28th June 2003, 22:09
I'm a little noob in jobfiles, is there a way to add the job file + the audience, with the jobfile with only prefilter settings? I'm a little lost... :confused:

Or I'll move all to job file.

karl_lillevold
28th June 2003, 22:22
i was a noob with job files too, still am, but I am learning.

I am not quite sure I understand your question, but if you look at the example above, you see how the audio "My audience" is specified in the job file. Then in the <mediaProfile> section of the <parOutputs> section I specify to use "My Audience" as the audience for this output.

I just tried to use an audience that's defined in its own file, and then run producer with -j jobfile -ad audiencefile, but was not able to. So as you can see, I am still learning job files too. Check the jobfile.htm documentation in the docs sub-folder, or this link:
https://helix-producer.helixcommunity.org/specs/Producer9.2/JobFile/index.htm
from the main Producer 9.2 specifications at
https://helix-producer.helixcommunity.org/specs/index.html

Sirber
28th June 2003, 23:16
Thanks Karl!

Your help is greatly appreciated :D

I'll move all to job file then.

Dark-Cracker
28th June 2003, 23:26
why not add this filter in the command line like "black level","deinterlace" or "video noise reduction" filter made by realnetwork ? somethink like :

-dup 3,2500,10000

could be more usable IMHO.

>Another reason is that for multi-channel encoding, you have to use separate input files for audio (ac3) and video (avs), and the only way to specify this is via job files.

if u use wavsource() in avisynth script using a 5.1 .wav file real producer will not support it ? using directely .ac3 file need use the directshow filter (and this not always work if u not have the right dshow filter).

Bye.

karl_lillevold
28th June 2003, 23:38
Originally posted by Dark-Cracker
why not add this filter in the command line like "black level","deinterlace" or "video noise reduction" filter made by realnetwork ? somethink like :

-dup 3,2500,10000

could be more usable IMHO.

Maybe so, but then I would have to have it added to the spec. Talk to the right Producer people, convince them it's a good idea, etc. etc. Not to mention recompile all of Producer, and not only this pre-filter DLL. More importantly, this is way more flexible, everyone can make changes, add/change parameters, re-compile, plug right in. Everyone has access to the source code for the filter itself. It is fully backwards compatible with previous producer versions. Everyone can add their own pre-filters, independent of Producer versions. You can do none of these things with cmd line parameters.

I know it's a lot of work for the GUI front-ends to move to using job files instead of cmd line paremeters, and I am sorry this is the case, but once you do, you will soon realize it is more flexible.


if u use wavsource() in avisynth script using a 5.1 .wav file real producer will not support it ? using directely .ac3 file need use the directshow filter (and this not always work if u not have the right dshow filter).

This may or may not be possible. I just have not had time to experiment enough (have been busy with video work), so please try it out and see what happens. It has to be a WAVEFORMAT EXT file, and it does not seem like Besweet produces such a file. Avisynth also needs to properly support 5.1 channels, and pass them on to Producer. This may or may not work, as well, just have not tried. Let's move this discussion to the thread I will start soon on RealAudio Multi-Channel encoding :)

Dark-Cracker
29th June 2003, 00:02
yes u are right add it to the command line it's less flexible i have only ask this because all other filter are in cli mode :) but for sure if this filter should evolute it's not a good idear to recompile a new producer version each time :)

u are right moving gui on job file request a lot of modifications.

for the audio u are right it's not the good thread.

Sirber
29th June 2003, 01:33
In your job exemple, there is this:

<disableAudio type="bool">true</disableAudio>
<disableVideo type="bool">true</disableVideo>

I guess I have to set false there?

[edit]

Producer is scaring me:

Diagnostic: keep max_ssd: 25927, frame_ssd: 5000
Diagnostic: keep max_ssd: 12928, frame_ssd: 5000
Diagnostic: keep max_ssd: 14845, frame_ssd: 5000
Diagnostic: keep max_ssd: 17365, frame_ssd: 5000
Diagnostic: keep max_ssd: 21805, frame_ssd: 5000
Diagnostic: keep max_ssd: 149094, frame_ssd: 5000
Diagnostic: keep max_ssd: 10352, frame_ssd: 5000
Diagnostic: keep max_ssd: 16384, frame_ssd: 5000
Diagnostic: keep max_ssd: 13038, frame_ssd: 5000
Diagnostic: keep max_ssd: 16600, frame_ssd: 5000
Diagnostic: keep max_ssd: 42320, frame_ssd: 5000
Diagnostic: keep max_ssd: 16174, frame_ssd: 5000

what's that?

karl_lillevold
29th June 2003, 01:44
ah, yes, my mistake, both should be 'false'. I was using it to compress T03 without audio, thus having one 'true' and one 'false' and edited both to 'true' after posting. Should have been 'false' Thanks! Fixed now.

Sgt_Strider
29th June 2003, 02:24
Can this filter be use for anything other than anime or animation type of DVD's? Is this filter effective for "real life" movies like black hawk down, matrix, Band of Brothers, and etc.?

Sirber
29th June 2003, 02:26
The filter removes frames with no motion, and in real movies, there is always some actions :D

karl_lillevold
29th June 2003, 02:29
@Sirber: I think I need to reduce the pre-filter logging messages. 'keep' means that the SSD is so high it has decided to keep the frame. When it says 'DROP' it has been dropped. when frame_ssd says 5000, it means it has exited early after finding that maxAvsSSD has been exceeded. max_ssd is the max areaSSD found before the function exited.

@Sgt_Strider: I don't think this filter will provide much benefit unless the frames are truly duplicated, but maybe someone can improve it or write another version that drops frames under other circumstances, like extreme high action, spatial complexity, but how this would look is another matter.

Sirber
29th June 2003, 03:18
DupeDrop is dropping about 3/4 of total frames. I may have to reduce the bitrate... :D

Kaiousama
29th June 2003, 15:57
I've tested this new filter....... very very good results:

It does exactly the needed work and the side effects are little and probably easily resolvable.

I've tested it on one of the first akira vobs, the one where testsuo is admiring kaneda's bike and all the next bike racer scene; in this PAL clip no one of the frames had to be dropped because it has Zero duplicated frames --> the filter drops only 20 frames (it is a fraction of a percentage point, so a little error amount)

The Second tested clip is the Juuni Kokki 6th episode, it has 24,16,8,4 fps scenes. The filter remove in the main of cases exaclty the needed frames with no fluidity degradation during the playback. There are only 2 macroscopic errors:
- The first is in the first 5 seconds during the opening, there is an almost white scene where the point of view is inside a fog that is flowing toward the watcher; in this case the filter wrongly drops many frames because the movement is very little and the changing is almost only chromatic
- The second is on the first 10 second in the ending, the screen is black and still and slowly appears a yellow mon, the filter didn't recognise even this scene (probably the same chroma missing feature)

Conclusions : in my opinion the most important next feature is chroma informations handling during the duplicates' recognition.

On another side, this test make me find even a new and (in my opinion) better default values for the filter, here they are:



<prefilters>
<!-- -->
<!-- Animation DropDupe pre-filter settings -->
<!-- -->
<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">5</maxDroppedFrames>
<maxAvgSSD type="uint">700</maxAvgSSD>
<maxAreaSSD type="uint">6000</maxAreaSSD>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>
<!-- -->
</prefilters>



I've increased the maxDroppedFrames to 5 in order to properly handle 4fps scenes (24fps / 4 fps = 1 good frame every 6, so 5 consequential drop frames)
I've decreased the other 2 values because both in akira (clean source) and Juuni Kokki (noisy source), duplicates' maxAvgSSD was around 300-500 and maxAvgSSD was near 2000-5000.
Try them yourself to check if my defaults are validable as generally usable default values. ;)

@ Karl:

It would be useful to have written into the log file even if EHQ was enabled or not during the encoding session (better with its strenght values).

a little question: when the Dupe filter checks for duplicate, is needed the frame maxAvgSSD AND maxAreaSSD to be verified for recognising the frame as duplicate or it's sufficient one of the 2 conditions to be true to drop the frame?

my opinions : The filter's log information are good as they are, and i think it's a good solution even to have the filter separated from the producer main application (jobs are all but difficult to use).
One thing, it would be good to add to log information the frames' number of each processed frame in order to visually check later on the source clip which are the dropped frames.

netterpaladin
29th June 2003, 16:19
Hi,
(a little of topic)

sounds like a very nice improvement for anime encoding. My question now is:

Real seems to be a very good choise as the video codec for anime

BUT matroska seems to be the container of choise for anime because of its awesome subtitle support.

Now if i remember right, real does not work with matroska? Is that true? It seems that the combination of real video and matroska subtitles is the PERFECT Combo. Is there any chance to see that in the future?


Daniel

Dark-Cracker
29th June 2003, 16:36
use the search button and u will find topic speaking about the possibility to add support of real video in matroska.

karl_lillevold
29th June 2003, 16:45
@Kaiousama: thanks for testing it out and the great feedback! This is just the kind of information I did not find from my limited test content. I will add chroma today, change the log to include frame number, and try out your suggested thresholds and possibly update the default values.

karl_lillevold
29th June 2003, 16:54
Originally posted by Kaiousama
It would be useful to have written into the log file even if EHQ was enabled or not during the encoding session (better with its strenght values).

a little question: when the Dupe filter checks for duplicate, is needed the frame maxAvgSSD AND maxAreaSSD to be verified for recognising the frame as duplicate or it's sufficient one of the 2 conditions to be true to drop the frame?
1) I am not sure I understand the first point here. I think the log file will be written to no matter what the EHQ setting is.
2) When just one of the thresholds is exceeded, the frame is considered to be a new frame, and it is passed through to the encoder.

I will add another param, a bool to enable or disable early exits, such that the actual max_ssd and avg_frame_ssd numbers will be printed in the log, not just what was found before the function early exit. This will slow down the detection slightly, but useful for tuning the thresholds.

netterpaladin
29th June 2003, 16:54
@ dark-cracker

Okay it seems, I have basically done the typical newby mistake... But actually i was reading threads about this. But i was only writing from memory.
So i just checked again, and you asked the same question as me.... I actually read this post before...

Here are the answers from Pamel:
1. Legal. Creating a Real file parser from scratch and distributing it may not be legal. Connecting directly to the Real dll's from the Matroska DirectShow filter may not be legal either. An email has already been sent to the Real mailing lists to find out about this.

2. Trying to connect to the Real dll's so that they render the data properly could be difficult. It is currently done through mplayer and mpc though, so it is definately possible.

3. For some reason, the real codec will output more than one data packet with the same timecode. Currently, all other codecs have one data packet per timecode, and while it is certainly possible to have more than one, it is easier if there isn't. Karl has said that the packets are merged when passing through the rvrender, before they get to the codec. If this is accurate, then it may be possible to just merge the packets before storing them in mkv.



The whole point of my post was to encourage the real people to think about matroska and combining the the advantages of both... I guess that would make many people happy. Thats why I posted here. I am pretty sure the matroska guys are more than happy to store real in their container. They seem wanting to pack everything in there....

sorry for the bad implementation on my side.

Alucard

karl_lillevold
29th June 2003, 17:02
Originally posted by netterpaladin
The whole point of my post was to encourage the real people to think about matroska and combining the the advantages of both... I guess that would make many people happy. Thats why I posted here. I am pretty sure the matroska guys are more than happy to store real in their container. They seem wanting to pack everything in there....

A little off-topic for this thread, but yes, we have received the request from the Matroska team to allow transcoding from RMFF to Matroska. ChristianHJW made a long list of very useful arguments, that would probably be nice to repeat in this forum, but not this thread. This was forwarded around to the right people at RealNetworks. Hopefully, this will lead to changes, but I can not promise anything or guess on a timeframe.

karl_lillevold
29th June 2003, 18:31
I added chroma, improved log info, and the ability to enable or disable detailed log info. Default is off. It would be great to have the chroma feature tested. I really don't know what good default values are, and without a suitable test clip, whether or not it does exactly what it is supposed to :) Hopefully we can find good default values, so tuning is not needed for every single clip.. I am also thinking perhaps the AvgThresholds are not needed.. Maybe the MaxThresholds (per area) are all that's needed?

Here is the updated parameter list, default values filled in:

<!-- -->
<!-- Animation DropDupe pre-filter settings -->
<!-- -->
<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">5</maxDroppedFrames>
<maxAvgSSD type="uint">700</maxAvgSSD>
<maxAreaSSD type="uint">6000</maxAreaSSD>
<maxAvgChromaSSD type="uint">200</maxAvgChromaSSD>
<maxAreaChromaSSD type="uint">1500</maxAreaChromaSSD>
<earlyExit type="bool">true</earlyExit>
<enableDetailedLogInfo type="bool">false</enableDetailedLogInfo>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>

Download from same link as listed in 1st post in this thread.

Sirber
29th June 2003, 19:08
I have problems since I updated the filter.

,Diagnostic,Command Line,2003/06/29 14:06:42,0,"C:\Documents and Settings\sirber\Mes documents\Progs\RealAnime\producer\producer.exe -j C:\job_tmp.rpjf -lc e,i,d,w -di auto" command line being run
,Diagnostic,File Reader,2003/06/29 14:06:43,8050,--------- Input File Properties from rn-avfile-directshow
,Diagnostic,File Reader,2003/06/29 14:06:43,8051,Input Filename: Y:\Sirber\Ranma\OAV08.avi
,Diagnostic,File Reader,2003/06/29 14:06:43,8052,File Size: 233.80MB
,Diagnostic,File Reader,2003/06/29 14:06:43,8053,Total Duration: 27:42.454
,Diagnostic,File Reader,2003/06/29 14:06:43,8080,Video Track
,Diagnostic,File Reader,2003/06/29 14:06:43,8081, Dimensions: 640 x 480
,Diagnostic,File Reader,2003/06/29 14:06:43,8082, Frame Rate: 23.976 FPS
,Diagnostic,File Reader,2003/06/29 14:06:43,8083, Format: XVID
,Diagnostic,File Reader,2003/06/29 14:06:43,8084, Duration: 27:42.454
,Diagnostic,File Reader,2003/06/29 14:06:43,8060,Audio Track
,Diagnostic,File Reader,2003/06/29 14:06:43,8061, Channels: Stereo
,Diagnostic,File Reader,2003/06/29 14:06:43,8062, Bit Depth: 16
,Diagnostic,File Reader,2003/06/29 14:06:43,8063, SampleRate: 48000 Hz
,Diagnostic,File Reader,2003/06/29 14:06:43,8064, Format: Uncompressed Audio
,Diagnostic,File Reader,2003/06/29 14:06:43,8065, Duration: 27:42.454
,Diagnostic,File Reader,2003/06/29 14:06:43,8054,--------- End Input File Properties
,Error,Command Line,2003/06/29 14:06:43,10533,Unable to create an encoding job from file "C:\job_tmp.rpjf"

and my job file is:

<?xml version="1.0" encoding="UTF-8"?>
<job xmlns="http://ns.real.com/tools/job.2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ns.real.com/tools/job.2.0 http://ns.real.com/tools/job.2.0.xsd">
<enableTwoPass type="bool">true</enableTwoPass>
<parInputs>
<input xsi:type="avFileInput">
<filename type="string">Y:\Sirber\Ranma\OAV08.avi</filename>
<prefilters>
<!-- -->
<!-- Animation DropDupe pre-filter settings -->
<!-- -->
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">5</maxDroppedFrames>
<maxAvgSSD type="uint">700</maxAvgSSD>
<maxAreaSSD type="uint">6000</maxAreaSSD>
<maxAvgChromaSSD type="uint">200</maxAvgChromaSSD>
<maxAreaChromaSSD type="uint">1500</maxAreaChromaSSD>
<earlyExit type="bool">true</earlyExit>
<enableDetailedLogInfo type="bool">false</enableDetailedLogInfo>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
<!-- -->
</prefilters>
</input>
</parInputs>

Anyone can help?

karl_lillevold
29th June 2003, 19:13
@Sirber: I think there is a missing <prefilter> around the DropDupe entry. Each prefilter needs to be wrapped in this tag:

<prefilters>
<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
[...]
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>
</prefilters>

Sirber
29th June 2003, 19:18
You're right. I deleted too much lines :D Thanks a lot!

[edit]

Diagnostic: Using RealAnime Audience
Diagnostic: Using video codec: rv9 (267959 bps)
Diagnostic: Using audio codec: cook (32041 bps)
Info: Starting analysis pass
Diagnostic: Setting video packet size to 1352

Using:

<codecProperties type="bag">
<encoderComplexity type="uint">80</encoderComplexity>
<customPacketSize type="uint">16000</customPacketSize>
</codecProperties>

The packetsize is wrong. Is it a bug or I missed something?

karl_lillevold
29th June 2003, 19:22
the packetsize printed by Producer is the default it wants to use. the codecProperties parameter over-rides this in the codec, but is not printed in the log file. I will see if it is easy to make the codec print in the log the codecProperties that are being set.

Kaiousama
29th June 2003, 22:51
@ Karl:

I've tryed the chroma feature but it has not the influence i hoped in helping me to solve the bad behaviours.
Maybe we've considered the problem from a too minimalistic point of view.... i would like to bring this sheets (taken from the noisy juuni kokki videoclip) to your attention :


This is a 8fps sequence :


,Diagnostic,Video Pre-filters,2003/06/29 22:47:46,0,DD: keep ( 123) max: 5062219 224017 108214, avg: 1133697 12001 10260
,Diagnostic,Video Pre-filters,2003/06/29 22:47:46,0,DD: keep ( 124) max: 9585 4975 2846, avg: 346 154 149
,Diagnostic,Video Pre-filters,2003/06/29 22:47:47,0,DD: keep ( 125) max: 12464 2541 927, avg: 623 130 137
,Diagnostic,Video Pre-filters,2003/06/29 22:47:47,0,DD: keep ( 126) max: 5694202 138885 72569, avg: 193173 3957 3439
,Diagnostic,Video Pre-filters,2003/06/29 22:47:47,0,DD: keep ( 127) max: 7594 3223 3917, avg: 551 260 278
,Diagnostic,Video Pre-filters,2003/06/29 22:47:47,0,DD: keep ( 128) max: 11758 740 899, avg: 495 44 40
,Diagnostic,Video Pre-filters,2003/06/29 22:47:48,0,DD: keep ( 129) max: 4627831 144591 68555, avg: 157821 2028 1295
,Diagnostic,Video Pre-filters,2003/06/29 22:47:48,0,DD: keep ( 130) max: 10662 2017 2679, avg: 762 136 152
,Diagnostic,Video Pre-filters,2003/06/29 22:47:48,0,DD: keep ( 131) max: 5646 250 198, avg: 323 31 33
,Diagnostic,Video Pre-filters,2003/06/29 22:47:48,0,DD: keep ( 132) max: 5584842 45894 26102, avg: 208398 867 1239
,Diagnostic,Video Pre-filters,2003/06/29 22:47:49,0,DD: keep ( 133) max: 9816 4695 8177, avg: 1182 369 499
,Diagnostic,Video Pre-filters,2003/06/29 22:47:49,0,DD: keep ( 134) max: 24003 3057 2977, avg: 1199 88 77
,Diagnostic,Video Pre-filters,2003/06/29 22:47:49,0,DD: keep ( 135) max: 3667489 206293 104577, avg: 200802 3666 3463
,Diagnostic,Video Pre-filters,2003/06/29 22:47:49,0,DD: keep ( 136) max: 8303 9368 2690, avg: 487 210 156
,Diagnostic,Video Pre-filters,2003/06/29 22:47:50,0,DD: keep ( 137) max: 8655 1408 1966, avg: 646 81 71
,Diagnostic,Video Pre-filters,2003/06/29 22:47:50,0,DD: keep ( 138) max: 2636697 98990 64107, avg: 176309 2547 3287
,Diagnostic,Video Pre-filters,2003/06/29 22:47:50,0,DD: keep ( 139) max: 12815 3113 2340, avg: 985 178 188
,Diagnostic,Video Pre-filters,2003/06/29 22:47:50,0,DD: keep ( 140) max: 12062 735 462, avg: 614 22 22
,Diagnostic,Video Pre-filters,2003/06/29 22:47:51,0,DD: keep ( 141) max: 2668548 75665 50678, avg: 177667 1681 2214
,Diagnostic,Video Pre-filters,2003/06/29 22:47:51,0,DD: keep ( 142) max: 8515 8143 6469, avg: 990 475 576
,Diagnostic,Video Pre-filters,2003/06/29 22:47:51,0,DD: keep ( 143) max: 5045357 91939 52252, avg: 309478 2527 3061
,Diagnostic,Video Pre-filters,2003/06/29 22:47:51,0,DD: keep ( 144) max: 10563 3670 10175, avg: 567 251 386
,Diagnostic,Video Pre-filters,2003/06/29 22:47:51,0,DD: keep ( 145) max: 10756 1584 1596, avg: 1480 187 171
,Diagnostic,Video Pre-filters,2003/06/29 22:47:52,0,DD: keep ( 146) max: 15377 1170 2085, avg: 1238 70 68


And this is the 24fps foggy scene :


,Diagnostic,Video Pre-filters,2003/06/29 22:31:00,0,DD: keep ( 25) max: 5558 422 352, avg: 820 73 69
,Diagnostic,Video Pre-filters,2003/06/29 22:31:00,0,DD: keep ( 26) max: 16457 720 866, avg: 1863 130 136
,Diagnostic,Video Pre-filters,2003/06/29 22:31:00,0,DD: keep ( 27) max: 7269543 52724 34709, avg: 2273066 5199 5254
,Diagnostic,Video Pre-filters,2003/06/29 22:31:01,0,DD: keep ( 28) max: 1530457 6323 6137, avg: 243766 1056 888
,Diagnostic,Video Pre-filters,2003/06/29 22:31:01,0,DD: keep ( 29) max: 27636 1869 558, avg: 2677 53 31
,Diagnostic,Video Pre-filters,2003/06/29 22:31:01,0,DD: keep ( 30) max: 1352847 4066 871, avg: 239012 158 86
,Diagnostic,Video Pre-filters,2003/06/29 22:31:01,0,DD: keep ( 31) max: 1232018 4125 1148, avg: 244567 250 124
,Diagnostic,Video Pre-filters,2003/06/29 22:31:02,0,DD: keep ( 32) max: 1258010 1435 1124, avg: 239088 164 97
,Diagnostic,Video Pre-filters,2003/06/29 22:31:02,0,DD: keep ( 33) max: 1260600 2664 634, avg: 244532 200 101
,Diagnostic,Video Pre-filters,2003/06/29 22:31:02,0,DD: keep ( 34) max: 1337209 3004 739, avg: 245091 232 114
,Diagnostic,Video Pre-filters,2003/06/29 22:31:02,0,DD: keep ( 35) max: 1671186 2550 820, avg: 246467 228 116
,Diagnostic,Video Pre-filters,2003/06/29 22:31:03,0,DD: keep ( 36) max: 1423471 2668 643, avg: 251653 201 93
,Diagnostic,Video Pre-filters,2003/06/29 22:31:03,0,DD: keep ( 37) max: 1362168 5740 3286, avg: 248824 190 95
,Diagnostic,Video Pre-filters,2003/06/29 22:31:03,0,DD: keep ( 38) max: 1354510 3793 3144, avg: 247833 330 229
,Diagnostic,Video Pre-filters,2003/06/29 22:31:04,0,DD: keep ( 39) max: 1434941 1264 1770, avg: 244878 86 41
,Diagnostic,Video Pre-filters,2003/06/29 22:31:04,0,DD: keep ( 40) max: 1399868 729 515, avg: 251547 86 40
,Diagnostic,Video Pre-filters,2003/06/29 22:31:04,0,DD: keep ( 41) max: 1421801 1828 560, avg: 247995 116 51
,Diagnostic,Video Pre-filters,2003/06/29 22:31:04,0,DD: keep ( 42) max: 1418843 2781 1282, avg: 244101 137 53
,Diagnostic,Video Pre-filters,2003/06/29 22:31:05,0,DD: keep ( 43) max: 1313546 2405 479, avg: 247928 156 50
,Diagnostic,Video Pre-filters,2003/06/29 22:31:05,0,DD: keep ( 44) max: 1529740 1615 295, avg: 249372 113 35
,Diagnostic,Video Pre-filters,2003/06/29 22:31:05,0,DD: keep ( 45) max: 1675041 3866 513, avg: 253427 153 41
,Diagnostic,Video Pre-filters,2003/06/29 22:31:06,0,DD: keep ( 46) max: 1405619 2819 533, avg: 255522 149 48
,Diagnostic,Video Pre-filters,2003/06/29 22:31:06,0,DD: keep ( 47) max: 1274292 3557 1070, avg: 256652 159 54
,Diagnostic,Video Pre-filters,2003/06/29 22:31:06,0,DD: keep ( 48) max: 1293300 3537 2203, avg: 261005 134 51


In the first sheet you can note a rhytmic cadence, i mean the difference of each duplicate from the good frame is 10^1 - 10^2 times, while the difference of one duplicate from the other is not so evident.
In the second sheet there is no rhitmical cadence, the difference of each good frame from the other is not too much but, more important, it is not repeating ciclically.

What i wanna tell you is that maybe comparing each frame with the previous in order to find duplicates is not the right way to walk, maybe it is better the filter stores in memory a buffer of values taken from the 5 previous frames (5 for the 24fps story i've told in the last reply), attemps to find if there is a rhitmical cadence and compares the new frame with the ipotetical good frame.
In this case the very first 5 frames of the clip will not be analized but this fact is not relevant because rarely the first 5 frames'movement is important/noticeable

A pratical example:

you have a 8fps scene ( "0" = duplicate, "A" = good frame)

A00A00A00A00A00A00A0A000

the filter analyzes the first 5 frames and stores them in the buffer that will contain:

A00A0

The filter, analyzing the buffer, will notice the difference between frame 2 and 3 is much smaller than the difference from 1 to 2 or from 1 to 3, the same cadence is returning from 4 to 5 so the filter has to expect the next frame (the 6th) will be a duplicate and will check the difference of this new frame not with the 5th frame but with the 4th (wich is the real good frame)
When the frames cadence changes the filter simply adapts itself to the new cadence (which, with a 5 frame-long buffer, is always "visible" to the analyzer).

What do you think about?

P.S. i've not well understood the producer.log numbers dislocation:


,Diagnostic,Video Pre-filters,2003/06/29 22:31:06,0,DD: keep ( 48) max: 1293300 3537 2203, avg: 261005 134 51


I was expecting only maxAreaSSD (3537) - maxAreaChromaSSD (2203) - avgAreaSSD(134) - avgAreaChromaSSD(51)
What the 1293300 and 261005 values stands for?

karl_lillevold
29th June 2003, 23:19
Thanks again for excellent suggestions.

You are right, the current approach is a simplistic starting point, put together in a few hours as a proof of concept and rough framework such that we have something to experiment with, even though it works well for relatively clean clips. It would be much better and reliable to take into account the history, such that the actual cadence can be found. We currently do this in the inverse telecine pre-filter. I will look into your suggestions when I have some spare time. You don't happen to have an inclination to try to work on the filter yourself? The source code is available. Let me know if you need some assistance in getting started.

The 3 numbers for max and for avg are for luma, and the two chroma components, Cr and Cb. The big number is luma. To simplify the output, I will combine the two chroma components into one in the log file, i.e. max for both components.

EDIT: If it is so that for most animations the cadence pattern is constant for at least each scene, it should be possible to use the history to achieve very high reliability, without having to adjust any thresholds.

Sirber
30th June 2003, 02:00
@Karl

Hum... I finished my test with the new filter. With Ranma OaV, E08, the framedrop causes wierd thing at 300kbps, on scene change. I'll send you the clip by FTP when it'll be working.

karl_lillevold
30th June 2003, 02:08
okay, that would be good. I will let you know. I am working on including a simple 5 frame history. That should remove the need for most thresholds, and make it more reliable.

Sirber
30th June 2003, 02:49
PM me when ready. I'll delay my encodes :D

karl_lillevold
30th June 2003, 03:36
@Kaiousama: I just realized you may have misunderstood the logging output, and you can fix this sequence, just by increasing the thresholds, maxAreaSSD from 6000 to 12500, and maxAreaChromaSSD to perhaps 12500 as well (hmm, this must be pretty noisy), and also nudge upward the other thresholds. Then the sequence would like like this, which is much better, but still not exactly the right cadence, which a history based analysis would help resolve:


keep ( 123) max: 5062219 224017 108214, avg: 1133697 12001 10260
DROP ( 124) max: 9585 4975 2846, avg: 346 154 149
DROP ( 125) max: 12464 2541 927, avg: 623 130 137
keep ( 126) max: 5694202 138885 72569, avg: 193173 3957 3439
DROP ( 127) max: 7594 3223 3917, avg: 551 260 278
DROP ( 128) max: 11758 740 899, avg: 495 44 40
keep ( 129) max: 4627831 144591 68555, avg: 157821 2028 1295
DROP ( 130) max: 10662 2017 2679, avg: 762 136 152
DROP ( 131) max: 5646 250 198, avg: 323 31 33
keep ( 132) max: 5584842 45894 26102, avg: 208398 867 1239
DROP ( 133) max: 9816 4695 8177, avg: 1182 369 499
keep ( 134) max: 24003 3057 2977, avg: 1199 88 77
keep ( 135) max: 3667489 206293 104577, avg: 200802 3666 3463
DROP ( 136) max: 8303 9368 2690, avg: 487 210 156
DROP ( 137) max: 8655 1408 1966, avg: 646 81 71
keep ( 138) max: 2636697 98990 64107, avg: 176309 2547 3287
DROP ( 139) max: 12815 3113 2340, avg: 985 178 188
DROP ( 140) max: 12062 735 462, avg: 614 22 22
keep ( 141) max: 2668548 75665 50678, avg: 177667 1681 2214
DROP ( 142) max: 8515 8143 6469, avg: 990 475 576
keep ( 143) max: 5045357 91939 52252, avg: 309478 2527 3061
DROP ( 144) max: 10563 3670 10175, avg: 567 251 386
DROP ( 145) max: 10756 1584 1596, avg: 1480 187 171
keep ( 146) max: 15377 1170 2085, avg: 1238 70 68


@Sirber: will do, but it will not be tonight, time to watch the Animatrix ;)

Kaiousama
30th June 2003, 06:52
It would be much better and reliable to take into account the history, such that the actual cadence can be found. We currently do this in the inverse telecine pre-filter.
If it is so that for most animations the cadence pattern is constant for at least each scene, it should be possible to use the history to achieve very high reliability, without having to adjust any thresholds
Yeah, this is what i'm also hopeing ;)
Merging the filter with the IVTC pattern recognition code could be a good solution to predict the right cadence :)

I will look into your suggestions when I have some spare time. You don't happen to have an inclination to try to work on the filter yourself? The source code is available. Let me know if you need some assistance in getting started.
Believe me Karl, i would like so much to work on this filter's code but i've a big problem: i don't know c++ language, only a little of VB (i'm a pharmaceutical chemist in real life), and i think i cannot fill this missing knowledge of mine in one night :(

The 3 numbers for max and for avg are for luma, and the two chroma components, Cr and Cb
Ah, ok .... i've mistunderstood them a little in the previous post ^_^

@Kaiousama: I just realized you may have misunderstood the logging output, and you can fix this sequence, just by increasing the thresholds, maxAreaSSD from 6000 to 12500, and maxAreaChromaSSD to perhaps 12500 as well (hmm, this must be pretty noisy), and also nudge upward the other thresholds. Then the sequence would like like this, which is much better, but still not exactly the right cadence, which a history based analysis would help resolve:
The posted parts are not representative of the whole clip, i mean there are much more clean scenes that will be wrongly dropped if i increase the thresehold, those logs were only functional to explain you the concept ^_^

time to watch the Animatrix
Yo, what a good anime! if you like it, i can do my next build's tests taking this source as clean default source; in this way if'll find wrong pattern recogntions on it i can simply tell you wich part is to gave you a better idea of the problem ;)

By the way, even if you read me find some wrong pattern recognition issues i think the filter is very good even if it is very young, it helps a lot in anime encoding and i think a proper handling for the exceptions is a reachable point, thanks Karl for your work.

Atamido
30th June 2003, 07:55
@karl_lillevold:

Is there any chance of creating more complex configuration options? For example, "Only drop frames if at least 3+ frames in a row are identical"? You could simply have a configuration option to take an interger of 2 or more. Combine that with a threshold setting and you would have a very powerful dupe-detection filter.

midiguy
30th June 2003, 20:04
I thought of any idea.
You could use a current filter (like dup or copysame or whatever) that already has all the bugs worked out, and let that decide what the duplicate frames are. Then just set up the DropDupe RV9 filter to only drop frames that are EXACT duplicates (100% exact) so then the avisynth filter would really be doing all the work and you wouldn't have to worry about any current bugs. Of coarse, this would just be a temporary solution until everything is worked out with the DropDupe filter.

Any thoughts?

Sirber
30th June 2003, 23:20
when I check my Dropped clip and an full one, in the dropped I have sound cut. I was using the lastest filter with max 5 dropped images. Is the dropped images related to audio frames?

karl_lillevold
1st July 2003, 00:02
@Pamel: Yes, such configuration options could easily be added, but I am not sure I understand completely. With such an option, and the number set to anything higher than 1, then it will never be able to skip just one frame, for instance in a 12 fps animation. right?

@midiguy: good idea! Should work well until I find time to iron out the wrinkles in this filter. I will give those a try. The RV9 pre-filter should then have an easy job.

@Sirber: it should not, but I will take a look. Thanks for testing!

midiguy
1st July 2003, 00:33
I know its a good idea :D

But is it possible to set the DropDupe filter to only pick up frames that are 100% exactly the same (to the pixel)?

karl_lillevold
1st July 2003, 04:32
Originally posted by midiguy
I know its a good idea :D

But is it possible to set the DropDupe filter to only pick up frames that are 100% exactly the same (to the pixel)?
You know it! Yes, just set the thresholds to one or very very low. SSD measures the differences between frames (sum of squared differences), so if everything is working correctly, they should all be zero for 100% exact copies. I have not yet tried your suggestion though.

snowcrash
1st July 2003, 06:58
Is there any way to use this filter with the third-party tools like AutoRV9? Thanks.

midiguy
1st July 2003, 19:31
Originally posted by snowcrash
Is there any way to use this filter with the third-party tools like AutoRV9? Thanks.

I predict that eventually it will be supported in AutoRV9, but you will have to ask Dark-Cracker (the author).

Sirber
1st July 2003, 22:16
this filter is for anime mostly, while AutoRV9 for DVD encoding.

[second taught]

I'm tired...

DVD (anime) --> AutoRV9 --> Happy Users :D

sorry... :(

karl_lillevold
2nd July 2003, 01:34
i was experimenting with Donald Graft's Dup Avisynth filter as a front-end for Helix DropDupe, and think I found a bug in Producer itself in that it does not handle dropping two or more frames in a row. Then it sets the frame timestamps to zero, right after the dropped frames. This will cause some issues, extra frames getting dropped, or perhaps audio dropouts during playback, like Sirber mentioned. So until this is fixed, maxDroppedFrames should be set to 1.

EDIT: i know there is a timestamp problem and with 4-5 maxDroppedFrames, there can be much larger gaps in the video, in fact whole scenes (short ones) can get lost, i don't know whether or not this can affect audio, but probably.

Sirber
2nd July 2003, 02:13
I'll try with 1 dropped and get you posted.

Atamido
2nd July 2003, 06:40
Originally posted by karl_lillevold
@Pamel: Yes, such configuration options could easily be added, but I am not sure I understand completely. With such an option, and the number set to anything higher than 1, then it will never be able to skip just one frame, for instance in a 12 fps animation. right? The idea would be to discover still scenes without a possible false-positive. For instance, the situation mentioned earlier for slow zoom scenes in CGI video. These would have a very high chance of registering as a dupe frame. But, if you put a minimum number of dupe frames at 5, then it would have to detect at least 5 frames in a row before it could start dropping frames. This would mean that even a slow CGI zoom scene wouldn't get any dropped frames, but a still CGI scene would get all but the first frame dropped.

A more dynamic filter would work on a curve, where the threshold to drop a single frame would be very high. But the more identical frames detected, the lower the threshold required to determine a frame is a dupe. Of course, you should have a minimum threshold. But, that would mean you would could drop an anime dupe frame easily, and a CGI still scene easily, but a CGI slow zoom would be safe.

midiguy
2nd July 2003, 08:04
It would be cool if someone could set up a program where you could choose which frames to drop (manually go through the frames and pick out the dupes), then the program cna just create a log file of some sort, and this log file could be used in producer? This would be good for very noisy sources..

Sirber
2nd July 2003, 12:26
There is no sound problem when dropping 1 frame.

[edit]

Problems still alive, and more evil and bad. :)

bill_baroud
2nd July 2003, 19:36
i agree here with midiguy, there is _always_ a scene that wont match with the treshold you choose for your video ... and then you're forced to lower treshold and miss some other scenes. A way to exclude some frame from duplicate remove would be great :)
(and hardcore anime encoder are used to ... who never did a manual decimation ?? ;) )

karl_lillevold
3rd July 2003, 03:19
Audio dropout problem fixed along with extra frame drops. Thanks to Sirber for noticing the audio dropouts. Please get new DLL, see 1st post.

I have also experimented with Donald Graft's Dup filter added to the AviSynth script used as input to Producer, as a front-end for DropDupe, and then set the DropDupe thresholds really low, like midiguy suggested. This works better than DropDupe's simplistic approach, and is the recommended method until the built-in detection is improved. I did a couple of test encodes that came out really well with this combination, with 5.1ch sound and everything.

I included this in the 1st post:
Use Dup (AviSynth filter) as front-end
Donald Graft's Dup Avisynth filter (http://shelob.mordor.net/dgraft/dup/dupnew.html) does a better job with noisy clips, and has more advanced options for look-ahead, blend, use-last frame in sequence. If you add this in your Avisynth script before feeding it to Producer with DropDupe set to low thresholds (since Dup creates true duplicated frames), you will get very good results:
In your AviSynth script, include
LoadPlugin("<path_to_plugin>\dup.dll")
Video=Dup(Video,show=false)
Then set show=true, and load avs in VirtualDub to make sure you tune the Dup thresholds to your clip. See Dup documentation. Defaults have worked well, or maybe a little too aggressive, for my limited testing.

Then set the all thresholds for DropDupe to l, before using the .avs script with Dup as your source to Producer. Remember to set show=false, otherwise DropDupe will detect changed frames, and not drop them. Besides you don't want to encode the debug info from the Dup filter.


@midiguy and bill_baroud: Adding a manual method via some kind of user interface would be great for the ultimate anime encode expert, but might take more time than I have right now. I can imagine a VirtualDub plugin with UI that writes out a log file that DropDupe reads.

@Pamel: another improvement that would have been really nice to have, either in an Avisynth front-end filter or directly in this filter.

iwod
7th July 2003, 06:13
So i am back... after leaving for a while.......

Really, i think if anybody wants more improvement, they should suggest it to Donald, where he can improve his Dup2.2,

I don't see why we should we invent the wheel if we alreay have a EXtremely good AVISyth filter. May be all this work about manual choosing which frame not to drop could goto avisyth filter, while the dropdupe filter will keep as a simple if it is the same then drop it approach. Which will save some time for karl to improve other things :D

As far as i can see Dup2.2 with Dropdupe works very well. I just don't understand what is the point of improving current dropdupe filter........

Edit...... Donald is currently running out of idea on how to improve on his dup filter. So you may want to suggest it on the Avisyth board.

Or you could post it here and i will merge all your idea and submit to him.

HomiE FR
8th July 2003, 12:47
Hi all,

This DropDupe along with Dup 2.20 is really a good team for anime encodes! But I have a little problem with DropDup which doesn't want to drop more than 3 frames in a row, no matter what I choose for maxDroppedFrames (in the example below it's 20, which equals to the parameter maxcopies choosed for Dup).

Below I put a few lines from producer.log which shows the problem:
DD: keep (25245) max: 561 216 328, avg: 27 33 34
DD: DROP (25246) max: 0 0 0, avg: 0 0 0
DD: DROP (25247) max: 0 0 0, avg: 0 0 0
DD: DROP (25248) max: 0 0 0, avg: 0 0 0
DD: keep (25249) max: 0 0 0, avg: 0 0 0
DD: DROP (25250) max: 0 0 0, avg: 0 0 0
DD: DROP (25251) max: 0 0 0, avg: 0 0 0
DD: DROP (25252) max: 0 0 0, avg: 0 0 0
DD: keep (25253) max: 0 0 0, avg: 0 0 0
DD: DROP (25254) max: 0 0 0, avg: 0 0 0
DD: DROP (25255) max: 0 0 0, avg: 0 0 0
DD: DROP (25256) max: 0 0 0, avg: 0 0 0
DD: keep (25257) max: 0 0 0, avg: 0 0 0
DD: DROP (25258) max: 0 0 0, avg: 0 0 0
DD: DROP (25259) max: 0 0 0, avg: 0 0 0
DD: DROP (25260) max: 0 0 0, avg: 0 0 0
DD: keep (25261) max: 0 0 0, avg: 0 0 0
DD: DROP (25262) max: 0 0 0, avg: 0 0 0
DD: DROP (25263) max: 0 0 0, avg: 0 0 0
DD: DROP (25264) max: 0 0 0, avg: 0 0 0
DD: keep (25265) max: 0 0 0, avg: 0 0 0
DD: keep (25266) max: 655 265 341, avg: 36 47 52

And here is the lines of the job used concerning DropDupe:
<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">20</maxDroppedFrames>
<maxAvgSSD type="uint">1</maxAvgSSD>
<maxAreaSSD type="uint">1</maxAreaSSD>
<maxAvgChromaSSD type="uint">1</maxAvgChromaSSD>
<maxAreaChromaSSD type="uint">1</maxAreaChromaSSD>
<earlyExit type="bool">false</earlyExit>
<enableDetailedLogInfo type="bool">true</enableDetailedLogInfo>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>

I'd really like to see this small problem solved (I believe this is just a hard limit that Karl put for some testings when there was a bug with timestamps, but this bug has been solved I think).

Thanks in advance.

karl_lillevold
8th July 2003, 20:38
this is true, I put in a limit of 3 frames, because I noticed that there were some problems with larger numbers, due to too large distances in the timestamp gaps between certain frames. I still have on my list of things to do, to figure out what the max number really is. Sorry about that, but most of the compression efficiency improvement is already there with 3, even though some scenes and videos have longer periods of completely duplicated frames.

HomiE FR
8th July 2003, 20:47
Ok thanks Karl, I think you're right when you say that most of the duplicates are already gone with maxDroppedFrames = 3. I only have to put maxcopies = 3 in Dup so that I don't do too much duplicates that DropDupe can't handle as duplicates. :)
And like you said, the compression efficiency improvement won't get much bigger with maxDroppedFrames = 20, it's just that I like when things are pushed to their limits. :p

karl_lillevold
11th July 2003, 00:15
Originally posted by iwod
Really, i think if anybody wants more improvement, they should suggest it to Donald, where he can improve his Dup2.2,

I don't see why we should we invent the wheel if we alreay have a EXtremely good AVISyth filter. May be all this work about manual choosing which frame not to drop could goto avisyth filter, while the dropdupe filter will keep as a simple if it is the same then drop it approach. Which will save some time for karl to improve other things :D

As far as i can see Dup2.2 with Dropdupe works very well. I just don't understand what is the point of improving current dropdupe filter..
Thanks, I have also realized this, and put any further fine-tuning of the detection algorithm in DropDupe on hold for now. Donald's Dup works quite well, and tuning the thresholds is very easy in VirtualDub.

P.S. There is an additional use for DropDupe, that did not even occur to me until someone said they were using it for this purpose, and this is encoding of screen capture training videos. Except for the mouse pointer, things should then be pretty duplicated for long periods of time. Maybe the SSD is hard to adjust for the mouse pointer though, I guess depending on whether or not you want to include it.

midiguy
13th July 2003, 05:05
Would it be possible to use to tkae the code from dup and implement it into dropdupe (both are opensource)? That way less processing would be involved (you wouldn't need to process it through two different duplication drop filters).

PS: I PMed you Karl. My email's outgoing server is down, although I can receive mail (and did receive your email).

iwod
13th July 2003, 06:27
Originally posted by midiguy
Would it be possible to use to tkae the code from dup and implement it into dropdupe (both are opensource)? That way less processing would be involved (you wouldn't need to process it through two different duplication drop filters).

PS: I PMed you Karl. My email's outgoing server is down, although I can receive mail (and did receive your email).

I don't think this is nessercery as you won't gain much in terms of resources if you combine both scripts.

Even through they are open source, in reality only karl and donald work on their script. By keeping them seperate if you want new features you could always request donald to make it since he is running out of idea how to improve dub. And left Karl to do the simple job of putting the final touch on Dopedrop. This way karl get more free time to improve other part of RV9 :devil:

I don't mean the idea was to shift the work load over to donald. Another reason for that is only because donald is "THE" master in these areas if you visit enough of avisyth forums. Which means he proberly does better coding than karl in this field. You could always try reading his highly technical homepage and his script such as decomb before worshipping him :D

karl_lillevold
13th July 2003, 17:32
Originally posted by midiguy
Would it be possible to use to tkae the code from dup and implement it into dropdupe (both are opensource)? That way less processing would be involved (you wouldn't need to process it through two different duplication drop filters).
Initially this may sound like a good idea, and I have considered it. However, with early exit and low thresholds in DropDupe, there is virtually no overhead in running the two back to back. Also, it's an advantage to preview your settings in VirtualDub, and this is only possible with Dup. Integrating the two would add extra work for me, extra maintenance when Dup changes, etc. If your source file is not already an Avisynth script, the only extra work is to write one to include Dup, like I showed above.

Another problem with integrating the two, is how Dup is released under the GPL. I am not an attorney, but have been told this is not compatible with the open source Helix RPSL. RPSL compatible licenses include the Lesser GPL and BSDI Open Source License.

Originally posted by iwod
donald is "THE" master in these areas if you visit enough of avisyth forums. Which means he proberly does better coding than karl in this field. You could always try reading his highly technical homepage and his script such as decomb before worshipping him
I have no doubt about that, and I am sorry to see him leave this forum (http://forum.doom9.org/showthread.php?s=&threadid=57337).

iwod
14th July 2003, 10:34
Originally posted by karl_lillevold
Initially this may sound like a good idea, and I have considered it. However, with early exit and low thresholds in DropDupe, there is virtually no overhead in running the two back to back. Also, it's an advantage to preview your settings in VirtualDub, and this is only possible with Dup. Integrating the two would add extra work for me, extra maintenance when Dup changes, etc. If your source file is not already an Avisynth script, the only extra work is to write one to include Dup, like I showed above.

Another problem with integrating the two, is how Dup is released under the GPL. I am not an attorney, but have been told this is not compatible with the open source Helix RPSL. RPSL compatible licenses include the Lesser GPL and BSDI Open Source License.


I have no doubt about that, and I am sorry to see him leave this forum (http://forum.doom9.org/showthread.php?s=&threadid=57337).

What the hell!!!!!!!!!!!!!!!! he is leaving!!!!!!!!!!!!!!!!!!!!!!!!!! :eek: :eek: :eek: :eek: :eek:

31 Flavas
8th August 2003, 03:27
hey uhm... i'm want to do a drop dupe encode using Donald Graft's Dup avs filter, but the link doesn't work... Would someone point me to a working link? Thanks.

Wilbert
8th August 2003, 13:55
http://neuron2.net/mine.html

Sirber
9th January 2004, 15:17
The videodupframedropper.dll included with producer 10 is 3 times larger than the one that we can find on Karl's site. Is it the same?

karl_lillevold
9th January 2004, 16:11
UPX

Sirber
9th January 2004, 16:22
:confused:

karl_lillevold
9th January 2004, 16:23
Sirber! : Google :) http://upx.sourceforge.net/

Sirber
9th January 2004, 16:32
ha! that UPX was lost somewhere in my legendary brain :D sorry :)

Gannjunior
26th March 2004, 18:26
Hi,
i'd like to test RV codec,'cause I've seen very interesting results in anime encoding at low and very low bitrate.I've downloaded RV10 producer.I'd like to test the setting "...RV9+EHQ+DropDupe, a new 250 kbps RV9 version with this new filter looks virtually identical to the 350 kbps version...",i've seen at the beginning of this discussion.I've downloaded easy real media producer 1.51 e autoRV10 1.0 beta2.My problem is that I need Audio stream with a bitrate of about 20Kbps,but autorv10 doesn't permit me to go under 44kbps,while media producer 1.51 yes.On the other hand,in autorv10 i can edit the 'audience file' to have the "RV9+EHQ+DropDupe@250kbps setting", introducing the code I've in the first page of discussion,that should permit me to use the videodupframedropper.dll in 'prefilter'...But in this case I have another problem,'cause I don't know the 'language' of audience file: in which point of audience file, produced by autorv10,i have to put the code of prefilter for 'dropdupe' etc...?
Any suggestion?
My objective is compress anime(source dvd,vcd,vhs...) with 250kbps in video stream and about a 20kbps music stereo high respons RA8 in audio stream,with the better result between encoding time and quality.And,if i've understood well,videodupframedropper could be very useful...
T.I.A.

ciao!:)

Sirber
26th March 2004, 19:25
yes, it would. Check RealAnime at in my signature :).

Dark-Cracker
26th March 2004, 19:49
u need add the dropdupe settings like this :

...
<parInputs>
<input xsi:type="avFileInput">
<filename type="string">e:\avis\t03.avi</filename>
<prefilters>
<prefilter xsi:type="videoDupFrameDropper">
<enabled type="bool">true</enabled>
<maxDroppedFrames type="uint">3</maxDroppedFrames>
<maxAvgSSD type="uint">700</maxAvgSSD>
<maxAreaSSD type="uint">6000</maxAreaSSD>
<maxAvgChromaSSD type="uint">200</maxAvgChromaSSD>
<maxAreaChromaSSD type="uint">1500</maxAreaChromaSSD>
<earlyExit type="bool">true</earlyExit>
<enableDetailedLogInfo type="bool">false</enableDetailedLogInfo>
<pluginName type="string">rn-prefilter-dupframedropper</pluginName>
</prefilter>
<!-- -->
</prefilters>
</input>
</parInputs>
...

i think audio at 20 kbps will not give u a correct sound even if u use dual mono but it's only my personnal opinion :)

Ps: out of topic and even if i don't like made personal publicity , next release of autorv10 will add support for dropdupe.

++

Gannjunior
29th March 2004, 13:16
Thanks 2 both you, 4 your answers!now i come in your respective encoders' discussion to make you some specific question.

Dark-Cracker,I've been succeed to do work the dropedrupe.the quality obtained,relationated to the size,is really 'embarassing' !! i'm an estimator of xvid,divx...but at bitrate under 350 kbps and 352x or 320x,can't do anything against r.v. ...


i think audio at 20 kbps will not give u a correct sound even if u use dual mono but it's only my personnal opinion
I obviously agree with you,but I've many animes' series in vcd format that "are made with feet" ('fatte con in piedi'),as we would say here in Italy.It should be enough to know a bit of avisinth language to do decent work even with mpeg1 format.These series are taken from very old tv registration on vhs and you can imagine the audio problems...so I can do not much...for this reason,in these chance (220mb of bad quality for 20 minutes!), i choose audio bitrate so low... ;)
ciao!

Sirber
29th March 2004, 14:10
I did some Futurama once, here's the settings:

300kbps total
32kbps Sterel Music HR
DropDupe @ 1, default
no filtering

Quality was like the original MPEG, with estimated average 17.71% difference ;).

rezard
30th January 2005, 06:12
this prefilter cause sync problem.

in my last encoding, with 65535 'maxDroppedFrames' value, this filter dropped 16870 frames out of original 30833 frames.
when playbacking the resulting file, the first part of source video is skipped. about 1 second.
so, the video is faster than the audio by about 1 second.

I compared the sync in media player classic, becaues seeking in vdubMod via directshowsource() is very unstable.
by the way, by vdubMod I can notice that the resulting file has 30812 frames which is less than 30833 frames of source.

with 3 'maxDroppedFrames' value, the video is faster than source by about 5 frames.
the total number of frames is 30827.

without this filter, the video is still faster than source by about few frames!
the 'encodeAllFrames' tag in job file seem not to work.
the total frames number is 30830.

E-Male
30th January 2005, 06:22
ist the maximum value accepted 3 or 5, with any higher value being ignored?

rezard
30th January 2005, 15:53
higher values than 3 are working.

rezard
30th January 2005, 17:19
by the way, I used Helix DNA producer 10.0.0.545 for last encoding, without changing the 'videodupframedropper.dll'.

Sirber
30th January 2005, 18:53
Originally posted by rezard
in my last encoding, with 65535 'maxDroppedFrames' value, this filter dropped 16870 frames out of original 30833 frames.Common sense say that the filter wasn't designed to skip frames up to 2730,625 seconds...

rezard
31st January 2005, 02:29
consecutive 2730 seconds(about 45minutes:confused: )?

because I set all thresholds of this filter to 1 and max consecutive copies number of dup() is 20, the number of max consecutive drops can hardly exceed 19.

Sirber
31st January 2005, 03:53
65535 frames / 24 FPS = 2730 seconds ;)

Why not just setting 20? Anyway... you know what you are doing :)

rezard
31st January 2005, 05:11
I think I will adjust values of thresholds someday to make this filter drop more consecutive frames than 19.

Dark-Cracker
31st January 2005, 09:24
@karl

does this filter is still usefull with the params "encodeallframe" or the new RC who doesn't produce VFR output.

Bye.

Sirber
31st January 2005, 17:15
encodeallframe is for old RC, which prevent it to drop frames at lower bitrate. The new RC should not drop frames, just get very ugly :)

I think the filter will ask to drop frames even if encodeallframe is set, since encodeallframe is just used in case of low bitrate.

Dark-Cracker
1st February 2005, 13:26
>I think the filter will ask to drop frames even if encodeallframe is set, since encodeallframe is just used in case of low bitrate.

i am not sure.

Sirber
1st February 2005, 13:40
Karl? :cool:

karl_lillevold
2nd February 2005, 00:39
encodeAllFrames makes the codec encode all frames that are passed to it without skipping any due to bitrate considerations. However, pre-filters like dropDupe might still drop frames. So encodeAllFrames does not affect the operation of the dropDupe filter.

Dark-Cracker
3rd February 2005, 16:10
ok, so in fact dropdup is useless for the new RC.

Sirber
3rd February 2005, 17:17
not recommanded :)

karl_lillevold
3rd February 2005, 18:01
Originally posted by Dark-Cracker
ok, so in fact dropdup is useless for the new RC.
It should work just fine. What the new RC needs is that the same number of frames are encoded in each pass, so as long as you don't change any settings that could affect how frames are dropped with the dropdup filter in between the two passes, it should function just fine.

Sirber
3rd February 2005, 19:24
yes, but at same bitrate, better tweak the RC than drop frames...

karl_lillevold
3rd February 2005, 19:29
with duplicated frames in animation, better drop the duplicated duplicated frames, for better coding efficiency.

Sirber
3rd February 2005, 20:56
In my tests, for animations, got better results using RC with 10% low boost than using dropdupe with old RC, similar filesize.

karl_lillevold
3rd February 2005, 21:13
yes, it is true it is hard to get DropDupe to work really well. In many cases, animation looks better when you do code all the frames, just due to the grain/noise differences from frame to frame.

Sirber
3rd February 2005, 21:47
no noise and no grain, MPEG4 wash it all ;)

rezard
4th February 2005, 04:09
but I worry that the 'lowBitrate boost' is likely to make fast motions blurry.
this filter is a must for me because it make output of more quality or less size.
the 'grain/noise differences from frame to frame' is not important at all to me.:)

by the way,
isn't there anyone who noticed that the codec skip (or drop?) the first few or some frames of original video, especially with dropdupe filter?(as I said before)
or is this a known problem?

Sirber
4th February 2005, 13:01
"lowBitrateBoost" prevent the curve to go too low. I had problems with bad motion artefacts in low motion scenes. Also, reducing high by 30% is also good. The best way is always to test :) I'm not using those anymore, I'm on Constant Quality @65% :D

lunaticmoon
7th February 2005, 07:14
Hi. Qustion! (Though I have not tried yet ^^; )

Could this technique neatly display the frame at equal intervals?

Anime's 24fps-ize requires blow:

A---A---A---A---O---O.....
|
v
A---A---A---A---O Drop ...
|
v
A----A----A----A----A..... display the frame at equal intervals

Behavior fo original RV10/9:

A---A---A---A---O---O.....
|
v
A---A---A---A---O Drop ...
|
v
A---A---A---A-------A..... It doesn't look smooth.

Is this problem solved?

Thanks.

Sirber
3rd March 2005, 13:36
Displaying 2 times the same frame or only 1 and keep it will produce the same. Or maybe I get it wrong :(

lunaticmoon
14th March 2005, 05:11
Sorry, I did not accurately understand English of Sirber.

Anyway, when IVTC was accurately done, the display interval is different though it is 24 frames per second on data.
Therefore, you are sure to have the sense of incompatibility at the same level as the display of 24fps part with 30fps. :(

Rumbah
14th March 2005, 14:21
I think what Sirber was trying to say is that it makes no difference if there are 25 equal frames shown in one second or 1 of the equal frames for the whole second.
There should be no difference in smoothness except the frames are not equal but similar.

lunaticmoon
17th March 2005, 01:49
At last, I understood. Thanks, Rumbah. :)

However, there is still a problem. DropDupe also seems not to be the same at all, and to drop the frame that looks alike.
(If a fine adjustment is done, it is not a problem ?)

For instance, the processing of the scene of a gradual scroll (pan or tilt ?) might become a problem.

However, it is extent that the anxious person is anxious. :(

Sirber
17th March 2005, 01:59
pan and small mouth moves are problems I got with DropDupe. Maybe it's set too wide...

Dark-Cracker
17th March 2005, 20:09
@lunaticmoon

>(If a fine adjustment is done, it is not a problem ?)

do you know that dropedup filter have some param to set the detection of duplicate frame ? why don't you try to made some tests by tweaking the values, instead of complaining it can skip some frames ? fell free to share the result of your tests with the people potentialy interested by this filter.

have a good day.

lunaticmoon
20th March 2005, 05:32
@Dark-Cracker

Thanks for your response. :)

I tried it only just a little.

However, it was noticed that it was necessary that I change the parameter of each scene at once.
In a word, after I divide the source, and frontend processes it individually, rmedit should unite them. :(
The process is a solution to which it very takes time equally to 120fps mixture processing.

I will compromise by the technique of the interlace maintenance.

This problem in DropDupe's operating by a static parameter will not be solved. It might be another if becoming adaptive DropDupe.

Sorry for my bad English.

Thanks.