no luck playing mp4 from an xcd

[avih]

ok, i need volunteers to test a new version.
improved features in the new version (test7) of the xcdsrc.ax filter:

1. has cache (currently 64 distinct sectors, each sector is cached on it's own {=~128K}. number of sectors is totally configurable).
i expect the cache to be usefull during the negotiation phases (start of playback, etc) because in these phases, the same sectors are read over and over again (as is the case with 3ivx, but not only).

2. reads more efficiently from the cd (the whole request in one go instead of the whole request sector-by-sector. usually request is for about 32K during actual playback, and 1byte-1K during negotiations phases).
i expect the improved reading mechanism to give a more fluent playback, (if it wasn't so far for u).

i don't know what other effects the new version might have (especially if there are bad sectors).
i didn't test it extensively, so new bugs may appear. adding the cache system was a bit more complicated than i expected, and much new code was written. if you wanna experiment (absolute descretion is a MUST), post on this thread.

if the tests by the volunteers prove that the new version is usable, i'll release it to the public asap. otherwise, i don't want ppl that are not specifically interested in testing using the buggy version (and it WILL spread as the 'latest' version if u let it).

cheers,
avih

[Phobos]

well, you have here a glad volunteer with plenty of material to test on, feel free to send it to my email or contact me as desired

[shitowax]

I can help as well :)

[ChristianHJW]

Link pls :D .... although MKV from XCD works, there is a lot room for improvement, especially for accessing tags and the like. Looking forward to test the new version of the filter avih ! You rock ! :)

[avih]

shitowax, chris: sent.
phobos, gimme your email, i can't find it.


chris, had a delivery failure with your email:
"Remote host said: 550 Error: Message content rejected"
gimme another address, or contact me by icq, etc.

[P0l1m0rph1c]

@avih: if you need anything for testing, please let me know. I am willing to test and give feedback :) I'm looking forward to that new filter.

Oh, btw, my mail is dcoder@alexandria.cc

[avih]

it crashed constantly for for shitowax. i wanna have another debugging session before releasing another private test version.
i'll let u know when it's ready, thx.

avih

[calinb]

Originally posted by avih
i wanna have another debugging session before releasing another private test version. i'll let u know when it's ready, thx.avih Okay, I'd like to test it, when you're ready. I have bunch of XCDs that use a variety of codecs and splitters.

[avih]

ok, version test7b was sent to shitowax, phobos and P0l1m0rph1c.
calinb, PM me your email if u're still interested.

reports here pls.
thx.
avih.

[P0l1m0rph1c]

Hi. I've tested this new version and it seems not to be working properly. :confused:

With AVI and MKV files, it crashes all players, caused by an access violation.

With MP4 files, it seems to work, and it even reads the XCD, but keeps telling "opening..." and doesn't do anything more (I'm using 3ivx's splitter).

I hope this helps.

Edit: btw, i've tried xcd2file and it looks nice, and works just fine. Nice proggie! ;)

[avih]

P0l1m0rph1c:
if xcd2file works well (meanning: the extracted file plays back ok), then the new system is probably working well.

both the filter and xcd2file use exactly the same core reading library (although in a slightly different manner), so i think that on files which xcd2file works correctly, the filter should at least perform as well as the test6b filter.

i didn't change any of the dshow stuff, i just made it read faster from the cd and use cache.

how did these files play with test6b? comparison with test6b is essenital (as i noted on my email) for me. pls also post your OS, and drive used. thx.

thx.

[P0l1m0rph1c]

On my machine (P4 2.0 GHz, 768 MB, Win2000 SP4, ASUS CRW-2410A):

With AVI and MKV files, it crashes all players (didn't try WMP9 here), caused by an access violation.

With MP4 files, it seems to work, and it even reads the XCD, but keeps telling "opening..." and doesn't do anything more (I'm using 3ivx's splitter). And the XCD2FILE extraction makes a perfectly playable mp4 file.

On another machine(P4 2.4 GHz, 256 MB, WinXP SP1, LG GMA-4020B):

With AVI and MKV files, it plays fine with WMP9, but with MPC or WMP6.4 it says "no combination of filters....".

With MP4, same beahavior as above, in all players.

Where i could play the XCD's, the video was just as smooth as always (i don't have much problems with that), but the seeking was way faster.

With Gabest's filter i was able to play all AVI and MKV XCD's, but not any of the MP4 ones, in both PC's. This also applies to the test6b filter.

[Phobos]

well, ill just post my impressions on the format given by avih:

functionality: ds filter still doesnt work, it acts just the same it did on b6, it just keeps "connecting" in wmp9 and does the same in bs player

size of file: 799MB

speed: a bit faster but still slow, it just took like 3 minutes opening until i got an error message (opposed to 10 min in b6), never played

strain: wmp9 and bsplayer both have normal cd use, in b6 wmp9 was slower and bsplayer did hundreds of head seekings in the drive

OS: Windows XP SP1 up to date

drive: LG DVD-ROM\CD-RW 16x\52X GCC 4520G

On what xcd2file matters, i succesfuly converted the same dat file to mp4 reading directly from cd, maybe its just me, but isobuster feels a little faster, but heck its good enough and saves the hassle of installing a program, not to mention its freeware... thnx avih :) The drive had no strain, it read smootly like any other app.

The output file played fine in any player, i just found that filesize was different:

original mp4
Size: 836,133,315 bytes
Size in Disc: 835,100,672 bytes

output from xcd
Size: 836,133,368 bytes
Size in Disc: 835,100,672 bytes

im not sure if the translation of my spanish explorer is okay, but you get the idea :rolleyes: i dont find it anoying, maybe its that i have ntfs compression enabled by default.

I hope we find the cure for xcd mp4s on directshow soon

:D

P.S.: of course i didn't share the file i got ;)

[shitowax]

To make the XCD source filter connect automatically to the 3ivx splitter, you need to add this to your registry:

[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"2"="4, 4, 66747970"

This helps DirectShow to recognize mov/mp4 files. It will of course be done automatically in the new 3ivx version. On my two tests systems, it works nicely with that. Feel free to report problems and congrat to avih ;):

- I tested successfully a medium file (180MB) and a small file (10MB) using both daemon tools and real CD on two different CD readers.
- with the registry addition, it works with both b6 and b7. With b6 it takes more than 1 minute to connect, with b7 about 7 seconds.
- the playback is perfect and maybe even better in b7 ...

hope that helps.

[avih]

thx shitowax, and congrats.

@P0l1m0rph1c, Phobos:
thank u very much for your tests and reports. just to summary the new riff/cdxa filter issues, can u pls confirm or show otherwise for each point (for all of your tests so far):

1. the new (test7b) is working in every scenario that test6b was working for.

2. test7b is sometimes working faster than test6b, but never slower that test6b (start/playback/seek).

3. in cases in which both test6b and test7b fail, they behave differently (7b causes a crash, 6b causes a... ? )

4. xcd2file is working properly always (i guess that's confirmed already).


i need these answers since i can't release test7b unless it's at least as good as test6b. if there's even one scenario in which 7b performs worse than 6b, then i must fix it before releasing.

remember that the thread is about xcd with mp4. it was suggested that the long time interval during start of playback may be the cause. i addressed this issue ONLY!, there are NO dshow improvements in 7b, only better cd usage.

it also seems that 3ivx splitter is working better now with shitowax's registery fix, as now (if i understand correctly) mp4/mov files can also have other file extensions and still be recognized by a dshow player. the improvements with xcd is a derived result (but still a good one).

thx.
avih

[Phobos]

Originally posted by shitowax
To make the XCD source filter connect automatically to the 3ivx splitter, you need to add this to your registry:

[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"2"="4, 4, 66747970"

how exactly do i add that to the registry?

what type? i tried alphanumeric and it didnt work :(

[shitowax]

You can either do it by hand using regedit or try to create an xcd.reg file, put those following lines in it and double-click it in explorer:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"2"="4, 4, 66747970"

[Phobos]

well, i copied the info you posted including the first "windows registry editor 5" line to a txt, renamed to reg, applied the changes and rebooted... still wmp just sits there.... :mad:

[P0l1m0rph1c]

Well, i added those values to registry too, but the result is the same.:confused: It doesn't do anything, no matter how much time i wait.

[calinb]

Originally posted by shitowax
[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"2"="4, 4, 66747970" Shitowax, are you sure that's correct? All the other entries have two commas, like this:

"2"="4, 4, , 66747970"

In fact, would you mind checking the entire key name and value again?

Thanks!

[calinb]

So far, I've only found one thing that's different with test7b -- Nero Showtime will now play a Nero Digital mp4/XCD whereas it just hung with 6b.

Otherwise, it still takes about a minute to load -- just like graphedit or the directshow players with the 3ivx splitter.

I played a Nero Digital mp4/XCD made with QT profile + bframes. HE-AAC 5.1 and CoreAAC 1.0b9 filter. I used both DivX 5.11 and 3ivX D4 4.5.1 decoders. I haven't tried 3ivx source yet but plan to try it soon.

I used a IBM Thinkpad T23 1.2GHz PIII Mobile, 512MB RAM, XP-Pro sp1a. I read directly from the CD and I read .dat files from the HD.

I have bunch of larger, faster machines and many XCDs to try, if avih has any further test cases or suggestions. Right now, not much difference from 6b :( and I'm not sure what else I should try.

Phobos, can you get anything to play by right clicking on the dat file >> open with >> browse >> graphedit? You can google for graphedit and download it. Take the player out of the test and go straight to a graphedit filter graph.

[shitowax]

You're right ... I wonder what happened cause I export the key with regedit :( ?!? Anyway you can try with that:


Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"2"="4, 4, , 66747970"

Originally posted by calinb
Shitowax, are you sure that's correct? All the other entries have two commas, like this:

"2"="4, 4, , 66747970"

In fact, would you mind checking the entire key name and value again?

Thanks!

[avih]

@calinb:
thx, that's a very good report. i guess your computer/cd are fast enough and there's no difference with 6b. i wonder, if shotowax's registry fix will make it work with the 3ivx splitter for you. if it will, i'd appreciate a speed comparision with test6b.

the nero player issue puzzles me, since as i said, it should be 100% the same as test6b, only with faster access to the cd. so the fact that it didn't work with 6b and does work with 7b may imply that it handles dshow connectivity differently. and since i didn't touch that part of the filter, i don't understand how is it possible. one possible solution to this 'mystery' is that with the old filter it's slow enough for the nero player to quit on a timeout, while the new filter reads faster, and the player manages to start the playback before reaching that timeout.

perhaps someone of the nero team could comment on this? (are there any nero ppl around here anyway?)

regarding other test cases, well, just throw at it the mode2form2 cds that u've got ;). i don't urge u to do that, but it will make me feel more comfortable to publicly release the new version.

one implication of the caching system is that it may cache bad sectors as well (it knows they're bad, but i decided to cache them anyway). so i think it may behave differently if there are bad sectors on the cd. comparisions of this kind are very welcome.

thx for a good report.

@phobos:
i'm still waiting for you to comment on my previous post. if you, too, report that it's not worse than 6b, then i'll release 7b to the public.

thx again.
avih

[P0l1m0rph1c]

Originally posted by avih

1. the new (test7b) is working in every scenario that test6b was working for.

2. test7b is sometimes working faster than test6b, but never slower that test6b (start/playback/seek).

3. in cases in which both test6b and test7b fail, they behave differently (7b causes a crash, 6b causes a... ? )

4. xcd2file is working properly always (i guess that's confirmed already).


As for the #1, here's confirmed. The problem with non-mp4 xcd's was a flaw in my system :(
I confirm the #2 too. #3 also.
#4 - xcd2file works in all cases.

However, even with the registry tweaks, mp4 doesn't work in any case here :eek:

anyway, thx for all your work :)

[avih]

@P0l1m0rph1c:
thx (and sorry for forgetting you, i remembered i sent it to 3 ppl, but i forgot i also sent to calinb afterwards.).

ok, that's good. one to go ;)

[calinb]

shitowax, Did you cut and paste the entire key? If so, you don't have a "0," "Source Filter," or a "1" value.

Here's what I have:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"0"="4, 4, , 6d646174"
"Source Filter"="{E436EBB5-524F-11CE-9F53-0020AF0BA770}"
"1"="4, 4, , 6d6f6f76"
"2"="4, 4, , 66747970"

I don't know what the other values do but I'd sure like to speed things up.

avih, I'll throw everything I've got at 7b tonight.

[shitowax]

This is correct. It should work. Btw, I wonder how you managed to play mp4+XCD with Showtime as the NDParser seems to use its own source filter and doesn't seem to be able to open .dat files at all ... Almost sure it uses the 3ivx splitter in this case =).

Originally posted by calinb
shitowax, Did you cut and paste the entire key? If so, you don't have a "0," "Source Filter," or a "1" value.

Here's what I have:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Media Type\{e436eb83-524f-11ce-9f53-0020af0ba770}\{E436EB89-524F-11CE-9F53-0020AF0BA770}]
"0"="4, 4, , 6d646174"
"Source Filter"="{E436EBB5-524F-11CE-9F53-0020AF0BA770}"
"1"="4, 4, , 6d6f6f76"
"2"="4, 4, , 66747970"

I don't know what the other values do but I'd sure like to speed things up.

avih, I'll throw everything I've got at 7b tonight.

[Phobos]

Originally posted by calinb
Phobos, can you get anything to play by right clicking on the dat file >> open with >> browse >> graphedit? You can google for graphedit and download it. Take the player out of the test and go straight to a graphedit filter graph.

Calinb: ok i have graphedit already, thats what i use to mux mp4s lol. Well i tried your suggestiong and after 5 minutes graphedit just pops up with "have you installed necessary filters?" error. Note that i have the registry values added as shitowax said. Would you post a noob proof way to enter the registry settings you have?

Avih: I'd be glad to confirm your statements but im still unable to open any kind of mp4/xcd... :( Here are my hardware specs:

P4 800MHz Bus @ 3GHz
1GB PC3700 DDR RAM
10000 RPM 36GB WD Raptor HD

I guess that my system speed aint a factor here...

any other suggestions?

[calinb]

Originally posted by shitowax
Btw, I wonder how you managed to play mp4+XCD with Showtime as the NDParser seems to use its own source filter and doesn't seem to be able to open .dat files at all ... Almost sure it uses the 3ivx splitter in this case =). Yeah, I was wondering the same thing. I tried Showtime with 7b on a lark. I could hardly believe it when it opened it from the CD drive and played. It must be using the 3ivx splitter but I don't know how to check. Just to be sure that I remembered it NOT working with 6b I uninstalled 7b and went back. I waited forever and nothing. I think I finally killed it with task mgr. because it had hung. 7b loaded the XCD file in about a minute, as with the other players. Weird, huh!

Phobos, sure I'll help with regedit tutoring, but it's like shitowax said--you can make a text file with notepad or whatever. Put the lines in it as he described above, but put the two commas in it, like I discovered and name it xcd.reg. Double click it and that should install it into the registry. You could then go to "start" button>> "run" and type "regedt32. Then navigate down to the key. The hierarchy follows the from the key path and name. Careful 'cause many of the keys are have very similar names and you've got to find the correct one. You shouldn't have to change anything, but you can find the entry for the "2" value, etc. and then just close regedt32.

I'll give you some more details later, but I've got to test these XCDs right now (had a long day a work so not much time). All the values except the "2" entry were in the registry when I opened the key with regedt32 to do the entry manually. I guess 3ivx install did the other ones. If your keys are different, maybe it's because I have the licensed version vs. demo 3ivx???...I dunno. I would think they'd be the same.

[calinb]

Originally posted by Phobos
P4 800MHz Bus @ 3GHz
1GB PC3700 DDR RAM
10000 RPM 36GB WD Raptor HD Ahaa! That's it! Your ultra fast system can't keep up with my ol' IBM T23 and 1.3GHz PIII :) It must be hyperthreading its little GHz's out.

avih, I've played mkv, mp4, and ogm containers from XCDs and they all work fine. The streams were combinations of DIV5, Nero Digital w/ HE-AAC, mp3, and DTS and they were played with the 3ivx splitter, DivX decoder, 3ivx audio and video decoders, CoreAAC.9b, MPC built-in filters, and gabest/intervideo DTS. I also tried a K7S5A/1.2GHz Duron machine with Win2K spk6a. The mp4 XCDs loaded even faster than on the T23 (the Duron box CD is faster.) It loaded a full 795MB mp4 from an XCD in less than 30 sec. Smaller files seem to load proportionitely faster than larger files.

I'll try to run some XCDs with 3ivx, AC3, and vorbis later this weekend and I might have some XVID too, but these 3 container formats all seem to be working. There's also the surprising new functionality of Showtime working with a Nero mp4 XCD too, which I reported above.

Looks like you've got a "keeper" here. Thanks so much!

[avih]

Originally posted by calinb
... It loaded a full 795MB mp4 from an XCD in less than 30 sec. Smaller files seem to load proportionitely faster than larger files....

... Thanks so much! [/B]

my pleasure :) glad it works better.

now, the proportional load time is an interesting issue indeed, and imho, can be addressed.

shitowax, i need some more info pls. regarging the 30 seconds it takes for a ~800M file, it probably means that it has cache misses. a lot of them. now the filter drops only the last used sectors from the cache when there's a need to use a new sector. this means that before starting the actual playback, it reads from sectors that sum up to more than 128K (!!) in a manner that's causing cache misses. could this be possible? coud you try to describe again your file access pattern? where does it read from during the negotiation phase? how much does it read? does it 'scan' the file repeatedly over a large area?

calinb, i'll make u a version with a larger cache (say, 256 sectors =~ 0.5M) and send it to u. i want to release it with the minimal cache size that still working well.

thx so much for the tests. invaluable diagnostic data ;)

avih

[shitowax]

The 3ivx splitter does a lot of small sequential readings. That means in theory, the only cache miss that it does is when it fills the source cache and keep on reading sequentially ... Basically, I wonder if the cache system wouldn't be better with pre-readings i.e when the cache is full and data after the cache is required, you could imagine flushing the oldest portion of the cache and prereading not 1 sector but 10 or 100 ... What do you think ?

[Phobos]

Originally posted by calinb
Phobos, sure I'll help with regedit tutoring, but it's like shitowax said--you can make a text file with notepad or whatever. Put the lines in it as he described above, but put the two commas in it, like I discovered and name it xcd.reg. Double click it and that should install it into the registry. You could then go to "start" button>> "run" and type "regedt32. Then navigate down to the key. The hierarchy follows the from the key path and name. Careful 'cause many of the keys are have very similar names and you've got to find the correct one. You shouldn't have to change anything, but you can find the entry for the "2" value, etc. and then just close regedt32.

well, i now tried the registry settings you posted above and still no luck... just a question, does your drive starts spinning madly at full speed or just a calm like 8x read speed? mine doesnt spin fast at all, i cant hear it but i can see the led working... maybe my supposedly 'b0rked' LG combo drive (as said in the xcd faq) is the responsible.

[avih]

Originally posted by shitowax
The 3ivx splitter does a lot of small sequential readings. That means in theory, the only cache miss that it does is when it fills the source cache and keep on reading sequentially ... Basically, I wonder if the cache system wouldn't be better with pre-readings i.e when the cache is full and data after the cache is required, you could imagine flushing the oldest portion of the cache and prereading not 1 sector but 10 or 100 ... What do you think ?

i intentionally didn't introduce a prefetch mechanism, since the amount of prefech is unknown to the filter. guessing the amount wrongly will result in decreased performance if the reader doesn't need the prefetched data. the only place that knows the amount needed is the function that requests the file (i.e. the splitter). and i don't want the filter to perform better with your splitter, if it's worse with others.

how about requesting on your 1st request (just once) the whole portion of the data that you're going to search? this will cause all of it to get into the cache in one go, and on subsequent requests it will be served much faster. you could also request chunks of 32K if you're not sure how large the cache is (atm it's about 128K, but u shouldn't count on it).

but the best solution would be, imho, that you improve YOUR reading mechanism. i.e. if u know you're going to scan for atoms over 8M, then read these 8M 1st, and then scan them from your memory.

every filesystem is much more efficient when reading/writing large chunks of data, instead of 4 bytes at a time. it just doesn't make sense, and the result is decreased performance.

i.e. try to read/write a 10M file on the HD in chunks of 4bytes. then try to do that in chunks of 1K, then in chunks of 1M. the 4bytes one will be MUCH slower than the others. filesystems are not built to access such small chunks of data.


regarding your assumption that the slowdown is only because of the missing prefetch.. it still sounds strange to me. let's say that you scan a section of 128K. it will access the CD only 64 times, once for each sector read. the rest of the requests will be served from the cache. that means that 64 sectors read are 30s on a fast CD, 1min on slower CD. sounds too much to me.
possible reasons:
- something's wrong with the cache system.
- you read much more than 128K. can u pls comment on this one?


let me know what u think.
cheers
avih

[shitowax]

hehehe ... yeah, of course, we could improve the 3ivx splitter reading scheme and indeed we already did (the current CVS has it's own internal caching system ...). But I still think there is a problem somewhere else:

XCD beta 6 : 30 seconds to open a 180 MB mp4 file.
XCD beta 7 : 7 seconds.
normal CD : less than 1 second.

During the duration of the building of the graph the CPU is at 100% and the CD doesn't do a lot of readings it seems. I also tried to connect manually the source to the splitter filter and the time is the same so it's not another problem of directshow registration ...

[shitowax]

On a .mp4 file containing a 2 hours movie, the splitter has to read about 2 MB before being able to play the first frame. Almost all those bytes are currently read sequentially with fairly small readings.

Originally posted by avih
regarding your assumption that the slowdown is only because of the missing prefetch.. it still sounds strange to me. let's say that you scan a section of 128K. it will access the CD only 64 times, once for each sector read. the rest of the requests will be served from the cache. that means that 64 sectors read are 30s on a fast CD, 1min on slower CD. sounds too much to me.
possible reasons:
- something's wrong with the cache system.
- you read much more than 128K. can u pls comment on this one?

[calinb]

Originally posted by shitowax

XCD beta 6 : 30 seconds to open a 180 MB mp4 file.
XCD beta 7 : 7 seconds.
normal CD : less than 1 second. ogm XCD beta 7 less than 1 second to open a 800MB file.
mkv XCD beta 7 less than 1 second to open a 800MB file.

Well, maybe I'm exaggerating slightly :)

shitowax, No criticism of 3ivx or mp4 intended! I think you guys are doing great work and this issue is new simply because we haven't had good directshow mp4 tools before. You guys are changing that and I very much appreciate that you're interested in making them universally compatible across many platforms and formats. Thanks to you and avih, I think were making good progesss and mp4/XCD is usable now :).

avih, I'll keep testing anything you send my way.

[avih]

Originally posted by shitowax

...During the duration of the building of the graph the CPU is at 100% and the CD doesn't do a lot of readings it seems. I also tried to connect manually the source to the splitter filter and the time is the same so it's not another problem of directshow registration ...

On a .mp4 file containing a 2 hours movie, the splitter has to read about 2 MB before being able to play the first frame. Almost all those bytes are currently read sequentially with fairly small readings.


ok. now we're getting somewhere.

what u describe (100% cpu) makes sense in the light of the new info (2M before the 1st frame). imagine requesting 2M in chunks of 4bytes. that's 500,000 (0.5M!!) requests from the cache!!!! for each of those, the cache searches for the matching entry, copies the memory.. yada yada yada. u get the point. i'm actually surprised it performed that well ;) only 7 seconds :) and that includes accessing the cd 500(!!) times.

what adds to it, is that since it's the 1st version of the cache, it searches the entries in a linear form and not with a hash table.. i somehow imagined that scanning 64 entries won't make such a cpu load for each sector read... but 500,000 requests.. too much :) (although it DOES NOT scan the entries if it's requested with the same entry over and over [sort of a mini cache of the last request within the cache itself].. which is the case most of the time with your splitter. it still copies 2K chunks 500,000 times. which is a total of 1GB copied memory).

obviously, a prefetch mechanism won't help here much (and it already implicitly prefetches 2K while it reads the sector. that's 2K prefetch for 4bytes request! you can't ask much more from a prefetch scheme ;) )

what needs to be done here is improving the reading scheme of your splitter. once that's done, i think that the whole issue will be a none-issue.

and regarding the 1 second from a normal cd issue. i think that the OS treats mode2 form2 files differently than normal files. that could be the reason. also, i intentionally wanted to leave cache and prefetch of files to the OS. it now seems like it doesn't give much of these on a mode2form2 filesystems. so a cache is what i'm currently supplying.. ;)

let me know if YOUR new cache indeed improves the performance.

avih

[calinb]

I tested several CDs today, including vorbis and 3ivx streams, and everything is working better than ever. I'm still using the 7b version you sent to me, avih. In fact, I can't figure out why both Nero Digital and 3ivx 800MB cds are now loading in 3-4 secs today compared to 30 secs yesterday on the same K7S5A/1200 GHz Duron Win2K machine. I haven't changed anything. I rebooted previsouly after making the registry entries so that's not it. I really don't know what happened. Device Mgr. said DMA's been enabled on my CD drive the whole time and I can't figure it out. Weird! It sure works great now.

Hey Phobos, come to think of it, have your checked that DMA is working on your CD drive? Windows has a nasty habit of rolling back to PIO mode after a certain number of read errors (like after reading a scratched CD). It will forever stay in PIO mode--even through reboots and even after attempting to select DMA in the Device Mgr. pulldown box. You either have to edit the registry to disable this "feature" (this approach is covered in the MS knowledge base somewhere and you know how to use regedit now :) ) or you have to use Device Mgr. to delete the IDE/ATAPI controller driver for the primary or secondary controller, as appropirate for your CD drive, and reboot. Plug and play will do its thing and you'll get your DMA mode back after you select it again. About your drive spin-up question, the drive in the Duron box spins like a top. The drive in the laptop is very quiet and I can't even tell if it's running. Of course it's a slower drive too. The Duron box has some kinda 52x read/48x write unit made by "China, Inc." According to the Nero tools, it's really that fast too, but no mfg. is identifiable.

[Phobos]

i confirm not even this works, i tried exactly what you sad and STILL i cant play the friggin mp4 from the xcd... while the original file is playable and an extracted mp4 from xcd2file works well...

now my question is, what in this world is different from calinb's putter to polimorphic's one and mine???

[avih]

phobos, it's known that xcd doesn't work with all drives. perhaps u've got one of those?

calinb, that's great news. thx again for the time you're investing in the new version (i hope you're at least enjoying the movies ;) ).

as i noted on my prev post, the filter can still be improved on 'edgy' cases as with the 3ixv splitter. that's what i'm currently working on. regardless, if the 3ivx splitter is modified to behave better, i'm sure it'll gain on other scenarios as well (other than xcd that is). i hope to release the new version shortly.

thx
avih

ps.
phobos, maybe u can find another cd drive which u can try? i feel for u ;)

[avih]

test7c was sent.
pls report back to this thread.

improvements: should cope better now with tons of tiny requests (4 bytes anyone? ;) ).

avih

[Phobos]

well, sorry to say no use here :( ... ill try with another drive ASAP

[P0l1m0rph1c]

Hmm...now with the new filter (7c), it does connect to 3ivx's splitter, after a couple of seconds. :)
However, i'm still unable to play it after that (I tried ffdshow and xvid's dsf for the task, and CoreAAC for audio).

Now i gotta get some sleep, i'll do more tests tomorrow.

[avih]

thx P0l1m0rph1c. reports on success stories (regardless of mp4) are welcome as well, since the new filter is NOT mp4 specific, and it will be released as an improved GENERAL riff-cdxa filter. so, again, general filter test results are welcome as well.

calinb, shitowax, i'd appreciate some comments. phobos, i just wish you'll be able to use it for any playback ;)

thx guys.
avih

[P0l1m0rph1c]

This is weird. Now, all filters connect correctly (using either MP4, AVI or MKV), but when trying to play, there's simply no action. Either in graphedit or any media player, it just sits there, doing nothing. Maybe this issue is from the dshow part of the filter (?) Dunno. But i see we're making progress here :)

[P0l1m0rph1c]

Oops, sorry, but i guess the problem is from my drive. I made a virtual image of the XCD's i have, and it played just fine with either MKV, AVI or MP4 (MP4 took a few seconds to connect).

So, i guess the problem here is either of the drive or its mode (DMA, PIO).

Keep up the excelent work ;)

[shitowax]

With the same 3ivx splitter and the same file:
XCD beta7b: 9 seconds to open
XCD beta7c: 6 seconds to open
CD : less than 1 second.

With XCD, the CPU is still at 100% and only a few physical access during the opening phase, so physical access don't seem to be the bottleneck here.

[avih]

Originally posted by shitowax
With the same 3ivx splitter and the same file:
XCD beta7b: 9 seconds to open
XCD beta7c: 6 seconds to open
CD : less than 1 second.

With XCD, the CPU is still at 100% and only a few physical access during the opening phase, so physical access don't seem to be the bottleneck here.

hmmm, that was not the improvement i was hoping for... :(
will u pls open debugview and make me a log of the messages during file open? (there should be a L-O-T of them, since it prints a message for every access to the filter, and according to your estimation (and my calculation), there should be about 500,000 accesses to the filter. maybe u could make a log for a very small file? say 10-20M? it should note that it retrieves from the buffer (if indeed it does) for about 500 streight requests before bufferring another sector. reading from the buffer should have NO overhead. exactly copying the number of requested bytes, with no intermediate bufferring. unless i have more bugs of course ;)

i'd greatly appriciate such log report (do zip it, or prefferably, 7zip it).

thx
avih

[calinb]

I'm not seeing much difference with 7c either. I haven't had time to do as much testing as before and I haven't quantified the improvement, but it's not much. Maybe have more time in a couple of days!