DVD interlaced at an angle

[nudelbubu]

..or at least it looks a bit like that. I recently bought the remastered version of Mr. Bean on DVD. Two of the episodes had parts, that looked rather weird, like interlacing, only at an angle. Here's some example frames, it's especially visible at the glass of wieners:
http://imgur.com/a/lY0yS
It may not look like too much, but the error moves sideways with every frame, which makes it very annoying when upscaled to a FullHD Screen.

Has anybody seen the problem before and how can I get rid of it? Thx in advance.

(Also, am I allowed to upload a short sample clip here? It is copyrighted material after all and I don't want to break the board rules)

[Guest]

Probably you are talking about the bad dot crawl. Look for dot crawl removers, such as CheckMate.

https://www.google.com/search?q=dot+crawl+removal&oq=dot+crawl+removal&aqs=chrome..69i57.3927j0j1&sourceid=chrome&espv=210&es_sm=93&ie=UTF-8#es_sm=93&espv=210&q=dot+crawl+removal+avisynth

You can upload a small sample under fair use. Use DGSplit to cut the VOB itself (say 50 MB), upload to mediafire.com, and post the link here.

[nudelbubu]

After reading up on dot crawl (which was much easier after knowing, what it is called ;)), I'm pretty sure that it is exactely that. Thanks a lot.

I've tried checkmate, but that only made the frames a bit smoother on default settings, it wasn't able to remove the crawl. So I'll have to play around with different setting and plugins to see what can be done, when I have some time. If anyone is willing to give it a try in the mean time, I would be grateful though. Short clip:
https://www.mediafire.com/?zry4wqkch4d6xvp

I use QTGMC as deinterlacer, if that affects anything. Encoding time does not really matter to me in this case, I don't care if it takes 2 hours or 2 days to encode. If it introduces artifacts on non-dot-crawl parts, that would be acceptable and I would be willing to fix those manually, it is still better than having to redraw all those frames in Photoshop.

[Guest]

Bad dot crawl is notoriously difficult to correct. Don't hope for miracles. Hopefully our members will help you find something that at least improves things somewhat. Good luck with it.

I will move this to Avisynth Usage as it will likely attract more attention there.

[Ghitulescu]

Unfortunately, like most other BBC features, Mr Bean was shot on tape, not on film. The crawling dots are part of the taped video (original) and not obtained via digitisation or other post processing. I have the "old" version (it doesn't mention Remastered - whether it's the same or not I can't tell) and, while I am critical about image quality, I was not disturbed too much about these artifacts. I was more disturbed by the lack of dynamics in image, "tipically tape" :)

[2Bdecided]

Mr Bean was ITV.

The BBC would have used their transform PAL decoder for the composite to component conversion, which would have removed the vast majority of the dot crawl.

Cheers,
David.

[Guest]

The BBC would have used their transform PAL decoder for the composite to component conversion, which would have removed the vast majority of the dot crawl. Interesting, David, thanks. Can you provide any further information and/or links on this technology? Maybe it is something that could be emulated (if only partially) in an Avisynth filter.

[nudelbubu]

@Ghitulescu: A friend of mine also has the non-remastered version, so I was able to make a comparison.
original release: http://i.imgur.com/cXylYNW.png
remastered version: http://i.imgur.com/tKgHFgj.png
As you can see, the dot crawl is there, just not as visible, due to the lower image quality probably.

From what you can see on http://www.jim-easterbrook.me.uk/pal/, that BBC decoder does a great job. Would be great to have that as an Avisynth filter, not sure if it is possible on digital video though. I could be completely wrong there, but it may need frequency information from between frames.

[Ghitulescu]

@Ghitulescu: A friend of mine also has the non-remastered version, so I was able to make a comparison.
original release: http://i.imgur.com/cXylYNW.png
remastered version: http://i.imgur.com/tKgHFgj.png
As you can see, the dot crawl is there, just not as visible, due to the lower image quality probably.
Well, what I could see form a still was that the remastered edition employed a "noise reducer"

[2Bdecided]

Two of the episodes had parts, that looked rather weirdIt's strange it's only parts of two episodes. There's, what, 16 episodes + extras across those discs? They were made over 7 years, but most (not all) is video, presumably entire episodes went through the same process. It's very strange that only parts have dot crawl.

Which episode was your clip from?

I've noticed a couple of badly damaged video frames on my disc 1, but not spotted any dot crawl yet (though I was watching for enjoyment, not critiquing the video quality, which looks antique in 2013 anyway).

Cheers,
David.

[nudelbubu]

I just noticed it on the two episodes while watching, as it was particularly visible there. The clip is from 'Do-It-Yourself Mr Bean' (DVD2/Ep10), the other one is 'Back to School Mr Bean' (DVD3/Ep11).
I'm not sure, if there's dot crawl in other episodes. I may simply not have seen it, like it didn't faze Ghitulescu when watching the non-remastered version.

I also noticed some bad frames and tried to fix them in photoshop, could upload the results if you're interested. (Two frames from Ep3 'The Curse of Mr Bean' and 7 in the Special 'The Library')

Apart from that, I still don't know how to get rid of the dot crawl at least to a point, where you don't notice it when watching. (In order to push this discussion btt :p)

[Ghitulescu]

I checked my version.
I played the original disc (from Universal, Germany edition) on a Pioneer DV-646A connected via SCART/FBAS on a Sony Trinitron 21". It's a pure analogue setup (matching its 4:3 DAR, too).
I could not see any dot crawl. Yes, there have been artifacts, mostly from MPEG compression (around edges), nothing serious, anyway. And of course the analogue noise, which seemed filtered in the Remastered edition.

[nudelbubu]

Yeah, I wouldn't have noticed it on the old version either, if I wouldn't have searched for it, even on my pure digital setup (mpc-hc with internal filters on my htpc connected to an 32" LCD over HDMI). In case that wasn't clear, the comparision frames were only to show, that the dot crawl is there, just not as obvious.

[2Bdecided]

I noticed a weird thing in The Trouble with Mr Bean. For two shots (after the electric shaver incident) it drops from 50i to 25p-in-50i. It's the same where this clip appears in the broadcast version of The Best of Mr Bean, so it must be like that on the master. Some mistake in editing probably.

There is rainbowing (we usually call it cross-colour) throughout on moving sharp edges, but nothing noticable on other things when watched in the same way as Ghitulescu. Given what a poorly decoded PAL composite signal looks like, this was already decoded pretty well. the film inserts look comparatively poor, but it's always like that in UK shows.

Pity the folks who "remastered" it couldn't have fixed the bad frames. Don't expect faults like that on relatively recent shows (considering there's BBC colour video from the 1970s that still plays perfectly!).

I don't think I'd have the patience to go through fixing it - I just want to enjoy the comedy, and it spoils it watching it with a "video restoration" mindset. I've spoilt a few CDs for myself by spending hours fixing them, and then not been able to enjoy the music for years after.

Cheers,
David.

[Ghitulescu]

Well, I'll try to capture the corresponding frames from my version.
It appears that in your captures, dot crawl appears at edges of yellow-to-brown surfaces.
It is not noticeable on analogue setups, like mine.
I'll check it next on my pure digital setup (pioneer to pioneer via HDMI).
I do not watch movies on laptops or computers.

[feisty2]

frfun7 is very good for dot crawl removal

[nudelbubu]

Finally had some time to try a few filters. I decided to use the following filter chain. It's far from perfect, but I think I can live with it:
checkmate()
QTGMC().SelectEven()
LRemoveDust()
...or so I thought. This apparently leads to some frames not being in the correct order. Back to testing. *gnargel*

@2Bdecided: I see, what you're getting at, but basically: What has been seen cannot be unseen. I now know it is there and I'll never be able to fully enjoy that scene, if I don't at least make it a little better.

@Ghitulescu: The worst dot crawl was on the edges of red things, the rest got removed by the filters pretty nicely. Also both versions I had here are the german ones, so your version is most likely the same my friend has.
I do not watch movies on laptops or computers.
htpc as in Home Theatre PC, horizontal casing with a remote, replacing DVD/BluRay-Player, SAT Receiver, TiVo, etc., connected to a 32" LCD TV ;)

@feisty2: I tried frfun7, but isn't it just a denoiser? Anyway, putting the values high enough to actually reduce the dot crawl made the rest of the frame real blurry, so I guess I'll pass on that one.

Thanks to all who helped so far.

[2Bdecided]

Why select even? You've just thrown away half of the temporal resolution.

[nudelbubu]

QTGMC doubles the framerate and I want it to stay at 25fps. That's why.
But maybe filtering at 50fps and applying SelectEven at the very end would yield better results, one more thing to try \o/

[TheSkiller]

Well, the frame rate has to be doubled to keep all the temporal information there is.
Remember, Mr. Bean is mostly shot on video, not film, hence you get 50 distinct fields per second (wrapped in 25 interlaced frames per second) which QTGMC turns into 50 frames per second (50p).

[nudelbubu]

I'm well aware, that I loose some of the temporal info that way, but this is supposed to go back to DVD, once I'm ok with the quality. And while I'm not sure, I think most DVD-players don't support 50p MPEG2.
Besides, we're talking about 50i being interpolated to 50p and converted back to 25p, so I don't really loose half the info compared to the original source.

The best would be to keep it interlaced probably, but I guess getting rid of the dot crawl will be even more annoying that way.

[poisondeathray]

Besides, we're talking about 50i being interpolated to 50p and converted back to 25p, so I don't really loose half the info compared to the original source.

The best would be to keep it interlaced probably, but I guess getting rid of the dot crawl will be even more annoying that way.


You really do lose half when going from 50i => 25p . 1/2 the fields are thrown out

The other option, if you need progressive filtering, is to reinterace it . ie. 50i => 50p + filters => 50i for DVD (note, 25i is the same thing as 50i, just different naming convetions; both indicate 50 fields per second interlaced)

[nudelbubu]

Maybe I got it wrong, but doesn't QTGMC basically make a full frame from every top field and every bottom field? In case of a "top field frame" it would then restore the missing (bottom field) lines from the existing (top field) lines _and_ the surrounding bottom fields. Which means at least some of the information from the bottom fields would be added to a "top field frame". Which means, even when discarding the "bottom field frames", some of the bottom field info is still available, in addition to all the top field info. So less than half is lost. Hope that made sense :)
Anyway, I can worry about that, once I found a good way to reduce the dot crawl.

As for the reinterlacing, is there a preferred way of doing that or would I simply use something like: (in case of TFF)
SeparateFields().SelectEvery(4,0,3).Weave()

[poisondeathray]

Maybe I got it wrong, but doesn't QTGMC basically make a full frame from every top field and every bottom field? In case of a "top field frame" it would then restore the missing (bottom field) lines from the existing (top field) lines _and_ the surrounding bottom fields. Which means at least some of the information from the bottom fields would be added to a "top field frame". Which means, even when discarding the "bottom field frames", some of the bottom field info is still available, in addition to all the top field info. So less than half is lost. Hope that made sense :)



Perhaps less than 1/2 of the spatial information is lost because of interpolation and reconstruction , but exactly 1/2 of the temporal information is lost . Motion fluidity is affected

Instead of 50 moments in time being represented per second, only 25 moments in time are represented per second with 25p . Exactly 1/2




As for the reinterlacing, is there a preferred way of doing that or would I simply use something like: (in case of TFF)


SeparateFields().SelectEvery(4,0,3).Weave()





That should work for the re-interlacing from 50p

[2Bdecided]

assumetff().separatefields().selectevery(4,0,3).weave()

Never assume that AVIsynth has remembered the field order.


For DVD, you absolutely need to take the 50p version and re-interlace it to 50i. You should not drop it to 25p, unless you actually want to change the footage from the smooth video look to the stuttery film look (which is a rather dramatic change).

Cheers,
David.

[nudelbubu]

Sorry that I didn't respond any sooner, but the last few weeks were really busy for me. Posting in this thread now almost feels like necromancy :x.
First of all, I obviously didn't get the distinction between spatial and temporal information, so thanks for explaining that.

Apart from that, even after playing with filters and options for quite a while, I wasn't able to produce a result, that made the dot crawl problem any better and kept enough detail. So I'll give up and keep it as it is.

Thank for all the help and bye for now.