Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th June 2017, 05:18   #1  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
Early South Park is driving me crazy

Seriously, what's with all the checkerboard noise? Why does it exist? Why is the image apparently composed of four fields instead of two? THE ABYSS IS GAZING INTO ME AND I AM BECOMING A MONSTER

__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.

Last edited by Katie Boundary; 27th June 2017 at 05:25.
Katie Boundary is offline   Reply With Quote
Old 27th June 2017, 15:45   #2  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,662
Is this from a tape capture? "Composite" analog signal? Is the pattern static or does it crawl when you play the video?
Sharc is offline   Reply With Quote
Old 27th June 2017, 20:27   #3  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
It's from the DVD. This artifact appears in the place of ghosting, in frames that would otherwise be ghosted. In this particular case, the yellow checkerboarding is from the school bus in the next frame...



...and here, you can see red checkerboarding from where the guy's shirt used to be.

I kind of expected that everyone here would already have the South Park DVDs and would have discussed this problem a long time ago.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.

Last edited by Katie Boundary; 27th June 2017 at 20:45.
Katie Boundary is offline   Reply With Quote
Old 28th June 2017, 05:07   #4  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
I'm 95% certain that the problem is dot crawl.

If you are playing this DVD on your TV set, and the DVD player is connected to the TV set via an analog composite cable (the RCA, round yellow connector), and you then captured this image, then this is almost certainly dot crawl. It certainly has the usual dot crawl attributes where the severity of the checkerboard changes depending on the hue and intensity of the color. Here is a simple explanation of dot crawl:

Dot Crawl

On the other hand, if you are playing the DVD on your computer (which is what I suspect is the case), then no such mechanism exists, and this image is, I presume, captured directly from your computer's media player. If that is the case, then the problem is "baked into the DVD."

Is this a commercial DVD, or did this come from some online capture, or from some other source? A lot of "shady" DVDs from China, Thailand, and other countries without any copyright laws are often created in the dumbest ways imaginable, as are many online videos posted on YouTube or on various torrent sites. It is possible that someone had a videotape of this episode, connected their capture equipment via a composite connection, encoded the result, and ended up with this sorry mess.

I had a similar problem and started a thread in this forum where I asked for help. Here is that thread:

Remove cross hatch pattern from Kinescope

The solutions posted should work for your video as well. One of them uses convolution kernels, something I know you are familiar with. I got the best results using one of the convolution matrices.

Since the scripts in that thread are nothing more than customized blurring functions, the fact that your video still retains its colors (mine had been converted to B&W) shouldn't matter, and the scripts should function just as well for you as they did for me.
johnmeyer is offline   Reply With Quote
Old 28th June 2017, 05:48   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
A quick search at AVS Forum and the web in general does seem to suggest that the early South Park episodes -- even the official DVDs -- have this problem. Wow, that stinks. I guess the fixup route is the only way to go.
johnmeyer is offline   Reply With Quote
Old 28th June 2017, 06:56   #6  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
Quote:
Originally Posted by johnmeyer View Post
It is possible that someone had a videotape of this episode, connected their capture equipment via a composite connection, encoded the result, and ended up with this sorry mess.
I suspect that this is how the official DVDs were created in the first place. Evangelion got the same treatment, but suffered capture static at the bottom of the screen instead of checkerboarding.

The first and second seasons both got re-releases in 2004. I sent Warner (the distributor) an e-mail asking if the reissues cleaned up this noise. No response yet.

EDIT: for future reference, all of my workflows are 100% digital for the simple reason that I do not own any analog equipment other than a Sega Dreamcast, and although I own a few VHS tapes, the last time I watched any of them was back when "Friends" was still on the air, *NSYNC was still together, and invading Iraq seemed like a good idea.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.

Last edited by Katie Boundary; 28th June 2017 at 08:03.
Katie Boundary is offline   Reply With Quote
Old 28th June 2017, 07:44   #7  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,662
DotCrawl was my first thought as well, but can it be THAT bad??
After a second thought I am afraid 'yes', for example when the monkey who captured the video tape cross-wired the Composite signal to the S-Video input of the capturing device, hence bypassing the comb filter which would suppress the dot crawl effect to some degree (depending on the quality of the comb filter). Secondly, I suspect that -- after seeing the damage -- the same monkey applied some strong temporal smoothing which eventually produced the ghosts.....
It can be difficult to fix dot crawl properly. There are some avisynth tools available, some are good for NTSC or PAL only (you have to know how it was captured), and are more or less effective.
I would recommend to try the brute force method first for this ugly case: Downscale to half width and upscale to original width. It will smooth the picture a bit but the loss of details may be acceptable (depending on the true resolution of the original).
Sharc is offline   Reply With Quote
Old 28th June 2017, 08:14   #8  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
Quote:
Originally Posted by Sharc View Post
I would recommend to try the brute force method first for this ugly case: Downscale to half width and upscale to original width. It will smooth the picture a bit but the loss of details may be acceptable (depending on the true resolution of the original).
I tried something similar involving separatefields() and blur(1.0), and ended up with scanline-like artifacts similar to the ones in Babylon 5. It was still an improvement.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 28th June 2017, 08:23   #9  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
Quote:
Originally Posted by Sharc View Post
DotCrawl was my first thought as well, but can it be THAT bad??
After a second thought I am afraid 'yes', for example when the monkey who captured the video tape cross-wired the Composite signal to the S-Video input of the capturing device, hence bypassing the comb filter which would suppress the dot crawl effect to some degree (depending on the quality of the comb filter). Secondly, I suspect that -- after seeing the damage -- the same monkey applied some strong temporal smoothing which eventually produced the ghosts.....
It can be difficult to fix dot crawl properly. There are some avisynth tools available, some are good for NTSC or PAL only (you have to know how it was captured), and are more or less effective.
I would recommend to try the brute force method first for this ugly case: Downscale to half width and upscale to original width. It will smooth the picture a bit but the loss of details may be acceptable (depending on the true resolution of the original).
I think it may be worse because of something Katie mentioned in her first post, namely that she is seeing four fields instead of two. I don't know exactly what they means, and I don't know if she meant that is true all the time, or whether it comes and goes. However, since it is animation, it likely has a really weird cadence with lots of duplicated fields, but not following a consistent pattern. I have no idea how that would affect the mechanism which produces dot crawl, but I suspect it might produce additional artifacts beyond the usual cross-hatching.

Without an actual clip, I don't know what the ghosting actually looks like and whether it might be reduced or eliminated by clever selection of "good" fields (or frames) which are duplicates of bad fields (or frames). Once the best of each duplicate is kept and all other duplicates eliminated, you could then try to re-do some sort of pulldown in order to keep the speed correct. Since it is animation, the audio can be allowed to go off by a few frames during this process, and as long as the final video is the right length, the audio will be close enough that no one will notice.
johnmeyer is offline   Reply With Quote
Old 28th June 2017, 08:37   #10  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,376
Your might benefit from Spectral adaptive denoising with dfttest. Maybe related, see Remove cross hatch pattern from Kinescope.

Please disregard if these suggestions are insufficiently ghetto
raffriff42 is offline   Reply With Quote
Old 13th July 2017, 22:15   #11  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
Quote:
Originally Posted by raffriff42 View Post
Your might benefit from Spectral adaptive denoising with dfttest. Maybe related, see Remove cross hatch pattern from Kinescope.

Please disregard if these suggestions are insufficiently ghetto
Cool suggestions but I'm mostly interested in what's causing the noise, not how to fix it. Most forms of noise disappear when you scrunch the resolution down to sufficiently ghetto VCD standards
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 14th July 2017, 01:56   #12  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
Quote:
Originally Posted by Katie Boundary View Post
Cool suggestions but I'm mostly interested in what's causing the noise, not how to fix it.
Why do you keep asking when I already gave you the answer (and the correct answer, I might add), weeks ago? To repeat myself: it is caused by dot crawl when the people who transferred the video to DVD used composite video as the source. Whether they did this because they were stupid, lazy, or just couldn't get the original source material, I don't know.

I should also note that this problem with the first few episodes is well-documented on the web, so you can easily find more information with a simple Google search. It took me less than thirty seconds ...

Here is one discussion you may find interesting:

Hideous dot crawl (on early South Park DVDs)

[edit]Here's another:

The trademark South Park dot crawl artefact was present at times on this DVD

and another

South Park Dot Crawl

Last edited by johnmeyer; 14th July 2017 at 02:02. Reason: add another link
johnmeyer is offline   Reply With Quote
Old 14th July 2017, 12:02   #13  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,286
Quote:
Originally Posted by johnmeyer View Post
Why do you keep asking when I already gave you the answer (and the correct answer, I might add), weeks ago?
Aren't you on the ignore list due to your inability to comprehend plain English?
hello_hello is offline   Reply With Quote
Old 14th July 2017, 16:53   #14  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
Quote:
Originally Posted by hello_hello View Post
Aren't you on the ignore list due to your inability to comprehend plain English?
Maybe that is it. We'll see. However, I'm working on my Russian so I can converse in her native language.
johnmeyer is offline   Reply With Quote
Old 15th July 2017, 00:06   #15  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
I was under the impression that "dot crawl" would resemble, you know, little dots crawling around, like I've seen on Buffy the Vampire Slayer. But maybe you're right and "dot crawl" is just some weird euphemism for "checkerboard ghosting", and the artifact that resembles little dots crawling around on the screen is actually called something else.

Quote:
Originally Posted by johnmeyer View Post
I'm working on my Russian so I can converse in her native language.
If you can find me a LEGITIMATE copy of the Пружинки album by Маффин, and not one of the millions of bootlegs, I will personally come to your house and service you sexually.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.

Last edited by Katie Boundary; 15th July 2017 at 00:12.
Katie Boundary is offline   Reply With Quote
Old 15th July 2017, 03:27   #16  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
You are absolutely correct: "dot crawl" makes no sense when you look at a freeze frame and see the checkerboard pattern. However, on live video (and perhaps animation as well), when you get strong transitions between colors, you do in fact see dots or "ants" crawling along those boundaries (pun not intended), depending on the movement. I think that is where the term originated.

This video shows some pathological case of dot crawl:

Dot Crawl Example Video

You said you are not interested at this time in fixing it, but if you do eventually want to do something about it, there are several approaches that work pretty well. I'll post the links if & when there is interest.
johnmeyer is offline   Reply With Quote
Old 15th July 2017, 18:51   #17  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
Interesting.

So what's the name for the artifact that occurs when something that should be moving appears semi-stationary instead, but with weird crawly dots, because the colors are too similar and the source was low-quality to begin with so MPEG's motion-search gets all screwed with? For example, the walls of Sunnydale High School in early Buffy DVDs?
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 16th July 2017, 01:22   #18  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,286
Quote:
Originally Posted by Katie Boundary View Post
Interesting.

So what's the name for the artifact that occurs when something that should be moving appears semi-stationary instead, but with weird crawly dots, because the colors are too similar and the source was low-quality to begin with so MPEG's motion-search gets all screwed with? For example, the walls of Sunnydale High School in early Buffy DVDs?
I'm not sure about that one. Someone else will have to chime in.
johnmeyer is offline   Reply With Quote
Old 19th July 2017, 21:01   #19  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
The 'crawl' motion is caused by two checkerboard patterns alternating, and our eyes tune into this as motion, either up or down. If you look at dot crawl on colour bars, you can amuse yourself by trying to see the dots at the edges of the bars moving up or down.

When dots are stationary, it's called hanging dots or Hannover bars, however that effect is usually seen on PAL material.

There is a special pattern to your sample, it's clear that there's dot crawl only where the fields were different. That would point to two fields mixing somehow, like some kind of field blending due to a rate conversion, a resize, or temporal smoothing. Maybe if I stepped through a sample and thoughtt about it enough, I could guess what kind of likely process lead to it.

And yes, I still call it dot crawl because there's no more precise term for it. It's not inaccurate, it's just a frozen dot crawl.. however the term implies the source of it, which is accurate.

As far as fixing it, brute force would be way too much. I think maybe you could subtract it out perfectly by blending the colour from the last field maybe..
jmac698 is offline   Reply With Quote
Old 20th July 2017, 15:57   #20  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 737
Quote:
Originally Posted by jmac698 View Post
If you look at dot crawl on colour bars, you can amuse yourself by trying to see the dots at the edges of the bars moving up or down.
Is this at all related to the moving checkerboard patterns that sometimes appear at the edges of the black overscan bars on fullscreen material?
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:05.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.