Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th April 2021, 01:19   #1  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,430
Horizontal noise bars - what are they called, and how to remove?

I have been asked to fix this video, but have not yet received the DVD. It is of a 1945 American football game. I don't have access to the original film, but I was told that the original film to video transfer was done professionally. Whatever created these horizontal bars happened after that time.

I have been told that they are static (i.e., they do not crawl up or down the screen as the video is playing).

Question 1: does anyone know what this interference pattern might be called (so I can Google it)?

Question 2: does anyone have some general ideas of what they would use to fix it? If they truly are static, then I might just be able to get away using a mask of some sort and then adjust the exposure and color saturation. If they move, I'll have to devise another approach.

Question 3: Does anyone know what might have caused this?

I ask the last question because the person who purchased this disc may be able to find a source that is closer to the original transfer if we can pinpoint what caused the problem, and by knowing that, figure out how to go around the person involved.

Thanks!

Here's a link to an un-edited clip (MPEG-2 format from DVD):

Noise Bar Clip

.
Attached Images
 

Last edited by johnmeyer; 9th April 2021 at 22:53. Reason: typo; removed external image link (image approved)
johnmeyer is offline   Reply With Quote
Old 8th April 2021, 02:46   #2  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,361
Uhm... I've seen those in Analog BetaCAM as well, but I thought (and I might be completely wrong) they were introduced by the live analog transmission method occurred at the time.
The reason why I'm saying this is that in some very old oxide tapes I've captured there were live feeds which have been recorded at the time and they had this kind of issue.
What got my attention, though, is that you're having this issue on something which isn't supposed to have been aired... ever... so I don't know how they got there.
Could it be that it was somehow transferred from the original 1945 reel to Oxide BetaCAM and then from Oxide Analog BetaCAM to DVD and that step introduced them?
I'm just guessing 'cause back in the days there were no SDI cables and there might have been something introduced by some kind of VTR, but I'm really just speculating...
If someone knows, please let me know, 'cause I'm very interested in finding out why they're there.
As to how I fixed it, I'm afraid I didn't... Well I wasn't "superficial" enough to let it the way it was, but I don't remember what I used... I remember I've got some poor results when I was attempting to fix it "digitally", though.
What I found out at the time was that the Thompson VTRs were much more prone to show this than the Sony ones as there was some analog integrated circuitry that was attempting to fix this with some very good results in some scenes and some pretty poor ones in some others in the official Sony ones. I've never found out the root cause, though, so I'm interested to see if someones knows.

Last edited by FranceBB; 8th April 2021 at 02:48.
FranceBB is offline   Reply With Quote
Old 8th April 2021, 03:38   #3  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 513
Well, while a better alternative appears, and seeing the problem, I remembered the birth of vinverse2, I don't know if making any modification to the Didée script can do something:

Here
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 8th April 2021, 04:23   #4  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,430
Quote:
Originally Posted by GMJCZP View Post
Well, while a better alternative appears, and seeing the problem, I remembered the birth of vinverse2, I don't know if making any modification to the Didée script can do something
Didée posted some amazing scripts, back when he was active, so when the video arrives, I'll definitely look at the script and see if I can get ideas.

Quote:
Originally Posted by FranceBB View Post
Could it be that it was somehow transferred from the original 1945 reel to Oxide BetaCAM and then from Oxide Analog BetaCAM to DVD and that step introduced them?
This is a very real possibility. The person I am working with has ties to NFL Films: he provides them with content, and they send him some of their content in return. He claims that a previous disc from them had similar problems. NFL Films does professional scans of the film and then, back when this was done (analog days) transferred the results to broadcast video formats.
johnmeyer is offline   Reply With Quote
Old 8th April 2021, 07:46   #5  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,813
Quote:
Originally Posted by johnmeyer View Post
Question 1: does anyone know what this interference pattern might be called (so I can Google it)?

Question 3: Does anyone know what might have caused this?
My guessing:
Q1: Mains interference (power frequency + harmonics) during analog signal transfer

Q3: Poor wire screening and/or groundloops between devices

Edit:
https://en.wikipedia.org/wiki/Mains_hum
https://en.wikipedia.org/wiki/Ground_loop_(electricity)

Quote:
In video systems
In analog video, mains hum can be seen as hum bars, (bands of slightly different brightness) scrolling vertically up the screen. Broadcast television frame rates are chosen to match the line frequency, to minimize the disturbance these bars cause to the picture. A hum bar can be caused by a ground loop in cables carrying analog video signals,[5] poor power supply smoothing, or magnetic interference with the cathode ray tube.

Last edited by Sharc; 8th April 2021 at 08:17. Reason: Links added
Sharc is offline   Reply With Quote
Old 8th April 2021, 08:47   #6  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,045
Q2: Chances are that some of vcmohan's plugins might help here. I think he has several which tackle (pun intended) this kind of area.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 8th April 2021, 10:56   #7  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 290
Here mod for vinverse:
https://github.com/realfinder/AVS-St...erse_avsi.avsi
kedautinh12 is offline   Reply With Quote
Old 8th April 2021, 17:22   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,430
Thanks for the link to Didée's vinverse. GMJCZP linked to something Didée wrote that apparently was the predecessor to that function. I'll definitely try it out.

I've been Googling "hum bars" ever since someone else said it might be some manifestation of AC hum. Most of the visuals I've found for that, as well as my own experience trying to remove hum bars with a "humbucker" (balun to interrupt ground loops) is that AC hum bars tend to be much, much broader (wider, top to bottom, with only a few horizontal lines over the entire frame). This looks more like a beat between two frequencies, like you sometimes get with OTA adjacent channel interference
johnmeyer is offline   Reply With Quote
Old 8th April 2021, 22:58   #9  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 513
If this is related to broadcasting interference, and if vinverse2 for some reason does not work, john, review this information, especially pay attention to the contribution of tritical. I uploaded the first two pages of the post because I couldn't see the images today. Remember, only use this Plan B if you are not satisfied with vinverse2, good luck.

Kurish post

Post original today: https://forum.doom9.org/showthread.p...25#post1404025
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 9th April 2021, 00:39   #10  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 984
Do we see a complete frame or a cropped one ?
Somehow I am missing upper/lower border artifacts...

Without a sample no clues for now, just tossing thoughts for association:
79 horizontal bars with square-modulated luminance...40 bars dark, 39 bright.
(Seems to be Y only to me, lowering Y alone would saturate colors)

A shift upwards a bit after half width.
Analog PAL has such behaviour that picture content of first line starts a tiny bit past half width.
But with US footage this should be out of the equation.

Still: any resizing using integers ?
(DLP menu preview had that, leading to 15/16 stuff, missing/doubled lines and so on.

Just a shot in the dark.
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain)
"Data reduction ? Yep, Sir. We're working on that issue. Synce invntoin uf lingöage..."

Last edited by Emulgator; 9th April 2021 at 00:46.
Emulgator is offline   Reply With Quote
Old 9th April 2021, 04:06   #11  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,430
The video was shipped to me today, so I'll have it on Friday or Saturday.
johnmeyer is offline   Reply With Quote
Old 9th April 2021, 22:56   #12  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,430
Here's an unedited clip from the DVD:

Noise Bar Clip

I found that the last two minutes of the DVD is a blank screen (the tape had run out) which included the noise bars. It provides a hint as to what caused the problem. I note that the bars are not horizontal and that they jitter up/down slightly. I've attached a snapshot of the bars.

The 3:2 pulldown must first be removed before doing anything else.

I'll post my results, after I try a few of the suggestions.
Attached Images
 

Last edited by johnmeyer; 9th April 2021 at 23:42.
johnmeyer is offline   Reply With Quote
Old 10th April 2021, 03:50   #13  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,361
Quote:
Originally Posted by johnmeyer View Post
I found that the last two minutes of the DVD is a blank screen (the tape had run out) which included the noise bars.
So it was a tape... My hypothesis turned out to be correct.

Quote:
Originally Posted by johnmeyer View Post
Here's an unedited clip from the DVD
Woah! This very much looks like what I had to deal with with those tapes, so I really bet it was an oxide analog BetaCam, 'cause I haven't seen those things in the Digital BetaCAM.



Quote:
Originally Posted by johnmeyer View Post
I note that the bars are not horizontal and that they jitter up/down slightly.
Yeah, if you were the one who was in charge of capturing it, your best allied would have been this monster: Sony DVW-A500P



one of the boards you see on the left hand side is a logic with a series of analog filter and the one close to it is for analog to digital conversion and it's done in a really good way once the VTR has been referenced with an SD 720x480 30i limited TV Range Colorbars via SDI as it would be able to not only attenuate that issue but also provide good levels.
Unfortunately, you don't have access to the original tape and in the file they sent you the "damage" has already been done 'cause whatever has been used to play it and capture it:

A) Had a really poor analog circuitry and TBC
B) Wasn't referenced

The problem with "A" is common in the sense that I've seen it being "bad" in many professional VTRs like the ones from Thompson or in the J-30 and I consider anything but the official Sony ones to be not good enough. About the "B" that is pretty bad 'cause if a VTR isn't referenced, anything that could go wrong WILL go wrong. Sure, VTRs can generate their own reference signal and adjust automatically according to the input, but what they do is generally a pretty poor scene-by-scene based guess which is anything but good. Personally I've been capturing tapes with the model I posted above (referenced with a PAL SD colorbars signal via SDI 'cause I live in PAL Land) and a BlackMagic Decklink configuration via SDI saving everything in v210 4:2:2 planar lossless as Derek (Blue_MiSfit) suggested (Emulgator also helped me when I had to remote it) and I've now captured around 300 tapes (which have been Remastered and sent to our LTO6 archive). Unfortunately there's nothing you can do about the capturing method and you've received a lossy file as well, so it's gonna be tough to remove those things digitally but I believe you will find a way ('cause the power of Avisynth is endless). Still, it saddens me that you have to do an extra work and spend a lot of time when your job would have been easier if whoever was in charge of capturing it did "the right thing"...


Keep us posted John and best of luck.

Last edited by FranceBB; 10th April 2021 at 03:52.
FranceBB is offline   Reply With Quote
Old 10th April 2021, 14:08   #14  |  Link
vcmohan
Registered User
 
Join Date: Jul 2003
Location: India
Posts: 804
From the image posted in the first post of this thread, I think as already mentioned by some one, the bars are due to power line frequency interference. This can be easily removed using my plugin manyPlus ( also libfftw3f-3.dll) in the path. F1Quiver can be used as long as the direction of noise bars do not change much in the frame. Need to do a turnLeft() prior and later TurnRight.
Alternative is to use F2Quiver. In place of F1Quiver you can also try Fan function. Same process of turnleft, Fan, TurnRight may work.
__________________
mohan
my plugins are now hosted here
vcmohan is offline   Reply With Quote
Old 10th April 2021, 15:23   #15  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,430
Quote:
Originally Posted by FranceBB View Post
So it was a tape... My hypothesis turned out to be correct.
I think I misled you when I said "the tape ran out." I meant to say, "the film ran out." This was, as I said in my OP, a transfer from a 1945 16mm color film. What I just received was a DVD (hence the MPEG-2 file which was losslessly cut from the rip).

However, you may very well be correct because, like with so much video, it has probably been through a tortured path, even though the source was, initially, from a very professional organization.

Of course the fact that pros dealt with this doesn't mean it was done correctly. At some point these bars got added, and whoever let that happen must have been legally blind to have not noticed, or they just didn't care.

Quote:
Originally Posted by vcmohan View Post
... the bars are due to power line frequency interference. This can be easily removed using my plugin manyPlus ( also libfftw3f-3.dll) in the path. F1Quiver can be used as long as the direction of noise bars do not change much in the frame. Need to do a turnLeft() prior and later TurnRight.
Alternative is to use F2Quiver. In place of F1Quiver you can also try Fan function. Same process of turnleft, Fan, TurnRight may work.
Thanks. I will give those a quick try. Defreq worked very well on the middle of the video, but left some residual bars at the top and bottom.
johnmeyer is offline   Reply With Quote
Old 11th April 2021, 00:40   #16  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Metropolitan City of Milan, Italy
Posts: 1,361
Quote:
Originally Posted by johnmeyer View Post
I think I misled you when I said "the tape ran out." I meant to say, "the film ran out."
Ah yeah, got it.

Quote:
Originally Posted by johnmeyer View Post
whoever let that happen must have been legally blind to have not noticed, or they just didn't care.
Yeah... I'm pretty sure they didn't care...
You would be surprised by how many people are in some professional organizations who just don't care and wanna get through the day to just get to lunch time and then home... It's a shame, really...
FranceBB is offline   Reply With Quote
Old 11th April 2021, 07:31   #17  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,735
"Timeliness" is the keyword here, in such companies.
Nobody wants to wait, therefore they use hardware solutions which provide excellent results even without too much operators' involvement.

If one doesn't like the material, he'll let on Auto and goes for lunch.
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 12th April 2021, 06:52   #18  |  Link
vcmohan
Registered User
 
Join Date: Jul 2003
Location: India
Posts: 804
Quote:
Originally Posted by johnmeyer View Post
Thanks. I will give those a quick try. Defreq worked very well on the middle of the video, but left some residual bars at the top and bottom.
Further I tested on your screenshot as the video had only a logo. Screen shot clips amplitudes so that sinusoids will become step functions, which can not be filtered out. These will have almost all harmonics. However I tested with following code using my manyPlus plugin.

Code:
imagereader("c:\images\BaseBall.jpg",end = 100)
turnleft()
crop(0,0,800,1060) 

#F1q=F1QTest(0,100,25,10,30,10,40,100,60,100,70,20,80,20,100,100, custom = true)
#
F1q=F1Quiver(0,100,25,10,30,10,40,100,60,100,70,20,80,20,100,100, custom = true)
F1q=turnRight(F1q)
you can see lot of residual noise. One way to reduce is use
F1q=F1Quiver(0,100,25,0,30,0,40,100,60,100,70,0,80,0,100,100, custom = true, morph = false, uv = true)
But it will introduce too many artifacts. Normally such sharp filters should not be used, but you can try.
__________________
mohan
my plugins are now hosted here
vcmohan is offline   Reply With Quote
Old 12th April 2021, 12:40   #19  |  Link
Frank62
Registered User
 
Join Date: Mar 2017
Location: Germany
Posts: 131
There is a step in the lines in the middle of the picture, about two pixels up seen from left to right. Could it be useful to handle the two halves seperately?
(Just saw it, if this is senseless, just forget... I don't know much of these brillant frequency plugins).
Frank62 is offline   Reply With Quote
Old 12th April 2021, 14:19   #20  |  Link
vcmohan
Registered User
 
Join Date: Jul 2003
Location: India
Posts: 804
Yes. There are several inclined line ups where more than 2 pixels shifts are seen. These will require possibly segmenting the frame into several parts and process separately. In 2D freq display appears to be too complicated to analyze. May be the original video will look better
__________________
mohan
my plugins are now hosted here
vcmohan is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.