TecoGAN Problem: TecoGAN produces vertical stripes

[Cary Knoop]

Converting a progressive SD video from RGB24 PNGs with TecoGAN produces vertical stripes:

For an image comparison showing the problem, check here:
https://user-images.githubusercontent.com/25874670/87980762-ed128b80-ca88-11ea-980c-61c39568a91b.png

Any ideas as to what could have gone wrong?

Installation in Windows 10


python .\main\main.py --cudaID 0 --output_dir png-out/gpu0 --summary_dir log/ --mode inference --input_dir_LR png-in/gpu0 --num_resblock 16 --checkpoint ./main/model/TecoGAN --output_ext png
Using TensorFlow backend.
input shape: [1, 576, 720, 3]
output shape: [1, 2304, 2880, 3]
Finish building the network
Loading weights from ckpt model
Frame evaluation starts!!
Warming up 5
Warming up 4
Warming up 3
Warming up 2
Warming up 1
saving image output_00001
saving image output_00002
saving image output_00003
..... etc

[feisty2]

seems like incorrect chroma upsampling

[Cary Knoop]

seems like incorrect chroma upsampling
I am investigating and it seems that:


QTGMC actually creates the vertical stripes, they are very faint. The interlaced material is more diffused, perhaps the deinterlacing algorithm misinterprets it as interlaced jaggies.
FFmpeg mp4 to PNG RGB24 (-v:q 0) seems to influence the chroma subsampling.


Edited to add, confirmed, there is definitely an error with the PNGs conversion from 4:2:2 YCbCr to RGB24 as a direct comparison between the video frame and the png images shows.

This is the command:
ffmpeg -video.mov -c:v png -pix_fmt rgb24 -q:v 0 png-in\gpu0\%%05d.png

Anyone knows of any known bugs or workarounds?
Note that the video source uses yuv420p10le, perhaps that could be a factor as well.

[Cary Knoop]

Ok, converting to 444 in the Vapoursynth process, capturing in prores 4444 and then converting to RGB24 PNG, finally seems to be doing the job!

There ought to be a law against chroma subsampling! /s