Working with Panasonic HDC-SD700 1080p50 footage

[2Bdecided]

I wondered if anyone had tried processing the 1080p50 footage from this camcorder in AVIsynth?

I don't have one, and I don't have an Nvidia graphics card either (i.e. I can't use DGDecNV), but I've downloaded some raw HDC-SD700 footage from the web (from this (http://vimeo.com/groups/native1920x108060pclips/videos) Vimeo group - the content less than 1 week old retains the original raw files for download - if you log in), and was surprised to find that DGAVCIndex 1.0.9 seems to handle it just fine.

I say "surprised", because this is supposed to be "non standard" / "not AVCHD" / "not supported by much at the moment" - but it seems they just upped the bitrate and left everything else as High Profile level 4.2 h.264 - so the decoder in DGAVC seems quite happy with it. (Unlike the BBC HD captures I've tried (1080i60, but with MBAFF) which DGAVC creates extra blocking and artefacts on.)

So, has anyone bought one of these? Is it really this easy to get it into AVIsynth without problems, or have you found issues which my quick test hasn't revealed?

I don't really have the budget for a new camcorder at the moment, but decent quality 1080p50 looks very tempting. No more deinterlacing - just imagine that! ;)

Cheers,
David.

[WorBry]

I'm seriously looking at this camcorder (maybe the NTSC 60p version since I'm now in Canada) for my long held-off quantum jump from DV into HD.....are there any of us left?? Remember our PM exchange about two years back speculating whether 50/60p HD capability would ever become available in consumer range cameras? You suggested it was just a question of time, and right you were.

Been playing around with some native HDC-SD700 and HDC-TM700 footage myself. I'm seeing some edge artifacts and pixelization in motion scenes, and a little aliasing on fine lines, but otherwise the quality is astounding for a 3 x 1/4" CMOS sensor set. I'd like to do some hands-on tests with low light sensitivity/gain noise suppression, but by all accounts, it's quite decent, considering the sensor size.

No problems decoding with DGAVC, like you say, or processing with AVISynth. The big dilemma for me is editing. Is there any commercial editing suite (Windows) out there yet that can handle 1080 50p/60p AVCHD footage natively or at least after conversion to a lossless codec intermediate? Wouldnt really want to resort to tedious cutting/appending with AVISynth/VDub.

[Ghitulescu]

I also think of a Panny camcorder, or a Canon (I hate Sony, but if they allow their camcorders to record on SD/SDHC/SDXC or variations and not on MS I won't exclude Sony).
Concerning the software: as long as the format is .TS/.M2TS it can be processed by most prosumer proggies, like Edius &Co. From what I understand the movie is binary cut into pieces (to fit FAT32 cards). So if after rejoining the clips into a single big one, and passing it through tsmuxer/tsremux or similar, practically most serious editors should be able to cope therewith. From experience, .TS is more compatible than .M2TS. I think Panny is MTS which is a short of M2TS.

[2Bdecided]

@Ghitulescu: There are reports of the 1080p50 and 1080p60 28Mbps footage crashing all kinds of software - though to be fair, most NLEs are quite crash-prone anyway. Next Edius is supposed to handle it properly, but Vegas HD is more my budget! You'd need a stupidly powerful PC to work with this. Since nothing smart renders AVCHD (never mind this currently non-standard variant), there's no harm using a fast editing intermediate - it's just going to eat up disc space if (like me) you don't always finish one project before starting the next one(!).

@WorBry: I haven't found any raw clips of challenging scenes yet (only just started looking, so I've missed most of the interesting clips being available as raw downloads via Vimeo), but I did notice the aliasing. You can kill this and still end up with sharper footage using the usual AVIsynth tricks, but who wants to process all their footage like this? :( I think I'd just put up with it.

You've got to make the jump sometime WorBry! IIRC your existing camera does really nice SD - but I was showing my daughter (just turned four) clips from throughout her life the other day, and it was a relief when we got to the HD stuff (my SD camera was quite naff indoors - but the difference in sharpness, and the crucial nature of getting the deinterlacing right in SD, is quite visible). I'd jump ASAP if I was you, assuming you can edit it (or will be able to do so, by the time you get around to it ;) ). I don't envy you having a mixed 50 + 60 Hz archive (quite a pain to edit together if you ever need to), but if you travel around a lot I guess you have little choice.

Cheers,
David.

[WorBry]

Since nothing smart renders AVCHD (never mind this currently non-standard variant), there's no harm using a fast editing intermediate - it's just going to eat up disc space if (like me) you don't always finish one project before starting the next one(!).

I'd likely take the "lossless intermediate" approach for editing too, with a view to re-encoding to x264. High capacity HDD drives are getting cheaper and I'm not planning on upgrading my PC any time soon.

Yep, I've got lots of unfinished projects. Ah, the demands of a young family, not to mention the old folks back home who always want the latest stuff. Difficult to keep up :o

That's one of the main appeals of this camcorder for me. Best of both worlds. 1080 50p/60p BD for my own viewing on the HDTV. Down-scale and interlace for the obligatory family DVD.

Out of interest, anyone know if WDTV Live can handle 1080 60p or 50p AVCHD video? The specs say technically "H.264 MP@L4.1 and HP@4.1 up to 1920x1080p24, 1920x1080i30, or 1280x720p60 resolution". I read somewhere that PAL video will play, but you cant use the HDMI connection, only composite.

I haven't found any raw clips of challenging scenes yet (only just started looking, so I've missed most of the interesting clips being available as raw downloads via Vimeo)

Not at home right now. I'll see what I can dig up. IIRC, all of the clips were off Vimeo.

....but the difference in sharpness, and the crucial nature of getting the deinterlacing right in SD, is quite visible. .

Too right. As great as TGMC is, it does tend to give a 'rubbery/oil painting' like appearance to alot of my home DV footage, especially outdoor shots with foliage, grass etc....hand held shots on parks/playgrounds with wood-chip ground cover are a nightmare. MCBob often fares better in that respect, but it becomes tedious testing which gives the better outcome, and fiddling endlessly with the parameters. Dont really have the time these days.

Cheers.

[Didée]

Could you provide a typical source sample that leads to rubbery and oil painting? Sounds like the problem is worth investigating.

[WorBry]

Sure. I'll see what I can pull up when I'm at home. Havent shot interlaced DV for a while. Using the pseudo-progressive (25p) frame mode (Procinema) on my DV camcorder more of late.

[2Bdecided]

Too right. As great as TGMC is, it does tend to give a 'rubbery/oil painting' like appearance to alot of my home DV footage, especially outdoor shots with foliage, grass etc....hand held shots on parks/playgrounds with wood-chip ground cover are a nightmare. MCBob often fares better in that respect, but it becomes tedious testing which gives the better outcome, and fiddling endlessly with the parameters.I get that look with TGMC too, but I think it's partly because that's what my camcorder actually captures! i.e. remove the interlacing and the noise, and that's what's left. mcbob leaves a little more noise, and maybe a little more jaggies in some cases, and maybe a little more blur when it's uncertain.

If it looks a little artificial or smooth, the TGMC output can benefit from a little grain.

TBH the best cure is usually to downres it somewhat. Lots of video looks far better at 75% or 66% or even 50%! That's not very helpful advice if you're starting from SD!

As you say, you can make EEDI2 connect fewer things, which helps in some cases but hinders in others. Pebbles on a beach amazed me - EEDI2 defaults lined them up into jagged rocks! (not in TGMC though - just simple EEDI2 2x upscaling I was using a few years back).

The only thing that worries me is that, while most of the people watching won't really notice any improvements (over dumb-bob), they will notice any artefacts. Thankfully I haven't seen any with TGMC yet on my own footage. Not bad going!

Cheers,
David.

[-Vit-]

As great as TGMC is, it does tend to give a 'rubbery/oil painting' like appearance to alot of my home DV footage, especially outdoor shots with foliage, grass etc....
I've noticed this too. The lossless mode of TGMC or QTGMC can help reduce the effect - restoring the original source grain/detail and reducing the "Photoshopped" look of the temporal blur / resharpen. Penalty is potentially more artefacts/shimmer. QTGMC has a "fake" lossless mode which has fewer artefacts, but isn't exactly lossless. Here are some suggestions:
# Real lossless, very accurate to source but a little combing remains
QTGMC( Preset="Slower", Lossless=1 )

# Fake lossless - fewer artefacts, in between basic TGMC and lossless
QTGMC( Preset="Slower", Lossless=3 )

# Fake lossless with wider temporal radius (less shimmer but slower) and a little sharper
QTGMC( Preset="Slower", Lossless=3, LosslessTR=3, Sharpness=1.3 )

# As above, but with some noise bypass/removal (lossless may retain too much source noise)
# I frequently use variants of this line with Lossless either 1 or 3 and different values for NoiseRestore
QTGMC( Preset="Slower", NoiseBypass=1, NoiseRestore=0.5, Lossless=3, LosslessTR=3, Sharpness=1.3 )
Note that these are slow (MT advised) and may need a SetMemoryMax to work.

[Bi11]

I use sharpness=0.2 because of the same 'oil painting' reason since TGMC, which solves the problem for me, but I never bothered to mention it because I thought it was just the default that was too high (not a problem with how the script works).
From my observations, the effect is similar to setting ss_x=1,ss_y=1,soft=0 in LSFMod.
For me, the 'oil painting' effect is prominent when video has low detail resolution and has some form of halos initially, which become amplified after sharping.
Edit: Doesn't QTGMC resharpen the temporal gaussian blurred clip (as said in the docs), so wouldn't that amplify the existing halos and add new halos?

[-Vit-]

Doesn't QTGMC resharpen the temporal gaussian blurred clip (as said in the docs), so wouldn't that amplify the existing halos and add new halos?
QTGMC does exactly the same as TGMC in this regard. But yes, over-sharpening from these scripts is partly the cause of this problem. Yet some measure of resharpening is needed, given that the TGMC deinterlacing method relies on blurring the input. I would expect Sharpness=0.2 to be rather softer than the source.

Still, I tend to agree that the default sharpness of both TGMC and QTGMC is too high. I decided to make the QTGMC defaults match TGMC to make changing over painless.

Interestingly, using lossless output makes the resharpening step somewhat irrelevant. Instead, the sharpness is restored after the temporal blur by adding a "deinterlaced" difference between the "first pass" deinterlace and the source. [I briefly experimented with a second order difference but it was ineffective - although there may still be scope for development in this area]

As an example, this line switches off all resharpening and still gives a decent result:
QTGMC( Preset="Slower", Lossless=3, SMode=0 )

I put a simplistic post-sharpening step in the lossless code so (confusingly) you can still control the sharpness somewhat in this case:
QTGMC( Preset="Slower", Lossless=3, SMode=0, Sharpness=1.3 )

[WorBry]

@David. I'm still away from home and so havent been able to access test clips. Cant seem to find the particular SD/TM700 footage on Vimeo that demonstrated motion artifacts, but there's lots of material there.

As for the 'oil painting' look. I had assumled, like Bi11, that this largely reflected the resolution limitations of my DV camcorder (Panasonic NV-GS400), compounded by it's tendency to produce haloing as a byproduct of the sharpening mechanism. Manually reducing the sharpness dampens this a bit, but not enough. Of course, image enhancing technology has greatly improved since this model came out (2004) and I'm seeing much less haloing on the newer Panasonic AVCHD camcorders. By all accounts, the anti-shake technology (OIS) is greatly improved also.

Cant say I've tried QTGMC as yet. I havent shot interlaced for a while. Maybe there is some scope for further tweaking. I'll dig out some sample footage when I'm home.

Cheers.

[WorBry]

@David. I'm still away from home and so havent been able to access test clips.

Found one raw mts clip I had on my laptop. It's a 1080/50p from an HDC-SD707 (some European version). Cant see it on Vimeo now, so I've put it up on MediaFire:

http://www.mediafire.com/file/yu4n8ncj044oa5x/SD707%201080%2050p.mts

[2Bdecided]

Thanks - that's a nice (quite challenging) sequence.

There are artefacts - but there would be more artefacts from my HV20, and I'd have to deinterlace it, and that would take days to do properly, and it still wouldn't be as sharp.

So all in all, I wish you hadn't shown me that! ;)

Cheers,
David.

[WorBry]

Still away from my home PC.

I've read some reviews that the edge and motion artifacts most noticeable at bitrates up to 13 Mbps (out-of-the box default), tend to disappear at 17 Mbps and above, but DGAVCIndex reveals that clip was shot at top whack 28 Mbps (averaging 23.5 Mbps).


Hmmmm......seen some great deals on ex-stock HV40's, as well as some beautifully shot raw 30p (PsF) footage. That softer HDV look still holds some appeal for me.

[2Bdecided]

If you're happy with 30p instead of 60p, and tape, then I'd go for it.

My HV20 has some things I don't like:
1. the LCD is appalling, but that's supposedly a little better on the HV40
2. the microphone is in the wrong place (facing the sky!) and picks up tape noise (manual audio level helps, but auto would be more convenient)
3. the zoom rocker lever control is poor
...but the image quality can be fantastic (for consumer 50i 1440x1080 MPEG-2).

I've never liked 24p/25p. I don't really like 30p either, and it's a pain in a PAL country. It's useful for PC use though, where you can't be sure people can cope with 50p or 60p properly, and usually 30p stutters slightly less than 25p (depends on the player and monitor refresh rate). If you like the look, it's easy to edit and use in an NTSC country (like 25p in a PAL country).

...I'm sure you know all this. Just like these discussions. It's cheaper to talk than to buy! ;)

Cheers,
David.

[Blue_MiSfit]

Don't use DGAVCIndex anymore.

neuron2 withdrew it for a reason.

Use DirectShowSource or DSS2, or remux to MKV and then use ffms2

Derek

[WorBry]

Didnt know that. I've always used the DGAVCIndex from MeGUI filter set and just assumed that to be the latest.

I've found ffms2. Where do I find DSS2 ?

Edit: OK, got it, uses avss.dll from Haali Media Splitter.

[knutinh]

I have some experience with these cameras and I like them a lot.

A very _different_ experience and image style than video DSLR cams.

-k

[Ghitulescu]

@Ghitulescu: There are reports of the 1080p50 and 1080p60 28Mbps footage crashing all kinds of software - though to be fair, most NLEs are quite crash-prone anyway. Next Edius is supposed to handle it properly, but Vegas HD is more my budget! You'd need a stupidly powerful PC to work with this. Since nothing smart renders AVCHD (never mind this currently non-standard variant), there's no harm using a fast editing intermediate - it's just going to eat up disc space if (like me) you don't always finish one project before starting the next one(!).
As I said I intend to buy one of these AVCHD camcorders, but I wait until some issues get fixed (like rolling shutter, manual controls) and the price would be atractive. However, from what I heard in other fora, some use Nero Vision without any problems (I think this also depends on what one does). I disregarded Nero some years and I still see no reason to come back to it again.

I have some experience with these cameras and I like them a lot.

A very _different_ experience and image style than video DSLR cams.

This thread is not whether one has experience with camcorders, but with helping people. So, what is your experience with these camcorders, some details on processing might be useful.

[knutinh]

So, what is your experience with these camcorders
My experience consists of using the camera for recording video, visually inspecting the results via HDMI on a suitable tv, and playing back the files using VLC.

-k

[WorBry]

...I'm sure you know all this. Just like these discussions. It's cheaper to talk than to buy! ;)


Call me cheap and indecisive (;) :D) but.... as tempting as it is, I think I'll still hold out for now.

For one, the SD700 (which uses only SD cards, no internal flash memory) isn't available here in Canada....(yet?).

Secondly, Panasonic are usually pretty good at responding to review critique and user feedback and I suspect that the New Year will bring further improvements in the image quality, particularly in relation to the motion artifacts.

Sure hope they don't put all their developmental energies into the trendy 3D technology thing. Goodness me, my kids came back from the cinema the other day after watching Toy Story 3 in 3D. Promptly wanted to watch their old Toy Story 2 DVD. Sat there with their free-bee 3D glasses on and complained why it wasn't 3D. Go figure. Time for some "Now when I was I lad, all we had was....."

No doubt, at some point, someone will be asking on this forum how AVISynth can be applied to convert 3D AVCHD video to normal (2D) format.

Edit: Turns out that Panasonic's 'world first consumer 3D camcorder' (HDC-SD7750), is really just an HDC-TS700 (internal HDD version) with a detachable 3D conversion lens slapped on....thank goodness.

http://camcorder-reviews.net/panasonic/panasonic-hdc-sdt750-review/

I cant help but chuckle:

"Panasonic has not forgotten that you need to show off your video recordings. Panasonic has included the HD Writer AE 2.6T editing software, which allows you to easily edit recorded 3D images, and save them onto PCs or Blu-ray/DVD discs. .........The software also enables easy uploading and sharing online without the need for any cumbersome processes, so that even an inexperienced user can post video clips on the web. When uploading 3D images from a PC onto the web, they must first be converted into 2D images"

[Terka]

how to open the tm700 (sd600..) mts file correctly?

[Terka]

no reply?
i want to do avisynth postprocessing on the footage.

[mariner]

Don't use DGAVCIndex anymore.

neuron2 withdrew it for a reason.

Use DirectShowSource or DSS2, or remux to MKV and then use ffms2

Derek

---------

[Terka]

Thank you, i tried. even if the footage is 60p, it contains artefacts similar to interlaced. look at the table front edge.
http://www.ulozto.cz/7082498/00026-zip
the shutter speed was 1/120, recorded at 60p.
im not sure if i do something wrong with camera settings
or the dss2() does something wrong.
or camera is somehow bad...
also the noise and sharpness are not as expected.

[mariner]

Did some testing with ffms2.dll, using vertical stacking to convert to 1920x2160/60. Result seems ok with --crf 17 setting.

Name: 700_v_2D3D.mkv
Size: 125.25MB
Description: 1920x1080/60p 2Dto3D Top/bottom
http://www.sendspace.com/file/hnw7q9

[Ghitulescu]

Thank you, i tried. even if the footage is 60p, it contains artefacts similar to interlaced. look at the table front edge.

the shutter speed was 1/120, recorded at 60p.
im not sure if i do something wrong with camera settings
or the dss2() does something wrong.
or camera is somehow bad...
also the noise and sharpness are not as expected.

I look for a camcorder. I remember reading that even if the camcorder provides you a progressive mode, due to production costs and other issues, the electronics still work interlaced. In the same article I think I've also read that no consumer camcorder is actually pure progressive.

[manolito]

Don't use DGAVCIndex anymore.

neuron2 withdrew it for a reason.

Use DirectShowSource or DSS2, or remux to MKV and then use ffms2

Derek
Could you elaborate a little? I did search this forum and also Neuron's own site, bit I could not find any information about this.

Did he withdraw it because it had serious issues, or was it a licencing problem with MPEGLA, or did he just want to protect his business model (his current AVC / H.264 indexers require that you buy a licence from him)?

In case it was just because he was scared that MPEGLA could come after him, I could not care less. I live in Germany where software patents are hardly enforceable, and the DGAVCIndex binaries are readily available everywhere on the net.

If it was withdrawn because of quality flaws, which of the alternative source filters are recommended? DirectShowSource has a reputation for not being frame accurate, is DSS2 or ffms2 the preferred choice?


Cheers
manolito

[WorBry]

Thank you, i tried. even if the footage is 60p, it contains artefacts similar to interlaced. look at the table front edge.
http://www.ulozto.cz/7082498/00026-zip
the shutter speed was 1/120, recorded at 60p.
im not sure if i do something wrong with camera settings
or the dss2() does something wrong.
or camera is somehow bad...
also the noise and sharpness are not as expected.

That's aliasing on high contrast edges. I've seen it in some other sample clips, but not quite as bad as this. Rather off-putting.

[Terka]

it looks like interlacing artefacts to me. also vertical lines are blurred. despite of 1/120s shutter.

[WorBry]

Actually, having now viewed the clip on a better quality LCD monitor, I'm seeing very little 'edge artifacting'; just a little on the desk drawers. Still think it's aliasing ('line twitter') though. I cant imagine how 'interlace artifacts' could arise.

[WorBry]

Appears that the upcoming 2011 models (HDC-HS/TM/SD900) will be touting an 'Advanced CMOS system' that boasts a 45% improvement in noise suppression when shooting under dim light conditions.

http://panasonic.net/avc/camcorder/hd/900_800_series/feature1.html

Might be worth waiting for if it lives up to it.

BTW - Came across this link to a collection of native 1080 50/60p clips from the 700 series camcorders.

http://hdcam.web-pda.info/

Many of the original Vimeo posts are gone, but the direct links to the raw files are still alive.