Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > NLE - Non Linear Editing

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th October 2007, 10:25   #1  |  Link
MaeWanto
Registered User
 
Join Date: Oct 2004
Posts: 72
DNxHD Encoding with FFmpeg

FFmpeg is now providing Avid DNxHD (SMPTE VC-3) encoding and decoding features:
http://www.fullres.blogspot.com/2007...th-ffmpeg.html

In conjunction with Ingex ( http://ingex.sourceforge.net ), such essence can be wrapped into MXF file format and directly ingested into Avid's NLEs without any rendering process.
MaeWanto is offline   Reply With Quote
Old 13th October 2007, 09:59   #2  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,547
hey, that's pretty cool.

i wonder if that ingex system could work like a virtual vtr? it'd be cool to be able to play out from an Ursa directly into MXF files and save myself some double-handling with tapes.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 13th October 2007, 13:18   #3  |  Link
MaeWanto
Registered User
 
Join Date: Oct 2004
Posts: 72
I guess it could ... I saw a demonstration at IBC 2006 where BBC R&D was showing Ingex' features ( based on a DVS I/O board ). More info on their workflow :
http://www.bbc.co.uk/rd/pubs/whp/whp...les/WHP133.pdf
http://www.bbc.co.uk/rd/pubs/whp/whp...les/WHP141.pdf
http://www.bbc.co.uk/rd/pubs/whp/whp...les/WHP155.pdf
MaeWanto is offline   Reply With Quote
Old 14th October 2007, 22:20   #4  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,337
Very nice. DNXHD is a hell of a format, and the more supported by libavcodec the better!

If only this had been around about a year and a half ago when I was responsible for managing an uncompressed HD workflow for a feature film! AviSynth helped a lot, but this would have been super cool.

~MiSfit
Blue_MiSfit is offline   Reply With Quote
Old 16th October 2007, 14:59   #5  |  Link
CruNcher
Registered User
 
CruNcher's Avatar
 
Join Date: Apr 2002
Location: Germany
Posts: 4,949
Quote:
Very nice. DNXHD is a hell of a format, and the more supported by libavcodec the better!
So it's better then AVC Intra @ the same bitrate ?
im building on such a AVC intermediate workflow (10 Generations) currently as my tests with Cineform and Canopus HQ where destructing both but i didn't test DNXHD yet

PS: My tests where done in the 80 mbit range reaching 51 dB for 1440x1080 30p (tough with compresed Mpeg-2 source)
Results= http://cruncher.mufflastig.com/hdtv/hdv/intermediate/
__________________
all my compares are riddles so please try to decipher them yourselves :)

It is about Time

Join the Revolution NOW before it is to Late !

http://forum.doom9.org/showthread.php?t=168004

Last edited by CruNcher; 16th October 2007 at 15:22.
CruNcher is offline   Reply With Quote
Old 16th October 2007, 20:24   #6  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,337
What about speed? I was under the impression that DNxHD and Apple ProRes422 were both real-time on reasonably fast systems. I can't see HD resolution AVC being real-time.

~Misfit
Blue_MiSfit is offline   Reply With Quote
Old 17th October 2007, 01:43   #7  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Bear in mind though that they're post-production codecs, designed to be IIRC a master transfer. 10-bit is especially useful for studios because it keeps more detail. Speed doesn't necessarily matter, quality does. That said, if it's fast, it'll find its uses.

EDIT: Apple's ProRes codec is likely to be fast on Macs, but slow elsewhere.
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld

Last edited by Inventive Software; 17th October 2007 at 01:52.
Inventive Software is offline   Reply With Quote
Old 18th October 2007, 07:52   #8  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,547
speed matters! it's probably second only to quality (if you're offline editing you'll want speed OVER quality).

the big deal with DNxHD and ProRes is they're heavily parallelised, and give enough compression to carry HD in the same bandwidth as uncompressed SD. meaning the hard disk bottleneck is offloaded onto the (increasingly multicore) CPU.

it means you can use cheaper hard disks to do the same tasks, and the compression is similar (possibly better than) to mastering tape formats.

...unfortunately people haven't quite got into these codecs yet, so i still have to wait forever to copy uncompressed stuff onto 1394 hard disks, tying up edit suite time and meaning lots of double-handling (ie, you can't capture uncompressed reliably over 1394, so you have to capture once then copy).

i'd love to see prores implemented into ffmpeg
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 18th October 2007, 15:28   #9  |  Link
easy2Bcheesy
Moderator
 
easy2Bcheesy's Avatar
 
Join Date: Feb 2002
Posts: 628
I'd like to use this to transcode my 720p/60 CineForm AVIs into Avid-compliant streams, but it looks as though this implementation is for 1080i/p only

Any idea on how fast this encoder is? Is it multithreaded?
easy2Bcheesy is offline   Reply With Quote
Old 18th October 2007, 23:12   #10  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,337
@ Mug Funky

My sentiments EXACTLY!!

I had such a headache capturing uncompressed 1080p24 in 8 bit 4:2:2 to a RAID, and then offloading it to a bunch of firewire drives. It takes WAY too much time. Capturing directly into an editing format like CineForm/ DNxHD / ProRes would have been a huge time (and space) saver.

Unfortunately, we were on Final Cut Pro, which at the time didn't have ProRes, and still doesn't have CineForm. DNxHD was a no-go for some reason..

~MiSfit
Blue_MiSfit is offline   Reply With Quote
Old 19th October 2007, 00:32   #11  |  Link
tateu
Registered User
 
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
As with almost every other codec, I have found (as I'm sure many of you have also found) that the FFMpeg implementation is significantly faster than the one in windows Quicktime.

Using QTInput in an avisynth script opened in VirtualDub, pressing F5, I get about 10.5fps. Using FFMpegSource, I get about 19.5fps.
tateu is offline   Reply With Quote
Old 19th October 2007, 08:53   #12  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,337
Hmm - that's still very slow. Does Avid have some sort of hardware acceleration??
Blue_MiSfit is offline   Reply With Quote
Old 19th October 2007, 19:51   #13  |  Link
tateu
Registered User
 
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
The Avid appears to have acceleration of some kind. I created 3 test quicktime clips: 23.976 1080p at 185Mb, 120Mb and 36Mb. As mentioned above, they do not play in realtime using FFMpegSource or QTInput. Quicktime player cannot play them in realtime either. The 36Mb file was pure white in quicktime and QTInput. FFMpegSource crashed when trying to play it.

I imported all 3 into a 23.976 1080p project on a 2 year old Avid Meridien Uncompressed system. The 185Mb and 120Mb files fast imported to MXF but the 36Mb was converted to MXF 115. I don't have hi-def out of the Avid but the 185Mb and 120Mb clips played back in real time, atleast to my eyes. The 36Mb file was pure white which means the quicktime codec could not decode the file.
tateu is offline   Reply With Quote
Old 20th October 2007, 06:21   #14  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,547
i think the deal with DNxHD and ProRes is they are heavily parallel. from what i've found (which is scant info to say the least), the codecs are extremely similar in philosophy and implementation, though of course apple made a lot more noise about it than Avid did (when they came out with it much earlier).

@ tateu: it'd be interesting to compare the quicktime and ffmpeg implementations of prores, assuming it gets ported eventually. there'll likely be massive gains in quicktime the more cores you have.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 22nd October 2007, 19:26   #15  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,337
By "heavily parallel", do you mean similar, or parallel in the sense of being heavily multithreaded?

I wonder how real-time ProRes or DNxHD 1080p ingest runs real-time??

~Misfit
Blue_MiSfit is offline   Reply With Quote
Old 22nd October 2007, 19:53   #16  |  Link
tateu
Registered User
 
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
Looking at CPU usage (dual Xeon 3.0GHz) on the machine I tested FFMpegSource and QTInput...

FFMpegSource playback through VirtualDub of an 185Mb DNxHD mov file is at about 80% CPU usage (19.5 fps). QTInput is at about 70% (10.5 fps). And I've started fooling around with making an FFMpeg input plugin for VirtualDub...with that, I get about 90% CPU usage and about 29.5 fps. Quicktime player uses exactly 50% (only one CPU). My guess with Quicktime, is that Avid's plugin uses their unoptimized code. I don't think we'll ever see any fast implementations there, for regular playback.

I wonder how the FFMpeg team made their DNxHD decode/encoder. Avid has made the source code available but the license terms are extremely restrictive...you cannot redistribute code that uses their source, you cannot modify it, you cannot optimize it (asm or otherwise), etc., etc.

I would particularly like the ProRes codecs to be ported to windows only because I have had some mac people deliver files to me in that format, lately, and I can't open them. But, as with DNxHD, I'll bet the Quicktime import plugins will be slow.
tateu is offline   Reply With Quote
Old 23rd October 2007, 16:01   #17  |  Link
merbanan
Registered User
 
Join Date: Apr 2006
Posts: 11
The encoder should support all bitrates at 1080i/p 8 bit depth. 720p is under development.

Encoder is multithreaded and should be real time on quad core cpus, quality should be better than what avid media composer produces (observed during the tests by the author).

Email the author if you find any files that crashes the codec. The address can be found here:

http://svn.mplayerhq.hu/ffmpeg/trunk...65&view=markup
merbanan is offline   Reply With Quote
Old 23rd October 2007, 16:40   #18  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,547
that's very good news

and yeah... to port ProRes, i'd say DNxHD is a very good place to start reverse engineering... if i know the imagination that apple usually apply to situations (ie, if we can't buy the codebase outright and shelve it for 2 years before releasing an inferior product, we'll just steal the idea and flood the market with our shoddy version).

...i'm an unashamed apple hater, which makes me a bit of a minority in my line of work. in my opinion there will always be something better out there than apple's version of anything.

i'm definitely going to have to check out ingex some more though - could save a bit of edit-suite time if Avid stuff can be done on a commodity PC rather than a stupid mac with some extra stickers on it.

Quote:
Originally Posted by Blue_Misfit
I wonder how real-time ProRes or DNxHD 1080p ingest runs real-time??
wont happen on anything less than a quad intel crapple. we do ours on an octo (which strangely will run Color below realtime on SD PAL quicktimes, but will crank out 2k DPX in realtime without breaking a sweat... way to go quicktime interface. obviously no multithreading in quicktime for most tasks).
__________________
sucking the life out of your videos since 2004

Last edited by Mug Funky; 23rd October 2007 at 16:45.
Mug Funky is offline   Reply With Quote
Old 24th November 2007, 15:02   #19  |  Link
AFord
Registered User
 
Join Date: Nov 2007
Posts: 7
ffmpeg script sample?

Hello folks,

I'm just getting started with Avisynth + "HV20Pulldown.exe" thanks to footage generated from a Canon HV20- an HDV format camera. I am shooting the camera in its 24p mode. capturing media with HDVSplit.exe and then running the script below to change the 30i m2t files to real 24p AVI. Once that is done, I am rendering the AVI in After Effects with the Avid DNxHD codec and importing the QuickTime into Avid Xpress Pro in a 23.976 Project.

I'd like to avoid the After Effects step if possible. I am hoping I can do it all by using Avisynth and ffmpeg- the problem is I'm the compleat script newbie. I need to know exactly what to add to the script as well as what and where to place any downloadable components. In short- please don't overestimate my uncanny ability to miss the obvious.

Anyone here got time to help with this request? Thanks very much for any replies..

Best Regards,
Abe

Here's the script I am using posted by LordTangent over at www.HV20.com that is working well in conjunction with "HV20pulldown.exe" a program that helps with the Inverse Telecine task:

### Lordtangents HV20 Uber Inverse Telecine Template + Full_Range_YCbCr2RGB
### v0.3 October 10 2007
###
### Requires: DGDecode.dll, TIVTC.dll
### Optional: TTempSmooth.dll and/or TNLMeans.dll for noise reduction
### http://neuron2.net/dgmpgdec/dgmpgdec.html
### http://web.missouri.edu/~kes25c/


### Configuration section

# Scale final output? ( 1 for true 0 for false )
Resize = 1

# # Uncomment your desired final resolution
# pixel aspect 1, 16:9 sizes ...
#==================================================
final_xres = 1920 final_yres = 1080 # "1080p"
#final_xres = 1280 final_yres = 720 # "720p"
#final_xres = 1024 final_yres = 576 # "1k"
#final_xres = 960 final_yres = 540 # "half res"
#final_xres = 864 final_yres = 486 # "486 high"
#final_xres = 720 final_yres = 406 # "720 wide"
#final_xres = 640 final_yres = 360 # "medium"
#final_xres = 320 final_yres = 180 # "Small"
#final_xres = 160 final_yres = 90 # Micro

# Set optional Noise/Grain Reduction level (requires TTempSmooth.dll & TNLMeans.dll )
# int 0-4 "0" means no noise reduction. Runs BEFORE resize for maximum effect.
# 1 is fast temporal smoothing only. Not too blury. Pretty fast.
# 2 is temporal smoothing plus some light 3D (spacial&temporal) means based smoothing. Still not to blury. Not too slow.
# 3 is heavier temporal smoothing plus heavier 3D means based smoothing. Softer. More grain killing power. Slow.
# 4 is even heavier temporal smoothing plus even heavier 3D means based smoothing. Soft. Huge grain killing power. Really Slow.
# 11-13 are special neat visualize modes

ReduceNoise = 0

##### MAIN -- Do not edit unless you really know what you are doing!

v=MPEG2Source("__vid__", upConv=1, idct=3, iCC=true, iPP=true, cpu2="ooxxox")
a=MPASource("__aud__")
audiodub(v,a)
tfm(d2v="__vid__")
tdecimate()

# Apply Noise reductioin...or not. Poormans switch function
(ReduceNoise==0) ? nop() : nop()
(ReduceNoise==1) ? TTempSmoothF(maxr=4, vis_blur=0) : nop()
(ReduceNoise==2) ? TTempSmoothF(maxr=4, vis_blur=0).TNLMeans(Ax=1, Ay=1, Az=1, sse=true) : nop()
(ReduceNoise==3) ? TTempSmoothF(maxr=5, vis_blur=0).TNLMeans(Ax=2, Ay=2, Az=2, sse=true) : nop()
(ReduceNoise==4) ? TTempSmoothF(maxr=6, vis_blur=0).TNLMeans(Ax=2, Ay=2, Az=3, sse=true) : nop()
(ReduceNoise==11) ? TTempSmoothF(maxr=4, vis_blur=1).invert() : nop()
(ReduceNoise==12) ? TTempSmoothF(maxr=4, vis_blur=2).invert() : nop()
(ReduceNoise==13) ? TTempSmoothF(maxr=4, vis_blur=3).invert() : nop()

clip=AssumeFPS(24000,1001)

# Full scale YCbCr --> RGB conversion before resize
ConvertToRGB(clip, matrix="PC.709")

# Resize ... or not ... not really anyway
(Resize==1) ? LanczosResize(final_xres,final_yres) : PointResize(Width(),Height())
AFord is offline   Reply With Quote
Old 2nd December 2007, 20:37   #20  |  Link
bcoudurier
Registered User
 
Join Date: Oct 2007
Posts: 2
720p support

FFmpeg SVN r11134 should now support 720p encoding and decoding.
Extensive tests are welcome.
bcoudurier is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.