About multi pc video encoding

[Slave01]

I was thinking about a way to enable video encoding on multiple pc. I would like to split the movie in two parts and put them on my 2 pcs. I think this is possible but are there problems about hystorical information during encoding? I ask you if the information during encoding process are statistical independent or i have to store some type of information to trasparently split the file in two, encoding and then join them? However is this possible?
Sorry for raw English.

Greetings

[DeathTheSheep]

I was thinking about a way to enable video encoding on multiple pc. I would like to split the movie in two parts and put them on my 2 pcs.
I do it all the time--it's not too hard.
I think this is possible but are there problems about hystorical information during encoding?
Hystorical? What do you mean? That the video doesn't retain its proper timeline?...
I ask you if the information during encoding process are statistical independent or i have to store some type of information to trasparently split the file in two, encoding and then join them? However is this possible?
OK, to split a video, assuming it's avi+vfw compatible, open it in virtualDub or VirtualDubMod (2 excellent freeware video editing programs). Then, at the very first frame, click the "mark in" button (2nd button from the right at the bottom). Hold down the shift button and press the right arrow key on your keyboard until you reach a spot roughly in the middle. Then, let go of the shift key (chosen frame should be a keyframe). Press the "mark out" button (the right-most button). Now, in the video menu, click on "Direct Stream Copy." Then go to file, "Save as AVI..." name it something, and press save.

Then, press "mark in" again (this should be exactly where you pressed mark out before. If you're not exactly sure, well, do it anyway :-) ). Then manually drag the slider all the way to the right and press "mark out." Then go to file -> "Save as AVI..." and name it whatever you please.

The 2 files you've created are the movie split into 2 seperate, independent, easily joinable files.

ENCODING THE PIECES.
If you decide to encode these 2 parts seperately, make sure you use the exact same encoder settings (encoder, fourcc, framerate, compression, frame dimentions...etc) for both clips. Then, after the encoding is done on both computers, you can join them in virtualDub. Just open the first clip, go to File -> "Append AVI segment...," select the second segment, and click Open. Then go to /Video-> "Direct Stream Copy," then file -> "Save as AVI..." Name it a good name and you're completely done.

Good luck.
Sorry for raw English.
Heh, I'm a lot harder to understand. You're fine. ;-)

[ChronoReverse]

Yeah, you can do that. This does incur a small penalty in quality though as the bits obviously can't be distributed from one segment to the other.

Probably isn't too much of an issue in many cases though. Hmm, I should try it and see.

[akupenguin]

It could be a very large penalty, if you don't do it right. It's quite common for 2 halves of a movie to need significantly different average bitrates. To reduce that problem, you can run both 1st passes at cq, then choose 2nd pass bitrates based on the cq bitrates (which is what 2pass ratecontrol does anyway). But to really eliminate the problem, you'll need to know the details of the ratecontrol algorithm and duplicate its decisions.

[ChronoReverse]

I wonder though, would it be possible to split a clip in half, run the CQ first passes, have the stats files compared and then the 2nd passes adjusted all done automatically.

Seems to me a "simple" way of taking advantage of dual-processor systems. Of course, this is from a non-developer point of view meaning it's probably all crock.

[akupenguin]

Of course it's possible. All you need to do is separate ratecontrol from codec, and expose a few variables that were previously internal. XviD has the separation, it just lacks the API.
There has been some discussion of such an app: ELDER (http://www.funknmary.de/bergdichter/projekte/index.php?page=ELDER).

[Slave01]

Originally posted by ChronoReverse
I wonder though, would it be possible to split a clip in half, run the CQ first passes, have the stats files compared and then the 2nd passes adjusted all done automatically.

Seems to me a "simple" way of taking advantage of dual-processor systems. Of course, this is from a non-developer point of view meaning it's probably all crock.

Thank everybody for the answers but i think that in first pass there are no problems. Right? (because single frames are correlated but only with previous 5-10 frames (i frames....and so on). I could overlap some part of the movie so the stat file will be the same of the one pc encoding. Right? In the second pass in fact i need to know the alghoritm of managing bitrate and some information about it will be appreciated.
Very thanks to everyone


Greetings

[xistan]

What about parallel encoding on multiple PC:s or even PC:s with multiple CPU:s? Is there any video encoder out there capable of running in parallel over a distributed network of PC:s?`

/Chris

[akupenguin]

Originally posted by Slave01
I think that in first pass there are no problems. Right? (because single frames are correlated but only with previous 5-10 frames (i frames....and so on).
Right. To make it perfectly equivalent to a 1 PC 1st pass, you'd have to overlap by enough to catch an I-frame. But you can probably cut it down to an overlap of 10 or so with negligible consequences.

[Slave01]

Please can you tell me where i can find information about stat structure? I tried one year ago but without success. And another question: all these problems should be go away with audio matter...right? Encoding an audio stream is always a without memory process..ok?


Thank you
Greetings

[alipp]

In few days i'll post Netix my distributed network encoding video.
Based on avs2avi, tcpdeliver...

[Slave01]

Cool!!!!

Greetings

[PatchWorKs]

# Easy to use interface to encode and play XviD Mpeg4 files

# Encodes video and audio in one step

# Vidomi Enhanced AVI™ allows you to add chapters and subtitles to your encodes

# Multiple Language Support provides international accessibility

# Distributed Encoding: Encode more in less time using using multiple computers on a network (FAQ (http://www.vidomi.com/docs.php?myfaq=yes&id_cat=27))

# Source range selection to encode only the video you want. Select up to 32 cut points for output as a single file

# Create job lists: up to 256 jobs of up to 64 files per job

# Variable Bit Rate for optimum quality and file size

# YV12 Encoding The fastest possible method of converting MPEG 2 to XviD

# Splits files to fit on CD-Rs and maintain audio synch

# Hyper functions automatically eliminate top and bottom black bars, adjust for aspect ratio, and scale video

# Advanced, automated post-processing (deinterlace, scale, re-sample)

# Improved Integrated player designed specifically for playing XviD files with keyframe and single frame advance and full-screen playback

# Playlist support for over 32,000 files

# And, it's fast!

Official website (http://www.vidomi.com/)

Note: Vidomi is actually deadlocked, anyway someone is "hacking" it...

HDTVtoMPEG2 (http://www.midwinter.com/~bcooley/)
Modified Vidomi Encoder (http://www.alexander-oelmaier.de/)

HaPpY CoDiNg !

[alipp]

Vidomi Encoder use only windows network share, contrary to Netix.
Netix is based on Network protocol and it can be used over internet with adsl or better...

[Selur]

netix sounds nice,..
got any additional infos about it?
(supported codecs, input/output formats,..)

[alipp]

Some infos:
Netix input :
avi, d2v, avs

Netix output :
avi, mkv, ogm

Netix codecs tested with success :
Video : xvid 1.x, x264, divx 5.x
Audio : AAC, mp3, ogg, ac3, ...

I've tested Netix on a LAN (100Mbps), the network's card occupation was
5% for 5 slaves without packets optimizations.

Packet optimization :
Frames can be compressed in huffman, gzip or lzh. Must be tested, for choose the best ratio

Sorry for my poor english, that's the best frenchie sentence ;-)

[Slave01]

Will all this meat be open source? Will it? Pleaseeeee....:D



Greetings

[pernas]

Originally posted by Slave01
I was thinking about a way to enable video encoding on multiple pc. I would like to split the movie in two parts and put them on my 2 pcs.
Perhaps not suitable for everyone but it exist here and uses DVD as source:
http://www.freebsd.org/cgi/url.cgi?ports/multimedia/dvdrip/pkg-descr
http://www.exit1.org/dvdrip/

Regards,
Per

[Slave01]

Originally posted by pernas
Perhaps not suitable for everyone but it exist here and uses DVD as source:
http://www.freebsd.org/cgi/url.cgi?ports/multimedia/dvdrip/pkg-descr
http://www.exit1.org/dvdrip/

Regards,
Per

Very nice! :( But it's only for linux...

Thank you

[CyberGuy]

I have experimented with the H.264/AVC Reference Encoder, http://iphome.hhi.de/suehring/tml, ever since the first codec appeared allowing me to play back H.264 content. Since then, the Nero/Ateme encoder has become my encoder for every day use, but I have always wanted to use the reference encoder to encode special videos to achieve the highest quality and experiment with the latest technology such as the High Profile. As those how have used the reference encoder know, it’s extremely slow and would take a single computer a month to encode one movie. I have several hundred computers available to me, and would like to devise a way to distribute the encoding across thirty to sixty computers which should reduce the time to less than twenty four hours to encode a two hour movie.

Using AviSynth, MEncoder, JM 9.4 H.264/AVC Reference Encoder, and some scripts, I can currently encode segments of a video, but I haven’t started working on the distributed encoding. Although I know how to do this, I was hoping to save time and see if someone else has already devised a way to do this with video files or another application that could be easily tailored to this purpose.

I also was hoping for feedback from other users as to whether there is any interest in using a distributed reference encoder as an alternative to the Nero/Ateme encoder. Also, if there is anyone who is interested in what I have so far, I can post more details, because there is always room for improvement. One area that I’m inexperienced in is the JM 9.4 encoder settings, specifically using High Profile with Transform8x8Mode set to 0 so that it will play with the Nero/Ateme decoder, and yield the best quality.

[thed33p]

Originally posted by CyberGuy
as to whether there is any interest in using a distributed reference encoder

I don't really care for the JM encoder but I would be very interested in a way do to distributed X264 encoding - I have 3 PCs available @ home and another2 @ the office.

That said, I know comparitively little about video encoding (only really what I've read on this forum for the past year or so) and even less about DC so I'll have to wait for someone with actual skills to pave the way :(