helix producer: output files gone after encoding completed?

[crackpot_religion]

have other people experienced this?

sometimes after the encoding is complete, the .rmvb output file is just gone. poof! probably helix producer deletes it? there is no error message, encoding is completed 100%, i've checked the destination is correct, the .rmvb output file is even already created after the encoding begins.

it doesn't happen everytime, just some of the times. it's really frustrating because you can't know for sure.

helix producer plus 9.0.1, win2k, using 2-pass encoding, mpeg-1 vcd input.

i now temporarily switch to using WMV9, but it's so sloowww... :-(

[JJ]

Maybe you've run out of temp space? Producer doesn't handle it very well...

[crackpot_religion]

found out the error after using the command line version (producer.exe).

Warning: Could not write to output file z:\tmp\XXX.rmvb. File saved to C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\temp__RWlc4eT4kG5u64sFUnAaA==.tmp.
Error: RealMedia File Writer Plug-in IGNORE error with UserCode 9: Unable to move "temp__RWlc4eT4kG5u64sFUnAaA==.tmp
...
Info: Total Bitrate = 446 kbps
Info: Encoding successful!
Done Errors: 1 Warnings: 1

so it seems that the temp space and the final destination needs to be on the same partition (drive), probably because producer uses rename() function/call or something instead of copy() or remove().

perhaps producer should not delete the temp file is the move failed.

and the GUI version needs to display this error too.

[crackpot_religion]

Originally posted by crackpot_religion
so it seems that the temp space and the final destination needs to be on the same partition (drive), probably because producer uses rename() function/call or something instead of copy() or remove().


umm, saving to different destination drive works for smaller files, so i'm not really sure what the cause of the problem is. both my C: and Z: drive have lots of free space so i don't think that is the problem.

[karl_lillevold]

@crackpot_religion: I have not heard of this problem before. there should not be any problems with having temp and output on different partitions. I will forward your error message to the producer team, and let you know any feedback I get. If you have time, it would be helpful if you could try one of the later Producer 9.2 Milestones. You can get them from the Helix Community, and it is also included with some of the GUI front-ends, like AutoRV9 and HPG (See RV9 Info (http://forum.doom9.org/showthread.php?s=&threadid=40392)). With these front-ends, producer.exe is located in a sub-directory of the front-end. With 9.2 you also get EHQ, which I never encode without anymore.

[TFC]

Well that arrived me twice :)
And twice my movie was allready in temp dir under its temp name : Ad4e212cfzf5erger45gercd54s54e.tmp (it's an example, just rename that file in movie.rmvb) check if any file in the temp have the size of yours

++
TFC

[crackpot_religion]

Originally posted by TFC
Well that arrived me twice :)
And twice my movie was allready in temp dir under its temp name : Ad4e212cfzf5erger45gercd54s54e.tmp (it's an example, just rename that file in movie.rmvb) check if any file in the temp have the size of yours

++
TFC

in my case, the temp file is always deleted at the end of encode, no matter what. so i can't get the encode result. i also tried hardlinking the temp file (using a utility called HARDLINK.EXE) to avoid the temp file being deleted. but both disappear at the end! strange...

[crackpot_religion]

Just a followup report.

I've tried 9.2M6 for a couple of weeks now and for smaller mpg files (around < 300 MB) no problem encountered. I now always put the input files and the output files on the same drive. My C drive is NTFS, +- 30GB free space. OS is Win2k.

However, I tried a +- 900 MPG1 file today and the disappearing happen again.

============================
Warning: Suppressing message 3167 for 18 hours.
Warning: Could not write to output file C:\tmp\3\020030908-1.rmvb. File saved to
C:\DOCUME~1\ADMINI~1.000\LOCALS~1\Temp\temp_CWyJC64YqEqMl51kgnDcUQ==.tmp.
Error: RealMedia File Writer Plug-in IGNORE error with UserCode 9: Unable to move "temp_CWyJC64YqEqMl51kgnDcUQ==.tmp
Info: Total | Audio | Video | Avg | Min | Avg |
Min | Pre- |Audience Name
Info: kbps | kbps | kbps | FPS | FPS | QI |
QI | roll |
Info: ad: 444 | 64.1 | 380 | 30.0 | 28.0@00:00:47 | 49 | 25
@00:33:03 | 4 |450k VBR Download
Info: Total Bitrate = 444 kbps
Info: Encoding successful!
Done Errors: 1 Warnings: 205
C:\tmp\3>
============================

The temp file is gone, I can't find it anywhere. There goes several hours of encoding. This is getting annoying.

Perhaps producer could report a more specific reason why it can't move the tmp file and why it erases the temp file anyway? (Sorry, modifying and rebuilding the source program myself is quite beyond me for now).

[karl_lillevold]

I am sorry to hear you still have this problem. It is very unusual, and no one in the Producer team has heard of anything similar. The error message just indicates that the operating system call to create the destination file failed. The tmp file should not have been deleted though, and you could always have used 'dtdrive' to make the tmp file playable.

I am assuming your c:\tmp\3 is writable, and that you do not have a file with the same name as your destination filename in this folder already, that for some reason can not be over-written.

The only advice I can think of is to run producer with an empty file called debug.txt in the producer directory. Make sure to use options '-lc e,d' and producer will now report more detailed error messages. Try on a short file first to make sure you do get the detailed messages [not to the file debug.txt, this is just the 'switch', but to stdout]

You could also defrag your C: drive, and run chkdsk on it, if you have not recently done so.

[crackpot_religion]

Thanks for the suggestions, karl.

Enabling diagnostic messages doesn't help. There doesn't seem to be relevant additional information about the error. I have even tried -lc e,w,i,d.

The bad news is I still can't figure out the cause. It's not the source file, many of the time when I run producer again to the same file it succeeded. It's not the free disk space. It's not filesystem corruption. It's not different drive/dir as I've tried many combinations. It's not strange input filenames as I've tried many combinations. It happens on all of my boxes: my home PC (Athlon XP 1800+, win2k) and my office PC (P4 1,4G, win2k). It happens on 9.1 as well as the latest milestone I tried (9.2M6).

For the producer's developers, I'd like to make a request: make sure the tmp file is not deleted if it cannot be moved/copied to the final destination. I can believe my problem is a rare one, but still this happens and is very annoying. I imagine having to comment about producer to my friends: "RV9 rocks, and producer does a good job of encoding RV9, but beware: sometimes you'll end up with the desired .rm/.rmvb file, and sometimes you'll end up with nothing at all."

Thanks.

[crackpot_religion]

an update.

the problem seems related to current directory location. usually the problem arises if either the input file or the output file is in the current directory (but for smaller files, <20MB usually this is okay). for example:

- i am in C:\, input file is c:\video\file.mpg, output is c:\file.mpg; or

- i am in C:\video, input file is c:\video\file.mpg, output is c:\file.mpg

the problem is fixed if i cd to another directory (not equal to input file's nor output file's directories).

[karl_lillevold]

interesting. I will forward to producer team.

One question: you say the problem is fixed when you cd to another directory. Does this mean, everything works if you cd to another directory, and type the path name fully for both input and output, like this:
producer -i c:\video\file.mpg -o c:\video\file.rmvb

and the problem occurs if your current directory is c:\video, even if you then also fully specify path to input and output?

[karl_lillevold]

The good news is, one of my audio codec colleagues has seen this problem:


>>On one of my machines here, it happens all the time. These could factor into the problem:
>>
>>- large encodes (2hr video+audio -> 700MB files)
>>- very little space on the hard drive(s)
>>- dual-proc machine

The bad news, now he cleaned up his machine and it does not happen anymore :(


> > When the encode did fail with the error mentioned, my hard drives were
> > extremely full (although not necessarily the drive that the encode was
> > targeted at). I believe that producer (or the filewriter) used temporary
> > files that were not on the target drive, but probably in the %TMP% or
> > %TEMP% directory.
>


I think the current plan is to re-fill up the hard-drive and try to reproduce again. This is a problem we would really like to resolve, and one fix has already been made to the output module to at least prevent the deletion of the temp file, but since we can not repro, we can not verify that it actually fixes the problem. Thanks for following up.

[crackpot_religion]

Originally posted by karl_lillevold
interesting. I will forward to producer team.

One question: you say the problem is fixed when you cd to another directory. Does this mean, everything works if you cd to another directory, and type the path name fully for both input and output, like this:
producer -i c:\video\file.mpg -o c:\video\file.rmvb

and the problem occurs if your current directory is c:\video, even if you then also fully specify path to input and output?

yes, the problem occurs less frequently if my sourcedir and outputdir is not the same as currentdir.

but actually, the problem still happens from time to time.

[crackpot_religion]

i think i have made the problem go away completely (so far) by changing TMP and TEMP environtment variables. my default TEMP value is:

C:\DOCUME~1\ADMINI~1.000\LOCALS~1\Temp

and the actual "long file name" of the above is (the C: drive is NTFS):

C:\Documents and Settings\Administrator.KOMODO.000\Local Settings\Temp

now i do this before encoding:

c:\winnt\system32> set TMP=z:\tmp
c:\winnt\system32> set TEMP=z:\tmp
c:\winnt\system32> cd \video
c:\video> c:\producer\producer -am voice -ad "450k VBR Download" -i alias1.mpg -o alias1.rmvb

Z: is a samba-shared mapped network dir. but C: and Z: both have ample free space (>20 GB).

so i'm wild-guessing the problem might have something to do with path length limit? i haven't tried a long new TMP dir nor a super-long MPG files though...