Samplerate and Sync.

[zerowalker]

Okay i am recording gameplays, and i get Desync overtime.
It seems to be the cause of some kind of sync clock between the sound and the CPU, i have 2 sound cards, 1 built in, 1 external.

Of both of them i get best results with the one built in.
Though i do get desync overtime, but it´s not delay, it´s the opposite, so in 5 hours of constant gameplay, the audio can be about 500ms earlier then the video.


Now i have "solved" this by manually adding DelayAudio(x.xx), but it´s not a good solution.

So now i started to think, isn´t there a way to make it go 500ms slower over 5 hours?

I don´t know how to calculate that, so i found a setting called AssumeSamplerate (tried playing with fps but it changed the video of course).

And if i use it to reduce the samplerate with 1, it´s pretty much in sync.

from 48000 to 47999.

Don´t know what changed there, but i guess it plays a small bit slower.

The problem is, i got a separate Intro, and i usually use SSRC() to get both of them into 48khz, but SSRC can´t change the samplerate of 47999 to that, so i have to reduce it to 44.1khz.


I normally don´t care about that becuase it´s not audible, but i want to know for learning reasons.

Is there a way to do what i am doing more properly?
Is there anything that´s wrong?

Thanks:)

[StainlessS]

I calculate sample rate for 0.5 delay over 5 hours (18000 secs) to be 0.999972222 of original
which at 48000 is 47998.66667 or about 47999 as you tried.

Suggest maybe

AssumeSamplerate(47999)
ResampleAudio(48000)

[zerowalker]

How did you calculate that?
This kind of calculation is not within my field if i say so;P

And yes i noticed that ResampleAudio works but SSRC doesn´t.
But SSRC is higher quality (atleast i read so).

But wondering, should i resample the to a standard samplerate, or should i resample the separate files to the 47999 samplerate?

[StainlessS]

((18000-0.5) / 18000.0) = 0.999972222

0.999972222 * 48000 = 47998.66667 ie 47999 would play about 0.5 secs longer.

Well, to lengthen the sample, you play 1 sample fewer per second so over 5 hours
the fewer played samples accumulate to about 0.5 and so plays longer (or later @ 5 hours)
but if you want to eg burn on DVD using standard samplerate then you have to resample
to be compatible. SSRC used be be a lot better than Resampleaudio, but not any more
http://forum.doom9.org/showthread.php?t=165321

SSRC is more restrictive in what rates it can convert and will often fail, it is also slower.

Suggest as posted in previous post, or if you want 44.1Khz then 44100 instead of 48000
in ResampleAudio().

You could also use TimeStretch filter, but over 5 hours, it may not work reliably (lack of precision).

[zerowalker]

Ah nice, learned some calculation, will go back many times for that one, i am really not that bright with math;P

Hmm as i understood SSRC was better thanks to the 32bit float, but now ResampleAudio has it aswell, so there isn´t really any difference, and SSRC has much more restrictions?

I am going for playback on PC, but this will go to youtube, which uses 44.1.

But what is the highest quality, or mostly intact to the originaly, resampling to 48khz (adding 1 samplerate), or downsampling to 44.1, or letting it stay at the Assumedsamplerate (i guess this one).
And is there a way to make AssumeSamplerate work with a more precise number (47998.66667)?

And another thing, as i understand, my sound is recording in about 1.3 samplerates to few each second, or something.
What is lost with that except losing some secs over a long time?

Is the audio Really slower?
As it doesn´t skip anywhere, so i can´t think of anything else then that it doesn´t follow the correct samplerate.

[StainlessS]

If you want to change the length then you need to do the AssumeSampleRate otherwise
the length will be wrong. After that, you choose whatever compatible rate you require.
If you dont resample, you could just play that on PC and be happy, so long as you dont want to
give it to YouTube or burn on DVD etc, you take your pick. If you want two different versions,
then convert to 44.1 for YouTube and decide what you want for yourself, 48 or 44.1khz would be
the better choice unless you just dont care. Which is best, well it probably does not matter too
much, as far as I know, adult humans can hear in the (roughly) 20hz to 20khz range (babies
probably a little higher, because they are smaller, and less damaged hearing over time). To
accurately represent 20khz you need double that, ie 40khz, so even CD 44.1 khz is a little
excessive to most needs, and DVD 48khz is well, un-necessary. Where high rates are good to have,
is in the studio where many transformations may be done, the higher sample rate keeps better
precision, but when editing and manipulation is complete, then 44.1 khz is probably quite sufficient.
I'm sure there are many here on D9 who would disagree, but they must have hearing that is out of
human range (I have long suspected that many of them have pointy ears). :)

[zerowalker]

I Will go with 47999 if there isn´t a way to be more precise.
And if youtube doesn´t downsample it, i will do it manually.

But what i don´t understand with the, humans hear 20 to 20k, is the thing with 44.1 and 48.
I understand that there is more precision from Analog to Digital with higher samplerate, and Bit depths and all that. But how come you can´t downsample from 40+ to 20khz, without it sounding like a radio, i thought we only heard till 20khz?

I guess it´s some technical thing and not constatly related to the human hearing samplerate estimation?

[StainlessS]

If you downsample 40khz+ to 20khz, then it can only accurately represent 20khz/2 = 10khz.
Hence the radio like sound. You need twice the sample rate.

[zerowalker]

Oh, why is that?
Why not just name it 10khz instead of 20 then?
Or am i missing something?

[librarian]

... then 44.1 khz is probably quite sufficient...


I agree, for normal adults 40 y.o. limits are often 40 - 14.000 Hz.
On Youtube there are some tests:

http://www.youtube.com/watch?v=R0A72A-c8EU
http://www.youtube.com/watch?v=igGroIcga3g

I ignore their precision (it depends on your sound card too), but for a rough measure may be enough.

@ zerowalker

Nyquist Theorem

[cretindesalpes]

Doing listening tests on with Youtube videos or with any lossy-compressed sound seems to be a bad idea! Audio coders often apply a low-pass filter on the source before encoding to limit the content bandwidth explicitly.

Back to the topic, you should resample your sound instead of just setting its sampling rate to a value extremely close to a standard rate. Some players might round it to 48000 Hz, like video decoders sometimes do with 29.970000 fps they round to the exact ratio 30000/1001 = 29.970029970029... Convert it to 44.1 or 48 kHz, it doesn't matter much, especially if you want to upload it on YT where it will be compressed and damaged anyway.

[zerowalker]

Okay i have uploaded it for now, will see if it´s anything wrong with it.
And for the decoding, it says the correct samplerate (LAV Audio;)).

But still wondering, as StainleSS calculated, the samplerate is in float, but AssumeSamplerate only works in Int.
Is there a way to workaround this?
It´s not a Big issue, but it would be nice to sync it more precise.

[zerowalker]

I think i learned a way to workaround the Int restriction.

If i use this: AssumeFPS(29.99921630094044, sync_audio=true)
for example.

a = video.
b = video_audio.AssumeFPS(29.99921630094044, sync_audio=true)

a.Audiodub(video_audio)


Is there anything wrong with this?
Is it worse/better or identical to the use of Assumesamplerate?

[StainlessS]

Samples may be in float, but samplerate is ALWAYS in int.

Do as already suggested, you are just wasting your time.

EDIT: You are also giving yourself a messed up video frame rate to match a messed up audio rate.

[zerowalker]

Okay, but how am i wasting time, doesn´t the thing i wrote above work accurately?
Or is there something that´s bad with it?

And your suggestion, do you mean stay with a rounded number, or use timestretch?
Cause timestretch works pretty much as what i am trying to do i think?

[StainlessS]

AssumeSamplerate(47999)
ResampleAudio(48000)

See previous EDIT.

Timestretch at 5 hours would probably work but due to float precision and the maths it uses, it would
likely result in your only being able to adjust length in steps of maybe 0.5 seconds, maybe 1 second,
something like that. I've done it before and ended up spending a LOT of time to little good effect.
If you want to use timestretch, suggest cutting into several pieces and adjust them individually.
It might not be 5 hours, may be about 7 hours, but at some point the precision is insufficient
and is just not usable. Try it yourself, but dont ask me to help, I've already been there.

EDIT: If you do try Timestretch, cutting into pieces, then give yourself a few extra seconds on the
ends of EACH piece.

[zerowalker]

Yeah i know that, it worked very well, but it´s not precise with the samplerate thanks to Int.

And as you say that Timestretch is not the best piece of work i haven´t tried it, and don´t think i really want to.

But i want to know as i wrote before, is there a reason not to use this:

AssumeFPS(29.99921630094044, sync_audio=true)
And take that audio and put it in the same clip but with original 30fps.

That way i don´t have to resample, and i get to work with float numbers to get a more precise sync:)

[StainlessS]

sync_audio=true, resamples.

EDIT: Oops, no it doesn't its same as as AssumeSampleRate, but to match the new length
after ASsumeFPS.

If you want to use AssumeSampleRate, then why do it via the back door using AssumeFPS
for a clip that you will throw away anyway.

EDIT: You're going to give your self a headache.

[zerowalker]

It´s because AssumeSampleRate uses Int and can´t be precise.
AssumeFPS uses Float and can be as precise as the calculation is.

It´s not necessary for this stuff to be Perfect no, but it´s better to learn it if i need it in the future:)

[StainlessS]

I'm not altogether sure what it is that you have learnt.

From Avisynth source

int audio_samples_per_second; // 0 means no audio


So no matter how you arrive at your destination, the sample rate is always going to be int.

[IanB]

Yes the audio sample rate property is an integer! So you cannot get the exact change in rate that you need. This does become more relevant the longer a clip becomes.

The difference between 48000 and 48001 becomes 200 milliseconds, considered a just noticeable offset, after about 2.6 hours.

Fortunately the ResampleAudio() filter has an optional new_rate_denominator argument which allows the internal workings of the filter to do extremely precise adjustment of the total audio length.
ResampleAudio(clip, int new_rate_numerator [, int new_rate_denominator])

ResampleAudio performs a high-quality change of audio sample rate. The conversion is skipped if the samplerate is already at the given rate.

When using fractional resampling the output audio samplerate is given by :

int(new_rate_numerator / new_rate_denominator + 0.5)

However the internally the resampling factor used is :

new_rate_numerator / (new_rate_denominator * old_sample_rate)

This causes the audio duration to vary slightly (which is generally what is desired). ... (http://avisynth.org/mediawiki/ResampleAudio)
Using it is a little bit barse-ackwards. You apply a precise correction as a rational pair, then you force the final sample rate back as it should be.
...
AdjustNumerator=36001 # 100ms per hour
AdjustDenominator=36000
Ar = Audiorate()
ResampleAudio(Ar*AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate
It's been a while since I last did this, so I may have the ratio reversed. Add the Info() filter to check the total audio duration and adjust the script accordingly.

[StainlessS]

Ooooo, thats a bit sneaky, one for my docs collection.

EDIT: Perhaps this might be a good candidate for some extra docs.

[zerowalker]

Oh that new_rate_numerator seems to be what i want, i read about it in wiki but didn´t know what it was or what it was used for.
But this seems to be what i may be looking for:)

But still wondering, how does
Assumesamplerate compare to AssumeFPS?

a=AVISource("Z:\w.avi")
b=AVISource("Z:\w.avi").AssumeFPS(29.99921630094044, sync_audio=true)

v1 = a.AudioDub(b)

vs

AdjustNumerator=36001 # 100ms per hour
AdjustDenominator=36000
Ar = Audiorate()
ResampleAudio(Ar*AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate


(they aren´t both 100ms per hour though*)

[IanB]

@StainlessS,

Improved doco, yes always ;)

@zerowalker,

As I said,Yes the audio sample rate property is an integer!so the AudioRate can only ever be the nearest integer value. Compare the calculation used in the core code used to arrive at the value with what we have been calculating in the posts above (note: the numerator and denominator values are intelligently extract from the input float value) :-AssumeFPS::AssumeFPS(PClip _child, unsigned numerator, unsigned denominator, bool sync_audio, IScriptEnvironment* env)
: GenericVideoFilter(_child)
{
if (denominator == 0)
env->ThrowError("AssumeFPS: Denominator cannot be 0 (zero).");

if (sync_audio)
{
__int64 a = __int64(vi.fps_numerator) * denominator;
__int64 b = __int64(vi.fps_denominator) * numerator;
vi.audio_samples_per_second = int((vi.audio_samples_per_second * b + (a>>1)) / a);
}
vi.SetFPS(numerator, denominator);
}AssumeRate::AssumeRate(PClip _clip, int _rate)
: GenericVideoFilter(_clip) {
if (_rate < 0)
_rate = 0;
if (vi.SamplesPerSecond() == 0) // Don't add audio if none is present.
_rate = 0;

vi.audio_samples_per_second = _rate;
}

Assuming the input clip is 30000/1001 NTSC, AssumeFPS() speeds the audio (makes it shorter) by 1.000973850574712681333, so a 48000 input becomes 48047. About 3.5 seconds per hour. Which does not gel with what you were asking for so I have to infer the input clip is not actually 30000/1001.

The script I gave with the 36001/36000 values slows the audio (makes it longer) by 100ms per hour. Which is what you asked for in the 1st post.

:Edit: The 29.99921630094044 resolves to 38249/1275 which is approximately 29.999215686274509803921568627451

[zerowalker]

Can you tell more how to calculate, let´s say 133ms per hour etc?

And with the 29.99921630094044, it was a calculation to play the 30fps clip a small bit slower. but it seems in the end it became 47999, as you said it took the nearest Int i guess.

i as you said want to make it a bit slower, and i will use the captures i got with 5 hours straight as with that amount i should be able to get a very precise sync done with about anything compared to a 30min clip right?

[IanB]

Milliseconds per hour is 60 mins per hour * 60 secs per min * 1000 ms per sec = 3600000

So slow by 133ms per hour use :-AdjustNumerator=3600000+133 # 133ms per hour
AdjustDenominator=3600000And of course to speed up by 133ms per hour would be 3600000-133

[zerowalker]

Ah easy enough thanks:)!

[zerowalker]

It doesn´t work if i use more numbers than (36000) meaning miliseconds can´t be used;S

[IanB]

Oops yes the product of Ar*AdjustNumerator has to fit in a signed 32 integer, max value 2147483647, so 44739 is the max AdjustNumerator for 48000 audio rate.

3600133/3600000=1.0000369444...

Reducing this ratio with continues fractions at the limit of 44739 gives

27069/27068=1.00003694399 or 132.99837 milliseconds per hour
...
AdjustNumerator=3600133 # 133ms per hour
AdjustDenominator=3600000
Ar = Audiorate()

Limit=Floor($7FFFFFFF/Ar)
Num=ContinuedNumerator(AdjustNumerator, AdjustDenominator, Limit)
Den=ContinuedDenominator(AdjustNumerator, AdjustDenominator, Limit)

ResampleAudio(Ar*Num, Den) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate

[zerowalker]

i don´t get any sound when i use that?
Do you get sound?

[IanB]

Yes, it works same as the original script, As it should, the numerator and denominator have just been scaled. Add an Info() call to the end of the script to make sure weird numbers are not happening. Try ColorBars() as a test source to dispel any source considerations.

[zerowalker]

got it to work i think:)

will this be 89.2 ms per hour?

AdjustNumerator=36000892
AdjustDenominator=36000000

thanks:)

[IanB]

36000892/10000-3600=0.0892 => 89.2, so yes ;)

[zerowalker]

It freezes if i use some other numbers, for example 62.5ms = 625 instead of 892, it has something to do with the Limit, though i don´t know what.

[IanB]

For Ar=48000Hz you get Limit=44739 which results in Num=44740, Den=44739,

48000*44740=2147520000 which is greater than the max value 2147483647, so you end up flipping a negative result.

Also 44740/44739 actually gives 80.466 milliseconds per hour, not the 62.5 you want. Lowering limit to avoid the overflow will make this worse.

You are running out of the available precision.

You could scale the AudioRate to crib a bit more precision but the current version of ResampleAudio also tests that the input AudioRate and the output AudioRate are actually different. If they are not then it does nothing. I have just committed a change to CVS that now tests the num_audio_samples instead, but that will not help you until a build is available.

This will get you very close at 62.4ms, with Limit=74565, Num=57693 and Den=57692...
AdjustNumerator = 36000625
AdjustDenominator = 36000000
XAr = 28800 # Lowest rate that supports 62.5ms/hour

Limit=Floor($7FFFFFFF/XAr)
Num=ContinuedNumerator(AdjustNumerator, AdjustDenominator, Limit)
Den=ContinuedDenominator(AdjustNumerator, AdjustDenominator, Limit)

Ar = Audiorate()
AssumeSampleRate(XAr) # Scale to crib a bit more precision
ResampleAudio(XAr*Num, Den) # Fine tune duration
AssumeSampleRate(Ar) # Put back original sample rateAdd this to you script as required to help you see what is happening.Return SubTitle(String(Num)+" "+String(Den)+" "+String(Limit))
With a build that includes the above mentioned fix the following will be possible :-...
AdjustNumerator= 36000625
AdjustDenominator=36000000
Ar = Audiorate()
AssumeSampleRate(1) # Scale to crib a lot more precision
ResampleAudio(AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate

[zerowalker]

Didn´t get everything with the numbers there, but i somewhat understand that there is a limit to how precise it can be thanks to many things.

Where do i look for the build, it´s very nice of you to commit a fix/tweak to it or what you all it so fast when you saw the "limits" or what it is!

Will do the code that you says work with the current build, and i think it´s somewhere around 60-70ms, it seems to be 62.5 but it´s hard to listen as i don´t know when exactly the sound is supposed to come so i have to guess more or less:)

But 10-20 ms here and there over some hours isn´t anything to worry about, it´s 50ms+ or - that´s noticeably if you ask me:)

As always thanks!

[zerowalker]

Does anyone know of a game or something, that has a very easy way to predict sound?

Like a flashing light that sounds exactly when it´s bright, or something?

As with games where you shoot, the sound doesn´t come Exactly when it´s fired, but some frames later depending on game of course.

Very Appreciated if anyone knows, it can be anything, just that it´s rendered somehow, even a clip will do i think:)

[StainlessS]

Just a suggestion if you want this for test purposes,
why not knock up test clip yourself, with maybe a grey blankclip with silence
and trim/splice in a white blankclip with tone as the test.
Maybe an integer FPS, perhaps even 1 FPS would do.

[zerowalker]

ah very true, will make a clip with AVS if i can (tip would be nice).
Will let it runt when i sleep so it will be like 8 hours or more hopefully.

EDIT:

Okay even if i had a clip that make a tone and change color, it won´t be precise when i record it´s playback, as it doesn´t record it Exact as it playsback, so i can´t look at the waveform and the video;S

Anyone have any ideas how it can be done?


EDIT 2:

please take a look at this and try to sync it with AudioGraph in avisynth, i can´t make it work for the entire clip.

http://www.sendspace.com/file/3sh7wm

[zerowalker]

Okay have to bring this up again.

I noticed that ResampleAudio gives alot of noise when converting from 47999 to 48000, and i can´t use the script above.
As it doesn´t seem to work, no matter what i write, it will still be the same lenght and delay.

EDIT:

It seems that the problem lies with how it works in the audio bit.
Cause if i "ConvertAudiotoFloat()" right before ResampleAudio(48000) the noise goes away.

[IanB]

What exactly is the added noise like?

What type of audio are you working with?

I am not sure exactly what you are saying, but the 16 bit path in ResampleAudio has it's problems, it always has. This is why SSRC was originally added and a sFloat code path later added. This script shows an issue with lack of precision :- ColorBars()
AmplifydB(-60.0)
ConvertAudioTo16Bit()
AssumeSampleRate(47999)
ResampleAudio(48000)
AmplifydB(57.0)The added buzz is quite substantial but then -60dB (1000 times) is a pretty low level, resulting in only ~5 bits resolution. For most medium quality audio in the -40 to -20 dB range the result are usually satisfactory. If you need better quality then using the sFloat path or SSRC is usually the answer.

[zerowalker]

The noise is, Hiss like.

And the audio is. Well don´t know what it´s called, but not Real Music, Gameboy SNES etc sound, i know it´s called something.

I understand that 16 bit has it´s problems, but doesn´t ResampleAudio work like SSRC now?
I remember that it was bad before, but they fixed it, by making it work in float, or something.

But as i can see, it doesn´t work in float, as if i Convert to float before the resample, and after convert it to 16 bit, it hiss isn´t there.
But that doesn´t really sound like an optimal way, but then again, i am not that good at this stuff.

If you need a sample, i will gladly provide it.

[Gavino]

I understand that 16 bit has it´s problems, but doesn´t ResampleAudio work like SSRC now?
I remember that it was bad before, but they fixed it, by making it work in float, or something.

But as i can see, it doesn´t work in float, as if i Convert to float before the resample, and after convert it to 16 bit, it hiss isn´t there.
ResampleAudio has two modes of working: 16 bit and floating point.
If you give it 16 bit, it will continue to work in 16 bit.
If you give it anything else, it works in float.

So if you are starting with 16 bit, you need to convert explicitly to float before calling ResampleAudio, if you want it to work in float mode.

[zerowalker]

I see, But.
What is good to do?
I mean, if i use for example, Virtualdubs Samplerate converter, at High Quality (haven´t tried without). I don´t get the hiss.

I don´t know how this resampling thing works, i just want best quality. Lossless if possible, but that isn´t i guess.

[IanB]

Resampling audio involves interpolating output samples from a collection of input samples. How you select and manipulate the input samples determines the faithfulness to the original that the output represents.

The very crudest interpolation would be a simple linear interpolation of the output sample from the 2 adjacent input samples weighted by the relative distance of the input position from the output position. So if going from 9 samples per interval to 10 samples per interval to generate output sample 5 we calculate this maps to input position 4.5 so we would take 0.5 of input sample 4 and add 0.5 of input sample 5. The other output samples would be similarly calculated based on their relative position to the input samples.

Even for simple sine wave input samples such a crude interpolation results in a noticeable amount of error. This error results in noise. So practical resamplers involve considerably more complex calculations, utilising 100's or even 1000's of input samples to generate each output sample.

The 16 bit code path uses uses 16 bit samples multiplied by 16 bit filter coefficients, accumulating 32 bit intermediate results. The interpolation position uses 24.8 fixed point arithmetic. All these compromises allow for a very fast implementation but result in small errors against the intended calculation. The original algorithm was intended for real time processing on a 80386 cpu. The accumulated error presents as noise.

If the 16 bit code path is not of sufficient quality use the sFloat code path, it may be slower but it is still many many times faster than real time on modern cpu's. The 16bit to sFloat back to 16bit is lossless.

[zerowalker]

How do i make it use sfloat?
Do i just convert it to float and then 16 bit after then resampling?
Or is it a way to make the resampling do it internally?

Thanks

[IanB]

Yes you explicitly convert to float.

You don't need to convert it back, the avi output layer will automatically convert it to 16 bit. It may actually be beneficial to convert to 24 bit if the output work flow can use it.

[zerowalker]

The original sound is at 16 bit, so shouldn´t i convert it to that?

But if i don´t convert, then it will still get converted, as it can´t stay Float i presume?
What will it get converted to?


And the end result will be 16 bit, (AAC or other lossy), would it still be beneficial to convert to 24 bit even if the source is 16 bit cause of the float resampling?

[IanB]

Once you process an audio stream at high bit depth, you should leave it in high bit depth format for the rest of the work flow. This minimises accumulating quantisation errors, e.g. if you simply amplify a stream by 3db a 16 bit input sample of value 1234 wants to become 1743.0713 and a value of 4321 want to become 6103.5747. Staying with 16 bit these 2 samples become 1743 and 6104. So the relationship between the values changes slightly what originally was a ratio of 3.5016207 becomes 3.5020080 the error is very small but if you keep performing this rounding it does eventually effect the sound.

By default the avi output layer will automatically convert float to 16 bit. The avisynth option "global OPT_AllowFloatAudio=True" can be used to control this behaviour if your work flow can actually accept IEEE floating point audio format. You can also explicitly ConvertAudioTo24Bit() or ConvertAudioTo32Bit() if your work flow can use those formats.

In most cases this does not matter, the background noise already present in the recording totally swamps all these considerations. You are experiencing a case where the additional noised caused by insufficient precision in the processing calculations adds noticeable extra noise.

If you want to know more about this issue, google for "audio noise shaping (http://www.google.com/search?q=audio+noise+shaping)"

[zerowalker]

Ah i see, or well, it´s a bit complicated. But i think i got a hold of it.

And currently this is my script:

v1 =Avisource("Z:\Yoshis Island\snes9x-unicode 2013-04-25 06-17-14-807.avi").Assumesamplerate(47999).Amplify(2.8).Convertaudiotofloat.ResampleAudio(48000).DelayAudio(-0.032)
Intro = avisource("E:\Fraps Projekt\Intros etc\Final 16-9.avi").Convertaudiotofloat.Amplify(1.77)

Clips = v1
AlignedSplice(Intro.resampleaudio(48000).Zoombox(Width (Clips), Height (Clips),ResizeMethod="BicubicResize").ChangeFPS((Clips)),Clips.ConvertToYV12(matrix="Rec709"))
ConvertAudioTo16bit()

A bit excessive maybe, but i am currently amplifying the clip that has wrong speed, right after i fix it, and before changing it to float for resampling.
The clip(intro) isn´t anything wrong with, so i just amplify it after float, and later resamples it to 48000 (it´s 44100).

And at the end i convert it to 16 bit. (I trim it after i convert it, but that shouldn´t affect anything).

And when i have listened to it. The wrong speed Clip, sound the same after this. well a bit slower, but it´s inaudible as it´s so slow. But i can´t hear any noise getting added, just the original noise that´s their. Proably cause of it being made in 16 bit from an Emulator, or maybe it´s within the sound, but i doubt it. (It´s midi like sound effects pretty much.)


But i think that as i can´t notice any difference, i will stay with 16 bit at the end. As i won´t make any more changes to the audio after that anyway, except encoding it.

[IanB]

To be pedantic Amplify after you ConvertAudioToFloat.

And as I keep hinting if your work flow will accept 24bit, 32bit or sFloat use it! Avisynths ConvertAudioTo16Bit just does bit length truncation (no noise shaping), good audio encoders like AAC can include noise shaping as part of their lossy compression.

[zerowalker]

Ah, so it´s better to Amplify with Float, if it´s going to be Float in any stage?

Well my work flow is pretty much, Edit it in Avisynth and then Encode with x264 and AAC for example.

But if lossy encodings can do better with the highest quality work flow (Float in this scenario as i am resampling) i will keep that in mind.

So to do that, i will have to do something like this, right?

global OPT_AllowFloatAudio=True
v1 =Avisource("Z:\Yoshis Island\snes9x-unicode 2013-04-25 06-17-14-807.avi").Assumesamplerate(47999).Amplify(2.8).Convertaudiotofloat.ResampleAudio(48000).DelayAudio(-0.032)
Intro = avisource("E:\Fraps Projekt\Intros etc\Final 16-9.avi").Convertaudiotofloat.Amplify(1.77)

Clips = v1
AlignedSplice(Intro.resampleaudio(48000).Zoombox(Width (Clips), Height (Clips),ResizeMethod="BicubicResize").ChangeFPS((Clips)),Clips.ConvertToYV12(matrix="Rec709"))

Then feed it directly to the Lossy Audio encoder?

EDIT:

Or wait, i don´t really need to use: global OPT_AllowFloatAudio=True, if i feed it into the encoder right?
As it´s not converted to AVI, it should be float. It´s only if i save it with Virtualdub for example, that i would need it to prevent 16bit conversion right?

[IanB]

You should aim to only have zero or one bit length truncation operation in your audio graph. Bit length truncation happens every time you need to round the result of a calculation. E.g. in 16bit Amplify does So=(Si * K + 65536) >> 17, K=131072 is unity gain. Converting sFloat to 16 bit does So=(short)(Sf * 32768.0f + 0.5f) which is exactly the same rounding. So if you do 1 Amplify or 1 MixAudio or 1 whatever with 16bit input and 16 bit output that okay. If you need to do 2 or more operations then it is advantageous to do all the operations in sFloat and take a single hit with the single output bit length truncation. And if you can avoid the bit length truncation altogether e.g by folding it into the noise shaping of an advanced lossy audio compression then .....

Now the noise levels we are discussing here are very small, usually only extreme audiophiles have equipment that can reveal the noise and thus care about it. The 16 bit path in ResampleAudio does many 100's of these rounding operations so exposes the noise quite dramatically.


Yes, OPT_AllowFloatAudio and other option only effect the AVI/VFW output layer. If an application uses the Avisynth API then whatever the application does with the API is what you get, e.g. I believe MeGUI can automatically append a ConvertToYV12() to the end of it's graphs. The global OPT_AllowFloatAudio=True costs you nothing in your script, applications are free to ignore the value or implement some appropriate function of it's choice.

[zerowalker]

Okay thanks, really helped me out.
Think i got hold of it.

I will do it as the script above, as i currently does a bit more job, and as you said, it´s better to do it all in float. And i do use MeGUI and it should be able to feed it directly to the Encoders.
And if a do need it in AVI in float for some reason, i will use that command.

Really appreciate it, thanks!:)

[zerowalker]

Okay once again i need help.

This time i have need for getting samplerate to work in float.

To be exact, i need 48000hz to work in 47998.88hz

The earlier way doesn´t seem to work, where you could calculate the ms per hour and use a special function and resampling/conversion to get it to the work in more precision than "Assumesamplerate".

[IanB]

Read this thread again from post 21.

Do you want to build "AssumeSampleRate(47998.88).ResampleAudio(48000)" which will increase the audio duration by 84 milliseconds per hour. So AdjustNumerator=3600084 and AdjustDenominator=3600000

Or do you want to build "ResampleAudio(47998.88).AssumeSampleRate(48000)" which will decrease the audio duration by 84 milliseconds per hour. So AdjustNumerator=3600000 and AdjustDenominator=3600084

[zerowalker]

It doesn´t work.

it does work if i use this:

XAr = 28800 # Lowest rate that supports 62.5ms/hour

Limit=Floor($7FFFFFFF/XAr)
Num=ContinuedNumerator(AdjustNumerator, AdjustDenominator, Limit)
Den=ContinuedDenominator(AdjustNumerator, AdjustDenominator, Limit)


that you added, but is it accurate( thinking about the XAr variable).

[IanB]

Are you using Avisynth 2.6.0 Alpha 4 [130114] or a recent build that includes this bug fix :-

* Fixed ResampleAudio NOP test to compare vi.num_audio_samples, not sample rate.

If you are, then you can use the much simpler method at the bottom of Post 35. This avoids all the dodging precision and word size limitations that have plagued you in this thread.

[zerowalker]

Nicely, Works!

But i have used the MT versions of 2.6.

What is the different (Except MT)?

//Edit

How is the resampling btw?

Is it better than Assumesamplerate(47999) and resampleaudio(48000) which introduced quite some noise?
Or will i haveto convert to float before resampling?

(Checked it, it´s the same, which make sense:) )

[zerowalker]

Okay it seems my calculations where off.

If you look at these pictures, you can see that it´s the exact same place, but at different times.

So i use this clip to try to calculate the desync problem.

As you can see, it´s very close in the waveform, but still not correct.

This is with:

AdjustNumerator=36000710 # 100ms per hour
AdjustDenominator=36000000
Ar = Audiorate()
AssumeSampleRate(1) # Scale to crib a lot more precision
ResampleAudio(AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate

It seems that it should be

3600070x

But i can´t work at that precision, as it doesn´t change anything.

[IanB]

You cannot use thisAssumeSampleRate(1) # Scale to crib a lot more precisioncheat unless you have a version that includes the above mentioned code change.

To see if the script is working as required carefully check the before and after "Audio Length: 1742584833 samples.". The number should change by 48 per (millisecond per hour) per hour correction you apply, i.e 34367 samples for the 71 ms and 10 hour example above.

[zerowalker]

Okay, will use the version without MT then.

Okay i checked, the Samples seem to change, but the waveform() doesn´t.

Perhaps it can´t "see" small changes?

Look at this.
AdjustNumerator=36000710
http://i40.tinypic.com/e5ldlc.png
AdjustNumerator=36000717
http://i44.tinypic.com/bg1s83.png

As you can see, there isn´t any change.

Or well, there is a , extremely tiny change in the wave, if you look at extremely carefully.
But it doesn´t move.

But if i change from AdjustNumerator=36000717 to 718, it will move drasticly.

Here you can see.
The left is very early in the clip, the right is at it´s end about 10 hours later.
The waveform should appear very similar (atleast the loud part).
But as you can see, it´s off by a bit after 10 hours, even when i try to correct.
This is because i can´t move it a tiny bit, only big movements (atleast according to waveform() )

http://i43.tinypic.com/alr8yo.png

[IanB]

Okay I see what you are saying, the AudioLength has changed by the expected 7ms in 10 hours, but the audio graph has not shifted the waveform by 7ms on a frame near the end of the clip.

I assume the audio wave display is 1/30 of a second, 1600 samples. And 7ms is 336 samples which would be 21% of the wave display window. What tool are you using to paint the waveform display, it seems to be mapping the audio samples onto the video frame inaccurately.

I lashed up this script to prove the current Avisynth works as expected :-ColorBars(640, 480, "YV12") # 1 Hour
ChangeFPS(100, 1) # 10ms resolution
Trim(0,0) # Force AudioLength == VideoLength
Loop(10) # 10 Hours
X=Last # Save original

AdjustNumerator= 36000010 # Plus 10ms
AdjustDenominator=36000000
Ar = Audiorate()
AssumeSampleRate(1) # Scale to crib a lot more precision
ResampleAudio(AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate

MergeChannels(Last, X) # Mux new and old
Info()
AmplifyDB(-48.0) # Scale for display
Histogram("AudioLevels")
yes the audio is 10hours and 10ms and yes the new right channel, 2, per second transitions happen 1 frame later than the original, 4, at the end of the clip.

[zerowalker]

Yes it seems to, which makes it hard to do what i want.

I am using this plugin: http://forum.doom9.org/showthread.php?t=165703

I looked through and as you say, it seems to do it´s work, the only thing is then that the waveform is incorrect.

[IanB]

Waveform works correctly with my test script....
AmplifyDB(-48.0) # Scale for display
Histogram("AudioLevels")
AmplifyDB(48.0) # Scale for display
Waveform(1, Under=True)

:script: Post your entire script, you must have something strange happening.

[zerowalker]

Weird, but sure.


directshowsource("Z:\snes9x-unicode 2013-04-25 12-50-25-985.avi").DelayAudio(-0.032)
Crop(94, 0, -94, -0)

ConvertAudioToFloat()
Amplify(15)
AdjustNumerator=36000706 # 100ms per hour
AdjustDenominator=36000000
Ar = Audiorate()
AssumeSampleRate(1) # Scale to crib a lot more precision
ResampleAudio(AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate
info()

waveform(under=true,window=1)
Trim(1036, 1036)++Trim(1076458, 1076458)
StackHorizontal(Trim(0,0),Trim(1,2))


I am not sure we fully understand each other, if so i made a picture to clear things out what i am trying to do.

Point A, is where the sound should be at, if you look at the left.
http://i41.tinypic.com/4qiv0h.png

[IanB]

Are you adjusting in the right direction? In the post above you seem to want the audio at the end to play earlier, in which case you would need to shorten the audio track. I would estimate by about 7 milliseconds total in 10 hours 5 minutes. Thus try AdjustNumerator=36000699 # 69.9 per hour, i.e. 0.7ms per hour less.

Of course you do realise 200 milliseconds is generally consider the point that an audio delay is just noticeable.

[zerowalker]

Problem is, it won´t detect small changes for some reason.
It´s hard to show, would like to give you the clip, but it´s 10hours and well, extremely big.

But here is an example.

http://www.sendspace.com/file/64n02i

As you can see, nothing happens, until "d8".
Hope that shows what i am trying to explain with the lack of accuracy.

And yeah i know that i am currently working with quite an overkill in terms of what human can notice.
But i am quite a "perfectionist" or what to say. When i know it can be fixed extremely correct, than i want to do it.

I have you to thank for this, you are helping me extremely with what can be done with this case, thanks:)

[IanB]

I can see you are getting "quantisation" in your script. I am at a loss know what is going on. There must be some wood in them there trees, I just cant see it :confused:

I modified my test script, please try it :-Function DoIt(Clip clip, Int AdjustNumerator) {
clip
AdjustDenominator=36000000
Ar = Audiorate()
AssumeSampleRate(1) # Scale to crib a lot more precision
ResampleAudio(AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate
}

ColorBars(640, 240, "YV12") # 1 Hour
ChangeFPS(100, 1) # 10ms resolution
Trim(0,0) # Force AudioLength == VideoLength
Loop(10) # 10 Hours
ConvertToMono()

A=Doit(36000706)
B=Doit(36000705)
C=Doit(36000704)
D=Doit(36000703)
E=Doit(36000702)
F=Doit(36000701)
G=Doit(36000700)
H=Doit(36000699)

MergeChannels(A, B, C, D, E, F, G, H)
Info()
Waveform(1, Under=True, Height=60)Frame 3599970 should encompasses the last per second transitions, and each transition should be 0.1ms earlier.

Run this script and post a copy of frame 1076458.Function DoIt(Clip clip, Int AdjustNumerator) {
clip
AdjustDenominator=36000000
Ar = Audiorate()
AssumeSampleRate(1) # Scale to crib a lot more precision
ResampleAudio(AdjustNumerator, AdjustDenominator) # Fine tune duration
AssumeSampleRate(Ar) # Put back sample rate
}

DirectShowSource("Z:\snes9x-unicode 2013-04-25 12-50-25-985.avi")
DelayAudio(-0.032)
Crop(94, 0, -94, -0)
ConvertAudioToFloat()
Amplify(15)

Spline16Resize(640, 240)
ConvertToMono()

A=Doit(36000706)
B=Doit(36000705)
C=Doit(36000704)
D=Doit(36000703)
E=Doit(36000702)
F=Doit(36000701)
G=Doit(36000700)
H=Doit(36000699)

MergeChannels(A, B, C, D, E, F, G, H)
Info()
Waveform(1, Under=True, Height=60)

[zerowalker]

Yes, the only thing that changes until the last picture, is small "jitter" in the waves, but it doesn´t move.

Here is pictures from your scripts.

http://i42.tinypic.com/a2xc0i.png
http://i40.tinypic.com/eg4tjm.png
http://i40.tinypic.com/dgqfpi.png

Something weird is going on, you script seems to work just fine though, but not for my clip:S

[zerowalker]

Noticed something, if i resample the audio from something, than back to 48000 right before changing the lenght, it seems to act quite different.

Here is with: ResampleAudio(48001) and back again.
http://i41.tinypic.com/348ghty.png

[IanB]

Try adding a Preroll(audio=10) after the DirectShowSource(), if that doesn't help try an EnsureVBRMP3Sync().

[zerowalker]

It seems that EnsureVBRMP3Sync(). may be working.
But it´s very hard to try it out, as it need to read the 10hour clip every time i change anything, which takes ages:(

Preroll(audio=10) didn´t do anything though.

Though may be worth mentioning, i am NOT working with MP3 here, it´s pure PCM.

[zerowalker]

I think i detected the problem, it comes from Directshowsource.

I am using a file saved as Lagarith, with PCM 16bit 48khz Audio, and if i use Avisource, the audio goes silent after 3:52:11 hours, so i use directshowsource, and it seems to lack accuracy with that.
Though i don´t understand why the audio goes silent.

I even tried saving the audio and using audiodub, but i get:

AVIFileSource: Coultn´t open file

And i am using WAVSource on an .wav file, i don´t see the problem.
But i think Avisynth may be limited on how long the wav file can be, which seems to be 3:52:11 hours.


But it does work if i load it using Directshowsource, like this:

avisource("Z:\snes9x-unicode 2013-04-25 12-50-25-985.avi",audio=true)
audiodub(directshowsource("F:\Desktop\snes9x-.wav"))

Then everything seems to be working.
Though i would like to get the audio to work without this through Avisource().


Worth mentioning is.

avisource("Z:\snes9x-unicode 2013-04-25 12-50-25-985.avi",audio=true)
audiodub(directshowsource("F:\Desktop\snes9x-.wav"))

With this, it´s all good.

directshowsource("Z:\snes9x-unicode 2013-04-25 12-50-25-985.avi",audio=true)
audiodub(directshowsource("F:\Desktop\snes9x-.wav")

With this the problem occurs.
So it has to do with the how the video acts or something.
You maybe have a guess?

EDIT:

The problem works with

directshowsource("Z:\snes9x-unicode 2013-04-25 12-50-25-985.avi",audio=false)
audiodub(directshowsource("F:\Desktop\snes9x-.wav")

So it works if i use Audio=False.

[IanB]

Looks like something to do with Direct Show is quantising the audio seek position for big values. In the bottom image of your post 70 above :-
http://i40.tinypic.com/dgqfpi.png
The channels 5 & 7 have glitches and the shape of 8 does not match the others.

Given seeking is screwed for high number samples I am not surprised Preroll does not help. And EnsureVBRMP3Sync() which uses brute force to make seeking work properly solves the issue (but creates a performance issue).

Not sure why the audio goes silent after 3:52:11 hours it is some odd value. For 48000 you usually run into 32 bit size limits at 12:25:39, 6:12:49 or 3:06:24. Maybe this is where the container crosses a 32 bit boundary :confused:. (For 44100 the 32 bit boundaries are at 13:31:35, 6.45:47 and 3:22:53)

[creaothceann]

What about WAVSource or other source filters (http://avisynth.org/mediawiki/External_filters#Source_Filters)?

[zerowalker]

Looks like something to do with Direct Show is quantising the audio seek position for big values. In the bottom image of your post 70 above :-
http://i40.tinypic.com/dgqfpi.png
The channels 5 & 7 have glitches and the shape of 8 does not match the others.

Given seeking is screwed for high number samples I am not surprised Preroll does not help. And EnsureVBRMP3Sync() which uses brute force to make seeking work properly solves the issue (but creates a performance issue).

Not sure why the audio goes silent after 3:52:11 hours it is some odd value. For 48000 you usually run into 32 bit size limits at 12:25:39, 6:12:49 or 3:06:24. Maybe this is where the container crosses a 32 bit boundary :confused:. (For 44100 the 32 bit boundaries are at 13:31:35, 6.45:47 and 3:22:53)

I see, well that explains it.

Hmm, well i don´t know either, maybe it has to do with the video codec (I am using LAV Filter, and i guess it takes over the decoding of Lagarith with Directshow). But i don´t see how that matters for the audio. Or maybe it´s the AVI Splitter, don´t know have Avisynth works with that.


What about WAVSource or other source filters?

WAVSource doesn´t even work with the wavefile for some reason.

I get:

AVIFileSource: Coultn´t open file

So i have to use Directshowsource:(

It´s very odd.

[zerowalker]

Well it doesn´t really matters as i normalyl don´t work with files that are 10 hours;P
It´s just a test sample to use for sync purposes.

But i wonder.

I Virtualdub when i capture, it can detect the actual rate compared to the intended.
IF you look at Relative rate under Audio.

Is it possible to use that method to get a get the sync value (hz value)?

As virtualdub can only use the methot on capturing, which seems a bit limited as it also has to capture video and all that.

And the Relate Rate isn´t float, it´s Int with 2 decimals, so it´s a bit lacking.

http://i43.tinypic.com/34ythxj.jpg

[raffriff42]

Very interesting discussion. I would advise not to resample the audio for such a minor speed correction, because of the artifacts. If you must resample, use SSRC ('http://avisynth.org/mediawiki/SSRC')(32-bit float + supersampling). Note the caveat on the SSRC Wiki page:
Sampling rates of 44.1kHz and 48kHz are popularly used, but the ratio of these two frequencies is 147:160, and they are not a small numbers. Therefore, sampling rate conversion without degradation of sound quality requires filter with very large order, and it's difficult to achieve both quality and speed. This program achieves relatively fast and high quality with two different kinds of filters combined skillfully. The author is referring to the need to find the least common factor of the two sample rates, supersampling the audio to that common frequency, and subsampling to the new frequency. This is the only way to do it without getting strange artifacts like you are seeing. Now if 147:160 is a problem, yours is impossible.

70 ms is two video frames at 30 fps. Can't you find two frames per hour to drop or repeat as required? This would be a much better way to go.

[zerowalker]

The problem is that SSRC is very limited, and doesn´t allow resampling here and there.
I only get problems with 16bit, if i use 24bit+ (preferebly Float), there is no artifacts that i can hear.

And as you say, it´s another way to solve it, by adding/dropping frames, which i looked into a little while ago.

But i am currently trying to analyze the problem.

I want to know, Is it the audio that goes out of sync, or the video.
As i only guess it´s the audio, but i actually don´t have anything to back it up.

The best thing also, would be to solve the core problem, as there shouldn´t be desync to begin with.
I would appreciate it, if you could try to reproduce the error on your side, raffriff42.

If you could try to record something that goes on and on for along time (while you sleep) and see if you get the sync problems.
A recommendations is Super Mario World, as the intro there just goes on and on the same all the time.

btw, thanks for joining the discussion:)

[raffriff42]

Windows multimedia timing, specifically how it chooses a reference clock, is a mystery, wrapped in a riddle, etc - at least to me. My guess is your audio card is supplying the audio sample rate clock, and it it running approximately 0.000016% off for some reason. Probably due to the digital frequency generator's limited precision. Whatever. I am a "git er done" type of guy and try not to obsess over little things like this. :)

[zerowalker]

Would really love a way to calculate the off sync compared to that reference clock so i can get an exact value.
There should be a way to get it, but i don´t know how, Virtualdub seems to know a way to calculate the desync somehow, but i don´t know how.
And i can´t find any information at all on this.

Yeah i know that i am to obsessed, but i am a perfectionist, i can´t tolerate 70ms per hours, it´s just not possible.
If i know there is a problem, i have to solve it, Sync problem is one such problem where i can´t turn blind eye at all.

This is also true for capturing Analogue material, i will get the same desync there, and it´s extremely important to have good sync there, you can´t just duplicate frames here and there.
Or well you can, but you will lose information that way, which isn´t something you want on such material if you are into archiving and restoration.

[StainlessS]

Methinks you are destined for serial disappointment.

[zerowalker]

Well probably;P

Does this application work for you?
http://www.agm.me.uk/blog/uploaded_files/SoundSpeed.zip

It seems to do what i am looking for, but not sure if it works correctly.
http://www.agm.me.uk/blog/2007/09/soundcard-clock-accuracy.php

To confirm this theory I coded up a small application that uses the Windows multimedia API test the clock speed of the sound card. It compares this with the system clock to work out the error.

[zerowalker]

Does anyone have knowledge in Graphstudio/Graphedit?

It seems that i may be able to use it to determine the clock sync between to devices/filters.

[StainlessS]

Search , Advanced mode, threads, tites only: "GraphStudio OR GraphEdit" (EDIT: Without quotes of course)

You will find plenty of threads where you may be better served.

[zerowalker]

Found this: http://www.nomadelectronics.com/Vidcap/DirectShow/ShowClock/ShowClock.aspx

Can anyone test it, i atleast don´t get a match of 1, i get about 0.999xx

[Ghitulescu]

maybe is a problem with the recording software ...

[zerowalker]

Sadly that´s not the case.
I have gotten it down to that it´s the difference between the capture device clock and the audio clock that makes the sync problem.

And the capture device clock in this case, is the PC itself, which is what is weird.
If you capture video from a capture card, and audio from a sound card, those 2 would have difference clocks.

But a soundcard and the PC itself, shouldn't really be such a problem.
I am trying to see if there is a way to alter the PC clock, or QueryPerformanceFrequency, which seems to pretty much be the clock, to get it closer to the audio clock.
But i can't find anything about it.


But please try the filter in Graphedit/studio i posted, and write the result you get on Ref/QPC, i am very interested.

[StainlessS]

Something from MSDN:
http://msdn.microsoft.com/en-us/magazine/cc163996.aspx

[StainlessS]

I tried the showclock.ax in graphStudio (Graphedit crashed as soon as I clicked on "DirectShow Filters"),

On start, shows Ref/QPC ~ 0.9997xxxx, after ~ 200 seconds, crept up to 1.00000

[zerowalker]

So you get a stable 1?

Hmm mine seems to not get there.
I tried changing CPU Skew to delay 400, just to see if it changes anything, and i think it did.
I am currently at 0.999996 and going up, extremely slow though (9 min).
But perhaps it is random and hadn't anything to do with CPU Skew.

But thanks, at least you get 1, and that says something.

[StainlessS]

So you get a stable 1?

Yes, stable 1.0

Probably totally un-related but I just put in a brand new CMOS/RTC battery this morning.

(Got a batch of about 8 on a card, in the 99 pence shop, only last about 6 months each)

[zerowalker]

May be related, IF the timer depends on the RTC and the battery is bad.
I tried testing the battery by disabling internet time sync, to see if the time would go off.
And it didn´t for the time i tested, but i didn´t make a really thorough test, i will keep the battery in mind, thanks for the information.

[Ghitulescu]

Sadly that´s not the case.
I have gotten it down to that it´s the difference between the capture device clock and the audio clock that makes the sync problem.

Then you should have the same problems during playing ....

[zerowalker]

That's the weird thing i am trying to look closer at.
But there seems to be audio renderer that solved that by increasing the audio clock (or decreasing), so i guess the issue exist on playback aswell.
Though it's not the same for some reason, i guess the sync is a bit more loose or something.

Or maybe it is the software or method, i would love to be able to solve it or know the problem anyways.


And, i tried recording on another PC and let it run overnight.
I ran probably for about 10 hours i think, and it had desync issues aswell, though not as i did.
I didn't check, but i would guess the desync was a bit less then mine. Maybe 200-300ms at 10 hours.

[zerowalker]

Okay i have gotten a bit more knowledge, but it has made me quite interested in a thing.

It seems to work like this.

There is always a Video Clock and an Audio Clock.
If these to are not the same (which they never are, except if they are on the same device and created from the same source) they will be different.

So for example.

1:00:00 in the Video Clock could be 1:00:55 in the Audio Clock.

So the only way to solve this except using the same clock (which don't seem to be possible, it's hardware releated), is to sync the video to audio, or audio to video.

But here is the dilemma, it may sound good at first, i mean, sync one source to the other, and all is well, right?

But, which clock is correct?
Which is reality, which has the correct time based on reality (not going to deep, but which has 1 sec, that is 1 sec).

By looking at the video, and it say, the video duration is 2:00:00, then that should be correct in my eyes.
But if i sync it to the audio, then it would be 2:01:40 (an example).
Does that just mean that the Audio is wrong, and the clock goes to slow?

Or is the Audio correct, and the Video goes to fast?

I know it may sound like a science project now, but i don't care about, Oh the precision in Time Space blabla.

But hope you get what i mean with it. Cause as both aren't the same, one or both must be wrong if you compare it to Real time.
But i don't know well how Time works, but i only care about, well the Clock in my kitchen so to speak.

[Ghitulescu]

There will always be 4 clocks in a PC. One is the clock of the CPU. Another one is the RTC, already mentioned. The third one is the video clock, provided by the video card. The fourth one is the audio clock.
There is also the possibility that each clock be a plurality (for instance some older cards had a clock for each refresh rate, some audio cards had one for 48k another for 44k1, and so on).

[zerowalker]

Well for this case, there are only 2 clocks that matter, the Video Clock and the Audio Clock.
the rest doesn't seem to matter when capturing goes drift.

I am pretty sure my Audio card only goes with one clock, as different frequencies doesn't change anything.
Same with video card, resolution (and tiny difference with hertz, like 60,59.8 etc) doesn't seem to matter, atleast to recording.

But i would like to compare the clocks to reality if possible, cause i guess sitting and counting numbers doesn't yield much precision.

If i would have to guess, it would be that my Sound Card has more precision than the Video clock, but that's just a guess.

[IanB]

When capturing the video clock is from the video source, i.e. TV station, VCR, ...

Advance capture hardware also provides a synchronised audio clock to keep the two is sync.

Avery Lee has discussed this issue on his VirtualDub site blog (http://www.virtualdub.org/), Google for "video audio clock site:www.virtualdub.org (http://www.google.com.au/search?q=video+audio+clock+site:www.virtualdub.org)"

[zerowalker]

Well when that is the matter, i don't know what to do.
Let's say i capture from a VCR, which clock should i sync after, the video clock, or the audio clock.
That's the dilemma here, as sadly, i don't have hardware that can keep everything sync.

I have read some documentation there, and it's very interesting, learned about video clock there, and how Virtualdub handles it.

http://i41.tinypic.com/2z5sr6h.png

For those who want to test a sync issue, i learned a way to easy see it.

Set up a graph like this, choose your video source and audio source however you want, and codec.

Then click on AVI Mux, go to Avi Specific, enable Capture Drift. And choose the audio pin (in this case it's pin 2, which is number 1 (1 is 0)).
Now capture and stop after awhile, maybe 1-2 minutes.

Now check the file, it should not have the correct framerate, it should be synced to the audio, and either han slightly faster fps or slower.

[Ghitulescu]

Finally I got it: you record the games by capturing them, like you'd do with say VHS. I didn't realize this before as I do not "do games" nor I like watching other people playing (I never understood why would one look at someone playing a game).
In this case, follow the advice of IanB (two posts above), and see how to capture properly when audio gets out of synch.

[IanB]

Fundamentally you have 3 disparate clocks sources. The incoming video frame rate, the PC clock and the audio card capture clock.

The goal is to correct the sync between the audio and the video.

The problem is you have audio rate and video rate measured from the PC clock. To use a standardise video playback rate you need to translate the skew between the video and pc clock through to extra virtual skew between the audio and pc clock.

For example you have a VCR or Gaming console that when measured from the PC clock appear to be running at 24.99875 fps. We also measure the audio card capture clock from the PC clock and it appears to be running at 48002 Hz.

So if we choose a standard 25.0 fps playback rate we need to include the 25 to 24.99875 fps skew into the audio skew thusly 48002 * 25.0 / 24.999875 = 48004.40.

So now we have normalised the video clock skew to the PC clock to zero and have an aggregate skew between the PC clock and the audio clock of 4.40, which is the actual skew between the video and audio clocks, which is the value we need to work with in doing the sample rate and sync correction.

Which clock was correct does not matter, it is the relative skew between the video source and the audio capture clock that matters. As I said earlier using a good integrated audio video capture card handles this automatically. So for the above video source the observed clock rate of an integrated audio clock should have been 47999.76 Hz

[creaothceann]

Even a real SNES will have slight differences in its video and audio rates. Real oscillators are affected by temperature etc. so their frequency varies from the norm.

PC monitors and sound cards don't run at exactly 60fps and 32/44.1/48/...kHz either. Emulators like ZSNES or SNES9x will drop or repeat a frame after some amount of time. The solution that byuu (author of higan, formerly known as bsnes) came up with is to dynamically resample the audio (http://board.byuu.org/viewtopic.php?f=8&t=3960&p=89174#p89174) (if VSync is enabled).

The only place you'll get perfect sync is in the world of mathematics and theoretical physics.

[raffriff42]

Capture timing and capture sync - VirtualDub blog (comment)
http://www.virtualdub.org/blog/pivot/entry.php?id=78

I worked on this problem in the late '90s for several years. The core problem is when you have more than one crystal, both trying to give you timing information -- there will always be some rate slew (CPU clock / video clock / sound card crystal / etc).

The only way to really solve this problem is to use a house clock, and make sure your audio and video gear all syncs off the same clock (for more on this, google "blackburst"). Needless to say, cheap consumer hardware won't go through the trouble of implementing this crucial feature, so we'll get wow and flutter from "compensating" software forever to come.

If you actually CARE about your multimedia, you need to get gear that can sync right, but expect an audio+video capture rig to set you back over $1,000. On the bright side, you'll be able to play back OUT with the same sync :-)

Jon

[zerowalker]

Totally missed all these posts.

But extremely interesting.

IanB, if i understand it correctly, for example.

If i capture from a VCR with PAL.

The audio should be 48khz, and the Video should be 25fps.

So i capture it without any sync to anything.

The result is, 25fps, But the audio is off sync, so if i sync the video to audio, it will be off, 24.9996 for example.
And as this is not correct, i will have to fix the audio, as the Video is supposed to be 25 as PAL is 25.

So in this scenario i will have to sync the audio to the video to get the correct video/audio.

Hope that's correct.


I have tried to make some experiments with my webcam, but even if i use it's internal microphone, it's of sync, it doesn't seem to record at 30fps, but a bit lower, so i am syncing the video to the audio there.
But i guess the webcam just have bad timestamps which causes even the internal audio and video to go off sync.

Cause as you say, when the Audio and Video are on the same device (not a PC though) they will often be synced to each other (take a video camera for example).


PC monitors and sound cards don't run at exactly 60fps and 32/44.1/48/...kHz either. Emulators like ZSNES or SNES9x will drop or repeat a frame after some amount of time. The solution that byuu (author of higan, formerly known as bsnes) came up with is to dynamically resample the audio (if VSync is enabled).

Funny that you mention that, i am currently investigating Higan a bit, and just got knowledge about the "repeat drop frame" thing.

raffriff42, i guess what's mentioned there is one of those "Master Clocks", and they seem to cost hell, and aren't really suited for "home work".


IanB, a question, as you never get an exact sync form something through the methods i use, and you have to fix it afterwards by syncing to it's intended format (fps/frequency).
Is there any Real loss of quality?

I mean, if there is dropped/repeated frames, then there is a real quality loss. But resampling the sync of audio, seems to be pretty transparent with a good resampler.
So is there anything else that's going on, or is it just there 2 things you need to think about?


EDIT:

Is this a correct way to sync audio to the video length, while maintaining it's samplerate?

b=36000000
AdjustDenominator=36000000
AR=AudioRate()
TR=Float(AR)*AR*FrameCount()/(AudioLengthF()*FrameRate())
AC=FrameCount/(Audioduration*FrameRate)
BB=int(b*AC)
AssumeSampleRate(1)
ResampleAudio(BB, AdjustDenominator)
AssumeSampleRate(AR)

Or can it be better, or is it way off?

[IanB]

From the bad old days when I used a Matrox rainbow runner and and a SBawe32 on a P-II 450 with a 40GB disk. I tried many many solution from that era and I found the best was to just capture the streams and collect a VirtualDub capture Timing Log (Capture -> Enable time log). I would import the log into Excel and use the Linear Regression (LINEST) to calculate the true video and audio rates and the initial start capture delay. My JVC deck ran slow, my Panasonic desk ran fast, my sound card ran fast, my P-II's clock ran slow. Now I use a usb capture device called a LiDiC it produces ~5Mb/s H.264 with stereo 48KHz AAC @192K, the audio clock is locked to the Video Hsync.

Fixing things after the capture allows you to choose much higher quality tools, during capture everything has to run in real time. And by not loading up the cpu during capture with "fix it" tools you can get a better capture (less dropped frames, less audio resync hunting).

And re-clocking the audio is so much easier and higher quality than trying to re-clock the video.
The goal is to correct the sync between the audio and the video.
Which clock was correct does not matter, it is the relative skew between the video source and the audio capture clock that matters.

[creaothceann]

i am currently investigating Higan a bit

This (http://web.archive.org/web/20130315081624/http://byuu.org/bsnes/accuracy) might be interesting (unfortunately much of the site's content is offline because of a recent server move).

[zerowalker]

I found the best was to just capture the streams and collect a VirtualDub capture Timing Log (Capture -> Enable time log). I would import the log into Excel and use the Linear Regression (LINEST) to calculate the true video and audio rates and the initial start capture delay.

Interesting, will look at this.
Sadly i am not all that familiar with how Excel works, i used it in school a bit, but it was just to make a diagram of thing, (like votes, which one got highest, that kind of stuff).

Fixing things after the capture allows you to choose much higher quality tools, during capture everything has to run in real time. And by not loading up the cpu during capture with "fix it" tools you can get a better capture (less dropped frames, less audio resync hunting).

That's very good to hear, as before i knew about this, i always captured with Virtualdub using dynamic resampling, and i think it has it's drawbacks, as it doesn't know the true difference (like the entire video vs audio), it just changes in realtime, which isn't always that accurate i think. And as you say, it's very limited in quality, you can't choose what you want etc, and it takes CPU cycles.

And re-clocking the audio is so much easier and higher quality than trying to re-clock the video.

how do you even re-clock a video?

I mean, if i capture a PAL VCR band, shouldn't this script be a way to perfectly sync it (if there is no capture delay).

CLIP
AssumeFPS(25, sync_audio=false)
ConvertAudioToFloat()
b=36000000
AdjustDenominator=36000000
AR=AudioRate()
TR=Float(AR)*AR*FrameCount()/(AudioLengthF()*FrameRate())
AC=FrameCount/(Audioduration*FrameRate)
BB=int(b*AC)
AssumeSampleRate(1)
ResampleAudio(BB, AdjustDenominator)
AssumeSampleRate(AR)

//Is this possible, or worthless

AdjustDenominator=ContinuedNumerator(PI())
b=ContinuedNumerator(PI())

////
Oh and btw if you didn't see that i edited my post.
Can this be improved (the audio sync to video length), as i pretty much copied a script i found from you, and edited it to fit into the new things you have allowed (AdjustDenominator etc).

And also, sadly, by doing this, i am limited to Avisynth's resampler, while it's not bad, atleast while working in Float. There is better.
Is it possible to make use of another resampler somehow?

This might be interesting (unfortunately much of the site's content is offline because of a recent server move).

Will look into that, thanks:)

[IanB]

With double precision, your adjustment calculation might be right, but Avisynth script floats are only 32 bit IEEE single precision. So you will probably hit a wall like you did earlier in this thread. I suggest you do all your calculations with integer arithmetic. Google for Rational Arithmetic.

Also VirtualDub capture is at the mercy of the video and audio capture drivers, so even though both are instructed to start and stop capturing at the same time sometimes they invariably do not. IIRC on my old P-II the Matrox driver could take up to 200 milliseconds to respond to both the stop and start commands. So yes the streams were different in length, but mostly not from the clock skew.

[zerowalker]

With double precision, your adjustment calculation might be right, but Avisynth script floats are only 32 bit IEEE single precision. So you will probably hit a wall like you did earlier in this thread. I suggest you do all your calculations with integer arithmetic. Google for Rational Arithmetic.

Double precision??

Do you mean that as Avisynth works in 32bit IEEE single precision (what does single/double precision mean?) the numbers it can work with is limited in this task?

I googled Rational Arithmetic, but i don't really understand it, i am not good at math at all sadly, and programming, even though i try to be.

But i am guessing you it's some kind of highest int number that Avisynth can work with, or something?

Also VirtualDub capture is at the mercy of the video and audio capture drivers, so even though both are instructed to start and stop capturing at the same time sometimes they invariably do not. IIRC on my old P-II the Matrox driver could take up to 200 milliseconds to respond to both the stop and start commands. So yes the streams were different in length, but mostly not from the clock skew.

Interesting, has happened to me on some occasion that the delay will be wrong, and the only way to solve it is to manually adjust the audio/video start.
Does the timing log record all this stuff, so you know exactly what's going on so you can fix it afterwards based on the log?

also.

AdjustDenominator=ContinuedNumerator(PI())
b=ContinuedNumerator(PI())

With using this, i am not that mathematical as i said earlier.
but i do know that PI is infinite, so will this be the biggest precision that Avisynth can offer with int?
(this can probably be way out of hand, as said, i am not that good with int math and stuff, so sorry if i am talking crap here)

[Ghitulescu]

If i capture from a VCR with PAL.

The audio should be 48khz, and the Video should be 25fps.

So i capture it without any sync to anything.

The result is, 25fps, But the audio is off sync, so if I sync the video to audio, it will be off, 24.9996 for example.
And as this is not correct, i will have to fix the audio, as the Video is supposed to be 25 as PAL is 25.

So in this scenario I will have to sync the audio to the video to get the correct video/audio.
You got pretty much the idea.
Now, almost every card can synch itself to VHS, provided the VHS signal is steady (eg ran through a TBC). I am not talking here about the possible bottlenecks that might hinder this, like the speeds of the CPU, memory, HDDs. The PC clock is not important here.
The failure to synch to VHS (I keep saying VHS, but in fact I refer to any other analogue source like camcorder) is immediately noticed by most people, thus the main time has to be provided by the video. We also know that VHS has a fixed and reproducible fps of 25Fps (or 50fps), or a granularity of 20 millisecond.
On the other hand, the audio is more forgiving. At normal sampling rates (44k1 or 48k) the granularity of audio is of only ~50 microseconds. People won't miss one audio frame, as they hardly notice a 20ms lipsynch, provided the dropped audio frame was correctly removed (no "Click" no "Pop"). And even if the audio has to be resampled (eg from 44k1 to 44056) this is far from being objectionable, as it would be the interpolation of video frames to match 25Fps.
Therefore audio has always to be synched to video, not viceversa. Again the PC clock is not at all important.

The PC clock becomes important however if the playback would take place on the PC. The PC clock would also become important if you've chosen to use time codes, as these are given by its clock.

[IanB]

@zerowalker, You really need to find some way to come to grips with the arithmetic here. Maybe a friend or college can explain it enough to get you started. And Google is your friend!

Double precision is a 64 bit floating point representation.

More than highest number, it's the smallest difference that can be represented.

If you have the required numerical skills, you can extract the required information from the log.

You said you knew how to do graphs with Excel. Well do a short capture with a log and graph the audio and video timestamps versus real time. You will get 2 roughly straight lines, jitter and drops will distort the lines. Where the lines intersect the 0 real time axis is the actual start delay for each stream, the slope of the lines is determined by the true audio and video rates. Methods like Linear Regression allow you to fit a "best" straight line to a set of jittery data points. With the "best" straight lines you will have the 2 startup delays and the 2 rates needed to sync the audio and video tracks. I used this method very successfully but it was well more than 10 years ago.

If you cannot get a grip on the maths involved here, perhaps you should consider getting a video capture device with synchronised and integrated audio capture.

=================================================

As for the continued fraction stuff, Pi=3.1415926535897932384.... this is a representation to 20 digits.

If we examine under the covers Avisynths version of PI()=3.141592741 which matches for the first 7 digits, which is the 6 to 7 digits you expect for a 32 bit float.

Using the continued fraction functions we can get a rational pair of integers that match the precision of the underlying 32 bit floating point number :-

ContinuedNumerator(PI())=93343
ContinuedDenominator(PI())=29712

And if we resolve the rational pair back to a 32 bit floating point number we get the original value back.

Float(93343) / 29712 = 3.141592741

But if we resolve the rational pair back at a higher precision we can see a deviation :-

93343 / 29712 = 3.1415926225094238018...

[zerowalker]

@zerowalker, You really need to find some way to come to grips with the arithmetic here. Maybe a friend or college can explain it enough to get you started. And Google is your friend!

Will try with google, as sadly, no one of my friends or that mathemetical;P
But hopefully i will grasp it sooner or later!

I will also do some tests to see if i get the Excel timing log, from your explanation, i think i should grasp it.

And with using AdjustDenominator=(ContinuedNumerator(PI()))
in the calculation for syncing the audio to video, i took it as it was very high.

Cause isn't a higher number better?
Meaning, the highest possible Integer number that IEEE 32bit allows, will be the highest precision it allows when calculation this, isn't that correct?

So can't i just use AdjustDenominator="highest int number", or something like that, if you get what i mean here.


AR=AudioRate()
TR=Float(AR)*AR*FrameCount()/(AudioLengthF()*FrameRate())
KScale=Int($7FFFFFFF/ TR)
ResampleAudio(Round(TR*KScale), KScale)
AssumeSampleRate(AR)

Here is the original script i took the idea from, which you posted.
And this one is before you fixed the limitations when working with small samples.


If i use function to get the correct AdjustDenominator/Numerator, i don't need any real knowledge about that mathematical going on.
Though of course i do want to know what's going on.

[pbristow]

It's horrifying to me that domestic capture devices are still unable to properly sync to a typical analog video source, given how many of them have been sold for the express purpose of hooking up to an existing VCR and "preserving your precious video memories" - a job that usually involves capturing an entire 2, 3 or 4 hour VHS tape in one go!

All that's required is for the hardware to derive a "theoretically 48kHz" audio clock from the incoming video line frequency, either directly through a PLL and divider network, or through a crystal-clocked timer that adjusts the rate of the audio clock. That way, if the video source is running slow (say), then the audio capture clock is slowed to match, and every block of 48000 audio samples stays reasonably well matched up to the corresponding second (25 frames) of video. Or, you could just over-sample the audio at a simple multiple of the line frequency (e.g. 8x) and then resample it down to the required rate. (With that much oversampling you can simply drop the unwanted samples, as any resulting jitter artifacts will be ultrasonic anyway. Let the audio purists pay for their expensive versions with "proper" interpolation, if they wish... ;) )

None of these should be expensive circuits to produce, if they were churned out en masse. Is there *really* no domestic or sub-£200 gear that does this?!?


Example:

25fps x 625 lines/frame = 15625 lines/second,
So the incoming (PAL) video is expected to contain Hsync pulses at a rate of 15625 Hz, but with some expected variation.

Multiply this by 8x using a PLL: 125kHz (but with the same, proportional variations). As long as the range of speed variation is les than about 5%, a PLL should have no problem tracking it - And I don't know of any VCR that runs that badly under/over speed!

Use this derived 125kHz signal to clock the audio capture, then downsample the audio by a fixed factor of 5.208333333... And hey presto! Audio is output at (theoretically) 48kHz, with the actual frequency from moment to moment automatically following any variations in the incoming video. You end up with a file on disk with an average of 48000 audio samples captured for each 25 frames of video, and with no individual moment of audio being more than a few samples (a tiny fraction of a frame) adrift of where it should be.

[zerowalker]

pbristow, indeed, it's very weird how the consumer market has been suffering with this all this time.


IanB, i am trying to figure out the higher int number i can use in AdjustDenominator, though i am not coming that far.

in a clip that has:

Audio length: 26422399 samples

I can use: 1721028160, anymore will break the calculation.
But this number is only correct for this clip, i am trying to find a way to get a function that works for every clip.
I am guessing the audio length has something to do with the number.

[IanB]

@pbristow,

Most integrated devices that have both audio and video connectors in to the devices are well behaved and lock the audio clock to either the video line (15625Hz/15734.27Hz) or field rate (50Hz/59.94Hz).

The problem comes when people try to use a free standing sound card for the audio capture. (Watch for capture cards that internally link to the free standing sound card).


@zerowalker,Audio length: 26422399 samplesA piece of string is 5 long :devil: (you have only provided part of the information).

From the context of past posts, we can assume you want to re-sample the audio track to make it the same length as the video track. Okay accepting the limited accuracy of a single precision float, this will give you the best pair of integers from the floating point ratio :-AR=AudioRate()
TR=Float(AR)*FrameCount()/(AudioLengthF()*FrameRate())
ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)Be aware that floats only have 24 bits of precision, so the finest ratio that can be represented is 8388608 / 8388607 = 1.00000012 or 8388607 / 8388608 = 0.99999988

[zerowalker]

A piece of string is 5 long (you have only provided part of the information).

Well what i meant was, the highest int number that can be used , for the highest precision, is limited depending on the audio samples, i think (you probably know this better than me).

From the context of past posts, we can assume you want to re-sample the audio track to make it the same length as the video track. Okay accepting the limited accuracy of a single precision float, this will give you the best pair of integers from the floating point ratio :-

Yes, that function just aims for clocking the audio to the video length. And for that i would like to have the highest possible precision.

And i know that Avisynth is limited to using Int for calculating the variation in the end.


AR=AudioRate()
TR=Float(AR)*FrameCount()/(AudioLengthF()*FrameRate())
ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)


Though from this, you only use Float?
It "works" when i tried, meaning it didn't complain on the float (or maybe it's int?). But the samples are way of, it's making a 7 hour clip becoming 180ms.
Fixed it, i forgot Assumesamplerate(1)!

Or maybe i have missunderstood something here?

[IanB]

I would not have thought scaling the audiorate would make any difference, 0.99999988 or 47999.99424 both have 24 bit precision. I will have to think on this for a bit.

Ding! :sly: Ah yes it will make a difference, because using unscaled will crimp the style of the 32 bit rational denominator, by almost 15.5 bits.AR=AudioRate()
TR=FrameCount()/(AudioDuration()*FrameRate())
AssumeSampleRate(1) # Scale to maximise available precision
ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)

7 Hours is 1209600000 samples, 180ms is 8640 samples. As float 1209600000.-8640. = 1209591296. or 1.209591296E9 not the anticipated 1209591360 an error of 64 samples or 1.33ms.

[zerowalker]

Okay wait, this is getting ahead of me, let's see if i get it correctly.

Ding! Ah yes it will make a difference, because using unscaled will crimp the style of the 32 bit rational denominator, by almost 15.5 bits.

So, by using AssumeSampleRate(1), it will increase the available precision, while not using it, will decrease it by "15.5bits"?

AR=AudioRate()
TR=FrameCount()/(AudioLengthF()*FrameRate())
AssumeSampleRate(1) # Scale to maximise available precision
ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)

Is this function the highest possible precision that avisynth allows for calculation the length of video/audio?
Is ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR)) the highest number that can be used?

7 Hours is 1209600000 samples, 180ms is 8640 samples. As float 1209600000.-8640. = 1209591296. or 1.209591296E9 not the anticipated 1209591360 an error of 64 samples or 1.33ms.

Not sure what you mean here, do you mean that without Assumesamplerate(1) i will generate the error instead of the assumed correction?
Meaning if i calculate a 7 hours audio to a video, i will get 64 samples wrong.?

[IanB]

So, by using AssumeSampleRate(1), it will increase the available precision, while not using it, will decrease it by "15.5bits"?Of the denominator!

Because we are talking about numbers around 1, to maximise the available precision of the rational pair we want both numerator and denominator to be the same order of size. Including a factor of 48000 in the numerator reduces the range of denominator values to MAXINT/48000 = 44739.
Is this function the highest possible precision that avisynth allows for calculation the length of video/audio?
Is ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR)) the highest number that can be used?
I have already answered this, and with an example. Please go and learn about floating point numbers and the precision of rational numbers. If you use single precision floating point numbers, the precision is 24 bits. If you use integer arithmetic with rational pairs you can calculate an exact result, but that result may not fit within a pair of 32 bit integers, however you can use the continue fraction method to calculate the closest pair of 32 bit integers.
Not sure what you mean here, do you mean that without Assumesamplerate(1) i will generate the error instead of the assumed correction?
Meaning if i calculate a 7 hours audio to a video, i will get 64 samples wrong.?With 32 bit floating point numbers the result is only accurate to 24 bits. For a 7 hour stream than means approximate to within 64 samples. Including a relativly large scale factor in the numerator will further reduce the available precision.

[zerowalker]

Of the denominator!

Because we are talking about numbers around 1, to maximise the available precision of the rational pair we want both numerator and denominator to be the same order of size. Including a factor of 48000 in the numerator reduces the range of denominator values to MAXINT/48000 = 44739.

Sorry, as said, i am bad at this.
I am reading around like mad and trying out stuff, i hope i am slowly getting more knowledge.
It's hard with the mathematics i have never used, especially when the words are in english (i am from Sweden), so there is some problematic there as well.

I have already answered this, and with an example. Please go and learn about floating point numbers and the precision of rational numbers. If you use single precision floating point numbers, the precision is 24 bits. If you use integer arithmetic with rational pairs you can calculate an exact result, but that result may not fit within a pair of 32 bit integers, however you can use the continue fraction method to calculate the closest pair of 32 bit integers.

Really;S
Sorry once again. I do know a bit about floating point from my programming classes. But the thing i know is pretty much, Float is much higher then int. that's pretty much it sadly, will have to really read up on the rational numbers.
Really need to know how to calculate with integer arithmetic, as if what you say is true (which it is), i can use integer arithmetic with rational pairs, get the exact value, and then use "continue fraction method to calculate the closest pair of 32 bit integers", and that will be higher precision than floating point, as it's 24 bit.

I did a bit of reading on that function, and i guess that's 24 bit single precision, as you stated in an earlier post (was pretty focused that it used Float when it must be int in the Numerator/Denominator).

With 32 bit floating point numbers the result is only accurate to 24 bits.

This proves how little i know.
As when i read this, it doesn't make much sense to me.

32 bit is only accurate to 24 bit.

So once again, i must really read up on the floating point.

Thanks as always, sorry for wasting your time with worthless questions when i don't get some stuff.

[IanB]

Back in post #119 there was an error I used AudioLengthF instead of AudioDuration, I have edited the post to make it right. :o

From Sweden, I would never have guessed, your written English is superb, better than most native speakers on this board. ;)

If we try a simplistic approach we run into integer overflow for quite small clips, less than 30 minutes for pal.# Typical values and limits
# FrameCount() 900000 for 10 hours with pal
# FrameRateNumerator() 25 or 50 for pal, 30000 or 60000 for ntsc
# FrameRateDenominator() 1 for pal, 1001 for ntsc
# AudioRate() 44100, 48000, 96000, 192000
# AudioLength() MAXINT for 12.4 hours at 48000
#
AR=AudioRate()
TRnum=AR * FrameCount() * FrameRateDenominator() # Goes bang at frame 44740 for pal, 45 for ntsc
TRden=AudioLength() * FrameRateNumerator() # Goes bang at 00:29:49.5 with pal, 00:00:1.5 with ntsc
AssumeSampleRate(1)
ResampleAudio(TRnum, TRden)
AssumeSampleRate(AR)
Lets dust of some mathemagical tricks to see what we can squeeze out.
AR=AudioRate()

VDnum=FrameCount() * FrameRateDenominator()
VDden=FrameRateNumerator()

ADint=AudioLengthHi(AR) # AudioLength64 / AudioRate
ADrem=AudioLengthLo(AR) # AudioLength64 % AudioRate

Snum=MulDiv(VDnum, AR, MaxInt)
Sden=MulDiv(ADint * VDden, AR, MaxInt)
Scale=Max(Snum, Sden) + 1 # Divisor to ensure MulDiv does not overflow

TRnum=MulDiv(VDnum, AR, Scale)
TRden=MulDiv(ADint * VDden, AR, Scale) + ADrem * VDden / Scale

AssumeSampleRate(1)
ResampleAudio(TRnum, TRden)
AssumeSampleRate(AR)
For a 12.4 hour clip Scale will be 26, which results in precision to within 38.4 ms, which is not quite as good as the floating point version, using continued fraction technology might improve on this.

The above code is untested!

[zerowalker]

From Sweden, I would never have guessed, your written English is superb, better than most native speakers on this board.
Well thank you. Though i use the inbuilt spell check on my browser to prevent bad spelling, as i am crappy on that part.

If we try a simplistic approach we run into integer overflow for quite small clips, less than 30 minutes for pal.

Luckily i know the normal stuff, audiorates/framerates and framecount, don´t get what you mean here though: # AudioLength() MAXINT for 12.4 hours at 48000. Do you mean that at 12.4 hours on a PAL video (25 fps) the audiolength if 48khz is used will be the maximum integer allowed?

And what does that code do, is it to show the limitation of Int, meaning when a certain length is reached with a certain framerate, it goes out of the boundaries for which is allowed, breaking the calculation?


For a 12.4 hour clip Scale will be 26, which results in precision to within 38.4 ms, which is not quite as good as the floating point version, using continued fraction technology might improve on this.

The above code is untested!

Just to see if i understand it correctly here.

When you say, precision within 38.4ms, do you mean that if the real answer it for example, 55.9 ms, it will off that answer by 38.4ms?
If that's what i means, it's very accurate. How accurate will floating point be then, as that's supposed to be less accurate then Integer with Rational Numbers with Continued Fraction.

Also, i can't test the code, i don't know what i am supposed to do with "MaxInt", there is no such function, so am i supposed to calculate or know the MaxInt on my clips?

And, i have done some testing, and noticed something,

If i use:

TR=float(AR)*AR*FrameCount/float(AudioLengthF*FrameRate)
it seems to be either more accurate, or less acurate, then when using

TR=float(AR)*FrameCount/float(AudioLengthF*FrameRate)
Assumesamplerate(1)

i actually think it's more accurate, though you can probably see if it is or not.


And hate to bring up Excel, but i can't get the graph to work as intended.
I can get 2 lines, but Audio and Video aren't even close to each other, the Audio goes way faster than Video, but they end at roughly the same size.
It seems that i was able to calculate the estimated framerate though, by taking the framecount / Vglobaltime (the last in the log), and then i would get the frame rate, then i could use that to recalculate the difference in the audio, and resample it to sync to 30fps.

Though it seems it doesn't work every time, so it may not be accurate at all, and simple just happens to be close, but this is just me testing.

[creaothceann]

MaxInt is 2^(32-1), i.e. 2147483648.

[zerowalker]

Okay thanks, added: MaxInt=Int(2147483648). Hopefully that's correct.
But the code still doesn't work as intended i think, 9 hours becomes 40 hours.
But either i am doing it wrong, or it's cause of it being untested.

[creaothceann]

Why the Int() ?

[zerowalker]

If i didn't use it, the script didn't work at all.
I think the maxint changes depending on the clip. I think that the longer the video, the less size is the MaxInt.
Not sure though, but it atleast seems to matter for some reason.

[Gavino]

MaxInt is 2^(32-1), i.e. 2147483648.
MaxInt (meaning the largest positive integer representable in signed 32 bits) is actually 2^(32-1)-1, or 2147483647 ($7FFFFFFF)

If 2147483648 is used in a script, it is converted to float since it is too big to fit in an int.

[zerowalker]

Ah that explains it, now it works.

Though it seems to be quite limited (guess that was the purpose?), and breaks on to long videos.

Also: Timing Log (http://www.sendspace.com/file/i79wfz)

I am trying to use this timing log to calculate the sync issue (in this particular capture).
But i am not able to do it, i have done an estimated calculation through listening, and that is:

ConvertAudioToFloat()

AR=AudioRate()
TR=float(AR)*(32260240)/(32400000)

ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR)) #AssumeSampleRate(1)
AssumeSampleRate(AR)


But that is just from me listening and calculating from that.
There must be a Correct way to do it, atleast when a timing log is available, but sadly i don't see it.

[IanB]

precision within 38.4ms,Uncertainty of the answer, the result can be off by up to that amount in either direction. It might be right, it might be low, it might be high, but it will be within that range.

And yes MAXINT is $7FFFFFFF

And geez I said start small. 46.6 MB capture log, 9 hours 23 minutes, 1012250 frame capture and NTSC not PAL like you have been implying.


Linear regression [y=mx+b] of the first 65k lines of the log. (The PC I am borrowing only has Excel 2002.)

=INDEX(LINEST(C2:C65536,A2:A65536,TRUE,FALSE),1), M value is 33.47367754
=INDEX(LINEST(C2:C65536,A2:A65536,TRUE,FALSE),2), B value is 55.7123293

=INDEX(LINEST(H2:H65536,G2:G65536,TRUE,FALSE),1), M value is 0.010415426
=INDEX(LINEST(H2:H65536,G2:G65536,TRUE,FALSE),2), B value is 15.8849696

Video capture started at 55.7123293 ms and we have 33.47367754 ms per frame.

Audio capture started at 15.8849696 ms and we have 0.010415426 ms per byte.

So the audio capture started 39.8273597 ms before the video capture.

Use a DelayAudio(-0.0398273597)

The measured video rate is 29.87421979 fps, the measured audio rate is 48005.717673 Hz.

For a standard NTSC rate of 30000/1001 fps the relative audio rate is 48159.6777261 Hz.

So the correction factor should be 0.99668441041

[zerowalker]

Uncertainty of the answer, the result can be off by up to that amount in either direction. It might be right, it might be low, it might be high, but it will be within that range.

And yes MAXINT is $7FFFFFFF

Great:)

And geez I said start small. 46.6 MB capture log, 9 hours 23 minutes, 1012250 frame capture and NTSC not PAL like you have been implying.

Sorry for that, i do have lower, why i uploaded a "massive" capture log was cause i thought, the longer the clip, the higher precision could be made.
And it's a Webcam, it's not PAL or NTSC really, it's 30fps (not 29.976), though Avisynth says it´s 30/33333333 or something like that, not sure which is correct there.

(why i used a webcam was just for testing, as it was an own device "outside" of the PC, and using a separate mic, so i could produce a similar scenario as when capturing other stuff).

Linear regression [y=mx+b] of the first 65k lines of the log. (The PC I am borrowing only has Excel 2002.)

Not really sure how you even do that, but will try to find my way there.

Cause from your information, i can get all the information out of that log, making a perfect sync. I am Amazed!


EDIT:

Okay was able to understand the function now, took some time.

But i wonder, is it possible for the audio to start at a negative number?
I get this with a test clip: -65.4628313190

Also, how do you calculate the relative rate of audio/frame?


EDIT 2:

Okay i got how you calculate the Framerate, 1000/(Frametime).
But i can't find anything about calculating the sample rate from ms and bytes:(

[IanB]

Because Linear Regression is about fitting a single straight line, y=mx+b, to a set of data points, such that the line gives the lowest overall error, a negative "b" is always possible.

However given the way capturing works a negative "b" is unlikely and I would be double checking my maths very carefully.

From the audio format you need to know how many bytes per sample to calculate sample rate from bytes and milliseconds.

For 16 bit mono it is 2 bytes per sample, for stereo is is 4 bytes.

So 0.010415426 ms per byte is 0.020830852 ms per sample => 1000/0.020830852=48005.717673 samples per second.

[zerowalker]

Because Linear Regression is about fitting a single straight line, y=mx+b, to a set of data points, such that the line gives the lowest overall error, a negative "b" is always possible.

However given the way capturing works a negative "b" is unlikely and I would be double checking my maths very carefully.

From the audio format you need to know how many bytes per sample to calculate sample rate from bytes and milliseconds.

For 16 bit mono it is 2 bytes per sample, for stereo is is 4 bytes.

So 0.010415426 ms per byte is 0.020830852 ms per sample => 1000/0.020830852=48005.717673 samples per second.

It seems to be correct. Did another and that one was 13ms.
What am i to do with a negative number?
Am i supposed to calculate from negative to positive?
Example:

-60 on audio
40 on video.

Audio = 100 delay ?


And also, how did you get a factor from the wanted framerate compared to the relative sample rate?

EDIT:

I think i got it:

TR=float(AR)*("Relative framerate")/float(framerate)*(AR/"Relative audiorate")

But i don't know if i am supposed to Delayaudio before or after resampling. I guess before though.

[zerowalker]

=INDEX(LINEST(C2:B65536,A2:A65536,TRUE,FALSE),1), M value is 33.47367754



I Noticed that i overlooked this. Is this a typo?
Should it be B2:B... or C2:C...?
Cause i can´t get anything from C2:B...

I am making a VBA script that will post everything, making this alot easier, will post it here once i am done:)

[IanB]

If the LR gives a negative B it just means the "start time" needs to considered to be earlier in following calculations. So -60 and 40 is 100.

It could also mean the audio capture driver is not starting with an empty buffer, so the first chunk of data arrives much earlier than expected, because there are samples from before you pushed the start button. It does not matter because the methodology discard audio samples before the start of the video and stretches/squeeze the remaining audio track to make the ends match.

And yes you just include any video scaling into the audio scaling calculations.

The DelayAudio goes first, but doesn't really matter, the change between before or after is so minuet it's lost in the precision available. i.e. -0.0398273597 versus -0.0396953085 thus 132 microseconds.

C2:B... yes it's a typo, I have fixed the post above.

[zerowalker]

If the LR gives a negative B it just means the "start time" needs to considered to be earlier in following calculations. So -60 and 40 is 100.

It could also mean the audio capture driver is not starting with an empty buffer, so the first chunk of data arrives much earlier than expected, because there are samples from before you pushed the start button. It does not matter because the methodology discard audio samples before the start of the video and stretches/squeeze the remaining audio track to make the ends match.

And yes you just include any video scaling into the audio scaling calculations.

The DelayAudio goes first, but doesn't really matter, the change between before or after is so minuet it's lost in the precision available. i.e. -0.0398273597 versus -0.0396953085 thus 132 microseconds.

C2:B... yes it's a typo, I have fixed the post above.

I didn´t quite get what you meant there.
If the audio has samples from before i started, and those are discarded, won´t that alter the way i should calculate the Delayaudio?
Shouldn´t i calculate as the audio was started as 0ms then?

And good to hear, will use DelayAudio before, even if it doesn't matter, it´s "better" to do it correct:)

Okay, good, was surprised how it was supposed to work if it wasn't.

But is is really C, i though B was the captured time frame, and C was the "wanted" time frame?

So shouldn't i compare the Audio to the actual video capture?



And here is the Macro:)
Took quite some time as i didn't understand the scripting that well, and got help with it.

It is extremely poorly optimized, and you don´t want to see the code;P

But the results should be precise.

Though i am trying to solve that Excel keeps truncate calculations, making them a bit less precise than they can be.

But still, from my testing, i can keep anything in sync by following the script (at least with what i have added).

Please try it, and if you have any thoughts of it, please tell.

btw: Webcam is my Webcam's FPS, which is 30.0000 (10000000/333333).
Also, i don´t know how to calculate 24 bit, so it´s probably not accurate at all*

[IanB]

The Linear Regression is about fitting a single straight line, y=mx+b, to a set of data points, such that the line gives the lowest overall error.

Note! Lowest error not necessarily zero error.

When processing the .CSV logfile be careful, the audio data ends much earlier than the video data, do not include the ",,,,," cells in your audio LINEST range.

If you get a negative B's just fold them into the calculations.

If you get an Audio B later than the Video B just fold it into the calculations.

The method is right. It just handles the numbers.

Your Webcam's FPS is purely nominal. It is stating 10000000/333333=30.00003 fps but you are measuring 29.87421979 fps. The claimed 30fps is bullshit.

Assign the most sensible rate for you final play back, i.e. 30000/1001 (ntsc) or 25/1 (pal) and scale everything to suit.

The 2 video times, B & C, are the lie reported by the driver and the actual time measured by VirtualDub. Analysing the driver reported time and comparing it measured time can be instructive when looking for dropped or duplicated frames.

[zerowalker]

The Linear Regression is about fitting a single straight line, y=mx+b, to a set of data points, such that the line gives the lowest overall error.

Note! Lowest error not necessarily zero error.

I see, why isn´t it possible to get Zero error?
Is it because of the limitation of numbers that Excel allows, or how Linest works, or simply the log that Virtualdub makes?

When processing the .CSV logfile be careful, the audio data ends much earlier than the video data, do not include the ",,,,," cells in your audio LINEST range.

I don´t understand this. I know that the Audio data ends much earlier, which is why i had to make a script that looks for every cell Until it´s blank. I guess that correct´and ",,,," = blank?

If you get a negative B's just fold them into the calculations.

If you get an Audio B later than the Video B just fold it into the calculations.

So, no matter if it´s negative or positive, i shall always look for the difference, (Lower audio = - delay, higher audio = + delay)?
Example: video start = 10ms, Audio start 100 ms, , that´s 90 ms Later, meaning Delayaudio(0.090) - Actually when i look at this, it makes more sense in Delaying audio with + if it´s starting earlier to make it the same as the Video, but i guess that´s wrong.

The method is right. It just handles the numbers.

Great:)

Your Webcam's FPS is purely nominal. It is stating 10000000/333333=30.00003 fps but you are measuring 29.87421979 fps. The claimed 30fps is bullshit.

So, the 30fps it´s saying it will do, is actually just wrong?
But if i change the FPS, the video will become longer/shorter, won´t that make it, inaccurate?

The 2 video times, B & C, are the lie reported by the driver and the actual time measured by VirtualDub. Analysing the driver reported time and comparing it measured time can be instructive when looking for dropped or duplicated frames.

When you say "lie" is that only for this device cause you said the "30" fps is way off?
And so it´s possible to compare B and C to look for dropped frames?

[IanB]

You cannot get zero error because the data is bumpy, it has jitter.

If the data was 10, 20, 30, 40, 50, 60, ... i.e exactly 10 each time then it would be possible.

But the data is like 9, 21, 29, 42, 49, 50, ... i.e sort of 10 each time but with jitter. You can get a best line but not all the data points are on that line.


For the ",,,," if you look at the .csv in notepad you will see about 1/15th down the file the audio columns become ",,,," which excel loads as blank cells. For video you get a line every frame, for audio you get a line every ~0.5 seconds.


Yes, lower audio = - delay, higher audio = + delay


The 30fps is nominal. i.e. the value specified and aspired to. In real life it is something close, measured at 29.87421979 fps using your PC's clock as a reference. Maybe it is truly 30.0 fps and your PC clock is wrong, maybe the driver code is slack. It does not matter. We just want to match the video and the audio. Then we want to play them back and again we need to specify a useful nominal frame rate. If you are smart you choose a nominal frame rate the same as everybody else does, i.e. 30000/1001 (ntsc) or 25/1 (pal).


I say lie because many capture drivers return bogus numbers. Your webcam driver appears to return a version of the PC clock, and given it is a webcam that is probably expected. The webcam driver probably uses the PC clock to wait for an interval of 1/30 of a second and then takes some 0.0001403442 seconds to give some instructions to the camera and have the camera respond with a frame and then simplistically waits for another interval of 1/30 of a second, hence the slightly slow measured 29.87421979 fps.

A helpful capture driver would return the count for the internal PLL clock synchronised to the analogue video sync pulses.

[zerowalker]

You cannot get zero error because the data is bumpy, it has jitter.

If the data was 10, 20, 30, 40, 50, 60, ... i.e exactly 10 each time then it would be possible.

But the data is like 9, 21, 29, 42, 49, 50, ... i.e sort of 10 each time but with jitter. You can get a best line but not all the data points are on that line.[/QUOTE

I see, isn´t it possible to make the log jitter free?
By increasing the intervals on the logging or something?

[QUOTE]For the ",,,," if you look at the .csv in notepad you will see about 1/15th down the file the audio columns become ",,,," which excel loads as blank cells. For video you get a line every frame, for audio you get a line every ~0.5 seconds.

Ah, well that is solved, the script ignores the blank cells.

Yes, lower audio = - delay, higher audio = + delay

Great:)

The 30fps is nominal. i.e. the value specified and aspired to. In real life it is something close, measured at 29.87421979 fps using your PC's clock as a reference. Maybe it is truly 30.0 fps and your PC clock is wrong, maybe the driver code is slack. It does not matter. We just want to match the video and the audio. Then we want to play them back and again we need to specify a useful nominal frame rate. If you are smart you choose a nominal frame rate the same as everybody else does, i.e. 30000/1001 (ntsc) or 25/1 (pal).

Well it matters also that i get hte correct time. For example, if i record, and i say "hi" at 140 seconds when i record. And on playback i have synced to NTSC, it will be off.
So the nominal is the most "correct", though it´s just this scenario, as if what you say how the capture driver acts. The this isn´t really something that i will run into.

The webcam was also just something i could capture to try with, as i currently don´t have any VCR and stuff up and running, other capture devices normally have something they are supposed to be running at (PAL = 25 etc).

I will go with syncing to the "Capture framerate" with the webcam, as if my thought are right, that will be the correct way if i would like to sync to something else.

For example:

I Have a video i record which is 25 fps. And along with that, i record webcam and talk about the video.

So, to make those synced to each other, the 25fps will have to be 25fps, and the webcam would have to be "webcam fps", and of course the audio should be synced to the respective device.

Hopefully i explained that correctly;P

[IanB]

No the jitter is in the real world. The log records when things actually happened.



Yes you can sync two clips the same as you sync the audio and video of one clip.

Syncing a 25 fps pal with a ~30 fps webcam will be a challenge. You need to make them both the same first then fine adjust the sync.

[zerowalker]

No the jitter is in the real world. The log records when things actually happened.

Don´t quite understand this,
Is there no jitter on the actual recordings, but when logging it appears compared to what´s expected?

Syncing a 25 fps pal with a ~30 fps webcam will be a challenge. You need to make them both the same first then fine adjust the sync.

It should work fine as long as i use the same Fps as the capture driver wants on both videos, and sync the audio to both videos.
The i should just need to put them together and adjust the timing (delay) of the two, then they should run in sync with each other (except the difference between the frames, as 30fps is more then 25, so 30 should be more precise per ms, but it´s negligible)

[zerowalker]

Okay, i am having a bit of a problem when doing calculations.

I was currently using a way to invert a calculation, but it seems it isn´t accurate for some reason.

For example:


25/24 = 1.041666666666667
1 / 1.041666666666667 = 0.9599999999999997
1 / 0.9599999999999997 = 1.041666666666667

There it works perfectly, i can invert it back and forth.
But now the problem appears.

24/25 = 0.96
1 / 0.96 = 1.041666666666667
1/1.041666666666667 = 0.9599999999999997

I get the same results afterwards, but they must be incorrect.
As by inverting, i should get from 1.04..... to 0.96, but instead i get 0.9599999999999997.


I am not mathematical so it won´t surprise me if there is a logical explanation for this.

[zerowalker]

Hate to open an old topic, but easier than starting a new and explaining.

I would like to know, is the code that stretches the audio portable to c++?

No programming skills to mention though, but as it's very stable and works as expected, i hoped that i could learn something from it so use in , well c++ for example.

Thanks

[foxyshadis]

You mean ResampleAudio? The implementation is in audio.cpp (https://github.com/AviSynth/AviSynthPlus/blob/MT/avs_core/core/audio.cpp#L1166). Or do you mean the Excel line-fitting?

[zerowalker]

Probably, and wow that was complicated.

I can work with Audio in a float array, like stereo to mono, 32bit to 16bit, such stuff. This is on a completely different scale.

Thanks though:)