RFC: Sampling Theorem

Message Bookmarked
Bookmark Removed
Can someone help me with a difficulty I have with the sampling theorem please? Here is one of many slightly different formulations of it I've seen. Quote from http://www.national.com/an/AN/AN-236.pdf

"The sampling theorem by C.E. Shannon in 1949 places
restrictions on the frequency content of the time function
signal, f(t), and can be simply stated as follows:

In order to recover the signal function f(t) exactly, it is
necessary to sample f(t) at a rate greater than twice its
highest frequency component.

Practically speaking for example, to sample an analog signal
having a maximum frequency of 2Kc requires sampling at
greater than 4Kc to preserve and recover the waveform
exactly."


I understand that this is the basis for claims that CDs have 'perfect'
reproduction for all sound frequencies below 20k Hz.

My problem is that 10*Pie Hertz == 31.42... Hz so a sound of this frequency can be stored 'perfectly' on a CD,
yet Pie is an irrational number with non-repeating decimal expansion and so can't be written down in any finite amount of space.

Have I missed some assumption inherent in the theorem? Perhaps the theorem assumes infinite vertical resolution (in digital audio that would be bit depth), but even that doesn't intuitively seem enough.

What's up? This has literally been bugging me for years.

mei (mei), Tuesday, 18 February 2003 14:35 (twenty-two years ago)

My problem is that 10*Pie Hertz == 31.42... Hz so a sound of this frequency can be stored 'perfectly' on a CD,
yet Pie is an irrational number with non-repeating decimal expansion and so can't be written down in any finite amount of space.

? what does this have to do with the theorem above? there's nothing intrinsic about hertz as a unit, is there, so surely your argument would mean that you couldn't store sounds of any frequency 'perfectly'? there's surely nothing to stop me choosing to write 10\pi Hz as 1, for example.

toby (tsg20), Tuesday, 18 February 2003 15:07 (twenty-two years ago)

Hi Toby.

What it's got to do with the theorem is this: the theorem says you can store a 10pi Hz frequency 'perfectly'. But my maths tells me that would take infinite storage space, because pi is irrational.

I think you're suggesting letting 1 (or some other number) in binary represent 10pi.

In that case you could store 10pi Hz perfectly but not 10Hz, because the number 10 would require a non-finite number of decimal places if we wrote it in base-pi numbering.

To paraphrase, I don't think you can represent both 10pi and 10 in any base-x numbering system without one of them having infinitely many decimal places (which wouldn't fit on a CD).

mei (mei), Tuesday, 18 February 2003 19:59 (twenty-two years ago)

"To paraphrase, I don't think you can represent both 10pi and 10 in any base-x numbering system without one of them having infinitely many decimal places (which wouldn't fit on a CD)."

Exactly! But what happens when you put a 10pi and 10 frequency together? You get crossharmonics, of more subtle frequencies, and those are what get lost, but the argt. is that what is lost is too subtle for the human ear.

Sterling Clover (s_clover), Tuesday, 18 February 2003 20:11 (twenty-two years ago)


In order to recover the signal function f(t) exactly, it is
necessary to sample f(t) at a rate greater than twice its
highest frequency component.

recovering f(t) exactly with f'(t) doesn't mean you actually regain the original f(t)... you merely get an f'(x) = f(x) for x that are multiples of the period rate over the time range of sampling. if you sample fast enough, you won't miss a significant bounce anywhere in the true f(t) wave with f'(t). that doesn't mean perfection however. there's still serious possibility for variation of x1 < t < x2 where f(t) <> f'(t). the deal is though, that as x1-x2 gets smaller and smaller human ears can't hear the variation because f'(t) is robust enough to cover any terms that the ear might have lost if the sample rate had been slower.

anyways... the pi aspect of a frequency rate doesn't really matter. if it does, it matters probably far less than the clock imperfections on a circuit board. in other words, the sample rate isn't even perfect out to too many decimal points. well, not unless you're using some sort of atomic clock to time your shit.
m.

msp, Tuesday, 18 February 2003 22:14 (twenty-two years ago)

Hiya msp.

You're saying that the the sampled waveform exactly equals the real waveform at the times when samples are actually taken. (Assuming that samples aren taken instantaneously and that there's sufficient (infinite) vertical resolution (bit-depth)).
In between those times it's merely an approximation - the shorter the gap the better the approximation.

That is _exactly_ what I think. But the sampling theorem seems to say that the waveform reconstructed from the samples is _perfect_ during the in between periods also. Which I don't believe.

This is not a practical question, we can assume that the clock is perfect, we have as much bit-depth as we want, instant sampling, pefect circuit boards etc. I just don't believe you can store and infinite decimal place, irrational number in a finite amount of space.

I'm not too concerned about pi in particular. I know it comes up in a lot of calculations to do with waveforms and rotations, but that's coincidental. I'm only using it because it's a well known irrational number. Replace all referrences to 10 pi with 100e if you want.

(msp your f'(t) is not the derivative, just the sampled version of f(t) right?)

mei (mei), Wednesday, 19 February 2003 09:12 (twenty-two years ago)

That is _exactly_ what I think. But the sampling theorem seems to say that the waveform reconstructed from the samples is _perfect_ during the in between periods also. Which I don't believe.

But there's no extra information between the gaps - it's removed by bandwidth-limiting the signal in the first place. If you remove all frequency components above n Hz in an analogue signal, you're left with a waveform which is simple enough to be perfectly represented by sampling 2n times per second.

If you like, there's only one way to connect the dots when you've removed those higher frequencies - anything else would suggest their presence.

Michael Jones (MichaelJ), Wednesday, 19 February 2003 13:59 (twenty-two years ago)

. . . and removing those frequencies means you lose combination tones.

Lynskey (Lynskey), Wednesday, 19 February 2003 17:55 (twenty-two years ago)

But there's no extra information between the gaps - it's removed by bandwidth-limiting the signal in the first place. If you remove all frequency components above n Hz in an analogue signal, you're left with a waveform which is simple enough to be perfectly represented by sampling 2n times per second.

Now we're getting somewhere. By 'frequencies' are you talking about frequencies of _sine_ waves? That's what I've always suspected the sampling theroem is talking about.

If you mean _all_ kinds of waves, then what about a a square wave at, say, 30Hz? It should be reproduced perfectly,
because 30 < 44,000/2. But such a square wave is the sum of sine waves with limitlessly high frequencies.

I think if you sampled a true 30Hz square wave at _any_ sample rate what you'd get is a square wave with the corners rounded off. Less rounded for higher sampling rates.

mei (mei), Wednesday, 19 February 2003 19:16 (twenty-two years ago)

It depends on how you reproduce it, dunnit? like if it interpolates into sine wavs or not? (i am v. much guessing on this point)Ã?

Sterling Clover (s_clover), Wednesday, 19 February 2003 20:07 (twenty-two years ago)

. . . and removing those frequencies means you lose combination tones.

Meaning heterodyning? But where does that happen? In the air, so captured by the mic? Then they're already captured within the 0-22k channel, yes?

I don't know where it's important to preserve supra-aural content (and hence difference/combination tones) - at the mic or beyond into the recording format. Any ideas?

Michael Jones (MichaelJ), Wednesday, 19 February 2003 20:32 (twenty-two years ago)

Hmm. If i remember rightly (which I frequently don't) combination tones are percieved by the ear, they don't actually exist. I may be talking bollocks.

Lynskey (Lynskey), Wednesday, 19 February 2003 21:01 (twenty-two years ago)

you add and cancel all the frequencies coming to the mic, & yr. gonna get a single waveform, right? reproducable as a fourier series. & my understanding is that the highest frequency componant would be the last term of a fourier series for the waveform. (and the ear only hears different tones at once by breaking down the waveform into constituant frequences too, no?) So if yr. combination tones are higher frequency than 1/2 yr. sampling rate, you lose them.

right?ˇ

Sterling Clover (s_clover), Wednesday, 19 February 2003 21:13 (twenty-two years ago)

The sampling theorem views waveforms in terms of their sine-wave partials, so that's all the nyquist limit is guaranteeing -- that you'll get accurate representation of the partial frequencies up to that limit (ca. 1/2 the sampling frequency).

Combination tones are a physiological/psychoacoustic phenomenon; they don't have a physical existence of their own during transmission through air (say). So if you lose high frequencies, you're potentially losing combination tones that are part of the listening experience -- as Sterling just said. But those high frequencies tend to occur at relatively low sound pressure levels, and the difference tones they induce are not salient under most listening conditions.

Anyway, if we're content to listen to mp3 -- which is VERY lossy compression -- then we can surely tolerate what's lost in CD-quality digitisation.

Paul in Santa Cruz (Paul in Santa Cruz), Wednesday, 19 February 2003 21:40 (twenty-two years ago)

The sampling theorem views waveforms in terms of their sine-wave partials, so that's all the nyquist limit is guaranteeing -- that you'll get accurate representation of the partial frequencies up to that limit (ca. 1/2 the sampling frequency).

Hiya Paul, you seem to know what you're talking about. Those 'partial frequencies' are the frequencies of sine waves then aren't they?
It's along time since I did fourier transforms and I wouldn't know how to go about learning them again. From what I remember you can write any wave as a sum of sines. These sines are the 'partials'?

mei (mei), Wednesday, 19 February 2003 21:58 (twenty-two years ago)

I'm not complaining about CD quality at all, I think they're great.

My problem really is with the sampling theorem and my misunderstanding of it. Which I now think I've resolved to the fact that the theorem only addresses sine waves (and that sort of includes any other waves indirectly because they can be written as the sum of sines).

mei (mei), Wednesday, 19 February 2003 22:02 (twenty-two years ago)

Yes, Fourier proved that any periodic wave can be expressed as a(n infinite) sum of sine waves; those are its partials.

Bear in mind how this theorem is usually applied: a hard-limit filter cuts off frequences above the nyquist limit prior to sampling, because if they stayed in they would "alias" (be represented inaccurately as lower frequencies in the sampled result). And at the output stage there's a similar filter at approx. the same cutoff frequency -- since no frequencies above this limit ENTER the sampling process, any frequencies in the result are due to error and should therefore be blocked from the output. Filter design can get pretty fancy (I imagine) but that's the basic idea.

you seem to know what you're talking about
DON'T BE FOOLED. (What little I know is residual from my Electrical Engineering days -- I double majored in EE and Music, and since then it's all been music.)

Paul in Santa Cruz (Paul in Santa Cruz), Thursday, 20 February 2003 00:37 (twenty-two years ago)

Anything with "resolution" will always be imperfect. All sampling theorem states is that anything up to a 22KHz tone can be theoretically represented by a 44KHz sampling rate.

In answer to your pi question, imagine plotting a graph of y=sin(pi*x/(10*pi)). No matter how small intervals you plot, the graph will always be approximate, yet will still be a plot of an irrational number.

Graham (graham), Friday, 21 February 2003 03:21 (twenty-two years ago)

graham those maths can't be right coz the pi on the top and bottom will cancel out and you just get sin(x/10).É

Sterling Clover (s_clover), Friday, 21 February 2003 06:08 (twenty-two years ago)


You must be logged in to post. Please either login here, or if you are not registered, you may register here.