*** MUS171 #15 02 22

@0000 Miller: Last time we ended up doing sequencing. But before we showed the sequencer, there was talk

es: @0000

@0015 about waveshaping and frequency modulation and, in particular, there was a patch that tried to show the equivalence of waveshaping and frequency modulation synthesis.

es: @0015

Actually, what I'll do is I'll get that patch out just to remind you that it was there. @0030 That would have been 2.17 FM. <<loading "2.17/2.FM.pd">> Here is the fixed-up patch with a smaller font size, but

es: @0030

@0045 you can see that this is making the point that over here there is waveshaping and ring modulation which was making sounds. Do we have sound?

es: @0045

I didn't check the sound yet. That was really smart of me. @0060 Sound. Carrier frequency.

es: @0060

And that, hopefully, you're willing to believe sounds something like what happens when I do frequency modulation, which is this thing where you take @0075 an oscillator like this, and start changing its frequency, but then start changing its frequency fast enough to get this sort of thing:

es: @0075

@0090 So that is the well-known sound of frequency modulation, and this patch was a description of why you could think about frequency modulation as waveshaping.

es: @0090

And if you want to know what the spectrum @0105 of a frequency-modulated sound is, that's to say if you want to know the strengths of all the partials are that it can make and what their frequencies are, then you can analyze it by thinking about what this patch would do.

es: @0105

That leads you into engineering mathematics -- You have to know Bessel functions, which @0120 I don't want to tell you right now. So Bessel functions aside, and in fact all of this aside, what I'll do this time is just go back to the basic frequency-modulation patch, build it from scratch, and show you what it's good for, just in terms of sound making.

es: @0120

@0135 The punch line is going to be that it's a thing which combines two oscillators. But, of course, you can combine 50 oscillators and all sorts of frequency-modulating networks where each one modulates the frequency of some other one, and then finally you listen to some oscillator at the bottom.

es: @0135

@0150 So frequency modulation is this very extensible, very complicatable thing. So I'm going to get rid of this and start from scratch. Just to be nice and pedagogical again.

es: @0150

@0165 So let's see. We have an output device. I'm just going to start with oscillator ... So the oscillator is ... let's get a frequency going, which is just a number. And then we're going to say

es: @0165

@0180 that's going to be the frequency. Just to be pedantic, what I'm going to do is make the oscillator out of a phasor.

es: @0180

Phasor is the actual oscillator; @0195 it's the thing that remembers what the phase was previously and gives you the next phase every time that time moves forward. Whereas, cosine is the thing which is the wave form.

es: @0195

It is actually done internally with a table lookup, so you can think of this as being @0210 generalizable by table lookup kinds of objects. Anyway, here is an oscillator divided up into the oscillator part and the wavetable part. And it sounds like it always used to sound, and this is a good check to make sure our computer is still working.

es: @0210

@0225 440, OK.

es: @0225

Now, we're going to take this and we're going to start messing with its frequency. And, again, just to be as pedagogical and as @0240 didactic as I possibly can, I'm going to do this in two different ways and make a claim about how they can be thought of as equivalent, -- if you ignore a couple of minor problems.

es: @0240

So the first thing is, we're going to take @0255 this in order to take us ... oh you know what? Before I do that, even before I duplicate it. Of course anytime you have an oscillator, you're likely to want to control its amplitude, and I'm going to want to control the amplitudes of all these oscillators.

es: @0255

@0270 So we're going to get a multiplier and a line~. Line~ because I want to make everything to sound nice and the line~ is going to get messages (I'm thinking back up the tree now ... ) so the messages are going to be packed.

es: @0270

@0285 Some amount of time that I'm going to make the line~ ramp at, and that's going to now be a control which I control with a number box.

es: @0285

@0300 Let me get this number box over here. So now what we have... sorry, this is all very repetitious... so now what we have is the 440 Hertz tone.

es: @0300

I can listen to it at my favorite listening level but I can also turn it on @0315 and off _that_ way. OK. Now, we have an amplitude controlled oscillator. And now I'm going to make frequency modulation out of it in two different ways.

es: @0315

So the first way is going to be by doing the real frequency-modulation thing: @0330 Which is to say I'll take this oscillator, but the input of the oscillator is going to be another oscillator. So it's going to be something plus 440. So I need to now have an adder.

es: @0330

@0345 And what I'm going to add to it is going to be a whole other oscillator. So I'm going to take this oscillator and make it nice and

es: @0345

@0360 compact because we're going to have several of these up on the screen very soon, unfortunately.

es: @0360

OK. So let's get another one of these puppies, put it up here. I didn't mean to do @0375 that scrolling thing. ... And now this one is going to be added to the frequency of the other oscillator, right? Sorry; this is boring ...

es: @0375

So now we're going to say this one is going to talk to us, and we'll control its amplitude ... Oh and we need this 440, @0390 it's not in the plus yet. And now, this oscillator and I forgot, of course to make a nice number box to set its frequency. This is the modulating oscillator,

es: @0390

@0405 which now can get a frequency and an amplitude. And now I don't know if it's audible but that's now making vibrato, or with different parameters, it's making frequency modulation.

es: @0405

@0420 So that's a thing. And now you can think of this as being two different oscillators. In fact I'm going to put this close to this oscillator, so that you can see them as being

es: @0420

@0435 essentially the same thing, this could have been plus 120, but I'm just not going to be that didactic.

es: @0435

So this oscillator now, if you think of it as an oscillator, it has a constant and a variable part to its frequency. And it has a thing for controlling @0450 its amplitude, and then it multiplies and that's it, right?

es: @0450

Oh yes, before I forget, I have to tell you one other bit of Pd lore that I probably haven't said before, @0465 which is: It would be natural if we tried to do this: We'll take this thing and disconnect it and just run the signal and the message into the phasor, expecting Pd to add these things automatically.

es: @0465

@0480 This will badly confuse Pd because Pd is seeing messages here and seeing a signal here. The phasor has to decide whether this inlet is a signal inlet or a message inlet. And what it will do, in this case, is it

es: @0480

@0495 will actually decide, "Gee, I'm a signal inlet because there's a signal connected to me, and these messages are superfluous."

es: @0495

If these messages were not superfluous -- that's to say if it just remembered these messages -- @0510 then you could get in some awfully bad problems because someone could send a number five into this phasor and you could forget it and have this phasor around with a number five and hook stuff into it, and wonder why everything is five or Hertz off.

es: @0510

So instead of allowing that to happen, when you have a signal @0525 and a message connection to the same thing, the message connections are simply ignored. So, now we have this rather embarrassing fact that this thing is now playing at 576 Hertz if I do this:

es: @0525

But if I @0540 connect this to it, it forgets the 576 Hertz because that got overridden by the incoming signal. So signals override messages. And that is why if you want these two things to add,

es: @0540

@0555 it's not OK -- or it won't work -- just to put them in the same inlet as if they were both signals. If they're both signals, they'll be added for you automatically. But if one of them is a signal and the other is control, Pd will not know what to do, and so you have to add explicitly,

es: @0555

@0570 like that, which is how I had it before.

es: @0570

So, that's a detail about Pd that can very easily be confusing so I wanted to be very overcautious about that. @0585 Now, the whole point to this exercise is to take this and to show you how to do it another way. And the reason for this is because the other way is the way it's actually done in practice. Which is: You take this oscillator all right; but instead of

es: @0585

@0600 having it be an oscillator with extra input for frequency, you make it an oscillator with an extra input for phase.

es: @0600

What that means is you have your phasor first, @0615 which is making phase, but meanwhile you can add your own phase to it. Actually, it's going to be better to work it this way:

es: @0615

So now what we have is ... @0630 how do I make this ... It seems unavoidable to add an extra object to make the oscillator uglier but I guess that's just what it is.

es: @0630

Now what we have is an oscillator with two inputs once again. Well one of them is just a @0645 number box, but this is now controlling the amplitude, and this is controlling the frequency, all right? And in fact, if I just tell this 440 and this something reasonable, I should hear the sound again:

es: @0645

Good. But, @0660 now the addition is not being added to the frequency, but it is being added to the phase, which is the stuff that's between the phasor and the cosine. And this is the point of splitting oscillator up into a phasor and a cosine 10 minutes ago, which was that I wanted to be able to get in there and add

es: @0660

@0675 something in between the phasor and the cosine in which the osc~ doesn't have an inlet to do. And therefore, I have to rewrite it in this more elementary form, or in terms of these more elementary objects. OK?

es: @0675

Now, so this is an oscillator with @0690 a frequency control signal input. This is an oscillator with a phase control frequency input. And again, I can just do all the same stuff. This up here is an oscillator without any controls at all. It's just being an oscillator. And now I can do this:

es: @0690

@0705 Let's see, just to try and get equivalent results, let's turn this on. Let's see....

es: @0705

So now we hear that one. You know what? @0720 I need to be a little louder. OK? Oscillator. -- And now I'll make this do the same thing. 0.3, oscillator.

es: @0720

@0735 Oops, sorry. Left this on. Let's turn that off. And we have the same thing. Now, we can make vibrato out of either of these two oscillators. This one, it's obvious how to do it. We'll take this thing and make it go six times per second -- five times per second.

es: @0735

@0750 Oh, sorry. Five times per second is the right one. And then there's a depth here. And for instance, just so that we can... all right, let's just do it by ear.

es: @0750

So we'll do 10 Hertz, @0765 which is a... No, five Hertz, vibrato. Not totally unreasonable setting. Let's do that over here. OK. Can you make vibrato by adding a sinusoid to the phase of the thing,

es: @0765

@0780 instead of adding a sinusoid to the frequency?

es: @0780

The answer is, unless the answer changes, the answer is you can do exactly the same thing. Because what a phasor @0795 really does is integrate the input over time. What that means is that I'm putting signals in here. And if I put for instance, a constant signal here like 440, what the phasor does is it fixes it so that its slope is proportional to 440 Hertz.

es: @0795

@0810 Or to put it in another way, what the phasor does really is: at every point it simply adds to its previous output, which is its phase, an increment -- a phase-increment which is proportional to the frequency.

es: @0810

@0825 So what it's doing is adding in values of frequency sample by sample, accumulating them. And of course, there's a little detail that when it hits 1, it wraps back around at 0 and that's really for numerical accuracy more than any other thing.

es: @0825

If we had infinite numerical accuracy, the phasor could simply be a straight line going off to infinity. @0840 So, if a phasor's an integrator, integration is linear. So we're integrating 440 and that gives us a nice ramp but we also can integrate a sinusoid

es: @0840

@0855 and integral of a sinusoid as you all learned in calculus is another sinusoid. So that the indefinite integral of the cosine function is sine and indefinite integral of sine is -cosine. So either way integration

es: @0855

@0870 just changes the phase and actually the amplitude. But why? Because depending on the frequency, there will be a different constant in there when you do the integration or differentiation. That's calculus; I'm not supposed to use calculus here. Why don't you forget I just said that?

es: @0870

@0885 OK, so at any rate ... What that is saying is that if I am adding, if I want to simulate adding a cosine to the frequency of the oscillator, I could do it by adding a cosine of a different phase

es: @0885

@0900 and amplitude to the phase of the oscillator. In other words, I could add the thing here or I could add its incremental sum here. And in fact

es: @0900

@0915 if you don't believe it, I'll play it for you and then you'll have to believe it.

es: @0915

So now I'll make this thing be five Hertz. It will be in the same frequency, but I'll have to give it a different value here. I don't know what it's going to be, yet, but it's going to be much smaller. Like that. @0930 So now I claim this signal is - what's the right word -- is "similar" to this signal. Oops. Give me that signal.

es: @0930

OK. In fact, we can even make them be @0945 the same by ear. Now this one had to have an amplitude of five because we're going to range from 440

es: @0945

@0960 +5 Hertz down to 440 -5 Hertz. This one had to have a much smaller amplitude because all we had to change this phase by was how much that 5 Hertz could get you

es: @0960

@0975 in the one-fifth of a second it takes this oscillator to cycle.

es: @0975

That's hand waving, but in fact this has to be in the order of a fifth as big as this because this frequency is five. @0990 This number is actually going to be 5/(2*pi). And now is it really true that one / 2π is about 0.13. Now we have to find

es: @0990

@1005 this out because otherwise, we won't ever know. Free open-source mathematics package.

es: @1005

@1020 0.159, 0.16 roughly.

es: @1020

So this number I claim, this number here is this number @1035 divided by five because the faster this thing goes, the less it accumulates; so, the faster this is going, the more you have to divide by.

es: @1035

I'm arguing by proportion and not by actual equations, right? So what @1050 we have over here is going to be inversely proportional to the frequency. Proportional to this number because we're trying to get the same sound and I'm just going to tell you that the factor that you have to throw in is 2*pi -- which you will get out of calculus class if you go there. All right?

es: @1050

@1065 So to try to see if this still holds, I'll try some other number here:

es: @1065

I'm sorry. Yes, this is a frequency. Now 30, and now I'll @1080 choose some horrendous value here. Now, I'll see if I can get that same sound over here and see if it's still true.

es: @1080

OK. So we're going to thirty, and that... oh my! That was too easy. @1095 Is that the same sound? No, it has to go up higher.

es: @1095

@1110 All right? Does that sound similar? Maybe. So let's take this thing and divide it by 30 and divide it by 2 pi and see if we get that, right? 70

es: @1110

@1125 divided by 30 divided by my 2 pi... 0.37 -- 0.38.

es: @1125

Ears are wonderful things. Your ears can do better mathematics @1140 than your eyes. Oh, yeah. That's actually true. I don't know how accurate your eyes are for seeing things spatially, or seeing colors, or seeing frequencies, but your ear can hear three cents difference in frequency which, let's see, three cents is a 30th of an

es: @1140

@1155 octave and you have roughly ten octave range of hearing. So that's a part of 3,000. That's the most accurate sense that you have in your body. That's not bad. OK.

es: @1155

@1170 It's fast, too. It's faster than vision. << Note -- the quick estimates here should be refined as follows: There are 100 cents in a half-step, and 12 half steps in an octave. So there are 1200 cents in an octave. If human hearing is 10 octaves then there are 12,000 cents total in the range of human hearing. So being able to distinguish a difference of 3 cents is 3/12,000 or 1 part in 4,000. >>

es: @1170

So anyway, let's go back to where we are here. So, what I'm claiming, although I'm just giving this to your ears, I'm doing the mathematics out, is that we can change the frequency or we can change the phase; and as long as it is true @1185 that the thing that we're modulating by happens to be a sinusoid...

es: @1185

Why does it have to be a sinusoid for this to be true? Because I made this hand-waving argument about you have to put the integral of this thing in here @1200 to get the same effect.

es: @1200

In other words, whatever you have here, you have to accumulate it here. And it turns out that if you accumulate a sinusoid by adding up values cumulatively, you get another sinusoid and that's a wonderful property of sinusoids that, in this case, @1215 makes it possible for us to rearrange this thing from this form to this form. But that only works for sinusoids. It does not work for other waveforms.

es: @1215

For other waveforms it turns out that this is a better @1230 thing to compute than this. And I don't know how to explain why very well. But this is more likely to give you what you want than this.

es: @1230

@1245 I'm not going to try to explain that.

es: @1245

Too complicated to get into. All right. Anyway, here's another good thing about this form. I've probably already let the cat out of the bag. This is @1260 the way it's always done in hardware. Why? For the unobvious reason and the very interesting reason that good values of

es: @1260

@1275 amplitude of oscillators that you use to modulate are in the same range as good values for listening to this stuff.

es: @1275

In other words, @1290 if you take these numbers that you have to choose in order to make this thing sound right are much larger than these numbers that you have to choose to make this thing sound right, the amplitudes have to be down here, but the widths

es: @1290

@1305 of frequency deviation have to be on the order of the frequency itself to have a reasonable effect, which is again saying why you have to divide this thing by this thing to get how strong an effect it is.

es: @1305

Here, the proportionality to the frequency is @1320 already built-in. So, for instance here, what I could say is, this is the same thing as deviating this thing not by 0.38 Hertz. This is deviating this frequency by 70 Hertz.

es: @1320

@1335 This is going from 440 plus 70 down to 440 -70.

es: @1335

This one is going from 440 to what? Well, we heard the same thing. So in fact, it's going approximately from 440 plus 70 to 440 -70. @1350 But -- this is a better way of saying it. It's going 440 plus or -0.38 -- 38 percent of itself divided by 2 pi.

es: @1350

@1365 So forgetting the 2 pi -- because 2 pi is close to one -- this is the proportional depth of frequency modulation, whereas this is the absolute depth. And the proportion depth is a better unit to be talking about frequency modulation in.

es: @1365

Miller: @1380 [laughs] Stony dead silence. Exactly

es: @1380

@1395 what we want. Every professor wants stony dead silence when they talk. ...

es: @1395

Now, with that as an excuse, now I can actually take the entire left-hand side of the patch and erase it and @1410 do something else instead, which is to take the right hand side of the patch and populate it with other stuff. Before I do that, I'm going to do something else, which is this:

es: @1410

I'm going to show you the spectrum of this again. This is @1425 going to be another proof that these things are sort of similar. In fact, this is going to be a test of whether our ears or our eyes are better at finding the similarity. So what I'm going to do is do a little

es: @1425

@1440 deus ex machina again. We have to go back ... (Ignore the fact that you saw the future in that open dialogue ... )

es: @1440

live-spectrum.pd, <<opening "2.22/live-spectrum.pd">> @1455 Here we are. OK, this is a nice patch which I developed for totally different reasons and which I do not want to explain. This patch, which you can get if you download the patches for the day, let's you do the good stuff

es: @1455

@1470 like "Hi I'm a spectrum and..."

es: @1470

Here we go -- ADC: OK I will not dwell on this because I will end up talking for hours about the wonderful properties of spectra of voices, but @1485 what you see here... Anyhow, I'm not going to not be able to stop myself from doing this: [voice in microphone]

es: @1485

There is a spectrum. This is the like the spectral analyzer I had out for talking about @1500 waveshaping a couple of sessions ago, but this is a real one which doesn't care that I use exactly frequencies which are aligned to the filter bank that I used to measure the spectrum. Never mind what all that was. ...

es: @1500

This is a general @1515 spectrum estimator in which peaks just look like peaks that can move up and down continuously without getting messed up.

es: @1515

Good enough. All right, And I stopped it now. @1530 What's really happening is that every twentieth of a second, this thing is making a new nice picture and showing us a new spectrum.

es: @1530

Now the reason I'm hauled this out is not to show off @1545 the spectrum of my voice so much as it is to show off the spectrum of frequency modulations. So what I'm going to do is I'm going to take this patch and use a wonderful feature of Pd -- which is that

es: @1545

@1560 you can send and receive signals from one patch to another -- to make it listen to this patch. "send~ spectraph"

es: @1560

@1575 And I'm doing this so that we can look at the spectrum.

es: @1575

@1590 So now here's the wonderful frequency modulating sound that I just made, and here is its spectrum, if I can

es: @1590

@1605 find that window again. ... Ta-da.

es: @1605

OK. The choice of frequencies I made was good for our ears to be able to find the same @1620 bunch of junk, but it's not so great for looking at. So I'll go change the parameters later to show you how this is all affected.

es: @1620

What I'm going to do now is check that this spectrum that we're looking at right now is actually kind of @1635 the same thing in these two techniques of doing frequency modulation. So there's the one and here's the other. So, remember what this

es: @1635

@1650 graph looks like, and now we do this one, and we get -- approximately the same thing.

es: @1650

So that's another non-proof, another sort of demonstration without proof, that in fact @1665 what we're doing here is in some sense equivalent to what we're doing here. You heard it -- Now you see it, in some sense.

es: @1665

Now I'm going to get rid of this and we're just going to be looking at spectra @1680 of frequency modulation networks. And I'm making room because this is going to grow as always. Stuff never shrinks. ... Save.

es: @1680

@1695 So now we listen to this and we're looking at the spectrum. And so now, as we remember, and if we turn this down to say zero, we're looking at a sinusoid which looks like a

es: @1695

@1710 peak in frequency-land.

es: @1710

And now as we turn this back on ... so one thing you see is that the thing is getting fatter @1725 and fatter and fatter, the bigger I push the deviations in the frequency.

es: @1725

Now, the other thing @1740 that we sort of already know is - let's get a bigger frequency, like a hundred, so we can see it. The spacing of the peaks that we have here is set by the modulating

es: @1740

@1755 frequency. So, terminology: I've been sloppy about not defining my terms as I use them, which is unfortunately normal for me.

es: @1755

So this is an oscillator. This, in common speech is called the @1770 "carrier oscillator." And this oscillator is called the "modulating oscillator." And I think, although I can't swear to this, that this goes back to the days of radio where FM was a way that you got signals from a radio station to a radio receiver, and

es: @1770

@1785 this signal, the carrier signal -- would be whatever it will be... 92.5, right?

es: @1785

Whatever radio station you listen to, you dial in the carrier frequency and this modulating frequency would be the frequency or @1800 frequencies that would be present and the signal which you're listening to on the radio.

es: @1800

So that's why this is called a "modulating frequency." And so this, your FM radio, @1815 doesn't sound like this when the announcer is silent, only because this frequency is 92 megaHertz say -- it's too high to hear.

es: @1815

These frequencies would be audio. In this case they're both audio, they're both in the audio range, @1830 20 to 20k. And again. As we change the strength of the modulating oscillator, what we see is that the spectrum that we started with grows ...

es: @1830

@1845 grows limbs.

es: @1845

And, furthermore, what we see is that those limbs don't move around; that is to say, they don't shift left or right. They stay in the same place, but they change amplitude. @1860 Oh yeah, negative does something similar to positive, as usual.

es: @1860

And where are these things? Well, everyone knows -- but I'll tell you anyway. So this frequency here is 440, and these are 440 @1875 plus and minus a hundred, plus and -200, and so or to put it on the way: All of these peaks are separated by 100 Hertz from each other.

es: @1875

And the other thing about that is if one of these things lands negative, an oscillator @1890 oscillating in a negative frequency is the same thing as an oscillator possibly with a different phase oscillating in a positive frequency because we can only hear the real part of these things.

es: @1890

And so, if I push @1905 this amplitude so that peaks further and further out from the center get energy, at a certain point I'll start to see funny stuff in the low frequencies because the peaks will start...

es: @1905

@1920 New peaks will start appearing. You can't see it with these choices of frequencies. Sorry, I'm going to now increase all these frequencies.

es: @1920

@1935 Let's go up to E <<660 Hz>> and this will be 200 Hertz.

es: @1935

Try it again. Yes. So now you see there's the carrier frequency. Here are side bands which @1950 are -- I don't know why they're called "side bands." But these are peaks which describe frequencies that are present in the signal that are 660 plus and -200 Hertz. So this is 460 (660 - 200).

es: @1950

@1965 This is 260, this is 60, and furthermore, there is going to be one that is -140, which is 60 -200. And -140 is the same thing as positive 140, if it's the frequency of an oscillator.

es: @1965

@1980 So you see, there's a peak trying to grow here and that peak is 140 Hertz. Furthermore, I can put more and more energy into it, and it's not just that we're going to get energy at 140, but we'll get energy at

es: @1980

@1995 340 and 540. Those are actually minus, well, morally speaking; these are -140, -340, and -540 Hertz. But we see them as positive.

es: @1995

So the amplitudes depend on @2010 this amplitude here. And the frequencies are all fixed forever, immutable.

es: @2010

All right. Now, one other piece of terminology before I forget to say it, which is @2025 that these things all have names: This is the "carrier oscillator" so this is the "carrier frequency." This is the "modulating oscillator," so this is the "modulation frequency."

es: @2025

This is the @2040 amplitude of the "modulating oscillator," which is also known as the "index of modulation." People who were talking about waveshaping back in the 70's stole the word "index," I think, from FM and started using it to

es: @2040

@2055 describe waveshaping. So in a waveshaping setup, where you have oscillator, multiplier, and non-linear table lookup, that multiplier there could also be called the "index" of waveshaping.

es: @2055

So @2070 "index" sort of means either the amplitude of an oscillator before you do something non-linear to it, like this or like waveshaping; or it could mean how much you're messing the sound up by doing something non-linear to it, which is the same thing

es: @2070

@2085 by what amplitude you throw it through this non-linear thing ... sort of -- I'm coasting over details there.

es: @2085

So what does it sound like? @2100 The original tone and then you get this kind of stuff.

es: @2100

You can get all the partials you want. Notice that the @2115 sort of characteristic pattern of frequency modulation: Which is that after a certain point the partials start appearing in pairs because negative frequency and positive frequencies ones are both going in an arithmetic sequence

es: @2115

@2130 with the same separation. So you should get this sort of one-two one-two one-two kind of pattern.

es: @2130

The other thing about it is the amplitudes of these things... (OK, let me @2145 drop the frequency again.) So I made the modulation frequency large then to show you all the reflection about zero, which happens in frequency. But the other thing to wonder about is how do the amplitudes act?

es: @2145

@2160 OK they're Bessel functions, you've all heard that ... but how do they act empirically?

es: @2160

The answer is: first off, the thing gets fatter and fatter as you push up the @2175 index of modulation, which is the amplitude of the modulation oscillator. The energy starts at the center frequency, and it goes out so the signal picks up bandwidth. The FCC gets very excited about that because,

es: @2175

@2190 of course, if you have two radio stations, the sum of their bandwidths ought not to be more than the distance between the two frequencies, or else they'll be cross-talking.

es: @2190

So, now talking about amplitudes, then, ... @2205 The amplitudes arrange themselves so that more and more energy appears further and further out from the center frequency. But without these frequencies changing, it's just that the amplitudes are changing in such a way to make the frequency appear to be spreading.

es: @2205

@2220 And the other thing is that: OK, to start with, you get nice, normal reasonable stuff like this. And it even sounds reasonable, I'm not sure but ...

es: @2220

@2235 Now, that sounds horrible, but that's because I chose bad frequencies for the nice picture, right? So the other odd thing that starts happening is that as you push the frequency harder, the

es: @2235

@2250 carrier frequency, the center peak, which is the peak at the carrier frequency, loses energy, but it looks like it's actually giving energy off to its side bands, which you could sort of pretend is happening. -- These are side bands. But it actually ends up

es: @2250

@2265 giving all of its energy off into the side bands. So as we push the index further, we actually lose the center frequency altogether. It happens at an index of about 0.38. I don't actually know what that number is. I've tried to figure it out once but I think it's

es: @2265

@2280 just a number. OK?

es: @2280

There's a number of which you just don't you don't have any carrier frequency left at all. But you have nothing but side bands. Furthermore, if you push it further, @2295 that amplitude which was going down keeps going down and goes negative. But, of course, negative amplitudes are the same thing as positive amplitudes.

es: @2295

So we're going back to zero. So the evolution of the @2310 amplitude of the first one is that it goes from large to zero to negative to zero to up to zero, to down and so on like that.

es: @2310

Meanwhile, watch this partial. Actually, these two will have the same amplitudes so watch either one of them. And they start from zero, @2325 and they start going up. That's all right, but at a certain point, they hit their maximum and start dropping, too. And, furthermore, they will eventually go through zero, as well.

es: @2325

Furthermore, the @2340 next ones will go through zero, as well, and so on like that so that you'll actually even have this sort of a wave or even a sequence of waves of energy going out from the frequency of the original carrier.

es: @2340

So this is the original carrier frequency. @2355 Now we have one lump here and another lump there. And if we start pushing the modulation index - index of modulation - up higher, you'll see more and more of these waves and you'll hear these partials appearing and disappearing in amplitude.

es: @2355

@2370 And that gives you a characteristic sound that you can sort of describe as a rolling sound. -- Which, you know,

es: @2370

@2385 you can think that sounds cool or you could just sort of think that sounds like the bad side of FM, depending what kinds of sounds you like.

es: @2385

@2400 But I will say that if you listen to John Chowning's music, which is worth doing, John Chowning being the person who invented frequency modulation as a synthesis technique for music, you'll find that his indices ...

es: @2400

@2415 I can't tell you a lot... You'll find that the beginning of his first piece, the indices of modulation all very nice and small but then he sort of starts feeling his oats and the indices start going up. -- So, never mind I said that.

es: @2415

@2430 Anyhow, these are the nice sort of classical sounds that have nice, smooth spectra that just sort of have a peak. And then you can get the funny sounds that are just complicated;

es: @2430

@2445 fraught full of energy all over the place, which sound like this:

es: @2445

OK. So @2460 the easy way of describing it, describing what's going on, is this sets the center of the energy -- this is the carrier frequency, which says where the energy is going to be centered. And this talks about the

es: @2460

@2475 bandwidth, and not the extent to which the energy is spread out over other partials besides the carrier frequency.

es: @2475

And this is the spacing of the partials. And, of course, now good things happen when you @2490 ask for the carrier frequency -- (220 say)-- and the modulating frequency to be

es: @2490

@2505 multiples of each other. Now we've set up a situation where the carrier frequency and the modulating frequency are the same.

es: @2505

And so to start with, we have this and now as we push the index up, the first sideband over here is going to be @2520 DC -- zero frequency -- and we won't hear it. The next one will be twice the fundamental, which will be up here, and, in fact, no matter how often you add and subtract integer multiples of 220 to 220, you get another

es: @2520

@2535 integer multiple of 220. And so what we have here, no matter what, is going to be periodic.

es: @2535

And its period @2550 is going to be consistent with the frequency of 220, that is to say this period will be 1/220th of a second. And this is the sound that a 1973

es: @2550

@2565 sounded to those computer musicians like a trumpet. So if you say "Make a computer music trumpet," that's the sound.

es: @2565

Now, how do you make the computer music clarinet? We'll just make this one be 440 like I did start with, @2580 and now the first peak is 220. But then you get 220 plus 440, which is 660, (220 - 440) is -220. So the reflection of this peak lands right where it

es: @2580

@2595 was and furthermore every multiple, that is (220 plus or minus any integer times 440) is 220, or 660 or -- whatever that number is -- five times 220, which is 1100,

es: @2595

@2610 and so on like that. And so now we have a sound that has only odd harmonics. And that, ladies and gentlemen, is the computer music clarinet from 1973.

es: @2610

@2625 So you've got your trumpet, 220. -- A trumpet is "modulating frequency equals carrier frequency," and the clarinet is "modulating frequency is twice the carrier frequency."

es: @2625

@2640 Here's another thing about that: I set the carrier frequency to be 220, and then we saw that all the possible

es: @2640

@2655 peaks that could arise in the side band would be odd number multiples of 220. But that could also be true if that number were 660, or 1100.

es: @2655

@2670 All I'm doing is I'm taking the carrier frequency and I'm placing it either here or here or here or here. And the result is always the same collection of possible harmonics.

es: @2670

@2685 The timbre changes. What's the next one? ... Got to add 440 ... 1540.

es: @2685

@2700 So what I'm doing is I'm moving the carrier frequency to occupy different peaks in the spectrum, but the spacing is always given as 440 and so the spectrum itself stays the same.

es: @2700

@2715 In general, this is true: No matter what these two numbers are, you could then add this number into that one or subtract it from it, and you would get different amplitudes, but you would get the same frequencies of the partials present.

es: @2715

@2730 So now for instance and maybe this is kind of obvious -- you can play additive synthesis games with frequency modulation by choosing carrier frequencies which are

es: @2730

@2745 chosen to be lying on a desired spectrum. The spectrum, though, of course, has to be a spectrum that you can get from FM in the first place.

es: @2745

Then you can through if voices of FM along any of these possible center frequencies and add them and you will get more @2760 complicated FM instruments, that still obey that same spacing of frequency.

es: @2760

And there are two simple ones that work well. This is the odd harmonic one and then there is the normal one -- @2775 the "trumpet" one, as I called it.

es: @2775

So now, the carrier and modulator frequencies are the same. But again, now I can take the frequency of the modulating oscillator and @2790 add it to the carrier oscillator any number of times, and I get these kinds of sounds. Notice, the fundamental

es: @2790

@2805 is the same, it's 220. But there are twice as many peaks in the spectrum; the other spectrum only had the odd peaks, and this one now has all the integer multiples of 220.

es: @2805

And also enjoy how -- since I @2820 chose a reasonable index of modulation that's below that wonderful number 0.37 something -- the peak is always going to lie where I put the carrier oscillator. Not really, but sort of.

es: @2820

@2835 The peak is sort of here. Let's try another one. ... Maybe we'll drop this back a little further, for that really to be true.

es: @2835

Oh, right -- Sorry: 0.37 is where the peak actually disappeared -- but there's also some point at which this peak @2850 quits being the tallest. And I don't know where that is; that's some smaller number.

es: @2850

So now we have 220, 440, 660, and now you can see that you can make spectra that variously have their energies @2865 centered in different places and you can superpose them.

es: @2865

What about the phases? The story about phases is ugly and you should look in the book if you want to find out how the phases operate. But the short answer is if you @2880 work it out so that the phases below the carrier frequency are all in phase, are all like cosine, then the ones up on the other side are like cosine, too, except that they are alternating in sign.

es: @2880

@2895 I don't know a good, simpler explanation for why such a thing would happen, so phase is a mess. Just try not to think about phase. Pretend it's on your side, or something.

es: @2895

And if you do care about phase, don't do frequency modulation, but do @2910 something that has a simpler spectrum than this. So the complexity of the spectrum here comes from two things: One is that you get that rolling effect as the index of

es: @2910

@2925 modulation goes up -- that sort of chaotic in-and-out of various frequencies. And the other thing that is odd about it is -- you don't see it -- but the phases of these things are not terribly well-behaved.

es: @2925

The good thing is that the amplitude of what comes out is @2940 really, really well-behaved. No matter what you do to the frequency or phase of this oscillator, you can modulate this oscillator until kingdom come, it's never going to get outside of the range from -0.3 to +0.3. And so it's going to be good to your Fender

es: @2940

@2955 Dual Showman amp -- in a way that some other algorithms that were chosen to have phase coherence here might not do for you.

es: @2955

And there, the example would be the phase-aligned formant synthesis technique which is described in Chapter 6 -- @2970 which shows you how to make these spectra with very nicely controllable amplitudes and phases of partials but which has another Achilles' heel, which is that it gets very spiky and very bad for amplifiers.

es: @2970

@2985 I don't know any way of getting both good amplitude and phase behavior and getting a signal whose behavior -- just in terms of what it ranges from and what percentage of the time it's actually giving you good energy --

es: @2985

@3000 are both controllable simultaneously. I don't know how you do that. Now, next thing about this ... Picturing yourself at Stanford University back in '73 ...

es: @3000

@3015 This only cost us two oscillators, that's to say it only took Chowning an hour of computation to hear about five seconds of two-operator FM back in the day on his Foonly F4 computer -- if I remember it correctly. << Chowning may have used an earlier model of Foonly ...>>

es: @3015

@3030 So why don't we make the thing take an hour and a half to get our five seconds of sound ... and add another oscillator! Where are we going to add it? Well, let's add it ...

es: @3030

OK, well @3045 we know what would happen if we add one down here. We could figure out what would happen pretty quickly if we add another one down here. OK, so we'll need an adder now. So we put these puppies down here. And then I'm going to just

es: @3045

@3060 add them because I might be adding other stuff in, too.

es: @3060

So here's a thing. I'm going to look at it, too. Good. So if I had another one of these things with @3075 other parameters -- but I'll reuse the modulating oscillator and just give myself two carrier oscillators and add them, what then would happen?

es: @3075

@3090 Well, we know what spectrum this thing is going to make. Well, at least we know how to talk about what kind of spectrum this thing makes. And this one is just

es: @3090

@3105 another of the same thing so it does the same thing, too, right?

es: @3105

And so now, we just superpose @3120 the two spectra, and now we have more control over the timbre of the sound.

es: @3120

@3135 (The light -- the wonderful motion-controlled (room) light -- either concluding that we're not moving enough or that we're moving too much. We will never know. ...)

es: @3135

So what happened there was -- @3150 maybe this is just too obvious for words but -- I'm reusing the signal but in fact I would have gotten the same thing that had two of these oscillators with the same parameters, more or less.

es: @3150

What's happening here is I'm just adding, I'm just using two @3165 carrier oscillators and what's coming out is just the sound of what would have happened if I'd done the two carrier oscillators separately and that's kind of obvious now that you look at it, right?

es: @3165

Except that if I were one of those crazy people @3180 who likes to do frequency modulation with waveforms that weren't pure sinusoids, one way that I could think about what that would be doing is I could think of the non-sinusoidal waveform as being

es: @3180

@3195 a sum of sinusoids of different frequencies.

es: @3195

And then you could think of the oscillator itself as being an additive synthesis, an equivalent additive synthesis patch that might have an infinitude of oscillators being added into it but anyway you could simulate @3210 any waveform you want with additive synthesis. I'm not going to prove that right now.

es: @3210

And so this would be a good description of what happens when you have a non-sinusoidal waveform as a carrier oscillator, which, @3225 by the way, the FCC cannot be happy about. That's OK, we're not radiating too much here other than acoustically.

es: @3225

So that is adding @3240 another carrier oscillator. Now, what we can think about is... Oh, terminology: By the time you do this,

es: @3240

@3255 then you're starting to get the idea that these oscillators and these controllable phases are building blocks, in a sense. And for some reason, the Yamaha Corporation got to calling these things "operators."

es: @3255

So this is @3270 three-operator FM. And of course, three-operator FM, you could invent other technologies. And, in fact, I'll show you another couple that could also be three-operator FM. The famous DX-7 synthesizer

es: @3270

@3285 that was FM for the masses for the first time was six-operator FM. So imagine all the good cool things you could do with six of these piling up together in different ways.

es: @3285

And the thing that made that all possible was @3300 the fact that it was phase modulation, as opposed to frequency modulation, so that the units that you describe the amplitudes in were all compatible. In other words, you didn't have to go choosing crazy different ranges of numbers for different oscillators.

es: @3300

@3315 So you could manage them very easily, even in old fashioned 1980's architectures -- 70's even. ... OK, so go back.

es: @3315

So that was putting the oscillator there - the extra oscillator. I could take this @3330 "operator" if you want. I could take the extra operator and put it up here and you still have three-operator FM. But now what we have is the two other oscillators. (Let's see how we get it so we can see all this.

es: @3330

@3345 There's not much hope anymore. So I can make this thing take less vertical space. We're just not

es: @3345

@3360 going to be able to do very well.) So let me turn all of these things off and let's see what we get.

es: @3360

@3375 The carrier frequency is 440 and I'm going to choose two modulating frequencies:

es: @3375

220, and here I'm going to choose a different one. What's a good choice? @3390 550.

es: @3390

So now, both of these oscillators are turned off in the sense that their amplitudes are zero. So now we just hear the @3405 carrier frequency, and now we know what this oscillator will do to it -- Sorry! -- I forgot the shift key.

es: @3405

There. That's two-operator FM. Here's what the other one @3420 does: The other one is 550, so I'm going to make it 550. -- And then it does this.

es: @3420

Isn't that sweet? Maybe it's sweet, maybe not, @3435 depending on how you feel about it. So just to be painfully slow about this: There's 440. There's 440 - 550 which is plus 110;

es: @3435

@3450 and then this is 440 + 550 which is 990. This is this one plus 550, so it's 660.

es: @3450

So we got 110, 440, 660, 990, and so on. @3465 It's 1, 4, 6, 9, 11 and so on times the fundamental frequency of 110. That's kind of cool. And it sounds like this.

es: @3465

And now, @3480 how do you think about what happens when you do both of these together? It turns out to be strikingly easy. Let's do it this way. Let's start with the 550:

es: @3480

@3495 [...]

es: @3495

@3510 So if someone gave you a two-operator FM network that was a sinusoid modulating but was a carrier that had a complex waveform, you could think of that as a sum of simpler two-operator FM

es: @3510

@3525 modules, where modulator was always a sinusoid and the carrier is always a sinusoid because those imaginary carriers would just arrange them to add up to a complex waveform.

es: @3525

This thing you can think of and analyze in exactly the same way, because @3540 you can combine these two. These are the ones we're listening to right now. You can think of this thing, whatever it is, it's periodic with 110 Hertz period.

es: @3540

And so it itself is a @3555 thing that at least in your mind you could describe as a sum of sinusoidal oscillators and here are the frequencies that they're at. And we could even, if you wanted to, write down a bad formula for what their amplitudes were.

es: @3555

@3570 And now when we start taking this complex waveform and modulating it with this sinusoid, what will happen is that it will act like each of these peaks was independently getting modulated. And then we wouldl get this extraordinary complicated thing,

es: @3570

@3585 which is each of these peaks sprouting its own side bands.

es: @3585

And of course, the sidebands are all mixed up in each other because the first two sidebands of this peak are here and here, @3600 but the first two sidebands of this peak are here -- and I'm not sure where to say the other one is, probably here again. I'm not sure.

es: @3600

And so now, we get @3615 all of the frequencies, which are... OK, so to go back. Now what we have is: This is two-operator FM. If I turn this one off, so now it's just these two, right? So it's 440 plus or minus an integer multiple of 550.

es: @3615

Now each one @3630 of those you can think of is being the carrier frequencies for a new two-operator FM setup and whatever these frequencies are, like this 440 again will sprout 440 plus or minus integer multiples of this.

es: @3630

@3645 So in sum, what you get out of the whole thing is 440 plus or minus any integer times this plus or minus any integer times that, which is potentially a very nice thick spectrum.

es: @3645

@3660 So we have that sound, and now we add this sound and we get this kind of stuff.

es: @3660

And now, it's idiot's delight. @3675 We have basically all the spectra that we can possibly wish for and the only thing that we can wish for that we don't have is some way of actually getting from one desired spectrum to another in a continuous and

es: @3675

@3690 ergonomic way -- or thinkable way, understandable way, comprehensible way. ... All right?

es: @3690

So you throw numbers in here and you get spectra. You can say where the frequencies are, and that's all right. But @3705 if someone tells you, "I got this spectrum. Can you make FM do it for me?" And then you just say, "Hmm." [laughs]

es: @3705

There are papers out about that because people in the early '70's got really excited about that, "Oh @3720 FM, that's this very powerful synthesis technique. Let's take the sound of this real trumpet, analyze it and then figure out how we're going to stuff parameters into these FM networks to sound like a real trumpet that was analyzed."

es: @3720

And the answer is you can get @3735 this waveform when you get that well, not even but you can imitate. You can get a bad imitation or a vague imitation of one set of partial strengths and of another.

es: @3735

And then you can try to get to @3750 continuously from the one to the other, but then you will find that the parameters that you had to do for this aren't actually in the neighborhood of the parameter that you had to get for that. They're somewhere else, in some mountain range of horrible parameter choices. And as a result, you can't actually make continuous paths and get between spectra in

es: @3750

@3765 any desirable way that you could wish -- as a general rule. So FM turns out to not be nirvana in terms of synthesizing sounds -- for the simple reason that if someone gives you a sound and asked you synthesize it with FM, it is usually just not possible. So what would you do?

es: @3765

@3780 Well, you can go back and do additive synthesis, probably the easiest way to do that.

es: @3780

OK, so this now is three-operator FM, with two of the operators... (Sorry these are oscillators but we call them @3795 operators now because that's what they call them), operating on the third one. And now, of course, you can also say, "Oh cool. ..."

es: @3795

Let me explain quickly again the asymmetry of this design or of this picture. This oscillator is like these others, @3810 except that I haven't thrown this adder in. I'm just saving real estate because I'm going to... It's going to be a tree structure, it's going to be a tree structure today.

es: @3810

And so, somebody's going to be on top, someone's going to be out on the leaf because you can't have @3825 a tree without leaves, as far as I know. And so this is going to be leaf. So now what we're going to do is we're going to say, "No. I don't want this thing to modulate this oscillator down here. I want it to modulate this oscillator over here."

es: @3825

And then what do you get? Well, you get @3840 yet another spectrum. ...

es: @3840

And even less of any reasonable way of describing what that spectrum is, @3855 except to say this. ... No, you can't even think about this. ... Don't even try to put this in your brains. This is another thing and you can do it because it's easy to do. But trying to analyze

es: @3855

@3870 what this does and I think it's just going to be hopeless. So, just don't try it.

es: @3870

Al right, let me tell you how hopeless it is: Analyze these two and you'll get an infinite number of frequencies here, which are this frequency @3885 plus or minus multiples of this frequency.

es: @3885

Now, this thing, then you could regard as an equivalent to an additive synthesis network with an infinite number of oscillators in it. Each of those is modulating this thing. @3900 And I told you how to think about two oscillators modulating this thing, which gave you a doubly infinite set of peaks.

es: @3900

Well this thing gives you, then, an infinitely infinite set of peaks because each of these infinite number of peaks is independently @3915 separately modulating this one. And their indices of modulation, furthermore, are given by the amplitudes of the components of this pair, which themselves are moving in this horribly complicated way.

es: @3915

So the whole thing is just completely beyond @3930 sort of any rational analysis at this point. But you can dial right up on your DX-7 and say you can enjoy all day and people have gotten good intuitively at making sounds out of this thing, even though it's impossible

es: @3930

@3945 to understand what's going on

es: @3945

So there is FM operators and all that stuff. Now, just to go back quickly to @3960 the question of... (Maybe I don't need this one anymore.) OK. Going back now to this thing, I said rather a simple thing about this which is that the

es: @3960

@3975 frequencies present are this thing plus or minus integer multiples of this plus or minus whole number multiples of this, I can even say.

es: @3975

Now, it turned out that all of those things were @3990 multiples of one number, which is 110, and you can tell from by looking at that this plus N times that always ends with integers that are all going to be multiples of 110. That's cool, right?

es: @3990

If I gave you any two numbers here, @4005 and said "What is the number of that this plus or minus N times this is always an integer multiple of?" The answer is going to be, I think the answer is always going to be,

es: @4005

@4020 and the best thing that you can get is going to be the greatest common factor of these two numbers. Greatest common divisor of these two, the GCD of them -- which is in this case,

es: @4020

@4035 110. But in this case, the greatest common factor of these two numbers is one.

es: @4035

And so even though it looks nice now, @4050 this thing, if you want to wait until this thing repeats, you have to wait an entire second because after a second this thing will have moved,

es: @4050

@4065 this thing will have done 551 cycles and this thing will have done 440 cycles and you'll be back to where we were.

es: @4065

But there's no other point at which these two phases both will be equal to what they were before. And just pushing this index a little bit, @4080 now we can do the usual cheap thrill FM thing which is walk through all of the wonderful frequencies.

es: @4080

@4095 Should I make this a little louder?

es: @4095

So if you've ever @4110 heard sounds like these, they're not atypical of frequency modulation synthesis. And of course there's a special case where this one and this one happen to be close to integer multiples of

es: @4110

@4125 something else. So in this case, we have positive and negative things that we didn't quite line up and so they're beating but this number, I believe, is about 3/2 of this number, is that true? 290 ... so half of that

es: @4125

@4140 is a hundred forty five. So three halves of this is ... I think -- 435

es: @4140

Miller: Yeah, so @4155 now what we have is 150 Hertz coming out. Actually we've got odd harmonics again, oddly enough.

es: @4155

@4170 So every once in a while, if we sweep either of these two, you will hit a situation, a sort of syzygy where these peaks whack into each other and do something nice -- well, sparse in terms of spectrum, and then you get those sounds. And between those sounds you get all

es: @4170

@4185 the very juicy, creamy sounds in between.

es: @4185

@4200 So as you're looking at the spectrum, you can imagine these peaks actually just moving through each other like ghosts through a wall.

es: @4200

@4215 That, in a nutshell, is the story of frequency modulation. Let's see what do I have to tell you other than what I've just said that's important?

es: @4215

@4230 That's what it is.

es: @4230

So things to take home about this are: First off, this is real easy to do and it's a cheap thrill, @4245 it's in your cell phone, that's how cheap a thrill it is. What do I mean by that? It's so cheap that you can do it in silicon using very few watts or microwatts, even. So it becomes a very easy thing

es: @4245

@4260 to build circuits to do, which is why you hear a whole lot of it.

es: @4260

It's very well behaved in terms of the amplitudes that you get out because of the fact that finally what you're looking at always ends in this cosine function. @4275 So the behavior is good. You control the amplitude because in the end it's really just an oscillator with a changing frequency.

es: @4275

The gotcha is you don't know how to do anything with it, other than the very simplest things, with any sort of @4290 actual predictability. The only way of finding things out about FM is to develop an intuition on how to get cool things out of FM, which people of course have spent many years doing because there are lots of people who spend all

es: @4290

@4305 their time programming. ... Yeah, literally there are people who spend all their time programming, like cell phone sounds that sound like trumpets and pianos and bells and all that good stuff. And you can enjoy their work

es: @4305

@4320 and millions of people can enjoy their work, too. But if you want to do it, you can count on spending many years messing around with these things yourself. Maybe that would be good, maybe it wouldn't. Depending on what you think do you want to do with your lives ...

es: @4320

@4335 That's it. That is the entire story of frequency modulation, unless I've forgotten something important. I don't think I have. We're done.

es: @4335