*** MUS171 #19 03 08  

Miller: @0000 Someone asked a really cogent question, which is grading weighting: What's the weight of the final versus all the homeworks. @0015 And the answer is the final is worth two homeworks. So out of 100%, the homeworks are all 10% and the final is 20%. 

The grading, whatever you call it, is the way it will be graded @0030 is the same as your homeworks, except that of course you will only have something like three minutes per person to look at the finals, assuming that there are 53 of you, which is how many people were signed up last I saw. Although all I see now 10? [laughs] @0045 But of course it's only three minutes after the beginning of the hour, so people will wander in.

So I think what we're going to do is set up some kind @0060 of a tag team thing where we'll set up a couple of tables and allow someone to be presenting while someone else is fumbling with their laptop.

And I'll ask this again later, but if you have a patch but don't have a laptop, @0075 then you'll want to use a laptop that I'll provide. But all of my laptops are running screwy OSs, so we're going to have to work on that, I'm not sure. Maybe Joe's got a laptop @0090 that's got a normal OS that you guys know about. 

All right. I want to do one thing @0105 to finish off filters, although it being only one thing doesn't make it necessarily short. I'm not sure how it's going to work. The first thing is an observation, which is that you can put delays on @0120 things. So filters are made out of delays.

So I'm going to make observation number one just using delays, so you can see and hear it with long delays -- that's to say audible ones. And then I'll go back and show you how it spins out with filters proper. @0135 And then I will drag all of you to complex plane again one more time just to show you how this spins out in calculations, which you can actually calculate the frequency response of any filter in the world. Maybe. OK. So @0150 the main observation I want to make is this: 

I'll make the recirculating delay loop, which is cool, and then I'll show you how to make the delays go away again. I didn't actually realize this worked until last year. @0165 (... Wait, I don't want to do this.)

So what we're going to do is we're going to take a signal to add a delay to it. OK, signal. Let's see. Let's just do the microphone for now. So do we have a microphone going? @0180 Microphone, so that we can do things like have the delays now. Is that amplifying, still? Because it shouldn't be. OK, that's me not knowing how to deal with my mixer. So anyway, I can @0195 drown it out by doing this:

So microphone. Now, what I'm going to do is make a nice delay just like you've seen before. So what that is, is you say delwrite~ -- @0210 I'm just going to call it delay 1 <<"del1">>.  And we can give this a nice long time because we'll maybe want to set the time that we read it from. @0225 And then we'll have a delread~  Snd we'll give it a nice delay time that we can hear, like a fifth of a second.

And I want this thing to have a gain maybe of less than one, so I'm going to say... @0240 (Sorry. I'm just realizing ... Ah, it's all right...)

OK, so I'm going to multiply it by some gain, and that's going to be a control. So I'll put down a nice number box. @0255 It might be good to have it in hundredths. So I'm going to say "divide by a hundred." That will be the gain of that, and we'll listen to it to see how it all works. OK?

So now if this network does what I think it does, @0270 we'll hear me and then --It doesn't do anything?! Oh, Duh: <<adjusts the echo volume up from 0>>  Here's the echo, and the echo has a volume that I can control. Let's do something reasonable. That's all right. Just set this to 0. "Hello"  -- @0285 Back to normal.  So delay line.

Now, how would you make this delay go away? Now why would you want this delay go away -- To make a point. The point is some delay networks have inverses. @0300 That means that some filters have inverses because filters are delay networks. So I want toshow you how to invert a filter.

And this is good because ... and there's a reason this is good ... Oh! -- @0315 Because it turns out that I can tell you how to figure out the frequency response of this network real fast. Basically you already sort of know what it is because the resonant frequency ... It's going to be a comb filter, and so it will have peaks at @0330 multiples of 5 Hertz in this particular example.

And if you want to do the math, you can even figure out what shape the peaks have as a function of this gain here. So if it's 0, it's flat. And if it's a hundred, then it's notching completely out 2.5 Hertz, @0345 and doubling 5 Hertz.  And in between, it's doing something in between. And you can compute that. And then if you can compute that, you can compute the one of its inverse --  because it's just going to be 1 over it.

First off, let me make the add explicit so @0360 we can talk about this as a single thing. So either it's a delay network or it's a filter, depending on how you think of it. And now, we're going to run very quickly out of space, so I'm going to just sort of -- @0375 squeeze it.  There. All right. So what's the opposite of this? The opposite of this is the following thing: 

We're going to take the filter again.

Oh, yeah, so how would you get rid of @0390 that echo? Well, all you have to do is you would make the same echo --you'd have to use a different delay line to make the echo, of course. So name it "del2." And then we're just going to subtract it. So we'll take this thing and multiply it by minus one ... @0405 and have that control this:

And then if I take this signal... So here's a signal with a delay on it. Let's see if this works... @0420 "So we're still talking and we're now making a delay..." So now if I subtract that, that would make it go away, right? The answer is: 

"Try again!" So what happens is, @0435 we'll take this thing and we'll make an echo of it that is minus the same multiple of the original. -- @0450 And then we'll try it.

And then we get... So -- what I forgot was ... @0465 there are two delayed copies coming out of here and they're separated by a fifth of a second <<200 milliseconds>>. And then I subtract that off. But of course it subtracts not only the original but it subtracts @0480 the delayed copy off.  So what I really get is signal minus the signal that is _400_ milliseconds late.  Oops! Right?

So how do I really get rid of it? The answer is I'll make it recirculate. @0495 So I'll take the signal. The signal now has a delayed copy.  Now I'll subtract the delayed copy, but that will subtract another twice-delayed copy, so I'll subtract it out and that will make a three-times delayed copy, and I'll subtract that out @0510 and it will make it a four-times delayed copy, and so on. And eventually I'll get them all subtracted out and there'll be nothing.

And the easy way to do this is to make this delay line recirculating. So what I'm going to do is, rather than just add @0525 the delayed signal into the original signal, I'll take the delayed signal, add it to the original signal, and feed it back. @0540 This is now a recirculating delay.

Oh ... can I prove that? Let's see if this is actually working as a recirculating delay by listening to it alone.

@0555 OK, we have clarity problems here. So there's the non-recirculating delay, here's a recirculating delay. If I did it right. And I'm going to listen to it and make sure it really is a recirculating delay. @0570 By the way, it's got a negative feedback coefficient.

All right. recirculating delay. Now if I take that recirculating delay and apply it to this delay, @0585 so I took out the original signal and now I'm putting in a signal and itself delayed times 86%.   And now we're listening to both delays and @0600 we've got rid of the delay.

One caveat about this that will immediately have occurred to you:  Of course, if I made this gain more than 1, then to get rid of it I would need a recirculating delay with @0615 a gain more than 1, and that would be unstable.  As a result, if I made this gain more than 1, at least this approach to finding the inverse filter is going to fail.  ...  @0630 But maybe we should not worry so much about that.

The next thing is ... So now I have a delay network that is just giving us the same thing, and slight observation is we could make @0645 this gain negative, and the same thing holds:  Now what we've got is the recirculating delay has a gain of 74% and the @0660 non-recirculating delay is subtracting a copy of it. This is actually easier to think about than the other case. And again, as you hear, I got rid of the delay @0675 that we had before.

Now I've told you this, although I don't think I've really emphasized this. Linear time-invariant things like this commute. @0690 So let's get this up here. What I'm going to do now is switch the order of the two delay lines. So now what we're going to have is, @0705 the original signal will go into this delay line, and it has 74% feedback.  And now we have the recirculating version:

And now I'm going to take that @0720 and throw that into this network here - the non-recirculating one. In fact, to save our sanity, @0735 maybe I should make them agree spatially with what I'm doing. So let's do this now.

So now we have a @0750 recirculating delay, that you just heard. And then we have a non recirculating delay that we will put this into and then we will listen to the output. @0765 And now -- Doh!? Why is it delaying? "Hello!"What am I doing wrong? Interesting.

@0780 I have no explanation as to why this isn't canceling out. @0795 Just to be sure that I'm not going crazy... So I'm adding ... @0810 I'm sending this thing into this delay line. Oh, that's not the output of the delay. Sorry, I did this wrong. I wanted this thing here. @0825 This is the output of the recirculating delay. Now, we have nothing again!  No delay. OK?

Now a quick analysis of the situation: @0840 (Sorry, I don't know how to make this neat.) This is maybe easier to understand than the other one. So now what I've done is almost miraculous. I have this messy recirculating delay network, which you put instantaneous sound in and out comes a thing @0855 that lasts forever, right? And here is a nice thing that cuts that infinite train of delays out, entirely.  Cancels it out.

Why did that happen? That happened because ... One way of thinking about it is --  @0870 When you studied geometric series in high school, they taught you that all you have to do to figure out the sum of a geometric series is you take the thing and multiply it by that number and that makes all the terms match up to all the terms but the first one, and @0885 then you subtract it and they all cancel out.

The exact same thing happens here. The result of this network is a train of delays, each one 77% as loud as the previous one, or having 77% @0900 of the amplitude of the previous one. And then, of course, if you apply a delay that takes the original signal and subtracts 77% of it, that on the first one cancels out the second one. And that delay of the second @0915 one cancels out the third one, and that on the third one cancels out the fourth one, and so on.

Or to put it another way, this recirculating delay makes a train of echos, each of which is dying out exponentially. Take that whole thing and delay it -- the same delay time -- and multiply it @0930 by minus 77%, then you cancel out perfectly the infinite train of echos. And the only reason you should believe this is because you just saw me pull it off in a patch.

@0945 You might think that this means that now you whenever you have a recording that was done in a bad space, all you have to do is make the inverse of that space. Right? So rooms are basically delay lines @0960 in a sense. So whenever you talk in a room, when you put a microphone somewhere or a pair of ears somewhere, you hear just a bunch of delayed copies of the sound -- In a very hand-wavy way of describing acoustics, right?

So all you have to do is make the network that cancels out all those delays @0975 and you could zero out the effect of any kind of room acoustics that you wanted to.  And start all over. So you made a nice recording of someone, but they were playing in a horrible acoustic space. Just take the space out. @0990 And then you can apply any other kind of treatment to it that you want, from the original raw signal.

Two things wrong with that:  One is notice that this only worked... @1005 For recirculating delays, I think it's fair to say for any stable recirculating delay, you can make the inverse of it and get rid of it. For non recirculating delays, you can only invert the thing if the @1020 echo is softer than the original sound.  When that spins out into a real acoustic situation, it turns into a statement that you don't actually know, or you don't know in advance that the inverse filter of  whatever your room is, @1035 is a stable filter.

A stable filter is, for instance, a recirculating delay that has a gain of less than one. An unstable one is going to have a gain bigger than one. So there might not be a stable inverse to @1050 a real live situation.

The other thing is that you can't do it because, first off, you can never measure perfectly what the response of the room is. Second off, the sound source is always moving. You almost @1065 can't get a sound source to stay completely still. And even if the sound source were perfectly still, the air temperature and air density in the room is constantly changing because there are air currents.

And as a result, the reverberation, the response of the room is @1080 always changing -- enough that if you canceled out the reverberation of the room at any given instant of time, it wouldn't be good for any other given other instant in time. So you'd be out cold.

So people have been chasing this fool's dream of trying to @1095 get rid of -- or trying to inverse-filter -- reverberant patterns for decades. Conveniently forgetting, or inconveniently forgetting, that it's actually pretty well-nigh impossible to do. @1110 However, in this nice completely artificial, simple setup, you can do it completely.

I told you that the reason that this was going to be interesting was because now we can make @1125 inverse filters. That's OK. But we can analyze the frequency response of a recirculating filter if we can analyze the frequency response of a non-recirculating filter.

So to make that @1140 audible, let me get rid of the microphone -- put in some noise.  @1155 But I now have to drop the delay time of the filter so this will be clear. So I'll put a number into the delread~ read objects to change their delay times to something tiny like 2 milliseconds. Then we put noise in.

And if @1170 I do it right... I get 92% recirculation. Oh, I'm @1185 changing the wrong one ... So now we've got a nice comb filter. And now I can take that filter and invert it and reconstruct the original sound.  But what that also means is that this filter and @1200 this filter -- I'll play now the original noise going into this filter like this:

So here's the inverse filter, if you like. @1215 This filter [sound] is the inverse of this filter. [sound]  This filter's kind of ugly-sounding [sound], but it's the inverse of that one.  All right?

And I told you that that was going to be easy because we can analyze this rather easily, @1230 and it will take more work to analyze this one. Now, when we take engineering courses or other kinds of signal processing or signal analysis kind of courses -- signals and systems -- they will make you do algebra to analyze this system. @1245 And what I'm doing is trying to avoid the algebra by just sort of analyzing it out of thin air -- by making this claim about these two systems being inverses.

@1260 I should tell you a thing about this: This is a perfectly good filter.  I could make one that has the exact @1275 same frequency response as that one, simply by exchanging the two delayed copies. So right now it's the original signal, and then it's minus 92% of the delayed copy. But I could take @1290 -92% of the original copy and then full-blast the delayed copy and that would sound exactly the same.

But that filter would not be invertible because the second echo is louder than the first one. @1305 So if I try to make a recirculating filter to cancel the echo out, it would be unstable.

There are two different forms of this filter. One of which, the one I've shown you here, it is invertible. @1320 And the other way that you could put it together, which is backwards in time, is not invertible -- Or at least it's not invertible with a stable causal filter.

How would you analyze the frequency response of this thing? @1335 Well, I've sort of told you ... In fact, you can almost do it in your head if we say the gain is flat-out 1. Yeah, if the gain here is either 1 or it's -1, @1350 then we can make a claim about what the frequency response should be pretty easily.  --  Think of putting a cosine wave in ...  What comes out is a cosine wave and itself delayed.  And the sum of the cosine wave and @1365 a delayed copy of that is trig -- which you can handle -- not quite in your head -- but it's pretty easy to do.

However, when this gain is not 1, it takes a little bit more work.  So I'm going to draw you a picture to show you how you might @1380 think about that.

As usual, as soon as the trig gets hard, the right thing to do is avoid dealing with the trig by jumping into the complex plane.  So we will spend ten minutes looking at the complex plane, and we then will forget the complex @1395 plane forever --  unless I can end up showing you the Fourier transform next time. But maybe we won't have time for that.

So here is @1410 the thing that I just showed you.  Now, this is how Chapter 8 of the book. We've set all the delay times equal to 1 now because we're making filters. So the deal there @1425 is that comb filters are filters --  perfectly all right. But if you make comb filter whose first resonant frequency is the whole sample rate, then it doesn't act like a comb filter at all; it acts like something that only does its thing once in the entire audible frequency range. @1440 It doesn't do the combing thing.

So this is the way you do a filter. And here what I'm doing is making a number capital Q for reasons that well ... you'll see in the book ...  I had to name the variables carefully. And then we'll subtract that from the @1455 incoming signal. The reason that we're subtracting here is because -- as shown in the patch -- by convention...

Yeah, it's convention. By convention, you think of the recirculating filter, this one, @1470 as having the positive coefficient. It doesn't have to, but I make its coefficient negative if I wanted to. Like that. All the same good things would happen:  We have that and this and those two things are still @1485 inverse filters.

But by convention, when one uses the recirculating coefficient as the variable that one names capital P or capital Q, depending on whether we're recirculating or not.  @1500 As a result, since we're going to be talking about inverse filters, the inverse filter is going to be subtracting --  subtracting some coefficient times its delread~ .

@1515 To tie this in with last time:  Of course this is a real-valued comb filter, and one of the things that I showed you last time is how to make a recirculating comb filter whose feedback coefficient was @1530 a _complex_ number.

And there is, in fact, no reason that all that this number has to be real number --  It could be complex number. I just made it real number so that I could make the network easily and show you this inverse property.  But if we were using complex numbers -- which means @1545 pairs of delay lines, and then doing complex arithmetic on them -- Then all of this stuff would still hold. And then we could be doing things like making bandpass and stop-band filters.

So now, in fact over here,.. (If @1560 you're reading through Chapter 8 while I'm doing this, these are already complex numbers. The hypothetical reader of the book is completely bathed in complex analysis by this time.)

Now, here then is how you @1575 analyze that: I'm going to skip the equations and just show you what you get. So this Q -- Here is a complex number. And this is the recirculation @1590 of a -- if you think of a pair of filters that are inverse, this was either going to be the recirculating coefficient of the inverse of the recirculating filter or it's going to be just the coefficient of the non-recirculating filter @1605 except it's going to be with the minus sign. So the minus sign is still up here.

So what happens up here? So again, @1620 we imagine that we are going to put a sinusoid into the system. And a sinusoid in complex number-land means the following thing. You choose a good number Z, a capital Z, which @1635 is on the unit circle. So here's the number Z.

And the number Z encodes the frequency of a sinusoid. So now you think that the sinusoid is the points @1650 1, Z, Z^2, Z^3, Z^4, Z^5, and so on forever.

And of course, when we listen to this, we're just going to take the real part. So we're going to project that onto the real axis and it's going to look like a cosine. @1665 But in truth, the real signal, the thing that's happening underneath,  is that there is complex number spinning around the unit circle. And the trick is that each new sample is simply Z times the previous one.

@1680 So now we know how to talk about delaying that 1 sample. Delaying it a sample is simply dividing the signal by Z. Why dividing? Because if you have 1, Z, Z^2 ..., and if you delay it, you get ... @1695 1, Z, Z^2, <<shifted>> and that's 1/Z times the signal that you have delayed. That's confusing, but it's important to remember that "one over." <<1/Z>>

1/Z, by the way, is just this number down here, which is what you get when you take Z @1710 and reflect it around the unit circle, Z to the -1 power. So it's down here:

So now what we have is the original @1725 signal, which is the signal times 1, and then we have the signal times Z^-1 because we delayed it, and times Q because we multiplied it by Q in the network. << 1 - Q Z^(-1)>>  @1740 So let's go back to the network and just check.

So here is QZ^-1. Here's Z^-1, which is the delay, and here's Q. And we're going to subtract. @1755 Now this isn't even figurative -- This is figurative when you're in electrical engineering, but this is real for us because we're thinking that we're putting a complex sinusoid into this. So this really is multiplication @1770 by 1/Z or Z^-1. And this really is multiplication by Q. And this is just subtraction.

So here is the left side of the thing, @1785 and here is the right side of the thing, which is the signal multiplied by Q Z^-1. In other words, the signal is multiplied by 1, and it's multiplied by Q Z^-1, then you subtract the two. So it's ( 1 - Q Z^-1 )  @1800 which is to say it is a complex number which gets from here to here.

So if you think of it as a vector, it's the short vector which starts here and points there. @1815 The way I drew it, it looks like that's a very small number. But the number, in fact, is in the range from 0 to 2. And this amplitude, this thing,( 1- Q Z^-1 ) --  Its size, its absolute value is @1830 the gain - the frequency response of the filter at the frequency Z.  So Q is a parameter of the filter. You can control it but you can treat it like a constant.

Z is the thing which depends on the frequency that's going in. In fact, @1845 you think of a signal perhaps as consisting of many or even an infinitude of sinusoidal components going in. all with different values of Z.

So Q is fixed and Z is everything at once in some sense.  And @1860 the easy way to think about that is not to think about how Q Z^-1 changes, but to multiply the whole thing by Z to get this picture. So now we have Z, which encodes the frequency of the sinusoid, @1875 and here's Q, which is the coefficient.

And now as we imagine looking at values of Z that could range all the way around the unit circle, you see there's an area of the unit circle where the values of Z are close to Q -- and those are areas where the gain of the filter is low,  @1890 it's small. And around here on the other side, the gains of the filter are large.

And the gain of the filter in fact is nothing but the absolute value of this @1905 complex number, that's to say the length of that segment. Furthermore, if you want to think about it, you can also get the phase response of it -- it's the angle of this thing. So if you care about phases, which well ... maybe you care about -- but you don't @1920 have to worry about that yet -- you know how to get the phase out of this diagram?  ... Yeah?

Audience:  So is Q fixed there, it's fixed in space there? Even as Z travels around and Z-Q will change? 

Miller:  That's right. @1935 So Q is the filter parameter -- it's the knob. Z is the frequency that you're thinking about going through the filter. And this is a way of thinking about for all possible values of Z, What does the filter do? And the answer is it multiplies it by that. 

Audience Member: So what is  @1950 (Q Z^-1) ? 

Miller:  This. This part of the diagram is explaining why this part of the diagram makes sense. So this is the real response @1965 of the filter. But this is harder to think about because this point is moving around. This point is not fixed. It's easier to think about the thing just rotating the whole thing about the Z so that this point is fixed and this point is moving, instead of having this point fixed and this point moving. Although @1980 you could think about it either way. This is the way engineers think about it. 

Now, the punch line to this is... Well, there are two punch lines. First off, if I took a bunch of filters and put them in a series @1995 (Only if I put them in series; parallel would be a mess.) ...  but series. And if all of them had this form, then I could tell you what the frequency response of the whole mess was, because you would simply multiply the frequency response of each @2010 stage in turn.

So now we can analyze the behavior of complicated filters using the behavior of simple filters. Or to put it another way, that could give us way of of designing complicated filters. Because we might want to @2025 design that filter with several stages in order to accomplish something or other. Of course, we're not really going to get there, but that's the thing that people think about. The other cool thing is this: 

Going back to the beginning of the lecture ... this is @2040 non-recirculating filters. But of course I've told you and showed you that recirculating filters can be thought of as the inverse of non-recirculating filters.

So here's the non-recirculating filter. @2055 And the recirculating filter, which is the inverse of that, is this filter:  (I have to go forward one. I have to go forward two, sorry. ... @2070 Sorry, I didn't have the diagram. Never mind that.)

You know what it should look like. It should look like this, except that you're recirculating the output of the delay line to the input and we're multiplying @2085 by Q and adding at the input of the delay line, instead of subtracting at the output.

Or to put it another way, we can go back to the patch:  It realizes that this is the recirculating @2100 filter where we are:  Delaying, we're multiplying by something, and then we're adding. This is the non-recirculating one where we're:  Delaying, we're multiplying by minus the thing and adding -- in other words we're multiplying by something and subtracting.

@2115 And those two things are inverses. And what that implies is that [...] @2130 for the non-recirculating filter, the frequency response is the length of this segment, which changes as Z changes, which encodes the frequency. <Z encodes the frequency>>  And for the recirculating one, @2145 the inverse of it, the frequency response is one over this. <<[1/(1 - Q Z^-1)]>>

So the non-recirculating filter has a notch right where Q points towards Z, then the @2160 recirculating one has a peak where that happens. And what's the smallest value that this thing can be, the shortest this segment can be? It's when Q lines right up with Z. And so that is 1 minus the absolute value of Q -- @2175 in other words the amount that Q fails to be right on the unit circle.

And so what that shows is that the recirculating filter has a greater and greater gain ... the closer Q gets to the unit circle, the greater @2190 the gain of the recirculating filter is right at the choice of Z which lines up with Q.

So the angle of Q chooses either the @2205 notch frequency of the non-recirculating filter, or the resonant frequency of the recirculating filter. And the absolute value of Q, this radius here called r ... <<edited from transcript as follows:>> -- controls the gain of the filter.

@2220 So with those two rules -- and with a lot of messing around -- you can design filters to any kind of specifications you want -- @2235 just by making assemblies of recirculating and non-recirculating filters with various coefficients.

But in particular, you can do the thing that I showed you last time, which is you can make a recirculating filter that resonates at any given frequency. @2250 And the way I described that last time was: You know what the impulse response should be -- It should look like a damped sinusoid.

And I know how to make a damped sinusoid because I can just make this funny @2265 recirculating complex filter. Just by pure thought, you can think about what the impulse response of that should be -- which is ringing. And therefore that recirculating delay line would @2280 act as a resonant filter.

Now what I'm doing is showing you analytically why that same recirculating delay line acts as resonant filter.  It's the same filter that I made last time, but I'm coming at it with a completely different line of reasoning now. This is the @2295 correct line of reasoning -- in the sense that now you can take this and actually go compute things. Wwhereas what I showed you last time was just sort of a phenomenological explanation of what's going on -- like picking shells up on the beach or something.

... @2310 Questions about this?

This is pretty much it for filter theory, I think. @2325 Yeah, because I don't know how to show you much more without... After that, it's just details, but the details go on and on and on. For instance, how would you design now a peaking filter, which is a filter that, @2340 around an area here has a little bump that either bumps up or bumps down, but around the rest of the circle it's basically 1-ish?

Well, the answer is you put a recirculating filter and a non-recirculating filter @2355 with their coefficients at the same angle, put them rather close to each other, and therefore anyone out here is about the same distance from both of them, and therefore the gain there is about one, because the two distances are almost the same @2370 and you're dividing -- because one's recirculating and one's not.

But in this neighborhood, if you put one of them closer than the other, then you can make the thing have a positive or negative gain, depending on the ratio of the two distances. And that's your peaking filter.

@2385 And reasoning like that.  You can make it as complicated as you want, but it's thoughts like that get you through all the elementary filters that people use every day in computer music.

@2400 So this is a much better way of thinking about it than is the sort of phenomenological: "Make something that rings." Because you can actually do reasoning on this complex plane and in a way that allows you actually figure out that would make @2415 the filter do something that you want.

Oh, and furthermore, I just told you how to make a peaking filter:   It's a recirculating @2430 and a non-recirculating filter. Now if you set the two coefficients to be exactly the same, those filters are exact inverses. And so that's how you get away with putting all these filters in series in an audio chain.

It's actually true that when you set the gain of a @2445 peaking filter to 0 dB -- neither up nor down -- so that the recirculating and the non-recirculating coefficients are exactly the same, the filter is unity. It is as if there were no filter there at all.

@2460 And you can put a hundred of these things in series and it won't change the original signal.  And so that's why you can actually have things like graphic equalizers that don't completely destroy your signals, because each one of those filters is actually nothing if you zero it out.

... @2475 Questions about this?

So that's filter design. And study with Tom or Shlomo and learn @2490 all the deep stuff if you want to go further in this. Now what I'm going to do drop this entire line of inquiry and start talking about graphics.  

Audience Member:  These results you've shown are all from complex analysis?  @2505 I might need to find some books on that. I wonder if it's the simplest approach?

Miller:  @2520 Yes, this is called the Z plane, just because one uses the letter Z to describe the complex number on the unit circle. Yeah, indeed. This is something that I wake up every morning wondering about. 

@2535 And the good news is you didn't have to know any calculus to do this.  You do have to do complex arithmetic, that's to say you have to understand about @2550 angles and magnitudes of complex numbers, but you don't need calculus. Although, sometimes calculus helps later on.  This is all high school mathematics. In fact, I've probably already said this, but this is the reason @2565 high school mathematics is interesting; so that you can do computer music. [laughter] 

Miller:  You cannot do banking, even. Bankers don't do algebra or geometry, but computer musicians use that all the time, and this is exactly how it comes up.  @2580 So people should be teaching computer music in high school because that would make the mathematics interesting and make it stick with people. 

But that will only matter if you become a high school mathematics teacher in your futures.  So I'm going to save this. ... @2595 And I've pretty much finished making all the points that I want to make about this.  So how we're going to go work with graphics.

Now I have to tell you something about graphics programming, which is that I don't do a lot of it. @2610 So I'm not exactly the right person to teach you this.  You'll see. I can't make great examples, because I'm not a person that does this kind of thing, and there are a lot of @2625 people who do.

So I'm going to quit here. And I'm actually going to change directories. And furthermore, I'm going to run GEM, and not Pd.

@2640 First thing about GEM --  You probably already see this ... when you start up ... If you downloaded Pd-extended as opposed to Pd, then you have GEM. And in fact, when Pd-extended starts up, you @2655 see all this kind of stuff.  This is Pd loading GEM.  GEM is a library that is larger than Pd, I believe, that Pd reads and uses @2670 to define a whole collection of new objects.

OK. Describing that depends on... I haven't said something I didn't say, which @2685 is that:  In Pd, most of the objects that I've shown you have been built-in objects. In fact I think all of them have been.  But if you type it the name is something in Pd that Pd doesn't know about,  (And if it's not an abstraction -- that is to say that if it's not the name of a patch.) ... another thing that it @2700 could be is a name of a file which is an object file which Pd will load in to define a new object.  So you can make objects in Pd that are C-code that fit inside boxes and do things that you want them to do instead of the things that I thought you might want to do @2715 with the built-in objects.

GEM is the... So the name stands for the Graphics Environment for Multimedia. This is by Mark Danks and is now managed by IOhannes Zmoelnig.  @2730 And all these names are also people who work on this.

What GEM is, is a collection of something like 200 objects that pertain to graphics @2745 in some way or another. Graphics in GEM-land is the thing which is called OpenGL. OpenGL -- it's the @2760 name of either a worldview or of an API, which regards computer graphics as being "drawing polygons in space."

That sounds really stupid until @2775 you find out that what you can actually do with it, which is not drawing polygons in space, but it is drawing one polygon in space and pasting images onto it. So I'll show you how all this works.

@2790 There are two points of view on computer graphics:  One is the 3-D point of view which is "Everything is a model."  The thing that you do if you make a dinosaur or something like that -- A dinosaur is a bunch of triangles which are @2805 a bunch of vertices, which few points in space, and a bunch of segments between them that describe polygons.

And then, if you want to render a nice dinosaur on your screens - it's called "rendering" -- for each one of those @2820 triangles that is described, you ask the computer to paint a picture of that triangle, except of course if it's wholly or partially occluded by some triangle that's in front of it, then you would paint the one in front of it in that place instead of the one behind.  @2835 So you do hiding and all that kind of stuff.

And this is exceedingly popular as a way of thinking of computer graphics, basically because of the influence of two industries. @2850 One is Hollywood, which started making high-budget, three-dimensional rendered animated films like "Toy Story" or "Up."

@2865 And the other thing is computer games. There are the interesting computer games and the stupid ones... The interesting ones are the ones that have a thought component to them, And the stupid ones are the ones where you're chasing things around and shooting at them.

And if you want @2880 to make a computer game where you chase something around and shoot at it, it turns out that 3-D rendering is a very good way of making that appear realistic. And so basically the first-person shooter games, FPS games, @2895 and Hollywood movies are the basic reason to have three-dimensional graphics.

Now, personally, turn your ears off, because I'm going to express a aesthetic opinion here: I think both of those things are aesthetically bankrupt. @2910 But they're there and, furthermore, both of those things aree multi-billion dollar industries. So if you know how to do this kind of stuff, you can actually make money at it.

GEM is @2925 OpenGL, which is the Open Graphics Library, which was originally called the Graphics Library.  It's one of two or three competing APIs which the world uses @2940 for doing 3-D rendering.  However all of the 3-D rendering things ended up having to work with images as well.

So why isn't all that stuff images?  Well, because, as I've told you @2955 but then got lost trying to explain, there really are two points of view on making pictures with computers. One is draw a 3-D model, which is what OpenGL and these other things are designed to do.

And the other is to think of it as painting on a screen.  @2970 In other words, the screen consists of a bunch of pixels. It's flat. And what you're really doing is you're concerned with the color of each of the pixels. And then the tools that are of interest are video cameras, because video cameras make images, they don't make 3-D models.

@2985 And tools like compositing, which is to say taking one image and painting onto it with another image, which could by, the way be the image of a blob made by a paintbrush, so that you could regard painting as a compositing operation where @3000 you put bits of one image, which is actually just paint, onto another image, which is the actual canvas that you're painting.

So OpenGL has a huge @3015 facility for doing image processing in this second sense, which is called "texture mapping."  Why? Because the only reason for doing images if you're back in chasing dinosaurs around @3030 in some computer game, the reason for having images is so that you could paint the images onto the skins of the dinosaurs to make them look scaly.  That's called "texture mapping."

You don't really want to have your dinosaur to look like a bunch of green polygons @3045 running around. You want it to look like dinosaur skin.  So what you do is you go hire artist to make a picture of what a dinosaurs might look like and you take that picture and you tile it onto the body of the dinosaur -- all over -- in a way that doesn't make it obvious that it's repeating.   And that's called "texture mapping."

So to make a 3-D picture... -- Is this stuff that you all know? Sort of? Maybe I'm repeating things here. ... @3075 So to make a 3-D image, really, you make a model but then you paint a texture on the model and texture on the@3060 image. And so there are image processing things which are basically there's 3-D texture mapping but they allow you also to do things like taking in @3090 parts of images and compositing them onto the other images and stuff like that.

And those are the cool things that you can do with GEM, as it turns out, at least from my point of view.

So now what I'm going to attempt to do -- and this is dangerous because I don't really know what I'm doing -- is show you the @3105 basic tools in GEM for making shapes and for mapping textures onto them.

Just to maximize the embarrassment, I'm going to do this from nothing so that you can see everything that you have to do in order to make a GEM object.  And then I'll @3120 fall back on some prepared ones because - you'll see ...  Things don't just work the first time in GEM because there are more details to keep track of than there are in audio land.

So what we're going to do is make a new window and @3135 before I do anything, I'm going to save it, and it's going to be 0.gemtrythis. <<Saving "gem/0.gemtrythis.pd">>  And I'm going to call it .Pd even though maybe it should be called .GEM. -- It's going to be a Pd patch but @3150 there are going to be a bunch of new objects which are GEM objects.

So the first one is going to be this thing ...  The first thing is, we need is a window to put the graphics output in. @3165 And there's an object called "gemwin", whose purpose in life is to maintain a window that will be the image that Pd makes.

And it takes messages, and messages are something like @3180 "dimension," which allows you to say how big you want the thing to be. So maybe 300x by 200y. I'm going to make it really small because my screen doesn't have much resolution. And @3195 that is just going to tell the GEM window how big a thing it should make, and there's a "create" message that makes it and there will be a "destroy" message that gets rid of it.

@3210 There's other stuff here that I'm not telling you about. So tell it what dimensions and create a window. And ta-da -- we have now a nice window --  which is just being a window that doesn't @3225 know what it's doing. It's just sitting there being a space on the screen. It isn't even being managed by the window manager right now. 

The next thing to do is to be able to throw it messages to start it and stop it. @3240 And the obvious thing to do would be to put a toggle. So you can send it the number 1, and the number 1 starts it rendering @3255 if you've set it so.

So now it's rendering, which means that it can redraw itself -- and it's just redrawing black every time it gets redrawn now. And then if I turn it off, then it's not rendering anymore and that it's just being catatonic like it was before.

@3270 Now me, when I'm making patches, I don't do this, I do this: It turns out that as soon as you create a window, it's time start rendering. And if @3285 you want to stop rendering you'd probably want to get rid of the window, too. So I alwsys just do that, "creating and on" or "off and destroy." << create, 1>><0, destroy>>

Now, making objects:  So I told everything's a polygon. So @3300 I'm going to show you as an easy-to-manage polygon that won't do much for you.  And then I'll show you a complicated polygon that will do more stuff for you, but it will take a lot of work. The easy one is a rectangle, @3315 and the complicated one is a triangle.

Why? I'll show you... Rectangle is this: Say "rectangle", and then we'll give it dimensions.

@3330 And then we've to say -- And here's the thing that I have trouble explaining.  But I'll just try to explain it:

Now what you need is to have, in some sense, the equivalent of the @3345 adc~ object -- The thing which just starts things rolling. So rectangle is an object which doesn't -- It looks like it has an output, but it doesn't.

It's at the bottom @3360 of a chain of things that we will do, which is we will first off make a source of messages that will go down this chain of objects. And eventually the chain will terminate in the rectangle, which is the command to draw something.

@3375 The top of the thing is an artificial object which is called "gemhead."  So we will have a gemhead at the top everything -- which is called a "GEM chain."   @3390 Now we have a rectangle. This is insulting your intelligences but ... Rectangles have dimensions. And so now we've got graphics. So now you can @3405 immediately tell that you could make patches that have abstractions that have thousands and millions of these things and start making Piet Mondrian paintings and stuff like that. In fact, you can do anything with this now. Almost.

Now this is, @3420 however, a little bit stupid because the rectangle is white and there's some other things stupid about it ... Let me tell you something good to know. The dimensions in GEM roughly speaking:  @3435 Everything is in three dimensions, although you don't see it yet. And this rectangle is on the Z=0 plane. Z is this direction, Z is positive towards you and negative away from you. (I hope -- @3450 I'll find out if I'm wrong.) This is Y and this is X.

And at Z = 0, you can see X and Y, if I remember correctly, between positive and negative 3. @3465 Now why positive and negative 3? It's because that's the way the coordinate system is set up. It's a thing that you could change, but you think of that as the camera.

In other words, there's a virtual@3480 camera looking at this scene, and that camera has a particular lens length and all that kind of stuff.  But it is such that this ranges from plus to minus 3 at Z=0.

@3495 Let's see if that's actually true. So I'll make the thing three. Three means three units wide. So we'll make it six units wide to fill screen.

So there's that. @3510 Now, first thing that you might wish to do - well, there are a bunch --would be to change the color. Anyway, it's a thing you'll eventually wish to do. @3525 Now we start with all the 200 objects:  I've shown you three objects, now we'll start with the other 197. How about @3540 "colorRGB" ... 

I have to say very soon I'm going to have to go consulting the help files, because @3555 all these objects have lots of inlets and have lots of complicated things that you can do with them. But right now, I'm just going to fly with this and hope for the best.

So if I remember correctly, RGB are actually these three inlets, @3570 and their values range from 0 to 1. (I'll find out if this is right or not as soon as I start doing this. I'm wrong.)  @3585 Change that to 1. Let me get rid of this. We're going to just not know what the last inlet does, and do this instead:

So this is also confusing because @3600 the color is white, which is:  R, G, and B are all equal to 1 to start with.  And so I had to actually send it 0's in order to turn it off. The inlets do not necessarily start out at zero in GEM. Although they almost always will in Pd.

@3615 So here's red, here's green - wait, no.  Sorry. And of course blue. And now everyone knows this but ... @3630 Take red and green and you get ooh -- Sort of yellow.

So this is video color rules, where colors add; they don't subtract. @3645 That isn't yellow either, is it? Better than the other.

And exactly what color you get depends on your projector or your screen. In fact I have a radically different color here than what you're looking at. So there's all that.

@3660 What is happening here is the following thing:  Maybe you all can understand this, but this is a thing that I find very, very mysterious.  @3675 The way GL thinks and the way GEM plays into GL... (OpenGL is the API that GEM is talking to. API is an "applications programming interface," - It's just a bunch of function calls as far as we're concerned.)

@3690 The way the API thinks about life is, you don't just render things. (So this is rendering a rectangle here, and you can say "render a rectangle."  But before you render the rectangle, you're allowed to do all kinds of nonsense @3705 which are called "transformations."

Transformations are things like rotating or translating the object in space, changing its color. (Now why is that a transformation? I don't know, but it is.)  @3720 And making it not opaque,  make it partly transparent. And worse yet, unbelievably wrong -- but this is not Mark Danks' fault, this is OpenGL's design @3735 that's just weird:  Another transformation of an object consists of saying what texture it's going to have mapped onto it as it's rendered.

So to put it another way, you go @3750 waltzing down this GEM chain until you get to the last thing, which is a command to draw something, which is a drawing command. But you collect all sorts of detritus along the way. And the detritus is things like rotations, translations, color changes, @3765 transparency changes, and textures.

So this right now is the color thing. And I believe it's true @3780 that if you do it again, it would simply replace the color with the new one. (I shouldn't be doing this, but we'll see...) Now this color is white. @3795 This color simply overwrote that color.

So as I told you, you can translate or rotate. So how about @3810 "translateXYZ"?  Sorry ... this is painfully stupid, isn't it. Now @3825 while we're here, if you want to do animations, the obvious way to think about doing animations is to make an object and then start it flying it around. Don't do that. It's just stupid. @3840 This is a stupid animation.

A good animation is representing motion by drawing things in sequence, which is not the same thing as drawing a thing and making it move around.  @3855 But the people who designed OpenGL were convinced that the right way to animate something would just be to make something and make it fly around.  And so there are all these wonderful things in OpenGL for making things fly around.

Next one, of course, @3870 we have translations. But of course, we're also going to have to have rotations. This one, I think, you can actually say "rotateXYZ." 

@3885 [...] @3900 So rotation: There are many ways that you could represent a rotation, but one possible way is as a 3-vector. @3915 By the way, it's a complete coincidence that the same number color dimensions is the same thing as the number of spatial dimensions. That didn't have to be true. That's our eyes didn't actually choose the number 3. But that's the number we have, right?

@3930 So rotating about XYZ is the following thing. You specify a vector and the magnitude of the vector is going to be the amount of rotation. @3945 What units do you suppose OpenGL would have proposed for that? If colors are going to be from zero to one, the obvious thing to do is have the rotation be in degrees, right? No.   But that's what they did.

@3960 So, for instance, if you want to rotate the thing on the plane that you're looking at, what you're doing is you're rotating about the Z-axis. We're going to describe the vector normal <<perpendicular>> to which you're rotating, and furthermore, @3975 the length of that vector is going to be how much you're rotating.

So here, we're rotating about  Z. And notice by the way some crud in the image. This staircase thing is called aliasing, @3990 and that's bad. You can fix that, but I'm not going to show you just right now. The other thing is you can also rotate like other axes and now suddenly you get to see the fact that you're being three-dimensional.

@4005 Now, this is a perspective drawing of the nice rectangle whose top edge is at 140 at whose bottom edge is pointing away from you. And compound rotations @4020 can be compound things. Now this is interesting for about 30 seconds and then it's profoundly boring.

So next @4035 I'm going to make two of these, so you can see how GEM thinks the things should superpose.

@4050 Let's see now what's happening>  This one ... They're the same size. We're going to rotate this one. Oh my! @4065 I have to tell you something about my computer. A pixel in this direction isn't the same size as the pixel in this direction. @4080 Anyway --  I didn't tell it to make it a square window. Probably that's what's really going on right now:  Since my window isn't square -- In fact why don't I make my window square because I'm going to be confusing here.

@4095 So you would never actually make a square window, but I'll make a square one so that you still only see the aspect ratio problem. So this shape should be the same that shape and I think it's not, but I think that's because my @4110 aspect ratio is messed up. The aspect ratio is whether the pixels are square or rectangular.

So now we have two things. Now, who said that the yellow @4125 rectangle is behind the white one? ... @4140 OK, so why is the yellow one behind the red one? 
Because I did this first. That's not good. It would be better to tell these gemhead's in what order they should be @4155 rendered.

What's really happening is that the yellow rectangle is being rendered first and then the red rectangle is being rendered. So that even though they are at exactly the same location, pretty much space, the red one shows one on top.

You can @4170 actually specify that by putting numbers here which are priorities, which would be the order in which that the things are going to be rendered. But rather than worry about that right now, I'm going to start translating again in the Z direction. @4185 I'm translating the yellow one.  And I'm going to move it toward us, and then away from us.

You get two options. What is that? @4200 "Z buffering" is I guess what you call this, where one thing is in front of or behind another. Or, to belabor the point horribly, let's take this thing and rotate it about X. @4215 Now we have sort of an engineering drawing where we have this piece of metal going through a slot on this piece of metal or something like that.

Now basically this is the way @4230 3-D rendering works:  You draw stuff and then this stuff occludes other stuff -- that's the basic deal.  Now comes the risky part. Let's start putting textures on these thing. And the reason for wanting to do this @4245 is just so that we can actually start working with images, which is much more interesting than working with shapes in space. ...  At least in my opinion.

So to do that -- and this is really strange, I guess. It's not really stupid, @4260 but it's just strange.  So I'm going to do yet another thing, which is to tell the thing that it has a texture. Now @4275 that is called "pix_texture".

And what's the texture going to be? What this thing does, pix_texture, is it says @4290 "Whatever the image is, that's what you're going to texture map on."  But what is the image? There's no way to specify that directly, and so what you do is just say "Part of the environment that @4305 we're in is there is a current image and the image is going to be set by pix_image -- this object." And furthermore, this object takes a message. @4320 The message is going to be "open test1.jpg."  <<Image name needed is actually "temp1.jpg" here>>

So I'm @4335 going to do this and I'm going to try to explain what it is. So pix_image is a thing which is memory, which contains an image which we can read the image in. It didn't work?!  @4350 Why can't I do this? 

You know what, I'm going to cheat. I'm going to go look and see. @4365 So we say pix_image open pix_texture. There's nothing wrong with this... @4380 Oh! "temp1" ! So why didn't I get an error message? Oh, right! The error message goes...  (Right! ... I got this earlier today @4395 and forgot: The error message getss thrown on the "standard error console".  Bad bad ...)

So we're going to say "temp1.jpg". @4410 And then do we get anything? OK, it's happy ... (Where's the rendering window ... It's gone because ... There -- Ha!)

OK, so let me @4425 go back and make this thing be white.  So that you can see the image in all its glory.  Here's the image. It's a stupid image, but you know, someone found this somewhere on the web... @4440 Mark I think found it on the web somewhere.  And now what's happening is:  This image is being texture-mapped onto that nice rectangle. So the operation is done in the following way: 

You have a pix_image, @4455 which is a buffer which contains the image. And pix_image's job is simply to say, "All right. If anyone needs an image, I've got an image and it's in this buffer."  And then pix_texture's @4470 job is to say, "Has anyone got an image? If so, let's texture map it."

And of course the person that's got the image is this one. So that becomes the image that gets texture-mapped. And that's the basic way that this thing gets mapped through. OK.

@4485 Now images are cool ...  But where do images come from?  Images come basically from three possible places that I'm aware of right now. @4500 The medium cool one is files, the really cool one is the camera, or your camera -- so you can take your camera and make it show up on this thing.

And then the uber-cool one is:  You can take the two images and composite them into a third image, @4515 so that you can actually work on various kinds of image synthesis, like you can take an image and punch it into a part of another image and blah, blah like that. I'm not going to have time to describe this in detail today, but I'll try to show you some of these examples next time.

And finally, @4530 on some OS's -- maybe all Os's now, I'm not sure -- you can take whatever you just rendered, make that an image and save it -- which is a way then of being able to paint stuff and haul it off screen into an image and then be able to use that as raw material @4545 in something else.

So, with all that, you have a ready source of a whole bunch of images that you can use in recursive ways to build up cool visual effects of one sort or another.  ... @4560 Yeah?

Audience Member: Can this run while you're running audios and stuff?

Miller:  Yes!  And furthermore, although I'm not ready to talk about it  -- Well, I don't have time to talk about this because we have two minutes left. But furthermore, the two can be throwing information back and forth to each other. 

@4575 So you can. ...  -- I tried to get this demo working today and couldn't.  But maybe by Thursday I will  ...   You can take the camera and point it at something and turn that into sound, like a waveform or a spectrum, @4590 or you can have a sound going into a microphone and have that effect how an image is happening  -- Like ... make an image jump around when you have a signal coming in to the microphone.

@4605 A lot of people have worked hard on coming up with cool and interesting ways of using this musically and visually. Some people call this "visual music" when you actually make designs that consist of @4620 both images and sound that are changing in time.

Although, of course, that's a music-centric name to give it. But the name "visual music" I think dates back almost a hundred years now. --  So it's not like the inventor of GEM @4635 thought that term up.

Yeah, so now you can all go out into clubs and make money because people will pay to dance to images jumping around the screen -- until they @4650 suddenly realize that that's not something you really dance effectively to and then it'll go out of fashion and then you won't be able to make money this way any more. [laughter]

So more on this next time.