*** MUS171 #20 03 10 Miller: @0000 We'll finish up with GEM today. This is an example of taking audio and graphics and in some sense, bridging the divide between them. And this is a very good patch @0015 for you to load on your laptops while your professor drones away. [laughter] Basically, well, you can tell exactly what's happening. So, as I was saying, GEM is all about polygons. @0030 And what you see here is a bunch of polygons. But so far, I've shown you two rectangles, so this is a whole sample with a whole mess. So what I want to do is just sort of show you how you put together something very simple like this. @0045 And I've got to say, this is not going to win you any prizes being able to do this kind of thing. But the interesting stuff that I think that you can do with GEM, with geometric kind of modeling like this @0060 is things where you algorithmically generate hundreds of thousands or millions of polygons to make shapes or other kinds of collections of stuff in space. And, I don't have any good examples of that sitting here right now. So, I'm not going to @0075 try to develop one. Instead, I'm just going to show you this kind of very simple sort of generating example of "OK, here's how you would use audio analysis to drive a picture." And then, we'll just sort of leave GEM @0090 for future years. And then I have five other topics in Pd that I'm not going to have time to do properly, so I want to mention that they exist and give each one of them maybe 10-ish minutes each. @0105 That will be plenty for today. And then, we're going to be done for the quarter -- except of course for final projects. So, the plan for today is not just one thing: It's a bunch of things but each of them is @0120 only just kind of on the surface. One is, I'm going to finish up with GEM with this example. And then, the example actually, has is an example using audio analysis and there are lots and lots of things that you can do with @0135 analyzing sounds. And so, I want to show you the basic tools that Pd has available for doing audio analysis and some of the things that you can do with that. And then, there are other things that you might want to know about which I will tell you about @0150 as I get to them; but, basically, four other classes of things. Each of these are just sort of quick topics. "netreceive and netsend" is making network connections between computers. @0165 "readsf~ writesf~" is spooling audio to and from disc so that you can do things like make sound with Pd and have the sound file afterward. And, pd~ is a trick for doing Pd with multiprocessing. It's a way of having Pd sprout child @0180 Pd's that can run on other processors. So, each of those things takes a few minutes to describe and then they're done. Fourier analysis and re-synthesis: I'm going to just show you one very simple example of this, but this is a thing which you could easily study for @0195 months or years. In fact, it's true that most people have to study this and think they learn it and then come back a year later and realize that @0210 there's actually another point of view on it that you wanted to know it from. And so on for three or four iterations before you really have enough points of view down that you really believe that you know Fourier analysis and synthesis fluently. So, this is a thing that which I want to tell you the existence of, @0225 it will not start actually going on. Because it is just a big thing. But, anyway, back to the GEM example. so what's happening? Every time I talk, the sound is going to the microphone. @0240 So, we're having to do two things. Really, we are, measuring the loudness of a sound. That's a technique which is old and it dates back to the, at least to the analog synthesizer days, it's called "envelope following" and it's one of the three kinds of analysis that I am @0255 going to be showing you in some more detail next time. But at least now, what I'll do is show you how it works into this example. So, envelope following. This is kind of a mess. I like actually running sound files @0270 because if you have a recording of your favorite politician giving a speech, this is a great way to listen to that. So, what I am going to do is try to figure out where all the stuff is... (Not there. @0285 Come on. What am I doing wrong? Got that, got that. Audio in. Oh, "pd works" . Here we are. This is the real thing.) , so what does a bird consist of? @0300 Last but not least is going to be "pd sound" where we are actually getting the loudness of the sound coming into the microphone. So, I'm going to save that because for continuity sake, I will just talk about the graphics first -- even though @0315 the flow of information is from sound to graphics. So, I'm going to be going upstream in information flow here for a few minutes. So, what would you do? Well, these things are all, @0330 well, these, I'll do this in some detail. This, these hundred rectangles are going to be the bird's body. So, part of the trick to drawing the bird is making this shape, @0345 which is a very irregular shape that you can't make very simply out of polygons. So there is a thing for making a hundred polygons that makes that shape that I'll show you. Then, there are isolated things which are... Let's see. @0360 This thing is a rectangle, this twig that it's sitting on. The legs are I believe trapezoids although I have to go check. And, these eyes are actually hexagons. That's cheesy, but @0375 that's what I did. And this beak is three triangles. There are two triangles for the top part of it and just one triangle for the bottom. And, those triangles are the only thing in the whole thing that's moving; and in fact, the only thing that's moving in this, are @0390 four points in space which are: First off, the two sides of the beak which are [clapping noises 06:33] chosen at random whenever it finds a new attack. So that, basically, the width of the beak is sort of changing word by word @0405 except it's not working too great right now. And then the, you can tell that the two points in front of the beak -- which if there's no sound going at all are one point there. There are two points @0420 which lie on a vertical segment and they're some fixed point plus and minus the envelope value that is coming in from the incoming sound. So, these two are random, but are set off by attacks of sound. @0435 And these two are the continuously changing envelope. That's the whole deal. So, how do you do it? What I want to do is find some simple stuff first and then show you the complicated stuff. @0450 Here is simple stuff: I told you that everything is triangles and then of course here, I'm making a four vertex polygon which is a @0465 quadrilateral. If you make quadrilaterals in OpenGL or GEM, make sure that the four points are coplanar because it will do the wrong thing if you give it a skewed quadrilateral, @0480 that's a quadrilateral whose vertices are not planar. In this case, almost everywhere, the Z value... <