Digital SoundLab Summer 2006

Our goals in this class will be threefold: to fill gaps in your understanding of the principles and basics of audio, to introduce you to digital audio technology, and to propose ways for you to use the same in your own creations. ITP has a wide assortment of talents, so our goal here is to adapt the course content to whatever will complement your own focus. For some, this may mean learning the basics of live recording, for others it may entail gaining proficiency in a particular audio app, etc.

The hardware and software we'll be using is nothing esoteric or difficult to obtain. You may very well be tempted to purchase your own copies during the course, and I'll be glad  to assist you in getting the best price. Student discounts are often available, both from the manufacturers and from third party distributors. There are several audio workstations on the floor, and our work will revolve around the software and hardware found in them.  We also have access to the multimedia workstations at 35 W.4th street (the ACF) 2nd floor, which has hardware and software similar to ours. Check with Gordy to find out how to get an account there. If you find it hard to book time in our labs, that may prove to be another useful source. From what I've seen, though, there is almost always space available on our own equipment.

This is an Audio class, not a Music class! While any sound used in multimedia, right down to the shortest button clicks, can be thought of in a  musical context, it is up to you to figure out what that means for your own artistic temperament. We will spend some time in the musical domain with MIDI, but your main focus will be the creation and manipulation of digital audio clips (MIDI is treated in more depth in my other class here at ITP).

GRADING

Grading will be determined by four elements: attendance (including PUNCTUALITY), participation, a midterm project, and a final project. Projects can be strictly sound based, or an integration of sound into another project, quicktime movie, Director project, web page, etc. Your attendance and participation are taken very seriously. This is a graduate level course and we sometimes move quickly, so missing a single class can easily cause you to fall behind. Likewise, if you come in late, you'll undoubtedly be the one to ask a question which has already been asked and answered…and cause your colleagues to give you dirty looks. You'll find that I often try to explain things by posing questions. By doing this I don't expect you to necessarily know the answer - rather, I hope you'll exercise your synapses a bit trying to think of possible answers or reasons. You know, the ol' Socratic method...

ASSIGNMENTS

Think of this class as a forum in which you're encouraged to contribute as much as possible through the weekly assignments. Each week has a suggested assignment, which you'll do in order to ‘concretize' the ideas and techniques we go over in class. I reiterate: you're all grad students, so I expect you to handle those assignments in the way which best fits your own needs. Many times you'll probably already have something in mind you want to try. In any case, you should always be prepared to play something!  This means you should always bring your work materials to class. At first this will mean a cassette tape of  your recording experiments, later it will mean a zip or jazz disk with digital audio and midi. If you don't bring in the materials then the punishment is that you have to listen to me lecture more!
More to the point on the above: As I mention below, we use the Blackboard site as a media storage facility, and we should try to make full use of it as a way for you to have material prepared to show. Once you start digitizing things, it will save a lot of time if you upload your work (smaller files, anyway) to the blackboard using the 'digital drop box' feature, as we'll explore in class. That way, you can just let me know at the beginning of class that you have something to share and we can quickly retrieve it from there, as opposed to having 20 people trying to find their stuff on USB drives and what not.

TEXTS

Although we don't follow a specific textbook, I do recommend a couple of books that parallel some of the topics we discuss. Many books on Pro Tools are on the market, and some are available at the NYU bookstore (and Amazon, of course). Sound Check by Tony Moscal (Hal Leonard, pub.) is a very readable and down to Earth introduction to the things you need to know about the electronics of sound. Another fine all around manual for analog aspects of audio production is Audio in Media, by Stanley Alton (ISBN 0-534-06156-7). It is expensive, but a good volume for your bookshelf. Science and Music, by Sir James Jeans is an excellent introduction to the general principles of audio. He covers a lot of ground in a small inexpensive paperback, which, despite the fact that it was written in 1937, still presents the material in a pleasant and readable form. Kenneth Pohlman's Principles of Digital Audio is a standard in the industry, and has just been updated. Not cheap, but it's very comprehensive. If you intend to go into audio systems design, or simply really want to understand what goes on in all that digital gear at the machine level, it would make a great reference. For MIDI, I use MIDI for the Professional,  by Lehrman & Tully. All of these books are available at the NYU bookstore. If you get excited about the musical side of things, you might want Curtis Rhoads's magnum opus The Computer Music Tutorial , available from the NYU computer store. I will hand out excerpts from various publications from time to time, and we shouldn't forget that source of sources, the WWW, from which we'll be extracting relevant information as it breaks. The audio field is in a continual state of flux, and the course content will be adjusted to reflect whatever new advances come our way during the semester, whether it's the latest form of audio compression, some new capturing technique, etc. For general reading and mind expansion I highly recommend The Elegant Universe, by Brian Greene, a highly readable exposition of the controversial topic of String Theory in physics (ISBN 0-375-70811-1), which anyone who thinks sound holds the secrets of the universe has to love. I recommend this book because it poetically postulates that the most fundamental material of the universe is composed of vibrations and harmonics, the essentials of sound! Nova has made a DVD on the book as well.

Gradually supplanting paper as a source of knowledge, of course, is the web itself. Here, for instance, is an excellent place to get started in your exploration of sound:
www.kettering.edu/~drussell/Demos.html

ONLINE HELP

ITP has a great resource page for help with the audio workstations. My nyu email is: daniel.palkowski@nyu.edu. In class I'll give you a preferred email address to use, though. I'll be around on Mon. afternoons before class, and will be available other times by appointment. You can also call me at work if you need assistance in the labs (can't get sound, midi, etc.)  I'll give the number in class. There is also a list where you can post questions, comments, news, etc., it is itp-sound. You get to it via home.nyu.edu, and subscribe with your net id and a password which I'll give in class. I'll also give you an IM address so you can send me an IM  from any pc if you have trouble or a question.  This course has a Blackboard account. You can find it under the Academics tab when you log into http://home.nyu.edu .As a class member you'll automatically be added to the blackboard list. The site serves as a repository for all sorts of class materials, and has things like email addresses for your colleagues, etc. Still, the itp-sound list has been around longer, so please use that to post your sound related questions. It's also frequented by such luminaries as Josh Goldberg, Luke Dubois and Jeremy Bernstein, who are always ready with a quick solution (or at least a snappy answer)…

Here, also, is a new link from Miller Puckett, one of the fathers of Max/MSP, an online version of his excellent treatise on digital sound, for your reference:

Audio hardware and software tend to be loaded with features and options, which has the side effect of making them complex to configure. Once they've been set, though, they tend to be quite stable... until someone comes along and decides to start changing settings. With the shift away from a hardware-centric workstation to the integrated soft-synth approach we're now using, things should be easier to maintain. That doesn't mean you should go into the Pro Tools I/O configuration and start rewiring the virtual routings, of course. Once everything has been installed, we'll configure the software so that you should be able to accomplish anything you want using the preset configs.

A Note on the Final Project

                You are encouraged to integrate your sound work as a component of a project for a different class, however, you must obviously be able to draw a clear delineation between the sound component and whatever work pertains to the other class. I'd say that the most successful projects tend to be those that were planned early in the semester. You'll notice that we introduce a lot of software in the class, and this is because each type of software — destructive digital audio, non destructive digital audio, MIDI software and file manipulation software — are the basic tools which you need to know. You probably won't use all of them for your project, so most of the time you'll need to be doing two different kinds of homework: that which will serve your project and that which will simply serve to give you some chops on the software itself. Please do follow the assignments, even if, for instance, the assignment is to create a MIDI file even though your project is a Pro Tools piece! It would be a good idea for you to try to conceptualize what you'd like to create as soon as possible. This will shape, for instance, the kinds of sounds you'll go out and collect.

Here is a general plan for material to be covered. The structure and content is subject to change, depending on your interests, new floor acquisitions, and the general state of a particular piece of hardware/software in the classroom:

WEEK 1: 09/12/05

 INTRODUCTION TO CLASS, orientation and introductions of class members; our goals;the syllabus

SOUND THEORY

Lecture: Theory of Sound: Structure. Frequency and amplitude, their nature and measurement. Wave structure and overtones. The oscilloscope. Resonance and sympathetic vibration demo. Examples of wave motion. Use of Pro Tools or Sound Forge to generate test signals. A couple of Sound Pieces.

Useful links:
http://positron.ps.uci.edu/~dkirkby/music/html/demos/PlaneWave/SoundWave.html
www.kettering.edu/~drussell/Demos.html
www.

main assignment:

Conceptualize (and, if possible, realize) a method for visually demonstrating wave motion. Be prepared to discuss, and if possible, show your demo. Again, refer to www.kettering.edu/~drussell/Demos.html for some ideas to get you started.

Use Audacity, Pro Tools or Sound Forge to create a set of reference sine tones at 0 dB. Each should be about 4 seconds long, at 10Hz, 30Hz, 60 Hz, 110 Hz, 220 Hz, 440 Hz, 1000Hz, 4000 Hz, and 10,000 Hz. Make a similar set of square waves at -10dB.  Also, play with the tone generator processors (I like Sound Forge's the best), particularly the FM tool, make a few short files with different settings. Be sure to preview as you play.

USE THE DIGITAL DROP BOX TO SEND ME ONE 5 SEC FILE CREATED AS ABOVE

in addition:

Familiarize yourself with the sound facilities at ITP. This includes the equipment which may be checked out of the equipment room (mics, recorders, MIDI gear, etc.), as well as the audio workstations themselves. Make sure you know how to use the ITP automated signout system.

selected terminology:

 * primary concepts

WEEK 2: 09/19/05

ANALOG AUDIO ELECTRONICS 1

Discussion:  your wave demos .      
                             
Lecture: Capturing sound electronically: Analog concepts. The Process of Transduction;  What is an input? What is an Output?; The Signal to Noise Ratio.. Lo and hi impedance devices; Balanced and unbalanced lines; What is a circuit?; Dealing with ground loops;  A.C./D.C.; Cables and Connectors, proper nomenclature. mic and line levels.

main assignment:

Go to the Pro Audio parts dept. at either Sam Ash (48th between B'way and 7th) or Manny's (ditto), and buy two mono ¼” phone plugs, a stereo mini plug, and some three conductor shielded cable (at least 10 feet). Radio Shack may or may not have the same things, unlikely that they'd have the cable. Solder yourself an adaptor cable from these parts, respecting the proper TRS routing. Make it well, you'll probably find it very useful. Here's a wiring diagram:

Sign out a recording device -- we have just purchased a slew of Marantz PMD660 solid state recorders, probably the best recording devices we've ever had on the floor -- and any two of the following: a Shure SM-57 mic; one of the (also new) Rode Condenser mics (if you use the latter, remember to activate Phantom Power on the recorder), or the Shure Shotgun mic (can use either a AA battery, or phantom power). You'll also need two standard M-XLR to F-XLR cables for the mics. Go to the WhisperRoom,  plug the mics into the two inputs of the recorder. Now you'll need to configure the device to match your inputs. These steps pertain to the PMD670, so I may need to modify them for the 660 by the time this gets to press, but should be fairly accurate:

Hold down the menu button until it starts flashing, then hit enter. Use the >> arrow to cursor till you see 'Algor/File'. Hit enter. Use >> and enter to set the recording to stereo, 48K, and PCM (each is its own setting). Once you've gone through the settings in the Algor/File dialog, hit menu again to store the settings. Now Plug your headphones into the device, so you can monitor sound as it comes in.  Now, take a few small objects, and use them to create short sound FX. The goal of this exercise is threefold: to make you comfortable with setting up basic recording, to familiarize you with a particular recording device, and most importantly, to start training your ear to work with sound electronically. Therefore, as you prepare the sounds for recording, listen to them both with and without the headphones, observe the way in which the mic colors and alters the sound.  Take notes & we will discuss your results in the following class.

NOTE: I highly suggest that you buy yourself a Compact Flash card. The digital recorders use the same kind of card as digital cameras. A 1 Gig card is recommended, so you don't have to constantly erase your work to make room for new.

USE THE DIGITAL DROP BOX TO SEND ME ONE OF YOUR FX (or more if you like)

selected terminology

WEEK 3: 09/30/05 (FRIDAY) 3:30-6:00

ANALOG SONIC ELECTRONICS 2

Discussion:  Review of workstation layout, routing, functionality.
Lecture/Demos: Microphones: types of, sensitivity patterns
The Mixing Board: Signal Flow, Matching impedances
I/O
Normalizing the studio
Sources of Sound  (line level vs. mic level)
 Recording Basics: using the cassette decks and mini disk recorders
Using sound FX libraries
Getting a good dub, common errors in recording.

main assignment:

Note: ITP doesn't have a relationship with the 11th floor film dept., so they may not like your using the CD transfer room. Used to be, if you handed them your NYU ID, that would be the end of it, but they may only allow film dept. students to use that resource. If that should happen to you, use the alternate method to collect your sounds.

Book some time in the CD Transfer Room on the 11th floor of this bldg. Bring along either a cassette,dat, DV Cam, or one of the digital recorders. If you use the latter, be sure to double check the recording parameters, as outlined above. If you made an adaptor cable, you can use the patchbay in the transfer room and plug directly into a DV Cam. They'll supply you with a couple of standard patch cables if you want to use the dat machine or cassette deck up there. The DV Cam will make a digital recording equal in quality to a DAT, provided you make sure to set it to 48K, (not 32).

As in class 2, Use the digital recorder to capture a series of 10" - 20" sounds. This time, from the moment you start hooking things together, keep a journal of the complete process, leaving out nothing, including a description of any problems you run into, and your solution to address them. SEND ME THIS USING THE DROPBOX. Keep a ‘take sheet'  showing each item recorded, listing location on the card, duration, and notes about the sound. You'll be using these sounds for a while, so take complete notes!

(example take sheet:):

selected terminology:

DIGITAL SOUND Basics 1

Lecture: Key discussion on principal differences between analog and digital techniques. What is easier, what is harder?  Is digital always better? An explanation of the digitizing process from ADC to DAC.   Frequency and Amplitude in the digital domain, focus on Frequency (Sampling rate)

Practical:  Setting up and capturing audio digitally using Sound Forge (pc), Pro Tools (mac), Audacity (PC), and Audition (mac). This part will be adjusted depending on the experience level of the class.

main assignment:

Study the audio digitizing systems on either a workstation, or if you have one, your laptop. On OSX (mac), make sure you understand the settings available in the built-in I/O, and compare the options with any external device you might have connected. In the workstations, that will be the DIGI 001.  On a pc, you can right click on the speaker icon on the task bar, and choose properties, then look at the settings for Recording and Playback. One neat trick, you can play anything while Audacity is launched and recording, and capture the streaming audio directly into it, by selecting MIX as the source.  Practice digitizing using the audio items you've recorded thus far. I suggest using 48K, 16 bit, stereo or mono depending on the source recording. Be sure to also capture a few clips at the lower settings (22k, 8 bit for instance) so you can compare the difference.

See this doc re: adjusting your input source and levels in Windows:

Adjusting Input/Output volume in Windows

Do a couple of complete recording cycles, from a live source to the completed digital file, make sure you understand each step of the process. Be prepared to discuss any problems you run into.

selected terminology:

DIGITAL SOUND Basics 2
DIGITIZING

Discussion: Review of previous weeks concepts, playback/critique of class work.

Practical: Intro to the Pro Tools GUI, comparison with Adobe Audition. Basic capturing techniques, level setting, ensuring a good transfer.  Working with dedicated hardware vs. built-in (software) systems. Theoretical: focus on Amplitude (Bit Depth). Comparison of PT features with Audacity and/or Adobe Audition.

main assignment:

Download and install the free 30 day trial of Adobe Audition. Take some of the sounds you've been recording and digitize them via any of the audio software we've explored (mac or pc). If you've recorded them on the digial recorder, then you'll just need to mount the device as a USB drive and pull the files onto your working drive. Experiment with the different save options (Sampling Rate and Bit Depth), and listen to the effect on the resultant sound files. Ultimately you'll want each file to be 16bit 48k, mono or stereo. Remember, digital audio is memory intensive. You'll have a more productive session if you work with shorter segments of sound, preferably no longer than a minute each. Start creating a personal library of sound bites, thinking carefully about how sounds should be organized for future ease of retrieval.

selected terminology/concepts:

Basic digital editing & DSP techniques

Discussion: Playback of class work.
Lecture: Basic Destructive Editing techniques.

Audigy/Sound Forge: Cutting, splicing, replacing, trimming, copying, etc. Integrating other programs for specific processes. Sample Rate conversion, bit depth conversion, making loops.  Playlists and regions. Basic DSP techniques.

main assignment:

Record a spoken paragraph by your favorite politician, newscaster, talk show host, televangelist, etc. Digitize the file and use basic cutting and splicing to make them say something funny. The trick is to make it sound natural and not spliced. Then make a couple of different versions of the final result with different basic DSP processes applied to each (reverb, echo, pitch bending or transposition, etc).

selected terminology:

Plug-Ins.

Discussion: Playback of vocal pieces.                                   
Lecture: Non-Destructive editing using Pro Tools and/or Digital Performer, with both realtime and nrt plug-ins. Further exploration of DSP techniques.

Practical: Creating a multi-track sound piece. Exploring RTAS plugins , which allow you to alter the parameters of a range of soundtracks. The idea of a virtual bus is introduced.

main assignment:

Create a new Pro Tools or Digital Performer session. Add several tracks, both mono and stereo. Import several of your sound bites into the session. Arrange them on the timeline to 1) make a non-rhythmic, ambient composition, and 2) make a periodic, rhythmic ‘beat' piece. Experiment with both the built-in DSP functions and the plug-ins. Make a few variations in the settings of the DSP and bounce the file each time, so that you have several similar ‘phrases'. Fly the phrases back into the session (mute the original tracks) and build a new piece out of them, Bounce the final to disk.  Note: you may have different sets of plug-ins depending on which machine you're on. Compare the nomenclature/implementation of DSP functions between the different apps. Decide and make a note of which program has the best implementation of each DSP feature (you'll be using them all sooner or later).

Using the same procedure, create a set of user interface sounds for yourself. These include button clicks, mouseovers, warnings, action sounds, etc. Create at least four sets, and try to give some consistency within each set. As you create these fx, use layering of tracks to give more interest, then bounce to disk. Apply various dsp functions to each one as you go.  I'd suggest naming them in a way that reflects what you intend for them to be associated with, e.g., mousedown1.wav, rollover4.wav, etc.

selected terminology:

MIDTERM PRESENTATIONS:

Using a single class of sounds (I recommend noise-based ones), create an abstract or narrative 1' sound piece which transforms the sounds in a number of ways, to synthesize something completely different from them. Use a multitrack environment, and demonstrate knowledge of volume and panning automation, plugins, creative use of reverberation and/or digital delay, etc. For class playback, the final bounced sound file will be fine, but please have a CD to hand in with the working files, including the Pro Tools, Audition or Audacity session.

Please send the bounced files to the Digital Drop Box, to avoid delays in setup.

MIDI

Discussion: Electronic Music. The Narrative vs. Interactive experience; Authorship vs. Anonymity; the question of sampling.

Lecture: Parts of an electronic music studio. A quick look at the MIDI specification. Intro to sequencer software.  The evolution of hardware into software. Propellerheads' Reason.

Practical: Intro to MIDI hardware and diagnostic software.  Getting sounds out of a synth, accessing the synth from the front panel. Basic connectivity: using a controller keyboard or input device.  Using a MIDI interface. Mark of the Unicorn Digital Performer. Recording MIDI. Linking Reason Adapted to Digital Performer or Pro Tools.

main assignment:

Run Digital Performer in one of the workstations. Download the Performer template from Blackboard and open it. Modify the settings to reflect the system you're using. Familiarize yourself with the Consolidated Controls and Tracks windows.  Using the ‘Output' column in the latter, set the output to one of the synths at the workstation. Caution: each station has a different set of synths, so the sounds you get will vary, often drastically, so be prepared. If 'General MIDI' (or 'GM') is available as a sound bank, use it.  Using the ‘default patch' column, browse through the available patches for that synth while playing notes on the keyboard to hear the changes. Make notes of patches that interest you.  After you've picked 10 or so patches, try the following:

1) Record notes from the keyboard in real time.

2) Step Record notes from the keyboard.

3) Insert notes using one of the editing windows.

The Performer preferences should set up the MIDI configuration correctly as soon as you open a new file.

SEND ME YOUR MIDI FILE VIA THE DROP BOX.

selected terminology:

MIDI and Digital Audio Integration

Practical: Combining digital audio and MIDI in Digital Performer and Pro Tools.

main assignment:

Make a backup copy of your MIDI file. Import soundbites from your library into the file and synchronize them to your midi tracks. Experiment with plug-in fx, and alter the tempo via the ‘change' menu. When you have it sounding like you want, create a new stereo voice audio track and record the sequence onto that track. Import the track(s) into a new Pro Tools session and apply plug-ins to it. Export the resultant file and compare it with your original.

selected terminology:

LAYING SOUND TO QUICKTIME VIDEO

Practical: Using Digital Performer to create sound for quicktime video.

main assignment:

Add music and or sound fx to one of the qt clips in the soundlab folder on the server using either Digital Performer  or Pro Tools. Use Quicktime Player Pro to combine the finished track with the video.  Suggestion: have two ambient tracks, four fx tracks, one stereo music track. Bounce the total down to stereo before adding to the movie. If Quicktime Player does not open into the Pro version, find a GA on the floor and get them to authorize it for you.

Sound on the Web

Practical:  Preparing your sound for use on the web. Using Flash, RealMedia and Windows Media. Synchronization of Audio with images, text, etc.. Types of compression, bitrates, 'quality IS quantity'.

selected terminology:

THINGS TO COME

 A demo of Opcode's MAX/MSP software, which comprises the main package in the MIDI class.

General summary, questions, help with projects, etc.

Final Presentations, Party, etc.