Dataflow @ ITP

Dataflow


ITP Help Pages


rss feed for this site


apple

RodsAndCones

Main.RodsAndCones History

Hide minor edits - Show changes to markup

April 26, 2012, at 12:31 PM by shf220 -
Changed lines 201-202 from:
to:
April 25, 2012, at 10:46 PM by shf220 -
Changed lines 193-194 from:
  • gen
to:

This week roughly will cover Jitter tutorials 41,42,43 and Appendix C

Additional resources:

  • cg shader tutorial
  • lighthouse tutorials
  • OpenGL shading Language
  • Syphon
April 19, 2012, at 10:28 PM by shf220 -
Changed lines 149-150 from:

<pre><code>

to:

[@

Changed lines 182-185 from:

</code></pre>

to:

@]

April 19, 2012, at 10:27 PM by shf220 -
Changed lines 144-184 from:
to:

Patches from class

Here's a patch with the correct way to fade between 2 videoplanes :

<pre><code>


begin_max5_patcher----------

1278.3oc4YssbpaCE8Y3qP0SejiGc218I5i8anSmLBr.bJXQrkykdly+d2Rf IPgDrMACSOSx.Xgrzxq8dsuH99vAASLupKCP+F5OQCF78gCF3GxMvfsWOHXk 50oKUk9oEL0rZkN2FLZy2Y0uZ2L9xro+MZgtPirFzLUpFMQaeQqyQqLOmAax 16XYVtdpoJ2eazsCNyjaKy9GsaLBMDuc30J6zEY4yenPO0tAkLIMjvivR9HD S.yDQDb2aL3Uzes25kqV4WufeuHSsrd6yR8iYl732Hw0ClWsJKeo15eDIuOn oxVOJ1M3OFNz8xnKjqrKznrxxJ30bje5nWTkH6BkE49tT8Z6hGz4pIK0Hk0V jMoxp8yoTac7Kwcmto9XlMb9xvmyR0l0KU45xPzeLC8loBo.SwbCvdGYPfab 0H+RMF1g7z5shfFevViOoMKp01LRDIjKYQbwHDkEFySXzHvvIkd6Gl0ZKm7J a4VoKKUy0GY4vg.dCQTLdG27AzvF.XeasdCGDDr6QbcgtDbKT1LS99d1wzP4 VO6HwNVB63H7dLzYDEQxc2pLxyuwsld4mjdomjdIcfdy0u.6yQr6u7MfbaKu NaoQYC9L1gCrIMAywBmOWbXBPTBfdXzPmCItszSxsgcbhPvQn09ciNv26DzC msyuifkgrHIkCel6UmsmdNs1jcR5g9EpMIgNcx0QatOGEwBY3XLMoYZyCt0K WaRuxNeeD6N0rzTf9Uxt+u.N9DjjLILJlwDRPV5zkDQDOZatgtPSrqLMAQcf Eoawp.A4DU97OkPjrjvXLSPiOLlk.2MQIemlXsp.lrUWrMKuOEYyylRuCIq8 CvSSfkjvvRZ2IKVxMjrtMBPIlFFIkXAPajjjPAiAdeWh.TdaxRdTAwPMvyMn w1BUd4LSwpGbQ3sHJZrpxZVAQ5mhvmqPXzXO8ux.UQW81yu0klYNxf.P8AX+ Kxd8r4nEIhcgCnBXMY7DA7YQLoa89vXAWW23epLOTvL3DLTP8vhngwIQTW5q Nadnjai4whlbve5BUotsAZ7wqG0922rYeJOSC4P8TtBUA.TWnJgD0s37TbKX 43ubQPA3RqK1J.7O7OrIHOTE6l+ae2P0bYCoQFTlgHBpXNx0YNsazHII31jR rPqRGgfxDRgrzesoC2q7KHzqjGCryHTrniMTSucICexF5O.Nz3p7rmp7QOUo p0V2GNcXyqQBrX760yBjqiQofCHcCk1AmtaXBrWxxSMurU35HJjjiQLIjGxW zp6L2pyQMoZ1LPj6tzTAIqpu9JKr4TnmANzPEzkJmjzMNN9+c5ZFTla8IQRR jgTLVjbAB6ne100zn2OzR2mcmLB+Rz0hqrOm0Le9xctbG8nmk+4mkHkTqv16 froG2m49sd+E0L4Wn+yS.LJLHZmBB0r.P6cTWh3NFhuihK+t3q8++7Ka4Ara 7CYrRSUwz5Gw55BQQ6.aptDBo6ORv8ljqpGz6OQKxRS046aRSyJcV4zO92fn U3AemgGwYviqCydBOtyZ3r7SOhGB6NCOzl3+v5W+G4cFdXmAOtVx6K+4FXt5 Q3Htufi2ZwOCdD8FbhZ.6zenw8S2dV3H6M3jz.zvw8FbZRdhj6Jzv5wr52Ud N7lTyCqGwSSRg1eYHZhvpGKvnIYH5pLeS47p0qeVWTtcM8HAZ04QSg6R4H+k Y4atzuhAE5mypmOena09wv+EQPoRT


end_max5_patcher-----------

</code></pre>

April 19, 2012, at 12:09 PM by shf220 -
Added lines 136-137:

This week includes items from Jitter tutorials 31, 32, 33, 34, 37, 40.

April 19, 2012, at 08:58 AM by shf220 -
Changed lines 7-8 from:
to:
 Also by popular demand Wed afternoons from 430-530 
April 19, 2012, at 08:56 AM by shf220 -
Changed line 145 from:
  • jit.gl.gridshapes
to:
  • shaders
Added line 148:
  • networking
April 19, 2012, at 08:53 AM by shf220 -
Added lines 128-129:

Patches from class

Added line 132:
  • jit.gl.gridshape
Added lines 135-141:

Supplemental reading

  • Vade's jitter movie playback optimization techniques
  • D. Schreiner et al., OpenGL Programming Guide (the "Red Book") (Reading: Addison-Wesley, 1999). ISBN: 0321335732
    An earlier version of the red book
  • E. Angel, OpenGL 1.2: A Primer (Reading: Addison-Wesley, 2001). ISBN: 0201741865
April 12, 2012, at 06:30 PM by shf220 -
Changed lines 119-122 from:
  • mxj and java in max for video

This week includes items from Jitter tutorials 27, 28, 45, 46, 47, 48, 51. There's also the above named objects. Additionally, there's a .pdf that explains (somewhat) what's going on with mxj.

to:
  • mxj and java in max for video

This week includes items from Jitter tutorials 27, 28, 45, 46, 47, 48, 51. There's also the above named objects. Additionally, there's a .pdf that explains (somewhat) what's going on with mxj.

Changed lines 126-127 from:
  • http://www.tudou.com/programs/view/kBpPBtQ48v0/
to:
  • Oskar Fischinger's Komposition in Blau
April 12, 2012, at 12:36 PM by shf220 -
April 12, 2012, at 12:35 PM by shf220 -
Changed lines 126-127 from:
to:
  • http://www.tudou.com/programs/view/kBpPBtQ48v0/
April 12, 2012, at 12:16 AM by shf220 -
Changed lines 115-122 from:
  • how to automate things with LFOs
  • audio to video
  • video to audio
  • scripting in max with javascript
  • mxj and java in max

This week includes items from Jitter tutorials 27, 28, 45, 46, 47, 48, 51

to:
  • how to automate things with LFOs (transport, phasor~, snapshot~)
  • audio to video (jit.poke~, jit.buffer~, jit.graph, jit.catch~)
  • video to audio (jit.release~, jit.peek~)
  • javascript in max for video
  • mxj and java in max for video

This week includes items from Jitter tutorials 27, 28, 45, 46, 47, 48, 51. There's also the above named objects. Additionally, there's a .pdf that explains (somewhat) what's going on with mxj.

Supplements

  • the Music Animation machine
  • Are you a Synesthete?
April 11, 2012, at 11:33 PM by shf220 -
Changed lines 112-113 from:

Patches from class

to:

Patches from class

Added lines 121-122:

This week includes items from Jitter tutorials 27, 28, 45, 46, 47, 48, 51

Changed line 137 from:
  • final questions
to:
  • final questions
April 06, 2012, at 01:23 AM by shf220 -
Added lines 112-113:

Patches from class

April 05, 2012, at 03:52 PM by bhl236 -
Added lines 44-46:

<<<<<<<

  • Byung han(Tony) Lim

=======

Changed lines 48-49 from:
to:

>>>>>>>

Changed line 133 from:
  • final questions
to:
  • final questions
April 05, 2012, at 01:24 PM by rs3836 -
Changed lines 44-45 from:
to:
  • Rose Schlossberg
April 05, 2012, at 11:45 AM by shf220 -
Changed lines 102-103 from:
to:
  • Famous new media artist jeremy bailey's public sculpture
April 05, 2012, at 11:29 AM by shf220 -
Changed line 66 from:

Patches from class\\

to:

Patches from class\\

Changed lines 79-80 from:

Patches from class : Patches from class

to:

Patches from class

Changed lines 123-125 from:
  • mxj
  • javascript
to:
  • gen
April 05, 2012, at 11:28 AM by shf220 -
Added lines 45-68:

Week 1:

  • why are we here, not in a metaphysical sense, but in a practical fashion?
  • What is video, how is it treated in Max via Jitter?
  • basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matrices.

differences between Max6 and PD. If you want to use an alternative (like softVNS for max5 or gridflow in PD or Quartz Composer, knock yourself out, but I will be of significantly less help)

  • Quick tour of the interface : browser, sidebar, attributes window. help patches and reference.
  • keyboard shortcuts for object creation.
  • Play a movie!
    • jit.qt.movie
    • attributes and arguments in Jitter
    • jit.pwindow vs jit.window
  • jit.fpsgui : matrix name
  • The jitter matrix!
    • Screen coordinates
    • planes, dims, types, named matricies, setcell, getcell, sends and receives in matrices.

Assignment : create a video player that moves to random points in a timeline. In class, we looked at building a scrubber with a mouse, expand on that idea and combine the player it with different types of input (ideas include Arduino, keypress, audio detection, etc etc etc). Look at the framebyframe.maxpat example for a starter.

Patches from class
Jitter Tutorials 1,2,3, 5, 14 & 16 are analogous to what we covered in class

April 05, 2012, at 11:26 AM by shf220 -
Deleted line 44:

http://itp.nyu.edu/~dis244/blog/?cat=21

April 05, 2012, at 11:26 AM by shf220 -
Changed lines 87-89 from:
to:
  • scripting in max with javascript
  • mxj and java in max
April 05, 2012, at 11:23 AM by shf220 -
Changed lines 73-74 from:

This week covers the cv.jit help patches, and Jitter tutorial 25. Also the max patcher storage Tutorials.

to:

This week covers the cv.jit help patches, and Jitter tutorial 25. Also the max patcher storage Tutorials.

Added lines 80-82:

Assignment
We looked at means of storing and retrieving data with the pattr system, as well as different video tracking techniques. Either take the effect based patch you had from last week and incorporate some sort of presets system using pattr, or create a fun* use of the computer vision tools we looked at in class.

April 04, 2012, at 10:49 PM by mdr361 -
Added line 42:
  • Matthew Rader
April 04, 2012, at 02:07 PM by dis244 -
Changed lines 42-43 from:
  • Johann Diedrick | DongIk(D.I)Shin]]
to:
  • DongIk(D.I)Shin
April 04, 2012, at 02:06 PM by dis244 -
Changed lines 42-72 from:

Some nice pages for insiration and reference
*Andrew Benson's Jitter recipes Book 1, Book 2, Book 3, Book 4.

Course Schedule

Week 1:

  • why are we here, not in a metaphysical sense, but in a practical fashion?
  • What is video, how is it treated in Max via Jitter?
  • basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matrices.
  • differences between Max6 and PD. If you want to use an alternative (like softVNS for max5 or gridflow in PD or Quartz Composer, knock yourself out, but I will be of significantly less help)
    • Quick tour of the interface : browser, sidebar, attributes window. help patches and reference.
    • keyboard shortcuts for object creation.
  • Play a movie!
    • jit.qt.movie
    • attributes and arguments in Jitter
    • jit.pwindow vs jit.window
    • jit.fpsgui : matrix name
  • The jitter matrix!
    • Screen coordinates
    • planes, dims, types, named matricies, setcell, getcell, sends and receives in matrices.

Assignment : create a video player that moves to random points in a timeline. In class, we looked at building a scrubber with a mouse, expand on that idea and combine the player it with different types of input (ideas include Arduino, keypress, audio detection, etc etc etc). Look at the framebyframe.maxpat example for a starter.

Patches from class

Jitter Tutorials 1,2,3, 5, 14 & 16 are analogous to what we covered in class

to:
  • Johann Diedrick | DongIk(D.I)Shin]]

http://itp.nyu.edu/~dis244/blog/?cat=21

April 02, 2012, at 01:37 PM by shf220 -
Changed lines 43-45 from:

Some nice pages for insiration and reference Andrew benson's Jitter recipes Book 1, Book 2, Book 3, Book 4.

to:

Some nice pages for insiration and reference
*Andrew Benson's Jitter recipes Book 1, Book 2, Book 3, Book 4.

April 02, 2012, at 01:37 PM by shf220 -
Changed lines 43-45 from:
to:

Some nice pages for insiration and reference Andrew benson's Jitter recipes Book 1, Book 2, Book 3, Book 4.

April 02, 2012, at 11:36 AM by shf220 -
Added lines 98-104:

This week covers the cv.jit help patches, and Jitter tutorial 25. Also the max patcher storage Tutorials.

Supplemental readings

  • Golan Levin's notes on computer vision for artists and designers
  • David Rokeby
April 02, 2012, at 11:31 AM by shf220 -
Added line 86:
  • video feedback and fractals
April 01, 2012, at 10:48 PM by shf220 -
Changed lines 89-90 from:
to:
  • Kurt Ralske
April 01, 2012, at 10:33 PM by shf220 -
Changed lines 94-95 from:
to:
  • pattr and presets
Changed lines 110-111 from:
  • syphon.
to:
  • syphon
  • mxj
  • javascript
April 01, 2012, at 10:29 PM by shf220 -
Changed line 86 from:
  • MacObject Database
to:
  • MaxObject Database
April 01, 2012, at 10:29 PM by shf220 -
Added lines 69-70:

Jitter Tutorials 1,2,3, 5, 14 & 16 are analogous to what we covered in class

Added lines 83-84:

We covered things that are in Jitter tutorial 8, 10, 11, 15, 16, & 17

March 30, 2012, at 08:53 PM by shf220 -
Changed lines 79-80 from:

Patches from class : Patches from class

to:

Patches from class : Patches from class

March 30, 2012, at 05:23 PM by shf220 -
Changed lines 79-85 from:
to:

Patches from class : Patches from class

Links and things mentioned :

  • MacObject Database
  • Auv-i
  • A-chaos library
March 29, 2012, at 01:42 PM by jjd401 -
Changed lines 41-43 from:
  • Johann Diedrick
to:
  • Johann Diedrick
March 29, 2012, at 01:42 PM by jjd401 -
Changed lines 41-43 from:
  • Your Name
to:
  • Johann Diedrick
March 29, 2012, at 01:01 PM by kb1364 -
Added line 40:
  • Kevin Bleich
March 29, 2012, at 12:37 PM by nv468 -
Added line 39:
  • Phan
Added line 42:
March 29, 2012, at 12:32 PM by jhm309 -
Added line 38:
  • Jihyun Moon
Changed lines 40-41 from:
  • Your Name
to:
March 29, 2012, at 11:32 AM by nj541 -
Added line 37:
  • Nicholas Johnson
March 29, 2012, at 11:13 AM by ajl443 -
Added line 36:
  • Andrew Lazarow
March 29, 2012, at 11:02 AM by shf220 -
Changed line 68 from:
  • jit.roata
to:
  • jit.rota
Added lines 72-74:

Assignment : Use at least one of the built in effects in Jitter, plus feedback, to make something interesting! If the built in tools aren't your cup of tea, try using a third party library like Auv-i.

March 29, 2012, at 10:24 AM by ml3003 -
Added line 35:
  • Mira Leytes
March 29, 2012, at 08:31 AM by ml3003 -
March 29, 2012, at 02:43 AM by ecl303 -
Added line 34:
  • Ezer Longinus
Changed lines 37-38 from:
  • Your Name
to:
March 29, 2012, at 12:18 AM by mk3981 -
Added line 33:
  • Mark Kleback
Changed lines 37-38 from:
  • Your Name
to:
March 28, 2012, at 09:26 PM by jwk328 -
Changed line 31 from:
  • Jee Won Kim
to:
  • Jee Won Kim
March 26, 2012, at 08:49 PM by lp1251 -
Changed line 32 from:
  • Lisa Park
to:
  • Lisa Park
March 26, 2012, at 08:48 PM by lp1251 -
Added line 32:
  • Lisa Park
Changed lines 37-38 from:
  • Your Name
to:
March 26, 2012, at 05:47 PM by jwk328 -
Added line 31:
  • Jee Won Kim
Changed lines 37-38 from:
  • Your Name
to:
March 23, 2012, at 04:10 PM by shf220 -
Added lines 30-37:

Student pages

  • Your Name
  • Your Name
  • Your Name
  • Your Name
  • Your Name
  • Your Name
March 22, 2012, at 04:55 PM by shf220 -
Changed lines 51-52 from:

Assignment : create a video player that moves to random points in a timeline. In class, we looked at building a scrubber with a mouse, expand on that idea and combine the player it with different types of input (ideas include Arduino, keypress, audio detection, etc etc etc)

to:

Assignment : create a video player that moves to random points in a timeline. In class, we looked at building a scrubber with a mouse, expand on that idea and combine the player it with different types of input (ideas include Arduino, keypress, audio detection, etc etc etc). Look at the framebyframe.maxpat example for a starter.

Patches from class

March 22, 2012, at 10:30 AM by shf220 -
Changed lines 22-23 from:

There is an activity for nearly every class. They are very short, simple activities. These are the basic steps you need to go through to understand the principles discussed in class each week. They're designed to help you not only to understand the technical details, but also to get a feel for what the technology can do, so that you can incorporate it into actual projects. You should at least complete the steps outlined each week, so that you understand practically what it is we're talking about. Document on your blog any discoveries you make, pitfalls you hit, and details not covered in the class that you think will be useful for your fellow students and future students in this class.

to:

There is a homework assignment for nearly every class. They are very short, simple activities. These are the basic steps you need to go through to understand the principles discussed in class each week. They're designed to help you not only to understand the technical details, but also to get a feel for what the technology can do, so that you can incorporate it into actual projects. You should at least complete the steps outlined each week, so that you understand practically what it is we're talking about. Document on your blog any discoveries you make, pitfalls you hit, and details not covered in the class that you think will be useful for your fellow students and future students in this class.

Added lines 26-29:

Final Project

At the end of the semester, you will be expected to present a final project that uses dataflow language for video in some compelling fashion. This can take the form of an installation, performance, software, what have you. You are free to combine this final with that of another class, but make sure you let the other instructor know what you're doing. We will start to discuss these in week 4.

March 22, 2012, at 08:50 AM by shf220 -
Changed lines 14-15 from:

It;s probably best to BYO laptop to class. It is very helpful to use while we go over particular topics, and learn by doing in class. However, they can also be a super distraction. Whenever classmates are presenting or we're in the midst of a class discussion, however, please keep your laptop closed. The quality of the class depends in large part on the quality of your attention and active participation, so please respect that and close your lid.

to:

It's probably best to BYO laptop to class. It is very helpful to use while we go over particular topics, and learn by doing in class. However, they can also be a super distraction. Whenever classmates are presenting or we're in the midst of a class discussion, however, please keep your laptop closed. The quality of the class depends in large part on the quality of your attention and active participation, so please respect that and close your lid.

Changed line 33 from:
  • differences between max6 and PD. Will be teaching in Max, if you want to use an alternative (like softVNS for max5 or gridflow in PD or Quartz Composer, knock yourself out, but I will be of significantly less help)
to:
  • differences between Max6 and PD. If you want to use an alternative (like softVNS for max5 or gridflow in PD or Quartz Composer, knock yourself out, but I will be of significantly less help)
Changed lines 38-39 from:
  • jit.qt.movie (file types, codecs read, autostart, loop, unique, gettime, frame, randomly jump around in timeline. adapt, dim) show how movie continues to play even when metro is off. looppoints. qmetro vs metro. @colormode uyvy)
  • use mouse to control playback, scrubbing. Use keyboard to ump to different points in a film
to:
  • jit.qt.movie
  • attributes and arguments in Jitter
Changed lines 45-46 from:
  • planes, dims, types, named matricies, setcell, getcell (pack vs pak) sends and receives in matrices.
to:
  • planes, dims, types, named matricies, setcell, getcell, sends and receives in matrices.
March 21, 2012, at 04:20 PM by shf220 -
Deleted lines 7-12:

Grading

Participation & Attendance: 40%
Production Assignments: 40%
Journal: 20%

Changed lines 14-15 from:

Laptop use is fine if you are using your laptop to present in class, or if we're in the middle of an exercise that makes use of it. Whenever classmates are presenting or we're in the midst of a class discussion, however, please keep your laptop closed. The quality of the class depends in large part on the quality of your attention and active participation, so please respect that and close your lid.

to:

It;s probably best to BYO laptop to class. It is very helpful to use while we go over particular topics, and learn by doing in class. However, they can also be a super distraction. Whenever classmates are presenting or we're in the midst of a class discussion, however, please keep your laptop closed. The quality of the class depends in large part on the quality of your attention and active participation, so please respect that and close your lid.

Changed lines 22-29 from:

There is an activity for nearly every class. They are very short, simple activities. These are the basic steps you need to go through to understand the principle discussed in class each week. They're designed to help you not only to understand the technical details, but also to get a feel for what the technologies we're discussing can do, so that you can incorporate them into actual applications. You should at least complete the steps outlined each week, so that you understand practically what it is we're talking about. Document on your blog any discoveries you make, pitfalls you hit, and details not covered in the class or the lab that you think will be useful for your fellow students and future students in this class.

Production Assignments

For production assignments, you'll be expected to present your project in class on the day that it's due. If you're working in a group, all group members should be present, and should participate equally in the presentation.

Journal & Documentation

You are expected to keep an online journal of your progress. The purpose of the journal is twofold. First, it is a valuable way for you to communicate to your instructor that you are keeping up with the work in the class. We read the journals to see how students are doing, so you should update your journal regularly throughout the semester. At a minimum, reference to each week's work is expected, and thorough documentation of the production projects and technical research. Second, the journal is a way to document your work for your own use and that of others. Many ITP students have found themselves using their journals as a place to store notes, code samples, and more.

to:

There is an activity for nearly every class. They are very short, simple activities. These are the basic steps you need to go through to understand the principles discussed in class each week. They're designed to help you not only to understand the technical details, but also to get a feel for what the technology can do, so that you can incorporate it into actual projects. You should at least complete the steps outlined each week, so that you understand practically what it is we're talking about. Document on your blog any discoveries you make, pitfalls you hit, and details not covered in the class that you think will be useful for your fellow students and future students in this class.

I expect you to have a blog/journal/publicly available website where you will be posting the weekly work.

Changed lines 28-32 from:

Week 1:

  • What is video, why are we here, what other questions can we ask about ourselves?
  • basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matricies.
  • differences between max6 and PD. Will be teaching in Max, if you want to use an alternative (like softVNS for max5 or gridflow in PD, knock yourself out)
to:

Week 1:

  • why are we here, not in a metaphysical sense, but in a practical fashion?
  • What is video, how is it treated in Max via Jitter?
  • basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matrices.
  • differences between max6 and PD. Will be teaching in Max, if you want to use an alternative (like softVNS for max5 or gridflow in PD or Quartz Composer, knock yourself out, but I will be of significantly less help)
Added line 36:
Changed lines 45-57 from:
  • planes, dims, types, named matricies, setcell, getcell (pack vs pak) sends and receives in matricies.

Week 2: using the video grabber, some built in effects, automating things. jit.rota

3 computer vision, jit.gen

4 LFOs, audio to video, video to audio

5 openGL for optimization. drawing with jit.gl.sketch

6 More GL, slabs, it.gl.gridshapes Using with other applications : syphon.

7 show your work, final questions

to:
  • planes, dims, types, named matricies, setcell, getcell (pack vs pak) sends and receives in matrices.

Assignment : create a video player that moves to random points in a timeline. In class, we looked at building a scrubber with a mouse, expand on that idea and combine the player it with different types of input (ideas include Arduino, keypress, audio detection, etc etc etc)

Week 2:

  • using live video as an input
  • some built in matrix based effects
    • jit.xfade
    • jit.roata
    • feeback
  • building effect chains

Week 3

  • computer vision and additional libraries in Jitter
    • basic tracking and presence detection
    • cv.jit package

Week 4

  • how to automate things with LFOs
  • audio to video
  • video to audio

Week 5

  • Intro to openGL for optimization.
  • drawing with jit.gl.sketch

Week 6

  • More GL
    • slabs
    • jit.gl.gridshapes
  • Using Jitter with other applications
    • syphon.

Week 7

  • show your work
  • final questions
March 20, 2012, at 03:31 PM by shf220 -
Changed lines 39-64 from:

1)basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matricies.

differences between max6 and PD. Will be teaching n Max, if you want to use an alternative (like softVNS for max5 or gridflow in PD, knock yourself out)

Quick tour of the interface : browser, sidebar, attributes window. help patches and reference.

keyboard shortcuts for object creation.

Play a movie! bang makes us do it

jit.qt.movie (file types, codecs read, autostart, loop, unique, gettime, frame, randomly jump around in timeline. adapt, dim) show how movie continues to play even when metro is off. looppoints. qmetro vs metro. @colormode uyvy

use mouse to control playback, scrubbing. Use keyboard to ump to different points in a film

jit.pwindow vs jit.window

jit.fpsgui : matrix name

The jitter matrix! Screen coordinates planes, dims, types, named matricies, setcell, getcell (pack vs pak) sends and receives in matricies.

2 using the video grabber, some built in effects, automating things. jit.rota

to:

Week 1:

  • What is video, why are we here, what other questions can we ask about ourselves?
  • basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matricies.
  • differences between max6 and PD. Will be teaching in Max, if you want to use an alternative (like softVNS for max5 or gridflow in PD, knock yourself out)
    • Quick tour of the interface : browser, sidebar, attributes window. help patches and reference.
    • keyboard shortcuts for object creation.
  • Play a movie!
    • jit.qt.movie (file types, codecs read, autostart, loop, unique, gettime, frame, randomly jump around in timeline. adapt, dim) show how movie continues to play even when metro is off. looppoints. qmetro vs metro. @colormode uyvy)
    • use mouse to control playback, scrubbing. Use keyboard to ump to different points in a film
    • jit.pwindow vs jit.window
    • jit.fpsgui : matrix name
  • The jitter matrix!
    • Screen coordinates
    • planes, dims, types, named matricies, setcell, getcell (pack vs pak) sends and receives in matricies.

Week 2: using the video grabber, some built in effects, automating things. jit.rota

March 20, 2012, at 03:28 PM by shf220 -
Changed lines 2-25 from:
to:

Boxes and Lines for Rods and Cones

Spring 2012

Grading

Participation & Attendance: 40%
Production Assignments: 40%
Journal: 20%

Participation & Attendance

Showing up on time, engaging in the class discussion, and offering advice and critique on other projects in the class is a major part of your grade. Please be present and prompt. Lateness will hurt your grade. If you're going to be late or absent, please email me in advance. If you have an emergency, please let me know as soon as you can. Please turn in assignments on time as well.

Laptops

Laptop use is fine if you are using your laptop to present in class, or if we're in the middle of an exercise that makes use of it. Whenever classmates are presenting or we're in the midst of a class discussion, however, please keep your laptop closed. The quality of the class depends in large part on the quality of your attention and active participation, so please respect that and close your lid.

Mobile Phones

Please put them on vibrate or turn them off before you come to class unless they are part of your project.

Changed lines 28-44 from:
  • There will be regular homework assignments to supplement the in-class learning. The assignments are designed around the specific class topics. If you are working on an outside project that is related to the current homework topic, work on that project instead of the assignment. It must be related to the current class topic though.
  • Upon completion, upload all homework patches to class wiki, in the Students section. All homework patches must be uploaded there.
  • There will be some class time to present homework. You can optionally present any homework assignment during the semester, you are strongly encouraged to present at least a few times throughout the semester.
  • The assignments and readings are always due the week after they are assigned (i.e. the next class).

Final Project

The central goal of this class is to start developing fluency in the Pd programming language so that you are able to express ideas in the form of functioning projects. The class focuses on the programming language itself, and works best when you bring in project ideas to work on. Then you can focus on building real projects while developing fluency in Pd. Therefore, this class ends with a final project. It should be a substantial chunk of work, but it need not be something entirely new. Working on a project for a good chunk of the class is the best for learning, but it is also acceptable to present some smaller chunks as long as you can demonstrate basic understanding of what it takes to complete a larger project.

Breakdown of Example Projects

One approach to becoming good at a programming language is to learn a set of building blocks very well, then think in terms of those building blocks. This is the approach that this class aims to teach. To that goal, we will see a demo of a related project every week, talk about how it was done, and break it down. The central idea is to focus on what is happening rather than whether we liked it or not. The discussion can cover any topic, the only thing that is not allowed are judgments, like "I liked it", or "that sucked", or "that's a nice guitar sound". Instead we should talk about how to implement it using the tools we know, whether the idea was well implemented, whether a different approach would have been more effective, etc.

Laptops

During much of the classtime, you will be following along and programming on your own laptop. Therefore it is essential that you bring your laptop to every class. That said, laptops are also very effective instruments of distraction. Everyone benefits if we all pay attention. I'll do my best to keep the class interesting, I hope you'll join me in this pursuit. You are welcome to use your laptop in class when I am speaking, or when it is relevant to the classwork being presented. However, during discussions and when your fellow students are talking, please be respectful of everyone's time and close the lid. If necessary, I'll remind you of this, but even better would be if everyone does so on their own.

to:

There is an activity for nearly every class. They are very short, simple activities. These are the basic steps you need to go through to understand the principle discussed in class each week. They're designed to help you not only to understand the technical details, but also to get a feel for what the technologies we're discussing can do, so that you can incorporate them into actual applications. You should at least complete the steps outlined each week, so that you understand practically what it is we're talking about. Document on your blog any discoveries you make, pitfalls you hit, and details not covered in the class or the lab that you think will be useful for your fellow students and future students in this class.

Production Assignments

For production assignments, you'll be expected to present your project in class on the day that it's due. If you're working in a group, all group members should be present, and should participate equally in the presentation.

Journal & Documentation

You are expected to keep an online journal of your progress. The purpose of the journal is twofold. First, it is a valuable way for you to communicate to your instructor that you are keeping up with the work in the class. We read the journals to see how students are doing, so you should update your journal regularly throughout the semester. At a minimum, reference to each week's work is expected, and thorough documentation of the production projects and technical research. Second, the journal is a way to document your work for your own use and that of others. Many ITP students have found themselves using their journals as a place to store notes, code samples, and more.

March 20, 2012, at 03:23 PM by shf220 -
Changed lines 24-128 from:

(1/28) Week 1: Pd Overview

  • Class introductions
  • Course overview
  • What is Pd? The Max family of programming languages
  • Pd and read/write tools
  • you own Pd, do whatever you want with it!
  • Example Project: Attach:camera_wavetable.pd
  • In-class Demo: ring modulate your voice and see it
  • Reading:
    • Hackers & Painters, by Paul Graham
    • FLOSS Manuals: Pure Data Intro, What Is Realtime Graphical Programming? and The Interface
  • Assignment:
    • download and install Pd-extended
    • In Pd-extended, go to the Help menu, then choose Browser, then navigate to Manuals -> 0.Intro. In there, play with patches 01.hello_world.pd through 27.Example-all_just_data.pd
    • try adding to the ring modulator patch we built in class

(2/4) Week 2: What is Dataflow?

  • History of Max and Pd
  • bang!
  • objects and connections
  • always running
  • time and [metro]
  • numeric and symbolic "atoms, pointers
  • "symbol", "float", "list"
  • execution order and [trigger]
  • hot and cold inlets
  • Example Project: song in a patch: christchurch.pd
  • In-class Demo: simple sequencer with metro, a counter, and select (here are some WAV files to play with: continuous soft and relaxing.zip)
  • Reading:
    • FLOSS Manuals Pure Data: Dataflow Intro, Sequencer, Messages, Math, Order Of Operations
    • Miller's Pd Documentation, chapter 2 "Theory of Operation", sections 2.2 "how to edit patches", 2.3 "messages"
    • Optional: Music Lectures by Dr.Hdez: Pure Data Lessions (video tutorials)
  • Assignment:
    • make a sequencer of sounds and samples using metro, a counter, trigger, select, readsf~, osc~, noise~, phasor~, etc.

(2/11) Week 3: processing generic messages

  • [route], [select], [list], [moses]
  • messages and selectors
  • lists vs. other messages
  • translating "if", "for", etc to dataflow
  • Example Project: Solenoid Concert, an example of sequencing
  • In-class Demo: sequence sound and video (example media: Pd-SMU.zip)
  • Reading:
    • FLOSS Manuals Pure Data: Lists, Cordless Connections
    • Miller's Pd Documentation, chapter 2 "Theory of Operation", section 2.6 "semantics" (2.6.1 through 2.6.5)
    • In Pd-extended, go to the Help menu, then choose Browser, then navigate to Manuals -> MessageOddness and browse thru these odd cases of messages.
  • Assignment:
    • use the mouse data from [cursor], or other source of sensor data like [hid] or [arduino] to make a simple instrument using sounds/samples, [line~], [trigger], readsf~, osc~, noise~, phasor~, etc.

(2/18) Week 4: lines and signals

  • functions to generate lines over time
  • [line], [line~], [vline~] and simple lists
  • control rate versus audio rate
  • special data type with distinct connection type
  • In-class demo: using lines to control sound and video
  • Example Project: frey-wubwubwub.tar.gz
  • Example Project: Terre Natale EXITS Part 2 (scoring to video)
  • Reading:
    • Programming Electronic Music in Pd: Line
    • FLOSS Manuals Pure Data: Envelope Generator
    • In Pd-extended, go to the Help menu, then choose Browser, then navigate to Pure Data --> 3.audio.examples, and read through A03.line.pd and A04.line2.pd
  • Assignment:
    • use a line object to control sound and/or video playback

(2/25) Week 5: storing data

  • storing numeric data in arrays
  • visual representation of that data
  • read, write, modify methods
  • arrays as audio, controlled by line~
  • In-class Demo: arrays and tables for storage and control
  • Example Project: procedural audio in EA's Spore
  • Example Project: The first patcher language, in User Interface I (Alan Kay), 23:30-26:30 (excerpt on local mirror)
  • Reading:
    • FLOSS Manuals: Pure Data, Arrays, Graphs, Tables
    • Miller's Pd Documentation, chapter 2 "Theory of Operation", section 2.8 "numeric arrays"
    • Optional: chapter 2 "Theory of Operation", section 2.9 "data structures"
  • Assignment:
    • Pick a final project and start working on it. Bring ideas, problems, and questions for the next class.

(3/4) Week 6: encapsulation

The key to building large projects is encapsulating ideas into reusable chunks. The difference between a code block and a function: subpatches are code blocks and abstractions are functions, kind of. The real power of abstractions is not, that they are in their own files so they can be used for code reuse. The real power is, that they can accept arguments.

  • subpatches
  • writing objectclasses (aka "abstractions")
  • using arguments, inlets, outlets
  • installing externals
  • In-class Demo: turning example patches into objects
  • Example Project: reacTable: improvisation demo
  • Example Project: Dinosaurs, by Damian Stewart, download: dinosaurs.zip
  • Reading:
    • FLOSS Manuals: Pure Data, Subpatches, Abstractions, Dollar Signs
    • optional: Miller's Pd Documentation, chapter 2 "Theory of Operation" 2.7 "subpatches"
    • optional: FLOSS Manuals: Pure Data, Graph-On-Parent (GOP)
  • Assignment:
    • Finish final project for presentation in the next class

(3/11) Week 7: Final Project Presentations

  • Show and discuss your project in class, we can spend about 8 minutes per presentation.
    • 8 minutes per project
    • one sentence about your idea
    • show your project
    • talk about one thing that you felt was super successful in your project
    • talk about one thing that was frustrating
to:

1)basics : you already know how to patch things together (thanks Hans!). Now we’re going to apply that knowledge to the world of video and matricies.

differences between max6 and PD. Will be teaching n Max, if you want to use an alternative (like softVNS for max5 or gridflow in PD, knock yourself out)

Quick tour of the interface : browser, sidebar, attributes window. help patches and reference.

keyboard shortcuts for object creation.

Play a movie! bang makes us do it

jit.qt.movie (file types, codecs read, autostart, loop, unique, gettime, frame, randomly jump around in timeline. adapt, dim) show how movie continues to play even when metro is off. looppoints. qmetro vs metro. @colormode uyvy

use mouse to control playback, scrubbing. Use keyboard to ump to different points in a film

jit.pwindow vs jit.window

jit.fpsgui : matrix name

The jitter matrix! Screen coordinates planes, dims, types, named matricies, setcell, getcell (pack vs pak) sends and receives in matricies.

2 using the video grabber, some built in effects, automating things. jit.rota

3 computer vision, jit.gen

4 LFOs, audio to video, video to audio

5 openGL for optimization. drawing with jit.gl.sketch

6 More GL, slabs, it.gl.gridshapes Using with other applications : syphon.

7 show your work, final questions

March 20, 2012, at 11:40 AM by shf220 -
Added lines 1-128:

This syllabus is liable to change

Assignments

  • There will be regular homework assignments to supplement the in-class learning. The assignments are designed around the specific class topics. If you are working on an outside project that is related to the current homework topic, work on that project instead of the assignment. It must be related to the current class topic though.
  • Upon completion, upload all homework patches to class wiki, in the Students section. All homework patches must be uploaded there.
  • There will be some class time to present homework. You can optionally present any homework assignment during the semester, you are strongly encouraged to present at least a few times throughout the semester.
  • The assignments and readings are always due the week after they are assigned (i.e. the next class).

Final Project

The central goal of this class is to start developing fluency in the Pd programming language so that you are able to express ideas in the form of functioning projects. The class focuses on the programming language itself, and works best when you bring in project ideas to work on. Then you can focus on building real projects while developing fluency in Pd. Therefore, this class ends with a final project. It should be a substantial chunk of work, but it need not be something entirely new. Working on a project for a good chunk of the class is the best for learning, but it is also acceptable to present some smaller chunks as long as you can demonstrate basic understanding of what it takes to complete a larger project.

Breakdown of Example Projects

One approach to becoming good at a programming language is to learn a set of building blocks very well, then think in terms of those building blocks. This is the approach that this class aims to teach. To that goal, we will see a demo of a related project every week, talk about how it was done, and break it down. The central idea is to focus on what is happening rather than whether we liked it or not. The discussion can cover any topic, the only thing that is not allowed are judgments, like "I liked it", or "that sucked", or "that's a nice guitar sound". Instead we should talk about how to implement it using the tools we know, whether the idea was well implemented, whether a different approach would have been more effective, etc.

Laptops

During much of the classtime, you will be following along and programming on your own laptop. Therefore it is essential that you bring your laptop to every class. That said, laptops are also very effective instruments of distraction. Everyone benefits if we all pay attention. I'll do my best to keep the class interesting, I hope you'll join me in this pursuit. You are welcome to use your laptop in class when I am speaking, or when it is relevant to the classwork being presented. However, during discussions and when your fellow students are talking, please be respectful of everyone's time and close the lid. If necessary, I'll remind you of this, but even better would be if everyone does so on their own.

Course Schedule

(1/28) Week 1: Pd Overview

  • Class introductions
  • Course overview
  • What is Pd? The Max family of programming languages
  • Pd and read/write tools
  • you own Pd, do whatever you want with it!
  • Example Project: Attach:camera_wavetable.pd
  • In-class Demo: ring modulate your voice and see it
  • Reading:
    • Hackers & Painters, by Paul Graham
    • FLOSS Manuals: Pure Data Intro, What Is Realtime Graphical Programming? and The Interface
  • Assignment:
    • download and install Pd-extended
    • In Pd-extended, go to the Help menu, then choose Browser, then navigate to Manuals -> 0.Intro. In there, play with patches 01.hello_world.pd through 27.Example-all_just_data.pd
    • try adding to the ring modulator patch we built in class

(2/4) Week 2: What is Dataflow?

  • History of Max and Pd
  • bang!
  • objects and connections
  • always running
  • time and [metro]
  • numeric and symbolic "atoms, pointers
  • "symbol", "float", "list"
  • execution order and [trigger]
  • hot and cold inlets
  • Example Project: song in a patch: christchurch.pd
  • In-class Demo: simple sequencer with metro, a counter, and select (here are some WAV files to play with: continuous soft and relaxing.zip)
  • Reading:
    • FLOSS Manuals Pure Data: Dataflow Intro, Sequencer, Messages, Math, Order Of Operations
    • Miller's Pd Documentation, chapter 2 "Theory of Operation", sections 2.2 "how to edit patches", 2.3 "messages"
    • Optional: Music Lectures by Dr.Hdez: Pure Data Lessions (video tutorials)
  • Assignment:
    • make a sequencer of sounds and samples using metro, a counter, trigger, select, readsf~, osc~, noise~, phasor~, etc.

(2/11) Week 3: processing generic messages

  • [route], [select], [list], [moses]
  • messages and selectors
  • lists vs. other messages
  • translating "if", "for", etc to dataflow
  • Example Project: Solenoid Concert, an example of sequencing
  • In-class Demo: sequence sound and video (example media: Pd-SMU.zip)
  • Reading:
    • FLOSS Manuals Pure Data: Lists, Cordless Connections
    • Miller's Pd Documentation, chapter 2 "Theory of Operation", section 2.6 "semantics" (2.6.1 through 2.6.5)
    • In Pd-extended, go to the Help menu, then choose Browser, then navigate to Manuals -> MessageOddness and browse thru these odd cases of messages.
  • Assignment:
    • use the mouse data from [cursor], or other source of sensor data like [hid] or [arduino] to make a simple instrument using sounds/samples, [line~], [trigger], readsf~, osc~, noise~, phasor~, etc.

(2/18) Week 4: lines and signals

  • functions to generate lines over time
  • [line], [line~], [vline~] and simple lists
  • control rate versus audio rate
  • special data type with distinct connection type
  • In-class demo: using lines to control sound and video
  • Example Project: frey-wubwubwub.tar.gz
  • Example Project: Terre Natale EXITS Part 2 (scoring to video)
  • Reading:
    • Programming Electronic Music in Pd: Line
    • FLOSS Manuals Pure Data: Envelope Generator
    • In Pd-extended, go to the Help menu, then choose Browser, then navigate to Pure Data --> 3.audio.examples, and read through A03.line.pd and A04.line2.pd
  • Assignment:
    • use a line object to control sound and/or video playback

(2/25) Week 5: storing data

  • storing numeric data in arrays
  • visual representation of that data
  • read, write, modify methods
  • arrays as audio, controlled by line~
  • In-class Demo: arrays and tables for storage and control
  • Example Project: procedural audio in EA's Spore
  • Example Project: The first patcher language, in User Interface I (Alan Kay), 23:30-26:30 (excerpt on local mirror)
  • Reading:
    • FLOSS Manuals: Pure Data, Arrays, Graphs, Tables
    • Miller's Pd Documentation, chapter 2 "Theory of Operation", section 2.8 "numeric arrays"
    • Optional: chapter 2 "Theory of Operation", section 2.9 "data structures"
  • Assignment:
    • Pick a final project and start working on it. Bring ideas, problems, and questions for the next class.

(3/4) Week 6: encapsulation

The key to building large projects is encapsulating ideas into reusable chunks. The difference between a code block and a function: subpatches are code blocks and abstractions are functions, kind of. The real power of abstractions is not, that they are in their own files so they can be used for code reuse. The real power is, that they can accept arguments.

  • subpatches
  • writing objectclasses (aka "abstractions")
  • using arguments, inlets, outlets
  • installing externals
  • In-class Demo: turning example patches into objects
  • Example Project: reacTable: improvisation demo
  • Example Project: Dinosaurs, by Damian Stewart, download: dinosaurs.zip
  • Reading:
    • FLOSS Manuals: Pure Data, Subpatches, Abstractions, Dollar Signs
    • optional: Miller's Pd Documentation, chapter 2 "Theory of Operation" 2.7 "subpatches"
    • optional: FLOSS Manuals: Pure Data, Graph-On-Parent (GOP)
  • Assignment:
    • Finish final project for presentation in the next class

(3/11) Week 7: Final Project Presentations

  • Show and discuss your project in class, we can spend about 8 minutes per presentation.
    • 8 minutes per project
    • one sentence about your idea
    • show your project
    • talk about one thing that you felt was super successful in your project
    • talk about one thing that was frustrating
  Edit | View | History | Print | Recent Changes | Search Page last modified on April 26, 2012, at 12:31 PM