I chose to write a report on a video camera that is ideal for Video Tracking applications.
This report is on the Imaging Source color firewire video camera DKF 21BF04, which uses the Sony ICX098BQ CCD to create its digital image.
The camera was bought online from The Imaging Source. The regular cost of the camera is $420. The staff are very happy to answer questions and advise on the best camera and lens for your application and installation. The camera body is bought separately from the lens. There is a wide range of lenses to choose from.
The DKF 21BF04 is an industrial imaging camera. It is small and compact, and uses 3 color CCDs to output a UYVY video format at 30fps (or BY8 video format at 60fps) at a 640 x 480 resolution, over firewire. It is used for applications in the areas of factory automation, quality inspection and medical systems. It is good for image comparison and shape recognition. It is also very well suited for video tracking applications. Scott Fitzgerald is currently using one of these camears for a motion-tracking application in a corporate lobby installation. In my work with Light Projects, I am planning to use these cameras for a motion-tracking application for a light and video installation for the Liberty Science Center, New Jersey. I will report back.
THE SONY ICX098BQ CCD
CCD stands for Charge Coupled Device. It is the type of sensor that is often used in digital imaging devices, to convert light (photons) into electrical charge (electrons) using a 2D pixel array of photodiodes, and then translate that data into visual images. The first operation a CCD has to complete is charge generation: the ability of the sensor to intercept incoming photons and generate signal electrons through the photoelectric effect. This process is described by a performance parameter called Quantum Efficiency (QE). To achieve high response, sensors need to minimize three types of loss - absorption, reflection and transmission. Reflection and transmission losses are inherent to the physical properties of silicon. Transmission loss takes place when incoming photons pass through the sensor's photosensitive volume (a region typically 10 um thick) without generating signal charge. This problem is pronounced at very long and very short wavelengths - ie, the near-infrared (above 700nm) and soft x-ray (below 0.2nm) spectral regions.
The Sony ICX098BQ is an Interline CCD (as opposed to a Frame Transfer CCD, or a Full Frame CCD). which means it is subdivided into light sensitive and storage areas, placed as arrays. It is the light sensistive area that captures the information and transforms it into electrical charges. The generated charges are transferred to the darkened shift register cell (storage area). From there the charges of the vertical shift register are transferred, row by row into the horizontal shift register, and from there serially read out. The Interline CCD was created for high-speed applications. The structure of the CCD, with it's alternating elongated arrays of photosensitive diodes and storage, means that the transfer speed of the charge is very fast. The QE of the pixel diode are is also good, however the diodes only cover a limited area of the CCD, meaning that Interline CCDs can be sensitive to a smaller range of the spectral region than Full Frame CCDs .
Here's a link to a useful page about different types of CCD.
Here's a link to an interesting article comparing CCD technology with CMOS technology
The Sony ICX098BQ uses R, G & B primary color mosaic filters, based on the Bayer system. In the process of converting photons into electrons, the photon's wavelength is lost, which means that CCDs are "color blind". In 1976 Bryce Bayer obtained a patent for the solution of this problem, and equipped every second pixel with a green filter and distributed blue and red filters evenly to the remaining pixels.
Here is the spectral range characteristics (taken from the manufacturer's datasheet)
The Sony ICX098BQ also has better results in low light conditions, because it uses Sony's HAD (Hole-Accumulation Diode) design. Again, see this useful page about different types of CCD - scroll down to the bottom of the page for explanation of HAD.
This CCD outputs square pixels, at a resolution of 640 x 480. DV video is almost always rectangular pixels. Computer generated imagery is usually square pixels. This is referred to as a difference in Pixel Aspect Ratio. I think it's better to do video processing with square pixels, because geometry based filters and effects and processes will work without distorition. Open GL always draws in square pixels. The difference exists because engineers wanted to make higher resolution television images without changing the number of scan lines.
The CCD has a low Smear - Smear is an undesired signal which appears as a brighter vertical stripe emanating from a bright part of the image. In an interline transfer CCD (like this one), it is produced by scattered photons tunnelling into the darkened vertical shift register, rather than being collected in the photodiodes of the image sensing area.
The CCD outputs progressive scan, which gives a better image quality than interlaced video.
USING THE SENSOR
I used the DKF 21BF04 in a number of different motion tracking Jitter patches that I had already made, and also into the cv-jit example patches. Using the DKF 21BF04 resulted in a much higher frame rate than using other IIDC cameras (like the iSight). The image was fast and crisp. I was able to grab live frames at a resolution of 640 x 480 (@ vmode 0) and convert the image to monochrome, and still get full frame rate (jitter was reading it as approx 100fps, which must include duplicate frames). In the cv-jit examples, I was able to grab a 640x480 image, convert to b&w, label and identify blob tracking and still return a Jitter fps averaging approx 90fps. This was on average approximately 20fps faster than a Panasonic DV camera. This frame rate is similar to the built in iSight in my Macbook Pro, however, the image from the DKF 21BF04 is crisper, and shows more detail.
If you only want to do motion tracking with the camera, then it might be worth buying a monochrome version, however, if you think that you might want to output imagery from the camera as well as using it for tracking, it's probably worth going for the DKF 21BF04.
The reasons I liked to use it is that it outputs IIDC protocol over firewire, which means that I don't need to buy an ADC, and I can use an array of cameras, and Jitter will be able to grab them in a correct, repeating order at startup. This camera is also bus powered (over firewire), which means that extra cabling for power does not have to run to the camera. Also it only has manual focus and manual iris, which means that you don't have to disable autofocus and autoexposure functions when tracking motion through a space. You can get software to remote control the other visual aspects of the image, but I think much of this can be done remotely in Jitter anyway.