The proof is in the pepper! This image, captured on the new Foveon 3.5Mp X3 chip, shows the color quality we can expect with 36-bit color. As Dr. Carl Sagan might have said, "millions and millions of colors." Photos © 2002, Tim Whitehouse/Photo Courtesy Foveon, All Rights Reserved

The introduction of a technological breakthrough in digital imaging that has global significance is one of the things photographers live for. But, for a product to be that significant, it must be one of the basic building blocks that all camera designers use. One such basic building block of a digital camera is the chip on which the image is captured. The Foveon (www.foveon.com) X3 CMOS imaging chip is both a technological breakthrough and a basic building block. Is it important enough that photographers might end up seeing most digital camera advertisements saying something like "X3 Inside"?

Backtracking To Understand The Future
To understand the how, why, and, even, "who cares" about the workings of the Foveon X3 chip means backtracking a bit to clearly understand how a traditional CCD (or CMOS) chip sees, interprets, and records color. For the sake of this discussion let's call the current crop of imaging chips "mosaic" capture devices and imagine that their surface is a three-colored checkerboard pattern of tiny green, red, and blue squares called pixels. Running horizontally, the top row of pixels are green squares alternating with red squares and the second row, just beneath it, is made up of green squares alternating with blue squares. The pattern repeats itself over the entire face of the chip…green and red, green and blue, green and red, green and blue…etc. Each of the little colored squares gets its color sensitivity from a little (again tiny) filter of that color resting on top of the pixel.

This illustration shows the difference between a current mosaic chip and the new Foveon X3 CMOS chip. In both, notice the pixel in the lower left corner. In the mosaic example the pixel is just reacting to the green component of light while with the multilevel technology of the X3 all three colors are recorded.

Forgetting the difficulty of manufacturing and correctly placing these gazillions (a technical term!) of little filters for a moment, let's look at what happens when a ray of light strikes one filtered pixel. Because of the little filters, the green filter on top of the pixel only allows green light into that pixel, a red filtered pixel only lets red light into it, and finally the blue filtered pixel only receives blue light. Because of this, each pixel is only reading 1/3 of white light that is falling upon it that matches its filter; either the red, green, or blue component. Furthermore, considering the layout of the red, green, and blue pixels, the final image is comprised of 25 percent red pixels, 50 percent green pixels, and 25 percent blue pixels.

Let us assume that we are concentrating on a red filtered pixel for a moment. Imagine it is being struck by red light and is therefore reacting by generating a tiny amount of electric current. It is also surrounded by eight other pixels, four green and four blue. When a digital camera processes this information it looks at all the neighboring pixels surrounding our red example pixel and depending on whether the surrounding pixels are being activated by the light it can decide on the color for all nine pixels together as a group.

If, for example, in addition to the red pixel being active all of the blue pixels are active but none of the green ones are, the computer assumes that the red and blue pixels taken together without any of the green pixels being active means the area being examined is purple (red and blue together) and it registers the area as purple when it maps out the image. If, on the other hand, all eight pixels (four green, four blue) surrounding our red pixel are all reacting to the light the computer will decide that the total area in consideration is white.

This illustration explains how film uses multiple layers to record different colors of light. When you compare it to the X3 technology shown (far right) you'll see that the X3 chip works in approximately the same manner.

In effect, the computer is making an educated guess at color by looking at the color of one pixel and comparing it to the surrounding ones. But, importantly, it is basing that guess on only 1/3 of the light falling on any given pixel because of the colored filters in front of each one. All of this computational effort is called color interpolation (as opposed to resolution interpolation) and a report from a major color print manufacturer's research department estimated that there are 100 computations for each pixel as the processor zeros in on a pixel's color value. While 100 computations per pixel seems insignificant with today's speedy chips, a 3Mp camera must make approximately 300,000,000 (yes, that's three hundred million!) computations to interpolate the final colors in a scene from the raw data the imaging chip captures.

Even considering the Herculean computational effort required, everything works pretty well until you get to the border between one color and another, or a large expanse of primarily (but not only) one color, or, lastly, a finely detailed, multicolored pattern. In these situations the monkey with the wrench rears its ugly little head and you end up with color fringing or a moiré pattern--two of digital imaging's Achilles' heels. Furthermore, in an effort to make the guessing of each color easier, the image collecting chip in most of today's digital cameras have a layer of diffusion material made from glass over them to soften the edges between colors and make the guessing job easier. Some cameras add this blurring effect in software but the net result is better color, less artifacts (false color pixels), and a softening of the image. In effect, this means that the 6Mp chip (as an example only) you paid for is giving less sharpness than it can optimally provide. To solve these problems what was needed was a way to capture all the light…a way to capture all three colors at every pixel location.

Notice the magnified image of a single pixel in this illustration. The X3 absorbs different colors of light at different depths.

Capturing All Three Colors
In 1997, Foveon, a Santa Clara, California based firm, was founded and its goal was to become a world-class player in the digital imaging field. Carver Mead, chairman of Foveon, and a heavy hitter in the Silicon Valley, lined up backers with deep pockets (National Semiconductor and Synaptics), put together a staff and went to work. Foveon's first system provided a means to capture all three colors (with little or no guessing) using three 2Kx2K chips positioned behind a beam splitting prism which broke incoming light into the three colors and sent each component to a separate chip. Because each color had its own channel the result was an amazingly sharp image that is free from color artifacts.

This effort also resulted in a camera that, while producing extremely high quality digital images, was ungainly to use in the real world. But, the quality made possible by recording each color individually was solidly demonstrated.

Getting Back To The Future
Foveon realized that silicon (used as both the supporting surface and slab of an imaging chip) absorbs each color of light (red, green, blue) at a different level within depth of the silicon slab. It seems blue light is absorbed at .2µm (2/10 of one millionth of a meter), green light is absorbed at .8µm (8/10 of one millionth of a meter), and, finally, red light is absorbed at 3.2µm (3.2 millionths of a meter).

The current crop of mosaic chips guess at two of the three colors for each pixel location while the new X3 chip captures all three colors at every pixel location.

The Quantum Leap
Someone at Foveon had the brilliant thought of putting the sensors, which until now rested on the surface of the silicon slab, at different levels within the silicon to take advantage of silicon's light absorption qualities. Next, the mosaic set of red, green, and blue filters were removed from the surface of the silicon and each pixel could both see and record all three colors. Foveon has designed the first chip that collected data from detectors located at different levels within the silicon slab as opposed to just resting them on the silicon's surface. The ramifications of this breakthrough in digital imaging terms are enormous.

First off, it will represent a huge jump from the price vs. quality standpoint. CMOS chips, which the X3 is based on, are roughly 50-60 percent the cost of a similar sized CCD chip. But wait…there's more than meets the eye here. Because each pixel now captures all colors (instead of one out of three) the performance of an X3 chip is about two times that of a mosaic chip. So, in real world terms, you can be safe in expecting 6Mp performance from a 3Mp X3 chip.

From another, practical point of view, a 3Mp chip will approximately double your storage capacity when compared to storing 6Mp captures. This brings up an interesting point when looking at the space needed for capture vs. storage as a TIFF file when discussing the X3 chip.

This is the sensor you might find in the next generation of digital cameras. It is an important enough breakthrough that the JCIA had to come up with new nomenclature to describe it! Meet the 3.5Mp X3 Foveon chip.

With mosaic chips a 6Mp file requires 18MB of storage space because each pixel of information captured must be stored as either a red, green, or blue color. If a pixel is blue it requires a "yes" in the blue 6MB storage file and a "no" in both the red and green storage files. The Foveon X3 chip captures each color separately so, for example only, a 3.5Mp chip captures a 10.5MB file because each color is captured individually. Furthermore, because the guessing of color on the computer's part has been eliminated, the storage space required for a TIFF file is the same sized 10.5MB file.

This aspect of the capture is so unique that the JCIA (Japan Camera Industry Association) came up with a new nomenclature to describe the Foveon X3. According to this new naming system the 3.5Mp, three color chip is called a "3.5Mp X3" chip with the "X3" denoting that all three colors are captured independently and the file size is three times what we've come to classify the chip's resolution by.

Just Scratching The Surface
The first X3 chip captures a 10.5MB raw file and is approximately the size of an APS frame of film. Foveon plans on producing other sized chips for various applications (amateur to pro level) and they will have a web site set up for the camera designers. From this site a manufacturer can download the specs and requirements for a specific chip so they can design a camera around it. At this reporting, there is already one camera ready for production (from Sigma) using the Foveon X3. Reportedly, a few other companies are already fast at work incorporating the new chip technology into their camera designs.

But that is then and as of now, using the 3.5Mp APS sized chip, Foveon feels that they can match current 6Mp image quality with this sized chip. But, there are a lot of other interesting possibilities attached to their multiple level sensor technology. One of these is a readout mode called "Variable Pixel Size" in which Foveon can link four or more pixels together into one giant (relatively speaking), super pixel. While this will reduce the resolution of the chip (in the four pixel linked example a 3Mp chip becomes a .75Mp chip) the big advantage will be in less light required for exposure. A four pixel super pixel needs two f/stops less light to record an image when compared to four individual pixels. In certain situations, such as digital video, low-light capability can be as important as resolution and the Foveon technology will let photographers choose between qualities that they deem most important.

According to Foveon there is no practical limit as to how large a chip can be produced and, if you add that to the user selectable super pixel's low-light capability, there is the possibility of medium format chips that will have the ability to work in available darkness. Because of the already strong interest in this new technology, and if it lives up to Foveon's claims, digital imaging has taken a quantum step forward. It just might be that your new digital camera is going to say "X3" inside.