The Cat and the Camera: Foveon's revolution is just a click away
Gilder, George
"The Cat and the Camera" BY GEORGE GILDER As I walked out with Dick Merrill on a cool February day through the Foveon Corporation parking lot in Sunnyvale to drive to the Lion & Compass restaurant, my hands were...
...Because the algorithms function best by incorporating information from a range ofnearby pixels, the guessing game for each can require hundreds of arithmetic operations...
...Ask Merrill who the inspiration for his own work is and he does not say Mead, his boss—though he shares in the general awe of the man...
...not conforming to standards of thought and speech and action derived by the conforrnants from the study of themselves...
...Maybe Foveon should have merely exploited the niche market for professional studio cameras that cost $70,000 apiece...
...Amid a growing flock of empty glasses, he quoted the apothems of Ambrose Bierce (the results are a chap-ter in Microcosm) until he fell asleep...
...Well, he cites Bierce: "Crazy—affected with a high degree of intellectual independence...
...In the future, he believed, such devices could be the basis 62 THE AMERICAN SPECTATOR MARCH/APRIL 2002 of improved forms of machine vision, or of a superior solid-state camera...
...On the fertile edge of the network, new semiconductor and optical innovations will bring radical new technologies to market, weaving the microcosmand telecosm together in a seamless web...
...But the human eyes and mind see and sense byanalogies, embodied in the neurochemistry and electrical patterns in the brain...
...The stocks of companies with capital outlays geared low enough to avoid debt—or sufficiently risky enough not to have attracted it—have remained afloat despite the crash...
...By 1997, most of the ideas that might make Mead's vision possible resided on Dick Lyon's desk...
...Instead of guessing, the signal processor would then combine the actual red, green and blue values for each pixel to produce a final image...
...But through the early 1990s, it made only fitful progress in fulfilling Mead's vision of neuromorphic devices and large-scale analog neural networks...
...The photoreceptor for each pixel—roughly speaking the smallest component point in the image: think dots per inch in your ink jet printer—is covered by a filter, which lets in only red, blue or green light (the three col-ors captured by the rods and cones of the human eye...
...Foveon has the single most powerful new commercial technology I have encountered since first meeting Mead at the Newark Marriott, generations ahead of existing digital cameras and video recorders...
...Whether it was God or Gordon Moore, whom Mead had been consulting up there in the Valley, Mead's citation of the brain was not unusual in computer science.What was radical was that rather than treating the digital computer as a possible model for an extended or ultimately superior brain, he was offering the analog brain as a model of an incomparably more powerful computer...
...Cameras, slide rules, radios, television sets, telephones, ovens, airplanes and automobiles, all operated with nary a digital circuit—except possibly a simple clock—until the mid-1980s...
...Then a string of ingenious mixed-signal inventions by Mead student Tim Allen broke through, in the realm of touch, where Mead himself had done little work...
...Traditional digital imaging is based on the charge-coupled device—a kind of silicon bucket brigade resembling a single stretched transistor with thousands of information-bearing "gates" CCDs convert incoming photons into electronic intensities, then pass them on to digital processors that convert them back into an image...
...MIT's gurus at the time were advocating optical computers that used photonic binary transistors, directly translated from the digital silicon model.They laughed out loud at my assertion, borrowed from Mead, that the absence of digital optics in the brain—to this day, the world's only fully successful image processor cast serious doubt on the prospects for digital image processing...
...Synaptics struggled for seven years before bringing a significant product to market...
...Consider the electron...
...Thus, as it turns out, there would be no quick, giddy ascent for Synaptics, or for Mead's new analog vision...
...Up at his house, I asked what the document was on his computer screen and he told me he's writing a book on gravity"—the final frontier of physics...
...After several turns and a shameful capitulation toward Route 101, Merrill announced that we had missed the key turnoff and overshot the mark...
...64 THE AMERICAN SPECTATOR • MARCH/APRIL 2002 march of digital seems inexorable, as every telephone, cell phone and network succumbs to the advantages of neat clean bits and bytes for bringing order to the noisy and nosy analog world we inhabit...
...Together, they create full-color pixels...
...Sky, Merrill points out, offers the best visible index of the noise level in an imager...
...Widlar was the source of a key principle of the Foveon process, and one which had been ingrained into National's culture: exploit the potential of the silicon material, wherever it led...
...Its explicit mission was to do what Synaptics had done, but on the far bigger playing field of imaging and artificial vision...
...Analog devices carry out the same operation in one step and so decrease the power consumption of silicon circuits by a factor of about 10,000 " Next the Caltech pair pointed to a little-recognized virtue of analog devices, which originated with Widlar's op-amp and would prove pivotal in future imagers: "They respond to differences in signal amplitude rather than to absolute signal levels, thus largely eliminating the need for precise calibration...
...CMOS designers face a fundamental problem: Between each cell's two transistors—the complementary negative and positive devices—is a potential bipolar transistor, called the "latch-up" or "parasitic" device...
...and pursuing lengthy roundabout paths and kludges to complex camera systems with three chips and exquisite precision optics and color supplied by epoxied prisms, the team frequently debatMARCH/APRIL 2002 • THE AMERICAN SPECTATOR 61 ed whether they really wanted to go to the destination anyway...
...There were other advantages, too...
...Steering against that tide of ones and zeros, Mead has long seemed a foolhardy surfer, sure to land hard on his face on digital silicon's ever-spreading beach...
...What matters in a camera, after all, is perception, not the scientific accuracy of frequency calculation...
...Pulling out a Sprint PCS Motorola, my colleague Richard Vigilante began nervously dialing the restaurant...
...Today, so it appears, most remaining analog systems are giving way to the digital tide...
...It took me six years of a Caltech education to under-stand the Schrodinger equation, on the wave behavior of electrons...
...Call back later when you get closer," she suggested.Actually, we had been driving in circles within a mile of the restaurant the entire time, and suddenly we found ourselves crossing the fateful freeway and taking another up fart before the Lion & Compass parking lot suddenly appeared, obscured by what Merrill had described as an "unreadable sign," presumably blurred by digital noise and aliasing...
...Few were awed by his claim that he could scale his device to densities a hundredfold greater.A quarter of a million monochrome pixels scarcely endangered Kodak or Sony...
...Creators of devices without the assurance of simple numerical inputs and outputs—none of those tidy ones and zeros—analog people are loners, full of black arts and trade secrets, volatile resentments and spiky prejudices...
...It took months to convince him to join Mead's foray into silicon's wild frontiers...
...The depth of the challenge becomes more apparent considering that before Foveon, no true silicon-based color imager existed...
...But in 1991, after some twenty iterations, Mead's "artificial retina" made the cover of Scienti icAmerican, with the blurrily captured image of a cat's face...
...Despite his prophetic early experiments and inventions with tunnel diodes and high frequency transistors, Mead's ideas for analogVLSI incurred solid resistance even from leading analog companies, including Analog Devices, Texas Instruments, Linear Technology and Maxim...
...Within a few months of the Caltech lecture, Mead launched Synaptics, with the brazen ambition of applying his break-through to all the human senses, from hearing and imaging to touch...
...For Foveon's first-generation cameras, that meant tossing out the usual red-bluegreen filters and substituting a prism—splitting an image's red, blue and green light, and directing each stream of photons to its own single-chip imager...
...Captured in only 2,500 pixels, Mead's cat hardly seemed a threat to the Moore's law digital juggernaut, which already propelled a thriving industry of machine vision for manufacturing applications...
...Chiefly engaging me, beyond looking for street signs, was Merrill's lecture...
...But Merrill still thought we would find the Lion & Compass any moment now, somewhere just across 101.We continued to thrash around for a total of some 35 minutes and several upfarts (which I am told is Finnish for U-turn), as the steering wheel slithered through my hands like a wet snake...
...Bob Widlar runs a close second...
...But its innovation in pattern matching is analog-based...
...Merrill is a man of medium height, a stubble of beard, blue eyes and a shy, abashed manner, as if he really wished he could return to work and escape these distractingly personal or intimate questions about semiconductor P/N—positive/negative—junctions and charge-coupled devices...
...Was Widlar crazy...
...The first results were far from usable, but Mead was encouraged enough to tell the others, "I think we have to bet the company on this...
...Single chips have a singular virtue:They can eventually be manufactured in volume for less than 80 cents a piece—about the price of the packaging...
...This jumble of apparently unrelated ventures embodies the singular new vision unleashed by Mead himself two decades ago in his classes at Caltech and brought to diverse fruition by an amazingly ingenious cohort of his students and associates...
...Used in radios, disk drives, keyboards, microphones and oscilloscopes, Widlar's op-amp chip launched an analog movement that provided much of theearly cash flow for the semiconductor industry, from Fairchild and National to AMD...
...But the design he was showing on the screen to such friendly hilarity was not a digital device at all...
...Merrill simply plugged his new microchip into a circuit board, installed the board in a nineteenth-century camera chassis, snapped the picture and transfigured an industry...
...Arbitrary color patterns trigger rainbows, checks and whorls where nature intended a blue shirt or a plaid jumper...
...By that time, RichardV...
...The story, by Mead and his student, the late Mischa Mahowald, was confident: "The behavior of the artificial retina demonstrates the remarkable power of the analog computing paradigm embodied in neural circuits...
...Later in the class, however, Mead presented the first actual example of such a machine, a silicon retina chip, modeled on the human eye, that could follow a rotating fan without aliasing (seeming to reverse direction, as spinning wheels do on movie film...
...Don't they believe in Mead, the industry's first and most profound prophet ofVery Large Scale Integration VLSI microchips...
...The less energetic green could sink one-and-one-half microns before it agitated the silicon enough to be absorbed...
...That will be fine...
...Chiefly engaging him, though, was a running denunciation of the new Nikon D1 digital camera, which he had used on a recent trip down the Mekong River with his Laotian wife...
...In particular, he has long maintained that what he termed "neuromorphic analog VLSI" offers the possibility of a radically more effective image processor...
...If it could be made to work—and manufactured—the single-chip color image plane had the potential to repeat the magic of the digital microprocessor...
...In Merrill's totem pole picture, the sky is impeccably blue...
...Together, each group of three photoreceptors does indeed capture a full-color image—but in the process, throws away two-thirds of the light and information that streams in...
...At the time Mead presented his brain model, analog devices were orders of magnitude larger thanVLSI implied...
...Because [in Claude Shannon's theory of information] only changes and differences convey information, constant change is a necessity for neural systems—rather than a source of difficulty, as it is for digital systems...
...Most of it, Lyon recounts, could be discarded...
...With my Telecosm list---my buy-and-fold portfolio from the Gilder Technology Report—and the bold propositions of my book all now in a seeming shambles of the monetary deflation and telecom crash, I am inclined to be churlish...
...MAY 1991 $3.95 MARCH/APRIL 2002 • THE AMERICAN SPECTATOR 65 based color imager that would yield the best, cheapest and easiest-to-use mass market cameras ever made...
...minimizing the leakage of electrons across that atom-sized gap is a classic CMOS problem...
...A pivotal point came in 1986, when theValley was roaring back from its last great cataclysmic slump, with revenues dropping some 45 percent in a year...
...In other words, Foveon's X3, as the marketers have dubbed it, will make possible throwaway cameras with a resolution and accuracy better than today's most costly Hasselblad...
...Flying under the radar, his team would be free to pursue its real goal—a single-chip, siliconSCIENTIFIC AMERICAN Exploring the genetic heritage of racehorses...
...Mead eventually broke with Faggin and relinquished his MARCH/APRIL 2002 • THE AMERICAN SPECTATOR 63 role as chairman, though retaining his shares...
...analog means continuous representations by currents and voltages, rather than by digital math...
...Indeed, although the GIF and JPEG standards are too crude to exploit the superior verisimilitude of a Foveon image, they seem sufficient to most users...
...Foveon and an older Mead-launched corporate sibling, Synaptics, had indeed traversed an ambageous route...
...With Foveon, Mead was determined to avoid throwing away information...
...Merrill installed it into an old camera and snapped his totem pole picture...
...Images flash across the Net as digital GIFs (Graphics Interchange Format) or JPEGs (Joint Photographic Experts Group...
...BY GEORGE GILDER As I walked out with Dick Merrill on a cool February day through the Foveon Corporation parking lot in Sunnyvale to drive to the Lion & Compass restaurant, my hands were sweating...
...As his long-time colleague and collaborator Dick Lyon puts it, they wanted "no guessing at all...
...By the early 1990s, camera-company laboratories were already experimenting with digital imagers offering resolutions far higher than Mead's...
...But following Mead's call to "listen to the technology," I still find myself rising in joy at its reveille, which is sounding with new urgency in the new year...
...If out-side buyers and suppliers lose faith, the company can come a cropper overnight, regard-less of how valuable its goods and services ultimately may prove...
...Numbers appeared on panels and consoles, but no bits or bytes coursed through the circuitry inside, where values transpired as levels of charge, current, volt-age, frequency and amplitude...
...Although radios remain dominantly analog, new digital radio systems are emerging, including Sirius and XM Satellite Radio...
...had managed to raise the Lion & Compass, but the woman who answered the phone knew none of the streets or company names Richard cited, except 101...
...Video increasingly spurns tape, preferring digital video or versatile disks—DVDs...
...Digital's progress has become an axiom, as if the reduction of all information to numbers was the essence of progress itself...
...Lyon recommended tweaking the technology so that the pattern of colors in the silicon correlated closely with the pattern in the eye...
...As a result, mosaic sensors capture only 25 percent of the red and blue light, and just 5o percent of the green...
...On the wall outside Mead's corner office at Foveon is a dramatic symbol of how far the new technology has come: a three-foot high image of the face of a cat...
...X3s are rolling off the line at its leading-edge fab in Portland, Maine...
...But the electron already knows the Schrodinger equation...
...Following the lead of Mead's bipolar photodetector, Merrill pro-posed burying P/N junctions at different depths on a chip, to separate and collect dif-ferent colors at each pixel...
...As I researched Microcosm, my 1989 book on the semiconductor industry, Widlar—whom my simple wish was to make a hero—led me on a wild geek chase from Lawrence Station in Sunnyvale to the Hilton in PuertoVallarta, Mexico, without yielding a single interview...
...Like the railroads that bankrupted a previous generation of visionary entrepreneurs—but built the foundations of an industrial nation—fiber-optic webs, storewidth breakthroughs, data centers and wireless systems installed over the last five years will enable and endow the next generation of entrepreneurial wealth...
...But more crucially, it is the first workingphotograph made with a single-chip, full-color imager...
...Yes, the totem pole picture is flawless, with exquisitely authentic hues and supreme resolution...
...overshooting key turnoffs toward simple solutions...
...As a result, Foveon's image sensors capture red, green and blue light at every pixel location...
...His "101" operational amplifier was the first one-chip device with a stabilizing feedback loop, enabling its reliable use in any situation that required accurate boosting of a frail signal from the real world...
...He had performed the crucial researches from which Moore's law itselfderived, ordaining the doubling of digital computer performance every eighteen months...
...It signals the first hot flare of revival from the devastation of the last year in technology...
...Mead and Merrill's Foveon is no mere "digital camera," full of chips and microprocessors and mirrors and shutters—it is a fully solid-state machine, based around a single chip and virtually no mechanical paraphernalia, capable (like the human retina) of both still and moving photography...
...As the highest frequency and highest energy color, blue would be captured near the surface, only a half micron down...
...The success of this venture will give rise to an entirely new view of information processing that harnesses the power of analog collective systems to solve problems that are intractable by conventional digital methods...
...Merrill is an analog-chip designer, and analog people are different from digital people, who author microprocessors and software suites and work in teams and earnestly entertain journalists...
...But right away at the restaurant, he clarified Foveon's route to launching a revolution in image catchers...
...The result: pictures of extraordinary quality that mocked their best digital competitors, and even rivaled the 8"x 10" large-format studio cameras that are the pinnacle for chemical-based photographic film...
...And the trip to the Lion & George Gilder is editor of the Gilder Technology Report and the author of Telecosm and other books...
...Merrill's solution was to keep both—the electrons and the holes—and balance them off, registering only the difference...
...Patents are a way to do something with an idea, without too much work...
...Four years before, industry eminence Carver Mead had brought Merrill here, hoping to enlist this silicon sage in his latest new company, Foveon...
...And by contrast, it exposes who merely slips along on a secondary slurry of hearsay and hype, schussing through the media access channel toward the presentation layer, making PowerPoints and strutting the runways of industry fashion shows...
...A high-resolution digital imager must take billions of steps, consuming scores of watts of power and heat, to calculate a single picture...
...Joining him was Federico Faggin, the builder of Intel's first microprocessors and inventor of the self-aligned silicon gate that made them possible...
...It was a significant first step toward creating a real-time imager on monolithic silicon...
...Mead had been urging me for years to meet with Merrill, whom he compares with the industry's most famously feral analog chip prodigy, the late legendary Robert Widlar...
...But instead of trying to neutralize it, Mead—in his own famous phrase—"listened to the technology" to what the silicon "wanted" to do...
...After twenty years as the industry's most authoritative proponent of the power of digital electronics, he was reversing direction and declaring the onset of a new era, the analog age...
...It would be better, cooler, cheaper and lower power than its rivals...
...Spotting Merrill's patent, he was intrigued by the fortuitous color-filtering capability of silicon...
...On the wall as you enter the Foveon building is a more significant photograph, one whose story portends the company's likely success in its imperial quest.Taken by Merrill, it depicts a vividly colorful totem pole in Vancouver, Washington, shot against a perfect blue sky...
...Why is this class laughing...
...The image resolves every hair, whisker, glint and gleam of the feline fur and renders the eyes of the cat with a lifelike glow that gives the viewer the distinct and disturbing feeling that a formidaIn conventional digital cameras, color filters are applied to a single layer of photodetectors, in a tiled mosaic pattern...
...It was no joke, but Merrill continued his discourse, politely ignoring my threat to his life.Turning onto a back street that featured a company sign with three X's in it whether adult or pre-IPO, I could not descry —I suggested nervously that perhaps we should return to Foveon and eat in the cafeteria...
...So superior were Synaptic's touchpads—used most visibly today as the navigation device for notebook computers—that they quickly took over the industry, and today hold an estimated 80 per-cent of the global touchpad market...
...You dump it on the patent attorneys ") In particular, while at Nation-al, he had tried to create a truly differential analog technology out of CCDs, focusing on differences in energy rather than their mere intensity and thereby minimizing distortions from external changes such as temperature...
...As part of the agreement with National and Synaptics that gave birth to the new startup, Foveon had inherited all the intellectual property on imaging held by both companies...
...I first encountered Carver Mead in a noisy restaurant at the Marriott near Newark Air-port...
...The impact of Kuwait% burning oil wells...
...In telecommunications, similarly, the Captured In only 2,500 pixels, Mead's cat hardly seemed a threat to the Moore's law digital juggernaut...
...If this were a dot-corn story or a Telecosmic tale from the last few years, we might expect an early IPO and swift ascent to success, followed by wild acclaim and a possible crash...
...Mead's revolutionary camera will make your screen brim with the intensity of a new era...
...MARCH/APRIL 2002 • THE AMERICAN SPECTATOR 67...
...But Merrill was embedded deep in the comfortable substrate at National Semiconductor...
...Itnpinj is focused on radical innovations in self-adaptive semiconductors...
...Most television sets now receive digital sound and images, compressed in MPEG2 (Motion Picture Experts Group) codes...
...But at gigahertz speeds, digital signal processors often hurtle blithely past crucial signposts from nature...
...In order for Foveon pictures to take over the world of photography and film, these dense images will require broadband communications and trillions of bytes of storage, melded in a combination that I have dubbed "storewidth...
...I don't eat much lunch anyway," I said...
...The first commercial Foveon camera's $50,000 price tag marked Hasselblad as its chief competitor and professional studio photographers as its only market...
...But all this was theory...
...National Semi-conductor retains a 49 percent share of Foveon...
...The first workable chip emerged from the Nation-al line a year later...
...Merrill was muttering directions and assuring us that we'd be there in no time at all, what with the short cut eliminating the jams and terrors of 101...
...It computes Schrodinger equations day and night...
...its technical leader, Vance Bjorn, is a former Mead student, and another knight inerrant in his campaign to trans-form the interfaces between digital technology and our analog world...
...Left would be only lenses, batteries and silicon—a true solid-state camera...
...Each Foveon picture ultimatelytranslates into a digital file of up to 40 megabytes (millions of characters...
...The ultimate success of Foveon, after all, depends on the continued onrush of digital electronics...
...No one was building chips that simulated brain functions...
...Even an artist of analog evasions could not escape during what Mapquest estimated as only a seven-minute jaunt—a few hundred yards up San Tomas Expressway from Foveon and then one exit down Route 101...
...Following Synaptics and Foveon will be at least two other Mead companies...
...Since silicon absorbs different colors of light at different depths, each layer captures a different color...
...Even as assembled by Japan's best miniaturizing wizards, he railed, the 2002-vintage digital cam-era and its charge-coupled-device photo-sensors were horribly flawed...
...Can anions explain hightemperature upercomlactivity...
...MR...
...The lowest energy photons—red—would penetrate some three microns down...
...He started drawing chip designs for me, on napkins and blank pages of his canonical book and on scraps of paper that piled up on the floor next to the booth...
...To the argument that analog processing would require too much power, he ran his chips at subthreshold voltages, like the micropower systems in digital watches...
...On the Fertile Edge On the surface, the year 2002 growls into view as a scavenger's feast—an epoch when much of the profit and property created during America's most cornucopian economic boom slips away into the maw of the most reactionary and obtuse parties: quasi-governmental bureaucracies such as AT&T and SBC, marginally competent foreign Internet companies such as Cable and Wire-less, para-statal monstrosities such as France Telecom, and class-action racketeers such as Milberg Weiss Bershad Hynes & Lerach, who are taking the lead as plaintiffs in the Enron case...
...Good enough for the Net, the CCD-based digital camera continues its advance, dominating newspaper and magazine photography and making its way steadily into the huge amateur market...
...They don't even that what they're doing is hard...
...Synaptics was supposed to be the answer...
...Applied Neurosciences has a probable breakthrough venture in speech recognition.A kindred company associated with Mead, Sonic Innovations, emerged in 1999 as the world's fastest-growing hearing aid company, no niche market in a graying world...
...An intimate of many of the founders of the digital age, from inventor of the integrated circuit Bob Noyce to microprocessor architect Federico Faggin...
...In the normal course of events in the silicon business, a new chip based on an original design and a novel manufacturing process goes through scores of iterations before it works the way its designers hoped...
...The GIFs and JPEGs of the current World Wide Web are grossly inadequate to capture the vividness and verisimilitude of Foveon pictures.When the announced new standard is perfected, there will be yet another image of a cat—this one will leap off the computer screen toward you...
...Meeting Merrill also summoned from the mazes of my memory the recollection of a frantic rush down Sixth Avenue in New York in 1985, still chasing the elusive Bob Widlar...
...Only with agonizing delays would his analog VLSI vision gain adherents outside the circle of his own students...
...The key breakthrough involved light-receiving photoreceptors...
...Out of the parking lot, I squeezed our ungainly gray Lincoln rental car across a small bridge and through a series of adjacent parking lots, snaking past Intel's Santa Clara plant and companies with names such as Swixx, Svee and Luminant...
...But Merrill knew a short cut...
...When Bill Gates launched his new XP network-centric operating system at a recent trade show, this king of the digital age did not begin by entering a password or clicking icons in a pop-up window Instead, he put his finger firmly on a glowing biometric touchpad that recognized him, loaded his personal settings, and opened access to his personal files and digital kingdom.The company that sup-plied this open sesame is called Digital Persona...
...Down the hall from Merrill's office at Foveon's Santa Clara headquarters is what appears to be a heap of colorful Navajo blankets, with a sheet of glass resting on the top...
...Thus the latch-up transistor became an effective photoreceptor—so effective that it outperformed ordinary photodiodes, even as it was integrated onto the chip...
...New portents suggest that the Mead revolution is going mainstream...
...Foveon's saga is really the life story of Carver Mead, which I first told in my book on the semiconductor industry, Microcosm...
...You only know you have a real camera when you want to take it mountain climbing," explained the wiry native Vermonter...
...Even today analog has an archaic sound, calling to mind the very first computers, used for aiming anti-aircraft artillery fire during World War II.They transmitted radar signals, received their echoes, then calculated the telltale delays and lead distances for targeting, all without ever converting any of the measurements to digital ones and zeros...
...Failure loomed...
...The cost is even higher when you move to the next stage—reassembling that information into a viewable image...
...It doesn't think it's hard...
...God or Gordon Moore...
...Mead's analog technologies will change the world, but the world will have to change, too, to accommodate these new capabilities...
...The company cannot control or even reliably predict the market itself...
...Every pixel would register real features of the image—and every color—rather than rely on digital simulations...
...The Lion & Compass is no run of the mall Silicon Valley flesh pit...
...I was pursuing the leonine drunk for a random quote on the street, when he abruptly turned back into the Hilton and headed for the bar...
...Under capitalism, where entrepreneurs depend entirely on the free responses of others, both customers and investors, the possibility of failure is a crucial filter...
...But the Mead-Conway model for integrating multiple functions on a single chip triumphed so widely and rapidly—if often in crude defiance of Mead's larger vision—that he, himself, played chiefly the role of laureate and guide...
...But as always in the digital realm, the preferred way over the rainbow is to do more with Moore: add more pixels and handle the burgeoning computational load with ever faster digital signal processors...
...Neuromorphic means derived from human models rather than from mechanical logic...
...Contemplating the potential, Mead and Lyon reintroduced Merrill to his own patent and suggested that he reduce it to a silicon device...
...Camera and eve would converge if all the colors were collected at every pixel—Mead's mandate from the outset.The result, Lyon predicted, would be an accurate rendition of colors as humans see them...
...Eventually, looking for a fab—silicon fabrication plant—that could manufacture Synaptic's revolutionary chips, Mead encountered Merrill at National Semiconductor...
...It could be the same feline that stared out from the cover of Scientific American a decade earlier...
...its computations are based on smoothly varying ion currents rather than on bits rep-resenting discrete ones and zeros.Yet neural systems work with basic physics rather than trying constantly to work against it" During a long walk over the bare brown hills above Pasadena, Mahowald had explained to me the magic of analog...
...Bipolar photodetectors repeated the original retina design: burying three junc-tions completed it...
...Gone would be not only film, but virtually all the precision machinery that evolved around it, including, ultimately, the shutter itself...
...With Japan's great camera makers all now frantically putting in their bids, Foveon is poised to take a share of the global camera and video components market at least as big as Synaptics' 80 percent share of the world-wide touchpad business...
...Merrill's breakthrough exploited a curious fact about silicon: that the frequency of light—its color, in lay terms—determines the depth on to which its photons will penetrate a chip's surface...
...By that point, we knew enough not to buy a Nikon D1, but Merrill still needed to explain how Foveon, with his help, had arrived at its miraculous alternative...
...Selling them for half that amount would still be cheap tuition for a trip down-the learning curve...
...It was a schematic of the human brain...
...As his model and inspiration as a chip designer, Merrill names Bob Widlar...
...At Widlar's last International Solid States Circuits Conference, in NewYork in 1985, he won the best-paper award for the fifth time...
...And ultimately pixel size is limited not by Moore's law, but by less tractable limits like the wavelengths of visible light—at roughly half a micron, already close to the smallest features on digital circuits—and the resolution of the human eye...
...Like the apparent glass on the top of the pile of carpets, your screen will disappear into a luminous new world of art and color...
...Let the competition scoff that handcrafted prisms would never be the basis for a viable consumer product...
...The filters let only one wavelength of light—red, green or blue—pass through to any given pixel, allowing it to record only one color...
...Ingenious as those software guessing games are, the original decision to toss away so much information permanently impairs picture quality...
...The problem was that existing CCDs captured electrons (negative energy), but threw away what silicon engineers call the "holes" (positive energy...
...In a stroke, Mead showed the way to create analog systems that scaled like digital ones, with costs that fall off cliffs in accord with Moore's law...
...In conventional digital cameras, the final picture is produced by an elaborate digital guessing game, an algorithmic approximation performed by speedy (but expensive) digital signal processors...
...But instead of blurred monochrome, the new image offers a full-color vividness and verisimilitude perhaps never before achieved in photography...
...But Merrill had been a compulsive patenter...
...the closest anyone was coming to neural computational models was basement-IQ artificial intelligence software...
...This was less than reassuring...
...Although a slight overlap of the blue, green and red levels in the silicon persuaded Merrill that the system would be noisy, Lyon observed that the human eye is noisy in almost exactly the same way...
...And instead of neutralizing it, he enlarged and enhanced the bipolar transistor, increasing the P/N junction's sensitivity to light...
...It was analog, not a bit or a byte in sight...
...Some never do...
...Thus, Foveon's full revolution...
...Narrowly missing a possibly fatal collision with digital camera monster Sony...
...VLSI signifies the kind of tiny geometries characteristic of leading-edge digital devices, such as the microprocessors that run your personal computer—tens of millions of transistors on a fingernail-sized chip...
...He added with a sneer at me: "Reporter—a writer who guesses his way to the truth and dispels it with a tempest of words...
...Moving abruptly into the next lane without first signaling or scanning the traffic, I put us in the path of an angry blue Mercedes, then wrenched us back into the other lane with inches to spare...
...li#VtlM...
...But this impeccable image was the first product of the very first chip to emerge, under Merrill's direction, from the first run at the National Semiconductor wafer fabrication plant in Portland, Maine...
...By dispensing with all now-superfluous electromechanical paraphernalia of a convention-al digital camera, the new chip could process images with virtually no delay.And it could do moving pictures—video—too, at unprecedented resolution...
...Lyon, an expert on vision technology formerly at Apple, had developed a retina chip of his own, independent of Mead's...
...And regardless of its academic significance—the clear signs of an advance in under-standing the nature of vision—the blurred cat on Scienti icArnerican's cover was a downer for most observers, belying both the authors' confident assertions inside and the grandiose plans of their company, Synaptics...
...When when he did finally come, he brought with him National Semi's money, manufacturing power and, most importantly, his own string of powerful patents...
...He still does...
...NI Iry !g smrg...
...But the trornpe l'oeil trick is just candy for your eye.This company has its vision set on far grander horizons...
...Compass would be short...
...Recalling that his student Tobi Delbruck, son of the physicist turned Nobel Medicine Laureate, had once proposed a similar idea, Mead was immediately impressed by Lyon's logic...
...Sixth Avenue Breakdown Mead calls Merrill the most creative engineer he has ever met, in the combined disciplines of wafer fabrication, circuit design, device physics and photography...
...Merrill surely would not be so hard...
...They are, if you will, indiscriminate photon counters, which capture color only with a costly tradeoff...
...It burst through the market doldrums this January, with the first major technology IPO since the 2001 crash, the first public vessel of Mead's analog vision...
...Projecting the design of a massively parallel processor on the screen, he proposed it as a model for a revolution in computing and said: "Now I've been up in Silicon Valley, talking to the guy who made this thing and...
...It was the solution Mead and Mahowald had proposed in 1991 in Scientific American...
...A neuron is an analog device...
...But their photoreceptors operate in black and white—more precisely, they measure the intensity of the light striking them, not its wavelength...
...Microsoft itself is developing a new standard for representing photographs on the net...
...Mead, one of Caltech's treasures and godfather to four decades of history-making technologies, was probably the industry's most important intellect...
...The unforgiving fields of silicon electronics quickly reveal who is real, who truly listens to the technology, who feels the physics behind the device and who can match the intrinsic properties of the material with the essential demands of the function...
...In a Caltech class-room in Pasadena, the eminent Gordon E. and Betty Moore Professor of Engineering and Applied Science, like many in his trade, seemed to be flaunting his august connections to the technologic eruption underway up north...
...his experiments with explosives in the National Semiconductor offices and with wild animals on the company lawn inspired many of the conversations down the street at Lawrence Station's bar and grill...
...So, at mega-pixel levels, the cameras consume inordinate time and power—or both...
...Rather than accept the prevalent designs or templates and try to work around them, as digital designers did, Widlar would tweak the material itself to achieve new unexpected functions...
...And maybe in the end they could lease the things for a few thou-sand a month, like a T 1 line, and get as rich as Pac Bell...
...66 THE AMERICAN SPECTATOR MARCH/APRIL 2002 ble animal is watching him.Yet the picture is of a kitten...
...At the outset, it targeted the three key human senses: touch, vision and hearing...
...They inflicted ghastly whirls on the pictures where there should have been whorls, forced delays with their epic battery drain and wasted power (no ready plugs in Laos...
...Over the long run, however, Synaptics' most valuable asset is likely to be its 15 per-cent stake in Foveon, which was spun outas a separate startup in August 1997...
...Taking this biological inspiration, Mead and Mahowald stressed the energy efficiencies of analog, which would soon be crucial to its triumph in handheld and other power-constrained devices: "In digital systems, data and computational operations must be converted into binary code, a process that requires about 10,000 digital voltage changes per operation...
...Foveon expects to capture the mind and enter the wallet of every photographer, still and motion, on the planet...
...Merrill doubted that the device would work cleanly enough to be usable in a high-precision application, but he submitted the idea to be patented, and then essentially forgot it...
...His crucial book Analog VLSI and Neural Systems—was completed in 1989, just ten years behind his classic text on digital chip design, Introduction to VLSI Systems (written with Lynn Conway...
...Lyon's knowledge of human vision made him more optimistic than Merrill about the technology's potential...
...Synaptics, after all, went off on its own back to the cafeteria, and hit the jackpot with world-beating computer touchpads...
...Movie theaters will soon download their fare in multi-gigabyte files, using technology from Qualcomm and Williams...
...Foveon's X3 chip features three separate layers of photodetectors embedded in silicon...
...As the personal computer in all its forms extends its domains across the Internet, the bulk of music takes the form of the MP3 files used by Napster and its descendants...
...Conventional digital cameras do capture light on silicon, translate that into bits and from there produce color images...
...Its active area is termed the P/N—positive/negative—junction...
...Only slowly did the analog retina chip eke forward...
...Fine with Mead...
...And to skeptics who argued that complex analog chips could never be manufactured economically, Mead offered the final audacious piece in his plan: he would use plain bulk CMOS—complementary metal oxide semiconductor silicon, the same manufacturing process used to fabricate the chips in your PC...
...Prone to dour observations about the perceptual powers of digital computers compared with those of, say, fruit flies, Mead believed even back in the 1980s that digital machines were reaching a dead end...
...Mead's retinal chip could also adapt to changing intensities of light...
...But the split and polish strategy also meant handcrafted modules of glue and prisms, mirrors and multiple microchips, all aligned with exquisite accuracy...
...Widlar's inventions had sustained most of the key companies of Silicon Valley...
...Indeed, Mead had taught them much of what they knew about the design of digital devices...
...There I trapped him at last for several hours while he himself, systematically cut off all routes of his own escape, subsiding into a plush chair in a haze of smoke...
...Eschewing digital couplings, charged or otherwise, its model was the surpassing king of all light processors: the utterly analog human retina.The road he, Mead and the rest took, however, more perfectly resembled our circuitous path to lunch...
...On this bright Silicon Valley noon, though, he seemed shifty, intense and quiet, as if he were planning a getaway...
...And it would scale, according to Lyon's calculations, to no fewer than 300 million pixels—far more than the human eye could absorb, with its 6 million color cones...
...Unlike Widlar, he was at least unlikely to be drunk, or to flaunt a gun...
...See also "The Spectator Interview: Carver Mead,'TAS, September/October 2001...
...I learned myself how hard such a path could be when I presented Mead's ideas to an audience at MIT in 1986, a few months after his classroom revelation of the analog retina chip...
...Talking to Merrill snapped inc back to my own past, nearly 20 years earlier, when all in electronics seemed new...
...He already believed that some day electronic imagers based on his model of the human retina and on his analog chip wizardry would sweep the world...
...Actually, it is a two-faced sign with "Lion & Compass" on both sides in large letters...
...As readers of the Gilder Technology Report will know, Synaptics is now fully engrossed in the touchpad business, with possible new markets beckoning in other fields of pattern recognition...
...a 400-page book is roughly a megabyte of information...
...She suggested we try that...
...Thus the key insight: For a digital computer to compute electron paths requires millions of individual calculations...
...And thus it will dominate the next era of the Net, as the broadband vessel of images and videos superior in resolution and quality to the finest film images, but as portable as a Web page...
...Only when you try to remove the glass do you realize that it is in fact a photograph, indistinguishable in hue and fiber from the pile of real blankets below Now you understand Merrill's impatience with his mere Nikon...
Vol. 35 • March 2002 • No. 2
