Great American Novel: JENSEN-HWANG — Episode 12: The Crazy Seeds of CUDA

Act V: The Silicon Empire

Part 1: The Calculus of the Sand

The silica sand of a Hayward pool filter did not register the panic of the Wall Street financial analysts. It did not monitor the daily price of copper, nor did it shudder when the corporate boards in New York declared that Nvidia’s massive investments in unified device architecture were a form of corporate suicide. The sand cared only about the law of gravity and the fluid resistance of water. Inside the heavy fiberglass tank, fifty pounds of microscopic quartz grains sat in the dark, acting as a massive, parallel filtration array. When the pool pump pushed the dirty water through the tank, the water did not pass through a single, neat channel. It shattered into millions of tiny, independent streams, winding through the microscopic gaps between the grains of sand. Every grain did a fraction of the work, filtering out a single speck of dust simultaneously, returning a crystal-clear flow of water to the Sunnyvale pool.

It was a bright, hot Sunday morning in July 2012. Jensen Huang was forty-nine years old. For the last five years, he had been engaged in a silent, brutal war with his own shareholders. He had spent over five billion dollars of the company’s cash to put a tiny, expensive block of calculation silicon called a CUDA core into every single consumer graphics card they sold. The gamers who bought the cards for their computers did not want CUDA. They did not even know what it was. They wanted only to play their shooting games at higher frame rates, and they were angry that Jensen was charging them a premium for a processing brain they did not use. The board of directors had spent the previous quarter suggesting that Jensen’s insistence on keeping CUDA in the consumer cards was a path to bankruptcy.

But on this Sunday morning, the billionaire CEO was kneeling in the damp dirt behind his garage, his hands covered in black sludge as he unscrewed the high-pressure PVC coupling of his pool filter. He was wearing his faded gray t-shirt, old jeans, and his black leather jacket, which was now splattered with gray sand and water. Lori stood on the grass nearby, holding a garden hose, her blue eyes watching him with that calm, dry affection that had been his only anchor since the basement of Dearborn Hall.

“Jense,” Lori said, her voice clear in the warm morning air. “The chief financial officer called the house three times. He says the gross margin on our desktop graphics cards has dropped below forty percent because of the extra silicone area we are giving away for CUDA. He says the analysts are preparing to downgrade our stock on Monday morning unless you announce a freeze on the compiler development budget.” Jensen did not look up from the pipe. He used his wrench to loosen the heavy plastic collar, his muscles tensed under his leather jacket, his fingers steady as the gray, dirty water spilled over his boots.

“The analysts are sequential thinkers, Lori,” Jensen whispered, his voice carrying that quiet, vibrating intensity of his youth. “They look at our chip and they see a consumer product. They think we are selling a toy for teenagers to play video games. But they do not understand that we are selling a new form of light. We are putting a supercomputer into the hands of every college student, every researcher, and every child who wants to think in three dimensions. If we do not build the hardware first, the software will never be written. You cannot plant the seed if you do not plow the field.”

“The board says the field is too dry, Jense,” Lori said softly, taking a step closer, her hand reaching out to touch his wet shoulder. “They say we have spent five billion dollars on a tool that only fifty university researchers in the entire world are using. They want to know when the harvest begins.” Jensen stopped his wrench. He sat back on his heels, his hands covered in wet sand, his glasses crooked on his nose. He looked down at the gray sand filter, then turned his face to look at his own reflection in the window of the garage.

He saw his graying hair, his tired eyes, and the deep lines of exhaustion on his forehead. But beneath the gray, he saw the nine-year-old boy of Tacoma, standing in the cold rain of the boarding school, watching the water find its way through the mud. “The sand is the proof, Lori,” Jensen whispered. “The CPU is a single, clean glass pipe. It can carry a high-velocity stream of water, but if you throw a handful of dirt into it, the pipe blocks instantly. But my parallel engine… it is the sand filter. It is millions of tiny paths, all working together, all calculating simultaneously. If one path blocks, the other millions simply carry the load. It is the only way the human mind can ever calculate the complexity of nature. The harvest will come, Lori. We just have to make sure our joints are tight when the water hits.”

He took the garden hose from her hand, washed the black dirt from his fingers with a sudden, happy laugh, and tightened the final PVC coupling with a sharp, heavy twist. “There,” Jensen said, his toothy grin wide and bright under the sun. “The filter is optimized. Now, let us go see what the children are doing in Canada.”

Part 2: The Feast of the Unwashed

The laboratory in the basement of the computer science building at the University of Toronto was a monument to the beautiful chaos of raw youth. It was November 2012, and the room smelled of cold pizza crusts, stale cigarette smoke, sour laundry, and the high-voltage ozone emitted by two home-built desktop towers that were running at ninety-nine percent thermal capacity in the corner. Around the cluttered wooden table sat Alex Krizhevsky and Ilya Sutskever, two graduate students in their mid-twenties, their hair uncombed, their clothes wrinkled, their eyes bloodshot behind their thick glasses. On the table between them lay three empty boxes of cheap delivery pizza and two plastic bottles of warm ginger ale.

They had just won the ImageNet global computer vision competition by an absolute landslide. Using a neural network they called AlexNet, they had achieved an error rate that was forty percent lower than any other team in the world. But they had not won by using the university’s massive, million-dollar mainframe. They had won by using two off-the-shelf Nvidia GTX 580 graphics cards, which they had bought with their grocery money and programmed using Jensen’s unified device architecture.

The heavy wooden door of the laboratory swung open with a sharp, echoing click. Jensen Huang walked in. He did not wear a corporate suit. He did not bring an executive team of vice presidents and attorneys. He wore his black leather jacket, his sleeves rolled up to his forearms, his canvas sneakers squeaking on the dirty linoleum floor. In his arms, he held three large, steaming boxes of fresh, high-quality local pizza and a cold two-liter bottle of classic Coca-Cola.

“Alright!” Jensen’s voice boomed through the quiet lab, loud, joyous, and carrying no executive distance. “Who is starving? The neural network is a high-performance engine, but the human brain requires an immediate input of caramelized sugar and grease! Clear those manuals! We need to allocate memory for the pepperoni!” The two graduate students stared at him, their jaws dropping as the billionaire CEO of Nvidia set the hot pizza boxes directly on top of their printed research drafts.

“Mr. Huang,” Alex Krizhevsky stammered, his face turning a deep red. “We… we didn’t expect you to fly to Canada. The university said you were in New York for the analyst meeting.” “Wall Street is a flat, sequential calculation, Alex,” Jensen laughed, pulling up a dusty wooden stool and sitting down directly between the two young men. He did not wait for an invitation; he reached into a pizza box, snatched a giant, grease-dripping slice, and popped it into his mouth with a loud, satisfying crunch. “They are talking about dividend payouts and cost-reduction strategies. But you… you are talking about the future of the human mind. I would rather eat cold pizza with the giants of tomorrow than drink vintage champagne with the ghosts of yesterday.”

He wiped his greasy fingers on his old gray t-shirt under his jacket, his eyes locking onto Alex’s face with an intensity so absolute, so warm, it felt like a spotlight hitting their desk. “Show me the code, Alex,” Jensen whispered. Alex’s fingers trembled as he clicked his keyboard, opening a window of green-on-black code. It was their CUDA implementation of a deep convolutional neural network. Jensen leaned forward, his spectacles sliding down his nose, his face close to the screen. He didn’t look at the high-level diagrams. He looked at the low-level memory allocation calls—the precise, neat lines where the young programmers had forced the graphics card’s parallel processors to calculate the weights of the virtual neurons simultaneously.

“This is beautiful, Alex,” Jensen said, his voice dropping into a quiet, vibrating reverence. “You didn’t just write a program. You… you built a brain.” “We didn’t have enough memory, Jense,” Ilya Sutskever said, his voice quiet but carrying a deep, intense passion. “The GTX 580 only has three gigabytes of RAM. We had to split the network across two cards, manual routing, back and forth, sixty times a second. If we had more memory… if we had a chip designed for the tensor math instead of the gaming pixels… we could train a network with a hundred million connections. We could make it see anything.”

Jensen looked at the two young men. He saw their poverty, their dirt, and their uncombed hair. But he also saw their fire. He saw his own youth in the Tacoma attic, trying to build a bridge of iron out of nothing but his own skin and a set of old tools. He reached into his leather satchel. He did not pull out a legal contract or a corporate NDA. He pulled out a large, heavy plastic case containing a prototype card that had not yet been announced to the world—the Nvidia Tesla K20.

“This has six gigabytes of high-bandwidth memory, Ilya,” Jensen whispered, setting the pristine, gold-trimmed card directly onto the greasy wooden table. “And it has twenty-four hundred parallel processing cores designed specifically for double-precision vector mathematics. It is yours. I have three more of them in my car outside.” Alex Krizhevsky gasped, his hand reaching out to touch the gold shroud of the card as if it were a sacred relic. “But Jense… this card costs five thousand dollars. We cannot pay you.”

“You have already paid me, Alex,” Jensen said, his face breaking into that wide, toothy grin that made his eyes crinkle. “You have shown me that my five billion dollars was not a waste. You have shown me that the parallel philosophy is the true language of the future. Go build your brain, freshman. I will make sure you never run out of silicon.” He stood up, his leather jacket creaking, his hand slapping Alex’s shoulder with a loud, resounding bang.

“Now, eat your pizza! We have an empire to build!”

Part 3: The Sermon of the Thought Engine

The annual shareholder meeting of Nvidia Corporation in the Santa Clara ballroom was a silent arena of cold, corporate hostility. It was May 2013. The company’s net income had dropped by fifty percent because of the massive, ongoing R&D expenditures required to maintain the CUDA software pipeline and build the new Kepler chip architecture. The institutional investors in the front row—men wearing identical charcoal suits, expensive gold watches, and blue silk ties—sat with their arms folded, their faces severe under the bright lights of the stage.

They had prepared a motion to remove Jensen Huang from the position of chairman, claiming that his obsession with high-performance computing was destroying the company’s core consumer business. Jensen walked onto the stage. He did not wear a suit. He did not bring a laser pointer or a ninety-slide presentation filled with bullet points and financial projections. He wore his black leather jacket, his blue jeans, and his canvas sneakers. He carried nothing but his small, brass pocket-compass, which he set on the wooden podium with a soft, metallic click.

The room went dead silent. “For thirty years, we have built tools to multiply the physical force of the human body,” Jensen began, his voice quiet but carrying a hard, vibrating resonance that cut through the low hum of the air-conditioning. “We built the steam engine to multiply our muscles. We built the sequential microprocessor to multiply our ability to file documents and calculate taxes. We built the flat, orderly world of the CPU.” He looked at the front row of bankers, his black eyes locking onto their faces with an intensity that was more like a preacher’s conviction than a CEO’s defense.

“But the CPU has hit a wall,” Jensen said. “Not because the engineers are lazy, but because the universe is not sequential. The human brain does not calculate one thought at a time. When you look at my face, your eyes do not read my glasses, then my nose, then my leather jacket in a sequence. Your brain calculates every pixel of light, every color, every depth, and every memory of my face simultaneously, in parallel. The human mind is a parallel machine, gentlemen.”

He slammed his hand against the podium, the brass compass shivered, and the loud, dry clap of his palm echoed off the high ceiling of the ballroom. “We are not a graphics card company,” Jensen roared, his voice rising in passion, his hands gesturing wildly as if he were drawing the vectors of the air. “We are not a video game company! We are building the engine of the next human age. We are building the machine that will allow our children to simulate the physics of the earth, to calculate the folding of proteins, and to build artificial minds that can think alongside us. The five billion dollars we spent on CUDA was not an expense. It was a covenant with the future of human intelligence.”

“But Mr. Huang,” an institutional investor in the second row stood up, his voice sharp and cold. “The market is not buying artificial minds today. They are buying graphics cards. Your stock price is down thirty percent. How do you justify this loss to your shareholders?” Jensen stood at the podium, his leather jacket flared open, his chest rising and falling. He did not look at the stock chart. He looked at the investor with that beautiful, un-bendable, and blind conviction that had survived the Tacoma rain, the Oneida reformatory, and the Portland Denny’s.

“I do not justify it, sir,” Jensen whispered, his voice dropping into a fierce, commanding rumble that made the entire ballroom lean forward in their seats. “I am telling you that you are looking at the dirt, while we are planting the seed. If you want a company that will give you a safe, predictable ten percent dividend next quarter, sell your Nvidia shares today. Go buy a utility company. Go buy a bank. They are very safe, and they are very flat. But if you want to be a giant… if you want to put your hands on the throat of the future and write the code of the next century… you stay on this ship. Because we are not turning back.”

He stood silent for five long seconds, his black eyes unblinking, his body solid as iron. The investor stared at him, his mouth slightly open, his charcoal-clad shoulders tensing under the sheer weight of Jensen’s belief. He slowly sat down, his hand reaching for his pen, but he did not sign the motion to remove the chairman. A low, deep rumble of applause started at the back of the room—not from the bankers in the front row, but from the young systems engineers and software developers who had sneaked into the back of the ballroom. Within ten seconds, the applause rose, filling the hall with a loud, triumphant roar that shook the glass light fixtures.

Jensen smiled—a wide, toothy grin that made his glasses slide down his nose. He picked up his brass compass, slid it into his leather pocket, and walked off the stage, his steps fast and precise. He was forty-nine. He was dirty from the garden sand. He was hated by Wall Street. But as he walked out into the bright California sun, he knew the truth. The seed was in the ground. The water was flowing. And the future was about to explode.