NVIDIA’s own corporate timeline records that the company unveiled the CUDA architecture in 2006, opening the parallel-processing capabilities of its graphics processors to science and research. CUDA, short for Compute Unified Device Architecture, lets developers write general-purpose programs in a C-like language that run across the thousands of small cores on an NVIDIA GPU, rather than using the chip only for rendering graphics. The first public CUDA software toolkit followed in 2007.
Before CUDA, using a GPU for non-graphics math required awkwardly disguising calculations as drawing operations. CUDA exposed the GPU as a straightforward parallel computer, making it practical to accelerate physics simulations, financial models, scientific computing, and, crucially, the matrix operations at the heart of neural networks.
CUDA turned out to be the hardware foundation of the deep learning era. When researchers trained large neural networks such as AlexNet in 2012, they relied on NVIDIA GPUs programmed through CUDA to make training fast enough to be practical. That dependence helped make NVIDIA one of the most valuable companies in the world.