pandas describes itself in its documentation as “an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.” Where NumPy gives Python a fast but unlabeled array, pandas adds the missing layer that day-to-day data work needs: rows and columns with names, heterogeneous column types, an index, and built-in handling of missing values. Its central object, the DataFrame, is the in-memory equivalent of a spreadsheet or a SQL table.
The library was created by Wes McKinney, who began the project in 2008 while working at the quantitative investment firm AQR Capital Management, where the daily reality of cleaning and aligning financial time series exposed how much friction Python had compared with tools like R or specialized statistical packages. pandas was his answer: a single library that handled loading messy data, aligning it on labels, grouping and aggregating it, joining tables, and reshaping between wide and long forms, all on top of NumPy’s fast arrays.
The design rationale was set out in McKinney’s 2010 paper “Data Structures for Statistical Computing in Python,” published in the Proceedings of the 9th Python in Science Conference (SciPy 2010, pages 56 to 61), which the project still lists as its founding academic citation. The paper introduced the labeled-axis data model and argued that statistical computing in Python needed first-class tabular structures rather than raw arrays. That model, the DataFrame with an explicit index and named columns, is what made pandas feel natural to analysts coming from spreadsheets and SQL.
Internally, pandas long built its columns on NumPy arrays, which gave it speed but also tied it to NumPy’s type system; later versions added extension types and Apache Arrow backing to better handle strings, nullable integers, and columnar interchange. Throughout, the public API stayed centered on the DataFrame and its one-dimensional cousin, the Series, so that operations read like manipulations of named tables.
pandas became the default first step of almost every data-science and machine-learning workflow in Python: read a file or query into a DataFrame, clean and reshape it there, then hand the resulting arrays to scikit-learn or a plotting library. That position as the universal on-ramp for tabular data, more than any single feature, is why it is often called the workhorse of the scientific Python stack.