On October 19, 2026, OpenAI published “Advancing coding in low-resource programming languages”, introducing two evaluations for languages with little training code, such as COBOL, Fortran and Visual Basic. MultiLangBench asks a model to build a SQLite-compatible database engine from scratch in one of 20 languages over an extended coding session. Programs receive a database and a SQL query and must produce the modified database, checked against the reference SQLite using tests adapted from SQLite’s own suite, so the same tests apply in every language. OpenAI is releasing the setup and test suite on GitHub.
At maximum reasoning effort, GPT-6 Astra reached an average test pass rate of 87.4% across the 20 languages, up from 26.8% for GPT-5.6 Sol, with gains in every language: COBOL 10.9% to 72.1%, OCaml 25.6% to 85.8%, Fortran 21.4% to 83.5%, Visual Basic 27.5% to 89.4%. Astra scored above 83% in 19 of 20 languages, COBOL being the hardest. The cheaper GPT-6 Luna averaged 40.3% at an estimated 0.77 dollars per sample versus 3.48 dollars for GPT-5.6 Sol.
The second evaluation migrates COBOL programs to C++, Java, Kotlin, Python or Rust, with hidden tests requiring byte-for-byte parity on observable outputs; a migration passes only if all tests pass. GPT-6.1 Sol scored 86.6% at low reasoning effort versus 5.4% for GPT-5.6 Sol at the same setting and at 61% of the cost, and 97.9% at higher effort, which OpenAI says nearly saturates the evaluation.
Why it matters: legacy languages run critical banking, weather and nuclear-safety software, and a language-agnostic, test-graded build task is a cleaner yardstick for long-horizon coding than per-function benchmarks.
What it does not show: both evaluations were built and scored by OpenAI on its own models, the COBOL migration eval is already near saturation, and OpenAI notes production migrations still need expert review against real business rules and dependencies.