A formal grammar is a finite set of rules that says how to build the strings belonging to a language. Each rule, called a production, lets you replace one symbol with a sequence of others; starting from a single start symbol and applying rules repeatedly, you generate the language’s valid strings, and only those. This turns the vague notion of “the sentences of a language” into a precise, mechanical definition.
The modern idea of a generative grammar in this sense comes from Noam Chomsky’s 1956 paper “Three Models for the Description of Language,” which treats grammars as systems of rules that produce the strings of a language and compares the power of several such systems. From that comparison came the levels of grammar that make up the Chomsky hierarchy, each tied to a kind of abstract machine able to recognize the strings the grammar generates.
That pairing of grammars with machines is what makes formal grammars useful for computing. The Stanford Encyclopedia’s account of computation lays out the same family of recognizing machines, from finite automata to Turing machines, so a grammar’s type tells a language designer exactly how powerful a parser the language will require.
In practice, the syntax of a programming language is written as a formal grammar, very often a context-free grammar expressed in Backus-Naur Form. That grammar serves two purposes at once: it is the human-readable specification of what counts as a legal program, and it is the input from which a parser is built, so that a compiler can decide mechanically whether source code is well formed and how its parts fit together.