Anderson wrote a couple of essays on that subject and said the reason for the decline was that the Terran Empire's days as a vital and energetic society had gradually given way to decadence. He modelled the Empire on the rise and fall of the Roman Empire.
The Empire's rival and eventual successor, the Mersians (who were ironically saved from extinction by David Falkayn's efforts for the League in the van Rijn era) were more like the men of van Rijn's day. They were modelled more along the lines of the Sassanid Persians - masculine virtues of the hunt and conquering your enemies were dominant. By Flandry's time the Mersians had decided that mankind would not be allowed to become a vassal state like other races had, but would be exterminated because they might be too great a threat if allowed to flourish again. They understood, while at the same time found it hard to believe given the way the noble classes behaved, that, as said by one of the Vach's, "The blood of conquerors once flowed in thier veins and might yet again."
It's all too easy to admire the Mersians and recognize in them some of the "good old days" when humans were first expanding among the stars, but the story "A Plague of Masters" shows the danger in admiring them too much. The scenes involving Flandry and his opposite Mersian agents trying to outsmart eachother, or out-drink eachother are great.