I think the main goal is not to make the AI clever, but convincing as a fun opponent, or crowd of opponents in this case.
This, a thousand times this. To me, a fun, immersive AI is better than a blood-thirsty, perfectly optimal AI.
I play a lot of strategy games, mostly 4X, some not (MOO2, GalCiv II/III, Civ 2/3/4/5, AoW 2/3, Hearts of Iron III, et al.). On every forum I've visited, for every game I have played, I see the same comments: "This AI sucks, it's total crap." Sometimes with "and Game X did it better." Go to Game X's forum and you see "Game Y does it better."
It seems to me that this is often a combination of two factors: a vast overestimation of the capabilities of modern strategy game AI, and an inherent disagreement with the purpose of any particular game's AI - whether it should be only out to "win", or be an immersive AI with character; whether it should be predictable, random, optimal, etc. And when it comes to challenging a player, many expect it to be able to play at human-levels of competence.
Disregarding the sheer improbability of that occurring in a game (the first person to code human-level AI will be ushering in an incredible revolution of technical development), some seem to forget the most basic fact, that "human-level" competence is already a widely varying attribute. Some people struggle with the easiest difficulty levels in games; if you could build AI to perfectly imitate those individuals, there would still be outcry of "how poor the AI is."
Anyway, in regards to the OP, I found GalCiv II to have quite decent AI, certainly comparable to other games, and often superior to them. The fact that Brad and Stardock continue to work on and support their games for years just ensures that the AI will continue to improve - and it already has a pretty good start.