Actually I think the issue is invading a whole planet across the gulf of space is a really tricky logistical proposition. How do you land troops, take control of and keep control of a population of billions a long way from your home world.

Some races though do get this tech earlier than others.

Shooting a missile at a factory is more in the prevue of sabotage and Guerrilla warfare than planetary invasion. I agree planetary bombardment weapons should be possible sooner than full blown invasions.

They haven't told us much about the mechanics of planatary invasion in GC3 yet either.

Just to reiterate, invasion is fine in "age of war", I just think that you should have some option in first stage when a race doesn't respect your controlled area and plops a colony next to yours.  Dropping stuff on their buildings and shooting missiles seems easy and hard to justify why this is a mid-game tech.  Destruction is easy, conquest is hard.

Build a couple of consulates and you'll culture flip that colony in no time. In fact, this is better then settling it yourself because if doesn't cost you the population. Also, I often build a small hull ship with a single laser and as many engines as I can fit. This is excellent for killing scouts, survey ships, and colony ships that wander into your territory.

As a CS/AI guy, I'm skeptical.  Data mining data points isn't that powerful.  Data mining strategies and intents (which you could maybe define as associations in time) is beyond corporate supercomputers.  The union of { Amazon, Walmart, Netflix, eBay } "AI"s together can't even data-mine your shopping behavior.  The entire industry of network security can't begin to data-mine hackers' intents.  All you see is the action (or, more generally a stream of actions across time), and that's not enough to deduce the reasoning behind it, nor to anticipate what comes next.

Consider only the tactic of mining starbase for range extension.  Suppose your AI doesn't know it, and I demonstrate it a thousand times, in a thousand different games (and win all of them because of it).  Every time, it's different, to match the map configuration.  Some games, I do it.  Some games, I don't.  Now: on what pattern trigger does the AI invoke this idea (and commit ~20 build-and-move turns and a pie-wedge of nearby assets to it)?

Somehow you have to extract a tactic as some kind of frozen, declarative data chunk that represents ... a recipe?  A pattern-response pair?  A temporal script?  It seems that it would need to contain conditional and iteration (because some tactics really are iterative, and all tactics admit alternatives).  Just those two make it a Turing-complete programming language(!).  But then ... you're in the business of automatic programming, which would be grand, but it's ... not solvable in this century, perhaps.

We'll see.  I'll test that AI by forcing it to reveal its decision-making abilities, i.e. I'm gonna data-mine its style of play .  Suppose it has 2 weak frontier colonies, I fork them with a transport + fleet that could take either one of them.  The AI has 3 small fleets nearby.  How do they move?  How does it decide?  Protect one?  Intercept me?  Retreat?  Suppose I deflection-sac a weak fleet just to pull its only strong stack 1 hex out of range from its homeworld('s shipyard).  Does it take me?  Does it think ahead and see the real threat?

• In Avalon Hill's Third Reich (boardgame), naval fleets are modeled abstractly as factors, from 1 to 9, with a national combat bonus: England +2, Italy -3(?).  Oddly, this enabled the Italians to flaunt their inferiority and swarm the Med with "naked" transport missions, ferrying land units from Europe to N.Africa.  England's embattled fleets holed up in Med ports could intercept any one bait and surely sink it ... but then Italy will counter-intercept with everything and some German loans, and force a showdown.  England is screwed if it does, irked if it doesn't.  Could the AI play either side of that correctly?

Planetary invasion should be available as soon as appropriate tech is discovered. And given that one has the capability to colonize alien worlds, I'd suggest this is easily within the grasp of a space faring civilization.  It is like saying, sure we have ships, but we aren't advanced enough to amphibious assault. Folks on earth have been doing this since (at least) the bronze age.

Make it inefficient, make it costly (in ground pounder units) if you must, but it should be part of the game from at least very early on. One has garrisons to defend against this sort of thing. (And anti-invasion tech, equivalent of AA for incoming drop ships/etc.)

For example, until drop ship/assault boat tech is developed, perhaps the most basic "AA" planetary defense system will devastate any invasion force, making the garrison's mop up operation a sure thing.

Couldn't this effectively be done via basic tech, and invasion tactic modding? I remember modding a, "destroy one building", invasion tactic into GCII where you could harass and even cripple an enemy planet, but not capture it. I believe there were several other modders that included similar in their public mods.

The bonuses the AI get are way more than this. See my reply in the Question about Current AI thread about the real bonuses the AI gets on the various difficulty levels.

No problem, and no reason the feel sorry. I just augmented your post to show the massive bonuses the AI gets.

Is s learning AI really possible? How about on GV3's budget? AI bonuses is really the cheap way to go, otherwise multi-player. 

As far as having ships destroy ground buildings. It seems something that should be doable in the "real world" but being a video game I hope not because I don't want to have a fleet come in at a time I'm vulnerable and wipe out all my stuff taking 30-40 turns to rebuild or even more in the late late game. Can you imagine having to rebuild everything in your empire after being a few hundred turns in...

I'm certainly not a programmer, but this does seem complex. Do any video games do this currently, or would SD be developing this independently?

Heck, Google and IBM combined don't do that yet.  It's good that somebody's envisioning grand goals, but reality wields cruel pruning shears.

It would be fun to create an academic research group and explore this niche for a couple of decades.  (Ahem )  Nothing like it exists yet in the CS / AI / gaming literature.  Basically, take any one sentence out of that paragraph, and by itself it's already far more complex than, for example, chess programming.  (Chess is tiny compared to a 4X, because it has fewer units and only one board size, so the branching factor is orders of magnitude smaller than in a 4X.  That makes chess just barely amenable to brute-force search with clever optimizations.  We "solved" that in ~30 years, where today it's stronger than a human, limited only by hardware.  So now it's a GM's tool, like a car or a side-scanning sonar.)

Netflix recently offered a $10M prize in an open data mining / machine learning challenge for the best solution to a much simpler problem, that of mining your future movie preferences from your previous movie rentals.  Inevitably (as these things go), university groups and a few commercial think tanks competed for weeks until about 48 hours before the deadline, and then ... it ended in a flurry of mergers, as they cut side-chat deals and formed ensembles of 2-3 teams, then 4-5 teams, each leapfrogging the previous.  The final winner was roughly an ensemble of 6 or 7 of the former top 12 (and they beat out the other ensemble).  Netflix paid, but it has not noticeably improved their recommendations

Basically, the union of { IBM, Netflix, Amazon, eBay, Google, Walmart } have not solved this yet.

This seems to require:

• an algorithmic representation of human visible actions (i.e. an uber-duber, semi-telepathic macro recorder that distills human actions down to a C-like code block)
• an uber algorithmic language
• an uber (not-just-a-)macro recorder

... at which point, Microsoft hires you away to replace Visual Basic for Applications in Excel and Word, because this would be simply superior.

Note that all of the above is not even the AI itself; as the OP puts it, this is only the adaptive component of some already-existing AI.  It's not merely unsolved, it contains unsolved sub-problems.  It's so off-the-wall that there's basically no money in it, anywhere.  Not even Google is hiring teams of people to work on this.  (Between this and self-driving cars, the cars are much easier (prototypes exist now) and surely more profitable.)

Ultimately, I do have some ideas on how this could be tackled ... but I also appreciate the sheer scale of complexity involved.  I conjecture that we should pursue this in tiers, as part of my vision (described elsewhere) of an adaptive UI that scales up.

• Phase I: Lisp scripting.  You do everything yourself, by writing code with the full power of Lisp.  You can automate any ship path, planet build template, wheel balancing, alert-when-itchy, fleet annotation icons, windows hiding, UI customization, whatever.  (This is already beyond any 4X.)
• Phase II: UI-as-student.  The fledgling "UI AI" is like a new secretary who doesn't know what you do, or a bright 8-year-old nephew whom you must babysit for two weeks.  It watches over your shoulder, and tries to ... decompile your stream of human actions into algorithmic code.  (I do not know how.)  It may attempt to deduce your intent, maybe with an internal representation of your belief-state as a graph-of-graphs.  It may interrupt you a hundred times before turn 20 with pop-up windows asking Why didn't you colonize Mars?  Why did you do that?  (Be careful what you wish for )  It ponders sequences of your play-flow and tries to extract chunks.  If it notices that you never use certain UI features / windows / menu selections, it may streamline the UI for you by moving them to deeper sub-menus.  If it sees that you always drill down this window-path to that datum, it may float them higher.  This would utterly revolutionize Office, CAD systems, web browsers, and every other app domain.  The UI and window frame boundary would no longer just be static elements that split your monitor space -- instead, they would change over time as you use the system, until the system morphs into something that fits the way you use it.  This is only Phase II
• Phase III: Max Headroom.  The end product of the UI AI is that it has re-created / distilled / generated, in uber-code, an algorithmic, executable representation that -- mimics you.  This uber-script "plays" the way you play, decides how you would, wants-prefers-reacts-emotes the way you do.  For yuks, you could choose Autoplay, load your Self(-script) vs. 3 vanilla AIs, and watch as the game auto-beats those AIs in the same way you would.  You could maybe trust this script to automate portions of the game for you.  (Corollary: This script is (or would subsume) your set of ministers / governors.  Conjecture: Any sufficiently satisfactory set of ministers / governors would become this script.)  You could play against "yourself".  (If you ever win, then go back to Phase II and teach it some more.)  Difficulty: This falls under the scope of automatic programming, which has been a pipe dream in CS since before Fortran.  (They thought Fortran would be the automation!)
• Phase IV: Script-mining.  Now that we have a code-like representation of a human, we can envision compilers, maybe latent semantic analysis, meta-reasoners, and other tools that crawl over such a representation and try to extract ... something.  What should the AI even look for?  Event-condition-action rules?  Pattern-matching?  We can't even do this for the much simpler problem of: for this programming assignment A, for that student's submission X, what is her score?  (That would be the mythical automatic grading, which would revolutionize university curricula.)  In the limit, any meta-reasoner you use to mine a script strays perilously close to the "CS theoretical event horizon" of a Turing machine accepting the description of another Turing machine and then simulating it, which is known to embed undecidability issues sufficient to fill an entire year of graduate-level textbooks.
• Phase V:Adaptive AI.  Having mastered the previous 4 phases, we (finally) have all of the ingredients we need: a vanilla AI, some scripts of human players, the script mining tools, and whatever output the script miners emit.  Have the AI slurp all those in, and somehow modify its behavior according to what's in this elixir-of-extracted-chunks.  I can't see that far, so I don't know what the details would be like.

It's a big vision, which I heartily endorse.  But it's so big ... that the combined intellects of 21st-century academia, tech, DoD, and hackers trying to beat them all, have not even dared to start on any part of it.  The sub-problems are too hard, the payoff is too obscure, there's no carrot to entice anybody to try.

If I ever found a dev company, I'll investigate this, starting with Phase I (I might have to cheat and not call it a "game" tho)