There's 3 factors:

Personality and strategy both have weight values to determine what the race 'prefers' at any given time. Then the techs themselves have weights on which determine how valuable they are to races with those weightings. The AI then creates a random weighted array of all the techs available and rolls a virtual dice - so say thre's one tech with 10 weight and one with 50, it has a 5/6 chance of picking the weight 50 tech.

Now, the only thing in there which changes at all once the game is in action is the strategy, which AIs will change after a given number of turns. There's 5 triggers for how the AI picks a strategy: turn, victory condition, at war, tech age and personality. You can't check 'have I met someone', or 'do I have this many colonies' or 'have I bumped into some pirates'. The AI will always pick the first strategy in the list which fulfills all the valid triggers, and ignore any others which also fulfill all the triggers. This means that we can't actually tell it to anticipate a war - it will only react once it's IN a war.

The AI's early strategies in vanilla already have strong negative weight on military techs, so it generally avoids them til turn 25, and then doesn't have much bonus weight to them til turn 150ish. Marigoldran plays against Godlike AIs, so he's probably just seeing a combination of the ones that that AI gets for free at that level, and the very, very, very occasional random roll that lands military.

Indeed it should. I did ask Paul if we could get some more triggers exposed in the xml for modders a few weeks back; he didn't really answer. There's a number of triggers in the hardcode which thus far aren't availalbe to us... but these don't seem to impact on which script it chooses to run atm.

Just being able to have it check it's cash balance as a trigger, so we could throw it into 2-3 turn mini-bursts of intensive saving when it's <50 cash would be useful. One to check for hostiles within X hexes of it's border would be awesome, too - so we could tell it to quickly research military tech and crank out a few warships when it bumps into pirates. And one to check for touching cultural borders.

Just these three basic things would revolutionize what we can do with the AI. And any three of a couple of dozen more, too

With the increased power from research techs, you will catch up, and most likely go further than if you researched weapon stuff early. The point here is you can't get punished for not researching those weapons tech early on, since planet invasions is a tech era later, so why bother? In addition, one properly outfitted modern ship can steamroll all those crappy early ships on it's own, making the investment in early weaponry totally moot (lambs to the slaughter). Hence the OP topic about no point in researching them at all. 

However, if there were some plunder mechanics or something, you might be right in your approach to strategy, but currently there isn't something like that.