No spamming in here please - stick to discussing the code. That's for everyone, not just Marigoldran (though he's usually the worst offender )

That also goes for suggestions - unless you can figure out how to code an AI strategy, please DO NOT suggest it. We can all name 50 things the AI could do better, but that isn't really helpful unless we can convert that into XML that actually does it. So we need to be able to figure out every line of AI-related script so that we can figure out how to get it to do something sensible.

AI effects in Factiondefs.xml:

<PersonalityTraits> Seems mostly just to unlock strategies. 

<AICategoryWeight> is used in conjunction with the modifiers in AIStrategyDefs.xml to determine research choices, probably in another random weighted array

<WarnduranceInit> Sets the AI's initial War Endurance - how much punishment they'll take before they throw in the towel.

So I made some changes and ran a soak on insane/abundant (with base 1600 around ~1000 habitables), spiral galaxy, 40 players, uncommon pirates, normal difficulty. Changes were:

* Made 'startstrats' last 50 turns

* Moved all the non-endgame and startgame strats apart from the 'expansionstrats' require turn 100 to start, so at level 50 they chose expand.

* Removed the tech age requirements from strats

* changed  the governors to concentrate on their speicalism during fillers.

* Removed loads of useless defense, influence, tourism and wealth buildings from industrial and research planet governors

* removed 'growth' world governors completely

* Changed blueprints to be less obsessed with life support, move keen on engines

* moved all AI strategy spending priorities to be more industrially focused.

* tweaked the faction defs to give more races 'expansionist' (though this doesn't actually seem to do much).

The effect of this was enormous. I ran the soak to turn 70ish.

The first thing I noticed was that the Thalan and Terran factions no longer sucked. The Terrans were about the 3rd largest empire, comparable in size to the Krynn and Altarians but quite a way behind the Drengin. The Thalans were a ways behind, but that was mostly due to their starting position; they still had 15+ planets (in my previous game with the same settings but no AI adjustments, they had 6 by turn 150). This was a game with 5-6 big empires and 20-odd medium ones, as opposed to 38 tiny empires and the Krynn.

In terms of planets, the AI was a lot more focused. It was generally possible to see what an AI planet was for. Average industrial production and research were way up, and that effect was felt across the board - even the most viable non-modded planets are only slightly above where the average planet now sits. With no ridiculous 'Growth' worlds dragging their approval down, the AI was able to concentrate on useful, productive planets.

The empires were much wider, and expanded much more rapidly - due almost entirely to the blueprint changes imo, as now they stack engines more effectively, and make good use of sensors (which the AI does need on normal). The also had more and better starbases.

The level of military was far more subdued, however. This wasn't a problem with few pirates and such a large map, but may cause issues in smaller or more pirate-intensive games. Given the AI's vastly better productivity, I expect them to militarize rather swiftly post-turn 100, but they're sticking rather strictly to the colony-spam strategy for now. 

Sadly, there doesn't seem to be a means of telling the AI to use different strategies based on things like "do you have close neighbours", "are pirates threatening our colony ships", etc. Its strategic choices are largely not based on what's happening in-game. 

For the next set of experiments, I'll try adding in more strategies. If that's successful, then I'd propose we do the following:

Create a number of strategies, each lasting roughly 10 turns and divided by personality and ideology. There should be 4-5 different options for each 10-turn chunk of the game. These will vary between going all-out research for 10 turns, to simple outright colony spam for 10 turns, to dedicating itself to building a small combat fleet. 

This will last until around turn 50, where we switch to 20-turn chunks on the same pattern; then at turn 150 we do 50-turn chunks up to turn 300ish, and then put them into endgame mod.

This would require around 80 different strategies to be written, with the AI switching between them rapidly enough that it never gets 'stuck' with a strategy which is truly terrible for it's given situation for too long. 

Hats off to you naselus.  But you will need to run LOTS of soaks, with LOTS of statistics to make any concrete conclusions (an output of the timeline into a file, would be extremely useful to evaluate soaks).  I'm sure this is how stardock tests it themselves, Probably being able to save the soak save game every turn, giving them the ability to go back and see what was going on when its empire made radical changes in power stats (or even just research, military, etc).

Seems the AI still needs some way to react to player military buildup.  Around turn 50 of an Immense map, I had over 100 planets and was preparing to invade the Drengin, but they kept right on building colony ships and constructors.  They still kept on building civilian ships even as my massive carrier forces were shredding their unarmed vessels by the dozen.  With the Yor, it was basically the same - only it ended faster because by then I had Huge strike carriers with 350+ move.

OK, now back by my main machine, which I left on that soak. Looks like my faction surrendered to someone on turn 259.

3 AI races are all neck-and-neck at around 1800 power:

The Iridium have reached 1000 population and have 95 planets, two of them taken from someone else. They've built 48 starbases, built 350 ships and won 78 battles.

The Salarians from the MAss Effect race pack are next, While they only have 48 planets and 25 SBs, they have built nearly 600 ships; they've managed to get 719 population. Their military power is around 800 alone.

Maybe 50 points behind those two are the Volus, also from Mass Effect (in fact, the Mass effect races have done very well generally with seven of the top spots.). 600 ships, 75 planets and 767 population.

The Thalans and the Terrans each have over 40 worlds; the Thalans built 350ish ships and the Terrans 250ish. I suspect the Thalans spent a lot more time at war, sincethey were in the middle of the galaxay and the Terrans were out on the fringe. None the less, both empires have done markedly better than previously.

What's less visible from the general stats is the 'new powers' which appear to be rising in the galaxy; both the Altarians and the Geth are producing huge amount of military production presently and are likely to move up the rankings shortly. The Altarians actually built the most ships (over 700), but have also lost well over 100, while the Geth have now achieved both the highest research output and the highest military manufacturing, and so are likely to become a superpower - despite only having 38 worlds!

Equally interesting is the losers: The Krynn managed only 14 planets, less than 26 other races, and the much-vaunted Yor achieved only 11. Every other vanilla race has at least 40. The minor races, which I have made major for this run, have largely managed a similar 40ish each; the Marauders and Klepart both reached 57 and produced 500+ ships (the Klepart were victims of a multiple-DoW around turn 120 and lost huge numbers of ships over the course of the game). 

While I'm still fairly convinced I would've been way ahead in this had I been playing, I must stress - THIS IS NORMAL DIFFICULTY. A completely unbuffed AI managed to get 95 worlds! The AI not only coped with using maintenance rather than approval, it actually excelled. There were 3 top-of-the-line powers. The Terran and Thalan empires were relevant. And this was from a couple of dozen minor AI script changes.I think we can achieve some pretty great scripts if we work on it. 

I've added them all into my Insane-Abundant balance mod for the moment (available here: https://forums.galciv3.com/467008/page/1/#3566625). I'd love some feedback with that, but it contains a whole bunch of other changes from vanilla - it's not just AI tweaks, and it's only intended to be played with insane maps and abundant planets (since that's the most unbalanced setting for Vanilla).

The thing is, due to the way the AI deals with strategies, what works on one map really won't work on others (since if you played with these changes on a Tiny map, the AI wuld still be producing colony ships long after all the worlds have been taken). I think ideally we need to be producing a tailored AI for each size setting.

Wowowowowowowowow.  You're awesome. Saw your mod.  Does it work on other smaller galaxy settings than Insane? 

This is what the AI needs, it needs short mini-strategies, medium more strategic strategies that last about 50 turns, and enduring goals, which last the length of the game.

The Enduring Goals should be easiest. What does this race want to accomplish? These should be the type of victory the empire goes for and they should be decided with high degree of endurance about 1/10 of the length of the average game scaled to the side of the map, for smaller maps they could be decided once an empire hits a marker like it is ahead in research by 25% or military by 25% and it should be in line with the type of empire it is. If it is malevolent xenophobic expansions and aggressive, it should be gobbling planets and its enduring goal should be extermination of anybody not aligned with it and interested in the same conquests. Should be a long term focus and preference for a certain kind of weapon, or ships dont buld any defense, they should have a certain specific trait that makes them unique and stand out against the rest of the empires in the same galaxy. 

Strategic goals should be lengthy but more in line with mid range goals, we've been declared on war by this enemy, sets objectives for elimination of that enemy and priorities. So if the enemy has 10 planets, which planets to go after first, which ones on our side to defend, which starbases to elminate, which strategy should be used, attack fleets, planets, shipyards or starbases? This is the kind of AI the game is weakest at. It can't determine a strategy for winning. It can fight, it can build weakly but it can't decide which type of target. This should all be figured out by formula, ships vs ships, ships vs starbases, planets vs planets, ships vs starbases, number of shipyards vs shipyards, each of these calculations should be performed and then ranked, and the AI should pick the top or 2nd best strategy and focus on it for 50 turns. If the AI decides it had 5 shipyards and the enemy has 3 it should focus on destroying the 3 shipyards, but has the same number of ships and same fighting strength it should attack the enemy shipyards for up to 50 turns. If it achieves destruction of enemy ship yards then it should re-evaluate and form a new medium term focus. Each fleet AI could have 3-4 of these mid term strategies for each race to keep it relevant.

Tactical AI would be the 10 turn AI where the AI evaluates its ability to cause damage. Set a goal based on how strong the AI is versus the enemy. If it is 50% stronger it should be able to wipe out 50% of the enemy strength in 10 turns, if it is 10% stronger, it should be able to wipe out 10% of the enemy in 10 turns. At the end of the 10 turns if it was successful in its attacks it should keep on the same strategy, if it wasn't it should kick the medium strategy in the ass and redo it.

Feedbacks from the short and medium term strategy could keep the AI playing a shit ton smarter and also give it the ability to really follow an overall strategy instead of playing so damn generic

Also, AI need to be designed so they're capable of an early rush.  

I've noticed the AI doesn't build any military until around turn 100, leaving their planets undefended.  So I guess I'm going to start building transport ships on turn 50.  

I'm not disagreeing, but it's beyond the scope of what we mere modders can do. Take it up with Brad.

This is much more helpful in this context - if you break down your own game into 'sections' and look at what your own short-term strategies are, then list them here, we can convert that into XML scripts. While we can't point things too accurately - so we can't tell it to stack mil bonuses on the shrinker for example - we can point it in the right direction for techs and tell it 'you should just go flat-out colonizer between turns 1-20'. 

Yeah, the tweaks I've put in IAB kinda rely on the player being slowed down by maintenance and the durantium soft-cap on ship hulls atm. Basically, the challenge for the player in the opening 100 turns is in expanding his empire without financial collapse rather than defeating his enemies - this is what's severely lacking in vanilla imo. 

I was genuinely playing a game of it last night, same settings as the soak, and while I was winning I didn't have a big planet advantage over the top AI players (in fact, I was behind two of them); however, I had 80 small gunships and they had a couple of interceptors by turn 100. 

That's mostly because, in the present build, there's only 2 strategy choices for the AI in turn 1-100 atm - start->expand and aggressive start->aggressive expand. What I'm aiming to get set up this week is:

Create a 'stratstrat' option from turn 1-20 which is just 90%manu, 90%mil colony spam.

Add in a 'buildup' strat from 20-30ish where the AI does some research and builds up social manufacturing

30-50 give it a 'pirate-proof' expansion phase, where it throws some small fighters in but still mainly makes colony ships and constructors

50-60 another 'buildup' phase

60-80 another pirate-proof expansion phase

80-100 a more military buildup point where it starts producing decent vessels.

Once we have this as a generic template, then I'll vary it up a bit depending on personalities - so aggressives will have more pirate-proofing and higher weight on attack ships; diplos will push research a little more aggressively for a few 10-turn chunks; expansionist civs will push out more colonies etc. Then I'll give each race at least 3 personality tags.

That should mean that, in a galaxy with several players, we start seeing some real differences between the civs. 

This is excellent work Naselus!

I'm in traveling this week, else I'd be helping out more here - looking forward to seeing how far along things are when I'm back