So, R-TEAM, as I understand your argument, it is, in a sentence:
Any races I do not have an Open Borders agreement with should not be allowed to fly through my territory.
Two things I see resulting from GalCiv3 implementing a rule that is "If Race B is in Race A's Territory where Race B and Race A's Relationship Does Not Equal Open Borders, Race B's ships blow up/Race B's move is an Act Of War and now without any hesitation, at Turn End Race A and Race B are At War.":
Lots of wars. Like, lots and lots very early if that mechanic is implemented to start a war automatically without any input from you, the player. This would have the good point - for those of us who dream of Tall Empires - of totally nerfing the Colony Rush as every race would need to be building up military from Turn 1 for the inevitable war when, inevitably, that Terran Survey Ship 4, just minding it's own business, looking for resources/money, enters your Area Of Control.
Also, it's seriously going to ruin where you, as a player, send your ships. Example: You're not at war with the Terrans. However, there's 4 sweet very useful resources clustered around the Arya star system that no Race has claimed, so you build a Constructor with the fastest engines you have to claim it. However, to get to those four clusters, you have to enter Terran controlled space. Do you really want to have to fight a war every time you step out of your own clearly defined Area Of Control? Or do you want to make the AI even more stupid and have it just nod politely at you being in it's Area Of Control but have to accept bloody, gory war if it so much as enters into one hex of yours?
My compromise, not that I honestly think GalCiv3 needs it, is like a Toll Road. Every time a non-No Borders race enters your Area Of Control, 10BC is transferred from their Treasury to yours for every Turn they are in your AOC. Obviously, as you say, Races who don't like you won't care and won't pay the 10BC and will suffer any diplomatic and/or military punishments accordingly.