Exactly. Who's to say the population isn't being moved with the materials for production. It also may defeat the purpose of having shipyards since you can't launch a large invasion of billions of people from you're border worlds, eventually you'll have ships and not enough population for transports.
You can't get a war going if there aren't many planets within six tiles and you have 20-30 turns to the front lines! Or if I have a few planets within range and production high enough to make 2-4 transports a turn will empty planets fast or production will stall, which sucks. Rage quit.
So good luck with this.
(INSERT AVATARS SIGNATURE) 
