I have the same problem sometimes too. I find it works faster if I am zoomed nearly all the way in prior to confirming long range movement. Another problem I also experience is if I happen to accidentally mouse over or zoom too close to a blackhole, freezes everything for a couple of minutes.
All in all, once you find the work around it's not much of a problem for me, I love the game. Just wish I could turn black hole animation off.
Intel core i7, 4gb nvidia graphics card and 16 gb ram.