I envision space combat like this:
1. 2D plane(3D plane like in Homeworld ads no value at all, it just makes you look at tiny dots all the time)
2. turn based combat because that's the best kind of combat in terms of tactical depth, emphasis on visuals and fits perfectly in a space game, X-com and many other games have proven time and time again the benefits of a turn based system.
3. just like in Might & Magic Heroes 6 your skills, researched technology and types of ships determines your battlefield setup when encountering enemy.
4. you don't give orders one unit at a time and then each unit performs an action, but you give sequence of orders, a plan of action which is then executed in a single turn for all the units, of course researched technology/tactics and types of ships determining which options are at your disposal
5. then, at the end of battle, you have an option to view the whole thing uninterrupted from start to finish and post such battles online. Also you would have an option of fixed camera vs free roaming.
6. but for all of this to work and feel great, animations, effects and graphics need to be top notch with dynamic lighting and shadows, sharp textures, particle effects, volumetric explosions, etc.etc.
Basically, you would feel like you are watching Battlestar Galactica
I really believe you would have a hit if combat would be like this instead of the underwhelming, tedious affair of Sins of a Solar Empire.