My first real game I played on normal everything, medium sized galaxy with three opponents. It was really hard and I just survived long enough to build an ascension gate on my homeworld while my borders were being destroyed and taken over. It was epic. I also learned a lot about the game.
My second real game i played on small with three opponents again, this time with better game knowledge and a lucky-ish start regarding planets. By the time I spotted all three other civs I was much more powerful in every way. They seemed cramped and didn't have too many planets each. It wasn't so interesting, not so epic. Unfortunately it took me about 5 hours to notice this.
So how should I adjust the difficultly for my next game? I don't want to get five hours in and have to restart again. What does galaxy difficulty do? What size should I play? I'd prefer the game to not go too long though.