There is as much as a 7% change in solar radiation based on orbital mechanics, over the long term (even if the Sun doesn't change its output, though that is the suspected cause of the Little Ice Age of a few hundred years ago). And there are weird things long term with the great ocean currents like the Gulf Stream, which have an effect on ice ages.
https://www.youtube.com/watch?v=CM_QS984JKI
In the long run for example the Sahara will be a lot wetter, as it was in the Neolithic. Man made global warming gets us there faster. On the other hand, the melting of the Greenland ice cap might break the Gulf Stream and induce a new ice age, which is a very dry period. So flip a coin.