Hi SeekerOfKnowledge.
It seems to me that when human souls go to the life between lives we shed our egos and our negativity. We exist in unity, peace, understanding and love. A far less chaotic existence than we get here on Earth.
It also seems that we know more of a Creators intentions. Why else would we consciously endure Earthbound incarnations? We seem to accept the madness. Perhaps in spirit we know that if you spare the rod you spoil the child. We know we need the human experience.
Personally I think that the most likely explanation is that life is a morality play. It seems that in reality every creature is impervious and immortal in spirit but we are given the illusion of mortality to experience harm, fear, doubt, uncertainty, our capacity for good, our capacity for evil, faith, lack of faith and on and on. These are things that the life between lives doesn't seem to be able to provide.
The following scene always meant something to me. Imagine Picard is Earthbound man challenging God ( Q). Wanting to know why he doesn't get a direct answer. 'All the galaxies a stage' refers to reincarnation being a stage. 'Life is but a walking shadow' refers to the life between lives and 'what a piece of work is man' refers to mans ultimate destiny and reasons for engaging in the earth play:
I truly believe we incarnate on Earth to morally grow towards God. Right now though, in the 21st century, we are still primatives...children playing in Earths sandpit.
My life lessons and experiences have reinforced that belief I have in mans ultimate destiny. I could be wrong though. I could just think that because of how my minds constructed.