Kam, you ask some very interesting questions, to which I have no answers to except for my personal experience.
First off, one can be lucid in one part of a dream and then revert back to a different state of consciousness, where the fear takes over...even from unknown cause.
For example, if you were still in that conscious state, you may have said to yourself this is only a dream and I have nothing to be afraid of because I can fly away, become invisible, transport myself to a different location etc.
The astral experience does not always maintain at the same level of consciousness, from my experience...we go in and out of lucid dreaming, just as we do with OBE's.
Another possibility is that there are some entities on the lower astral that are testing and playing with you and your fears to feed from that. They sometimes are the protectors of the astral if you will to prevent us from going there and the knowledge within.
This is a complex issue with no definitive answers.
Also our own fears, worries and our subconscious ...level of consciousness plays a part in this.
I'm looking forward to others input to your questions and this topic...just my experience and MHO

Bookmarks