Memories are part of the illusion. All the memories are reality within the waking experience. The waking experience is unreal on the standpoint of the soul, which is the true self. Memories are based on the individual and individuality is falsehood. Basing the judgment on the memories will keeps one in the grip of duality. Duality is cause of ignorance and ignorance is the cause of experience the illusion as reality. Memories of the past are part of the illusion. Without the physical self(ego) the memories are non-existent.
Memories of form, time and space are reality within the illusion. They become unreal on the standpoint of the soul, which is the true self. All the memories of the waking experience disappear in deep sleep. Memories of The dream experience and deep sleep experience are present in the waking experience. When all the three states are unreal on the base of the soul, which is the true self, then what is it that is aware of the coming and going of the three states in succession?