I'm already into the second book, and that question really just hit home.
We have a "self" image, but if the physical stuff isn't the same, can we safely see ourselves the same?
It is similar to the question of cloning...think of the great "Oh Human Star" comic by @bluedelliquanti