I read some new Life patch notes the other day that said sleep deprivation debuffs actually don't disappear after restful sleep and will keep stacking throughout the life of your player character. Restful sleep does deminish it, but it doesn't go away, like rings in a tree.