I have never tied this kind of behavior on my Touch to a specific book, but it happens. I have never had it spontaneously fix itself. On my Touch, it happens with time. After a while (a few days of using the Touch), things begin to slow down. The slowdown includes page turns in books, and context switches (like pushing the home button, or starting the web browser). Occasionally I have seen reboots.
To fix it, I press and hold the power button for 20+ seconds to force a shutdown/hard reboot.
Then things are normal again.
Although others have suggested that this is a hardware problem, I am a hardware engineer by trade, and I know a software problem when I see one
It's like Windows slowing down over time and needing to be reinstalled . . .