Observing time isn’t like ordinary time. You’re working nights, for a start, and in many cases arrive at the observatory with a significant level of jet lag. That’s certainly the case for me at the moment. And then, for Mauna Kea at least, there’s the effect of the altitude. At 14000 feet you’re missing about 1/3 of the usual level of oxygen so nothing, and especially your brain, works properly.
For an impression of what it’s like, have a look at this video – Hotel Mauna Kea.