Yep, if I can't see my pin, or line my sight up with my peep, there's no use hunting after that, unless there's a deer around me, then I just wait til he leaves.
Like others have said, based solely on being able to see my pins. I will sometimes prepare for the end of the hunt not long before if I haven't been seeing deer but if I have then everything waits til I'm done hunting. Slowly pack up and slip out of there just as quietly as I did getting to the stand.