Actually, when phenolphtalein turns pink the pH might be quite high (say >9), so if you wait too long and/or swirl the solution too vigorously, some CO2 can be absorbed, making it turn colourless again.
In practice, however, the end point is when you see the pink colour persist after all the solution has been thoroughly mixed (i.e. when it's homogeneous).