The mean value is an exception. If it were always the case, then reactions wouldn't release nor absorb heat.
So I don't see why it needs a justification. P-H is just one lucky value.
Or are there deeper reasons? Cl-Cl is weak because the molecule has too many electrons. The best bond is N≡N; O=O has to accommodate two electrons more on antibonding orbitals, Cl-Cl four, which makes it reactive. But the same reasoning is difficult for P-P which is only a part of a more complex molecule.