I would say that the stability of the conjugate acid is by far the driving force here.
But, there are a lot of period 2 properties that do not "jive" with the overall trend including this one. Because the atoms in the period 2 are so small, you are trying to force a lot of negative charge close together, closer to other negative charges, and like charges repeal each other.
The neutral atom of oxygen, yes, is more electronegative then S. But once you add and extra electron to oxygen, its negative charge density is much larger then for sulfur, because sulfur has more room to spread out the negative charge density. As such, oxygen is more happier to form a bond and get ride of some of that negative charge density.
But, if you looked at just that alone you would think it would not want to even form in the first place; it is a delicate balance between electronegativety, charge density, and conjugate pairs.