To ionize something, you are removing an electron from the HOMO (although see note below), and the ionization potential is a measure of the amount of energy it takes to remove an electron. If the HOMO of compound A is destabilized compared to compound B (i.e., the HOMO of A is higher energy than in compound B), it takes less energy to remove an electron located in that orbital. Electron affinity would have similar considerations with the LUMO, because the LUMO is usually the electron accepter.
(note below) Note that orbital energies are not the same as state energies. When you remove an electron you not only have to consider the orbital energy, which is usually determined from model 1 electron systems. In real chemical systems you also have to consider the energy of interaction between other electrons, which can be substantial when electron correlation is strong. For qualitative comparisons of "similar compounds", the difference in orbital energies is usually sufficient to predict trends, but if you are trying to quantitatively calculate the ionization potentials, electron-electron interactions cannot be neglected. You may well read about
Koopman's Theorem.