Polar protic solvents will solvate anions (such as the leaving group and nucleophile) by hydrogen bonding - only protic solvents can participate in hydrogen bonding.
Using an aprotic solvent makes the nucleophile more reactive because it cannot hydrogen bond to it. It is true that an aprotic solvent will also decrease the efficiency of the leaving group for the same reason - good leaving groups and strong nucleophiles are required for SN2, and if that is the case the gain in nucleophilicity of the nucleophile in the absence of hydrogen bonding outwieghs the lack of solvation of the leaving group.
In an SN1 reaction, any gain in nucleophilicity is irrelavent because the nucleophile does not appear in the rate expression (as it is not involved in the rate-determining step). Hence an aprotic solvent can only retard SN1 by diminishing the efficiency of the leaving group.
It is worth pointing out that solvent effects are quite subtle in comparison to nucleophilic strength, leaving group ability, and the stability of any potential carbocations when determining whether a reaction will proceed by SN1 or SN2. For example, you can still do (and I have done) SN2 reactions in polar protic solvents if the nucleophile is strong enough and the leaving group is good enough.