It's fundamentally a problem of risk perception and the fact that experts perceive risks different than laymen:
No one seems to deny that experts rationalize hazards against dosage and exposure. The public does not. For example, "the public would have more of an all or none view of toxicity....[T]hey appear to equate even small exposures to toxic or carcinogenic chemical with almost certain harm". As well put elsewhere, when a young child drops a lollipop on the floor, the brief contact with dirt causes the parent to throw it away rather than washing it off and returning it to the child.
(From: Berube, D. M. in Nanotechnology and Society: Current and Emerging Ethical Issues (eds Allhoff, F. & Lin, P.) 91–107 (Springer, 2009).)
The media doesn't help to dispel the notion that dosage in unimportant, and nor does it help to dispel the common perception that chemicals are just plain dangerous. In fact, people don't even seem to understand what a chemical is. When I see a consumer product like a lotion or shampoo being advertised as "chemical free", I just groan with annoyance.