I found the pre-meds typically do the best.. some of the chem majors actually did the worst (which I found strange).
My advice: You don't know until you try! But, know what you're getting into ahead of time. General Chemistry (the first year course in chemistry) is typically not too difficult in terms of material. The hardest part for freshmen is adjusting to how professor's teach and test at a university level- you are expected to work very independently instead of the typical "spoon feeding". The exams are more application based rather than regurgitation of what was written on the board or on the powerpoint. Basically, do the problems on the syllabus and read the book using the professor's lectures as simply a "supplement". But, you have a lot to learn about chemistry.. organic, physical, analytical, advanced inorganic are very different from the material taught in high school. But again- you don't know until you try. After your first year in chemistry you generally will know if you like the stuff or if you just thought you did.