re: Crackpot ASOIAF Theories SPOILERS

Why do people automatically think Bloodraven has bad motives? (Note: I've only read D&E once, while on a plane, so pardon any errors in my comprehension.) Bloodraven was talked about negatively by some in the Dunk and Egg stories but other than using magic didn't seem to be all that evil once we finally met him. Likely people just feared him and talked bad about him because he was so powerful and as Hand would use his power from a highly visible position. Of course those on the Blackfyre side would talk negatively of him. Am I missing something that should be easily seen?