A colleague in social psychology once told me that he believes affect drives all

A colleague in social psychology once told me that he believes affect drives all attitudes. In other words we have negative attitudes towards others due to emotional reasons and not cognitive ones. There is definitely some evidence that distrust of stigmatized persons is due to affect. Do you think this generalization is true? Might some areas be different from others? [2 References and 2 Citatations needed]