Guilt and Manipulation?

I always use to get mad instead of dealing with feelings and kicking my mentally abusive ex to the curb. I seldom get mad at anything but anytime now I disagree with my adult daughter or my ex they bring up my anger issues from the past and I don't get mad I even have people monitor when I disagree to make sure because I am so paranoid of being how I once was. Its gotten to a point I isolate myself because everytime they bring up the past I feel so guilty. I feel like I am being abused and manipulated all over again and I don't know how to deal with it without distancing myself.