My husband turns me off, why?

We have had marital problems from the day we married. I had emotional issues which helped me make a bad decision. Then vows and guilt kept me there. I have always been open about my feelings and needs and we've been through tons of counseling. Even though my husband and I have made changes in leaps and bounds, I still am always irritated and annoyed by him. He almost disgusts me. I don't know if it's past memories, his looks, his bad habits, or what it is. We've been married for 15 years and I have filed for divorce twice, but never completed it because I was afraid of being alone and trying to support myself. Also he always told me he didn't want me to leave and he'd do whatever it took to try to change. So, guilt, feeling like I didn't want to hurt his feelings and that he deserved a chance, I stayed and tried to find love for him. I do love him dearly, but it's more like an annoying brotherly kind of thing. He is always angry and irritable, although he's been trying to do better, every irritable remark just raises the hair on the back of my neck. When he tries to kiss me I have to force myself to relax and not cringe. Sex is not good, even with a road map, he's rough. I can't get turned on by him at all. I have a very good sex drive, get turned on by just talking to a guy friend, so I know it's not my libido. So, why is it that I'm so turned off. Could it be that I'm just not in love with him? What should I do? I feel trapped and deprived.