I don't know what to do

My boyfriend called me selfish at my brothers funeral. Just a little background, I told my boyfriend the night before that I have to go to my brothers funeral in the morning, during that time he was spending time with his mom who lives all the way in Ireland and it was the last few hours then. I couldn't listen to the voice notes he sent me and I told him that I couldn't at the current moment and he said "You are very selfish" and it's affected me ever since. This happened 2 months ago and I know it's kind of my fault for not getting over it, but it's made me question how much I actually mean to him. He's done a lot more but I don't have space to put it all into here.