It's all about the context surrounding how and when it's said.
I'd suggest being more specific about why you're proud of what you claim to be proud of. I'm proud that I passed on one grandfather's work ethic, and didn't pass on my other grandfather's alcoholism, that I was able to raise my girls to think for themselves, proud of my career which helped others. If I try to jam all that into "Proud to be Irish/White" it just seems kind of stupid to me. It's far too general to be meaningful.
So, when I hear anyone say proud to be (insert general race/skin tone/heritage) I just kind of chuckle at them and wonder what they're thinking.
I am neither proud, nor ashamed to be white, because I don't identify myself with the billion other white people who live vastly different lives than I do.