After reading all of the replies, here is my thoughts:
1) The original tweet, without context, is extremely racist. I would assume it is widely known that the term 'monkey' can be used as a racist term. Think of the movie Remember the Titans when Denzel gave the white coach a banana because the white coach called him a monkey.
2) Pat could have easily followed up his tweet with a second tweet providing some context - without context I don't know how anyone could look at it and say it isn't racist. I don't get offended very easily and my first thought was "wow, the old man fell off his rocker."
3) After it has been confirmed that Brass Monkey was indeed played directly before his tweet this turns out to be a big nothing burger.
I guess where I differ from Pat is if I had his platform and tweeted something that was viewed as racist I would quickly explain myself.