Verb
rub someone the wrong way (third-person singular simple present rubs someone the wrong way, present participle rubbing someone the wrong way, simple past and past participle rubbed someone the wrong way)
(idiomatic) To bother, disturb, irritate, or annoy.
It's a small thing, really, to leave towels unfolded, but it rubs him the wrong way.