Sure, there are other things that matter: the work must be grammatically-correct, compelling, original, strong. But even so, if an agent/publisher doesn't handle your type of genre, then those things are suddenly and utterly moot.
It's like trying on a pair of jeans. Sometimes it takes dozens of "try-ons" before we KNOW - "This is the one! A perfect fit!" But here's the thing -- the ones we discard in the process might have been so close, but in the end, just weren't right. It doesn't even mean they were inferior. It might only mean they weren't THE ONE.
In some ways, this analogy is comforting. The notion that it's not the jeans' "fault" for not being selected. That it's more about the "fit," that it's out of the jeans' control, whether the fit is right or not. And -- that there's always the hope that if a pair of jeans is discarded on the shelf by one person, that the next person who picks them up to try them on could finally be "the one!"
I think if we see the publishing process in this light, we're a lot less likely to take things personally...