I don't know that they should be banished altogether. But I do think they should used incredibly (<---ha!) sparingly. Tightening one's writing should always include examining the overuse of adverbs (and adjectives).
Here's a great Writer's Digest article by William Noble that explains why this is true, and gives clear examples.
I'm sure I'm still guilty of using too many adjectives/adverbs in my writing (I do believe that, at least in women's fiction, writers can get away with them a bit more), but now I'm more aware of the ones I don't need, and I think/hope my writing is better for it!