In Melville’s ‘Bartleby, The Scrivener’, the eponymous protagonist has a standard response to all requests from his boss : "I would prefer not to."
As uncooperative responses go, this is hard to beat. It's not exactly an out and out refusal, but an expression of preference. Had Bartleby been subjected to a disciplinary procedure, a half-decent union rep should have been able to have the case dismissed almost immediately.
Even if it's not technically a refusal, the outcome is the same: the task that has been requested remains not done.
But there's more to it than that. A response such as "No, I'm not doing that" is confrontational and aggressive, and invites a robust response. The Bartleby response, on the other hand, is more likely to result in bafflement: "Did he really just refuse to do what I, his boss, has asked him to?"
The lame pause that such rumination involves is, albeit momentary, enough to put the boss in what Stephen Potter would have called a “one-down position".
(My favourite of Potters’ suggested responses or interjections is intended to stop the conversation dead, to silence a speaker, rather than to express a refusal to do something. Known as the Canterbury Block, "But not in the South" is extremely effective, but not applicable in the current context.)
A more aggressive expression of non-co-operation is to be found in Eric Frank Russell's short story, And then there were None. The inhabitants of the planet about to be colonised meet virtually every enquiry with the response "Myob". It's an acronym: mind your own business. Effective enough but, as I say, quite confrontational.
The beauty of the Bartleby response and its variations ("I would prefer not to say", "I would prefer not to go" and so on) is that it's deflationary rather than inflammatory. It has a somewhat depressing effect and, being unexpected, takes the wind out of the requester's sails. A response to the question, "How are you," of "I would prefer not to say " is virtually certain to prevent any further discussion or indeed any further enquiry. As such, it is perfect.
I like how you've correctly characterized "I would prefer not to" as non-confrontational. It's one of those delicious passive-aggressive turns of phrases, like Francis Urquhart's "You might think that, but I couldn't possibly comment."