Potential Neg argument

The current resolution is: In a democratic society, felons ought to retain the right to vote.

For neg, the only thing I can think of so far is the social contract; felons didn't uphold their end, so the state doesn't have to uphold its own. However, when I was looking at Mr. Anderson's definitions of democracy, I noticed that most of them say that the people wield supreme power through elected officials or directly.

So here's my line of thought: Initiatives are either voted on by people, or by officials elected by people. So the "supreme power" of the people is basically wielded vicariously. For an individual, that supreme power would be represented by voting. So by allowing felons currently serving time to vote, you've given supreme power to someone whose actions indicate that they will act against the best interests of the society. Is that the kind of person we want to give supreme power to?

Does this sound like a reasonable argument? Or does it sound like basically a re-wording of the social contract argument?