I've heard various ideas such as limiting it only after mods have expended their points (this will require implementing a cooldown to prevent a user from getting points again too soon). I want to hear your feedback, and I'll roll together something for the next major update of the site. Leave your comments
I don't think you can automate it, as all four +1 might be full of shit, while the single -1 might have a clue. You just can't know.
I tried metamodding for a while, and could not see the point of it. A 'peer review' system where peer means 'anyone who wanders past often enough' will be by its nature unfair, and since the 'fix' (metamod) didn't seem to actively do anything... why bother? so I quit doing it.