I've heard various ideas such as limiting it only after mods have expended their points (this will require implementing a cooldown to prevent a user from getting points again too soon). I want to hear your feedback, and I'll roll together something for the next major update of the site. Leave your comments
Looking at my post I'm not sure those numbers line up, but you know what I'm getting at. I think 1-2 point downward point slots is good for filtering purposes. Probably one point less for posting anonymously, and the lowest level for truly troll posts.