I've heard various ideas such as limiting it only after mods have expended their points (this will require implementing a cooldown to prevent a user from getting points again too soon). I want to hear your feedback, and I'll roll together something for the next major update of the site. Leave your comments
When the site first launched, the value was set at 24 hours. The problem was that we got stuck with people hoarding points, and the new algo needs points going in/out (it tries to maintain X points at all times, so its dependent on rapid expiration, or people using points). Dropping it down to 4 was an emergency fix until I could get around to revising the mod algo.
Unfortunately, when I wrote it, I had no hard numbers to work again, and its working on some faulty assumptions. I keep putting rewrite on the list, but its not SUPER high priority.