I'd also say keep it open. In almost every case, spammers are only hurting themselves with spam on sites like this, so it's not like you're enabling them in any way.
If you have the resources to implement it, you could have a tag-as-spam option and let the users handle it. (That is the 2.0 way, right? 🙂 Anything which gets two spam tags gets hidden, and you can review it at your leisure to see if anything was mis-tagged.