The meat of the idea seems to be discussed here.
The biggest thing I was worried about was a lack of legal understanding on the part of participants in the system, but they seem to have that pretty well addressed. Basically, their heart is in the right place, but they think that people will do this out of the kindness of their hearts and devotion to the betterment of society. Or, essentially, for recognition.
While there are people like that out there, I have a feeling that when it comes time to put up or shut up, there just won't be enough people to sustain the system. The truth is, patents are quite boring, but they're also quite lucrative, which is why there are so many people actually involved.
Another worry I had is the amount of time that such a system would require, both on the part of the peers and the examiners. Things like, "Those who submitted will be sent an email requesting their participation over a 4-week follow-on period in an expert panel.", "Once convened, this online panel will have one month in which to discuss the patent application.", and "In every case, the participant should receive feedback from the examiner about the final results of the process." seem to lengthen the examination process, not expedite it. It seems that this peer process is highly concerned with quality, but is willing to sacrifice timeliness. One suggestion was to start the process prior to when the examiner takes up the case. That really seems like the only practical solution, though I doubt there'd be any time for consultation after or during examination.
At any rate, it'll be interesting to see what will happen. Generally, I think the submission of relevant prior art by an expert public is a great idea, but in practice really won't happen all that often. First, there needs to be a large, dedicated community. Second, that community has to be as broad as it is deep, with regards to the subject matter. Thirdly, that community has to be willing to deal with learning not only patent law, but also the peer review system.
I think what will actually happen is that certain computer technology savvy individuals will at least be initially interested in computer technology fields of patent endeavor. A few other subject areas will also see some interest, but probably won't reach the level of peer review desired for discussion. Some companies/inventors will receive a disproportionate amount of attention. A lot of irrelevant, unranked art will be submitted. Eventually people will mostly give up on trying to affect the patent system as a whole, but importantly, sometimes some application will spark some public interest and a system may be in place for interested members of the public to affect its prosecution.