I am a hermit fiction writer (as you say, most of us are) with the particular problem of not having anyone around who reads English. I have friends who are avid readers or successful writers of fiction in Spanish, but not one that I can show an English piece to for a considered opinion. I surfed a few writing sites, but was not happy with what I found and really hated to spend any more time looking. I preferred spending it writing! While I was writing my historical Taino Tales, I set up a private group on Facebook with the specific purpose of having beta readers for the Taino stories. I invited “friends” I knew were fans of fiction or English major-types! It was quite successful for me — three of my beta readeers were teachers, very careful readers and quite perceptive. I incorporated many of their suggestions. I dissolved the group after I completed the main tales: I had a feeling that I was imposing on them. I was consuming their time and they got little in return. I looked for beta reader groups, but couldn’t find any that I felt would be a good fit. (They seem to abound for romance, scifi and fantasy, but those are not my genres.)
What would I like to see? I like the nuts and bolts idea you have, but I would be passionate about some kind of beta reader arrangement that included reciprocity: I read and comment yours, if you read and comment mine. Critical reading of someone else’s fiction is a difficult and delicate matter. It takes time. I am not good at numbers, but someone might be able to devise a point system of some kind. There also would have to be some way to channel preferences. Fantasy writers might not want to critique police procedurals, for example. Many stories are perishable: a writer doesn’t have a lot of time to wait for a beta reader to get around to commenting. Sometimes there is a looming deadline — or simply the writer is desperate to finalize the story! An ideal system would include time limits for responses. I like that there is a way (I haven’t done it myself) to post a story privately.
Also: since you asked <g> I don’t think using the word Gypsy is a good idea. I suspect there are people who would find it offensive, just as calling a site “Indian Giver” or something like that could only bring trouble.
That said, sign me up!