Bullshit detectors, detect yourselves

Evgeny Morozov defines “solutionism” as an “intellectual pathology that recognizes problems as problems based on just one criterion: whether they are ‘solvable’ with a nice and clean technological solution at our disposal.” What happens when you apply solutionism to a somewhat contrived problem of fake news (that even danah boyd labels “so-called”)? Well, there’s an app for that. In this case, it’s BS Detector (by Self Agency — cool name, bro).

As if neither Miguel de Cervantes nor Pierre Menard ever existed, this noble knight of an app boldly attacks “the proliferation of fake news”, to great amusement of onlookers.

BS Detector

That this app somehow manages to put “metaphysics” (metaphysics, Carl! Karl!) under “Junk science” rubric but yet makes no attempt to flag philosophy sites only means that the authors have no clue what they are talking about. That alone should be enough, but there’s more.

The app authors also feel the need for including a category such as “Satire: Sources that provide humorous commentary on current events in the form of fake news”. When you need an app to discern satire, well, what can you say about both the developers and the audience? (I suppose one can argue Poe’s law in their defense, but… Nevermore.)

Finally, BS Detector disavows any responsibility (“The B.S. Detector is powered by OpenSources[…]Neither the B.S. Detector nor the Self Agency LLC assume liability for the accuracy of OpenSources’ data”).

BS Detector became what it purported to fight even before it started
the fight. Must be a record. But now — just what is “OpenSources”?

OpenSources

There are so many problems with this, the only way to deal with them is with bullets:

• It is claimed to be a database of online sources “maintained by
  professionals who have analyzed each source”. Who are these professionals, and what their credentials are, is not immediately clear. (Other than that they’re “headed by one Melissa Zimdars (@mishmz) of Merrimack College”). Hell, it’s not clear at all. It’s not even clear what their professions are. They could be professional locksmiths or podiatrists or chefs. Given that the site lives on Github, it is more likely that they are professional computer programmers — solutionists, in other words.
• There’s a chat one can use to interact with these mysterious professionals, but it is largely pseudonymous. Browsing through the repository contribution history, one sees most commits from a BigMcLargeHuge (a bigly tremendous handle). Who is, well… a computer programmer.
• This, of course, does not just raise the question of “quis custodiet…”. It also begs the question. And then there’s the third question: why would one delegate the diligence of checking unknown sources to some other unknown source? By adding this extra layer transparency is reduced, and who knows what these “professionals” will decide tomorrow.
• Indeed, looking at the chat gives one pause — one is liable to encounter chilling things like “Is anyone working on anything that would bake-in verifying a source into the browser, sort of like SSL”. Nothing can go wrong there. Nothing.
• Implementation-wise, trying to enumerate all possible domains seems even more hopeless than chopping Hydra’s heads off. Especially now that we are not restricted to just a few TLDs anymore — look, even the BS Detector app has a .tech (and its author, an .agency) top-level domains.
• It is from here that BS Detector critically uncritically took the classification of “metaphysics” as “junk science”. It is decided here that “satire” needs to be labeled. (Decidedly humorless bunch, these “professionals” are, no?) BS Detector is just a front.
• Now as for methodology. Generally most of it is reasonable (and even repeated by NPR). Reasonable and simple — so simple that an automation of it is a solution in search of a problem — unless, perhaps, the goal is for Zimdars et al to insinuate themselves as arbiters of what is true and what is not, the proverbial custodes. No, thanks.
• MSM definitely is guilty, several times a day, of some sins listed in the methodology (forget even about Gell-Mann amnesia). It’s just that burying the lede has more gravitas than “click-bait”, while ultimately is the same thing, isn’t it?
• The list of “credible” sites does not even reach 20. Srsly? And Slate made it. Slate, Karl! Neither AP nor Reuters did. Lolwhut?
• Wikileaks.org is in the unreliable list, with a comment “analyzing this info is likely beyond the scope of our project at this time”. This is despite Wikileaks having published cryptographically verifiable information. Dr.Zimdars, maybe enlist some, I don’t know, people dealing with proofs? Since you’re on the “truth” crusade and all…
• The rest is sheer sloppiness. Much lulz can be had from the sheer absurdity of the “unreliable list”. Some dead sites, The Onion, some known partisan sites like DailyWire all thrown into the same pile with no method whatsoever. Blog of a reputable (albeit ideological — but openly so) Cato Institute somehow makes it into the unreliable list with the comment of “credible”. This is all straight out of “you had one job” Tumblr (is that fake?)

To summarize, on the subject of BS Detector, Open Sources and other such endeavors, bullshit — that’s how I feel. Total. Fucking. Bullshit.