How the Million Dollar Hackathon Went, and Continues, South
It’s almost laughable at this point. Either we are looking at complete fraud or utter, impressive incompetence. Salesforce has chosen to attempt to rectify its problem by giving the second place A MILLION DOLLARS.
Citing that a portfolio investment doesn’t constitute a related company (contrary to this investor words definition), the top brass at Salesforce thinks the only thing better than one ineligible winner is TWO!
Whoever is running damage control there thinks that if they aim the plane at the ground, they can bust through to the other side.
Where the developers don’t think the company is kicking them in the groin.
[UPDATE] BREAKING NEWS. Second place comes from Salesforce INVESTED company. How dirty was this competition?
Neither you nor any of your team members are an employee ofsalesforce.com or its related companies as of September 1, 2013 or during the Hackathon; nor an immediate family member (parent, sibling, spouse, child) of or household member to an employee.
I’ve been working hard to try to collect more than 13 of the 144 other entries to a list. But should I be considering that there weren’t more than 13 valid entries to begin with ?
Kudos to all of the journalists following this story. That is, all but @alexwilliams. While a judge and a journalist, he doesn’t seem to be interested in applying his skills to the farce in which he participated, intentionally or not.
[UPDATE] I’ve been triangulated. It took longer than expected — I wasn’t trying to hide it; I used the same name — but someone from Salesforce did identify many of the posts in various forums as me. If Salesforce really believes that this is an improbable series of poor decisions and not malicious intent, reach out to me to discuss how to start building the redemption story. Company and contestants together. It is the holidays and I’d like to watch a Charlie Brown Christmas less jaded.
Even though I’m Jewish.
It’s not surprising that controversy errupts as competition prizes grow. I’m sure right about now some executive is regretting their decision not to have “just given away a f*#cking iPad”. But here we are.
Googling Salesforce and Hackathon now just produces a page of scandal and controversy (probably not marketing’s first goal). The first place entry looks like they knowingly violated the rules, possibly with the permission of the company. And most of the contestants - developers whom the company wanted to lure into their ecosystem - now rightfully feel like they were seat fillers at the Oscars, their hard work never even seen or tested.
At this point in the cartoon, Wile E. Coyote is off the cliff with a sign that reads, “Yikes!”.
I’ve been looking into what happened ever since the wonderful paragraph’s long consolation email that was the digital equivalent of, “Thanks for all your hard work. Goodbye”. Not the classiest, but I accepted it. You don’t win every contest and this one had lots of competition. Then I saw the analytics of my app (that is now apparently the contest legal version of a healthcare.gov rework) during the review period. No usage. Then multiple reports of contestants’ apps not getting used when I inquired. And then, finally, the first post that the winning entry had been demoed weeks before the contest ever began.
If I disliked Bowties on hipsters to begin with, I downright hated them now.
So here’s a rundown of how this thing has gone south, as far as I can tell. An active investigation performed by Encyclopedia Brown. I’m not going to lead with the most obvious: that Upshot’s (and now Healthcare.Love’s) entry violates the rules. That’s so demonstrable by now it seems boring.
1. WHO ARE THE CONTESTANTS?
That seems like an easy one to piece together. But trust me. It’s not. Traditionally in these kinds of events there’s usually a submission gallery of entries and contact information. It’s done so that contestants can share in what was one of the founding laudable goals of a Hackathon: share ideas and let people see what was constructed.
Salesforce, for reasons only known to themselves, is still choosing to keep the submission gallery filtered to only five contestants — the five finalists.
For a brief period of time, one user [CITE] reported that after the entry deadline passed on Wednesday the site showed the full compliment. 149. It’s a number far shy of the 6000 they stated was the limit of entries. And it’s possible they aren’t showing the gallery out of embarrassment. But they are robbing contestants of one of the fundamental goals of a Hackathon. Not everyone can win, so camaraderie is a baseline expectation. You get to see who did what. And talk to them about it. You bond. You let the loss fade away.
Who are the contestants? I can’t say for sure. But from a brief polling of people through the boards they have come not just out of state, but out of country as well. They are in most cases professional, working developers. And perhaps that’s why salesforce is in so much trouble now. Next time, perhaps Salesforce should target a high school science competition. If the outcome is a spectacle, there’s always this week’s Vampire pic to distract. But some of these contestants have lawyers and bought plane tickets. Ouch.
BTW, I’m still trying to piece together the contestant picture for many reasons. So if you participated, add your entry to Simone’s list here or just add your info to this article and I’ll add it in.
2. THEY BAITED PROFESSIONALS INTO DEVELOPING FUNCTIONAL APPS THAT WOULDN’T BE TESTED
When marketing makes statements like
Last, but definitely not least, start coding! That’s the fun part. That’s what you do. That’s what you love. But it’s not going to be easy—$1 million is going to bring out the best of the best. So don’t wait until Dreamforce, you’re going to want to get started now
It seems like a pretty obvious expectation that the reason they want a working app is so they can test a working app. That is the reasonable reading of their own Official Rules. They even went so far as to require you to provide the source code.
However, it started emerging after the announcement that many, if not most, of the applications did not get tested. Participants would have been better served coming up with a glossy logo and making a non functional wireframe to get to the final round of judging.
I’ve seen that happen in other Hackathons. I myself started to re-orient my approach to presentation in these things. But that’s on a four hour hack. Or a weekend hack (where it’s really tough to get fully functional). That is NOT what people were led to believe by the company’s baiting statements.
There are even people who have asserted that their video submissions did not receive views. That’s insult to injury. Is a hackathon supposed to reward the caption or the work? If your intention is to lure developers, I would assume the latter. But if you just wanted the sales folks, well… I guess you did call yourself salesforce.
Again, we’re trying to put together a complete picture of the analytics so that there can be a review of what got seen and what didn’t. So PLEASE add your app to the list or to the google spreadsheet here
3. SECRET JUDGING
I’ve been trying to get visibility into the judging process. Adam’s response seems to assert that a team of 80 experts were on the problem. But if it does turn out that ONLY 149 entries exist, that would be 4 minutes to review in parallel if all they did was look at the 2 minute videos. And 8 minutes if they had each person look at two different entries? That is quite the long task to have empathy for. I think that’s the length of Led Zeppelin’s Kashmir.
I spoke to some of the developer relations people during the Hackathon. There was supposed to be a Chinese wall between the helping staff and the judges (possibly referenced as being in another timezone). So does that 80 number include the staff that was at the Hackathon? Is it only the judges?
We don’t know, but applying the judging criteria
Innovation: Is it an amazing new scenario?
Business Value: Does the application create value? Does it save money, create opportunity, connect customers, partners, and employees?
User Experience: Does the app have a user interface that is gorgeous, easy to use, and fast?
Use of Salesforce Platform: Does the application take advantage of the features and capabilities of the Salesforce Platform?
It’s hard to justify why these things weren’t launched at least. Was there a deadline to get all testing accomplished within a half hour? Were you boiling water that needed to be tended to?
It’s been as difficult to penetrate their judging process as to figure out what happens in a FISA court. This is something that if they rejudged in the open wouldn’t be a problem. But this is the inherent problem of closed door proceedings, especially ones that seem unwarranted. And this is yet another reason they are taking heat.
4. THE WINNER VIOLATED THE RULES, OR WAS ALLOWED TO VIOLATE
Now we’ve gotten past the filthyness of the darkness and into the light. Right? Well, wrong.
Set aside that Thom Kim worked for Salesforce and was allowed in to the competition. The winner, Upshot, had already demonstrated their technology that they had been working on for a long time here in violation of the Official Rules here.
The real nastiness on behalf of salesforce is that it is alledged here that a call took place with premiere partners that granted them access to enter the competition with pre-existing work. Supposedly, however, the judging panel would have been directed to look at the work on the merit of the new contributions.
Let’s set aside for a second the impossibility of this task (if I showed you a picture of a Tesla and a picture of a Datsun and asked you which camera shot the better image, what would you come up with? the Datsun camera?) If those directions were followed, then that appears to reduce Upshot to: baby learned to use an iframe (first hand account from Oct 8th Meetup).
Now, while an iframe was a worthy contribution in the late 90's, I still think even then it wasn’t a million dollar entry. So you have to wonder if those instructions were passed on to the judging panels. Was it an oversight? Or was it an intentional “look the other way”. We’d have to know the instructions to this FISA court to find out.
The easiest move by Salesforce at this point would be to cut and run on Upshot. However, this would place them in another precarious situation. They get it from Upshot for renegging on their deal. And they reveal the deal itself. Which comes to
5. MODIFICATION OF THE OFFICIAL ENTRY RULES
The Official Entry rules themselves state that in the section:
WHAT OTHER CONDITIONS AM I AGREEING TO BY ENTERING AND PARTICIPATING
We reserve the right to modify these Official Rules at our sole discretion
Except here’s the problem. The alleged rule ammendment comes in a backwater discussion forum (or on a secret call ;). Not exactly an obviously stated common reference point for the contestants (why not where the official rules sit?).
Are we to expect that if Salesforce determined that contestants are required to turn over their intellectual property or children to the contest in a hidden url that this is something we are magically bound to?
At the end of the day, you did reserve the right to modify, but you didn’t modify the document that probably is required. And even if that were the case, is it correct (or legal) to modify the rules after participants already were well into development? If they had this knowledge would they have participated against applications that had traction and time? By modifying the rules mid stream, do you also require re-agreement? Do you invalidate other parts of the agreement, like liability protection?
[UPDATE] More on this to come. It looks like this may indeed be illegal. They changed the nature of a contest mid contest. And without reasonable notification.
THE UPSHOT :)
This thing has gone south in just about every way it could. Much of it foreseeable. Even if you strip Upshot of the title, who do you give the prize to? The Healthcare.gov app that you put in the 5? Or the Healthcare.gov app that was one of the MANY you didn’t test? (and, btw, if part of the point was to address the folks who needed to be brought into the healthcare marketplace, making an assumption the target audience all operate iPhones is, um, maybe not correct).
It seems like this all was inevitable. Companies running hackathons and treating contests with loose rules that favor partners would eventually get here. And Salesforce is the first to eat it. But this is a bed they made themselves by creating expectations that they possibly had no intention of living up to.
The only way to prevent this damage from sliding into the brand is to accept that the entire thing has to be rejudged openly under a set of conditions formulated with and agreed to by contestants — including those you baited in from out of state and out of country.
In Adam’s statement, they are already looking forward to next year’s hackathon. But are they really expecting that without mea culpa they will have the trust of any developers? And, even if so, will the city of San Francisco knowingly allow another set of its citizens to get duped.
The clock is ticking and this story is going to jump media very soon. Google salesforce and hackathon and it’s a full page of scandal and controversy. By the time it jumps you’ll want to be looking like your working with the contestants and not against them.
Or the term, “I got salesforced”, will stick.