There we three attempts to add a 5th track to the conference to accommodate more of the talks. We wanted to accept many more talks than we did; however, we have a big enough event on our hands now for a first one. Then we wrapped up with a vote on the conference as a whole. The question was, “Did anyone feel that any one talk would detract from PyTennessee being a great conference?” After that passed, we have the list you see today
In the first round of meetings, we went through each proposal individually. The goal here was simple to determine if a given talk – reviewed in isolation – is potentially good enough for PyTN. That is, in the first round we reviewed talks strictly on their merits and avoid holding them up against similar talks (we do that next, in Thunderdome).
We reviewed talks, one at a time. We gave a couple minutes for the champions (identified by the website votes) to make an argument for the talk. Then, a few minutes of general debate. Finally, we voted yay/nay on each talk. Majority rules - a majority of “yay” votes will indicated that the talk was accepted – it moves onto Thunderdome. A majority of “nay” votes indicated that the talk was rejected – it’s out. The chair will break ties.
After round one has ended, we’re left with a set of talks that are all “good enough” for PyTN, and so we pit similar talks head-to-head and try to select the best.
In each “round” of Thunderdome we reviewed 3-5 talks, roughly grouped by theme/content/topic/etc. Out of each group we’ll got one or more winners, possibly some damaged talks, and some talks that are out. We did this by discussing, and then the committee voted for the talks they liked. Members voted for any number of talks (including none or all of them).
The winners were the talks from the group that absolutely should be at PyTN. It advanced and was safe from future cuts. It’s on the program! The winners were talks that had a 75% supermajority.
Damaged talks were ones that had been voted down by Thunderdome but aren’t quite out of the running yet. There may not be any damaged talks in a given group, or all talks may end up damaged. Damaged talks are any talks that didn’t win, but did receive votes from at least 50% of the committee. Talks that were selected by fewer than 50% of the committee in attendance are out – they will not appear at PyTN.
We then allowed a champion to speak for each damaged talk, and allowed the rest of the program committee to respond. We then took another vote. This process continued until all of those talks were refined down to the list you see on the schedule, and a set of backup talks.
There we three attempts to add a 5th track to the conference to accommodate more of the talks. We wanted to accept many more talks than we did; however, we have a big enough event on our hands now for a first one.
Then we wrapped up with a vote on the conference as a whole. The question was, “Did anyone feel that any one talk would detract from PyTennessee being a great conference?” After that passed, we had the list that appeared at the conference.
We attempted to do a fair and honest review of the talks, did we do a perfect job… No. Program Committee members were not allowed to submit talks, share talk reviews, or beseech outside opinions. During Kittendome and Thunderdome people abstained from voting for coworkers, bosses, close friends, etc. In my case, that meant I cast less than 5 votes the entire dome process for individual talks. :( I’m certain people could find flaws in our process, or point out some sort of bias.
###I can be bought with Cookies, and that’s really no secret.
So what kind of drama did we have?
Well first, we were clear from the onset that we wanted to be a different conference. We picked a diverse group of talks, and didn’t limit ourselves to just python or even code. In fact, two of the highest vote getting talks were on Systematic Bias and Mental Health. Those were contentious in the organizers group, but the amount of “not python” was a shock to people. I got emails calling me stupid, saying I destroy the conference, and that I was doing Python a disservice.
Secondly, we choose to allow presenters to have multiple talks. During the domes, we didn’t pay attention to how many times a speaker names appeared in our selected talks other than to decide if we could vote or not due to some potential bias. After the committee members were done selecting talks, they noticed that a few speakers had multiple talks chosen. A discussion then occurred, and the decision was made to say these were the talks chosen, they are the talks that should be presented. Unfortunately, many of these people were friends or close to a few of the organizers. This also caused several emails accusing us of playing favorite or placating big name presenters, etc.
Finally, A few people we turned down sent nasty emails about how well known they were, or how great a presenter they are and how stupid I was for turning them down. In a few cases the submitted proposal was one line about the content and several lines about past presentations. As you can imagine the committee voted that talk down over one with a nice full proposal. I still got told I had no business running a conference, I obviously knew nothing about development, and that “my talk would have been selected at” conference X.
These three days lead to a massive cry!!! Thanks to strong support from the committee and a few members of our local community I made it through. I knew we had a great set of talks selected, and I knew that it was done as fair as possible. And a week later the drama had died down and people began to get excited about the talk selection and were tweeting about it.
And now looking back after the conference. It was totally worth all the drama. We had an overwhelmingly positive response at the event. I and the rest of the committee were ecstatic with the results.