Sunday, May 11, 2014


This is part III of an article about the PGSC score metric to quantify Idol placement:

*Part I contains the ranks of the Idol contestants on a season-by-season basis, based on the PGSC score. Think of this as the way the contestants should be ranked, assuming neutral ground and zero pimpage. It is also the way I would rank them.
*Part II ranks the Idol contestants based on PGSC score, according to genre.
*Part III, which is this article, attempts to use historical rank data since AI3 to put actual rank numbers into contestants, based on PGSC score, and based on the features added in at Part II. Unlike Part I, this is how we predict the audience would vote based on these features, and this model gets more informed as we get more data points (more seasons).

As explained in the title above, from part II of my study I utilized the four features I implemented for the PGSC score, and for greater accuracy of specific trends within the PGSC score, I used the 8-degree polynomial function  for each of those features, and took averages of the four. Remember, the smaller the number, the better!--that means you're ranked higher. The average rank corresponds to the average rank within the top ten--an average of 2.54, as was Scotty McCreery's case, means the model is predicting an outcome between 2nd and 3rd place for that particular contestant, or if that contestant is the highest rank of that season, as was in McCreery's case, then he/she is the predicted winner of the season. There are certain errors for sure, but it attempts to create a model using prior knowledge of past seasons to update the current knowledge on how to peg contestant placement. It's not Bayesian, but it's informed to a certain degree.

There are some quibbles--we can ignore Curtis Finch's high scores, because he has no growth so his numbers are skewed. For all intents and purposes, he doesn't count.

Alex Preston has an enormously high rank based on his geography--the reality is, the model hasn't seen anyone from the Northeast have a PGSC score within his range, so it is uninformed. I considered using a logarithmic adjustment which will produce a fairer score for Preston here, though. From this link, a quick extrapolation using the logarithmic adjustment suggests that Alex's score due to geography is about 5.98 or thereabouts, but it is far less accurate. There is a general consensus that the order is Jena, Caleb and then Alex in that order, and that's what my algorithm predicts. But Alex is really docked off based on geography and the very few data points the model has of him within that PGSC inset--otherwise, his other numbers, particularly his singer-songwriter genre, is something that is ABSOLUTELY winning material. I think it's closer than the model thinks, but I think the general order is correct.

My informed model has the ranks generally like this; as you can see below, it's predicted the correct top 4 of AI10, had three of the top five in AI11, had four of the top five in AI12, had three of the top five in AI13, had three of the top five in AI3 as well as the right top two, had three of the top five in AI4 as well as the right top two, had four of the top five in AI5, was almost nearly perfect in relative order in AI6, had the correct top two in AI7, had four of the top five in AI8, and had four of the top five in AI9 and the correct final three. Below is the results of that:

1 Scotty McCreery 2.539981 AI10
2 Lauren Alaina 2.90495 AI10
3 Haley Reinhart 3.508148 AI10
4 James Durbin 3.922791 AI10
5 Casey Abrams 4.941457 AI10
6 Jacob Lusk 6.909581 AI10
7 Stefano Langone 7.03343 AI10
8 Paul McDonald 7.044193 AI10
9 Pia Toscano 7.118285 AI10
10 Naima Adedapo 10.11886 AI10
1 Hollie Cavanagh 2.80847 AI11
2 Phillip Phillips 2.825383 AI11
3 Skylar Laine 5.119702 AI11
4 DeAndre Brackensick 5.632362 AI11
5 Elise Testone 5.827453 AI11
6 Joshua Ledet 6.1286 AI11
7 Jessica Sanchez 6.350618 AI11
8 Erika Van Pelt 6.648269 AI11
9 Colton Dixon 7.10468 AI11
10 Heejun Han 7.511101 AI11
1 Angie Miller 3.092654 AI12
2 Janelle Arthur 3.104103 AI12
3 Kree Harrison 3.463586 AI12
4 Candice Glover 3.650698 AI12
5 Burnell Taylor 4.941735 AI12
6 Amber Holcomb 6.307405 AI12
7 Lazaro Arbos 6.751065 AI12
8 Devin Velez 6.836016 AI12
9 Paul Jolley 7.014493 AI12
10 Curtis Finch 77.59738 AI12
1 Jessica Meuse 3.641191 AI13
2 Jena Irene 3.774365 AI13
3 Caleb Johnson 3.841272 AI13
4 Sam Woolf 6.359719 AI13
5 Majesty Rose 6.388388 AI13
6 Malaya Watson 6.516274 AI13
7 Dexter Roberts 6.718637 AI13
8 CJ Harris 6.78554 AI13
9 Alex Preston 7.053049 AI13
10 MK Nobilette 9.139962 AI13
1 Fantasia Barrino 2.775265 AI3
2 Diana DeGarmo 2.805256 AI3
3 Jennifer Hudson 3.761465 AI3
4 George Huff 4.333605 AI3
5 Amy Adams 4.55241 AI3
6 LaToya London 4.878473 AI3
7 John Stevens 6.409483 AI3
8 Jasmine Trias 6.461608 AI3
9 Jon Peter Lewis 6.883238 AI3
10 Camile Velasco 8.982225 AI3
1 Carrie Underwood 3.638107 AI4
2 Bo Bice 4.361273 AI4
3 Nikko Smith 4.912391 AI4
4 Constantine Maroulis 5.372998 AI4
5 Scott Savol 5.705556 AI4
6 Jessica Sierra 6.16085 AI4
7 Anwar Robinson 6.267822 AI4
8 Anthony Fedorov 6.488099 AI4
9 Vonzell Solomon 6.511731 AI4
10 Nadia Turner 6.53735 AI4
1 Chris Daughtry 3.105895 AI5
2 Paris Bennett 4.068508 AI5
3 Bucky Covington 4.257597 AI5
4 Taylor Hicks 4.517381 AI5
5 Elliott Yamin 4.795046 AI5
6 Mandisa 6.23848 AI5
7 Kellie Pickler 6.41849 AI5
8 Katharine McPhee 6.608421 AI5
9 Ace Young 6.937166 AI5
10 Lisa Tucker 7.050085 AI5
1 Blake Lewis 2.86715 AI6
2 Jordin Sparks 3.915961 AI6
3 Melinda Doolittle 4.885022 AI6
4 Phil Stacey 5.12974 AI6
5 Chris Richardson 5.337774 AI6
6 LaKisha Jones 5.629854 AI6
7 Gina Glocksen 6.120664 AI6
8 Sanjaya Malakar 6.470392 AI6
9 Haley Scarnato 6.630615 AI6
10 Chris Sligh 7.779865 AI6
1 David Cook 2.777708 AI7
2 David Archuleta 4.774595 AI7
3 Kristy Lee Cook 5.07886 AI7
4 Chikezie 5.54108 AI7
5 Carly Smithson 5.98385 AI7
6 Jason Castro 6.108227 AI7
7 Syesha Mercado 6.212621 AI7
8 Michael Johns 6.513024 AI7
9 Brooke White 6.762671 AI7
10 Ramiele Malubay 9.361013 AI7
1 Adam Lambert 2.186894 AI8
2 Kris Allen 3.651781 AI8
3 Matt Giraud 3.95741 AI8
4 Allison Iraheta 4.558554 AI8
5 Anoop Desai 4.910723 AI8
6 Danny Gokey 6.332847 AI8
7 Lil Rounds 6.762913 AI8
8 Scott MacIntyre 7.042587 AI8
9 Megan Joy 8.765894 AI8
10 Michael Sarver 9.435499 AI8
1 Lee DeWyze 1.79351 AI9
2 Crystal Bowersox 4.169243 AI9
3 Casey James 4.372274 AI9
4 Siobhan Magnus 5.468904 AI9
5 Michael Lynche 6.082346 AI9
6 Aaron Kelly 6.168853 AI9
7 Tim Urban 6.429306 AI9
8 Andrew Garcia 6.695333 AI9
9 Didi Benami 6.743782 AI9
10 Katie Stevens 7.131676 AI9
The model is presented below, in data format. Play around with it!

No comments:

Post a Comment