Avatar

1st SAPO Data Challenge – Final Results

20 Junho 2011
Sem comentários

The First Data Challenge

First of all, we wish to thanks all challengers for their effort. We hope that this has been a good opportunity for working with real data (real data is nasty!) and to learn more about this type of prediction problems.

We know that the problem was relatively hard: it was more than simply predicting missing points from a single time-series, given a few others data points from that time-series.  In fact, the challenge was to predict several complete time-series based on the behavior of other series.

Twenty six participants hailing from nine countries came forward to face this 1st SAPO Data Challenge. Seven of our challengers provided their mighty predictions. We evaluated the results according to two metrics:

  • Cumulative Absolute Error
  • Cumulative Relative Error

And we computed three rankings, one for each of these metrics, and another combining the results of both. Here are the results.

1. Ranking based on Cumulative Absolute Error

RunCAE
1. nick_ruiz_3.tsv102534
2. nick_ruiz_2.tsv102554
3. nick_ruiz_1.tsv103504
4. diogo_costa_3.tsv104152
5. diogo_costa_2.tsv104926
6. jose_santos_1.tsv107211
7. diogo_costa_1.tsv109789
8. luis_marujo_3.tsv111236
9. luis_marujo_1.tsv111431
10. filipe_rodrigues_3.tsv112272
11. filipe_rodrigues_1.tsv113110
12. filipe_rodrigues_2.tsv116703
13. fabio_pedrosa_rr.tsv127600
14. jose_santos_3.tsv135568
15 .jose_santos_2.tsv138940
16. fabio_pedrosa_rs_1.tsv153101
17. fabio_pedrosa_rs_2.tsv153101
18. joao_oliveirinha_1.tsv160905
19. luis_marujo_2.tsv165364
20. joao_oliveirinha_2.tsv167264
21. joao_oliveirinha_3.tsv169048

 

2. Ranking based on Cumulative Relative Error

RunCRE
1. diogo_costa_1.tsv547
2. diogo_costa_2.tsv5688
3. nick_ruiz_1.tsv5741
4. nick_ruiz_3.tsv6703
5. nick_ruiz_2.tsv6704
6. diogo_costa_3.tsv7913
7. jose_santos_1.tsv8595
8. luis_marujo_3.tsv8739
9. luis_marujo_1.tsv9038
10. filipe_rodrigues_2.tsv14087
11. filipe_rodrigues_1.tsv15685
12. filipe_rodrigues_3.tsv15852
13. fabio_pedrosa_rr.tsv18494
14. jose_santos_3.tsv22196
15. jose_santos_2.tsv31148
16. joao_oliveirinha_2.tsv34729
17. joao_oliveirinha_3.tsv35517
18. joao_oliveirinha_1.tsv36808
19. fabio_pedrosa_rs_1.tsv37627
20. fabio_pedrosa_rs_2.tsv37627
21. luis_marujo_2.tsv39758

3. Combined Ranking

 

RunRanking(CAE) + Ranking(CRE)
1. nick_ruiz_3.tsv5
2. nick_ruiz_1.tsv6
3. diogo_costa_2.tsv7
4. nick_ruiz_2.tsv7
5. diogo_costa_1.tsv8
6. diogo_costa_3.tsv10
7. jose_santos_1.tsv13
8. luis_marujo_3.tsv16
9. luis_marujo_1.tsv18
10. filipe_rodrigues_2.tsv22
11. filipe_rodrigues_1.tsv22
12. filipe_rodrigues_3.tsv22
13. fabio_pedrosa_rr.tsv26
14. jose_santos_3.tsv28
15. jose_santos_2.tsv30
16. fabio_pedrosa_rs_1.tsv35
17. joao_oliveirinha_2.tsv36
18. joao_oliveirinha_1.tsv36
19. fabio_pedrosa_rs_2.tsv37
20. joao_oliveirinha_3.tsv38
21. luis_marujo_2.tsv40

 

The Winners *

Winners are found by subsequently choosing the best ranked Challengers from each rank, by the following order:

  1. Best Challenger in the Cumulative Absolute Error Ranking
  2. Best Challenger in the Relative Absolute Error Ranking
  3. Best Challenger in the Combined Ranking

All the runs of a winner of a given ranking will not considered when choosing more winners from subsequent rankings. Therefore, there will be three different winners in this 1st Sapo Data Challenge. And the winners are….

  1. Nick Ruiz
  2. Diogo Costa
  3. José Santos

Congratulations to the Winners!

As you know the prize will be a free registration in the Lisbon Machine Learning Summer School – http://lxmls.it.pt/ – from July 20th to 25th.

 

 



Sem comentários