Quintessa

Sports Rating Algorithm Predicts UEFA EURO 2020 Knock-out Stages

Quintessa mathematicians and scientists enjoy analysing data. Recently, Simon Rookyard presented predictions of the results in the group stage of the on-going European Football Championship, based on an algorithm developed within Quintessa. With the group stage of the competition now complete, it is possible to assess the performance of the algorithm so far, and present predictions for the remainder of the competition.

In a recent article we used our “N-Estimates” algorithm, created to rate sports teams, to predict the results of the 36 opening round matches of UEFA EURO 2020. Each of these predictions was accompanied by a graph to denote the most likely prediction (green) and the set of predictions within a 1-σ confidence interval (yellow). Figure 1 presents the EURO 2020 first round results again with the observed results highlighted.

11th Jun, 2021, Italy vs. Turkey. Central prediction: 2 - 0. Confidence range for the goal difference (Italy minus Turkey): 1 to 3. Actual Result: 3 - 0.12th Jun, 2021, Denmark vs. Finland. Central prediction: 3 - 0. Confidence range for the goal difference (Denmark minus Finland): 2 to 5. Actual Result: 0 - 1.12th Jun, 2021, Russia vs. Belgium. Central prediction: 0 - 3. Confidence range for the goal difference (Russia minus Belgium): -5 to -2. Actual Result: 0 - 3.12th Jun, 2021, Wales vs. Switzerland. Central prediction: 1 - 1. Confidence range for the goal difference (Wales minus Switzerland): -1 to 0. Actual Result: 1 - 1.13th Jun, 2021, Austria vs. North Macedonia. Central prediction: 1 - 0. Confidence range for the goal difference (Austria minus North Macedonia): -1 to 2. Actual Result: 3 - 1.13th Jun, 2021, England vs. Croatia. Central prediction: 4 - 1. Confidence range for the goal difference (England minus Croatia): 2 to 5. Actual Result: 1 - 0.13th Jun, 2021, Netherlands vs. Ukraine. Central prediction: 2 - 1. Confidence range for the goal difference (Netherlands minus Ukraine): 0 to 3. Actual Result: 3 - 2.14th Jun, 2021, Poland vs. Slovakia. Central prediction: 2 - 0. Confidence range for the goal difference (Poland minus Slovakia): 1 to 3. Actual Result: 1 - 2.14th Jun, 2021, Scotland vs. Czech Republic. Central prediction: 1 - 1. Confidence range for the goal difference (Scotland minus Czech Republic): -1 to 1. Actual Result: 0 - 2.14th Jun, 2021, Spain vs. Sweden. Central prediction: 4 - 1. Confidence range for the goal difference (Spain minus Sweden): 2 to 4. Actual Result: 0 - 0.15th Jun, 2021, Germany vs. France. Central prediction: 2 - 1. Confidence range for the goal difference (Germany minus France): -1 to 2. Actual Result: 0 - 1.15th Jun, 2021, Hungary vs. Portugal. Central prediction: 0 - 1. Confidence range for the goal difference (Hungary minus Portugal): -2 to 0. Actual Result: 0 - 3.16th Jun, 2021, Italy vs. Switzerland. Central prediction: 1 - 0. Confidence range for the goal difference (Italy minus Switzerland): 0 to 2. Actual Result: 3 - 0.16th Jun, 2021, Russia vs. Finland. Central prediction: 3 - 2. Confidence range for the goal difference (Russia minus Finland): -1 to 2. Actual Result: 1 - 0.16th Jun, 2021, Turkey vs. Wales. Central prediction: 0 - 0. Confidence range for the goal difference (Turkey minus Wales): -1 to 1. Actual Result: 0 - 2.17th Jun, 2021, Denmark vs. Belgium. Central prediction: 0 - 0. Confidence range for the goal difference (Denmark minus Belgium): -1 to 1. Actual Result: 1 - 2.17th Jun, 2021, Netherlands vs. Austria. Central prediction: 2 - 0. Confidence range for the goal difference (Netherlands minus Austria): 1 to 3. Actual Result: 2 - 0.17th Jun, 2021, Ukraine vs. North Macedonia. Central prediction: 1 - 0. Confidence range for the goal difference (Ukraine minus North Macedonia): 0 to 2. Actual Result: 2 - 1.18th Jun, 2021, Croatia vs. Czech Republic. Central prediction: 1 - 2. Confidence range for the goal difference (Croatia minus Czech Republic): -2 to 1. Actual Result: 1 - 1.18th Jun, 2021, England vs. Scotland. Central prediction: 4 - 0. Confidence range for the goal difference (England minus Scotland): 3 to 5. Actual Result: 0 - 0.18th Jun, 2021, Sweden vs. Slovakia. Central prediction: 1 - 0. Confidence range for the goal difference (Sweden minus Slovakia): 0 to 2. Actual Result: 1 - 0.19th Jun, 2021, Germany vs. Portugal. Central prediction: 2 - 1. Confidence range for the goal difference (Germany minus Portugal): -1 to 2. Actual Result: 4 - 2.19th Jun, 2021, Hungary vs. France. Central prediction: 0 - 1. Confidence range for the goal difference (Hungary minus France): -2 to 0. Actual Result: 1 - 1.19th Jun, 2021, Spain vs. Poland. Central prediction: 4 - 1. Confidence range for the goal difference (Spain minus Poland): 1 to 4. Actual Result: 1 - 1.20th Jun, 2021, Italy vs. Wales. Central prediction: 2 - 1. Confidence range for the goal difference (Italy minus Wales): 1 to 2. Actual Result: 1 - 0.20th Jun, 2021, Switzerland vs. Turkey. Central prediction: 1 - 1. Confidence range for the goal difference (Switzerland minus Turkey): -1 to 2. Actual Result: 3 - 1.21st Jun, 2021, Denmark vs. Russia. Central prediction: 4 - 0. Confidence range for the goal difference (Denmark minus Russia): 3 to 5. Actual Result: 4 - 1.21st Jun, 2021, Finland vs. Belgium. Central prediction: 0 - 4. Confidence range for the goal difference (Finland minus Belgium): -5 to -3. Actual Result: 0 - 2.21st Jun, 2021, Netherlands vs. North Macedonia. Central prediction: 3 - 0. Confidence range for the goal difference (Netherlands minus North Macedonia): 2 to 4. Actual Result: 3 - 0.21st Jun, 2021, Ukraine vs. Austria. Central prediction: 1 - 1. Confidence range for the goal difference (Ukraine minus Austria): -1 to 2. Actual Result: 0 - 1.22nd Jun, 2021, England vs. Czech Republic. Central prediction: 2 - 0. Confidence range for the goal difference (England minus Czech Republic): 1 to 4. Actual Result: 1 - 0.22nd Jun, 2021, Scotland vs. Croatia. Central prediction: 2 - 1. Confidence range for the goal difference (Scotland minus Croatia): 0 to 2. Actual Result: 1 - 3.23rd Jun, 2021, Germany vs. Hungary. Central prediction: 3 - 1. Confidence range for the goal difference (Germany minus Hungary): 1 to 3. Actual Result: 2 - 2.23rd Jun, 2021, Portugal vs. France. Central prediction: 0 - 0. Confidence range for the goal difference (Portugal minus France): -1 to 1. Actual Result: 2 - 2.23rd Jun, 2021, Spain vs. Slovakia. Central prediction: 6 - 0. Confidence range for the goal difference (Spain minus Slovakia): 4 to 7. Actual Result: 5 - 0.23rd Jun, 2021, Sweden vs. Poland. Central prediction: 0 - 0. Confidence range for the goal difference (Sweden minus Poland): -1 to 1. Actual Result: 3 - 2.
Figure 1. Predictions and Results from the First Round. The plots use a traffic-light scheme, with the central prediction in green, medium confidence results in amber, and low confidence results in red. The observed result is highlighted using a black cross-and-circle.

We consider four metrics of algorithm performance: the percentage of matches in which the correct outcome (correct winner or draw) was predicted; the percentage of predictions with the correct goal difference; the percentage of predictions with the correct exact scoreline; and the percentage of matches inside the approximate 1-σ region. Table 1 compares the algorithm’s performance in each of these metrics against a benchmark value (the expectation if winners/goal differences/scorelines were selected randomly, or the 68% of the 36 matches which would be expected to fall within a 1-σ confidence interval).

Table 1. Metrics of algorithm performance. The expected values achieved by random selection of results and predictions from the leading BBC pundit are also given for comparison.
Metric Number of Matches
Expectation from Random Predictions N-Estimates Algorithm BBC Pundit Competition Leader
Correct Outcome 13 20 20
Correct Goal Difference 6 10 10
Correct Scoreline 3 5 7
Inside 1-σ Region 25 22 Confidence levels not provided

It is clear that the algorithm performed well overall. In fact, the outcome, goal difference and scoreline were all correctly predicted around 67% more often than would be expected by chance. Particular successes included predicting some unprecedented events, such as England winning their opening EURO game for the first time, and Austria’s first ever win at a European championship. However, there was more variability in the first round than anticipated; only 22 results fell within the approximate 1-σ region, compared with an expected 25 matches. The algorithm’s worst predictions were not randomly distributed, but instead were associated with particular teams whose performances were systematically better or worse than expected (all but one of the matches in which the predicted and observed goal differences differed by more than two involved England, Scotland, Spain or Poland). One interesting finding relates to the effect of a team playing in their own country. In the historical data used to train the algorithm, there is a clear advantage to the home team. For this reason, home advantage was included in the predictions where relevant. However, across the group stage, for the matches where home advantage was expected, our predictions favoured the home team by 1.08 goals per game more than the observed goal difference, on average. For matches played in a neutral country, the average of the predicted goal differences differed from the average observed goal difference by just 0.08 goals. This is a strong indication that, unlike most football matches, home advantage is absent from EURO 2020.

It is also interesting to see how the algorithm compares to expert human judgement. BBC Sport has been running a prediction competition amongst its pundits, and so Table 1 includes statistics for the pundit who is leading the BBC’s competition at the end of the group stage. We can see that the algorithm has performed similarly well to the best pundit, although the pundit was more successful at predicting exact scorelines.

We have updated the teams’ ratings following the first round results and our predictions for the remaining rounds of the competition are below. See Table 2 for the round of 16 predictions, Table 3 for the quarter final predictions, Table 4 for the semi final predictions and Table 5 for the final prediction. The predictions are for the score at the end of normal time. Where this is predicted to be a draw, we have indicated which team we expect to progress (either in extra time or after a penalty shoot-out); this is simply the team with the higher rating before the match according to the N-Estimates algorithm. In response to our discovery from the first round matches, home advantage has not been included in any of these predictions. Matches for the later rounds will be added when the competing teams are known.

Table 2: Predictions for the round of 16 scores at the end of normal time. The confidence indicator plots show 'first team' goals on the x-axis and 'second team' goals on the y-axis. A traffic-light scheme is used, with the central prediction in green, medium confidence results in amber, and low confidence results in red. The * symbol indicates the team predicted to win if the match should go to extra time or penalties.
Date First Team Predicted First Team Score Predicted Second Team Score Second Team Confidence Indicator
Wales 0 2* Denmark 26th Jun, 2021, Wales vs. Denmark. Central normal time prediction: 0 - 2*. Confidence range for the goal difference (Wales minus Denmark): -3 to -1.
Italy 2* 0 Austria 26th Jun, 2021, Italy vs. Austria. Central normal time prediction: 2* - 0. Confidence range for the goal difference (Italy minus Austria): 1 to 3.
Netherlands 1* 0 Czech Republic 27th Jun, 2021, Netherlands vs. Czech Republic. Central normal time prediction: 1* - 0. Confidence range for the goal difference (Netherlands minus Czech Republic): 0 to 2.
Belgium 2* 0 Portugal 27th Jun, 2021, Belgium vs. Portugal. Central normal time prediction: 2* - 0. Confidence range for the goal difference (Belgium minus Portugal): 1 to 2.
Croatia 0 3* Spain 28th Jun, 2021, Croatia vs. Spain. Central normal time prediction: 0 - 3*. Confidence range for the goal difference (Croatia minus Spain): -4 to -2.
France 1* 0 Switzerland 28th Jun, 2021, France vs. Switzerland. Central normal time prediction: 1* - 0. Confidence range for the goal difference (France minus Switzerland): 0 to 1.
England 0* 0 Germany 29th Jun, 2021, England vs. Germany. Central normal time prediction: 0* - 0. Confidence range for the goal difference (England minus Germany): -1 to 2.
Sweden 1* 1 Ukraine 29th Jun, 2021, Sweden vs. Ukraine. Central normal time prediction: 1* - 1. Confidence range for the goal difference (Sweden minus Ukraine): -1 to 1.
Table 3: Predictions for the quarter-final scores at the end of normal time. The confidence indicator plots show 'first team' goals on the x-axis and 'second team' goals on the y-axis. A traffic-light scheme is used, with the central prediction in green, medium confidence results in amber, and low confidence results in red. The * symbol indicates the team predicted to win if the match should go to extra time or penalties.
Date First Team Predicted First Team Score Predicted Second Team Score Second Team Confidence Indicator
Switzerland 0 1* Spain 2nd Jul, 2021, Switzerland vs. Spain. Central normal time prediction: 0 - 1*. Confidence range for the goal difference (Switzerland minus Spain): -2 to 1.
Belgium 1* 0 Italy 2nd Jul, 2021, Belgium vs. Italy. Central normal time prediction: 1* - 0. Confidence range for the goal difference (Belgium minus Italy): 0 to 2.
Czech Republic 0 2* Denmark 3rd Jul, 2021, Czech Republic vs. Denmark. Central normal time prediction: 0 - 2*. Confidence range for the goal difference (Czech Republic minus Denmark): -3 to 0.
Ukraine 0 1* England 3rd Jul, 2021, Ukraine vs. England. Central normal time prediction: 0 - 1*. Confidence range for the goal difference (Ukraine minus England): -2 to 0.
Table 4: Predictions for the semi-final scores at the end of normal time. The confidence indicator plots show 'first team' goals on the x-axis and 'second team' goals on the y-axis. A traffic-light scheme is used, with the central prediction in green, medium confidence results in amber, and low confidence results in red. The * symbol indicates the team predicted to win if the match should go to extra time or penalties.
Date First Team Predicted First Team Score Predicted Second Team Score Second Team Confidence Indicator
Italy 1* 1 Spain 6th Jul, 2021, Italy vs. Spain. Central normal time prediction: 1* - 1. Confidence range for the goal difference (Italy minus Spain): -1 to 1.
England 0 0* Denmark 7th Jul, 2021, England vs. Denmark. Central normal time prediction: 0 - 0*. Confidence range for the goal difference (England minus Denmark): -1 to 1.
Table 5: Prediction for the score at the end of normal time for the final. The confidence indicator plots show 'first team' goals on the x-axis and 'second team' goals on the y-axis. A traffic-light scheme is used, with the central prediction in green, medium confidence results in amber, and low confidence results in red. The * symbol indicates the team predicted to win if the match should go to extra time or penalties. The ratings calculated by the algorithm for the two teams are very similar, leading to a calculated 51% chance that England will win and a 49% chance that Italy will win.
Date First Team Predicted First Team Score Predicted Second Team Score Second Team Confidence Indicator
Italy 0 0* England 11th Jul, 2021, Italy vs. England. Central normal time prediction: 0 - 0*. Confidence range for the goal difference (Italy minus England): -1 to 1.

Quintessa is not affiliated in any way with UEFA or the BBC. Its application of the N-estimates algorithm to the UEFA EURO 2020 competition is an independent and non-commercial endeavour. The UEFA EURO 2020 logo is copyright of UEFA.

Update 1 July 2021: Quarter final predictions added.

Update 4 July 2021: Semi final predictions added.

Update 8 July 2021: Final prediction added.