Recently, I have had the pleasure of conducting some exploratory research with a Championship football club to support them in their recruitment strategy. The research question was to understand how we can better estimate the change in players’ performance metrics when they transfer from one league to another.
This has been an ongoing question in football analytics for some time, so the first step I took was to put some feelers out to some analytics organisations, to understand what work was being done to answer this question. A quick reply from Soccerment was:
“You have actually just touched upon one of the hardest but – in our view – most important tasks that the football analytics world needs to work on.”
…followed up by a reply from Matchmetrics:
“Translating rating differences between different competitions is a non-trivial task, and not exactly being helped by a comparatively low amount of data.”
No pressure, then. A unanimous agreement that this was a difficult question, but a hugely important one from a recruitment perspective. In the current global climate, where the purse strings are tighter than ever within professional football clubs, minimising mistakes in the transfer market is essential.
. . .
First, let’s use an example to be clear on what exactly this challenge is.
Kasper Junker has scored 27 goals for Bodø/Glimt in the Norwegian Eliteserien this season, at a rate of 1.21 goals per 90 mins. Therefore, he is likely be a noteworthy player to look at across many recruitment teams in Europe. However, given that the Eliteserien is a weaker league compared with (for example) the Championship, would we expect Junker to score at the same rate if he moved to England? Probably not.
From a recruitment perspective, there are a couple of options that could be taken when considering the change in a player’s performance metrics between leagues:
Option 1 – Apply an ‘intuitive’ adjustment between leagues
Of course, there may be clubs who do not apply an adjustment of metrics at all when looking at a player in other leagues. This might have the benefit of allowing recruitment staff to look at all player’s metrics on a level playing field, but this is of course susceptible to error – you could overrate players who have good metrics in a weaker league, and miss out on a player who might have average metrics in a strong league.
Scouts and recruitment staff may apply their own intuitive adjustment when looking at a player’s performance in different leagues. Simply knowing that one league is stronger than another can help in the discussion of “how would he fare in our league?”. This could also be based on the experience of the recruitment staff, which is valuable, but similarly relying on this method alone might be prone to error due to the subjective nature of this approach. As is often the case in modern football, combining the use of data with traditional scouting methods can be the most valuable method to work from.
So how can we provide a more objective way to inform the adjustment in a player’s performance metrics if he did make the move to another league?
Option 2 – Apply a league adjustment based on previous transfers
There has been some really neat public work which has looked at exchange rates, or adjustments of certain metrics between leagues – most notably from Omar Chaudhuri at 21st Club, Ben Torvaney in his OptaPro Analytics presentation, and from Andy Watson at a team level between English leagues (see also, Smarterscout FAQs).
However, as discussed above, these exchange rates may be relying on a “comparatively low amount of data”, in which predictions are made based on transfers that often have been made between those leagues. For example, there are many examples of players who have moved between the Top 5 European Leagues, so we can find some relatively accurate exchange rates based on the data here. But how can we look beyond the data that we have? Can we predict how a player from the Brazilian Série A would do if they came to the Championship, without previous data of that transfer?
. . .
We aimed to devise a more accurate prediction of a player’s performance metrics when they move to a new league. From this, we generated a new option:
Option 3 – Apply a bespoke adjustment – per position, per metric
There are many great resources available which can provide an estimation of a team/league strength around the world – including ClubElo, FiveThirtyEight, and UEFA club coefficients. Methods such as this allow us to compare leagues relative to each other in a more objective way using numerical values, and see whether the difference in the league strength can predict the difference in a player’s performance metrics when they move between leagues.
This objective way to estimate that one league is stronger/weaker than another is a great start, but we wanted to look deeper by exploring how this league strength difference has an effect:
- between player positions (i.e. Central Defender vs. Centre Forward)
- between performance metrics (i.e. Aerial Duels vs. Goals)
When adjusting between leagues, ‘taxing’ a Centre Forward equally on their number of goals and their number of aerial duels hardly seems fair, as they require different skills. So it is important to look at the change of each metric separately, for each position.
An additional factor we considered was the effect of league style towards the change between leagues. Let’s go back to the Kasper Junker example. If he is scoring 1.21 goals per 90 mins in a weaker league, that is one thing – but it is also important to note that the average number of goals per team (per game) is 33% higher in the Eliteserien compared with the Championship, suggesting it is a more liberal goalscoring league in Norway which must be accounted for in the adjustment.
The Analysis
The approach we took was a pragmatic one – while there are certainly complex methods by which we could answer this research question, the main aim was to get some effective output in the data that would provide a more accurate method than the one that is currently used.
I must ensure I retain the confidentiality of the information that has been agreed with the Championship club, but together we have agreed to share a broad approach that we used to answer this research question.
As with all predictive frameworks in football analytics, we need to use historical data to provide insight into what might happen in the future. To do this, we used a multiple regression analysis. Many people reading this will be aware of this analysis – for those who are not, multiple regression is a statistical method that allows us to identify relationships between certain outcome variables (e.g. % change in a performance metric between seasons) and a series of predictor variables (e.g. change in league quality and change in league style). Crucially, this regression analysis creates a ‘model’ which we can apply to predict outcomes beyond the data that we have.
We used Wyscout data from 30 leagues to look at players who had transferred leagues between the 2018/19 and 2019/20 season (total of 1838 players). We broke the analysis down per position, by Goalkeeper, Centre Back, Full Back, Centre Midfielder, Attacking Midfielder, and Centre Forward. It is worth noting that we did parse out players who moved leagues internationally vs. those who moved leagues domestically due to a promotion/relegation, to determine whether there were any noticeable discrepancies in the results.
As discussed above, the analysis allowed us to look at all transferred players within the 30 leagues, which created a model that can go beyond the data we have. So even if we don’t have the data on a player who has transferred between from the Brazilian Série A to the Championship per se, the model can use the difference in league quality and league style to calculate a predicted change in a player’s metrics, if they were to make the transfer.
. . .
Back to Kasper Junker.
If Junker moved to the Championship, the model could provide us with an indication of how many fewer goals we can reasonably expect to see, given the higher quality league and a lower number of goals per game in the Championship.
However, if Junker moved to the Eredivisie – which has a similar league quality to the Championship – the model predicts that he will retain more of his performance output, given there are a higher number of goals per game in the Eredivisie compared with the Championship, that is more comparable with the Eliteserien.
The example above simply shows a proof of principle of the performance metric adjustment applied when moving to a new league, depending on the difference in quality and style of that league. Importantly, we applied this method to provide a bespoke ‘tax’ for all metrics, and all positions, to significantly improve the predictions in player performance between leagues.
Closing Thoughts
Of course, from undertaking this analysis we wouldn’t expect a player to produce an output that is exactly at the predicted rate, not least with the Junker example who has had an exceptional season in front of goal for Bodø/Glimt in 2020. What is crucial is that this new method provides a more accurate indication of the predicted difference between leagues, based on multiple indicators.
Secondly, not all metrics were statistically predicted by league quality and/or league style – so it is important to see where we can improve our understanding further across all metrics. Thirdly, the conversation around player recruitment across Europe is further influenced by the new Brexit rules from January 1st 2021, meaning that transfers to Premier League and EFL clubs will operate within a points system to determine their eligibility to receive a work permit.
There are further avenues to take this work to add further granularity to the model – e.g. the effect of team style and player age being two variables that come to mind. There are also many intangibles that simply cannot be added into the model, such as the time taken to settle in the country, or the ability to speak a new language. But as I discussed throughout, the approach taken was a pragmatic one, in helping to improve upon the current methods that are used to provide an increasingly accurate estimation of player performance when transferring leagues.
Finally, I want to thank the Championship club for the opportunity to conduct this piece of work, and hope that the findings will help to support them in their recruitment for the coming seasons.
Carey Analytics 