When “EPSOnic” hits the iceberg

Generation 2004 held a conference on Internal and external competitions: what (legal) action to take? on the 15th of November.

You can find the recording and the  PowerPoint presentation here.

The famous story of the Titanic’s tragic sinking in 1912 serves as a haunting reminder that disasters can often be averted if warnings are heeded. Unfortunately, history seems to repeat itself, and more than a century later, we find ourselves witnessing preventable crises once again.

Regrettably, on 16 October 2023, despite numerous warnings spanning a year, the EPSOnic ship collided with an iceberg and ultimately sank in the European ocean. This catastrophe can be attributed, at least in part, to arrogance and negligence from the ship’s management company[1]. The consequences of this incident are dire, encompassing financial losses, energy drain, damage to reputation, and shattered hopes. Let us now leave behind metaphors and delve into the stark reality.

The reasons behind the ongoing suspension of EPSO competitions

in October EPSO declared “a temporary time-out” on new competitions due to the many challenges faced by EPSO during the implementation of automated remote proctoring[2] (introduced 04.09.2023) under the New EPSO Model. There were many concerns about the intrusiveness of live/human remote proctoring, which involved personal and environmental checks by remote invigilators. The time-out/pause does not affect competitions from 2022 and those which have already passed the testing phase. Consequently, with the exception of competitions for Nuclear Safeguards Inspectors and Policy officers in the area of nuclear energy (published on 19.10.2023) and two CAST sessions (on 23-24.10.2023 and 6-7.12.2023) all other competitions originally planned for 2023 are now postponed until 2024.

Difficulties with retesting

The test provider, Prometric, could not ensure the desired quality with its automated proctoring platform. This led to a huge rise in technical problems and legitimate complaints introduced by the candidates to EPSO. It resulted in an exceptionally high retesting rate (ranging between 25% and 35%), which far exceeded what would be considered acceptable even for EPSO. The norm is typically between 2% and 3%, with an upper limit of 5%. The significant number of complaints and testimonies point to automated proctoring as the primary cause of the various issues faced by candidates during the testing phase. Retesting would typically occur one week later but due to the huge number or candidates requiring retesting, it was impossible to accomplish within such a short timeframe. This created a perceived  unfair advantage for those who had to retest, as they had more time to prepare than the initial test-takers. EPSO counters the accusation of unfairness, stating that it conducts psychometric analyses after each retesting session to identify any differences between testing sessions and neutralise  any biases.

EPSO has also announced details about the rescheduling of the retesting for the competitions in the fields of Economics (13.11.2023) and Crisis management, migration, and internal security (24.11.2023). These competitions are now experiencing delays beyond their originally announced retesting dates. However, the difficulties encountered with the testing platform appear to be even more significant than anticipated. Consequently, on 8.11. 2023 EPSO decided to further postpone the retesting for the Economics competition (EPSO/AD/402/23) due to concerns about ensuring “adequate testing conditions”. Notably, all candidates were informed in their EPSO accounts that they would be given the opportunity to retest, regardless of whether they requested it, raising questions about the obligation to retest and its implications for other competitions. Now legitimate questions arise as to whether they are obliged to retest or not and whether the same situation might occur in the other competitions. In our humble opinion, the issues were so serious and the number of complaints so high that EPSO had to make this offer of resits to mitigate the risk of candidates taking legal action.

Additional information about the remaining retesting sessions for Intellectual Property (EUIPO/AD/01/23) and Crisis management, migration and internal security (EPSO/AD/403/23) will be communicated by EPSO on their website.

Escalating the situation

In the aftermath of EPSO’s decision, they tried to portray in a recent Politico article that the main reason behind the postponement to a “legal minefield” was related to 24 languages and a recent lawsuit by France. Apparently “even if we had had an excellent platform to test candidates, this alone [the ruling from the Commission’s legal services] would have made us postpone the publication of the competition.” We dare to claim otherwise! This explanation seems dubious, and it appears that other factors have contributed to the situation. It is crucial to acknowledge the significance of ignoring and downplaying concerns raised by various stakeholders.

 Ignoring and downplaying our concerns

 Could this situation have been better managed? Certainly, it could have, yes. Generation 2004, the Central Staff Committee (CSC) of the European Commission[3] and the Common Front repeatedly brought these issues to the attention of EPSO, its management board, and HR services for nearly a year. Regrettably, our concerns were consistently either downplayed or ignored, and staff representation has so far not been given a fair opportunity to provide essential feedback within the framework of the social dialogue. Consequently, the escalation of technical problems and resulting reputational damage could have been avoided, had these worries been addressed earlier.

 Price already paid for remote testing

Moreover, the shift to remote testing has created an unintended consequence – it has introduced new costs for candidates which may or may not replace the travel costs of the past[4]. Many individuals now face the expense of securing a physical testing space which both meets the EPSO criteria and has reliable internet access. Candidates are renting computers and making every effort to meet other requirements. This change has made taking an EPSO exam financially burdensome in a different way from before and some candidates may have difficulty meeting the new conditions. In effect, the new testing model exacerbates inequalities among candidates in a new and different way. In addition to the cost/access issues of the past (issues which previously affected only those living far from a test centre, but which now affect a much larger group) EPSO testing will now include hurdles relating to IT/technical prowess, the digital divide/home internet quality and the alignment of the stars on the day of your exam (i.e. will the neighbour decide to drill a hole in the wall next to you, will there suddenly be a fire alarm/will the test freeze for hours while you desperately try to call the provider?). How well does this align with the goals of the EPSO testing process and with the stated desire to diversify and include?

When will EPSO resume its core business?

Everyone is now asking what will happen with the EPSO competitions that were suspended and when will they resume its business as usual. Unfortunately, EPSO has not provided any specific timeline for the ending of the postponement. Generation 2004 is quite sceptical – in our opinion, it will take more than several months to return to normalcy. However, informal indication suggests that no significant progress is expected until at least the end of January 2024. The CAST Permanent competition might resume by April 2024. EPSO is also committed to publish the long awaited “Generalists” competition still in 2024. This competition is very tricky as they expect about 44000 candidates, which will be a challenge to manage. Additionally, the EU knowledge test and Digital skills test must also be translated into 24 languages, which will cause additional delays. Nevertheless, this informal information predates the postponement of the retesting.

Prometric must also solve numerous issues around the automated proctoring, the availability of helpdesk support and adherence to the tender specifications, which have not been, to a large extent, respected as requested (this seemed to be perfectly fine for EPSO as they did not object) [1].   It is somewhat ironic that the complaints in the last three competitions were consistent, given that many of the issues such as black screens,  and platform connectivity problems  have persisted since the introduction of the new model. EPSO has promised informally to provide a list of issues that candidates faced during the testing. We are wondering whether this outsourcing (‘externalisation’) was the right way to go. Our experience shows over and over that cost-saving measures to the EU budget  often compromise the quality.  Apparently, the EU is not a lucrative enough customer to make a difference (the contract was worth ‘only’ 29 million EUR.) The worst thing is that the current contract with Prometric is only in its first year of four-year contract. The reason why this company was chosen is because there was no other candidate. Perhaps those in charge of the tender should have posed themselves a question on why it looks like Oracle, IBM, Microsoft, PMI and others moved away from Prometric.

The future

Furthermore, it seems that remote testing is here to stay according to EPSO because Prometric has dismantled all of the physical test centres. Most likely EPSO took this decision and Prometric implemented it. Perhaps considering the possibility to have a hybrid way of testing would provide more options and increase access to the competitions, this is rather for future discussion.

If the situation wasn’t already grave enough, recent developments within the EPSO Management Board have raised further alarms. The Central Staff Committee of the European Commission have an observer who regularly attends the Board’s proceedings. However, prior to the last meeting(07.11.2023), an unusual disclosure was made: a portion of the meeting would be conducted in a “restricted format.” This development is perplexing, as it deviates from the current rules governing the EPSO Management Board. It serves as yet another indication of underlying issues within the Board.

It is known that some decisions are often prearranged in other meetings, which not all stakeholders are privy to. However, conducting such proceedings in such an overtly secretive manner and excluding observers is deeply concerning. It underscores the lack of transparency within the Board and imposes an information embargo on the most pressing issue at hand – the suspension of the competitions. Someone must ultimately bear political responsibility for EPSO’s mission failure. It is imperative that those in leadership positions – much like a ship’s captain taking responsibility for a sinking vessel – acknowledge their role and take the necessary steps. If this isn’t a glaring example of reputational damage, one must question what it would be.

Conclusion

The situation surrounding EPSO’s recent crisis is deeply concerning, as it has revealed a series of systemic issues, ranging from technical problems and questionable management to a lack of transparency and accountability. The impact extends far beyond financial losses and reputational damage, affecting candidates and stakeholders alike.

To resolve these challenges and regain trust, EPSO must embark on a comprehensive reform process. This includes re-evaluating its partnership with Prometric, addressing the issues that have plagued recent competitions, and reconsidering the move to remote testing to ensure fairness and accessibility for all candidates. Furthermore, EPSO must commit to genuine engagement with the staff representation and take immediate steps to establish transparent communication channels.

The unfolding situation underscores the need for accountability within EPSO and its Management Board. It is imperative that those responsible for the failures take appropriate action, including resignations, if necessary, to restore the faith in the organisation’s ability to fulfil its mission. This crisis serves as a clear example of how accepting and addressing constructive criticism can avert disaster.

As always, we would love to hear from you. Please do not hesitate to get in touch with us or leave a comment below. 

If you appreciate our work, please consider becoming a member of Generation 2004. 


[1] EPSO Management Board – an interinstitutional body

[2] ‘Proctor’ is a US term. In the UK this is ‘invigilator’.

[3] One of Generation 2004’s Vice-Presidents of the CSC was mandated with this matter and was the leading force behind the initiatives.

15/12/2022 Serious problems related to competition EPSO/AST/154/22 + Annex HR
17/01/2023

[4] To reach test centres, mostly in capital cities, and costs at this point in a competition were never reimbursed.

Leave a Reply