You are on page 1of 13

https://docs.google.

com/document/d/1WQAFUebP
x5TNvgUKuIaUFPa80cdBuJhDbhwlh15EHz0/edit
#

Complaint to the European Ombudsman about


Elsevier and the Open Science Monitor

PLANNED TIMELINE:
We will attempt to have this ready in final version by mid-morning CEST Wednesday July 5
and then give people until noon to sign (or check if they want their signature to remain)

Complaint form:
https://www.ombudsman.europa.eu/en/atyourservice/secured/complaintform.faces and
information guide: https://www.ombudsman.europa.eu/en/resources/staffguide.faces#/page/1

Catalyst: https://www.theguardian.com/science/political-science/2018/jun/29/elsevier-are-
corrupting-open-science-in-europe

UPDATE: Elsevier have published a direct response to this.


https://www.elsevier.com/connect/elsevier-serves-the-global-research-community-to-deliver-
open-science

Comment on metrics that will be used:


https://www.makingspeechestalk.com/ch/Open_Science_Monitor/

KEY ISSUE: Why Elsevier have been subcontracted by a consortium on behalf of the
European Commission for the Open Science Monitor. The original tender award notification
can be found here.

--------------------------------------------------

Introduction
We, the below signed, wish to raise a complaint to the European Ombudsman about the
recent announcement that Elsevier, a publisher and data analytics provider, has been
subcontracted to monitor the future of Open Science in Europe. We became aware of
Elsevier’s involvement in this on March 12, 2018, when information about the Open Science
Monitor was first publicized, and then again on May 30 2018 when further details were
released, including the first version of the monitor and the underlying methodologies.

Elsevier are the single subcontractor for a consortium consisting of the Centre for Science
and Technology Studies, The Lisbon Council for Economic Competitiveness and Social
Renewal, and the ESADE Business & Law School. The contract was awarded to this
consortium, with the consortium in charge of subcontracting part of the work and deciding
who it should be awarded to. The proportion of the work to be subcontracted is estimated at
10% in the tender award notification.

We see a number of critical administrative issues with the process of the subcontract award
and that will have a detrimental impact on the future of Open Science and innovation in
Europe, the livelihoods of European citizens, and the legitimacy of the European
Commission (EC) as an institute.

These can be broadly separated into two main issues. Firstly, there appears to have been a
lack of care and transparency with the process of awarding the subcontract. Secondly, and
as a consequence of this, Elsevier are now in a position where they will be monitoring and
evaluating the very same science that they, and their competitors, sell as their primary
products. Furthermore, the metrics and data sources used in the evaluation, are biased
towards those owned and operated by Elsevier, which creates an inherent bias and conflict
of interest, to the exclusivity of their competitors and other primary data sources (e.g., Web
of Science, Dimensions, Crossref, Datacite, BASE, SHARE, PubMed, and other publishing-
based services).

Transparency in the awarding process


 General lack of transparency in the decision process leading to this point. This
includes specific points regarding the award of the tender to the consortium, which
remain relevant in this context:
o How did the 3 bids received for the tender score on the specific criteria that
were used to select the contractor? Why is this information not required to be
made public?
o Who evaluated the suitability of each candidate? Were independent external
experts involved in the evaluation process?
o Was there a consultation process involved?
o Why are tenderers only required to identify subcontractors whose share of the
contract is above 15%?
o Was the identity of this subcontractor made known to the EU during the
tender process?
o Was a risk analysis performed as to the ramifications of the choice of
subcontractor?

And specific points focussing on the subcontractor themselves:

 How do the consortium and the EU resolve the incredibly high profit margins (~37%)
of Elsevier with the intrinsic motivations behind open science, including financing and
governance?
 How is the proportion of work in this project that is allocated to the subcontractor
(10%) calculated?
 Who is accountable for the process itself, including resolution of internal disputes
during (not just the performance of the contract as a whole), with the data providers
(Elsevier), European Commission, and data analysts operating as three non-
independent parties?
 Given the EU’s emphasis on Open Science, including Open Data, why is there
(apparently) no requirement to insist that the Open Science Monitor must be based
upon open data, open standards, and open tools (with appropriate licenses for
accessibility) as a matter of principle?
 How will the comments on the indicators, many of which specifically also mention the
bias towards Elsevier services (including my own), be handled as part of the
consultation?

Consequences of Elsevier as the sole subcontractor


 What impact will this position of power from Elsevier have on the future of a fair
scholarly publishing market in the EU? From a scientific perspective, selectively
choosing data sources to the exclusion of others (as clearly indicated in the methods
for the Monitor), is generally considered to be bad practice, and in cases even
fraudulent.
 By using predominantly, and for many indicators almost entirely, Elsevier-based
services, such as Mendeley, Scopus, and Plum Analytics, subcontracting to Elsevier
creates an inherent bias in the data sources. The potential direction and size of these
biases is unknown at the present.
o This is partly a function of the products (metrics) and data themselves being
proprietary, and an irresponsible approach towards metrics usage for
evaluation.
o The metrics proposed to be used for the monitor are not acquired by an
independent body, but based on Elsevier products and services, creating an
inherent bias in the data sources.
o The fact that Elsevier is a publisher offering services that monitor scholarly
publishing also presents a serious conflict of interest, and does not respect
current competition laws.
o This also actively discriminates against the competitors of Elsevier, creating
unfair market conditions around Open Science evaluation and metrics.
 It is as yet unclear whether, as Elsevier services feature so prominently, these
services will become more of a requirement (either formal or implicit) for EU
researchers and research institutions to use, as this is what they are becoming
primarily assessed on.
o A potential consequence of this is that researcher careers become more
dependent on Elsevier’s product workflows.
o A further consequence of this is that many other competitive services and
workflows will be discriminated against.
o This creates an inherent conflict of interest in having Elsevier as the sole
subcontractor.
 Whether or not Elsevier will benefit by being able to sell more of their metrics and the
underlying data sources.
o This creates another inherent conflict of interest in having Elsevier as the sole
subcontractor here.
o They are now in a position to evaluate the very same science that they and
their competitors sell.
o As these data will be used to guide public policy in the EC in the future,
Elsevier clearly stand to benefit from being subcontracted.
 Will the data sources be made openly available for independent inspection and
analysis? Will data gathering protocols be transparent and replicable? How will the
EC ensure that the closed, profit-driven practices of Elsevier match and align with the
expectations of government accountability that makes the EU so successful?
 What ‘sanity checks’ for the data will be emplaced to compensate for the risk
associated with using a single data source?

It should be noted that the EC and the consortium deserve some credit in their handling of
this, by making the methods and data sources transparent (in that we can see what is being
done), as well as for inviting feedback to improve the indicators. However, we believe that
this almost complete lack of transparency in the administration and functionality of this
service, as well as the process that led to Elsevier being awarded the subcontract, which
potentially violates the fundamental rights of European citizens. Furthermore, it is discordant
with the Jussieu call for for Open science and bibliodiversity signed by many EU institutions.
Furthermore, it comes at an intriguing time in Europe when many university consortia are
cancelling their licensing contracts with Elsevier and their kin over disputes to do with
transparency, pricing, and unfair market conditions.

Further related concerns


[Elsevier lobbying weight may be too strong] There is also a concern that Elsevier already is
very present at many places where the EC is helping to shape the future of Open Science.
We know that RELX, the organisation that owns Elsevier, has 6 lobbyists within the EC, 3 of
which have direct access to parliamentary premises, and which have an estimated
expenditure in the EC of up to half a million euros each year. Furthermore, RELX also have
a representative for the Horizon 2020 expert group on Future of Scholarly Publishing and
scholarly Communication, again indicating that they are having at least some control on the
future of scholarly communication in the EC.

A lack of dialogue between the EC and all relevant stakeholders to the Open Science
Monitor will erode trust and legitimacy for the EC, and does not follow the principles for good
administration and governance for these sorts of processes. Finally, we find the agreement
to outsource such a critical task to an organisation with a history of fighting against the very
system it will be monitoring, to be evidence of low operational standards from the EC.

Co-signed (in an independent capacity):


Please note here that many of these signatories are from beyond the EU, demonstrating the
wider concerns with the present issue, and the far-reaching geographical impact it could
potentially have,

Jonathan Tennant (UK)


Chris Hartgerink (Netherlands)
Monica Gonzalez-Marquez
Sylvie Vullioud (Switzerland)
Tony Ross-Hellauer (Austria)
Marius Ostrowski
Anson Mackay (UK)
Bryan Lougheed
Gustav Nilsonne
Tobias Steiner (Germany)
Jonathan Dugan (USA)
Bianca Kramer (Netherlands)
Dorothy Bishop (UK)
David Colquhoun (UK)
Christiaan Monden
Benjamin Dean
Jon Agirre
Jennifer Beamer
Marin Cvitanović
Michael Ward
Dasapta Erwin Irawan (Indonesia)
Tristan Bekinschtein
Jenny Eriksson
David Adger (UK)
Jennifer Salomonsson
Daniel Albertsson
Julien Colomb (Germany)
André Hoffmann (Switzerland)
Dirk Verdicchio (Switzerland)
Miguel Moreira (Switzerland)
Christian Hauschke (Germany)
Natalie Bireima (Switzerland )
Florian Jug
Ricardo Ramiro
André Dazy
Stephen Eglen
Jorge Luis Saturno
Samuel Lampa (Sweden)
Lluís Revilla Sancho
Michael Schiltz (Belgium)
Debora Weber-Wulff (Germany)
Jason Serviss (Sweden)
Laurent Gatto (UK, Belgium)
Rolf A. Heckemann (Sweden)
Hauke Heekeren (Germany)
Simon Worthington (Germany)
Michael R. Crusoe (Lithuania, USA)
Miguel Maravall (UK)
Shalini Kurapati (Netherlands, Italy)
Jean-Sebastien Caux (Canada, Netherlands)
Elena Giglia (Italy)
Maria Chiara Pievatolo (Italy)
Plinio Cabrera Casarotto (Finland, Brazil)
Thorsten Rissom (UK, Germany)
Dino Buzzetti (Italy)
Tom Johnstone (UK, Australia)
Surya Dalimunthe (Indonesia)
Afrilya (Indonesia)

Sylvie Vullioud
3:11 AM Yesterday
Thank you. Could you tell me if offers by different contractors were also published on
SIMAP as response of this award notification? I cannot find any?

Bianca Kramer
6:55 PM Yesterday
Me neither, but I haven't seen that for any tender. Have you?

Sylvie Vullioud
2:15 AM Today
Well, in fact on the Swiss Simap, it is sometimes possible to read some commercial offers
and names. Maybe this parallel is not possible with EU Simap?
Jon Tennant
12:42 AM Yesterday
This is still open to signatures from anyone, whether you contribute to the document or not.

Christina Riesenweber
7:08 PM Yesterday
I'm kind of waiting for a final version before signing. Is this it?

Lluís Revilla
7:48 PM Yesterday
I think so, maybe there should be a deadline to final version of the letter and a deadline to
send the letter

Jeroen Sondervan
8:01 PM Yesterday
Do you need to have a certain number of signatories for filing a complaint like this?

Bianca Kramer
8:02 PM Yesterday
I dont think so (number of signatories). As far as I understand, 1 person can file a complaint
as well.

Bianca Kramer
11:39 PM Yesterday
Now added the aspirational timeline at the top of the document cc +jon.tennant.2@gmail.com
Sylvie Vullioud
1:59 AM Yesterday
The method was criticized on Twitter (and what about a real post-reviewing platform?). The
only assessment tool was Scopus. The choice of this methodology is not explained: how and
why where other data sources not choosen WoS, Dimensions, Pubmed, Crossref, BASE,
preprints servors per domains Arxiv , Biorxivpreprint, PsyArXiv, Socarxiv, EarthArXiv,
ChemRxiv, Paleorxiv, Lawarxiv, MarXivPapers, AgriXiv, EngrXiv, ECSasXiv,
Thesiscommon, etc ?

Whole methodological choices should always being explained to readers. It is also essential
for reproducibility, robustness and reliability of the data, as recommended by H2020 projects
and Data Management Plan (DMP) for good data management practices.

The publication on the method was only published in June 2018, just before the
announcement of Elsevier being chosen as subcontractor for OA monitoring by CWTS, The
Lisbon Council for Economic Competitiveness and Social Renewal, and the ESADE
Business & Law School.

CWTS and Elsevier worked for at least 3 years together.


2015 The Leiden Manifesto for research metrics points problems with WoS but not Scopus
https://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-
1.17351

2017 Open Data the researcher perspective


https://www.elsevier.com/__data/assets/pdf_file/0004/281920/Open-data-report.pdf

Signed by
- Paul Wouters
Professor of Scientometrics Leiden University and director of CWTS
https://www.cwts.nl/people/paulwouters

- Wouter Haak, Vice President,


Research Data Management,
Elsevier. Previously was responsible for Elsevier’s product strategy for Scopus,
ScienceDirect and Mendeley.
https://www.stm-assoc.org/people/wouter-haak/

2018 April 2018 Trends for open access to publications report


https://www.elsevier.com/connect/elsevier-and-cwts-release-report-on-data-sharing-
perceptions-and-practices-among-researchers

2018 June 2018: announcement of Elsevier as subcontractor for OA monitoring by CWTS,


Lisbon Council for Economic Competitiveness and Social Renewal, and the ESADE
Business & Law School
Show less

Sylvie Vullioud
2:14 AM Yesterday
Questions:
1. Who was contracted by EU for the choice of OA monitoring? CWTS company
http://www.cwtsbv.nl/ and/or CWTS Leiden University Institute?

2. How could CWTS/University of Leiden issue a report on OA monitoring at the same time
on selection of subcontractor? What about possible conflicts of interest?

3. CWTS worked for many years on the development of journal indicators based on Scopus
as mentionned on http://www.journalindicators.com/. What was the nature of putative
collaboration?

4. Technically, how data from Crossref can be used to identify green OA publications
remains the well-kept secret of CWTS Institute and/or Commercial (by @Lib4RI) preventing
transparency and reproducibility, and suggesting possible commercial interest and putative
conflict of interest?

5. Did all persons and partners selecting subcontractors declare their interests (CWTS, Lisbon
Council for Economic Competitiveness and Social Renewal, and the ESADE Business &
Law School), such as done in peer-reviewed articles? Is it public?
6. Being a subcontractor for OA monitoring is an important contract that offers should have
been published on the EU platform SIMAP Information System for Public Procurement
http://simap.ted.europa.eu/. If not, why?

7. The Leiden manifesto of 2015 specifies that relying only on major databases WoS or
Scopus is not reliable because:
- social, economic, educational, psychology, history and law journals are poorly indexed by
these databases
- local research is not indexed by these dataases
- non English articles are not indexed by these databases
Given this early awareness of 2015, how can CWTS rely only on Scopus and Elsevier tools
only for the OA monitoring in 2018?
Show less

Jon Tennant
3:22 AM Yesterday
So much great information here, thank you! I'll start integrating it in :)

Dasapta Erwin Irawan


7:15 AM Yesterday
Thank you Sylvie for the explanation. Yes, CWTS is built upon Scopus dataset, so we can
count CWTS and Elsevier as one entity in this activity.
Bianca Kramer
5:21 AM Yesterday
Another interesting aspect [gah, seems I'm starting all my sentences like that today] is that the
"Value or proportion likely to be subcontracted to third parties" is 10% (proportion) (tender
award notice point V2.5) Apart from the fact that I sincerely doubt that the value of the
Elsevier subcontract is only 54K (total contract value = 543K)....
Show less

Bianca Kramer
5:22 AM Yesterday
... if I remember correctly the % of the contract value or proportion to be subcontracted out
needs to be less than 15% to allow the subcontractor to not be named/specified. Now where
did I read that... I'll look it up

Bianca Kramer
5:31 AM Yesterday
Ah, yes, it's in the tender specifications
(https://etendering.ted.europa.eu/document/document-file-
download.html?docFileId=32281#) under 1.5 Subcontracting:

Subcontracting is permitted but the contractor will retain full liability towards the Contracting
Authority for performance of the contract as a whole.
Tenderers are required to identify subcontractors whose share of the contract is above 15%
and whose capacity is necessary to fulfil the selection criteria.
During contract performance, the change of any subcontractor identified in the tender or
additional subcontracting will be subject to prior written approval of the Contracting
Authority.
Show less

Sylvie Vullioud
2:31 PM Yesterday
"Elsevier subcontract is only 54K (total contract value = 543K)"....It may be possible in the
light of collaboration developed for years between CWTS and Elsevier. Elsevier may even
pay CWTS Institute and/or commercial branch to develop Elsevier indicators!
Amount of money may not be the real issue for Elsevier. Issue may be to play against WoS
main competitor to develop institutional ranking indicators based on Elsevier products at the
end. Market strategy only.
Show less

Bianca Kramer
3:46 PM Yesterday
Yes. Although the reference is to 10% of the proportion of the work, not 10% of the contract
value, sorry. So my remark regarding the value was actually less relevant (though the issue
remains relevant of course)

Sylvie Vullioud
4:02 PM Yesterday
Thanks for correcting my mistake. Your remark is anyway relevant for our understanding.
Anyway thank you for sharing your sharp knowledge about EU public bid system!

Bianca Kramer
5:26 AM Yesterday
The subcontractor could also already have been part of (or at least mentioned in) the bid on
the basis of which the contract was awarded. So suggest to remove the implied temporal
sequence of events?

Bianca Kramer
5:44 AM Yesterday
But we just don't know because they didn't HAVE to be identified in the tender process
because of the 15% rule..

Lluís Revilla
7:11 PM Yesterday
Elsevier don't sell science, sells publishing, licenses to access papers or databases. Could that
be rephrased? and "their competitors relay on".
Bianca Kramer
2:04 AM Today
Added one more...
Bianca Kramer
6:59 PM Yesterday
Still not entirely clear to me, sorry. At least for the relationship between the consortium and
the subcontractor, I think the stipulation in the tender requirements (under 1.5) makes it clear
that the final responsibility lies with the consortium. I take that to be for the process as well
as for the final product, or is that exactly what is questioned here?
Show less

Anonymous
8:57 PM Yesterday
Probably better to intro as different processes, then have a single sentence sub-bullet point
per issue. Otherwise the Ombudsman might consider questions out of scope.

Simon Worthington
9:20 PM Yesterday
I would add being open-source, and add reusable to accessibility

Simon Worthington
9:34 PM Yesterday
The EC does have a strategy history with Open Source Software that goes back to 2000, even
including the release of a EU Licence, European Union Public License (EUPL), and a 10
point strategy, see: https://ec.europa.eu/info/departments/informatics/open-source-software-
strategy_en
Tom Johnstone
7:52 AM Today
I think this part is crucial. One of the core principles of Open Science is that fully transparent
data and protocols allow for independent reanalysis and validation. For the outputs of the
Open Science Monitor to be accepted by the Open Science community, this must be the case.
Bianca Kramer
12:42 AM Yesterday
It's an interesting question whether the responsibility for the monitor lies with the consortium
or with the EC. Can we split this into the action (of the consortium) and the responsibility (of
the EC)?

Or simpy add: "the consortium and the EC deserve (...)"?

Jon Tennant
3:14 AM Yesterday
This is mentioned in a point above too:

Jon Tennant
3:14 AM Yesterday
Who is accountable for the process, with the data providers (Elsevier), European
Commission, and data analysts operating as three non-independent parties.

Jon Tennant
3:15 AM Yesterday
Modified now too

Jon Tennant
3:15 AM Yesterday
Marked as resolved

Peter Murray-Rust
3:27 AM Yesterday
Re-opened
Sylvie Vullioud
3:09 AM Yesterday
Addition of Jussieu call http://jussieucall.org/?

Jon Tennant
3:16 AM Yesterday
Go for it! :)

Jon Tennant
7:42 AM Yesterday
How do you feel this fits in here..?

Sylvie Vullioud
2:05 PM Yesterday
...violates their fundamental rights and Jussieu call for for Open science and bibliodiversity
signed by many EU institutions.

Jon Tennant
5:42 PM Yesterday
Yeah, get it in there! You should be able to edit :)

Lluís Revilla
7:06 PM Yesterday
Added, but not sure if now the sentence is too long.
Jon Tennant
12:18 AM Today
Modified, looking good!

Jon Tennant
12:18 AM Today
Marked as resolved

Lluís Revilla
12:38 AM Today
Re-opened
In the next sentence there is another "Furthermore"... I think it still needs some rewriting.

Sylvie Vullioud
2:20 AM Today
Thank you Lluis and Jon for insertion, I did not dare direct editing of the document.
Sylvie Vullioud
3:01 PM Yesterday
Elsevier is already very present at important place for the 'shaping' of EU Open Science.
Putative conflict of interests should be examined?

1. Elsevier (RELX) is an expert organisation (type C) in Horizon 2020 on Future of Scholarly


Publishing and scholarly Communication group (E03463)
http://ec.europa.eu/transparency/regexpert/index.cfm?do=groupDetail.groupDetail&groupID
=3463

2. CWTS (maybe bound to Elsevier?) participates to EU OA future EU platform vision: "with


their proposal the consortium, hosting a number of renowned journals (Elsevier?), with the
backing of a number of collaborating partners such as the Max Planck Digital Library and
CWTS (Leiden University), thinks to offer an alternative to both a new publication platform
and the ‘big deals’ with other publishers".
in https://www.scienceguide.nl/2018/05/native-open-access-publishers-propose-a-new-
publishing-deal/
Show less

Tom Johnstone
7:55 AM Today
Would it be worth submitting a FOI request to find out if the lobbyists had been promoting
their inclusion in the Open Science Monitor, or suggesting anything to do with the way it was
set up or tendered?
Sylvie Vullioud
4:43 PM Yesterday
I wonder if non EU citizen signing the recourse should be listed separately, for the reader
transparency?

Jon Tennant
5:18 PM Yesterday
Is that a joke about me being English? ;)

Simon Worthington
10:07 PM Yesterday
Its worth noting the strapline of the EU science policy from 2015 | 'Open Innovation, Open
Science, Open to the World'. | See: SPEECH 22 June 2015
Carlos Moedas – Commissioner for Research, Science and Innovation
Brussels,. Ending with - QUOTE - 'We owe it to the European Citizens.
We owe it to the future generations.
Let's dare to make Europe open to innovation, open to science and open to the world.' END
QUOTE Copy here as EU's own copy is no longer up
https://drive.google.com/file/d/18Ux0xGPvpb-ygNDgF3SsFtrdUu5Ds2dq/view?usp=sharing
Show less

Sylvie Vullioud
3:51 AM Today
Jon, the Brexit is not effective yet. You are still an English citizen!

Sylvie Vullioud
3:52 AM Today
So, an EU citizen!