Evaluation of Science 2.

0
New generation of publications means new generation
of evaluation of publications

Daniele Tosi

Monday, April 12, 2010

 Framework
• Universities and research centers need some
form of evaluation of researchers

• Number and impact factor of publications

represent the common way to evaluate the
quality of researchers

• But how to extend the evaluations to 2.0

publications?

• Redefine the criteria for publications and review

Monday, April 12, 2010

 The actual situation
Monday, April 12, 2010
 Evaluation

• Number of publications
• Impact factor
• Citation index
Score ~ # Publ. x IF x CI

Monday, April 12, 2010

 Reviewing

• Double blind review

• Random reviewers,
picked up manually

• On/off review, score

transparent for readers

Monday, April 12, 2010

 Problem 1
➡ How to compare different researchers?

VS

Publications + Patents + Prototypes Publications

Monday, April 12, 2010

 Actual situation
• Standard publications dominate the evaluation
of performances for researchers
• Unfair situation: theoretical and experiments-
only researchers achieve better score
• Technology transfer does not return enough
publications, leading to an inferior score
➡ Need to evangelize a new scoring system for
scientific publications

Monday, April 12, 2010

 Problem 2
➡ How to integrate Science 2.0 publications?

VS

Publications Publications

Science 2.0

Monday, April 12, 2010

 Actual situation
• Open Science and Science 2.0 publications
are not accounted
• General-purpose publications and reports are
not included
• Penalty for scientists who switch to Science
2.0, advantage for standard publications
➡ Need to evangelize the inclusion of Science
2.0 publications into researchers’ records

Monday, April 12, 2010

 <
Publications Publications

Science 2.0

<
Publications + Patents + Prototypes Publications
Monday, April 12, 2010
 1 =
Publications Publications

Science 2.0

=
Publications + Patents + Prototypes Publications
Monday, April 12, 2010
 Step 1

• Struggle for introducing a new system for

evaluating researchers
• To set guidelines for researcher and let
them open to Science 2.0 without penalty
• To be accepted as a standard from research
institutions

Monday, April 12, 2010

 2 >
Publications Publications

Science 2.0

>
Publications + Patents + Prototypes Publications
Monday, April 12, 2010
 Step 2

• Publications 2.0 are designed to achieve a

wider read, more extended reviews and a
network connection to interested people
• They should therefore receive a higher score
• A researcher is willing to switch from
traditional to 2.0 publications

Monday, April 12, 2010

 Mission
To set 2.0 publication as the new publication standard

Monday, April 12, 2010

 Objectives

• Generate awareness of 2.0 publications

among researchers
• Provide an evaluation tool that ranks 2.0
publications and authors
• Provide a legal and official process to
embed 2.0 publications in research projects

Monday, April 12, 2010

 Strategy

✓Generate awareness of 2.0 publications

among researchers
• Viral growth from SocNet users up to
conservative scientists

• Identify decision makers for publications and

evangelize them

• Advertise equality between 1.0 and 2.0

publications

Monday, April 12, 2010

 Strategy

✓Provide an evaluation tool that ranks 2.0

publications and authors
• Evaluate and account the increase number of
document reads

• Recruit early-adopter scientists to test and

arrange a new evaluation system

• Ensure no penalty for 2.0 users

Monday, April 12, 2010

 Strategy

✓Provide a legal and official process to

embed 2.0 publications in research projects
• Identify decision makers among official research
projects commissioner and evaluators

• Broadcast 2.0 publications as a recognized output

of research projects

• Embed scientific SocNets with shortcuts for

research project management

Monday, April 12, 2010

 Framework

• This draft is NOT a real market analysis

• Comments and corrections are welcome

Monday, April 12, 2010

 Thank you
Stay tuned for the next episode

Monday, April 12, 2010