Ph.D. from the University of Tokyo Graduate School of Arts and Sciences; Assistant Professor and then Associate Professor at the National Institution for Academic Degrees and University Education (NIAD-UE); Professor at the National Institute for Academic Degrees and Quality Enhancement of Higher Education (NIAD-QE); Current position from 2018; also works as a member of various councils and evaluation committees at the Ministry of Education, Culture, Sports, Science and Technology, the Cabinet Office, and the Association of National Universities
Research interest:
Evaluation system, method, and index for research activities and S&T policies
A: I was studying science history in my undergraduate days. It was when the global environment issues were emerging and the issue of how the society could control the S&T development began to be recognized. Thus, I chose Technology Assessment (evaluation of technologies before they are introduced to society) as the title of the bachelor's degree thesis. This has motivated me to be interested in evaluation.
I specialized in S&T policies at the graduate school. It was the days when the systematization of research evaluation started in Europe to evaluate the policies so that the research evaluation was an important issue. Japan also established the guidelines for R&D evaluation in 1997 and how to evaluate research was the topic of those days. Prof. Ryo Hirasawa, my mentor at that time, is an expert in that area, which led me to choose evaluation of research as my main research theme.
The reason that I was interested in research evaluation with Prof. Hirasawa's guidance is that research evaluation is essential for making S&T policies. As the S&T R&D requires highly specialized knowledge, it is difficult for non-experts to comment on the activities. On the other hand, the public funds are invested in the R&D based on the expectations from the public. The expectations include academic development, economic development brought by the S&T progress, and solution of social issues such as global environment problems. The research evaluation functions to connect the policy and the public expectation. Through evaluation, the value of R&D and the reason the government invest in it are made clear. Thus, to study research evaluation means to think about the basic of S&T policies.
After completing the graduate education, I joined the National Institute for Academic Degrees and University Evaluation (NIED-UE) where I studied how to evaluate universities, including both their research and education. NIAD-UE was established when the Japanese government decided to evaluate universities afresh. I was involved in the organization's activities from the beginning of the establishment. This means that I not only conducted research there, but also actually experienced establishing the university evaluation framework, which fulfilled my interest.
A: You might think of peer review for academic journals when you hear the phrase of evaluation of research. It is for the research community to manage the quality of research activities. However, as the R&D has recently been closely connected with innovation and supported by policies, the objects of evaluation have been expanding and the evaluators and evaluation indexes have diversified. I am clarifying such transition and its background as well as extracting the issues and improvement points of the evaluation methods and evaluation indexes, and studying how to use the knowledge gained from the research in managing research in the future.
For example, such evaluation index that is based not only on the academic results, but also on the effects of the research results on economy, society, and environment is called for both in and outside of Japan. This requires establishment of indexes and development of measurement methods. To attain this, it is essential to study the systems in and outside of Japan and analyze the past evaluation results and R&D cases.
Also, the bibliometric method that is to analyze the database for papers has recently been taken in evaluating research. The method enables not simply to count the number of papers, but also to analyze their relationship with industry-university collaboration and research funds. It is important to analyze quantitatively how these indexes are related with peer review and how to use the indexes for each case. In case of evaluating a university as an organization, their organizational activities, including the research promotion policy, are analyzed via surveys. By combining these multiple methods, to be analyzed are what indexes should be taken from what kind of view, what kind of factors are needed for R&D and innovation, and what situation the university is in.
A: When I was with the NIAD-UE (it was later renamed as NIAD-QE), I made clear what kind of research evaluation indexes are used in academic fields. As it has recently become easy to use the paper database, the indexes became possible to be used incorrectly even in such a field where the research results cannot appropriately be applied to. This has been a concern to the researchers. To try to correct this, I made clear, by using the data for research evaluation process, what kind of indexes the evaluators in each field are prioritizing when making evaluation and reflected them on the evaluation items. Also, besides evaluation of research, I established guidelines on establishing a university's 'internal quality assurance system' based on complex analyses of the past evaluation results, current university status, overseas cases and other factors. The guidelines are now used in evaluating universities.
I am currently focusing on the analyses on the relationship between university evaluation and allocation of funds. Japanese national universities became independent entities in 2004. Since then, the non-competitive funds from the government to the universities have decreased by 1% every year. On the other hand, the amount of competitive funds has increased. So, the public funds allocated to the national universities have increased as a whole. Even so, the universities are not financially healthy, which is causing many issues, including no employment of new faculty members. This was caused because something is wrong with the current system. The solution is called for. The educational organizations overseas have shifted to 'performance-based funding' that is to allocate the required costs to the activities when they are endorsed by the evaluation and measurement of the education and research results. I believe that Japan should also establish a system where stable allocation is exercised, while having a competitive funding system.
A: The basic of research evaluation is peer review. It enables to evaluate various research activities and results. However, it is yet difficult to allocate the funds only based on the peer's subjective judgment. It is expected to make a good balance of qualitative index and quantitative index by, for example, taking informed peer review that is to support peer review by index.
It is also important to have a view that has impact beyond the academic field and accomplishment as mentioned above. For example, the research results in human literature and social sciences cannot be evaluated only by the number of papers and publications, but also by non-qualitative indexes such as translation of books, compilation of historical materials, and management of art museum exhibition materials. If these non-qualitative indexes are neglected, these fields will fade away. Those who are versed with the field should establish the qualitative indexes, not only the quantitative indexes. As to the social and economic impacts, the academic research are becoming to be evaluated based on the evidence for social impact. For example, politics became to be evaluated based on 'whether the research results are reflected on the policies' and medicine on 'whether the research results are applied and effective in lowering the disease rate.'
It is important for the researchers in each field to make such evaluations, but it does not mean to make evaluations closed only in one field. It is important to recognize where your research field is in the whole academic and social pictures. This is not much discussed in Japan compared with other countries, which concerns me. I believe that it is important for each academic community to make clear its value and enhance activities to communicate more with the society. The research on evaluation should contribute to support such activities.
A: I would like to pursue my research on how to exercise university reform and bring financial stability to the universities through evaluation. In addition to that, as I have been having more opportunities to meet MEXT and Cabinet Office officials since I joined GRIPS, I have more time for discussion with them on the evaluation of R&D programs than before. I would like to make maximum use of such opportunities in studying what kind of effects the Japanese government R&D funds in the past have brought to the Japanese R&D system and innovation creation.
