Section outline

    • Introduction to Responsible Research Assessment (RRA)

    • What is Research Assessment?

      Research assessment systematically evaluates the quality, impact, rigor, and significance of scholarly work. It significantly influences crucial decisions related to funding, hiring, promotions, tenure, and institutional strategies. Traditional approaches primarily utilize quantitative metrics, such as Journal Impact Factors and citation counts, to gauge research success.

      Why Reform is Necessary

      Traditional research assessment practices tend to prioritize publication quantity and journal prestige, often neglecting qualitative measures and diverse scholarly outputs such as datasets, software, community engagement, mentoring, and interdisciplinary collaboration. Such practices can lead researchers to prioritize quantity over genuine quality and societal impact, thereby undermining innovation, fairness, inclusivity, and transparency in scholarly evaluation.

    • Multiple global initiatives have driven significant advancements in responsible research assessment practices:

      A. San Francisco Declaration on Research Assessment (DORA, 2013)

      • Advocates against using Journal Impact Factors for individual researcher evaluations.

      • Promotes evaluation of research based on the actual content and diversity of outputs (e.g., software, data, mentorship).

      • Emphasizes transparency, inclusivity, and a holistic evaluation approach.

      B. Leiden Manifesto (2015)

      • Presents ten guiding principles for the responsible use of research metrics.

      • Reinforces the importance of expert judgment alongside appropriate quantitative metrics.

      • Encourages transparency, contextual sensitivity, and regular assessment method evaluations.

      C. Coalition for Advancing Research Assessment (CoARA, 2022)

      • Establishes a global coalition committed to systemic reform of research assessment.

      • Balances qualitative insights with responsibly applied quantitative indicators.

      • Emphasizes accountability, transparency, inclusivity, and ethical oversight.

      CoARA has established 13 operational working groups tackling the reform of research assessment from different perspectives. These include working groups on open infrastructures, multilingualism, responsible use of metrics, artificial intelligence and ethics, narrative CVs, and more. One of these groups specifically focuses on developing principles, frameworks, and tools for Open Infrastructures.

      Open Infrastructures are not merely supportive tools; they are the backbone of a modern research assessment ecosystem. By offering transparent, community-governed, interoperable systems, they create the foundational layer that enables all other reforms to take root and scale. Without open infrastructures, efforts to adopt narrative CVs, multilingual evaluation practices, responsible metrics, and ethical AI would remain fragmented and difficult to operationalize. Their role is critical in ensuring that assessment practices are FAIR (Findable, Accessible, Interoperable, Reusable), trustworthy, and resilient over time.

      Open Infrastructures: A Foundation for Implementing RRA

      Open infrastructures (OIs) are essential for effectively implementing responsible research assessment, aligning with global reform initiatives like CoARA, DORA, and the Leiden Manifesto. OIs offer transparent, inclusive, interoperable, and ethically grounded environments to ensure a fairer and more holistic approach to research evaluation. The CoARA Working Group ‘’Towards Open Infrastructures for Responsible Research Assessment’’ has conducted related work in mapping the frameworks and principles or OIs fit for RRA (Open Infrastructures for Responsible Research Assessment: Principles and Framework):

      Core Characteristics of Open Infrastructures

      • Technical Robustness: Ensures data integrity, traceability, reproducibility, interoperability, and scalability. Transparent integration of advanced technologies like AI, Explainable AI (XAI), and NLP enhances accuracy and fairness.

      • Operational Efficiency: Offers streamlined workflows to reduce administrative burden, while valuing a wide range of scholarly contributions. Continuous capacity-building maintains stakeholder engagement and proficiency.

      • Community-Centred Governance: Promotes inclusive governance with active participation from diverse stakeholders, ensuring transparency, adaptability, and sustainability.

      • Ethical and Inclusive Practices: Maintains fairness, ethical data management, transparent technology application, and robust ethical oversight to guarantee impartial assessments.

      Implementation: Strategy and Practical Steps

      To effectively implement responsible research assessment using open infrastructures:

      • clear, institution-specific action plans guided by the CoARA framework.

      • Offer regular training and resources to stakeholders to ensure understanding and active participation.

      • Establish periodic evaluation cycles, incorporating community feedback to ensure ongoing improvement and adaptability.

      Benefits of Adopting Responsible Research Assessment

      Adopting responsible research assessment through open infrastructures:

      • Promotes genuine innovation, interdisciplinary collaboration, and societal impact.

      • Strengthens transparency, fairness, equity, and inclusivity in research evaluation processes.

      • Enhances institutional accountability and builds trust among researchers and stakeholders.

      Conclusion and Call to Action

      Implementing responsible research assessment practices through open infrastructures and aligning with international reform initiatives is essential for fostering inclusive, fair, and impactful scholarly environments. Stakeholders must actively engage and commit to these transformative reforms to ensure ongoing excellence, integrity, and societal relevance in research.

    • From Transition to Adoption — Practical Steps and Best Practices

    • Overview

      This module guides institutions and stakeholders through the transition process from traditional assessment to Responsible Research Assessment (RRA) using Open Infrastructures.

      Transition Roadmap

      • Assess Current Practices: Conduct an institutional audit of existing assessment procedures, metrics, and tools.

      • Define Objectives: Set clear goals for adopting RRA (e.g., improving transparency, recognizing diverse outputs).

      • Engage Stakeholders: Involve researchers, administrators, and funders early to build buy-in and co-create strategies.

      Adoption Steps

      1. Build on and follow related Policies: Draft or revise assessment policies to reflect RRA principles and frameworks.

      2. Select or Build Open Infrastructures: Identify systems meeting CoARA criteria or develop internal systems aligned with openness and inclusivity.

      3. Pilot Implementation: Run small-scale pilots to test workflows, data quality, and user experience.

      4. Training and Capacity Building: Provide training on new tools, open metrics, and narrative CVs.

      5. Evaluate and Scale: Gather feedback, measure impact, and refine processes before institution-wide rollout.

      Best Practices

      • Transparency: Communicate policies and criteria clearly.

      • Inclusivity: Recognize contributions beyond publications.

      • Flexibility: Adapt frameworks to discipline-specific contexts.

      • Continuous Improvement: Establish cycles of review, feedback, and refinement.

      Conclusion

      A deliberate, participatory approach ensures sustainable adoption of responsible research assessment, empowering institutions to drive excellence and fairness in research evaluation.