Quelques repères pour faire une relecture d’article médico-scientifique.
Auteurs et références
- Dr MAEKER Eric
- Médecin gériatre et psychogériatre, France.
- Président de l’association Emp@thies, pour l’humanisation des soins. empathies.fr
- MAEKER-POQUET Bérengère
- Infirmière diplômée d’État, France.
- Secrétaire de l’association Emp@thies, pour l’humanisation des soins. empathies.fr
- Correspondance : eric.maeker@gmail.com
- Les auteurs ne déclarent aucun conflit d’intérêts.
Mots clés MeSH pour la recherche de bibliographie
- Peer Review. Une procédure organisée et menée par un comité restreint de professionnels pour évaluer la performance d’autres professionnels dans le respect des normes de leur spécialité. L’examen par les pairs est utilisé par les éditeurs dans l’évaluation des articles et autres articles soumis pour publication. L’examen par les pairs est également utilisé dans l’évaluation des demandes de subvention. Elle s’applique également à l’évaluation de la qualité des soins de santé fournis aux patients. Année de lancement : 1971
- Peer Review, Research. L’évaluation par des experts de la qualité et de la pertinence des recherches ou des propositions de recherche d’autres experts dans le même domaine. L’examen par les pairs est utilisé par les éditeurs pour décider quelles soumissions méritent d’être publiées, par les organismes subventionnaires pour déterminer quelles propositions devraient être financées, et par les établissements universitaires dans les décisions de titularisation. Année de lancement : 1994
- Peer Review, Health Care L’examen simultané ou rétrospectif par des médecins praticiens ou d’autres professionnels de la santé de la qualité et de l’efficacité des pratiques de soins aux patients ou des services prescrits ou exécutés par d’autres médecins ou d’autres professionnels de la santé (tiré du Dictionnaire des faits sur la gestion des soins de santé, 1988). Année de lancement : 1994
Quelques repères bibliographiques utiles
- Voir les instructions aux évaluateurs du BMJ Open,
Effets du Peer Reviewing
. Effects of editorial peer review: a systematic review. JAMA. 2002 Jun 5;287(21):2784-6. doi: 10.1001/jama.287.21.2784.
[PMID: 12038911] [DOI: 10.1001/jama.287.21.2784] Context: Editorial peer review is widely used to select submissions to journals for publication and is presumed to improve their usefulness. Sufficient research on peer review has been published to consider a synthesis of its effects.
Methods: To examine the evidence of the effects of editorial peer-review processes in biomedical journals, we conducted electronic and full-text searches of private and public databases to June 2000 and corresponded with the World Association of Medical Editors, European Association of Science Editors, Council of Science Editors, and researchers in the field to locate comparative studies assessing the effects of any stage of the peer-review process that made some attempt to control for confounding. Nineteen of 135 identified studies fulfilled our criteria. Because of the diversity of study questions, methods, and outcomes, we did not pool results.
Results: Nine studies considered the effects of concealing reviewer/author identity. Four studies suggested that concealing reviewer or author identity affected review quality (mostly positively); however, methodological limitations make their findings ambiguous, and other studies' results were either negative or inconclusive. One study suggested that a statistical checklist can improve report quality, but another failed to find an effect of publishing another checklist. One study found no evidence that training referees improves performance and another showed increased interrater reliability; both used open designs, making interpretation difficult. Two studies of how journals communicate with reviewers did not demonstrate any effect on review quality. One study failed to show reviewer bias, but the findings may not be generalizable. One nonrandomized study compared the quality of articles published in peer-reviewed vs other journals. Two studies showed that editorial processes make articles more readable and improve the quality of reporting, but the findings may have limited generalizability to other journals.
Conclusions: Editorial peer review, although widely used, is largely untested and its effects are uncertain.
[PMID: 12038911] [DOI: 10.1001/jama.287.21.2784] Context: Editorial peer review is widely used to select submissions to journals for publication and is presumed to improve their usefulness. Sufficient research on peer review has been published to consider a synthesis of its effects.
Methods: To examine the evidence of the effects of editorial peer-review processes in biomedical journals, we conducted electronic and full-text searches of private and public databases to June 2000 and corresponded with the World Association of Medical Editors, European Association of Science Editors, Council of Science Editors, and researchers in the field to locate comparative studies assessing the effects of any stage of the peer-review process that made some attempt to control for confounding. Nineteen of 135 identified studies fulfilled our criteria. Because of the diversity of study questions, methods, and outcomes, we did not pool results.
Results: Nine studies considered the effects of concealing reviewer/author identity. Four studies suggested that concealing reviewer or author identity affected review quality (mostly positively); however, methodological limitations make their findings ambiguous, and other studies' results were either negative or inconclusive. One study suggested that a statistical checklist can improve report quality, but another failed to find an effect of publishing another checklist. One study found no evidence that training referees improves performance and another showed increased interrater reliability; both used open designs, making interpretation difficult. Two studies of how journals communicate with reviewers did not demonstrate any effect on review quality. One study failed to show reviewer bias, but the findings may not be generalizable. One nonrandomized study compared the quality of articles published in peer-reviewed vs other journals. Two studies showed that editorial processes make articles more readable and improve the quality of reporting, but the findings may have limited generalizability to other journals.
Conclusions: Editorial peer review, although widely used, is largely untested and its effects are uncertain.
Outils d’évaluation de la qualité
. Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research. BMJ Open. 2020 Jun 8;10(6):e035604. doi: 10.1136/bmjopen-2019-035604.
[PMID: 32518211] [PMCID: 7282387] [DOI: 10.1136/bmjopen-2019-035604] Objective: To develop a tool to assess the quality of peer-review reports in biomedical research.
Methods: We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis.
Results: A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1-5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer's comments).
Conclusion: Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers' work, and also as an outcome when evaluating interventions to improve the peer-review process.
[PMID: 32518211] [PMCID: 7282387] [DOI: 10.1136/bmjopen-2019-035604] Objective: To develop a tool to assess the quality of peer-review reports in biomedical research.
Methods: We conducted an online survey intended for biomedical editors and authors. The survey aimed to (1) determine if participants endorse the proposed definition of peer-review report quality; (2) identify the most important items to include in the final version of the tool and (3) identify any missing items. Participants rated on a 5-point scale whether an item should be included in the tool and they were also invited to comment on the importance and wording of each item. Principal component analysis was performed to examine items redundancy and a general inductive approach was used for qualitative data analysis.
Results: A total of 446 biomedical editors and authors participated in the survey. Participants were mainly male (65.9%), middle-aged (mean=50.3, SD=13) and with PhD degrees (56.4%). The majority of participants (84%) agreed on the definition of peer-review report quality we proposed. The 20 initial items included in the survey questionnaire were generally highly rated with a mean score ranging from 3.38 (SD=1.13) to 4.60 (SD=0.69) (scale 1-5). Participants suggested 13 items that were not included in the initial list of items. A steering committee composed of five members with different expertise discussed the selection of items to include in the final version of the tool. The final checklist includes 14 items encompassed in five domains (Importance of the study, Robustness of the study methods, Interpretation and discussion of the study results, Reporting and transparency of the manuscript, Characteristics of peer reviewer's comments).
Conclusion: Assessment of Review reports with a Checklist Available to eDItors and Authors tool could be used regularly by editors to evaluate the reviewers' work, and also as an outcome when evaluating interventions to improve the peer-review process.
. Tools used to assess the quality of peer review reports: a methodological systematic review. BMC Med Res Methodol. 2019 Mar 6;19(1):48. doi: 10.1186/s12874-019-0688-x.
[PMID: 30841850] [PMCID: 6402095] [DOI: 10.1186/s12874-019-0688-x] Background: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research.
Methods: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis.
Results: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of 'quality'. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer's comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18).
Conclusion: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.
[PMID: 30841850] [PMCID: 6402095] [DOI: 10.1186/s12874-019-0688-x] Background: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research.
Methods: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis.
Results: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of 'quality'. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer's comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18).
Conclusion: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.
Rôles et tâches du relecteur
. A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals. BMC Med. 2019 Jun 20;17(1):118. doi: 10.1186/s12916-019-1347-0.
[PMID: 31217033] [PMCID: 6585141] [DOI: 10.1186/s12916-019-1347-0] Background: Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals.
Methods: Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes.
Results: After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers' tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items).
Conclusions: Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.
[PMID: 31217033] [PMCID: 6585141] [DOI: 10.1186/s12916-019-1347-0] Background: Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals.
Methods: Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes.
Results: After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers' tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items).
Conclusions: Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.
Le Co Peer Reviewing
. How to bring peer review ghostwriters out of the dark. Mol Biol Cell. 2021 Mar 15;32(6):461-466. doi: 10.1091/mbc.E20-10-0642.
[PMID: 33720779] [PMCID: 8101444] [DOI: 10.1091/mbc.E20-10-0642] Early career researchers are frequent and valuable contributors to peer review. Systemic changes that acknowledge this fact would result in ethical co-reviewing, peer reviews of greater quality, and a reduction in peer reviewer burden.
[PMID: 33720779] [PMCID: 8101444] [DOI: 10.1091/mbc.E20-10-0642] Early career researchers are frequent and valuable contributors to peer review. Systemic changes that acknowledge this fact would result in ethical co-reviewing, peer reviews of greater quality, and a reduction in peer reviewer burden.
En résumé
Modèle proposé de rapport d’évaluation
Titre du manuscrit :
1 Notes concernant l’évaluateur
L’évaluateur ne déclare aucun conflit d’intérêts concernant l’article ci-dessous analysé. Il assure la complète confidentialité des documents qui lui ont été transmis.
2 Commentaire général
Résumé de la proposition
Appréciation globale du manuscrit
Quelle est la première question que le l’évaluateur se pose sur le sujet ? Et trouve-t-il une réponse à cette question ?
L’évaluateur trouve-t-il des publications similaires dans le domaine analysé ? Si oui, comment se positionne la présente étude par rapport à cette bibliographie générale ?
3 Pertinence et originalité de l’étude
4 Adhésion aux codes d’éthique médico-scientifique
5 Interprétation des résultats de l’étude
6 Robustesse de l’étude
Discuter les forces et faiblesses des méthodes d’étude (par exemple, la conception de l’étude, les résultats, le risque de biais, etc.)
Disponibilité des données et du matériel d’étude ?
6.1 Général
6.2 Méthodes
6.3 Méthodes statistiques
Adéquation des méthodes statistiques ?
7 Interprétation et discussion des résultats de l’étude
Les conclusions de l’étude répondent-elles à la ou aux questions de recherche et résument-elles correctement les résultats de l’étude ?
Les limites de l’étude sont-elles reconnues?
Comment sont évalués l’applicabilité et la généralisation (validité externe) des résultats de l’étude ?
8 Organisation et présentation du manuscrit
8.1 Corrections à apporter dans le texte
Noter les coquilles, et évaluer le niveau de langage.
Apprécier le niveau de congruence du manuscrit avec les recommandations aux auteurs de la revue.
8.2 Organisation du manuscrit
8.3 Présentation des tableaux
8.4 Présentation des figures
8.5 Présentation des références
8.6 Présentation des annexes
9 Conclusions de l’évaluateur
- Accepté en l’état
- Révisions mineures
- Révisions majeures.
- Révisions majeures. L’évaluateur apprécierait de réviser les corrections.
- Proposition à rejeter
10 Références utilisées pour l’évaluation
Modèle proposé de rapport d’évaluation (en texte simple)
Titre du manuscrit : 1. Notes concernant l’évaluateur L’évaluateur ne déclare aucun conflit d’intérêts concernant l’article ci-dessous analysé. Il assure la complète confidentialité des documents qui lui ont été transmis. 2. Commentaire général Résumé de la proposition Appréciation globale du manuscrit Quelle est la première question que le l’évaluateur se pose sur le sujet ? Et trouve-t-il une réponse à cette question ? L’évaluateur trouve-t-il des publications similaires dans le domaine analysé ? Si oui, comment se positionne la présente étude par rapport à cette bibliographie générale ? 3. Pertinence et originalité de l’étude 4. Adhésion aux codes d’éthique médico-scientifique 5. Interprétation des résultats de l’étude 6. Robustesse de l’étude Discuter les forces et faiblesses des méthodes d’étude (par exemple, la conception de l’étude, les résultats, le risque de biais, etc.) Disponibilité des données et du matériel d’étude ? 6.1. Général 6.2. Méthodes 6.3. Méthodes statistiques Adéquation des méthodes statistiques ? 7. Interprétation et discussion des résultats de l’étude Les conclusions de l’étude répondent-elles à la ou aux questions de recherche et résument-elles correctement les résultats de l’étude ? Les limites de l’étude sont-elles reconnues? Comment sont évaluées l’applicabilité et la généralisation (validité externe) des résultats de l’étude ? 8. Organisation et présentation du manuscrit 8.1. Corrections à apporter dans le texte Noter les coquilles, et évaluer le niveau de langage. Apprécier le niveau de congruence du manuscrit avec les recommandations aux auteurs de la revue. 8.2. Organisation du manuscrit 8.3. Présentation des tableaux 8.4. Présentation des figures 8.5. Présentation des références 8.6. Présentation des annexes 9. Conclusions de l’évaluateur - Accepté en l’état - Révisions mineures - Révisions majeures. - Révisions majeures. L’évaluateur apprécierait de réviser les corrections. - Proposition à rejeter 10. Références utilisées pour l’évaluation