Evaluation Criteria for ACM Editors-in-Chief

Reviewed April 2018

The ACM Publications Board appoints all Editors-in-Chief (EiCs) for all ACM periodicals, e.g., its journals, transactions, and magazines[1]. EiCs are appointed for three-year terms, with the possibility of one three-year renewal or a second three-year renewal in the case of some ACM magazines. At the end of an EiC’s first term the Board evaluates the status of the publication and, if the EiC is willing, will consider reappointment. At that time, the Board asks for a statement from the EiC summarizing the state of the publication, the accomplishments of the preceding three years, and plans for the next term. This document outlines the criteria and metrics that the Board uses when evaluating the performance of its EiCs. Such evaluations occur formally when EiCs are being considered for reappointment. However, the underlying principles also drive consideration of new EiC candidates.

Evaluation Criteria

The overarching goals for ACM EiCs are to produce high quality content consistent with the editorial charter of the publication, and to do so in a manner consistent with the Roles and Responsibilities in ACM Publishing[2]. The following sections provide additional details.

A. Editorial

1. ACM sets high standards of quality for its publications. It is paramount that the EiC uphold these standards in the development and selection of content. The Board tracks acceptance rates, usage, and citation statistics as indirect indicators of quality for its publications.

2. Maintaining and strengthening ACM’s reputation in the scientific and scholarly community is an important goal for ACM. For ACM magazines, visibility, site activity and DL downloads are important measures of success and reputation in the community. While there are no established guidelines for minimum levels of usage, an overall positive trend over the EICs term is the expectation.

3. EiCs are responsible for faithfully executing the editorial charter of their publication. EiCs should have a clear vision of the field and the role of the publication within it. The contents of the publication should be balanced, accurately representing the full spectrum of high quality work ongoing in the field. It should not be a forum for the EiC’s own viewpoint. In the case of ACM magazines, it is customary for the EiC to regularly publish editorials that may highlight that EICs own singular opinion on particular topics or issues.

4. EiCs should be proactive in adapting the publication to changes in the field, working with the Publications Board to adjust the publication’s charter when necessary.

5. The ability of a publication to attract and select the best content is also a function of the quality of its Editorial Board. Associate Editors should have high stature and respect in the community. In this regard, appointment of pre-tenure faculty is discouraged both because of their junior standing in the field and to avoid placing them in a potential conflict of interest situation. The number and technical breadth of the editors should be sufficient to manage the publication’s workload. Finally, the editorial board should reflect the diversity of the research community with respect to geography, gender, and types of institutions represented.

6. The publication should have appropriate and well-defined procedures for refereeing and review, which are fair, unbiased, and faithfully executed.

B. Operations

1. ACM publications appear on a fixed schedule, e.g., quarterly. ACM has a legal responsibility to deliver its publications to subscribers according to the contracted schedule. It is the responsibility of the EiC to ensure delivery of editorial content to ACM HQ in accordance with this schedule. In most cases this means that all material must be at HQ at least 90 days before the first day of the month of publication.

2. EiCs should take appropriate action to deal with backlogs of accepted papers that are either too large or too small. (A two issue backlog of accepted papers is considered "healthy".)

3. EiCs should ensure that manuscripts are given a timely review, and which produces understandable and (whenever possible) actionable feedback. ACM has set a goal that each review cycle for its publications (i.e., the time from submission to each accept/revise/reject decision) take no more than 90 days on average. Data on review cycle time is available from Manuscript Central, and is monitored by the Board.

4. ACM has contracted with Clarivate to provide the Scholar One and Aries to provide the Editorial Manager submission systems for use by its publications. Centralized systems of this type provides the convenience of uniformity and stability for authors, referees, and editors, and is important for facilitating transitions between successive EiCs. In addition, it provides the Publications Board and ACM HQ with the ability to track the status of all publications. EiCs should ensure that these electronic submission systems are used in their operations and that the records maintained within it are accurate. They should make appropriate efforts to learn the system used for their journal (training is available for all EiCs and Associate Editors upon request), so that it is used as effectively as possible. Finally, problems with the system should be promptly reported to ACM HQ.

5. Each ACM publication operates a Web site to provide essential information for its authors, referees, and readers. This information includes the publication's charter, listing of the Editorial Board, information for authors, information for referees, lists of forthcoming papers, and links to the ACM Digital Library. EiCs are responsible for ensuring that the web site is effective and up-to-date. In the case of ACM magazines, the EIC’s responsibilities for maintaining and updating the publication web sites are particularly critical to the success of those publications, as in some cases the web sites and the publications are one and the same.

C. Relationships

1. EiCs should work to ensure that authors have a satisfactory experience in their interactions with the publication. Concerns and complaints of authors should be handled in a timely, courteous, and fair manner.

2. EiCs should work to ensure that referees have a satisfactory experience in their interactions with the publication. Referees should not be abused, i.e., asked for immediate turnaround, asked to evaluate papers which are clearly unsuitable or which are far outside their area of expertise, asked to review multiple papers within a short time span, etc. Referees should be appropriately recognized for their efforts.

3. EiCs should work to handle unusual cases, such as violations of ACM policy during submission and review in a timely, effective, and fair manner, informing the ACM Director of Publications of serious cases. (Note: Violations of policy discovered after publication should always be referred to ACM Headquarters.)

4. EiCs should maintain good relations with appropriate ACM SIGs. Comments of SIGs are solicited when considering EiC appointments.

5. EiCs should seek help and advice, and ensure coordination with other ACM units (e.g., HQ, the Publications Board, other EiCs, the SIGs) as necessary. In particular, timely communication of problems to ACM staff, the Publications Board and the ACM Director of Publications, as appropriate, is essential. Effective handling of submitted papers that fall outside of the journal’s scope is another example. EiCs should forward good papers to other ACM journals. If a pattern of submissions emerges for which there is no natural home within ACM publications, the EiC should consult with the Publications Board to either expand the scope of the journal or start up a new one.

D. Innovations

Finally, ACM particularly values EiCs who are proactive in developing innovations consistent with the overall goals of ACM and its publishing program. Examples of the general nature of such innovations follow.

1. Innovations in generating content and reaching out to new constituencies, such as special issues.

2. Innovative and effective use of electronic media in providing supplemental content for the ACM Digital Library, such as electronic appendices, archived data, software, and multimedia.

3. Innovations that improve operations, providing a more satisfactory experience for authors, referees, and editors.

Suggestions for EiC’s Report to the Board

EiCs are expected to provide a written report to the Board before they are formally considered for reappointment. It would be most helpful to the Board if that report specifically addressed the following points.

• Evaluate current charter or scope statement of the publication: Is it up-to-date? Has the field or the publication drifted regarding their emerging sub-areas that deserve increased emphasis, or are there new topics that the publication should consider serving? What plan is there for adjusting the publication’s area of coverage?
• For each of the evaluation criteria areas detailed above (i.e., editorial, operations, relationships, innovations), please identify the current strengths and weaknesses of the publication. What are your plans for addressing weaknesses?
• What additional support do you need from the Publications Board or ACM staff to do your job better?

An EiC may provide any additional information to the Board. This report will be circulated to the entire Publications Board, as well as to all current ACM EiCs and SIG Board Chairs for their comments. If any information supplied is confidential, it should be identified as such and submitted separately.

 

[1] We use the term “publication” in this document to refer to periodicals of this type.

[2] Roles and Responsibilities in ACM Publishing https://www.acm.org/publications/policies/roles-and-responsibilities