“Natural Language Processing” Referee Guidelines

These Guidelines set forth the roles of the Editorial Committee, referees, and authors of papers submitted for publication in the Journal of Natural Language Processing (including papers, reports and survey papers, hereinafter “Paper(s)”). Note that non-Paper manuscripts and commissioned manuscripts will first be read by one editor in charge, who will request revisions to the author if necessary, and then approval or denial will be decided by the Editorial Committee.

1. Accepting submissions

The Editorial Committee will send a notification of receipt indicating the date of receipt and receipt number. Incomplete submissions will be rejected and returned. The manuscript will be returned without a review, if the Editorial Committee determines that it falls short of qualifying for a review.

2. Designation of the editor in charge

After receipt of the Paper, the Editorial Committee will designate one editor in charge.

3. Selection of referees

Based on the recommendation of the editor in charge, the Editorial Committee will choose two referees. It is preferable for the two referees to be in different fields of expertise. The editor in charge will send a copy of the Paper to each referee, and request refereeing. If the referee accepts the request, the Editorial Committee will send a Review Form to the referee, and within one month of receipt of the Form, the referee will report the refereeing result to the editor in charge and the office. If the referee declines the request, a new referee will be selected in the same manner and requested to review the Paper.

4. Refereeing

Members' interests
Whether the manuscript addresses a topic that interests members of the Association for Natural Language Processing

Credibility
Whether methods, experiments and arguments presented in the manuscript are logical and credible

Organization and Readability
Whether the manuscript is properly organized and coherent without ambiguous expressions

Usefulness and Novelty

1. General Paper
   Usefulness
   Whether the manuscript is useful from academic, engineering or social perspectives.
   Novelty
   Whether the manuscript clearly shows research contributions in the subject area in relation to other published research work and presents new findings. The new findings here refer not only to new technologies and methods. As an example, if the author discusses standards and design policies for creating language resources, presenting findings beneficial to readers, etc., the manuscript is deemed to be novel. As another example, if the author applies existing methods to a new area or data and thus presents new findings based on examination of the research results, the manuscript is deemed to be novel.
   
2. System Paper
   Usefulness
   It is desirable to show as objectively as possible that a constructed system can deal with realistic problems. If it includes objective evaluation, a manuscript is positively evaluated. Even if its performance is difficult to quantitatively evaluate, a proposed system may still be quantitatively evaluated in terms of, for example, how much it is used in the real world. If the quantitative evaluation cannot be indicated due to certain circumstances, the usefulness of a system is still demonstrable through clearly authorized qualitative evaluation (questionnaire results from the users, etc.) after specifying the reason for the decision.
   Novelty
   Even with a combination or integration of existing technologies, novelty of combination, novelty of the whole system, novelty of the design concept, novelty of the insights obtained by the designed /developed system are subjects for novelty. It is not necessarily required for a manuscript to include completely new research and development technology.
   Other
   Although they might not be concerned in theory, problems that must be solved when developing a practical system, as well as their solution method and evaluation (or observation) are very useful knowledge and should be positively evaluated.
   
3. Technical Report
   Regardless of “novelty”, “usefulness” will be the main point for evaluation.
   Usefulness
   Is it useful for academic, engineering or social purposes?
   
4. Survey Paper
   Usefulness
   Has prior research been thoroughly investigated?
   Are the essential citations provided?
   Is the overall view of the techniques concerned understood?
   Is the described content considered concrete and reliable?
   Novelty
   Is it summarized from a new viewpoint / perspective?
   

Policy regarding data sharing and reproducibility
It is not mandatory for the linguistic data (corpora) and software programs used in the paper to be publicly available. However, if the authors have made them available to the public, it will be considered as a plus from the viewpoint of usefulness and reproducibility (credibility).

Policy regarding citations of relevant previous studies
Regardless of whether a cited study is made public in the form of a journal article, an oral presentation, or a preprint, if it becomes available more than three months before the submission of a paper, it is in principle regarded as a previous study. However, referees adopt the following policies flexibly when reviewing a paper:

1. It is desirable for authors to cite relevant studies that become available less than three months prior to the submission of their respective papers.
2. If a relevant preprint becomes available more than three months prior to submission but is not widely cited, authors may be excused for not being aware of it.

After reviewing the Paper using the above criteria, the referee will make one of the following decisions, and inform the editor in charge of the decision by returning a completed Review Form (comments and inquiries to be included as needed).

A Approved for publication
B Inquire author
The author will be contacted for some answers or revisions, after which the Paper will be refereed again.
B1 Conditional acceptance
Only when inquiries are on minor matters and do not affect the main part of the Paper.
B2 To be refereed after inquiry
Only when there is sufficient prospect of improvement after matters such as errors, unexplained points, dubious points, incomplete format as a Paper, redundancy, need for additional description, etc., are inquired into and remedied.
C Denied publication
In the following cases, the manuscript will be denied publication, and returned to the author with the reason for denial.

1. When the manuscript is written in a field unrelated to the Association.
2. When the manuscript contains one or more essential errors, or the content is not objectively discernable.
3. When the level of content is low and lacks novelty and importance.
4. When the Paper has already been published in another journal or magazine, or its content has already been publicly known. (A manuscript that has already been published in periodical publications, such as other academic journals, corporate magazines and general magazines will not be accepted for publication. However, the material used in an oral presentation at an academic society’s meeting, workshop, symposium or international conference may be accepted for publication only when permission is given by the copyright owner.)
5. When the content is excessively difficult or incomplete, with no prospect of improvement.
6. When the Association for Natural Language Processing determines that it does not sufficiently emphasize “8 Precautions when writing” in the Manuscript Guide to Journal Publication.

5. Handling of referee results

Based on the reports from the referees, the editor in charge will take one of the following measures:

1. The editor in charge will inform the Editorial Committee of the evaluation result and the reasons behind it.
2. The editor in charge will request the author to answer all inquiries received from referees.

6. Inquiries

When inquiring the author, the Editorial Committee Office communicates matters to be inquired to the author, wait for a reply, and then request re-refereeing to the referees. Inquiries must be replied to in writing. When making changes to the manuscript in relation to the inquiries, clearly indicate where the changes were made and why, and send the revised manuscript to the Editorial Committee Office.

7. Decision for approval/denial

Upon reporting the decision, the editor in charge will submit all versions of the Paper and the Referee Reports to the Editorial Committee, explain the referee content at a committee meeting, and announce the decision of approval or denial as the editor in charge.

The Editorial Committee will make the final decision on approval/denial of the Paper based on the report from the editor in charge, and the Editorial Committee office will notify the result to the author. If approved, the Committee will also inform the author when the Paper is planned to be published, and request author introduction.

8. Non-disclosure of names

• The names and affiliations of the editor in charge and the referees will remain undisclosed to the author.

9. Contact from the author

Authors will contact the Office for any inquiries or withdrawal of the Paper.

10. Papers by editors

Members of the Editorial Committee will not participate in any proceedings related to Papers written by them.

A diagrammatic overview of the above procedure is shown below.

    Submission      Designation of the     Request for     I.Referee report         Notification
                    editor in charge       refereeing                               of decision
Author ----> Editorial -----------> Editor ---------> Referee -------> Editor in charge ----> Editorial 
             Committee              in charge          ↑               and Office             Committee
                                                       ↑                   ↓                      |
                              Request for re-refereeing|                Office                    |Notification
                                                       |      II.Inquiries ↓                      ↓of decision
                                                     Office <-----------Author                  Author 
                                                                 Reply

Typical time required for the review process
(1) from Submission to Selection of referees: within three weeks
(2) Refereeing: within one month
(3) from Referee report to Notification of decision: within one week
(4) from Inquiries to Reply: according to "The Editing Schedule"