Home

Papers

Submissions

Submission guidelines and editorial policies

Acceptance criteria

Author guidelines

Reviewer guidelines

Action Editor guidelines

Ethics guidelines

Code of conduct

Editorial board

Expert Reviewers

Contact

Frequently Asked Questions

Journal of Machine Learning Research (JMLR)

Proceedings of Machine Learning Research (PMLR)

Data-centric Machine Learning Research (DMLR)

Submission Guidelines and Editorial Policies

Scope

TMLR’s objective is to publish original papers that contribute to the understanding of the computational and mathematical principles that enable intelligence through learning, be it in brains or in machines.

To this end, TMLR invites authors to submit papers that contain

• new algorithms with sound empirical validation, optionally with justification of theoretical, psychological, or biological nature;
• experimental and/or theoretical studies yielding new insight into the design and behavior of learning in intelligent systems;
• accounts of applications of existing techniques that shed light on the strengths and weaknesses of the methods;
• formalization of new learning tasks (e.g., in the context of new applications) and of methods for assessing performance on those tasks;
• development of new analytical frameworks that advance theoretical studies of practical learning methods;
• computational models of natural learning systems at the behavioral or neural level;
• reproducibility studies of previously published results or claims;
• new approaches for analysis, visualization, and understanding of artificial or biological learning systems;
• surveys that draw new connections, highlight trends, and suggest new problems in an area.

Dual Submissions and Originality

Unlike many other journals, TMLR only accepts original contributions that don’t reuse the authors’ own prior work. In particular, we do not accept submissions that are expanded versions of conference papers. There should not be any reuse of written text, figures or results between the submitted paper and any paper which has been published, accepted for publication, or submitted in parallel at another archival, peer-reviewed venue. It is acceptable for a submission to overlap with the author's previous work if it was shared at venues or tracks that are publicly declared, in writing, to be non-archival, such as workshops, or on preprint servers such as arXiv and bioRxiv.

Review process

TMLR uses an open-reviewing, double-blind process. An open reviewing model, where papers, reviews, and discussion are visible to the public, provides several benefits, including transparency of the process, reviews which are higher-quality and more constructive, and easier communication between authors and reviewers. Since there is ample evidence that single-blind reviewing may lead to biased outcomes, we follow a double blind model where the identities of authors and invited reviewers are withheld. Reviewers are expected not to take any actions that would violate this double blind state, e.g. searching for related presentations or arXiv preprints.

The review process can be summarized in the following four phases:

1. Submission. The manuscript is submitted to OpenReview in anonymized form, and the authors recommend an action editor to oversee the review process.
2. Reviewing. An action editor will be assigned to the submission within a week. They will evaluate the submission, and may reject it immediately for being out of scope or of poor quality, or having format violations (contingent on Editor-in-Chief approval). Otherwise, the action editor will assign at least three qualified reviewers and the paper will become public. Members of the public may volunteer to review the paper; accepting such offers is up to the discretion of the action editor.
3. Rebuttal and discussion. Reviews will be visible to the authors as they are submitted, but the reviews will not be visible to the public nor to the other reviewers until all the reviews are submitted in order to keep them independent. Authors may post rebuttals and update their papers in response to the reviews, and the reviewers and editor may privately discuss the paper. Authors can respond to a review as soon as it is posted, however we recommend waiting until all 3 reviews have been submitted before submitting any revised version of the PDF manuscript. Reviewers will be able to submit their final recommendations once at least two weeks have elapsed after all 3 reviews became public.
4. Decision. The discussion period ends when the action editor enters a decision for the paper, which is one of the following:
   
   • Accept as is: the author is notified, and, once the camera ready version is submitted and verified, the manuscript is published.
   • Accept with minor revision: the author is notified of the required revisions, and, once the revised camera ready version is submitted and verified, the manuscript is published.
   • Reject: the author is notified of the rejection, and the action editor indicates whether they would be willing to consider a significantly revised version of the manuscript.  The submission remains publicly available on OpenReview and the authors are given the opportunity to reveal their identity.  Doing so will preclude them from submitting a revised version of the manuscript to TMLR. 

Authors of a rejected submission may revise and resubmit their paper, but it will need to be entered as a new submission and a link provided to the previously rejected submission as well as a description of the changes made since.  

Authors of accepted papers must provide a “camera ready” version, verified by the action editor, which would include a link to the review page for the paper. The author is encouraged to submit additional materials that can help to support their paper, such as a video presentation or code.

An author may decide to withdraw their submission before a decision has been entered, or they may decide to retract their submission after acceptance. In the former case, authors may trigger the withdrawal themselves, through OpenReview. The author will need to enter a reason for the withdrawal, and the submission will remain on OpenReview but its status will be explicitly marked as withdrawn. The authors will be given the option to reveal their identity, otherwise the submission will remain public but anonymous.

If an author decides to retract their paper after it is accepted, they will need to enter a retraction statement in OpenReview. The retraction statement will be reviewed by the Editors-in-Chief and all authors will be contacted. Once the retraction is confirmed, the paper will remain on OpenReview but its status will be marked as retracted.

The Editors-in-Chief may also decide to withdraw a submission or retract an accepted paper if it is found to contain unethical research, any plagiarized content (including figures), or findings that are erroneous (due to misconduct or honest error).

Our Ethics Guidelines describe TMLR’s ethical principles and establish common ground for editors, reviewers, and authors to evaluate the potential for harm of a proposed work. Submissions which describe work with the potential for harm should clearly describe the risks and mitigations in an (otherwise optional) Broader Impact Statement. In situations where a reviewer with ethics expertise is required to evaluate the impact and potential for harm, or where the action editor suspects that no statement would make the work ethically acceptable, reach out to the Editors-in-Chief.

Evaluation criteria

The acceptance decision for a submission is based on the answers to the following questions:

• Are the claims made in the submission supported by accurate, convincing and clear evidence?
• Would at least some individuals in TMLR's audience be interested in knowing the findings of this paper?

Papers should be accepted if they meet the criteria, even if the contribution or significance of the work is modest. Papers that should not be accepted include papers that make bold statements unsupported by empirical or rigorous evidence, papers that aren’t clearly written, papers that incorrectly claim novelty over existing published work, and papers that merely re-implement an idea that has already been reproduced before. See the Acceptance criteria for a more thorough discussion.

Certifications

Accepted papers will be automatically considered for a number of certifications that may be awarded by the action editors or editors-in-chief of TMLR. These certifications will act as public endorsements of the paper in a number of categories. The most prestigious award is the Outstanding certificate, which is jointly approved by all editors-in-chief:

• Outstanding Certification. The editorial board of TMLR jointly awards this certification to papers which are deemed to be exceptionally high quality and broadly significant for the field. The certification may be given well after the paper’s initial publication in TMLR (a year or more), and is equivalent to a best paper award at a top-tier conference.

Three other certifications are nominated by action editors and must be reviewed and approved by at least one editor-in-chief:

• Featured Certification. This certification may be awarded to papers that are very high quality. These papers present significant contributions which are novel, clearly explained, and well supported with evidence, theory, or analysis. If this paper was submitted to a top-tier conference, it would likely be presented as an oral/spotlight.
• Reproducibility Certification. This is awarded to papers whose primary purpose is reproduction of other published work. Beyond simple verification, the paper must contribute significant added value through additional baselines, analysis, ablations, or insights.
• Survey Certification. Papers that not only meet the criteria for acceptance but also provide an exceptionally thorough or insightful survey of the topic or approach may be awarded this certification.

Finally, an Expert Certification are awarded to papers that are authored by highly-rated TMLR reviewers (i.e. our Expert Reviewers), to recognize their contributions and expertise.

Open access policy

TMLR makes all published content immediately available to the public free of charge. TMLR imposes no fees or payments to authors, reviewers, action editors, or editors-in-chief.

LLM Use policy

LLMs may be used as general-purpose assistive tools. Whichever tools are used, authors and reviewers are fully responsible for content on which they are listed as (co-) authors or reviewers. This includes, but is not limited to, content generated by LLMs that could be construed as plagiarism or scientific misconduct (e.g., fabrication of facts). Low-quality contributions (be they submissions or reviews) that appear to be largely LLM-generated will be closely examined for evidence of the issues mentioned previously, such as scientific misconduct. LLMs are not eligible for authorship. We will periodically revise this policy as new information about the use of LLMs in the scientific process becomes available.

Editorial structure

TMLR uses action editors to execute the review process. The action editor role is similar to the role of area chairs in conference reviewing, but action editors are recruited for a two-year renewable term and papers are assigned throughout the year. Action editors are responsible for acceptance decisions. In addition, TMLR has three editors-in-chief, with renewable terms up to 5 years, who are responsible for assigning submissions to action editors, approving ethics and misconduct decisions, and approving some certifications. There is also a TMLR advisory board, whose members have 3 year, renewable terms.

Conflicts of interest

Authors and co-authors will use their OpenReview profile to identify domain and personal conflicts. Domain conflicts, entered in Education & Career History, are used to identify institutions with which you have a conflict of interest from (at least) the last three years. Be judicious with domain conflicts and only use them if you have a genuine conflict. It is forbidden to state a false conflict in order to block potentially negative reviewers from an institution.

For current or recent collaborations, including internships, you should generally use personal conflicts, which are recorded in the Advisors, Relations & Conflicts section. The following constitutes a personal conflict:

• Family or close personal relationship
• Ph.D. advisee/advisor relationship
• Current, frequent, or recent collaboration (where recent means within the past three years)

In some cases, you may have a personal conflict that is not covered by the definition above, but would nonetheless significantly compromise the fairness of the review process.  You may choose to mark such a conflict as hidden from your OpenReview profile by changing its visibility.  If an action editor or editor-in-chief of TMLR has any reason to doubt the validity of such a conflict, the editors-in-chief may confidentially inquire into its nature.

If a TMLR Editor-in-chief submits a paper to the journal, the other editor(s)-in-chief will choose an action editor that is not in conflict with any of the editors-in-chief to oversee the review. If a TMLR action editor submits a paper, it will be assigned to a different, non-conflicting action editor to oversee the review.

Criteria for Authorship

Individuals should be listed as authors on a TMLR submission if and only if they satisfy all of the following criteria:

• They have made a substantial intellectual contribution to the work described in the submission. This may include, but is not limited to: conception, design, or analysis of the work.
• They have contributed to drafting or revising the submission.
• They take full responsibility for all content in the final published work.

Furthermore, individuals listed as authors on a TMLR submission testify that:

• They are aware the submission has been submitted to TMLR.
• They agree to be held responsible and accountable for issues related to correctness or integrity of the work.
• To the best of their knowledge, the work submitted is original, all individuals who satisfy the authorship criteria outlined above are listed as authors, and the submission is an honest representation of the underlying work.

All authors must be listed on OpenReview at the time of submission, and may only be added later under exceptional circumstances at the discretion of the Editors-in-Chief.

TMLR gives no prescription on how to order authors, and this may be decided by the authors themselves in accordance with the customs of the subcommunity the work contributes to (e.g., alphabetically, in order of contribution, etc.). Authors may choose to include a statement indicating author contributions.

Some parts of this policy are adapted from similar policies of ACM and IEEE.

Ethical publishing

Allegations of author, reviewer, or editor misconduct should be sent to the editors-in-chief, except that if the allegations concern an EiC, the allegations may be sent directly to the personal emails of the non-accused subset of the EiCs.

To handle an allegation or discovery of misconduct, TMLR follows the Committee on Publishing Ethics’s (COPE) guidelines listed below. According to these guidelines, a determination of misconduct may result in TMLR contacting the author institutions, funding agencies, and/or potential victims.

Intellectual property, including plagiarism or redundant (duplicate) publication:

• COPE Council. Suspected plagiarism in a submitted manuscript.
• COPE Council. Suspected plagiarism in a published manuscript.
• COPE Council. Suspected redundant publication in a submitted manuscript.
• COPE Council. Suspected redundant publication in a published manuscript.

Research Fraud:

• COPE Council. Suspected fabricated data in a submitted manuscript.
• COPE Council. Suspected fabricated data in a published manuscript.

Reviewer or editor misconduct during the review process:  

• COPE Council. Reviewer suspected to have appropriated an author’s ideas or data.

TMLR follows the guidelines for retracting articles by COPE Council. In the event that TMLR retracts a publication because evidence shows that the findings are unreliable, either because of misconduct or honest error, or that the findings have been plagiarized or previously published, or that the research is unethical), a Notice of Retraction will be attached to the paper on OpenReview.

© TMLR 2024.