Retracting CS (conference) papers

1,805

Correcting errors in conference papers is certainly possible---I have done so myself.

Most reputable peer-reviewed conferences are associated with some sort of society or organization that maintains their archives, e.g., IEEE, ACM, AAAI, and that society will generally have a process for correcting or retracting their conference publications that looks much like their process for correcting or retracting the articles of the journals they maintain.

In many cases, however, a conference paper is superseded by a following journal paper, which effectively becomes the final and archival version. In this case, while the authors have been less than honest about the flaw in their original paper, the later correction means that adjusting this particular paper in the archive is likely not a critical issue.

Share:
1,805
Blaisorblade
Author by

Blaisorblade

Updated on September 14, 2020

Comments

  • Blaisorblade
    Blaisorblade about 3 years

    Suppose you learn of a confirmed fundamental error in a CS conference paper (for a reputable conference) that smells of gross negligence. Is there any way to get it retracted? Whenever I've asked, people told me that CS conferences don't do that—and I've surely never seen it for PL conferences (though this paper is on algorithms).

    While we can't talk specifics, please assume I've already done all the due diligence to confirm the error, down to getting the authors to confirm it.

    I understand such errors won't kill people directly as errors in life sciences could, but I'm still concerned. Maybe a more serious version of https://xkcd.com/386/, but still.

    The specific case

    Years ago (around 2009) I've confirmed the flaw with other experts of the field (let's call them A), who confirmed it with the authors of the paper (let's call them B). I was just an undergrad and not working in the field, so I didn't participate in the conversation. This paper is still cited years later by the authors (as a "preliminary version") without mentioning the error—though a later paper citing it changes a key equation and (maybe) fixes the problem.

    *Those experts avoided publicizing the flaw out of professional courtesy—making enemies of the author would allegedly have risked hurting their publication chances. However, they (had to) go as far as citing the paper without mentioning issues in follow-up work. Out of respect, I hence won't mention even the subfield of algorithms or the conference.

    I'm also outside of the field—I got involved simply because I was friends with one undergrad doing research with experts A and he just asked me help when he couldn't make sense of the paper by B.

    EDIT: I forgot to specify why I say "gross negligence", and I agree that was bad on my part. Here are the neutral facts as best as I can recall them/verify them:

    • the included pseudocode has a type error in a crucial routine—if you index a matrix you need two indexes, not one. We started investigating because we couldn't make sense of the pseudocode. This was on a second paper specifically about the implementation. EDIT: All 8 accesses to the array in 21 lines have this type error, so it's no typo.
    • the algorithm fails on the running example. EDIT: by running example here meant the example used throughout the paper. But I rechecked and the offending example is just the 2nd they use.
    • the paper implements and benchmarks the algorithm it describes against a naive implementation, yet it didn't detect that the algorithm results are (EDIT) incorrect. After you've done 90% of the work, comparing results is pretty easy.

    I wouldn't talk about negligence for an error in a correctness proof (which the paper seems to implicitly claim). And maybe the above is not gross negligence in this setting, but it does feel rather sloppy. I'm happy to be corrected though.

    EDIT2:

    • The question starts with "Suppose you learn [of a case that...] smells of gross negligence", then focuses on retraction (not misconduct). You can question that assumption only if no such case exists. I do sketch one example but that's just supporting evidence; the focus of the question cannot be trying the specific case. I'm fine with also discussing this case, but I'm just trying to argue plausibility.
    • I still think in such a context one should compare expected and actual results, at least once you have implemented both. The guidelines of the German Science Foundation say "good scientific practice includes the following fundamental principles:... consistently and critically questioning all results". Which sounds common sense to me.
    • I'm sure aware of honest errors (which should also be corrected somehow). Nobody expects all proofs to be error-free and formalized.
    • Google suggests "negligence" can only be "gross" if it endangers people's life. I'm surely not suggesting that. If that's how it's understood, I apologize for that and amend to negligence.
    • Jon Custer
      Jon Custer about 7 years
      Tossing around 'gross negligence' with respect to having an error in a paper makes this a highly loaded question. All researches make (and publish) 'errors' - we have early results, we misinterpret, we find out more later and reanalyze the problem. And then, we write a better, hopefully more correct paper. Now, the original paper with an error may still be useful to cite if it was the first to raise and describe the problem, even if the 'solution' was wrong.
    • Blaisorblade
      Blaisorblade about 7 years
      The first paper cites 3 other solutions to the same problem and already a few variants. (Also I've edited the question to clarify whence this comes from, which I should have done earlier).
    • Davidmh
      Davidmh about 7 years
      The first "gross negligence" looks like a typo. The second, a setup-specific quirk, or that they submitted the wrong example. The third can be sloppy analysis, or that they didn't bother to mention thinking it was obvious.
    • Blaisorblade
      Blaisorblade about 7 years
      @Davidmh Based on your comment I must have been unclear and tried clarifying what I meant.
    • FooBar
      FooBar about 7 years
      "if you index a matrix you need two indexes, not one." This is wrong. Just declare if the matrix that the matrix is row-major and access entries as [i+j*#cols] instead of [i,j] and you are good to go. (And that's how matrices are implemented in the end)
    • Blaisorblade
      Blaisorblade about 7 years
      @FooBar I'm aware of row-major order, but nowhere else is this paper using this notation and it doesn't make sense here. We tried repairing the routine, but we managed to prove that the rest of the algorithm can't work whatever this routine does—the "type error" was just a starting point. But frankly, once the authors confirmed the paper is faulty, and since I won't name the paper (and nobody here does), I don't think it's productive to defend a paper you have not seen.
  • O. R. Mapper
    O. R. Mapper about 7 years
    "Whoever is interested will not trust the original conference paper anyway" - this is somewhat field-specific. In some CS subfields, the conference paper is the main thing, not necessarily followed up by any other article.
  • Uwe
    Uwe about 7 years
    I don't know whether the situation has changed since then, but when I asked Springer in 2008 whether is it possible to add a revised version of an LNCS contribution (or a link to an externally stored revised version) to SpringerLink, in case the published version contains a factual error, the answer was "We are not permitted to add or change anything to the lncs papers on SpringerLink. The only thing we could try to do would be to introduce an erratum text explaining the error. We can apply for this, but our erratum department may not approve it."
  • Karl
    Karl about 7 years
    @O.R.Mapper This doesn't say good things about the relevance of said paper, does it? I mean if it is the only paper, and well cited, OK, but otherwise, i'll say it was just published because the author needed to be first author of another paper to get his PhD. ;-) Admittedly i'm not familar with the customs in CS.
  • O. R. Mapper
    O. R. Mapper about 7 years
    "This doesn't say good things about the relevance of said paper, does it?" - it doesn't allow for any conclusion about the relevance of the paper. "Admittedly i'm not familar with the customs in CS." - no, obviously you are not :) (Note that CS is diverse. The customs vary even between subfields.)
  • Karl
    Karl about 7 years
    Well. The OP says the conference paper was followed by a regular one, which he thinks was done correctly. And be that as it may, i do not believe that people in CS or wherever are not very suspicious when reading conference papers. They are writtten with a fixed deadline, and are not given the same care as regular papers by the average reviewer.
  • Karl
    Karl about 7 years
    Where is an erratum (which is in fact a new article with doi and everything) for a conference paper published? If it's a special issue of a regular journal, OK, next issue, but otherwise? In a year, with the proceedings of the next conference? Not likely. Of course it would be simple to just publish it online, but that's still not the way journals work, is it?
  • O. R. Mapper
    O. R. Mapper about 7 years
    Well, the OP also says they are "outside of the field" and admittedly, journal papers are indeed the primary publication channel in most fields. Still, for all intents and purposes, in some fields, conference papers are regular papers.
  • jakebeal
    jakebeal about 7 years
    I don't know how Springer does (or doesn't) do it, but IEEE attaches a correction to the "abstract" page in IEEE Xplore.
  • Blaisorblade
    Blaisorblade about 7 years
    I'm indeed from a different field of CS, but multiple sources confirm that across CS conference papers are often what matters most, see e.g. academia.stackexchange.com/q/38086/8966.
  • Blaisorblade
    Blaisorblade about 7 years
    Ah, found info on errata for IEEE and Springer, but not ACM, at: supportcenter.ieee.org/app/answers/detail/a_id/1012/~/… and springer.com/gp/authors-editors/journal-author/…