• 29 Posts
  • 236 Comments
Joined 1 year ago
cake
Cake day: June 22nd, 2023

help-circle
  • In computer science, garbage in, garbage out (GIGO) is the concept that flawed, biased or poor quality (“garbage”) information or input produces a result or output of similar (“garbage”) quality. The adage points to the need to improve data quality in, for example, programming.

    There was some research article applying this 70s computer science concept to LLMs. It was published in Nature and hit major news outlets. Basically they further trained GPT on its output for a couple generations, until the model degraded terribly. Sounded obvious to me, but seeing it happen on the www is painful nonetheless…











  • adr1an@programming.devtoScience Memes@mander.xyzMental hell
    link
    fedilink
    English
    arrow-up
    12
    ·
    edit-2
    6 days ago

    It’s difficult to know if this might just be a correlation to age of onset, plus the effect of new project or work. Maybe other graduates of similar careers (i.e. chemists going into big pharma) that doesn’t go into PhD programmes would be a nice control. But there’s no curve there.

    Alas, I know academia is difficult. But I wouldn’t dare drawing conclusions without a proper comparison.