spoiler

When I say LLMs I always mean opensource, openweight models not OpenAI/Anthropic/X-AI crap, these companies are trying their darndest to spread misinformation and lobby to quell and prevent the development of these opensource models, they should be punished by creating an alternate ecosystem untouched by the opinions and whims of these feudalistic tech lords

The exam/criteria should be broad and rigorous, The license issued and the exam should be private(tied to a handle not PI name), unique and publicly verifiable.

Right now the onus is on individual projects to take a stance for/against LLM use for any contributions in their codebase.

With all the opinions floating around regarding LLM use for programming(most of which i feel is either misinformation, or technophobia), I think this would at least stem some of the legitimate concerns regarding competency and responsibility.

  • Artemis_Mystique@lemmy.mlOP
    2·
    6 days ago

    What kind of criteria or tests are you imagining?

    I am imagining a sandbox environment where the test taker is provided with a medium tier capable LLM, and they are given tasks things like

    • given a huge codebase, can you implement a new feature following the coding standards of the codebase and minimally changing the existing code
    • a job to change something and follow the cascading effects through all the files it touches, fixing it without breaking things elsewhere
    • ability to reign in a LLM that is doing more than you asked it to do

    I am sure more experienced people would lay out more scenarios to test for

    A certificate after a test would only show capability, not continued practice. So my confidence in it wouldn’t be very high. Demonstrated practice seems more applicable than some cert some time ago. Some people chase certs to check boxes rather than prove or apply the practice.

    True, but then again a guarantee that the person knows what they are doing is better than no guarantee at all