policy: Added to the PR template, and an AGENTS.md, refusing AI contributions. - #15353
Conversation
|
Marked draft for discussion. I think even if this is our policy, we should wordsmith it to be more gentle. |
|
Under the assumption this is the policy, I'd like to get the pull request template down to one line, and, uh, less shouty. :) Or maybe just a check box: I think AGENTS.md is fairly reasonable, but I'll accept any feedback or rewrites. |
I think this would be the gentler approach and would probably be the way to go? Generally both approaches work.
|
|
For the check box, maybe something like: |
|
For AGENTS.md, how about something like this? |
|
Whatever we add here should be also added to the SDL satellite libraries and propagated to all release branches. |
I'd say maybe replace
|
I'm not sure if "of unknown origin" helps there, because
|
Updated with "may not": |
Good point. Simplified: |
👍 Nice, no complaints with this now. |
|
I've updated the PR with feedback. I've made small grammar changes, and wordwrapped AGENTS.md, but that's all. |
|
This looks good to me. I’d socialize this on Discord to see if there’s any reason we shouldn’t have this policy that we missed. |
|
I'm not an SDL contributor but if I were to write a patch, I would not be able to tick that box. I use clion which uses a proprietary code completion engine. If I write something like f(1);
f(2);
f(then it will suggest completing the last line to f(3);This falls under AI since some intelligence is required to detect the pattern and to infer the developer's intention. |
CLion uses machine learning auto completion, so yes, it wouldn't be allowed. You can disable it within the settings to get standard LSP autocompletion though, so you would need to contribute using either that or another editor that does not use ML/AI completions (e.g. KDE Advanced Text Editor (KATE), VSCodium + Clangd) |
|
"It notices that there's a simple pattern within a specific programming language's syntax and offers to fill it in for me if I agree with the autocomplete" feels like a long way from "I vibe-coded a pull request where I might not have even read the code, let alone understood it." I get that LLMs are also predictive models looking for patterns, so if you squint at both they might have similarities...but I think it probably doesn't need clarification. |
|
The checkbox mentions neither LLMs nor vibe coding. It says AI-generated code. |
We know what our intent is, but we probably want to be clear somewhere. Other people may not know how militant or relaxed our intentions are. |
|
I don't really think it was necessary, but it doesn't hurt, so I clarified this in db525f6. |
|
FWIW I still wouldn't tick that box because Clion is proprietary and I have no way of knowing if the underlying technology is "LLM". I think what you actually want to say is what you wrote above:
|
CLion's documentation/readme describes it. I do think it's your responsibility as someone who makes pull requests to understand the technology you used to write the code being used in the pull request. |
|
Where can I find the readme? |
|
I've taken a quick look at the settings and it suggests that it uses language models 
I don't know if it is a "large" language model. |
If it's using Jetbrains Mellum, then it is an LLM (Jetbrains describes it as such) Otherwise, I don't know, you will probably need to look into the files to determine that. |
|
As I understood @icculus above, he wants to allow this kind of simple completion whether or not it's an "LLM". |
|
Yeah, I don't want to get into the weeds trying to define this thing. If the current wording leaves someone legitimately confused, I don't think more words will help, and it's counter-productive to try to manage a complete list of all known tools. I don't believe this will be the threshold where someone feels uncomfortable contributing, or confused about whether they've used generative AI to create patches. |
|
Last call on this before I press the Merge button! |
|
I am not a big fan of IA but I think it's too strict :) |
AI has an incredibly bad record with regards to quality, legality and ethics. I feel that we could live in 2026 and not support slave labor. |
The definition of slave: "a person who is forced to work for and obey and is considered to be their property" Kenya is a free country. Workers must not have been forced to work, nor they are the property of a company. btw, for legality and ethics, you focus on IA, but there are other industrial sectors that are much more concerned: food, cloth, medicine, electronics, etc. I may not know all the ethics issues with IA, but my opinion to accept it right now (I mean this can change) is that I am optimistic enough to imagine that IA have pros that will at some point overtake by far all cons. |
|
artifical intelligence would have more pros to speak of if the primary motivator of the companies making these models wasn't money. Ultimately using cheap labour and littering the environment is cheaper than giving a fuck. We shouldn't support that. |
This is a very strange angle to take, not just because a lot of things are unethical right now means that we shouldn't strive to be more ethical, and even when we leave the entire topic of ethics in general and only think of this topic purely in terms of quantifiable measures and results, AI still isn't exactly fit for that either. LLMs have been proven to get worse over time as they scale, being practically unable to rationalize (of course, they are just words on a chain), and produce dubious quality of code, with bots and users here in this platform generating entire PRs and issues using Claude and similar tools, without them actually understanding what the hell is going on in the codebase to begin with. I do not think that this is worth it, nor a good compromise at all. |
|
Just to be clear, @1bsyl has written a lot of excellent code for SDL, over many years, so he has certainly earned the right to have his opinion heard here. |
If AI changes significantly in the future, I'm sure the guidance here would be updated accordingly at that time. |
Yes, certainly. As I said, this is evolving rapidly, so I think we'll have some discussions about this every few months. |
|
Just want to continue answer : It's clear that there are now ethical issues. As said, there are using "using cheap labour", "littering the environment". etc. I believe there are laws for this and there are ethical commissions to guide this. So, at some point, company will be pursued for this. And also, there are people, professionals, whose job is to sort this out. It's clear we can refuse IA's because of this. and then that's it. and I am not going to defend them. but we can look at the technical aspects:
for SDL:
My experience, is that if you instruct clearly, step by step, and also if you can evaluate what you're doing. you can manage to have something of good quality and save time. I just think IA can be used as another tool (like grep, sed, awk, indent, semantic patch) with more potential but also more difficult to use. And If someone provide PR with some part done IA. If something is erroneous, the IA isn't to blame, but the author of the PR. |
|
Update: I missed that the wording was already changed. Please ignore my comment, although I'll leave it here just so people can see what I initially proposed: I feel like one could argue it would be best to also disallow LLM auto-complete, not just AI agents. Due to this: Chan-jo.Jun.Co-Pilot.Copyright.Infringement.webmThis relationship of LLMs copying seems to be directly related to the output quality and therefore potentially inherent: https://www.sciencedirect.com/science/article/pii/S2949719123000213#sec6
While LLM auto-complete is commonly regarded as less problematic due to it leading to more involved human review, it doesn't seem like that would avoid plagiarizing items. So unless you plan to have a alwyer review every item you add to check if it's short enough to not risk a lawsuit, which I would propose you probably don't want to do, I'm not sure if you would really want to allow LLM auto-complete. While a ban is hard to enforce, it feels like 1. the responsibility would be moved to a malicious contributor with a ban and the project would be able to claim whenever an issue is caused by this that they did their best to avoid unlicensed problematic contributions, 2. I think it's safe to assume it'll at least reduce the contributors trying to submit LLM code. However, I'm not a lawyer and this isn't legal advice, please ask your own lawyer if you want legal advice on this. |
|
Ideally we should probably distinguish between AI as in machine learning (the traditional definition) and AI as in LLMs (what everyone thinks of when you say AI these days), but that can get complex when the people who use LLMs don't even know what it is they're using. And, I mean, that's exactly the problem, isn't it? LLMs are extra problematic because they can (simplifying a bit) essentially only combine things that they've already seen, and they've all been trained on all kinds of source code in violation of terms of use and licenses, without consent. Proprietary code, GPL code, code that needs a patent license in some jurisdictions, everything. That's still happening right now. If someone used an LLM exclusively trained on their own code and got something useful out of it I'd have less of a problem with that, but as far as I know that's not practical yet, if ever. (Personally I think LLMs are a dead end. They're certainly not intelligent in any sense of the word) This doesn't get into all the other problems with LLMs, all the ethical issues, code quality and correctness, and so on. I don't think those should be ignored, but that one issue is already a blocker as far as I'm concerned, and it's not going to change any time soon. For me at least, this isn't about simple code completion like inserting a[2] on the next line when the previous two had a[0] and a[1]. I mean that doesn't need an LLM, it's not even copyrightable, and there's still a human in charge anyway. Also, I feel like the arguments for allowing LLM are akin to saying "but there's so many people who don't care about licenses and copyright and code quality so really we just have to allow it". That's not a very good argument. That's just arguing for giving up. |
|
I tend to use the term GenAI for LLMs and diffusion models that were trained on huge datasets and used to (attempt to) replicate human creativity. The term "AI" became pretty much meaningless after it being used so much for marketing. Still, I do not like the term GenAI all that much, because there's nothing intelligent about it. But I rather have a term people can recognize than one no-one will recognize. |
Good points all around, would not be opposed to explicitly stating that this policy is for LLMs, given that all of my issues with "AI" are explicitly regarding LLMs (and, by extension, Generative "AI").
Generally speaking, most of the arguments I have seen for LLM use falls into small amount of categories: manufactured consent (couldn't think of an example for this one but I have definitely seen a fair few of these), giving up ("the genie is out of the bottle" and similar rhetoric), disinformation ("produces higher quality code"/"saves time and money") and downplaying ("its just a tool") |
|
As a big fan of handcrafted software made with love, I want this PR to proceed. Although I do have some suggestions:
My ideas might be overkill but that's what I'd use in any open contribution project I'd have. |
I'm guessing you mean "non-LLM code completion", based on traditional grammar-based fixed rules? I'm asking for clarity. I agree that traditional completion seems unproblematic. (However, LLM completion does not.) |
This is why AGENTS.md says this:
We would deal with this on an individual level, in the same way we might if someone is being disruptive in general, from speaking to them directly to outright banning them from the libsdl-org GitHub organization. As such, we aren't writing down specific guidelines for this at this time.
I'm not against adding something to AGENTS.md for this, but I don't know what the wording would be. I also don't want to drive away people running fuzzing/static-analysis tools, or even just fixing typos in comments, which are usually tool-driven but not genAI. But we haven't received these things in staggering quantities so far, and maybe I'd feel different if I did. |
|
Okay, I'm clicking merge on this. Further concerns can be discussed in followup issues/PRs. Thanks, everyone! |
|
Quite frankly, moronic. |
|
@rebasecase I assume you're meaning to make a case for AI use? You should probably elaborate then, if you want your disagreement to be constructive. |
|
Going to lock this so conversation occurs in one place: the original issue. And frankly, if the quality of conversation doesn't improve rapidly, I'll be locking that, too. |
14 participants
This is just a proposed solution; if we go a different way (even the same direction but with gentler text), it's okay to close this and do something else.
This will warn people, when they try to create a PR, that we won't accept AI-generated contributions. It also adds an AGENTS.md file, which is what Claude/Copilot/etc read for instructions on how to work with the project (in this case, we tell it not to).
Fixes #15350. (Which one should read fully before pressing Merge here.)