Why are you asking here and not Codeberg then? Nobody can answer for them.

There's no way you're going to be able to train an LLM from scratch without a giant corpus of real code and English text. Even if you managed to procure that ethically, it would only address one of a number of objections people have to these things.