Hi.
I'm currently making some changes to bootstrap in order to be able to cross-compile rustc with rustc_codegen_gcc.
In my branch, I added a target config variable to specify the path to the libgccjit.so for that target (since GCC cannot target all arches from a single driver like LLVM can).

I'm now trying to build rustc itself for a new target (m68k) and I'm not sure where in bootstrap I should add this build of rustc.
I tried adding it here, but this creates a cycle:

    
        
        
        
    
Cycle in build detected when adding Assemble { target_compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu, forced_compiler: false } }
    Assemble { target_compiler: Compiler { stage: 2, host: m68k-unknown-linux-gnu, forced_compiler: false } }
    Assemble { target_compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu, forced_compiler: false } }
    Assemble { target_compiler: Compiler { stage: 2, host: x86_64-unknown-linux-gnu, forced_compiler: false } }

Any idea where I can add this?
Thanks.

bootstrap drives the build of the compiler for selected targets, so that shouldn't be added in code

The command only works if I add rust.local-rebuild = true in bootstrap.toml and then fails with errors like:

    
        
        
        
    
/home/user/projects/rust/library/std/src/thread/current.rs:118:(.text._ZN3std6thread7current11set_current17h8dfbdf6be4d06606E+0xc): relocation truncated to fit: R_68K_GOT16 against symbol `_GLOBAL_OFFSET_TABLE_' defined in .got.plt section in /home/user/x-tools/m68k-unknown-linux-gnu/usr/m68k-unknown-linux-gnu/sysroot/usr/lib/crti.o

when building "stage0 library artifacts (stage0:x86_64-unknown-linux-gnu -> stage0:m68k-unknown-linux-gnu)".
I didn't get these errors when building stage 2 (I believe) library artifacts with m68k.

Is rust.local-rebuild = true necessary here?
Without it, I get this panic:

    
        
        
        
    
thread 'main' (34368) panicked at src/bootstrap/src/core/builder/mod.rs:1273:21:
It is not possible to build the standard library for `m68k-unknown-linux-gnu` using the stage0 compiler.
You have to build a stage1 compiler for `x86_64-unknown-linux-gnu` first, and then use it to build a standard library for `m68k-unknown-linux-gnu`.
Alternatively, you can set `build.local-rebuild=true` and use a stage0 compiler built from in-tree sources.

Thanks for your help.

local rebuild shouldn't be used. the error is weird, hmm. are you also passing --build somewhere? perhaps in bootstrap.toml?

Jakub Beránek said:

local rebuild shouldn't be used. the error is weird, hmm. are you also passing --build somewhere? perhaps in bootstrap.toml?

I don't think so.
My command is PATH=$HOME/x-tools/m68k-unknown-linux-gnu/usr/bin/:$PATH ./x build compiler --host=m68k-unknown-linux-gnu and my bootstrap.toml is:

    
        
        
        
    
profile = "compiler"  # Includes one of the default files in src/bootstrap/defaults
change-id = 140732

[build]
jobs = 2
target = ["m68k-unknown-linux-gnu"]
#local-rebuild = true

[rust]
codegen-backends = ["gcc"]
deny-warnings = false
debug-assertions = false
debug-assertions-std = false

[gcc]
download-ci-gcc = true

[target.x86_64-unknown-linux-gnu]
libgccjit-dir = "/home/user/projects/gcc-repo/gcc-build/build/gcc"

[target.m68k-unknown-linux-gnu]
cc = "/home/user/x-tools/m68k-unknown-linux-gnu/usr/bin/m68k-unknown-linux-gnu-gcc"
libgccjit-dir = "/home/user/x-tools/m68k-unknown-linux-gnu/usr/lib"

yeah sorry, I didn't realize that you don't even have the stdlib available. so yeah, you could enable local-rebuild and build a stage0 stdlib for m68k (this was enabled by https://github.com/rust-lang/rust/pull/145876).

I'm not sure if bootstrap will actually allow you to use that custom built stdlib for further build of m68k rustc though. But I suppose that it might work, if you copy the built stdlib to the sysroot of the stage0 compiler.

Shouldn't cross compilation use stage 2? So the bootstrap compiler builds a stage 1 with support for m68k and then stage 1 builds stage 2 for the host and for m68k and then uses the stage 2 for the host to build the standard library for m68k.

either stage 2 is needed or it will auto bump the stage, I don't remember

Jakub Beránek said:

using stage 2 should be simpler than doing these hacks with stage0, yeah

So, what command or config would I need to do this?

./x build library --host=<m68k> --stage 2

And then ./x build compiler --host=<target> --stage 2?
This will skip the stage0 stuff?

if you want only the compiler, then

build compiler --host

if you also want the library for m68k, theb

build library --host --target

yeah, this won't do anything stage0 releated with m68k

Thanks, I managed to get further.
It seems I'm having issue choosing the right libgccjit.so here:

    
        
        
        
    
*** Rustc target: m68k-unknown-linux-gnu
Using "/home/user/x-tools/m68k-unknown-linux-gnu/usr/lib"/libgccjit.so
Building stage2 compiler artifacts (stage1:x86_64-unknown-linux-gnu -> stage2:m68k-unknown-linux-gnu)
   Compiling proc-macro2 v1.0.101
libgccjit.so: error: unrecognized argument in option '-march=x86-64'
libgccjit.so: note: valid arguments to '-march=' are: 68000 68010 68020 68030 68040 68060 cpu32 isaa isaaplus isab isac
libgccjit.so: error: unrecognized command-line option '-masm=intel'

I assume proc-macro should use x86-64 since they run on the host, right?
As you can see from my debug prints, I use the m68k libgccjit.so because the target I get is m68k-unknown-linux-gnu.
Here's the commit where I added the code to select the proper libgccjit.so depending on the target.

Assuming I'm right about proc-macro, how can I know the right target to use here?

Maybe try setting the libgccjit.so in src/bootstrap/src/bin/rustc.rs? There is a target variable in there which either contains the target for which you need to load libgccjit.so or if it is None, you need to load the libgccjit.so for the host.

Thanks for the idea.
In src/bootstrap/src/bin/rustc.rs, how would I have access to my bootstrap.toml config that specifies the directory for libgccjit per target:

    
        
        
        
    
[target.x86_64-unknown-linux-gnu]
libgccjit = "/path/to/libgccjit/host"

[target.m68k-unknown-linux-gnu]
libgccjit = "/path/to/libgccjit/m68k"

?

Ok, there are many things at play here that I don't understand.
Which tool copies the libgccjit.so in ~/.rustup/toolchains/?

Then, is it loaded by librustc_codegen_gcc.so because ~/.rustup/toolchains/<TOOLCHAIN>/lib is added to LD_LIBRARY_PATH? If so, where is it added?

How do you suggest we make rustc select the right libgccjit.so? Do you think manually doing dlopen in rustc_codegen_gcc would be best? If so, would we expect that users select the correct libgccjit.so via a flag? Or via some directory structure like ~/.rustup/toolchains/<TOOLCHAIN>/lib/m68k-unknown-linux-gnu?

What are your thoughts on all this?

antoyo said:

Thanks for the idea.
In src/bootstrap/src/bin/rustc.rs, how would I have access to my bootstrap.toml config that specifies the directory for libgccjit per target:

    
        
        
        
    
[target.x86_64-unknown-linux-gnu]
libgccjit = "/path/to/libgccjit/host"

[target.m68k-unknown-linux-gnu]
libgccjit = "/path/to/libgccjit/m68k"

?

You could put those in env vars you pass to cargo and then read from the rustc wrapper in bin/rustc.rs.

bjorn3 said:

You could put those in env vars you pass to cargo and then read from the rustc wrapper in bin/rustc.rs.

Is there any advantage to doing that rather than what I was trying to do, e.g. selecting the correct libgccjit.so on the bootstrap side?

antoyo said:

How do you suggest we make rustc select the right libgccjit.so? Do you think manually doing dlopen in rustc_codegen_gcc would be best? If so, would we expect that users select the correct libgccjit.so via a flag? Or via some directory structure like ~/.rustup/toolchains/<TOOLCHAIN>/lib/m68k-unknown-linux-gnu?

What are your thoughts on all this?

those are good questions, but this is for a much bigger discussion in t-compiler, I'd say. this is really uncharted territory, as far as I know. I'll create a thread there

antoyo said:

bjorn3 said:

You could put those in env vars you pass to cargo and then read from the rustc wrapper in bin/rustc.rs.

Is there any advantage to doing that rather than what I was trying to do, e.g. selecting the correct libgccjit.so on the bootstrap side?

From the bootstrap side you can only pick a single set of configs to pass to cargo. If you need to distinguish between host and target crates, you need to do this in bin/rustc.rs..

bjorn3 said:

From the bootstrap side you can only pick a single set of configs to pass to cargo. If you need to distinguish between host and target crates, you need to do this in bin/rustc.rs..

Thanks for the precision.
From what I can see in this file, I guess I would need to use target when it's Some and use the host target when it's None.
Do I understand the code right?

Yes