Compiling with GCC

Hello,

I need to deploy Sensu on Gentoo Linux running on Pentium 3. My current setups on amd64 have been successful but it turns out that dev-lang/go does not run on Pentium 3 due to insufficient opcode support. So I’ll need to get a different Go compiler and GCC just happens to have it.

Can anyone please help with this? This could also help bring Sensu where Google doesn’t let their reference compiler go. Pun unintended.

Hey,
Just to clarify…
Are you saying you need to compile on a p3? Or are you saying the statically cross-compiled linux/386 builds you compile on amd64 with reference go compiler don’t run on the p3?

There is a envvar for go compiler GO386 that can be set that impacts the 386 build target.
It looks like you can use that envvar GO386=softfloat to make the go compiler use software defined floats instead of sse optimizations.

The specific value of GO386 has changed over time, but if you are using the most recent go compiler release ‘GO386=softfloat’ is probably want you want.

Sadly I don’t have any p3s in service that I can run this myself as a local test for you to be 100% sure.
But wow… I’m both amazed and sorry that you have to deal with this… that’s a real old 386 target now.

Hopefully this gets you unstuck.

references:
https://golang.org/pkg/cmd/go/internal/help/
https://github.com/golang/go/issues/42503

I can cross-compile for p3, but I would prefer if it’s self-hosted, i.e., the system intended to run sensu can also compile sensu. That’s how it should be on Gentoo Linux. No system should be left behind so that it can’t be self-sufficient in case the compilation cluster is down.

Here’s a quick rundown.

  • net-analyzer/sensu::src_prepare-overlay depends on dev-lang/go if USEflag gcc is disabled
  • dev-lang/go has a build-dependency on dev-lang/go-bootstrap
  • dev-lang/go-bootstrap is a binary package, it is not built from source and the x86 binaries herein do not run on Pentium 3 because of insufficient opcode support in P3

Here’s paperwork that proves so:

I don’t want to use soft floating point. Pentium 3 has SSE so it should be able to perform floating point ops in SSE, just not from any numbered set of SSE instruction sets that affect fp, like SSE3/4/4.1/4.2. SSE2 was only for integers.

So, to answer my question, how do I build sensu with GCC’s Go compiler?

Here’s how Gentoo builds GCC:

  1. [see next post link #2]
  2. From this point onward, inherited eclass toolchain will supplant the rest of the steps like src_configure, src_compile and src_install

[1] go-bootstrap-1.13.6.ebuild « go-bootstrap « dev-lang - repo/gentoo.git - Official Gentoo ebuild repository
[2] gcc-10.3.0.ebuild « gcc « sys-devel - repo/gentoo.git - Official Gentoo ebuild repository

Yes, I need to compile on p3 just in case.
No, I don’t cross-compile go-bootstrap. This is done by whoever supplies the tarballs in https://dev.gentoo.org/~williamh/dist, as specified in BOOTSTRAP_DIST of dev-lang/go-bootstrap package, which is then used in the variably resolved SRC_URI

Please be aware that due to the use of 64-bit atomic primitives, sensu-backend is not supported on 32-bit platforms. This is because these primitives have bugs on 32 bit platforms, see atomic - The Go Programming Language

On ARM, 386, and 32-bit MIPS, it is the caller’s responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically. The first word in a variable or in an allocated struct, array, or slice can be relied upon to be 64-bit aligned.

The etcd database, which is a dependency of Sensu, does not attempt to ensure that this alignment happens reliably. You might experience crashes or other undefined behaviour on 32 bit platforms.

I can cope with that. GCC has compile-time functions that re-align pointers if they caused an alignment exception.

If you have gccgo installed, then that should come with a replacement go tool that supports go build

After that its just a matter of using the correct --gccgoflags and related flags some such to tune for your target.

I just did a quick build of the sensu-agent using gccgo provided by Fedora repositories following the build instructions in the sensu-go repo Readme. On Fedora gccgo installs a go tool replacement in /usr/bin/go.gcc and uses the alternative system to make sure /usr/bin/go points to it

go build -o bin/sensu-agent cmd/sensu-agent

The only problem I ran into was the package github.com/modern-go/reflect2 failing… but once I pulled the development tip of that package using go get github.com/modern-go/reflect2@333559e1834b0c1840d283d79220bf121d704022 I was able to get past that problem
and the build succeeded and appears to work locally. Looks like that package has had some work since last official release to support both gccgo and llvmgo compilers

The resulting sensu-agent seems to work just fine as a drop in replacement with a quick local test.

Good Luck, I’m not much help with telling you the exact gccgo flags you’ll need for that target, but the codebase does appear to compile with gccgo without much effort on amd64 at least.

Thank you for letting me know about those tools! I’ve managed to go forward a little bit, but I’m still blocked by random packages getting stuck in compilation, for example github.com/spf13/cobra:

mkdir -p $WORK/b319/
mkdir -p $WORK/b319/_importcfgroot_/github.com/spf13
ln -s $WORK/b063/_pkg_.a $WORK/b319/_importcfgroot_/github.com/spf13/libpflag.a
cd /var/tmp/portage/net-analyzer/sensu-5.21.3/homedir/go/pkg/mod/github.com/spf13/cobra@v0.0.5
/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0/gccgo -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-pkgpath=github.com/spf13/cobra -o $WORK/b319/_go_.o -I $WORK/b319/_importcfgroot_ ./args.go ./bash_completions.go ./cobra.go ./command.go ./command_notwin.go ./powershell_completions.go ./shell_completions.go ./zsh_completions.go
echo '  .section .go.buildid,"e"' >> $WORK/b319/_buildid.s
echo '  .byte 0x58,0x76,0x35,0x4d,0x38,0x44,0x5f,0x69' >> $WORK/b319/_buildid.s
echo '  .byte 0x56,0x39,0x4d,0x47,0x73,0x4f,0x50,0x64' >> $WORK/b319/_buildid.s
echo '  .byte 0x5a,0x59,0x31,0x69,0x2f,0x58,0x76,0x35' >> $WORK/b319/_buildid.s
echo '  .byte 0x4d,0x38,0x44,0x5f,0x69,0x56,0x39,0x4d' >> $WORK/b319/_buildid.s
echo '  .byte 0x47,0x73,0x4f,0x50,0x64,0x5a,0x59,0x31' >> $WORK/b319/_buildid.s
echo '  .byte 0x69' >> $WORK/b319/_buildid.s
echo '  .section .note.GNU-stack,"",@progbits' >> $WORK/b319/_buildid.s
echo '  .section .note.GNU-split-stack,"",@progbits' >> $WORK/b319/_buildid.s
echo '' >> $WORK/b319/_buildid.s
/usr/x86_64-pc-linux-gnu/gcc-bin/10.3.0/gccgo -xassembler-with-cpp -I $WORK/b319/ -c -o $WORK/b319/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D GOPKGPATH=github.x2ecom..z2fspf13..z2fcobra -m64 $WORK/b319/_buildid.s
ar rcD $WORK/b319/_pkg_.a $WORK/b319/_go_.o $WORK/b319/_buildid.o
/usr/libexec/gcc/x86_64-pc-linux-gnu/10.3.0/buildid -w $WORK/b319/_pkg_.a # internal
cp $WORK/b319/_pkg_.a /var/tmp/portage/net-analyzer/sensu-5.21.3/homedir/.cache/go-build/be/be10e89cf7bf0e555356a7e526fef70d76c92d6098c5b41aa3505f3259b5e7c8-d # internal

It has been stuck like that for a couple days now. This is the build code that got me here:

src_unpack() {
        set -x
        if [ -z ${I_KNOW_WHAT_IM_DOING} ]; then
                local I_KNOW_WHAT_IM_DOING=0
        fi
        GO_TOOL="go"
        if use gcc; then
                GO_TOOL="go-$(gcc --version | head -n1 | sed 's/^.*Gentoo\ //;s/\..*$//')"
        fi
        if use systemd; then
                die "We don't have systemd unit files yet."
        fi
        if      use backend && \
                (use x86 || use arm) && \
                ! use gcc && \
                [ ${I_KNOW_WHAT_IM_DOING} -eq 0 ]; then
                die "Backend does not work reliably on 32bit hosts. If you really want this, either re-emerge with I_KNOW_WHAT_IM_DOING=1 or enable USE flag gcc"
        fi
        tar -xf "${DISTDIR}"/sensu-${PV}.tar.gz
        cd ${PN}-go-${PV}
        local TODAY=$(date +%Y-%m-%d)
        local BUILD_SIGNATURE="-X github.com/sensu/sensu-go/version.Version=${PV} -X github.com/sensu/sensu-go/version.BuildDate=${TODAY}"
        if (use x86 || use arm); then
                CFLAGS+=" -mstackrealign"
        fi
        if use agent; then
                ${GO_TOOL} build -x -v -gccgoflags "${CFLAGS}" -gcflags "${CFLAGS}" -ldflags "${BUILD_SIGNATURE}" -o bin/sensu-agent ./cmd/sensu-agent >/dev/null
                cp "${DISTDIR}"/"${PN}"_"${PV}"_agent.yml "${S}"/agent.yml.example
        fi
        if use backend; then
                ${GO_TOOL} build -x -v -gccgoflags "${CFLAGS}" -gcflags "${CFLAGS}" -ldflags "${BUILD_SIGNATURE}" -o bin/sensu-backend ./cmd/sensu-backend >/dev/null
                cp "${DISTDIR}"/"${PN}"_"${PV}"_backend.yml "${S}"/backend.yml.example
        fi
        ${GO_TOOL} build -x -v -gccgoflags "${CFLAGS}" -gcflags \""${CFLAGS}"\" -ldflags "${BUILD_SIGNATURE}" -o bin/sensuctl ./cmd/sensuctl >/dev/null
        set +x
}

Ideally, the ebuild could also pass CFLAGS defined in /etc/portage/make.conf. I tried a couple ways, but all was in vain. I can’t even remember which ones those are since it’s been at least a week since.

The goal is that each invocation of gcc’s Go compiler would also add something like “-O2 -fomit-frame-pointer -march=native”, or if we detect we’re a 32bit host trying to compile with GCC, we force to compile with -mstackrealign to make sure that 64bit atomics are satisfied.