Big-Endian Testing with QEMU

74 points - today at 1:28 PM

Source

Comments

AKSF_Ackermann today at 2:46 PM
> When programming, it is still important to write code that runs correctly on systems with either byte order

What you should do instead is write all your code so it is little-endian only, as the only relevant big-endian architecture is s390x, and if someone wants to run your code on s390x, they can afford a support contract.

bluGill today at 3:59 PM
What I really want is memory order emulation. X86 as strong memory order guarantees, ARM has much weaker guarantees. Which means the multi-threaded queue I'm working on works all the time on development x86 machine even if I forget to put in the correct memory-order schematics, but it might or might not work on ARM (which is what my of my users have). (I am in the habit of running all my stress tests 1000 times before I'm willing to send them out, but that doesn't mean the code is correct, it means it works on x86 and passed my review which might miss something)
electroly today at 2:48 PM
> When programming, it is still important to write code that runs correctly on systems with either byte order

I contend it's almost never important and almost nobody writing user software should bother with this. Certainly, people who didn't already know they needed big-endian should not start caring now because they read an article online. There are countless rare machines that your code doesn't run on--what's so special about big endian? The world is little endian now. Big endian chips aren't coming back. You are spending your own time on an effort that will never pay off. If big endian is really needed, IBM will pay you to write the s390x port and they will provide the machine.

susam today at 4:59 PM
I wrote a similar post [1] some 16 years ago. My solution back then was to install Debian for PowerPC on QEMU using qemu-system-ppc.

But Hans's post uses user-mode emulation with qemu-mips, which avoids having to set up a whole big-endian system in QEMU. It is a very interesting approach I was unaware of. I'm pretty sure qemu-mips was available back in 2010, but I'm not sure if the gcc-mips-linux-gnu cross-compiler was readily available back then. I suspect my PPC-based solution might have been the only convenient way to solve this problem at the time.

Thanks for sharing it here. It was nice to go down memory lane and also learn a new way to solve the same problem.

[1] https://susam.net/big-endian-on-little-endian.html

zajio1am today at 5:07 PM
There is one reason not mentioned in the article why it is worth testing code on big-endian systems โ€“ some bugs are more visible there than on little-endian systems. For example, accessing integer variable through pointer of wrong type (smaller size) often pass silently on little-endian (just ignoring higher bytes), while read/writ bad values on big-endian.
siraben today at 7:42 PM
Without installing anything, this can also be reproduced with a shell script that uses a Nix shebang to specify the cross compilers.

https://gist.github.com/siraben/cb0eb96b820a50e11218f0152f2e...

1over137 today at 7:41 PM
>But without access to a big-endian machine, how does one test it? QEMU provides a convenient solution. With its user mode emulation we can easily run a binary on an emulated big-endian system

Nice article! But pity it does not elaborate on how...

ncruces today at 4:46 PM
If you're using Go on GitHub (and doing stuff where this actually matters) adding this to your CI can be as simple as this: https://github.com/ncruces/wasm2go/blob/v0.3.0/.github/workf...

On Linux it's really as simple as installing QEMU binfmt and doing:

   GOARCH=s390x go test
bluGill today at 3:55 PM
For most code it doesn't matter. It matters when you are writing files to be read by something else, or when sending data over a network. So make sure the places where those happen are thin shims that are easy to fix if it doesn't work. (that is done write data from everywhere, put a layer in place for this).
eisbaw today at 2:49 PM
I did that many years back, but with MIPS and MIPSel: https://youtu.be/BGzJp1ybpHo?si=eY_Br8BalYzKPJMG&t=1130

presented at Embedded Linux Conf

beached_whale today at 5:42 PM
I've used docker buildx to do this in the past. Easier to work with than qemu directly(it does so under the hood).
pragmaticviber today at 2:21 PM
It's all fun and games until you have to figure out if the endianness bug is in your code or in QEMU's s390x emulation.
throwaway2027 today at 3:08 PM
Is there any benefit in edge cases to using big-endian these days?
IshKebab today at 7:07 PM
> When programming, it is still important to write code that runs correctly on systems with either byte order

Eh, is it? There aren't any big endian systems left that matter for anyone that isn't doing super niche stuff. Unless you are writing a really foundation library that you want to work everywhere (like libc, zlib, libpng etc.) you can safely just assume everything is little endian. I usually just put a static_assert that the system is little endian for C++.