ARM BoF ======= DC18, Hsinchu Ports update ============ arm64 * Current port, first released with Jessie * Working well * More devices available now, more coming * Real server hardware! * Simple choice of kernel with DTB and/or ACPI armhf * Current port, first released with Wheezy * Hard-float ABI, v7, VFPv3-D16 + Standard for 32-bit Arm Linux distros, so we should have binary compatibility with other distros * armmp kernel & DTBs + Potentially massive set of supported devices * UEFI is becoming more of a thing on armhf, and will make it easier to run armhf VMs on top of arm64 hardware armel * Current port, first released with Lenny * Soft-float ABI, used to be v4t, now v5te * I'd been planning to drop from testing/release, but others have stepped up to keep armel running * Still supported upstream for most languages, but can also be a problem where things newer language runtimes expect v7 as a minimum baseline. Fixable, but takes effort. People will need to keep working on this. We'll need to manage expectations as to what is supported/supportable in armel Buildds and hardware ==================== * Existing Marvell Armada XP hardware we're using for armel/armhf is not great in terms of supportability, e.g. needs a manual button press to reboot after power failure. The machines are quite fast and powerful, but they're still development boards and are missing features we'd like such as support for more than one disk to do RAID. * There's a wider choice of arm64 machines available, e.g. Seattle, X-Gene, ThunderX, Macchiatobin, Centriq, Synquacer, to suit a range of budgets and requirements. I'm hoping that we'll soon see a commercially successful, readily available arm64 server machine that we'll be able to just buy off the shelf... * DSA would like to get rid of the Marvell machines, replacing them with more manageable server machines. Supporting dev boards is hard work. AFAICS there are not any worthwhile 32-bit Arm server machines that we could sensibly use as buildds. Hence, we'll need to start switching to using arm64 machines as buildds for our 32-bit ports too. There was already some discussion about this on the mailing list - see the thread around https://lists.debian.org/debian-arm/2018/06/msg00062.html Building 32-bit software on arm64 ================================= There are some issues here :-( * Some arm64 machine won’t run the A32 ISA; server vendors are often targeting 64-bit only and cramming more cores into a CPU by leaving out the 32-bit support * We've started using an arm64 machine (arm-arm-01, a Seattle-based machine which includes A32 support) to build armhf. Unfortunately, there have been some problems detected already: + Alignment - Arm CPUs care about alignment, but we've been running the 32-bit buildds with kernel support enabled for fixing up user-space alignment problems in software. No such option exists in the arm64 kernel, so we get SIGBUS errors. + #define mixups - glibc fails its testsuite due to a mismatch in 32- and 64-bit definitions for SIGMINSTKSZ. Patch already developed by the Arm kernel team, but not upstream yet. + It seems that our armhf Haskell binaries are mis-targeted (ARMv6, not ARMv7?), so as well as needing lots of alignment fixups they're also using old-fashioned primitives for barriers. Looks like it will need fixing up and completely rebuilding... + I'm doing rebuilds of the armhf *and* armel archive to check if there are any other problems to be found. [UPDATE: very nearly finished, expect results and analysis very shortly...] + Possible that we could mask some of the building problems using 32-bit VMs running on the arm64 machines, but that won't solve the problems properly - these binaries still won't run properly on arm64 machines like they should. Much better to fix things like alignment issues properly, IMHO. + Fixing alignment problems also helps other ports like m68k and sparc, which have long had the same issues. They only exist because developers can get away with invalid assumptions on x86 where the hardware will fix things at runtime. Even there, proper alignment will typically improve performance. Discussion ========== * arm64 hardware for reproducible builds? Use reproducible builds to systematically detect arm64 vs. armhf issues * Ubuntu are also building armhf on arm64 and are seeing the same alignment problems we are. Should we disable the alignment fixups on the armhf buildds? Probably, to get consistent behaviour. Even better, turn on reporting of alignment fixups so we can find and file bugs. We should also do this for autopkgtest to find more alignment faults at runtime. * I'm expecting to pick up several Synquacer machines (24-core Cortex A53) to use for Debian, donated by Linaro. Some will become buildds, wanting to get more to use for autopkgtest, debian-ci, reproducible builds etc. [UPDATE: 3 of these are now in Vancouver, ready to set up as buildds] * When will people be able to get hold of arm64 server machines readily? It's a depressing story - lots of vendors in theory, but not much commercial success so far :-( * We've had arm64 openstack images for a while now. * Lots of the large cloud-based services (e.g. Travis) are starting to support arm64 too. It's an ongoing process. * Vagrant has been experimenting with newer U-Boot versions which include the new UEFI support. Should make installation easier on some arm64 and armhf devices, for example. Possible issues with devicetree here - discussion about problems with the devicetree moving target, and hence the move to ACPI for some devices like arm64 servers. Standards like SBBR and EBBR are meant to help here, defining lowest common standards for how systems should work. * Is armel going to continue? Are the buildd support issues the only blocking problems? Basically... Clarifying - the new arm64 builders for armhf *should* also work for armel too. Wait for rebuild results to see how that works, I'd expect it to be OK. We apparently have offers of donations to help keep armel alive. Let's see!