Banana Pi BPI-CM4 Node
2023.08.27
Spoiler alert: The BPI-CM4 ultimately is considered by Electricbrain to not be sufficiently compatible with the Raspberry Pi CM4. Update - after many hours of attempted kernel hacking, I ended up zeroing the BPI-CM4's eMMC and loading Armbian's Linux on an SD-Card. The Waveshare board instantly sprang to life, albeit without USB (read no keyboard). However, I was able to login via SSH and get started. This is currently an evolving story. Read about the journey and how close it came is now.
Built after the end of the Coronavirus pandemic lock downs the supply of parts is still very limited. Alternative supplies to the Raspberry Pi parts are just beginning to reemerge. Among those alternatives is the Chinese manufacturer Banana Pi. This node is built on their BPI-CM4 board which claims to be largely compatible with the original CM4 module.
Purchased on AliExpress after a previous scam attempt from another supplier failed to materialize. AliExpress refunded me entirely protecting me from the scam.
Getting the board running on the Waveshare Mini Base Board (A Rev. 4) with Banana Pi's Ubuntu 20.04 was a bit of an ordeal. Essentially the BPI-CM4 suffered precisely the same issue as the original Raspberry Pi CM4. Namely the USB doesn't work out of the box (see Software Compatibility Issues) which of course means no keyboard or mouse.
Overcoming the no keyboard issue involves hacking at the SDCARD files once the Ubuntu Server image is installed on it. Getting the OS image onto the SDCARD is covered on the wiki page. Luckily the downloaded image has SSH enabled which means that all one needs to do is setup /root/.ssh in the usual way with your public ssh key.
Once the hacked image is prepared the SDCARD is inserted into the card slot on the bottom of the Waveshare Mini Base Board and the monitor attached (although there is a bug in the 4.9 kernel CEC driver which causes thousands upon thousands of lines to be output into dmesg and the journal). Finally applying power sees the BPI-CM4 boot the SDCARD image.
Once BPI-CM4 is alive the OS seems to set about resizing partitions and running a remote-unattended-upgrade (so DO NOT cut the power until it has gone quiet). It is possible, at this point, to login as the root user, via SSH, using the private key associated with the hacked in public key.
When everything has gone quiet, the SDCARD partitions will have been resized and eveything else has been upgraded. At that point it's possible to copy the original OS image (using scp) on to the SDCARD (it was put in the /root directory in a folder named Downloads. From there it's possible to follow the instructions on the wiki page which outline installing the image on to the eMMC flash drive. Once again it's important to mount the root partition of the eMMC and hack the /root/.ssh stuff into the file system. cp -a /root/.ssh /mnt/root should do the job after mounting the eMMC.
Once that's done the next step is to power off the machine (use something like systemctl poweroff) and the SDCARD is removed. From that point on, as far as I can work out, it is no longer possible to boot from the SDCARD. It seems the boot sequence looks for a blank eMMC and if found to be blank it then proceeds to try and boot from the SDCARD. Update: As it turns out the boot sequence for Ubuntu 20.04 is tad more sophisticated than that. It seems that the OS loader from the onboard eMMC looks on the first parition of the SDCARD for a boot.ini. If it finds one it proceeds to boot from there. Essentially that means you can play around with the OS by plugging an SDCARD in and out, leaving the BPI-CM4's eMMC alone.
Once everything is booting from the eMMC there's the usual house keeping stuff like TZ and hostname to take care of. Then docker is installed as per the wiki page. Once successfully installed I had a go at trying to join the cluster. While the installation process indicated success, Portainer could not deploy the agent to the node due to some networking error and it kept cycling retries forever.
Faced with a non running node I set about uninstalling/purging docker, updating everything I could and then reinstalling. Somehwhere along the line I've managed to corrupt the stage 2 boot loader in the eMMC.
As it turns out that is: "3. A extreme situation is bootloader or uboot corrupted, Rom load it from EMMC but hangup in u-boot or BL2, for example the boot process will hangup in BL2 of EMMC if dram init failed, The only way is format the EMMC with usb burning tool, or download the Android image completely and then try other ways to erase EMMC or flash Linux image to EMMC."
So with no functioning USB it becomes imperative to have the BPI-CM4IO base board to carry out the necessary functions to recover. Fortunately that's quite economical to do as the board is available on AliExpress for only $AUD 23. Now waiting for delivery..... So that leaves the situation at a bricked BPI-CM4 for the time being.
... to be updated.
Well the io board turned up. Unplugging the module from the waveshare and replugging into the new io board complete with temporary USB power cable and Samsung 5Vx3A PSU saw the board immediately boot in the normal way. It sat there for a while trying to resync the RTC clock to the current time which succeeded.
Anyway to cut a long story short, one reads stories of bad cables causing mayhem. Well this is one of those stories. The cable is a UGreen high end USB cable too. Oh well. The long and the short of it is: there was pretty much nothing wrong - Doh!.
All the OS updates have run through in the usual way too. Docker is happily running and the board is now part of the cluster. Power measurements are showing the board consumes 4 watts when s-tui and stress maxes it out on all 6 CPU. When the clock frequencies drop to 500MHz power consumption drops to 1 watt.
... update to the update
The board BPI-CM4 failed again. Simply unplugging the board from the Waveshare board and plugging it in to the BPI-CM4IO base board saw it immediately fire up again. To rule out a software issue, the exact same process was used to bring the board up from nothing after wiping the onboard eMMC.
The final conclusion here is that there is some hardware issue preventing the BPI-CM4 from booting on the waveshare board. At this point electricbrain considers the BPI-CM4 is not compatible enough with the Raspberry Pi CM4 module. Essentially enough is enough.
... time passes: The BPI-CM4 has been happily running on it's own I/O board for some time. Occasionally it kernel oopses, which I put down to the CEC thing. It's had the boot sdcard copied over to the eMMC module now in the hope that this would rule out some issue with the Waveshare CM4-IO-BASE-A (rev4). But no change. It still won't boot anymore. Still runs fine on its own I/O board though.
... update to CEC issue - resolved
Ref: https://forum.odroid.com/viewtopic.php?t=26588
The solution is to completely disable the HDMI CEC module in the 4.9 kernel.
In a nutshell, edit /boot/firmware/boot.ini (on the vfat boot partition) and add the line:
fdt rm /aocec
Essentially that goes after the line:
fdt addr ${dtb_loadaddr}
So the few lines together:
# Load kernel, dtb and initrd
fatload ${devtype} ${devno}:1 ${kernel_loadaddr} Image.gz
fatload ${devtype} ${devno}:1 ${dtb_loadaddr} ${fdtfile}
fatload ${devtype} ${devno}:1 ${initrd_loadaddr} uInitrd
fdt addr ${dtb_loadaddr}
fdt rm /aocec
Armbian Linux with Kernel 6.2.12-meson64
The Ubuntu 4.9 kernel oops issue never went away. Running BackupPC from within a docker container saw the node crash and reboot with various "Tainted" messages. I began to follow up what the messages meant but decided I really don't want to know that much about obsolete kernels. That's when I began to come across posts from various people that had tried a bleeding edge distribution named Armbian Linux. Banana Pi themselves claim to have a high level of paid support from Armbian too.
In desperation the BPI-CM4's eMMC was wiped. The Armbian Linux image was imaged onto an SDCARD. Bingo everything lit up. Then the BPI-CM4 was plugged into the WaveShare board - double bingo! It just worked. I was able to login via ssh, since the keyboard and mouse don't work due to the Waveshare USB hub chip issue.