Managing Raspberry Pi images: Difference between revisions

From Wildsong
Jump to navigationJump to search
Brian Wilson (talk | contribs)
mNo edit summary
Brian Wilson (talk | contribs)
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Creating SD cards ==
== Creating SD cards ==


There is a Windows program that some people use but I only use the Raspberry Pi imager, or sometimes Balena Etcher.
There is a Windows program that some people use but I only use the Raspberry Pi '''imager''', or sometimes Balena Etcher.


== Copy IMG ==
== Copy IMG ==
Line 13: Line 13:
  sudo apt install dcfldd gparted
  sudo apt install dcfldd gparted


== Building new images ==
== Building new images with pi-gen ==


[https://github.com/RPi-Distro/pi-gen pi-gen] is what the Raspberry Pi folk use. See /boot/issue.txt on a standard image. For example,
[https://github.com/RPi-Distro/pi-gen pi-gen] is what the Raspberry Pi folk use. See /boot/issue.txt on a standard image. For example,
Line 20: Line 20:
  Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 2acf7afcba7d11500313a7b93bb55a2aae20b2d6, stage2
  Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 2acf7afcba7d11500313a7b93bb55a2aae20b2d6, stage2


== Loading an SD card ==
I have to run pi-gen on an x64 computer; I tried on a Pi5 and it would not run. I use the Docker version. I use [[Murre]] as my build machine, and I run an apt cache on [[Bellman]] to speed up repeated builds. I set up a pi-gen folder (on both Murre and Bellman).


Thing is, I could use [[Bellman]] but that does not help Mick since it's 800 miles away from him. So I am working with [[pi5]] instead
You have to download pi-gen and you have to install the apt package "qemu-user-static" so that you can build arm64 images. You probably only need qemu on the builder not the cacher.  
since he has access to his own inventory of Pi hardware.


I put the experimental svxlink image into a SanDisk card reader and popped it into a USB slot on pi5. The builtin card reader shows up in
<pre>
/dev/mmcblk0. SD cards are fine but not roomy enough for what I am about, and slow anyway.
cd ~/Documents/Projects/TARRA
For pi5, I put the image on an NVME drive so I'd have enough space to work on images. It won't do to build 8GB images on an 8GB card for example.
git clone https://github.com/RPI-Distro/pi-gen.git
After booting it looks like this.
cd pi-gen
</pre>
 
=== Building for 32-bit bullseye ===
You need to switch to a different branch, this works,
git checkout remotes/origin/bullseye
 
=== Basic config file ===
 
RELEASE=bullseye
IMG_NAME=tarra_bullseye
TARGET_HOSTNAME=svxlink
WPA_COUNTRY=US
WPA_SSID=tarra
WPA_PASSWORD=tarra
ENABLE_SSH=1
LOCALE_DEFAULT=en_US.UTF-8
TIMEZONE_DEFAULT=Americas/Mountain
KEYBOARD_LAYOUT="English (US)"
KEYBOARD_KEYMAP=us
# leave the user "tarra" available                                                                                                                                
DISABLE_FIRST_BOOT_USER_RENAME=1
FIRST_USER_NAME=tarra
FIRST_USER_PASS=XXXXXX
# Allow running in a emulator                                                                                                                                      
QEMU=1
# On Murre, use Bellman as an apt proxy when doing build_docker.sh                                                                                                
# On Bellman, be the apt proxy (run "docker compose up -d")                                                                                                        
APT_PROXY=http://192.168.123.2:3142
     
STAGE_LIST="stage0 stage1 stage2"
 
On the "builder" machine in the pi-gen folder, I run "./build-docker.sh" and wait. I get a zipped img file in the deploy/ folder. From the console then I can burn the image using the Raspberry Pi imager to an SD Card or an NVME drive. The image will be named after the name in the config and today's date, for example, '''2024-03-10-tarra.img'''


<pre>
To build again, remove the docker container and (possibly edit config) and run the builder again. 
sudo fdisk -l /dev/sda
docker rm -f pigen-work
Disk /dev/sda: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
 
Disk model: RTL9210B-CG
==== Don't build everything! ====
Units: sectors of 1 * 512 = 512 bytes
Build a "lite" image only by touching some files.
Sector size (logical/physical): 512 bytes / 512 bytes
touch stage3/SKIP stage4/SKIP stage5/SKIP stage3/SKIP_IMAGES stage4/SKIP_IMAGES stage5/SKIP_IMAGES
I/O size (minimum/optimal): 512 bytes / 512 bytes
Building all 6 stages takes '''FOREVER''' and is useless for '''TARRA builds''' which is what we're doing here'''.'''
Disklabel type: dos
Disk identifier: 0xdcf0eea7


Device    Boot  Start        End  Sectors  Size Id Type
I am running it again with just a lite image and started it at 1638. Tick tock. All apt packages should be cached now. SO. How long? 1700. Great - takes about 20 minutes. This is acceptable.
/dev/sda1          8192    1056767  1048576  512M  c W95 FAT32 (LBA)
/dev/sda2      1056768 1000215215 999158448 476.4G 83 Linux
</pre>


The SD card in the card reader shows up in /dev/sda.
== Running arm images on Linux x64 ==
You don't have to burn the image to an SD card, you can run it on Murre.  


<pre>
=== ARM8 (64 bit) ===
sudo fdisk -l /dev/sdb
THERE IS NO QEMU KERNEL FOR BOOKWORM. STOP NOW.
Disk /dev/sdb: 28.82 GiB, 30941380608 bytes, 60432384 sectors
Disk model: SDDR-B531 Reader
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x570b8731


Device    Boot  Start      End  Sectors  Size Id Type
=== ARM7 (32-bit) ===
/dev/sdb1        8192  532479  524288  256M  c W95 FAT32 (LBA)
Fortunately, this project (TARRA) '''has''' to use a 32-bit Bullseye image because of svxlink. So I should be able to run it in QEMU on Murre.
/dev/sdb2      532480 60432383 59899904 28.6G 83 Linux
</pre>


Now I need tools. I need pi-gen and that needs Docker.
See https://github.com/dhruvvyas90/qemu-rpi-kernel and also perhaps [https://linuxconfig.org/how-to-run-the-raspberry-pi-os-in-a-virtual-machine-with-qemu-and-kvm this article].
sudo apt-get update && sudo apt-get install qemu-system-arm qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virtinst libvirt-daemon virt-manager
git clone <nowiki>https://github.com/dhruvvyas90/qemu-rpi-kernel</nowiki>


<pre>
# customize the '''bold''' parts
curl -sSL https://get.docker.com | sh
sudo virsh --connect=qemu:///system net-start default
sudo usermod -aG docker $USER
sudo virt-install \
mkdir Documents
  --name rpios --arch armv6l --machine versatilepb \
cd Documents
  --cpu arm1176 --vcpus 1 --memory 256 --import  \
sudo apt install git dcfldd
  --disk '''2021-01-11-raspios-buster-armhf-lite.img''',format=raw,bus=virtio \
git clone https://github.com/RPI-Distro/pi-gen.git
  --network bridge,source=virbr0,model=virtio  \
</pre>
  --video vga --graphics spice \
  --boot 'dtb='''qemu-rpi-kernel/versatile-pb-buster.dtb''',kernel='''qemu-rpi-kernel/kernel-qemu-4.19.50-buster''',kernel_args=root=/dev/vda2 panic=1' \
  --events on_reboot=destroy
I put the long command in a file of course, called start_qemu.sh


[[Category: Raspberry Pi]]
[[Category: Raspberry Pi]]
[[Category: Embedded]]
[[Category: Embedded]]

Latest revision as of 23:06, 16 March 2024

Creating SD cards

There is a Windows program that some people use but I only use the Raspberry Pi imager, or sometimes Balena Etcher.

Copy IMG

Mick warns that pi-shrink is unreliable. Here it is all the same.

https://www.instructables.com/How-to-BackUp-and-Shrink-Your-Raspberry-Pi-Image/

I did this on Bellman and worked there I think. I think. Did I make notes?

sudo apt install dcfldd gparted

Building new images with pi-gen

pi-gen is what the Raspberry Pi folk use. See /boot/issue.txt on a standard image. For example,

Raspberry Pi reference 2023-12-11
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 2acf7afcba7d11500313a7b93bb55a2aae20b2d6, stage2

I have to run pi-gen on an x64 computer; I tried on a Pi5 and it would not run. I use the Docker version. I use Murre as my build machine, and I run an apt cache on Bellman to speed up repeated builds. I set up a pi-gen folder (on both Murre and Bellman).

You have to download pi-gen and you have to install the apt package "qemu-user-static" so that you can build arm64 images. You probably only need qemu on the builder not the cacher.

cd ~/Documents/Projects/TARRA
git clone https://github.com/RPI-Distro/pi-gen.git
cd pi-gen

Building for 32-bit bullseye

You need to switch to a different branch, this works,

git checkout remotes/origin/bullseye

Basic config file

RELEASE=bullseye
IMG_NAME=tarra_bullseye
TARGET_HOSTNAME=svxlink

WPA_COUNTRY=US
WPA_SSID=tarra
WPA_PASSWORD=tarra

ENABLE_SSH=1

LOCALE_DEFAULT=en_US.UTF-8
TIMEZONE_DEFAULT=Americas/Mountain
KEYBOARD_LAYOUT="English (US)"
KEYBOARD_KEYMAP=us

# leave the user "tarra" available                                                                                                                                 
DISABLE_FIRST_BOOT_USER_RENAME=1
FIRST_USER_NAME=tarra
FIRST_USER_PASS=XXXXXX

# Allow running in a emulator                                                                                                                                       
QEMU=1

# On Murre, use Bellman as an apt proxy when doing build_docker.sh                                                                                                 
# On Bellman, be the apt proxy (run "docker compose up -d")                                                                                                        
APT_PROXY=http://192.168.123.2:3142
     
STAGE_LIST="stage0 stage1 stage2"

On the "builder" machine in the pi-gen folder, I run "./build-docker.sh" and wait. I get a zipped img file in the deploy/ folder. From the console then I can burn the image using the Raspberry Pi imager to an SD Card or an NVME drive. The image will be named after the name in the config and today's date, for example, 2024-03-10-tarra.img

To build again, remove the docker container and (possibly edit config) and run the builder again.

docker rm -f pigen-work

Don't build everything!

Build a "lite" image only by touching some files.

touch stage3/SKIP stage4/SKIP stage5/SKIP stage3/SKIP_IMAGES stage4/SKIP_IMAGES stage5/SKIP_IMAGES

Building all 6 stages takes FOREVER and is useless for TARRA builds which is what we're doing here.

I am running it again with just a lite image and started it at 1638. Tick tock. All apt packages should be cached now. SO. How long? 1700. Great - takes about 20 minutes. This is acceptable.

Running arm images on Linux x64

You don't have to burn the image to an SD card, you can run it on Murre.

ARM8 (64 bit)

THERE IS NO QEMU KERNEL FOR BOOKWORM. STOP NOW.

ARM7 (32-bit)

Fortunately, this project (TARRA) has to use a 32-bit Bullseye image because of svxlink. So I should be able to run it in QEMU on Murre.

See https://github.com/dhruvvyas90/qemu-rpi-kernel and also perhaps this article.

sudo apt-get update && sudo apt-get install qemu-system-arm qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virtinst libvirt-daemon virt-manager 
git clone https://github.com/dhruvvyas90/qemu-rpi-kernel
# customize the bold parts
sudo virsh --connect=qemu:///system net-start default
sudo virt-install \
  --name rpios --arch armv6l --machine versatilepb \
  --cpu arm1176 --vcpus 1 --memory 256 --import  \
  --disk 2021-01-11-raspios-buster-armhf-lite.img,format=raw,bus=virtio \
  --network bridge,source=virbr0,model=virtio  \
  --video vga --graphics spice \
  --boot 'dtb=qemu-rpi-kernel/versatile-pb-buster.dtb,kernel=qemu-rpi-kernel/kernel-qemu-4.19.50-buster,kernel_args=root=/dev/vda2 panic=1' \
  --events on_reboot=destroy

I put the long command in a file of course, called start_qemu.sh