Uncategorized

Docker architecture problem

Asked 
Active today
Viewed 61 times
2

I have installed a fresh raspbian buster and docker from get.docker.com on a raspberry pi zero W [Edit: Does not work on raspbian jessie or stretch either]

I have run docker on that machine before, but now docker run <image> just starts and terminates about 18 seconds later, no output. I have tried several different Dockerfiles which works on a regular Linux laptop.

This is output to daemon.log while docker run runs. I see no errors, just a warning that (...)/mnt/shm can not be found. [Edit: This warning is also shown on working docker on pc, so it’s not relevant.] Note: I have now discovered that docker container ls -l shows that the containers end with error code 139, which is SIGSEGV

tail -f /var/log/daemon.log|grep docker &
root@pi:/# docker run hello-world
..8:44 pi systemd[1]: var-lib-docker-overlay2-8a2da5298e1(...)4ed0b781f\x2dinit-merged.mount: Succeeded.
..8:44 pi systemd[463]: var-lib-docker-overlay2-8a2da5298e1(...)4ed0b781f\x2dinit-merged.mount: Succeeded.
..8:44 pi systemd[1]: var-lib-docker-overlay2-8a2da5298e1(...)4ed0b781f-merged.mount: Succeeded.
..8:45 pi systemd[463]: var-lib-docker-overlay2-8a2da5298e1(...)4ed0b781f-merged.mount: Succeeded.
..8:48 pi systemd[1]: run-docker-runtime\x2drunc-moby-6c47caad0e(...)086e9ac535ba-runc.8fnFt5.mount: Succeeded.
..8:48 pi dhcpcd[277]: docker0: carrier acquired
..8:48 pi dhcpcd[277]: docker0: IAID 29:1f:af:46
..8:48 pi dhcpcd[277]: docker0: adding address fe80::f3ff:1d1a:3346:f173
..8:48 pi avahi-daemon[241]: Joining mDNS multicast group on interface docker0.IPv6 with address fe80::f3ff:1d1a:3346:f173.
..8:48 pi avahi-daemon[241]: New relevant interface docker0.IPv6 for mDNS.
..8:48 pi avahi-daemon[241]: Registering new address record for fe80::f3ff:1d1a:3346:f173 on docker0.*.
..8:48 pi dhcpcd[277]: docker0: soliciting an IPv6 router
..8:48 pi dhcpcd[277]: docker0: soliciting a DHCP lease
..8:54 pi dhcpcd[277]: docker0: probing for an IPv4LL address
..8:59 pi dhcpcd[277]: docker0: using IPv4LL address 169.254.156.109
..8:59 pi dhcpcd[277]: docker0: adding route to 169.254.0.0/16
..8:59 pi avahi-daemon[241]: Registering new address record for 169.254.156.109 on docker0.IPv4.
..9:00 pi dockerd[324]: time="2019-10-26T23:29:00.407340153+02:00" level=info msg="ignoring event"
     module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
..9:00 pi systemd[1]: run-docker-netns-1f13548fc883.mount: Succeeded.
..9:00 pi systemd[463]: run-docker-netns-1f13548fc883.mount: Succeeded.
..9:01 pi dockerd[324]: time="2019-10-26T23:29:01.039055309+02:00" level=warning msg="6c47caad0e(...)086e9ac535ba
    cleanup: failed to unmount IPC: umount /var/lib/docker/containers/6c47caad0e(...)086e9ac535ba/mounts/shm, flags: 0x2: no such file or directory"
..9:01 pi systemd[1]: var-lib-docker-overlay2-8a2da5298e1(...)4ed0b781f-merged.mount: Succeeded.
..9:01 pi systemd[463]: var-lib-docker-overlay2-8a2da5298e1(...)4ed0b781f-merged.mount: Succeeded.
..9:02 pi dhcpcd[277]: docker0: carrier lost
..9:02 pi dhcpcd[277]: docker0: deleting address fe80::f3ff:1d1a:3346:f173
..9:02 pi avahi-daemon[241]: Withdrawing address record for fe80::f3ff:1d1a:3346:f173 on docker0.
..9:02 pi avahi-daemon[241]: Leaving mDNS multicast group on interface docker0.IPv6 with address fe80::f3ff:1d1a:3346:f173.
..9:02 pi avahi-daemon[241]: Interface docker0.IPv6 no longer relevant for mDNS.
..9:02 pi avahi-daemon[241]: Withdrawing address record for 169.254.156.109 on docker0.
..9:02 pi dhcpcd[277]: docker0: deleting route to 169.254.0.0/16

Edit, to answer tflong01

tail -f /var/log/daemon.log keep printing the log from miscellaneous daemons |grep docker send output to grep to show only lines containing docker & keep running in background, but release keyboard back, so I can type the next command

docker takes a couple of disk images: a “disk” with the operating system, another with the command and mounts them on top of each other, creating a virtual machine.

at 8:48 this machine is powered up, it gets an IPv6 address and starts looking for an IPv4 address by DHCP. There is no DHCP on that network, so

at 8:54 it starts creating its own 169.254.0.0/16 adress, which it has done at 8:59

at 9:00 the machine looks ready to run the “hello-world” command, but no output is done

at 9:01 the machine powers down, the ip-adresses get released and the disk images unmounted and the routing entries cleared.

All looks normal except nothing is being run in the machine. Or it doesn’t output anything.

  • I am a linux newbie and don’t know nothing about “daemon.log” or “dhcpcd”. I saw that you are running “docker run hello-world”. I have just installed my docker, but I don’t how to setup to run “hello World”. I remember I did run Hello World some weeks ago and found it OK, but that was for Rpi3B+ stretch. I am using Rpi4B buster. I am going to run Hello World again. The following is my installation record. Perhaps you might like to compare my version with yours: penzu.com/p/b0b16fa9. – tlfong01 yesterday   
  • I was too lazy to check my old penzu diary to find last time what did I do to run hello world. So I just scribbled random commands to try out. Luckily the docker daemon is smart enough to guide me with newbie friendly error message to after a couple of trials and errors got the correct terminal command to successfully run hello world. The docker ghost is indeed very newbie friendly, discovering that I am a newbie (I guess using some sort of AI UI stuff), knowing no command and no “image” to run. The ghost went to her “hub”, fetched hello image (I think called image! ). / to continue, … – tlfong01 yesterday   
  • This is my hello world image execution penzu diary entry: penzu.com/p/43f37930. I found that you used the newbie scary command “tail -f /var/log/daemon.log|grep docker &”. I never can remember the meanings of “tail”, “|”, “grep”. Perhaps I just run it blindly and let you know the results. – tlfong01 yesterday   
  • I saved a log for reference. penzu.com/p/2f287724 I did not read it, because I found too many things I don’t understand: (1) Why mount and mount? Is it because the docker container is like a hard disk with a virtual machine needing setup and close down? (2) I noticed that there is something related to “dhcpcd”. Does this mean that my hello world program is sent up to the cloud and results comes back from there? If this is true, then I am not interested to learn more, because I always think online, send/process/receive to/from cloud is slow, compared with offline local processing. – tlfong01 yesterday   
  • 1
    I have explained the logfile in details. – Lenne yesterday
  • Oh my goodness, as I used to say, there are too many things I don’t know that I don’t know. I am grateful for your very detailed explanation, and I am surprised that you can write it up so concisely., and geared to the knowledge level of docker newbies. Last time I tried docker because I was trying to install a package which needs a couple of different versions of components which I found crashing each other, and I had the feeling that even experienced ninjas cannot solve the problem. Then I heard that docker is a “new” way to prevent version conflict. / to continue, … – tlfong01 yesterday    
  • Last time I did spent some 30 minutes reading Wikipedia to get a rough idea, and my quick and dirty conclusion was that docker is just like a light weight “Virtual Machine” which I learnt in IBM VM370 days (I was running CDC6600/7600 those years!). Now I am glad to have a mini VM in my micro. Anyway, Wiki was only scratching the surface, so your write up is digging one level deeper and I am very glad that you also explain that the the log daemon is logging several other little daemons on the fly. I am also surprised to know that IPv6 and DHCP network is playing in the background. Cheers! – tlfong01 yesterday   
  • 1
    I guess my explanation doesn’t get me further in solving my problem. But I hope it gives me some karma points 😉 – Lenne yesterday
  • I found Docker is not compatible with RpiZW buster. – tlfong01 18 hours ago   
  • I compared your log with mine and found them very similar. I first wild guess is that both Rpi4B and RpiZW completed successfully. The only catch might be in the display messages to the terminal problem. The messages might have indeed gone through to the display driver which is NOT compatible to docker. In other words it is the terminal display driver that causes problem. When I first tried RpiZW, I found the USB hub that used to work with Rpi3B does not work with RpiZW. I swapped USBs to check out which work. I found all my good quality USB hub do not work, / to continue, … – tlfong01 16 hours ago   
  • Only an old cheap USB hub works with RpiZW. So I conclude that it is the USB hub driver problem. – tlfong01 16 hours ago   

2 Answers

3

One needs to run the right architecture.

The kernel on my raspberry pi zero W is armv61

Whatever mechanism docker have to select the right arch is not solid enough.

For instance alpine:latest have images for

  • linux/386
  • linux/amd64
  • linux/arm/v6
  • linux/arm/v7
  • linux/arm64/v8
  • linux/ppc64le
  • linux/s390x

But the docker image inspect show both arm/v6 and arm/v7 as just “arm”. So docker pull alpine happens to select arm/v7, while RasPIzW need arm/v6

I can get the right arch by looking up the digest on dockerhub, and do a docker run alpine@sha256:29a82d50bdb8dd7814009852c1773fb9bb300d2f655bd1cd9e764e7bb1412be3 but that locks me to that particular version, so I’ll not automatically can use alpine:latest in my Dockerfile or docker-compose

I have solved it in this case, by using arm32v6/alpine, but that only works because some user called arm32v6 makes images specially for arm32v6

This also means I can’t have the same dockerfile on my pc as on the raspberry, if they use binary images.

I’d like a way to say

if arch="amdv61" then
  FROM arm32v6/alpine
else
  FROM alpine
end if

  • Ah, your explanation is very good. I need to make in my answer an apology later. Cheers. – tlfong01 37 mins ago   

0

Question

How come Docker running Hello World on RpiZW returns nothing?


Answer

Update 2019oct2901

The OP just found the cause of the problem. See his answer above. Let the docker newbie, ie, me, try to summarize the OP’s explanation:

(1) Rpi3/4 uses arm/v7, while RasPIzW uses arm/v6

(2) Docker by default uses architecture for arm/v7

(3) So it is RpiZW’s “architecture” not compatible to Docker, not just the display driver as I have wrongly guessed.

My apologies for the wrong and misleading guess. Many thanks to the OP for teaching me how to run and troubleshoot Docker stuff.


Update 2019oct2801

The OP just said the following:

“Note: I have now discovered that docker container ls -l shows that the containers end with error code 139, which is SIGSEGV”

I think this may mean my conclusion might be totally wrong. So I have stricken out my conclusion below, and wait for the OP to do more research work.


Experimentation docker

(1) I installed docker on Rpi4B with buster and ran Hello World OK (Refs 1, 2, 3).

(2) I cloned buster image from Rpi4B and used it to run hello world on RpiZW.

(3) Docker in RpiZW buster returns nothing. In other words, bad luck as the OP.


Discussion

I compared and contrasted your RpiZW log with mine. I found them did similar things and seem to end successfully in a roughly same time period.

In other words, both docker guys think they have completed the Hello World job. The only catch is that the Docker has no eyes to look at the terminal display that it does display any thing.

In short, the Rpi4B terminal display driver gets the Docker message and display happily. But the RpiZW driver does not do anything, just sleeping!


Conclusion

Docker is not compatible with RpiZW buster.

Errata – As mentioned in the update above, the OP has found the error message which might indicate the real cause of the problem, not at all related to my quick and dirty conclusion of a vague remark on incompatibility.

Updated conclusion

It is Docker’s default Rpi architecture not compatible, not just driver incompatible, though implied.

See the OP’s answer for a very detailed explanation.


References

(1) Docker Rpi4B buster Installation Record v0.1 – tlfong01 Sun. 10/27/2019

(2) Docker Rpi4B buster Hello World Execution Record v0.1 – tlfong01 Sun. 10/27/2019

(3) Docker Rpi4B buster Hello World Execution Log v0.2 – tlfong01 Sun. 10/27/2019

(4) Docker Learning Notes V0.3 – tlfong01 Sun. 10/27/2019

(5) Understanding Docker Container Technology for Beginner – GeekFlare

(6) Docker Architecture and its Components for Beginner – GeekFlare

docker


End of answer

  • 1
    I tried a fresh raspbian stretch, same result, Docker does everything in building, starting and stopping containers. Except actually running the container. – Lenne 16 hours ago
  • I am not surprised, my RpiZW with stretch or buster does not like any good quality USB hub except the old dirty cheap HUB I nearly gave away to charity. – tlfong01 16 hours ago    
  • Dinner time. See you late evening or tomorrow. Cheers. – tlfong01 16 hours ago   
  • 1
    My RpiZW runs just fine, except for the docker issue – Lenne 16 hours ago
  • Yes, I agree. The docker knows how to talk to the Rpi4B terminal display driver, but not to the RpiZW display driver. – tlfong01 14 hours ago   

Categories: Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: