Google Coral

First snow in 2020. Actually, it is ALSO the FIRST snow for the winter from 2019 to 2020.

First Snow 1 First Snow 2 First Snow 3
First Snow 1 First Snow 2 First Snow 3

Both my son and the Chinese New Year are coming. Let’s start the mode of celebrating. Today, I’m going to do the hotpot.

Hotpot 1 Hotpot 2 Hotpot 3
Hotpot 1 Hotpot 2 Hotpot 3

It looks in 1-day time, everybody is doing the edge computing. Today, we’re going to have some fun of Google Coral.

1. Google Coral USB Accelerator

Image cited from Coral official website.

Google Coral Accelerator

To try out Google Coral USB Accelerator is comparitively simple. The ONLY thing to do is just to follow Google Doc - Get started with the USB Accelerator. Anyway, let’s test it out with the following commands.

Make sure we are able to list the device.

1
2
3
➜  classification git:(master) ✗ lsusb
Bus 002 Device 003: ID 1a6e:089a Global Unichip Corp.
...

We then run the example.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
➜  classification git:(master) ✗ python3 classify_image.py \
--model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels models/inat_bird_labels.txt \
--input images/parrot.jpg
INFO: Initialized TensorFlow Lite runtime.
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
12.6ms
3.9ms
4.3ms
4.2ms
4.1ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.76172

BTW, I’m going to discuss

sooner or later. Just keep an eye on my blog.

2. Google Coral Dev Board

In the following, we’re going to disscuss Google Coral Dev Board more. Image cited from Coral official website.

Google Coral Devboard

2.1 Mendel Installation

2.1.1 Mendel Linux Preparation

Google Corel Mendel Linux can be downloaded from https://coral.ai/software/. In our case, we are going to try Mendel Linux 4.0.

2.1.2 Connect Dev Board Via Micro-USB Serial Port

On the host, we should be able to see:

1
2
3
4
5
6
7
8
9
10
11
12
➜  mendel-enterprise-day-13 lsusb
......
Bus 001 Device 020: ID 10c4:ea70 Cygnal Integrated Products, Inc. CP210x UART Bridge
......
➜ mendel-enterprise-day-13 dmesg | grep ttyUSB
[45021.091322] usb 1-8: cp210x converter now attached to ttyUSB0
[45021.092681] usb 1-8: cp210x converter now attached to ttyUSB1
➜ mendel-enterprise-day-13 ll /dev/ttyUSB*
crwxrwxrwx 1 root dialout 188, 0 Apr 6 21:33 /dev/ttyUSB0
crwxrwxrwx 1 root dialout 188, 1 Apr 6 21:25 /dev/ttyUSB1
➜ mendel-enterprise-day-13 screen /dev/ttyUSB0 115200
.....

Now what you see is a black screen. After having connected the Type C power cable, you should be able to see:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
......
[ 2871.285085] NOHZ: local_softirq_pending 08
[ 2871.376306] NOHZ: local_softirq_pending 08
[ 2872.117716] NOHZ: local_softirq_pending 08
[ 2874.283909] NOHZ: local_softirq_pending 08
[ 2875.286859] NOHZ: local_softirq_pending 08

U-Boot SPL 2017.03.3 (Nov 08 2019 - 22:29:30)
power_bd71837_init
pmic debug: name=BD71837
Board id: 6
check ddr4_pmu_train_imem code
check ddr4_pmu_train_imem code pass
check ddr4_pmu_train_dmem code
check ddr4_pmu_train_dmem code pass
Training PASS
Training PASS
check ddr4_pmu_train_imem code
check ddr4_pmu_train_imem code pass
check ddr4_pmu_train_dmem code
check ddr4_pmu_train_dmem code pass
Training PASS
Normal Boot
Trying to boot from MMC1
hdr read sector 300, count=1


U-Boot 2017.03.3 (Nov 08 2019 - 22:29:30 +0000)

CPU: Freescale i.MX8MQ rev2.0 1500 MHz (running at 1000 MHz)
CPU: Commercial temperature grade (0C to 95C) at 64C
Reset cause: POR
Model: Freescale i.MX8MQ Phanbell
DRAM: 1 GiB
Board id: 6
Baseboard id: 1
MMC: FSL_SDHC: 0, FSL_SDHC: 1
*** Warning - bad CRC, using default environment

In: serial
Out: serial
Err: serial

BuildInfo:
- ATF
- U-Boot 2017.03.3

flash target is MMC:0
Net:
Warning: ethernet@30be0000 using MAC address from ROM
eth0: ethernet@30be0000
Fastboot: Normal
Hit any key to stop autoboot: 0
u-boot=> [A

That is shown on the screen monitor of Google Coral Dev Board. After having connected the Type C OTG cable, we should be able to see on the host:

1
2
➜  mendel-enterprise-day-13 fastboot devices
101989d6f32efb39 fastboot

2.1.3 Flash Corel Dev Board

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
➜  mendel-enterprise-day-13 ls
boot_arm64.img flash.sh partition-table-16gb.img partition-table-64gb.img partition-table-8gb.img README recovery.img rootfs_arm64.img u-boot.imx
➜ mendel-enterprise-day-13 bash flash.sh
Sending 'bootloader0' (991 KB) OKAY [ 0.055s]
Writing 'bootloader0' OKAY [ 0.190s]
Finished. Total time: 0.266s
Rebooting into bootloader OKAY [ 0.024s]
Finished. Total time: 0.125s
Sending 'gpt' (33 KB) OKAY [ 0.018s]
Writing 'gpt' OKAY [ 0.309s]
Finished. Total time: 0.346s
Rebooting into bootloader OKAY [ 0.022s]
Finished. Total time: 0.122s
Erasing 'misc' OKAY [ 0.069s]
Finished. Total time: 0.079s
Sending 'boot' (131072 KB) OKAY [ 5.321s]
Writing 'boot' OKAY [ 3.632s]
Finished. Total time: 8.972s
Sending sparse 'rootfs' 1/4 (368422 KB) OKAY [ 14.792s]
Writing 'rootfs' OKAY [ 36.191s]
Sending sparse 'rootfs' 2/4 (408501 KB) OKAY [ 16.646s]
Writing 'rootfs' OKAY [ 18.944s]
Sending sparse 'rootfs' 3/4 (389107 KB) OKAY [ 15.881s]
Writing 'rootfs' OKAY [ 37.021s]
Sending sparse 'rootfs' 4/4 (231325 KB) OKAY [ 9.482s]
Writing 'rootfs' OKAY [ 65.204s]
Finished. Total time: 214.205s
Rebooting OKAY [ 0.005s]
Finished. Total time: 0.105s
➜ mendel-enterprise-day-13

2.1.4 Boot Mendel

Screen Snapshot Reboot

After a while, you’ll see:

Screen Snapshot Login

Now, login with

  • username: mendel
  • password: mendel
1
2
➜  ~ mdt devices
mocha-shrimp (192.168.100.2)

You will be able to see Google Coral Dev Board is NOW connected. If you don’t see the EXPECTED output mocha-shrimp (192.168.101.2), just plug out and plug in the Type C power cable again.

Unfortunately, mdt tool does NOT work properly.

1
2
3
4
5
6
7
8
9
10
11
➜  mendel-enterprise-day-13 mdt shell
Waiting for a device...
Connecting to mocha-shrimp at 192.168.101.2
Key not present on mocha-shrimp -- pushing

It looks like you're trying to connect to a device that isn't connected
to your workstation via USB and doesn't have the SSH key this MDT generated.
To connect with `mdt shell` you will need to first connect to your device
ONLY via USB.

Cowardly refusing to attempt to push a key to a public machine.

This bug has been clarified on StackOverflow. By modifying file vim $HOME/.local/lib/python3.6/site-packages/mdt/sshclient.py line 86, from if not self.address.startswith('192.168.100'): to if not self.address.startswith('192.168.10'):, problem solved.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
➜  mendel-enterprise-day-13 mdt shell
Waiting for a device...
Connecting to mocha-shrimp at 192.168.101.2
Key not present on mocha-shrimp -- pushing
Linux mocha-shrimp 4.14.98-imx #1 SMP PREEMPT Fri Nov 8 23:28:21 UTC 2019 aarch64

The programs included with the Mendel GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Mendel GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Feb 14 10:12:02 2019
mendel@mocha-shrimp:~$ ls
mendel@mocha-shrimp:~$ pwd
/home/mendel
mendel@mocha-shrimp:~$ uname -a
Linux mocha-shrimp 4.14.98-imx #1 SMP PREEMPT Fri Nov 8 23:28:21 UTC 2019 aarch64 GNU/Linux
mendel@mocha-shrimp:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Mendel
Description: Mendel GNU/Linux 4 (Day)
Release: 10.0
Codename: day
mendel@mocha-shrimp:~$ ip -c address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 7c:d9:5c:b1:fa:cc brd ff:ff:ff:ff:ff:ff
3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 3000
link/ether 7c:d9:5c:b1:fa:cd brd ff:ff:ff:ff:ff:ff
4: p2p0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 3000
link/ether 00:0a:f5:89:89:81 brd ff:ff:ff:ff:ff:ff
5: usb0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether 02:22:78:0d:f6:df brd ff:ff:ff:ff:ff:ff
inet 192.168.100.2/24 brd 192.168.100.255 scope global noprefixroute usb0
valid_lft forever preferred_lft forever
inet6 fe80::cc6d:b3d4:f07e:eed1/64 scope link tentative noprefixroute
valid_lft forever preferred_lft forever
6: usb1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 02:22:78:0d:f6:de brd ff:ff:ff:ff:ff:ff
inet 192.168.101.2/24 brd 192.168.101.255 scope global noprefixroute usb1
valid_lft forever preferred_lft forever
inet6 fe80::5bf4:c217:d9c9:859c/64 scope link noprefixroute
valid_lft forever preferred_lft forever
mendel@mocha-shrimp:~$

After activate the Internet by nmtui, we can NOW clearly see the wlan0 IP is automatically allocated.

1
2
3
4
5
6
7
8
9
10
11
mendel@mocha-shrimp:~$ ip -c address
......
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 3000
link/ether 7c:d9:5c:b1:fa:cd brd ff:ff:ff:ff:ff:ff
inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic noprefixroute wlan0
valid_lft 86367sec preferred_lft 86367sec
inet6 2001:569:7e6e:dc00:d1c4:697a:f60e:b5a4/64 scope global dynamic noprefixroute
valid_lft 7468sec preferred_lft 7168sec
inet6 fe80::e10b:9dc6:60c4:b91b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
......

Of course, we can setup a static IP for this particular Google Coral Dev Board afterwards.

2.1.5 SSH into Mendel

In order to SSH into Mendel and connect remotely, we need to do Connect to a board’s shell on the host computer. You MUST pushkey before you can ssh into the board via the Internet IP instead of the virtual IP via USB, say 192.168.100.2 or 192.168.101.2.

2.2 Demonstration

2.2.1 edgetpu_demo --device & edgetpu_demo --stream

Let’s ignore edgetpu_demo --device for I ALMOST NEVER work with a GUI mode. The demo video is on my youtube channel, please refer to:

On console, it just displays as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
mendel@deft-orange:~$ edgetpu_demo --stream
Press 'q' to quit.
Press 'n' to switch between models.

(edgetpu_detect_server:9991): Gtk-WARNING **: 07:56:57.725: Locale not supported by C library.
Using the fallback 'C' locale.
INFO:edgetpuvision.streaming.server:Listening on ports tcp: 4665, web: 4664, annexb: 4666
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37536
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37538
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.1.200:37536] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37540
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37542
INFO:edgetpuvision.streaming.server:Number of active clients: 3
INFO:edgetpuvision.streaming.server:[192.168.1.200:37538] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37540] Rx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:37544
INFO:edgetpuvision.streaming.server:[192.168.1.200:37538] Tx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37542] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:37542] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 4
......

2.2.2 Classification

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mendel@mocha-shrimp:~/Downloads/tflite/python/examples/classification$ python3 classify_image.py \
> --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
> --labels models/inat_bird_labels.txt \
> --input images/parrot.jpg
INFO: Initialized TensorFlow Lite runtime.
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
14.2ms
2.7ms
2.6ms
2.6ms
2.9ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.76562

2.2.3 Camera

2.2.3.1 Google Coral camera

The Google Coral camera can be detected as a video device:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
mendel@mocha-shrimp:~$ v4l2-ctl --list-formats-ext --device /dev/video0
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture

[0]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 2592x1944
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 0x0

2.2.3.2 Face Detection Using Google TPU

My youtube real-time face detection video clearly shows Google TPU is seriously powerful.

On console, it displays:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
mendel@deft-orange:~$ edgetpu_detect_server \
> --model ${DEMO_FILES}/mobilenet_ssd_v2_face_quant_postprocess_edgetpu.tflite

(edgetpu_detect_server:4081): Gtk-WARNING **: 10:40:45.436: Locale not supported by C library.
Using the fallback 'C' locale.
INFO:edgetpuvision.streaming.server:Listening on ports tcp: 4665, web: 4664, annexb: 4666
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:33950
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Tx thread finished
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.1.200:33950] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 0
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:33952
INFO:edgetpuvision.streaming.server:Number of active clients: 1
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Tx thread finished
INFO:edgetpuvision.streaming.server:New web connection from 192.168.1.200:33954
INFO:edgetpuvision.streaming.server:Number of active clients: 2
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Stopping...
INFO:edgetpuvision.streaming.server:[192.168.1.200:33954] Rx thread finished
INFO:edgetpuvision.streaming.server:[192.168.1.200:33952] Stopped.
INFO:edgetpuvision.streaming.server:Number of active clients: 1
......