Today's concert: ONE WORLD : TOGETHER AT HOME. Yup, today, I've my previous blog updated. A lot of modifications. Khadas VIM3 is really a good product. With Amlogic's A311D with 5.0 TOPS NPU, the board itself comes with a super powerful AI inference capability.
AI inference unit used to be in USB sticks, such as:
What a sunny day after the FIRST snow in this winter. Let me show you 3 pictures in the first row, and 3 videos in the second. We need to enjoy both R&D and life…
Green Timers Lake 1
Green Timers Lake 2
Green Timers Park
A Pair of Swans
A Group of Ducks
A Little Stream In The Snow
After a brief break, I started investigating Khadas VIM3 again.
1. About Khadas VIM3
Khadas VIM3 is a super computer based on Amlogic A311D. Before we start, let’s carry out several simple comparisons.
1.1 Raspberry Pi 4 Model B vs. Khadas VIM3 vs. Jetson Nano Developer Kit
2. Install Prebuilt Operating System To EMMC Via Krescue
2.1 WIRED Connection Preferred
As mentioned in VIM3 Beginners Guide, Krescue is a Swiss Army knife. As of January 2020, Krescue can download and install OS images directly from the web via wired Ethernet.
2.2 Flash Krescue Onto SD Card
1 2 3 4 5
➜ Krescue sudo dd bs=4M if=VIM3.krescue-d41d8cd98f00b204e9800998ecf8427e-1587199778-67108864-279c13890fa7253d5d2b76000769803e.sd.img of=/dev/mmcblk0 conv=fsync [sudo] password for longervision: 16+0 records in 16+0 records out 67108864 bytes (67 MB, 64 MiB) copied, 4.03786 s, 16.6 MB/s
2.3 Setup Wifi From Within Krescue Shell
If you really don't like the WIRED connection, boot into Krescue shell, and use the following commands to set up Wifi:
➜ ~ ping 192.168.1.110 PING 192.168.1.110 (192.168.1.110) 56(84) bytes of data. 64 bytes from 192.168.1.110: icmp_seq=1 ttl=64 time=140 ms 64 bytes from 192.168.1.110: icmp_seq=2 ttl=64 time=54.0 ms 64 bytes from 192.168.1.110: icmp_seq=3 ttl=64 time=13.1 ms ^C --- 192.168.1.110 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 13.191/69.145/140.193/52.936 ms ➜ ~ ssh root@192.168.1.110 The authenticity of host '192.168.1.110 (192.168.1.110)' can't be established. RSA key fingerprint is SHA256:0t0PZw/24nWc8hWaCJkltYtwCduMMSlRuux2Nn865Os. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.1.110' (RSA) to the list of known hosts.
BusyBox v1.28.4 () built-in shell (ash)
OpenWrt 18.06.3, r7798-97ae9e0ccb
__ _____ Khadas ## hyphop ## / //_/ _ \___ ___ ______ _____ / ,< / , _/ -_|_-</ __/ // / -_) /_/|_/_/|_|\__/___/\__/\_,_/\__/ extreme tiny and fast rescue system
=== WARNING! ===================================== There is no root password defined on this device! Use the passwd command to set up a new password in order to prevent unauthorized SSH logins. -------------------------------------------------- root@Krescue:~# uname -a Linux Krescue 5.4.5 #4 SMP PREEMPT Thu Apr 9 22:07:48 +09 2020 aarch64 GNU/Linux
2.5 Flash OS onto EMMC (WIRED Connection Preferred)
Let's take a look at the SD card device:
1 2
root@Krescue:~# ls /dev/mmcblk* /dev/mmcblk1 /dev/mmcblk1p1 /dev/mmcblk1p2 /dev/mmcblk2 /dev/mmcblk2boot0 /dev/mmcblk2boot1 /dev/mmcblk2rpmb
➜ ~ ssh khadas@192.168.1.95 The authenticity of host '192.168.1.95 (192.168.1.95)' can't be established. ECDSA key fingerprint is SHA256:Q59XrIX7bSWsphZCpgHBSnVH5ETgCY9iLfDEuvRKtOw. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.1.95' (ECDSA) to the list of known hosts. khadas@192.168.1.95's password:
khadas@Khadas:~$ uname -a Linux Khadas 4.9.206 #13 SMP PREEMPT Tue Dec 31 00:37:47 CST 2019 aarch64 aarch64 aarch64 GNU/Linux khadas@Khadas:~$ cat /proc/cpuinfo processor : 0 BogoMIPS : 48.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4
processor : 1 BogoMIPS : 48.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd03 CPU revision : 4
processor : 2 BogoMIPS : 48.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd09 CPU revision : 2
processor : 3 BogoMIPS : 48.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd09 CPU revision : 2
processor : 4 BogoMIPS : 48.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd09 CPU revision : 2
processor : 5 BogoMIPS : 48.00 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 CPU implementer : 0x41 CPU architecture: 8 CPU variant : 0x0 CPU part : 0xd09 CPU revision : 2
Serial : 290b1000010c1900000437304e424e50 Hardware : Khadas VIM3 khadas@Khadas:~$ clinfo Number of platforms 1 Platform Name ARM Platform Platform Vendor ARM Platform Version OpenCL 2.0 git.c8adbf9.ad00b04c1b60847de257177231dc1a53 Platform Profile FULL_PROFILE Platform Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp16 cl_khr_icd cl_khr_egl_image cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_create_command_queue cl_arm_core_id cl_arm_printf cl_arm_thread_limit_hint cl_arm_non_uniform_work_group_size cl_arm_import_memory cl_arm_shared_virtual_memory Platform Extensions function suffix ARM
Platform Name ARM Platform Number of devices 1 ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Device Name <printDeviceInfo:0: get CL_DEVICE_NAME size : error -6> Device Vendor ARM ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Device Vendor ID <printDeviceInfo:2: get CL_DEVICE_VENDOR_ID : error -6> Device Version OpenCL 2.0 git.c8adbf9.ad00b04c1b60847de257177231dc1a53 Driver Version 2.0 Device OpenCL C Version OpenCL C 2.0 git.c8adbf9.ad00b04c1b60847de257177231dc1a53 Device Type GPU Device Profile FULL_PROFILE Device Available Yes Compiler Available Yes Linker Available Yes ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Max compute units <printDeviceInfo:17: get CL_DEVICE_MAX_COMPUTE_UNITS : error -6> ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Max clock frequency <printDeviceInfo:21: get CL_DEVICE_MAX_CLOCK_FREQUENCY : error -6> Device Partition (core) Max number of sub-devices 0 Supported partition types None Max work item dimensions 3 ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Max work item sizes <printDeviceInfo:36: get number of CL_DEVICE_MAX_WORK_ITEM_SIZES : error -6> ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Max work group size <printDeviceInfo:37: get CL_DEVICE_MAX_WORK_GROUP_SIZE : error -6> ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Preferred work group size multiple <getWGsizes:671: create context : error -6> Preferred / native vector sizes char 16 / 4 short 8 / 2 int 4 / 1 long 2 / 1 half 8 / 2 (cl_khr_fp16) float 4 / 1 double 0 / 0 (n/a) Half-precision Floating-point support (cl_khr_fp16) Denormals Yes Infinity and NANs Yes Round to nearest Yes Round to zero Yes Round to infinity Yes IEEE754-2008 fused multiply-add Yes Support is emulated in software No Single-precision Floating-point support (core) Denormals Yes Infinity and NANs Yes Round to nearest Yes Round to zero Yes Round to infinity Yes IEEE754-2008 fused multiply-add Yes Support is emulated in software No Correctly-rounded divide and sqrt operations No Double-precision Floating-point support (n/a) Address bits 64, Little-Endian ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Global memory size <printDeviceInfo:74: get CL_DEVICE_GLOBAL_MEM_SIZE : error -6> Error Correction support No ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Max memory allocation <printDeviceInfo:80: get CL_DEVICE_MAX_MEM_ALLOC_SIZE : error -6> Unified memory for Host and Device Yes ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Shared Virtual Memory (SVM) capabilities <printDeviceInfo:83: get CL_DEVICE_SVM_CAPABILITIES : error -6> ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Shared Virtual Memory (SVM) capabilities (ARM) <printDeviceInfo:84: get CL_DEVICE_SVM_CAPABILITIES_ARM : error -6> Minimum alignment for any data type 128 bytes Alignment of base address 1024 bits (128 bytes) Preferred alignment for atomics SVM 0 bytes Global 0 bytes Local 0 bytes Max size for global variable 65536 (64KiB) Preferred total size of global vars 0 Global Memory cache type Read/Write ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Global Memory cache size <printDeviceInfo:97: get CL_DEVICE_GLOBAL_MEM_CACHE_SIZE : error -6> ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Global Memory cache line size <printDeviceInfo:98: get CL_DEVICE_GLOBAL_MEM_CACHELINE_SIZE : error -6> Image support Yes Max number of samplers per kernel 16 Max size for 1D images from buffer 65536 pixels Max 1D or 2D image array size 2048 images Base address alignment for 2D image buffers 32 bytes Pitch alignment for 2D image buffers 64 pixels Max 2D image size 65536x65536 pixels Max 3D image size 65536x65536x65536 pixels Max number of read image args 128 Max number of write image args 64 Max number of read/write image args 64 Max number of pipe args 16 Max active pipe reservations 1 Max pipe packet size 1024 Local memory type Global Local memory size 32768 (32KiB) Max number of constant args 8 Max constant buffer size 65536 (64KiB) Max size of kernel argument 1024 Queue properties (on host) Out-of-order execution Yes Profiling Yes Queue properties (on device) Out-of-order execution Yes Profiling Yes Preferred size 2097152 (2MiB) Max size 16777216 (16MiB) Max queues on device 1 Max events on device 1024 Prefer user sync for interop No ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. Profiling timer resolution <printDeviceInfo:145: get CL_DEVICE_PROFILING_TIMER_RESOLUTION : error -6> Execution capabilities Run OpenCL kernels Yes Run native kernels No printf() buffer size 1048576 (1024KiB) Built-in kernels Device Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_3d_image_writes cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp16 cl_khr_icd cl_khr_egl_image cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_create_command_queue cl_arm_core_id cl_arm_printf cl_arm_thread_limit_hint cl_arm_non_uniform_work_group_size cl_arm_import_memory cl_arm_shared_virtual_memory
NULL platform behavior clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) ARM Platform clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [ARM] ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. clCreateContext(NULL, ...) [default] <checkNullCtx:2694: create context with device from default platform : error -6> ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) <checkNullCtxFromType:2737: create context from type CL_DEVICE_TYPE_DEFAULT : error -6> clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) <checkNullCtxFromType:2737: create context from type CL_DEVICE_TYPE_GPU : error -6> clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform ERROR: The DDK (built for 0x70030000 r0p0 status range [0..15]) is not compatible with this Mali GPU device, /dev/mali0 detected as 0x7212 r0p0 status 0. Failed creating base context during DDK compatibility check. clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) <checkNullCtxFromType:2737: create context from type CL_DEVICE_TYPE_ALL : error -6>
ICD loader properties ICD loader Name OpenCL ICD Loader ICD loader Vendor OCL Icd free software ICD loader Version 2.2.11 ICD loader Profile OpenCL 2.1 khadas@Khadas:~$ cat /proc/partitions major minor #blocks name
khadas@Khadas:~$ gcc --version gcc (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) 7.5.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Name: OpenCV Description: Open Source Computer Vision Library Version: 3.4.3 Libs: -L${exec_prefix}/lib -lopencv_dnn -lopencv_ml -lopencv_objdetect -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videostab -lopencv_calib3d -lopencv_features2d -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_video -lopencv_photo -lopencv_imgproc -lopencv_flann -lopencv_core Libs.private: -ldl -lm -lpthread -lrt Cflags: -I${includedir_old} -I${includedir_new} khadas@Khadas:~$ sudo apt remove opencv3 Reading package lists... Done Building dependency tree Reading state information... Done The following package was automatically installed and is no longer required: libllvm8 Use 'sudo apt autoremove' to remove it. The following packages will be REMOVED: opencv3 0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded. After this operation, 1024 B disk space will be freed. Do you want to continue? [Y/n] Y (Reading database ... 118978 files and directories currently installed.) Removing opencv3 (3.4.3-2) ... khadas@Khadas:~$ ls /usr/lib/libopencv* ls: cannot access '/usr/lib/libopencv*': No such file or directory khadas@Khadas:~$ cat /usr/lib/pkgconfig/opencv.pc cat: /usr/lib/pkgconfig/opencv.pc: No such file or directory
It looks current OpenCV on current VIM3_Ubuntu-xfce-bionic_Linux-4.9_arm64_EMMC_V20191231.img** is a kind of outdated. Let's just remove package opencv3** and have OpenCV-4.3.0 installed manually.
To flash either of the above systems onto a TF/SD card is simple. However, both are ONLY for SD-USB, instead of EMMC. For instancen:
1 2 3
➜ Manjaro burn-tool -b VIM3 -i ./Manjaro-ARM-xfce-vim3-20.04.img Try to burn Amlogic image... ERROR: Try to burn to eMMC storage, but the image installation type is 'SD-USB', please use 'EMMC' image!
In this section, we're testing the computing capability of Khadas VIM3's NPU.
Before everything starts, make sure you have the galcore module loaded, by using command modinfo galcore.
4.1 Obtain aml_npu_sdk From Khadas
Extract the obtained aml_npu_sdk.tgz on your local host. Bear in mind that it is your local host, BUT NOT Khadas VIM3. Relative issues can be found at:
Afterwards, the models applicable on Khadas VIM3 can be obtained by following Model Conversion. Anyway, on my laptop, I obtained the converted model as follows:
Do I need to emphasize that I'm using Tensorflow 2.1.0 ? Anyway, check the following:
1 2 3 4 5 6 7 8
➜ ~ python Python 3.6.9 (default, Apr 18 2020, 01:56:04) [GCC 8.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import tensorflow as tf 2020-04-29 03:11:24.272348: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.2 >>> tf.__version__ '2.1.0'
4.3 Build Case Code
4.3.1 Cross-build on Host
You can of course cross-build the case code on your local host, instead of Khadas VIM3 by referring to Compile the Case Code. (The document seems NOT updated yet.) Instead of using 1 argument, we specify 2 auguments, one for aml_npu_sdk, the other for Fenix.
inceptionv3 now should be ready to use, but in my case, it's NOT working properly. It's probably because Fenix is NOT able to provide/represent the correct cross-compile toolchains for my installed VIMx.Ubuntu-xfce-bionic_Linux-4.9_arm64_V20191231.emmc.kresq. Anyway, this is NOT my preference.
khadas@Khadas:~$ sudo apt install aml-npu Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libllvm8 libssh-dev Use 'sudo apt autoremove' to remove them. The following NEW packages will be installed: aml-npu 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. Need to get 0 B/3318 kB of archives. After this operation, 1024 B of additional disk space will be used. Selecting previously unselected package aml-npu. (Reading database ... 136037 files and directories currently installed.) Preparing to unpack .../aml-npu_6.4.0.3_arm64.deb ... Unpacking aml-npu (6.4.0.3) ... Setting up aml-npu (6.4.0.3) ...
And with command line dpkg -L aml-npu, you'll see what's been installed by aml-npu. However, due to its commercial license, I may NOT be allowed to show anything here in my blog.
{% emoji no_mouth %}
4.4.2 Step 2: Install aml-npu-demo and Run Demo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
khadas@Khadas:~$ sudo apt install aml-npu-demo Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libllvm8 libssh-dev Use 'sudo apt autoremove' to remove them. The following NEW packages will be installed: aml-npu-demo 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. Need to get 0 B/19.7 MB of archives. After this operation, 1024 B of additional disk space will be used. Selecting previously unselected package aml-npu-demo. (Reading database ... 136098 files and directories currently installed.) Preparing to unpack .../aml-npu-demo_6.3.3.4_arm64.deb ... Unpacking aml-npu-demo (6.3.3.4) ... Setting up aml-npu-demo (6.3.3.4) ...
Where is the sample to run? /usr/share/npu/inceptionv3.
khadas@Khadas:~$ cd /usr/share/npu/inceptionv3 khadas@Khadas:/usr/share/npu/inceptionv3$ ./inceptionv3 ./inception_v3.nb ./dog_299x299.jpg D [setup_node:368]Setup node id[0] uid[4294967295] op[NBG] D [print_tensor:136]in(0) : id[ 1] vtl[0] const[0] shape[ 3, 299, 299, 1 ] fmt[u8 ] qnt[ASM zp=137, scale=0.007292] D [print_tensor:136]out(0): id[ 0] vtl[0] const[0] shape[ 1001, 1 ] fmt[f16] qnt[NONE] D [optimize_node:312]Backward optimize neural network D [optimize_node:319]Forward optimize neural network I [compute_node:261]Create vx node Create Neural Network: 37ms or 37726us I [vsi_nn_PrintGraph:1421]Graph: I [vsi_nn_PrintGraph:1422]***************** Tensors ****************** D [print_tensor:146]id[ 0] vtl[0] const[0] shape[ 1001, 1 ] fmt[f16] qnt[NONE] D [print_tensor:146]id[ 1] vtl[0] const[0] shape[ 3, 299, 299, 1 ] fmt[u8 ] qnt[ASM zp=137, scale=0.007292] I [vsi_nn_PrintGraph:1431]***************** Nodes ****************** I [vsi_nn_PrintNode:159]( NBG)node[0] [in: 1 ], [out: 0 ] [10587cb0] I [vsi_nn_PrintGraph:1440]****************************************** I [vsi_nn_ConvertTensorToData:750]Create 268203 data. Verify... Verify Graph: 1ms or 1811us Start run graph [1] times... Run the 1 time: 28ms or 28075us vxProcessGraph execution time: Total 28ms or 28091us Average 28.09ms or 28091.00us I [vsi_nn_ConvertTensorToData:750]Create 2002 data. --- Top5 --- 208: 0.819824 209: 0.040344 223: 0.009354 185: 0.002956 268: 0.002829 I [vsi_nn_ConvertTensorToData:750]Create 2002 data.
The program runs smoothly.
{% emoji smirk %}
4.4.3 Step 3: Build Your Own Executable and Run
Clearly, ALL (really???) required development files have been provided by aml-npu, in such, we should be able to build this demo inceptionv3 out by ourselves.
Besides aml-npu from repo, in order to have the demo inceptionv3fully and successfully built, you still need aml_npu_sdk from Khadas. In my case, you do need acuity-ovxlib-dev, and let's do export ACUITY_OVXLIB_DEV=path_to_acuity-ovxlib-dev.
4.4.3.2 Build inceptionv3 from Source
We don't need to copy the entire aml_npu_sdk onto Khadas VIM3, but ONLY demo/inceptionv3. Here in my case, ONLY demo/inceptionv3 is copied under ~/Programs.
In fact, you still need to modify common.target a little bit accordingly. However, to disclose it in this blog is still NOT allowed I think. Anyway, after the modification, let's make it.
1 2 3 4 5 6 7 8 9 10
khadas@Khadas:~/Programs/inceptionv3$ make cc -c -I/opt/acuity-ovxlib-dev/include -o bin_r/vnn_pre_process.o vnn_pre_process.c cc -c -I/opt/acuity-ovxlib-dev/include -o bin_r/vnn_inceptionv3.o vnn_inceptionv3.c cc -c -I/opt/acuity-ovxlib-dev/include -o bin_r/main.o main.c cc -c -I/opt/acuity-ovxlib-dev/include -o bin_r/vnn_post_process.o vnn_post_process.c cc -Wl,-rpath-link /opt/vivante_sdk/drivers bin_r/vnn_pre_process.o bin_r/vnn_inceptionv3.o bin_r/main.o bin_r/vnn_post_process.o -o bin_r/inceptionv3 -L/lib -lOpenVX -lOpenVXU -lCLC -lVSC -lGAL -lovxlib -lm -ljpeg -lrt bin_r/inceptionv3 Usage: bin_r/inceptionv3 data_file inputs... /opt/vivante_sdk/common.target:64: recipe for target 'bin_r/inceptionv3' failed make: *** [bin_r/inceptionv3] Error 255
Don't worry about the error. It just failed to run the demo, but the executable inceptionv3 has already been successfully built under folder bin_r.
This is the original status of ALL files under bin_demo. Let's copy and paste our built bin_r/inceptionv3 into this folder bin_demo. The size of the executable seems to be dramatically decreased.
Now, let's copy the built inception_v3.nbfrom host to Khadas VIM3. It seems inception_v3.nb built by Tensorflow 2.1.0 on host is of the same size as provided by Khadas.
khadas@Khadas:~/Programs/inceptionv3/bin_demo$ ./inceptionv3 ./inception_v3.nb ./goldfish_299x299.jpg D [setup_node:368]Setup node id[0] uid[4294967295] op[NBG] D [print_tensor:136]in(0) : id[ 1] vtl[0] const[0] shape[ 3, 299, 299, 1 ] fmt[u8 ] qnt[ASM zp=137, scale=0.007292] D [print_tensor:136]out(0): id[ 0] vtl[0] const[0] shape[ 1001, 1 ] fmt[f16] qnt[NONE] D [optimize_node:312]Backward optimize neural network D [optimize_node:319]Forward optimize neural network I [compute_node:261]Create vx node Create Neural Network: 64ms or 64785us I [vsi_nn_PrintGraph:1421]Graph: I [vsi_nn_PrintGraph:1422]***************** Tensors ****************** D [print_tensor:146]id[ 0] vtl[0] const[0] shape[ 1001, 1 ] fmt[f16] qnt[NONE] D [print_tensor:146]id[ 1] vtl[0] const[0] shape[ 3, 299, 299, 1 ] fmt[u8 ] qnt[ASM zp=137, scale=0.007292] I [vsi_nn_PrintGraph:1431]***************** Nodes ****************** I [vsi_nn_PrintNode:159]( NBG)node[0] [in: 1 ], [out: 0 ] [6df47cb0] I [vsi_nn_PrintGraph:1440]****************************************** I [vsi_nn_ConvertTensorToData:750]Create 268203 data. Verify... Verify Graph: 2ms or 2951us Start run graph [1] times... Run the 1 time: 28ms or 28835us vxProcessGraph execution time: Total 28ms or 28873us Average 28.87ms or 28873.00us I [vsi_nn_ConvertTensorToData:750]Create 2002 data. --- Top5 --- 2: 0.832520 795: 0.008316 974: 0.003586 408: 0.002302 393: 0.002016 I [vsi_nn_ConvertTensorToData:750]Create 2002 data.
By comparing to imagenet_slim_labels.txt under current folder, let's take a look at our inference results. Only the FIRST inference is qualified because of the probability.
dual boot by selecting devices: EMMC or TF/SD Card. On Boot Images from External Media, it's recommended as Via Keys mode (Side-Buttons) - the easiest and fastest way, which is the FIRST option on page How To Boot Into Upgrade Mode. Therefore, by following 4 steps as follows(cited from How To Boot Into Upgrade Mode), we should be able to boot into SD-USB.
Power on VIM3.
Long press the POWER key without releasing it.
Short press the ‘Reset’ key and release it.
Count for 2 to 3 seconds, then release the POWER key to enter into Upgrade Mode. You will see the sys-led turn ON when you’ve entered Upgrade Mode.
multiple boot via grub: Reasonably speaking, 2 operating systems may even have a chance to be installed onto a SINGLE EMMC
ONLY 1 operating system is preferred. Why??? Khadas VIM3 board comes with a large EMMC of size 32G.
After a VERY long time struggling, I would really like to emphasize the quality of Type C cable and power adaptor again. Try to buy things NOT from Taobao.
