Detectron2 - 1
This is also an updated version of my previous blog detectron2 published on October 13, 2019. I lost the previous source for this blog, but, let’s re-write it.
The autumn in both Vancouver and Seattle is gorgeous…
| Overview Seattle on Space Needle | Around Space Needle |
|---|---|
 |
 |
| Space Needle | So Heavy |
 |
 |
| Ferris Wheel | Pier 55 |
 |
 |
| Vancouver Maple | UBC Poisonous Mushroom |
 |
 |
Alright, let’s rapidly test Detectron2.
Installation is detailedly summarized in INSTALL.md.
We can simply follow GETTING_STARTED.md for some simple demonstrations. Make sure you’ve downloaded the demo pictures from Detectron1 demo and save under Detectron2’s folder demo.
1 | ➜ detectron2 git:(master) python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \ |
Let’s take a look at the result:
And of course, my pictures taken in Seattle and Vancouver:
| Overview Seattle on Space Needle | Around Space Needle |
|---|---|
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
| Overview Seattle on Space Needle | Around Space Needle |
 |
 |
Tensorflow 2.X
This is actally an updated version of my previous blog Tensorflow 2.0 published on October 12, 2019. What’s unfortunate is: I lost the source of that previous blog. What’s fortunate is: I have my Tensorflow updated from 2.0 to 2.1. Anyway, let’s begin:
- Previous blog: Flying back to China soon. Before leaving, here comes a simple blog testing Tensorflow 2.0. In this blog, I strictly follow Amita Kapoor and Ajit Jaokar’s free book Getting Started with TensorFlow 2.0.
1. Check GPU
1 | ➜ ~ python |
Then, for simplicity, let’s try it out directly:
2. Tensorflow 2.X
3. Tensorflow Dataset
Corona Virus
Hello everyone… As you all know that currently, my hometown Wuhan, China is suffering the MOST SERIOUS infectious disease - Corona Virus in her history. My parents are still in this locked city. People have been suffering the threaten from the death for over 20 days. I would have to write something, and WISH ALL MY BEST to my beloved parents, and all my country fellows who are still in Wuhan.
In this blog, all data are obtained from WHO Corona Virus Reports.
And, all plots are generated by altair.
So far, I can ONLY make that much…
| Confirmed Cases In China | Death Over The World |
|---|---|
 |
 |
God bless Dr. Li WenLiang, and shame on rubbish in Chinese government.
Updated on:
- February 10, 2020
- February 14, 2020
- February 15, 2020
- February 16, 2020
- February 17, 2020
- February 19, 2020
- February 21, 2020
So far, the data is still lack of info such as age, gender, etc. Let’s keep an eye on Corona Virus.
HiSilicon Hikey 970 - Shunya Desktop Image For HiKey970
I tried HiKey970 half a year ago, but NOT quite focused on it. There seems to be NO good operating system for this particular board. During this god-damn-it COVID-19, I now have some time, and seriously would like to fullfill some of my ideas on this particular 96board:
- test out Ubuntu 20.04
- build an operating system with the up-to-date kernel for HiKey970
- build the up-to-date Tensorflow from source with OpenCL
- test the capability of HiKey970‘s NPU
1. Hackerboards Comparisons
1.1 HiKey970 vs Hikey960
Hisilicon‘s official products’ website provides a detailed comparison of these two open source boards.
By far, HiKey970 is exactly the most powerful edge computing SBC I’ve ever met.
1.2 Comparison Across Various Popular SBCs
It looks HackerBoards has done a thorough comparison of almost all types of SBCs. Let’s just select the boards that we’re interested in and carry out our comparison. Clearly, HiKey970‘s performance is outstanding.
2. Build Our Own OS For HiKey970
2.1 Set up Cross_Compiler
1 | ➜ ~ export CC="aarch64-linux-gnu-" |
3. Flash HiKey970
1 | ➜ binaries sudo ./hikey_idt -c config |
4. Conclusion
If:
- HuaWei HiAI Engine ONLY supports Android instead of Linux
- HiSilicon
Build Toolchain Using Crosstool-NG
Today, we’re going to build our own toolchain using crosstool-NG. There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient. In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.
In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler. Some references can be found at:
1. Installation
How to install crosstool-NG are thoroughly summarized at its official website. In my case, I had it installed under folder /opt/compilers/crosstool-ng. Let’s take a look:
1 | pwd |
And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH.
2. Configuration
Under any directory that you want to save your .config file, we can configure our target cross compiler.
1 | ct-ng menuconfig |
According to elinux: RPi Linaro GCC Compilation, we need to do the following selections:
- Paths and misc options:
- Try features marked as EXPERIMENTAL: ticked
- Prefix directory: input the full path where you want to save the built toolchains
- Number of parallel jobs: 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B, but Raspberry Pi 3B does have 4 cores.
- Target options:
- Target Architecture: arm
- Default instruction set mode: arm
- Use EABI: ticked
- append hf to the tuple (EXPERIMENTAL): ticked
- Endianness: Little endian
- Bitness: 32-bit
- Emit assembly for CPU: cortex-a53
- Use specific FPU: vfp
- Floating point: hardware (FPU)
- Toolchain options:
- Tuple’s vendor string: rpi
- Operating System
- Target OS: linux
- Version of Linux: 4.20.8. Currently, I flashed Raspbian Buster with desktop and recommended software, 2019-07-10 onto my Raspberry Pi 3B, it comes with kernel of version 4.19.66, which is NOT in the list. That’s why I decided to select 4.20.8 and try my luck.
😝
1 | ➜ ~ uname -a |
- Binary utilities:
- Binary format: ELF
- Version of binutils: 2.31.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19➜ ~ apt show binutils
Package: binutils
Version: 2.31.1-16+rpi1
Priority: optional
Section: devel
Maintainer: Matthias Klose <doko@debian.org>
Installed-Size: 95.2 kB
Provides: binutils-gold, elf-binutils
Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
Suggests: binutils-doc (>= 2.31.1-16+rpi1)
Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
Homepage: https://www.gnu.org/software/binutils/
Download-Size: 56.9 kB
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU assembler, linker and binary utilities
The programs in this package are used to assemble, link and manipulate
binary and object files. They may be used in conjunction with a compiler
and various libraries to build programs. - C-library:
- C library: glibc
- Version of glibc: 2.28
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25➜ ~ apt show libc-bin
Package: libc-bin
Version: 2.28-10+rpi1
Priority: required
Essential: yes
Section: libs
Source: glibc
Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
Installed-Size: 3,015 kB
Depends: libc6 (>> 2.28), libc6 (<< 2.29)
Recommends: manpages
Homepage: https://www.gnu.org/software/libc/libc.html
Download-Size: 657 kB
APT-Manual-Installed: yes
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C Library: Binaries
This package contains utility programs related to the GNU C Library.
.
* catchsegv: catch segmentation faults in programs
* getconf: query system configuration variables
* getent: get entries from administrative databases
* iconv, iconvconfig: convert between character encodings
* ldd, ldconfig: print/configure shared library dependencies
* locale, localedef: show/generate locale definitions
* tzselect, zdump, zic: select/dump/compile time zones - C compiler:
- Show gcc versions from: GNU
- Version of gcc: 8.3.0
- gcc extra config: –with-float=hard
- Link libstdc++ statically into the gcc binary: tick
- C++: tick
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20➜ ~ apt show gcc
Package: gcc
Version: 4:8.3.0-1+rpi2
Priority: optional
Section: devel
Source: gcc-defaults (1.181+rpi2)
Maintainer: Debian GCC Maintainers <debian-gcc@lists.debian.org>
Installed-Size: 46.1 kB
Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
Recommends: libc6-dev | libc-dev
Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
Conflicts: gcc-doc (<< 1:2.95.3)
Download-Size: 5,200 B
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C compiler
This is the GNU C compiler, a fairly portable optimizing compiler for C.
.
This is a dependency package providing the default GNU C compiler.
After we save the configuration to file .config, we Exit the crosstool-NG configuration dialog.
3. Build
Please remember to:
unset LD_LIBRARY_PATHbefore building. Otherwise, you’ll meet some ERROR messages.mkdir ~/srcbefore building. Otherwise, whenever you tried to rerunct-ng build, you’ll have to download ALL required packages from scratch.
1 | ct-ng build |
This process may take a while. I’m going to sleep tonight. Continue tomorrow…
Alright, let’s continue today. Glad to know it’s built successfully.
😄
Let’s FIRST take a look at what’s under the current folder.
1 | ls |
And then, let’s take a look at what’s built under the specified destination folder.
1 | ls ~/....../CrossCompile/RPi |
Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B.
1 | arm-rpi-linux-gnueabihf-gcc --version |
Additional issue: It seems current crosstool-NG does NOT officially support Python. Please refer to my issue at crosstool-NG.
4. Compile/Build a Package with Generated Toolchain
For most of the packages nowadays, they are
- either supported by make:
./configure -> make -> make install - or supported by CMake:
mkdir build -> cd build -> ccmake ../ -> make -> make install
How to use our generated toolchain to compile/build our target packages?
- For the FIRST option, you can follow the crosstool-NG Using the toolchain.
- For the SECOND option, you are welcome to follow CMake Toolchains.
Today, we’re going to take the package flann as an example, which are to be built with CMake.
4.1 CMakeLists.txt Modification
4.1.1 CMake Toolchains - Cross Compiling for Linux
By following CMake Toolchains - Cross Compiling for Linux, we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann).
1 | cmake_minimum_required(VERSION 2.6) |
- CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
- CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
- tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder
${tools}/bin.
4.1.2 Ignore hdf5
In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt.
1 | #find_hdf5() |
4.2 Cross Compile
Now, let’s start cross-compiling flann.
1 | mkdir build |
and press c and then t, you’ll see the cross compiling toolchains have been automatically configured as:
1 | BUILD_CUDA_LIB *OFF |
The LAST step before make is to modify some of the parameters accordingly, in my case:
- BUILD_PYTHON_BINDINGS: ON -> OFF
- CMAKE_CXX_FLAGS:
-I/usr/include(for lz4.h from my host Ubuntu 19.04) - CMAKE_C_FLAGS:
-I/usr/include(for lz4.h from my host Ubuntu 19.04) - CMAKE_VERBOSE_MAKEFILE: OFF -> ON
- CMAKE_INSTALL_PREFIX:
/usr/local/-> under which directory you want to install, which can be IGNORED for now
Now, press c, you’ll see:
1 | CMake Warning at CMakeLists.txt:99 (message): |
A warining of missing hdf5 is clearly reasonable and acceptable. Then press g.
Finally, it’s the time for us to cross build flann.
1 | make -j8 |
VERY Important:
- If you build flann from source on a Raspberry Pi 3B, your system is going to hang HERE, possibly due to lack of memory..
- Raspberry Pi 3B has 4 cores ONLY, but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.
Finally, you’ll see flann has been successfully cross compiled as:
1 | ....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]': |
You can now:
make installto install the built/generated libraries installed under CMAKE_INSTALL_PREFIX- copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.
BTW: do NOT forget to install the header files.
5. Additional Issues
Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch. Some related issues are enumuated as the end of this blog.
- Multilib caveats from Official Notes on specific toolchain features
- fatal error: gnu/stubs-hard.h: No such file or directory from crosstool-NG
- fatal error: gnu/stubs-hard.h: No such file or directory from gcc-cross-compiler
- Build error: “features.h: No such file or directory” from risc-v tools
- crosstool-NG risc-v linux multilib issue
- raspberrypi toolchain issue
Alright, that’s all for today. Let me go to bed. Good bye…
😪
😴
Stereo Vision on VCSBC nano Z-RH-2 - PART I
Today, we are going to talk about a fabulous project: stereo vision on a zynq-7010 board.
1. VCSBC nano Z-RH-2
1.1 Hardware
We are using a VCSBC nano Z-RH-2 board for today’s experiment. The board adopted looks like the following:
| Front | Back | Connector |
|---|---|---|
 |
 |
 |
More detailed specifications, please refer to Vision Components’s official website.
1.2 Software
After you set up a static IP for this Vision Components SBC, it’s pretty straightforward to ssh into the system.
1 | ssh user@192.168.1.79 |
Currently, VC provides Linux Kernel 3.14.79.
1 | user@VC-Z:~$ uname -a |
And, let’s take a look at the dual ARMv7 CPUs on zynq-7010.
1 | user@VC-Z:~$ cat /proc/cpuinfo |
2. Stereo Vision
Sorry everybody. Today, I ONLY test stereo vision on ARM. I’ll try to figure out how to flash an open source IP of stereo vision onto zynq-7010, or write my own ASAP.
Hmmmmmmmm… It’s better I keep my code in dark???
😝
2.1 Classical Image Pairs
Let’s try out the stereo vision on some .pgm image pairs FIRST.
1 | user@VC-Z:~/longervision$ ./pgmpair ../images/aloe_left.pgm ../images/aloe_right.pgm |
My GOD… It’s UNBELIEVABLY SLOW.
| aloe_left | aloe_right |
|---|---|
 |
 |
| aloe_left Stereo | aloe_right Stereo |
 |
 |
2.2 Live Video Pairs
The BEST demo code to test Vision Componenets Stereo Vision is Eclipse_Example_Projects_VC_Z.
2.2.1 imageCaptureTest
1 | ./imageCaptureTest |
2.2.2 imageCaptFPS
1 | 03:41:47[root@VC-Z] /home/user/vc/Eclipse_Example_Projects_VC_Z/imageCaptFPS |
The above 2 examples are directly run on Vision Components’s board without display, for the given cable is of VGA connector, which is ALREADY outdated for many years. Therefore, in order to show the captured image pairs, in the next section, we’ll have to stream the captured data to a host computer, and display the real-time video pairs.
2.3 Stream/Display From Host Computer
Due to Vision Componenets’ Official documentation VCLinux_Getting_Started.pdf, images captured from camera can be displayed on a remote hosting PC.
2.3.1 Eclipse_Example_Projects_VC_Z (Not Preferred)
Eclipse_Example_Projects_VC_Z.zip provides some source code, which displays images captured from camera on the hosting PC’s Eclipse, by adding the camera as a Remote System to Eclipse.
Anyway, by using this method, you need to prepare ALL the following software and packages in advance.
- Eclipse
- CDT: install by Check for Updates with p2 software repository: http://download.eclipse.org/tools/cdt/releases/9.8
- Direct Remote C++ Debugging: installed from Eclipse Marketplace
2.3.2 vcimgnetclient.py
Besides the above method, a much more straightforward method is to adopt python script vcimgnetclient.py provided by Vision Components. However, vcimgnetclient.py is ONLY python2 compatible, and Vision Components has NO plan to provide a python3 compatible version of vcimgnetclient.py.
Therefore, the KEY to use the 2nd method is to make vcimgnetclient.py python3 compatible.
2.3.2.1 2to3
1 | 2to3 vcimgnetclient.py |
2.3.2.2 pygi-convert
Please refer to General Porting Tips
1 | ./pygi-convert.sh vcimgnetclient.py |
2.3.2.3 Try Running
1 | python vcimgnetclient.py |
2.3.2.4
For ALL vbox.pack_start, add one parameter 0 at the end for each case. For instance:vbox.pack_start(self.menu_bar, False, False)
tovbox.pack_start(self.menu_bar, False, False, 0)
Oh my god … There are STILL SO MANY things to do, in order to make vcimgnetclient.py Python3 compatible. Therefore, I implement my own vcimgnetclient_qt.py based on PyQt5.
2.3.3 Longer Vision’s vcimgnetclient_qt.py
2.3.3.1 Server
1 | 22:42:22[root@VC-Z] /home/user |
2.3.3.2 Client
Sorry, I’m NOT going to show my code, but the performance can be demonstrated as follows:
Install Armbian Debian onto Longer Vision Technology H3 Board
Today’s weather is perfect for this party: annual summer alumni party of Wuhan University.
| Alumni | Me |
|---|---|
 |
 |
Today, I’ve got another GOOD news: our ARM Linux board based on AllWinner H3 is demonstrated successful.
1. Hardware
1.1 Appearance
Yeah, this is how OUR OWN board looks like:
1.2 Specifications
From the appearance, it’s NOT hard to tell the specifications:
| Device | Version | Units |
|---|---|---|
| USB | 2.0 | 2 |
| Wired | 100Mbps | 1 |
| HDMI | 1.4 | 1 |
| Power Adaptor | 5V | 1 |
2. Flash Armbian Debian
2.1 Armbian for AllWinner H3
Armbian provides abundant info for AllWinner H3:
2.2 Armbian for Orange Pi PC
I’m so lucky today that I tried ONLY ONE TIME and have this Debian Buster with Armbian Linux for Orange Pi PC successfully installed and run on our board. You just need to download and extract Armbian_5.91_Orangepipc_Debian_buster_next_4.19.59.img out, and have it installed by dd.
2.3 Longer Vision Technology H3 Board Test
It’s NOT hard to find out your board’s IP address
- either from within your router
- or boot your board with a monitor and run the command
ip -c address.
Now, you should be able to SSH into your H3 baord directly as follows:
After you run
sudo apt update and sudo apt upgradesudo armbian-config
you should be able to upgrade your Debian Buster with Armbian Linux from 4.19.59-sunxi to 4.19.62-sunxi.
1 | longervision-GT72-6QE% ssh longervision@192.168.1.96 |
3. Issues
It’s a kind of late. Let me finish this up for now ASAP.
3.1 Hardware Issues
- ONLY 1 USB 2.0 port is working properly.
- It’s EASY to reach the critical temperature, as showing:
3.2 Features Expected
We really need a better low-cost chip to provide the support for:
- USB 3.0
- GigE
JeVois Smart Machine Vision Camera
😄
JeVois is an ARM Linux based camera module, delivered by JeVois Inc.. So far, this is the ONLY ARM Linux based camera module that I’m recommending, for its well-designed community as well as plenty of source code on Github.
1. Specifications
Clearly:
- JeVois Hardware Page: JeVois is based on the Made In China ARM Linux chip Allwinners A33
- JeVois Software Page: JeVois runs a full Linux operating system.
2. Getting Started with Jevois
The BEST place to start with JeVois is the following Youtube video:
I’m NOT going to repeat this step-by-step setup but move forward to the demonstration step directly.
3. Connect JeVois with Host
After having flashed JeVois’s TF card with the LATEST Linux image, we connect JeVois camera module into the laptop. This TF card will NOT mount onto the computer as a device automatically, but can be detected by lsusb as follows:
1 | lsusb |
4. Simple Demonstration
4.1 Guvcview
| Get Started | Image Control Default Settings |
|---|---|
 |
 |
| Video Control Settings 640x480 | Video Control Settings 640x360 |
 |
 |
4.2 JeVois Inventor
4.2.1 JeVois Tutorial
Again, a Youtube video is suggested to follow for the demonstration:
4.2.2 My Test
You can either download JeVois Inventor directly from the Official Website, or like me, build JeVois Inventor source code from scratch, based on my own QT, etc. Anyway, after JeVois Inventor has been successfully installed, it’s the time for us to test it out.
| JeVois Inventor Info | JeVois Inventor Parameters |
|---|---|
 |
 |
| JeVois Inventor Console | JeVois Inventor Camera |
 |
 |
| JeVois Inventor Config | JeVois Inventor Code C++ |
 |
 |
| JeVois Inventor Console | JeVois Inventor Camera |
 |
 |
| JeVois Inventor Console | JeVois Inventor Camera |
 |
 |
Being quite busy these days. Seriously, ARM board based on AllWinner H3 from my company Longer Vision is manufactured, and under test. Let’s have some fun.
😀
In the end, better to listen to some music while blogging.
ESP32-Cam
1. Preface
My ESP32-Cam has finally arrived today. The brand of AI Thinker has been demonstrated to be reliable.
In fact, I strictly followed randomnerdtutorials‘s blog ESP32-CAM Video Streaming and Face Recognition with Arduino IDE to demonstrate this low-cost video surveillance camera. So, in this blog, I ONLY enumerate some KEY POINTS that should be emphasized.
2. Connection
2.1 Schematic
ESP32-Cam can be connected to my laptop exactly the same as shown (cited directly from ESP32-CAM Video Streaming and Face Recognition with Arduino IDE).
2.2 Real Connection
The real connection looks as:
2.3 lsub
1 | lsusb |
3. Key Points on Flashing ESP32-Cam
3.1 Code Modification
Example code CameraWebServer should be trivially modified as:
1 | // Select camera model |
3.2 Verify/Compile and Upload
3.2.1 Verify/Compile
1 | Sketch uses 2241942 bytes (71%) of program storage space. Maximum is 3145728 bytes. |
3.2.2 Upload
1 | esptool.py v2.6 |
3.3 Disconnect GPIO 0 from GND
Before testing the code, be sure to disconnect GPIO 0 from GND.
3.4 RESET ESP32-Cam Board
It’s VERY important that you open the Serial Monitor dialog to monitor the serial port. You will see the following outputs after resetting ESP32-Cam board.
1 | ets Jun 8 2016 00:22:57 |
3.5 Video Streaming
By visiting the allocated IP address within any web browser, you’ll see the following picture:
And you’ll notice the streaming info from serial port:
1 | ets Jun 8 2016 00:22:57 |
4. Multiple Camera Streaming with Static IPs
I’m NOT going to talk about how to set up static IPs for IP cameras (well, yes, clearly, ESP32-Cam can be looked on as a low-cost video surveillance IP camera for sure). We ONLY demonstrate 4 ESP32-Cams are under streaming simultaneously with 4 statically allocated IP addresses.
We show the capturing effects FIRST:
Then, we show how the cables are connected by using a power bank and an extra bread board. Here, FTDI is just used as a power adaptor to provide 5V voltage.
