0%

For my FIRST blog Detectron2 - 1 has already demonstrated its stunning capability, I NEVER thought I'll have a 2nd chance to write a 2nd blog about detectron2.

In this crazy year 2021 (yeah, today is THE HOTTEST EVER DAY recorded in Canada ), detectron2 has some updates as in its What's New. Today, I'd love to test out Densepose in depth. By the way, today, I finally received OpenCV AI KIT. We'll have some fun on that next week.

1. My Environments

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
➜  ~ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
➜ ~ python --version
Python 3.8.5
➜ ~ pip show torch
Name: torch
Version: 1.10.0a0+git3137bbe
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: ~/.local/lib/python3.8/site-packages
Requires: typing-extensions
Required-by: torchvision, torchaudio, torchtext
➜ ~ pip show torchvision
Name: torchvision
Version: 0.11.0a0+0013d93
Summary: image and video datasets and models for torch deep learning
Home-page: https://github.com/pytorch/vision
Author: PyTorch Core Team
Author-email: soumith@pytorch.org
License: BSD
Location: ~/.local/lib/python3.8/site-packages/torchvision-0.11.0a0+0013d93-py3.8-linux-x86_64.egg
Requires: numpy, torch, pillow
Required-by:
➜ ~ pip show detectron2
Name: detectron2
Version: 0.4.1
Summary: Detectron2 is FAIR's next-generation research platform for object detection and segmentation.
Home-page: https://github.com/facebookresearch/detectron2
Author: FAIR
Author-email: None
License: UNKNOWN
Location: ~/.local/lib/python3.8/site-packages/detectron2-0.4.1-py3.8-linux-x86_64.egg
Requires: Pillow, matplotlib, pycocotools, termcolor, yacs, tabulate, cloudpickle, tqdm, tensorboard, fvcore, iopath, future, pydot, omegaconf, hydra-core, black
Required-by:
➜ ~ apt show opencv
Package: opencv
Version: 4.5.2-6
Status: install ok installed
Priority: extra
Section: checkinstall
Maintainer: root@lvision-MS-7C59
Installed-Size: 2,188 MB
Provides: build
Download-Size: unknown
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status
Description: Package created with checkinstall 1.6.3
➜ ~ gcc --version
gcc (GCC) 10.3.1 20210627
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

2. DensePose in Detectron2

2.1 Demo on Single Image

We first extract a single image from a youtube video Dance Workout- EXO: Love Shot, follow by Kkardio. The reasons why we select this video are: there are two parts of the video: - the left part contains multiple front-facing dancers - the right part contains a single back-facing fllower

2.1.1 Command Line for IUV

1
2
3
4
5
6
7
8
9
10
11
12
➜  DensePose git:(master) ✗ python apply_net.py show configs/densepose_rcnn_R_50_FPN_s1x.yaml ./pkl/IUV/R_50_FPN_s1x.pkl image.png dp_contour,bbox --output image_densepose_contour.png 
The checkpoint state_dict contains keys that are not used by the model:
proposal_generator.anchor_generator.cell_anchors.{0, 1, 2, 3, 4}
~/.local/lib/python3.8/site-packages/torch/_tensor.py:585: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ../aten/src/ATen/native/BinaryOps.cpp:461.)
return torch.floor_divide(self, other)
~/.local/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at ../c10/core/TensorImpl.h:1292.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
....../detectron2/projects/DensePose/densepose/vis/densepose_results.py:133: UserWarning: The following kwargs were not used by contour: 'cfg', 'texture_atlas', 'texture_atlases_dict'
plt.contour(u, self.levels, extent=extent, **self.plot_args)
....../detectron2/projects/DensePose/densepose/vis/densepose_results.py:134: UserWarning: The following kwargs were not used by contour: 'cfg', 'texture_atlas', 'texture_atlases_dict'
plt.contour(v, self.levels, extent=extent, **self.plot_args)

2.1.2 Command Line for CSE

It looks smpl_uvSnapshot_colors.jpg is missing?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
➜  DensePose git:(master) ✗ python apply_net.py show configs/cse/densepose_rcnn_R_50_FPN_s1x.yaml ./pkl/CSE/R_50_FPN_s1x.pkl image.png dp_vertex,bbox -v
[06/28 16:19:54 apply_net]: Loading config from configs/cse/densepose_rcnn_R_50_FPN_s1x.yaml
[06/28 16:19:54 apply_net]: Loading model from ./pkl/CSE/R_50_FPN_s1x.pkl
The checkpoint state_dict contains keys that are not used by the model:
pixel_mean
pixel_std
proposal_generator.anchor_generator.cell_anchors.{0, 1, 2, 3, 4}
[06/28 16:19:56 apply_net]: Loading data from image.png
~/.local/lib/python3.8/site-packages/torch/_tensor.py:585: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at ../aten/src/ATen/native/BinaryOps.cpp:461.)
return torch.floor_divide(self, other)
~/.local/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at ../c10/core/TensorImpl.h:1292.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
[06/28 16:19:57 apply_net]: Processing image.png
~/.local/lib/python3.8/site-packages/torch/nn/functional.py:3609: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
warnings.warn(
[06/28 16:19:58 apply_net]: Output saved to outputres.0001.png

2.1.3 Results

Dance Workout- EXO: Love Shot, follow by Kkardio
Dance Workout- EXO: Love Shot, follow by Kkardio
Dense Pose Contour IUV
Dense Pose Contour IUV
Dense Pose Continuous Surface Embedding
Dense Pose CSE

From the CSE result, it's quite obvious that detectron2 Densepose CSE is able to tell the left leg from the right leg.

2.2 Demo on Video

The original video is embedded in the following:

2.2.1 Code

The code I used for my demo heavily relies on detectron2 issue 481 and detectron2 Densepose apply_net.py. Please modify the code by yourself according to your own applications.

2.2.2 Results

2.2.2.1 IUV

IUV Model Zoo

2.2.2.2 CSE

CSE Model Zoo

Again, from the CSE result, it's quite obvious that detectron2 Densepose CSE is able to tell the left leg from the right leg.

Back to Canada for around 1 month. It's the time for me to write something. Good to know that PTI is released open source a couple of days ago. Let's take a look at its performance today. Sorry that I seriously have NO time to investigate the details, having been busy in a robotic arm project. Therefore, a couple of resultant images are displayed as follows to demonstrate PTI's performance.

Style Change on Myself

When I was young Aligned Image
When I was young Aligned Image
Inversion before PTI PTIed Image
Inversion before PTI PTIed Image
Rotation Left before PTI Rotation Left PTIed Image
Rotation Left before PTI Rotation Left PTIed Image
Rotation Right before PTI Rotation Right PTIed Image
Rotation Left before PTI Rotation Left PTIed Image
No Smile before PTI No Smile PTIed Image
No Smile before PTI No Smile PTIed Image
Smile before PTI Smile PTIed Image
Smile before PTI Smile PTIed Image
Young before PTI Young PTIed Image
Young before PTI Young PTIed Image
Old before PTI Old PTIed Image
Old before PTI Old PTIed Image

Style Change to Another Person

afro
Afro
mohawk
Afro

Music day. This is THE traditional Chinese culture. Revolution culture HAS NEVER BEEN.

ENet was published in 2016... Hmmmm... It's a shame that I just got to know it... Anyway, the performance seems to be REALLY GOOD...

{% emoji +1 %}

In fact, this video is somewhat stolen. I would like to advertise for SOME publication. I'll let you know a couple of months later.

It's sad to hear Donald J. Trump is detected as positive and now a patient, please refer to Trump's positive Covid-19 test throws country into fresh upheaval. I really would suggest everybody, particularly me, to be ALWAYS positive and patient. Kelowna, I'm coming to you again.

Long time NO see...
{% emoji wink %}

Have NO time for blogging these days.

https://youtu.be/UibIz3hLLLo

In this blog, we're going to explore Kubernetes. However, there is a VERY FIRST question to be answered: what is the relationship between Kubernetes and Docker? Let's start our journey of today:

1 Kubernetes

Kubernetes (commonly stylized as k8s) is an open-source container-orchestration system for automating application deployment, scaling, and management. (cited from Wikipedia)

On Raspberry Pi, a lightweight variant of Kubernetes is normally preferred. A variety of choices are available:

Minikube vs. kind vs. k3s - What should I use? elaborates the differences among Minikube, kind and k3s. Its final table is cited as follows:

minikube kind k3s
runtime VM container native
supported architectures AMD64 AMD64 AMD64, ARMv7, ARM64
supported container runtimes Docker, CRI-O, containerd, gVisor Docker Docker, containerd
startup time: initial/following 5:19 / 3:15 2:48 / 1:06 0:15 / 0:15
memory requirements 2GB 8GB (Windows, MacOS) 512 MB
requires root? no no yes (rootless is experimental)
multi-cluster support yes yes no (can be achieved using containers)
multi-node support no yes yes
project page minikube kind k3s

Here in my case, I'm going to use k3s to manage and monitor the cluster. The following 2 blogs are strongly recommended from me. - Run Kubernetes on a Raspberry Pi with k3s - Kubernetes 1.18 broke “kubectl run”, here’s what to do about it

2. Preparation

Let's take a look at the IP info of ALL 4 Raspberry Pis. Let's take pi04 as the example this time. pi01, pi02, pi03 are having very similar IP info as pi04.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether b8:27:eb:c1:b8:76 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.247/24 brd 192.168.1.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 2001:569:7fb8:c600:ca9b:d758:2534:8a9d/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 7480sec preferred_lft 7180sec
inet6 fe80::80b5:3a3e:defe:24aa/64 scope link
valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether b8:27:eb:94:ed:23 brd ff:ff:ff:ff:ff:ff
4: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether e8:4e:06:35:16:6d brd ff:ff:ff:ff:ff:ff
inet 192.168.1.245/24 brd 192.168.1.255 scope global dynamic noprefixroute wlan1
valid_lft 86060sec preferred_lft 75260sec
inet6 2001:569:7fb8:c600:5996:3c23:ef08:c367/64 scope global dynamic mngtmpaddr noprefixroute
valid_lft 7480sec preferred_lft 7180sec
inet6 fe80::a0ff:56e3:77f5:9bf2/64 scope link
valid_lft forever preferred_lft forever

As mentioned in A Cluster of Raspberry Pis (1) - Configuration, pi04 is an old Raspberry Pi 3 Model B Rev 1.2 1GB, which is unfortunately with a broken Wifi interface wlan0. Therefore, I've got to insert a Wifi dongle in order to have Wifi wlan1 enabled.

3. k3s Installation and Configuration

3.1 k3s Installation on Master Node pi01

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
pi@pi01:~ $ curl -sfL https://get.k3s.io | sh -
[INFO] Finding release for channel stable
[INFO] Using v1.18.3+k3s1 as release
[INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v1.18.3+k3s1/sha256sum-arm.txt
[INFO] Downloading binary https://github.com/rancher/k3s/releases/download/v1.18.3+k3s1/k3s-armhf
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
pi@pi01:~ $ sudo kubectl get nodes
NAME STATUS ROLES AGE VERSION
pi01 Ready master 2m42s v1.18.3+k3s1

If we take a look at IP info, one additional flannel.1 interface is added as follows:

1
2
3
4
5
6
7
8
4: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default 
link/ether 3e:53:91:ce:b9:46 brd ff:ff:ff:ff:ff:ff
inet 10.42.0.0/32 brd 10.42.0.0 scope global flannel.1
valid_lft forever preferred_lft forever
inet 169.254.238.88/16 brd 169.254.255.255 scope global noprefixroute flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::59c5:fbf9:aaf7:46fb/64 scope link
valid_lft forever preferred_lft forever

3.2 k3s Installation on Work Node pi01, pi02, pi03

Before moving forward, we need to write down node token on the master node, which will be used while the other work nodes join in the cluster.

1
2
pi@pi01:~ $ sudo cat /var/lib/rancher/k3s/server/node-token
K10cedbc6396ab68f6bee0d2df3eb005f0ff9ea17275aed2763b6bf07a06e83ce47::server:9a0d84f3bd8044b341c95f967badd5d3
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
pi@pi0X:~ $ curl -sfL http://get.k3s.io | K3S_URL=https://192.168.1.253:6443 \
> K3S_TOKEN=K10cedbc6396ab68f6bee0d2df3eb005f0ff9ea17275aed2763b6bf07a06e83ce47::server:9a0d84f3bd8044b341c95f967badd5d3 sh -
[INFO] Finding release for channel stable
[INFO] Using v1.18.3+k3s1 as release
[INFO] Downloading hash https://github.com/rancher/k3s/releases/download/v1.18.3+k3s1/sha256sum-arm.txt
[INFO] Skipping binary downloaded, installed k3s matches hash
[INFO] Skipping /usr/local/bin/kubectl symlink to k3s, command exists in PATH at /usr/bin/kubectl
[INFO] Skipping /usr/local/bin/crictl symlink to k3s, command exists in PATH at /usr/bin/crictl
[INFO] Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent

, where X=2, or 3, or 4.

3.3 Take a Look on pi01

1
2
3
4
5
6
pi@pi01:~ $ sudo kubectl get nodes
NAME STATUS ROLES AGE VERSION
pi02 Ready <none> 5m7s v1.18.3+k3s1
pi03 Ready <none> 4m18s v1.18.3+k3s1
pi01 Ready master 20m v1.18.3+k3s1
pi04 Ready <none> 11s v1.18.3+k3s1

3.4 Access Raspberry Pi Cluster from PC

We can further configure our PC to be able to access the Raspberry Pi Cluster. For details, please refer to Run Kubernetes on a Raspberry Pi with k3s. On my laptop, I can do:

1
2
3
4
5
6
 ✔  kubectl get nodes
NAME STATUS ROLES AGE VERSION
pi02 Ready <none> 3h27m v1.18.3+k3s1
pi03 Ready <none> 3h26m v1.18.3+k3s1
pi04 Ready <none> 3h22m v1.18.3+k3s1
pi01 Ready master 3h42m v1.18.3+k3s1

We can even specify the role name by the following command:

1
2
 ✔  kubectl label nodes pi0X kubernetes.io/role=worker
node/pi0X labeled

, where X=2, or 3, or 4.

Let's take a look at all nodes again:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
 64  ✔  kubectl get nodes
NAME STATUS ROLES AGE VERSION
pi04 Ready worker 2d7h v1.18.3+k3s1
pi03 Ready worker 2d7h v1.18.3+k3s1
pi01 Ready master 2d7h v1.18.3+k3s1
pi02 Ready worker 2d7h v1.18.3+k3s1
66  ✘  kubectl get pods --all-namespaces
 ~
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system helm-install-traefik-m49zm 0/1 Completed 0 2d7h
kube-system svclb-traefik-djtgw 2/2 Running 2 2d7h
kube-system svclb-traefik-cn76f 2/2 Running 2 2d7h
kube-system local-path-provisioner-6d59f47c7-5gz67 1/1 Running 1 2d7h
kube-system svclb-traefik-7bwtl 2/2 Running 2 2d7h
kube-system coredns-8655855d6-wg68z 1/1 Running 1 2d7h
kube-system traefik-758cd5fc85-vn7xg 1/1 Running 1 2d7h
kube-system metrics-server-7566d596c8-v2vgp 1/1 Running 1 2d7h
kube-system svclb-traefik-r9x5v 2/2 Running 4 2d7h
kubernetes-dashboard dashboard-metrics-scraper-6b4884c9d5-ct6lh 1/1 Running 0 43h
kubernetes-dashboard kubernetes-dashboard-7b544877d5-s4spx 0/1 CrashLoopBackOff 489 43h
67  ✔  kubectl get pods
 ~
No resources found in default namespace.

4. Create Deployment

1
2
3
 73  ✔  kubectl create deployment nginx-sample --image=nginx
 ~
deployment.apps/nginx-sample created

After a while, nginx-sample will be successfully deployed.

1
2
3
4
5
6
7
8
9
10
11
12
 79  ✔  kubectl get deployments
 ~
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-sample 1/1 1 1 12m
80  ✔  kubectl get pods
 ~
NAME READY STATUS RESTARTS AGE
nginx-sample-879f5cf56-kg6xf 1/1 Running 0 12m
81  ✔  kubectl get pods -o wide
 ~
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-sample-879f5cf56-kg6xf 1/1 Running 0 12m 10.42.0.15 pi01 <none> <none>

Now, let's expose this service and take a look from the browser:

1
2
3
4
5
6
7
8
 82  ✔  kubectl expose deployment nginx-sample --type="NodePort" --port 80 
 ~
service/nginx-sample exposed
83  ✔  kubectl get services
 ~
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 96m
nginx-sample NodePort 10.43.195.237 <none> 80:30734/TCP 72s