Longer Vision Technology

Github Blog

0%

OpenMV

Posted on Edited on In ,

It's almost July 10, let me continue testing something else: OpenMV. I used to have purchased 2 OpenMV cameras, respectively: * OpenMV Cam M4 V2 with STM32F427VG * OpenMV Cam M7 with STM32F765VI

No matter which version of OpenMV is connected to the laptop via USB, command lsusb is able to list the device out as:

1
Bus 001 Device 020: ID 1209:abd1 InterBiometrics

Dilemma of OpenMV-IDE

Conclusion FIRST: NEVER build OpenMV-IDE from scratch using a different version of Qt!!!

To build OpenMV-IDE is seriously headache.

1
2
3
4
5
6
7
➜  openmv ./openmv-ide-linux-x86_64-2.2.0.run
This application failed to start because it could not find or load the Qt platform plugin "xcb"
in "".

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, xcb.

Reinstalling the application may fix this problem.

Test OpenMV

In the following, we ONLY use picocom to carry out the test.

1. OpenMV Cam M4 V2 (OPENMV2 with STM32F427)

Current Contents In OpenMV Cam M4 V2
OpenMV Cam M4 V2 Properties
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
➜  ~ picocom /dev/ttyACM0
picocom v3.1

port is        : /dev/ttyACM0
flowcontrol    : none
baudrate is    : 9600
parity is      : none
databits are   : 8
stopbits are   : 1
escape is      : C-a
local echo is  : no
noinit is      : no
noreset is     : no
hangup is      : no
nolock is      : no
send_cmd is    : sz -vv
receive_cmd is : rz -vv -E
imap is        : 
omap is        : 
emap is        : crcrlf,delbs,
logfile is     : none
initstring     : none
exit_after is  : not set
exit is        : no

Type [C-a] [C-h] to see available commands
Terminal ready
MicroPython v1.9.3-4922-ge0f81c867 on 2018-04-23; OPENMV2 with STM32F427
Type "help()" for more information.
>>>

Press Ctrl+A then Ctrl+x, picocom will terminate its execution.

1
2
3
>>> 
Terminating...
Thanks for using picocom

2. OpenMV Cam M7 (OPENMV3 with STM32F765)

Current Contents In OpenMV Cam M7
OpenMV Cam M7 Properties
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
➜  ~ picocom /dev/ttyACM0
picocom v3.1

port is        : /dev/ttyACM0
flowcontrol    : none
baudrate is    : 9600
parity is      : none
databits are   : 8
stopbits are   : 1
escape is      : C-a
local echo is  : no
noinit is      : no
noreset is     : no
hangup is      : no
nolock is      : no
send_cmd is    : sz -vv
receive_cmd is : rz -vv -E
imap is        : 
omap is        : 
emap is        : crcrlf,delbs,
logfile is     : none
initstring     : none
exit_after is  : not set
exit is        : no

Type [C-a] [C-h] to see available commands
Terminal ready
MicroPython v1.9.4-4510-g23e8457de on 2018-06-29; OPENMV3 with STM32F765
Type "help()" for more information.
>>> 
Terminating...
Thanks for using picocom
➜  ~

Flash OpenMV Firmware Onto OpenMV Board

1. Entering DFU Mode

Similar to what's discussed in our previous blog, we need to enter DFU mode before flashing. * Disconnect the power for OpenMV board FIRST * Connect BOOT pin and RST pin afterwards * Then, reconnect the USB cable for flashing.

Before entering DFU mode: 

1
2
3
4
➜  ~ lsusb
...
Bus 001 Device 027: ID 1209:abd1 InterBiometrics
...

After entering DFU mode: 

1
2
3
4
➜  ~ lsusb
...
Bus 001 Device 028: ID 0483:df11 STMicroelectronics STM Device in DFU Mode
...

2. Flashing OpenMV's Firmware

Checkout the NEWEST OpenMV. Find the corrensponding version of OpenMV board, flash it by command python pydfu.py -u <firmware.dfu>.

1) OpenMV Cam M4 V2 (OPENMV2 with STM32F427)

1
2
3
4
5
6
7
8
9
10
11
➜  openmv git:(master) python ./tools/pydfu.py -u firmware/OPENMV2/firmware.dfu 
./tools/pydfu.py:80: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  if 'length' in inspect.getargspec(usb.util.get_string).args:
File: firmware/OPENMV2/firmware.dfu
    b'DfuSe' v1, image size: 834757, targets: 1
    b'Target' 0, alt setting: 0, name: "ST...", size: 834472, elements: 1
      0, address: 0x08010000, size: 834464
    usb: 0483:df11, device: 0x0000, dfu: 0x011a, b'UFD', 16, 0x9dfe8e73
Writing memory...
0x08010000  834464 [=========================] 100% 
Exiting DFU...

Let's try picocom directly:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
➜  openmv git:(master) picocom /dev/ttyACM0
picocom v3.1

port is        : /dev/ttyACM0
flowcontrol    : none
baudrate is    : 9600
parity is      : none
databits are   : 8
stopbits are   : 1
escape is      : C-a
local echo is  : no
noinit is      : no
noreset is     : no
hangup is      : no
nolock is      : no
send_cmd is    : sz -vv
receive_cmd is : rz -vv -E
imap is        : 
omap is        : 
emap is        : crcrlf,delbs,
logfile is     : none
initstring     : none
exit_after is  : not set
exit is        : no

Type [C-a] [C-h] to see available commands
Terminal ready
6792
18.86792
18.86792
18.86792
18.86792
18.86792
18.86792
......
Traceback (most recent call last):
  File "/main.py", line 51, in <module>
KeyboardInterrupt: 
MicroPython v1.9.4-4553-gb4eccdfe3 on 2019-05-02; OPENMV2 with STM32F427
Type "help()" for more information.
>>> 
Terminating...
Thanks for using picocom

Check file main.py, which starts running automatically while testing with picocom.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
# Face Detection Example
#
# This example shows off the built-in face detection feature of the OpenMV Cam.
#
# Face detection works by using the Haar Cascade feature detector on an image. A
# Haar Cascade is a series of simple area contrasts checks. For the built-in
# frontalface detector there are 25 stages of checks with each stage having
# hundreds of checks a piece. Haar Cascades run fast because later stages are
# only evaluated if previous stages pass. Additionally, your OpenMV Cam uses
# a data structure called the integral image to quickly execute each area
# contrast check in constant time (the reason for feature detection being
# grayscale only is because of the space requirment for the integral image).

import sensor, time, image

# Reset sensor
sensor.reset()

# Sensor settings
sensor.set_contrast(1)
sensor.set_gainceiling(16)
# HQVGA and GRAYSCALE are the best for face tracking.
sensor.set_framesize(sensor.HQVGA)
sensor.set_pixformat(sensor.GRAYSCALE)

# Load Haar Cascade
# By default this will use all stages, lower satges is faster but less accurate.
face_cascade = image.HaarCascade("frontalface", stages=25)
print(face_cascade)

# FPS clock
clock = time.clock()

while (True):
    clock.tick()

    # Capture snapshot
    img = sensor.snapshot()

    # Find objects.
    # Note: Lower scale factor scales-down the image more and detects smaller objects.
    # Higher threshold results in a higher detection rate, with more false positives.
    objects = img.find_features(face_cascade, threshold=0.75, scale_factor=1.25)

    # Draw objects
    for r in objects:
        img.draw_rectangle(r)

    # Print FPS.
    # Note: Actual FPS is higher, streaming the FB makes it slower.
    print(clock.fps())

2) OpenMV Cam M7 (OPENMV3 with STM32F765)

1
2
3
4
5
6
7
8
9
10
11
➜  openmv git:(master) python ./tools/pydfu.py -u firmware/OPENMV3/firmware.dfu
./tools/pydfu.py:80: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  if 'length' in inspect.getargspec(usb.util.get_string).args:
File: firmware/OPENMV3/firmware.dfu
    b'DfuSe' v1, image size: 1297173, targets: 1
    b'Target' 0, alt setting: 0, name: "ST...", size: 1296888, elements: 1
      0, address: 0x08020000, size: 1296880
    usb: 0483:df11, device: 0x0000, dfu: 0x011a, b'UFD', 16, 0xeff0bf62
Writing memory...
0x08020000 1296880 [=========================] 100% 
Exiting DFU...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
➜  openmv git:(master) picocom /dev/ttyACM0
picocom v3.1

port is        : /dev/ttyACM0
flowcontrol    : none
baudrate is    : 9600
parity is      : none
databits are   : 8
stopbits are   : 1
escape is      : C-a
local echo is  : no
noinit is      : no
noreset is     : no
hangup is      : no
nolock is      : no
send_cmd is    : sz -vv
receive_cmd is : rz -vv -E
imap is        : 
omap is        : 
emap is        : crcrlf,delbs,
logfile is     : none
initstring     : none
exit_after is  : not set
exit is        : no

Type [C-a] [C-h] to see available commands
Terminal ready
MicroPython v1.9.4-4553-gb4eccdfe3 on 2019-05-02; OPENMV3 with STM32F765
Type "help()" for more information.
>>> 
Terminating...
Thanks for using picocom
➜  openmv git:(master)

As you can see, firmware has NOW been updated to MicroPython v1.9.4-4553-gb4eccdfe3 on 2019-05-02.

Stream OpenMV Video without OpenMV-IDE

1. Snapshot

We FIRST test OpenMV snapshot code on OpenMV Cam M4 V2 (OPENMV2 with STM32F427).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Snapshot Example
#
# Note: You will need an SD card to run this example.
#
# You can use your OpenMV Cam to save image files.

import sensor, image, pyb

RED_LED_PIN = 1
BLUE_LED_PIN = 3

sensor.reset() # Initialize the camera sensor.
sensor.set_pixformat(sensor.RGB565) # or sensor.GRAYSCALE
sensor.set_framesize(sensor.QVGA) # or sensor.QQVGA (or others)
sensor.skip_frames(time = 2000) # Let new settings take affect.

pyb.LED(RED_LED_PIN).on()
sensor.skip_frames(time = 2000) # Give the user time to get ready.

pyb.LED(RED_LED_PIN).off()
pyb.LED(BLUE_LED_PIN).on()

print("You're on camera!")
sensor.snapshot().save("example.jpg") # or "example.bmp" (or others)

pyb.LED(BLUE_LED_PIN).off()
print("Done! Reset the camera to see the saved image.")
Snapshot on OpenMV Cam M4 V2

2. Video Recording

We THEN test OpenMV video recording on OpenMV Cam M7 (OPENMV3 with STM32F765).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# MJPEG Video Recording Example
#
# Note: You will need an SD card to run this demo.
#
# You can use your OpenMV Cam to record mjpeg files. You can either feed the
# recorder object JPEG frames or RGB565/Grayscale frames. Once you've finished
# recording a Mjpeg file you can use VLC to play it. If you are on Ubuntu then
# the built-in video player will work too.

import sensor, image, time, mjpeg, pyb

RED_LED_PIN = 1
BLUE_LED_PIN = 3

sensor.reset() # Initialize the camera sensor.
sensor.set_pixformat(sensor.RGB565) # or sensor.GRAYSCALE
sensor.set_framesize(sensor.QVGA) # or sensor.QQVGA (or others)
sensor.skip_frames(time = 2000) # Let new settings take affect.
clock = time.clock() # Tracks FPS.

pyb.LED(RED_LED_PIN).on()
sensor.skip_frames(time = 2000) # Give the user time to get ready.

pyb.LED(RED_LED_PIN).off()
pyb.LED(BLUE_LED_PIN).on()

m = mjpeg.Mjpeg("example.mjpeg")

print("You're on camera!")
for i in range(200):
    clock.tick()
    m.add_frame(sensor.snapshot())
    print(clock.fps())

m.close(clock.fps())
pyb.LED(BLUE_LED_PIN).off()
print("Done! Reset the camera to see the saved recording.")
Video recorded on OpenMV Cam M7

It looks it's NOT easy to demo with the SD card. Please refer to my symptoms on the OpenMV forum.

3. Wifi Module

As expected, OpenMV NOW also privides its Wifi module now. Let's just test the example code from openmv's repo.

OpenMV Wifi Shield Connection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Connect Example
#
# This example shows how to connect your OpenMV Cam with a WiFi shield to the net.

import network

SSID='' # Network SSID
KEY=''  # Network key

# Init wlan module and connect to network
print("Trying to connect... (may take a while)...")

wlan = network.WINC()
wlan.connect(SSID, key=KEY, security=wlan.WPA_PSK)

# We should have a valid IP now via DHCP
print(wlan.ifconfig())

However, it's NOT that handy to have the Wifi module tested.

My conclusion: Please use esp32-cam instead.

  1. 1. Dilemma of OpenMV-IDE
  2. 2. Test OpenMV
    1. 2.1. 1. OpenMV Cam M4 V2 (OPENMV2 with STM32F427)
    2. 2.2. 2. OpenMV Cam M7 (OPENMV3 with STM32F765)
  3. 3. Flash OpenMV Firmware Onto OpenMV Board
    1. 3.1. 1. Entering DFU Mode
    2. 3.2. 2. Flashing OpenMV's Firmware
      1. 3.2.1. 1) OpenMV Cam M4 V2 (OPENMV2 with STM32F427)
      2. 3.2.2. 2) OpenMV Cam M7 (OPENMV3 with STM32F765)
  4. 4. Stream OpenMV Video without OpenMV-IDE
    1. 4.1. 1. Snapshot
    2. 4.2. 2. Video Recording
    3. 4.3. 3. Wifi Module