Bash Skills

Back to Vancouver, Canada now. Start writing something again. Just wanna write down some very useful bash scripts.

Skill 1 - How to upgrade all Python packages with pip?

Solution

Please refer to https://stackoverflow.com/questions/2720014/how-to-upgrade-all-python-packages-with-pip. Bash script is:

1
pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U --user

Test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
➜  ~ pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U --user
Collecting distro-info
Installing collected packages: distro-info
Found existing installation: distro-info 0.0.0
Uninstalling distro-info-0.0.0:
Successfully uninstalled distro-info-0.0.0
Successfully installed distro-info-0.0.0
Collecting Markdown
Downloading https://files.pythonhosted.org/packages/c0/4e/fd492e91abdc2d2fcb70ef453064d980688762079397f779758e055f6575/Markdown-3.1.1-py2.py3-none-any.whl (87kB)
|████████████████████████████████| 92kB 4.5MB/s
Requirement already satisfied, skipping upgrade: setuptools>=36 in ./.local/lib/python3.7/site-packages (from Markdown) (41.0.1)
Installing collected packages: Markdown
WARNING: The script markdown_py is installed in '/home/lvision/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Markdown-3.1.1
Collecting pycups
Using cached https://files.pythonhosted.org/packages/00/ce/836a0deb8b24bcd5f850f8fb97f99fb4abd7374e078b9e6df5a0838f8eb5/pycups-1.9.74.tar.bz2
Building wheels for collected packages: pycups
Building wheel for pycups (setup.py) ... error
ERROR: Complete output from command /usr/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-9jc5iw0k/pycups/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-k2dtuswe --python-tag cp37:
ERROR: running bdist_wheel
running build
running build_ext
building 'cups' extension
creating build
creating build/temp.linux-x86_64-3.7
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION="1.9.74" -I/usr/include/python3.7m -c cupsmodule.c -o build/temp.linux-x86_64-3.7/cupsmodule.o
cupsmodule.c:23:10: fatal error: cups/cups.h: No such file or directory
#include <cups/cups.h>
^~~~~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for pycups
Running setup.py clean for pycups
Failed to build pycups
Installing collected packages: pycups
Running setup.py install for pycups ... error
ERROR: Complete output from command /usr/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-9jc5iw0k/pycups/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-imbt0ikc/install-record.txt --single-version-externally-managed --compile --user --prefix=:
ERROR: running install
running build
running build_ext
building 'cups' extension
creating build
creating build/temp.linux-x86_64-3.7
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION="1.9.74" -I/usr/include/python3.7m -c cupsmodule.c -o build/temp.linux-x86_64-3.7/cupsmodule.o
cupsmodule.c:23:10: fatal error: cups/cups.h: No such file or directory
#include <cups/cups.h>
^~~~~~~~~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/usr/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-9jc5iw0k/pycups/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-imbt0ikc/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-9jc5iw0k/pycups/
Collecting Pygments
Downloading https://files.pythonhosted.org/packages/5c/73/1dfa428150e3ccb0fa3e68db406e5be48698f2a979ccbcec795f28f44048/Pygments-2.4.2-py2.py3-none-any.whl (883kB)
|████████████████████████████████| 890kB 3.8MB/s
Installing collected packages: Pygments
WARNING: The script pygmentize is installed in '/home/lvision/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Pygments-2.4.2
Collecting PyGObject
Using cached https://files.pythonhosted.org/packages/c1/36/c31a6ce9411b22a8ac0759c9b5ffda09368d5e0643450bb8ed83c271588c/PyGObject-3.32.1.tar.gz
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Requirement already satisfied, skipping upgrade: pycairo>=1.11.1 in ./.local/lib/python3.7/site-packages (from PyGObject) (1.18.1)
Building wheels for collected packages: PyGObject
Building wheel for PyGObject (PEP 517) ... done
Stored in directory: /home/lvision/.cache/pip/wheels/30/7a/00/cdefec8644e21ce0d5a438f96ce0ba65b51eb648c4e4614d62
Successfully built PyGObject
Installing collected packages: PyGObject
Successfully installed PyGObject-3.32.1
➜ ~ pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U --user
Collecting distro-info
Installing collected packages: distro-info
Found existing installation: distro-info 0.0.0
Uninstalling distro-info-0.0.0:
Successfully uninstalled distro-info-0.0.0
Successfully installed distro-info-0.0.0
Collecting pycups
Using cached https://files.pythonhosted.org/packages/00/ce/836a0deb8b24bcd5f850f8fb97f99fb4abd7374e078b9e6df5a0838f8eb5/pycups-1.9.74.tar.bz2
Building wheels for collected packages: pycups
Building wheel for pycups (setup.py) ... done
Stored in directory: /home/lvision/.cache/pip/wheels/bb/4a/db/d9ea6edeead55ff1b909f3e7aa4384db28ff6ce3ecef0c94f7
Successfully built pycups
Installing collected packages: pycups
Successfully installed pycups-1.9.74
➜ ~ pip list --outdated
Package Version Latest Type
----------- ------- ------ -----
distro-info 0.0.0 0.10 sdist
➜ ~

In the above test, clearly, we ran the following bash script twice:

1
➜  ~ pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U --user

For the 1st time, we successfully had distro-info, Markdown, Pygments, PyGObject installed, but failed to install pycups.
For the 2nd time, after libcups2-dev had been installed from Ubuntu repository, we had pycups installed successfully.

However, we still have one LAST issue: why distro-info is still oudated? _ Can anybody give me an explanation?

Skill 2 - How to find all connected devices with nmap?

Solution

Please refer to https://vitux.com/find-devices-connected-to-your-network-with-nmap/. Bash script is:

1
nmap -sP 192.168.1.254/24

Test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
➜  ~ nmap -sP 192.168.1.254/24
Starting Nmap 7.70 ( https://nmap.org ) at 2019-06-03 08:03 PDT
Nmap scan report for 192.168.1.64
Host is up (0.082s latency).
Nmap scan report for 192.168.1.65
Host is up (0.0092s latency).
Nmap scan report for 192.168.1.66
Host is up (0.010s latency).
Nmap scan report for 192.168.1.67
Host is up (0.037s latency).
Nmap scan report for 192.168.1.71
Host is up (0.032s latency).
Nmap scan report for 192.168.1.73
Host is up (0.032s latency).
Nmap scan report for 192.168.1.74
Host is up (0.061s latency).
Nmap scan report for 192.168.1.98
Host is up (0.044s latency).
Nmap scan report for 192.168.1.103
Host is up (0.15s latency).
Nmap scan report for lvision-GT72-6QE (192.168.1.200)
Host is up (0.00047s latency).
Nmap scan report for _gateway (192.168.1.254)
Host is up (0.015s latency).
Nmap done: 256 IP addresses (11 hosts up) scanned in 8.21 seconds
➜ ~

Skill 3 - How to install OS img with dd?

Solution

Please refer to Installing operating system images on Linux for Raspberry Pi . Bash script is:

1
sudo dd bs=4M if=Armbian_5.83_Nanopineo_Ubuntu_bionic_next_4.19.38.img of=/dev/mmcblk0 conv=fsync

Test

1
2
3
4
5
➜  ~ sudo dd bs=4M if=Armbian_5.83_Nanopineo_Ubuntu_bionic_next_4.19.38.img of=/dev/mmcblk0 conv=fsync
[sudo] password for lvision:
223+0 records in
223+0 records out
935329792 bytes (935 MB, 892 MiB) copied, 89.9065 s, 10.4 MB/s