综合实践样例

根据Linux笔记中已经学习的命令，综合运用于实际实践中。

01 录制特定时间的bag包

如果文件夹存在不执行操作，不存在就创建文件夹; 计算该文件夹内的文件个数(不包括文件夹)。

  • ls -l,统计当前文件夹,不包括子文件夹; ls -lR, 统计当前文件夹,包括子文件夹;
  • "^-",统计文件个数; "^d",统计文件夹个数
  • wc -l, 统计输出信息的行数，因为已经过滤得只剩一般文件了，所以统计结果就是一般文件信息的行数，又由于一行信息对应一个文件，所以也就是文件
mkdir -p cloud_rtk
cd cloud_rtk

num=`ls -l |grep "^-"|wc -l`

rosbag record --duration=2 -o $num /lidar1/points

02 远程下载或上传文件

  • 首行是标记使用的脚本的类型，此处为expect脚本，也可使用bash脚本。
  • spawn: 执行一个命令
  • expect EOF means that all interacts are finished and return to original terminal/user. also eof
  • send: 用于向进程发送字符串. Quotation "" can’t be omitted and must be English punctuation
#! /usr/bin/expect

set timeout 30 # 设置超时时间, unit is second

spawn scp source_filepath destination_filepath # fork一个子进程执行scp

expect "password:" # 捕获到密码; note the colon is Chinese or English

send "password\n" # 输入密码并回车
expect EOF

03 远程连接并执行命令

  • sleep是指停止1秒后再继续执行后面的脚本
  • send后的字符串末尾的\n是转义字符，相当于输入命令后的回车键。
  • 末尾处的exit表示退出此次登录, it’s executed by the user we login。
  • interact means that stay at current user. For example, if we omit exit, we will stay at the user we login.
#!/usr/bin/expect

set timeout 100
set username [xxxx]
set password [123456]
set ip   [1.1.1.1]

spawn ssh -l $username $ip

expect {
    "(yes/no)?" {
        send "yes\n"
        expect "password:"
        send "$password\n"
    }
    "password:" {
        send "$password\n"
    }
}

sleep 1
send "cd /home/nnnn/bag\n"
sleep 1
send "rosbag record -a\n"
sleep 2
exit
interact

04 执行任务并定时关闭

首行是标记使用的脚本的类型，此处为bash脚本。
两个命令间用&连接，前面的命令执行后挂在后台并立即开始执行后面的命令。用&&连接，只有前面的命令执行完毕并退出后才会执行后面的命令。
第一行开启roscore后，计时两秒后结束roscore。kill -2相当于Ctrl+C。-2就是Ctrl+C发出的siginit。

#!/bin/bash
roscore &
sleep 3 && kill -2

05 解压文件名有规律的系列文件

这一系列的文件名为“17.7z”、“18.7z”…“21.7z”。解压密码为“blue”。

#!/bin/bash

set -u

for i in {17..21}
do
  filename=$i".7z"
  7z x $filename -pblue
done

解压所在文件夹内的所有以“.7z”结尾的文件，解压密码为“blue”。

ls | while read line
do
  file=$line
  if echo $file | grep -q -E '\.7z$'
  then
    echo $file
    7z x $file -pblue
  fi
done

06 内嵌expect脚本并检查IP地址连通性

  • ping [-dfnqrRv][-c<完成次数>][-i<间隔秒数>][-I<网络界面>][-l<前置载入>][-p<范本样式>][-s<数据包大小>][-t<存活数值>][主机名称或IP地址]
  • 参考链接
ip="11.11.11.11"
if ping -c 1 -w 1 ip >/dev/null;then
  echo "可以连接，继续检测"
  /usr/bin/expect<<-EOF   #shell中使用expect
    spawn git push
    expect {
      "yes/no" {
        send "yes\n"
        expect "password:"
        send "$password\n"
      }
      "password:" {
        send "$password\n"
      }
    }
    
  interact
  expect eof
  EOF

else
  echo "无法连接"$ip
fi

if ping -c 1 -w 1 ${ip} >/dev/null;then
  echo ${ip}
  /usr/bin/expect<<-EOF   #shell中使用expect
    set timeout -1
    spawn ssh $username@${ip}
    expect {
      "yes/no" {
        send "yes\n"
        expect "password:"
        send "$password\n"
      }
      "password:" {
        send "$password\n"
      }
    }
    
    sleep 1
    
    send "cd ~/Documents\n"
    
    send "exit\n"

	expect eof
	EOF

else
  echo "无法连接"$1": "${ipmap[$1]}
fi

07 修改某文件夹下的特定文件

遍历所在指定文件夹及其所有子文件夹中的markdown文档，并把第四行修改为<日期+时间+时区>

#!/bin/bash

files=$(find "." -type f)

# 遍历文件
for file in $files; do
    # 检查文件名是否符合特定首尾条件
    if [[ "$file" == *_posts* && "$file" == *.md ]]; then
        # 获取文件的最近修改时间
        last_modified=$(stat -c "%Y" "$file")

        # 将时间戳转换为日月年时分秒的格式
        last_modified_formatted=$(date -d @"$last_modified" +"%Y-%m-%d %H:%M:%S %z")

        # 替换文件的第四行为最近修改时间
        sed -i "4s/.*/date:   $last_modified_formatted/" "$file"

        echo "Updated date in $file to $last_modified_formatted"
    fi
done

09 批量删除某个名称的文件夹

find . -type d -name .git -prune -exec rm -r {} \;
  • .: 指定查找的范围，即在呢个文件夹内查找
  • -name: find命令的参数，后面接要查找的文件夹名称
  • .git: 我们想要删除的文件夹的名称，此处我想删除的文件夹为“.git”
  • -exec:
  • {}: can be read as “for each matching file/ folder”
  • \; is a terminator for the -exec clause

下面的这个命令也可以使用。

rm -rf `find . -type d -name a`

10 递归处理文件

#!/bin/bash

# 指定需要搜索的根目录，如果运行脚本时不指定目录，则默认为当前目录
root_dir="${1:-.}"

# find . -type f ! -name "*.webp" : 输出所有没有特定后缀的文件名

# 使用find命令在指定目录及其子目录下查找所有.webp文件
find "$root_dir" -type f -name "*.webp" -exec sh -c '
  for img_path; do
    # 使用dwebp工具将webp转换为png
    dwebp "$img_path" -o "${img_path%.webp}.png"
    
    # 如果转换成功，则删除原webp文件
    if [ $? -eq 0 ]; then
      rm "$img_path"
    fi
  done
' sh {} +
命令 解释
-exec 这个选项后跟一个命令模板，find 命令将为每个找到的文件执行这个命令。在这个脚本中，命令模板包括一个小型的 sh 脚本
sh -c 这个命令启动一个新的shell，并执行后面单引号中的命令。这允许对每个找到的文件执行多个命令
代码块’ … ‘ 一个小型的shell脚本，它为每个传递给它的文件路径执行一系列命令
for img_path; do ... done 这是一个循环，遍历所有传递给shell脚本的文件路径。在这个例子中，每个路径都是一个 .webp 文件
{} 这是 find 命令的占位符，代表当前找到的文件路径。每次 find 命令找到一个符合条件的文件时，都会将这个文件的路径替换到 {} 的位置
+ 这个符号告诉 find 命令将多个找到的文件路径聚集起来，一次调用 -exec 后的命令来处理这些路径，而不是对每个文件单独调用一次命令。这提高了效率，因为它减少了需要启动的shell的数量

11 统计文件名

ls | while read line
do
  file=$line
  if echo $file | grep -q -E '\.bag$'
  then
    if [ 18 = `echo ${#file}` ] 
      then
      # time=`rosbag info $file | grep start`
      # str=`date -d @${time: 0-14: 13} "+%Y-%m-%d %H:%M:%S"`
      time=${file: 0: 14}
      echo $time >> date_t.txt
    fi
  fi
done

12 simplify if statement

[ "`cat "params.yaml" | grep realtime`" ] && echo "0" || echo "1"

13 query ros topics

topic_name[0]="/image"
topic_name[1]="/lidar"
topic_name[2]="/rtk"

for topic in ${topic_name[@]};
do
  echo $topic
  timeout 3 rostopic hz $topic
  echo "----------------------"
done

14 bind several processes via service

We can create several peocesses and kill them in one time via service. Here we create a user service, which is different from system service.
First, # create a folder, which will contain service file and shell script. Assume username is “bs”.

mkdir -p /home/bs/.config/systemd/user/ 
cd /home/bs/.config/systemd/user/

Second, create a service file named myservice.service like below.

[Unit]
Description=Discribe your service here
# This service starts your custom script after the network is available. If this service id fully independent and does not rely on the state of any other services, you can delete this line
After=network.target

[Service]
Type=simple
# Use `ExecStart` to call a wrapper script to ensure that the script handles the termination of all its child processes upon receiving a signal to stop.
# shell script, it must be absolute path
ExecStart=/home/bs/.config/systemd/user/shell.sh
# ensures that all processes started by the service (including any child processes started by the script) will be terminated when the service itself is stopped.
KillMode=control-group
# on-failure: restart this service if this service dies; no: Don't restart the service automatically
Restart=on-failure

[Install]
WantedBy=default.target
# This service will be started at the default runlevel

Third, reload the user-level systemd configuration.

systemctl --user daemon-reload

Fourth, enable the service.

systemctl --user enable myservice.service

Fifth, don’t forget to create shell script. Now let’s create a script named shell.sh like below. Here I take ros as example.

#!/bin/bash

# Function to terminate all roslaunch processes
cleanup() {
    echo "Stopping all ROS nodes..."
    kill -SIGINT "$roslaunch1_pid" "$roslaunch2_pid" "$roslaunch3_pid"
}

# The trap command in the script sets up a handler for SIGTERM, which is the signal sent by systemd when the service is stopped. This handler function, cleanup, will explicitly kill all the background processes.
trap cleanup SIGTERM

# Start roslaunch commands in background
sleep 1 && roslaunch your_package launch_file1.launch & roslaunch1_pid=$!
sleep 2 && roslaunch your_package launch_file2.launch & roslaunch2_pid=$!
sleep 3 && roslaunch your_package launch_file3.launch & roslaunch3_pid=$!

# The wait command makes the main script wait for all the background processes, which means the main service process remains active as long as the background processes are running.
wait $roslaunch1_pid $roslaunch2_pid $roslaunch3_pid

We have completed service and shell file. We can use this service now.

systemctl --user start myservice.service # start service
systemctl --user status myservice.service # check service
systemctl --user restart myservice.service # restart service
systemctl --user stop myservice.service # stop service

Note that if you make some changes to service file, don’t forget to run this cmd systemctl --user daemon-reload to make it effect.

15 update date from local ntp server

create ntp server
Assume that ip of ntp server is 192.168.1.100 and host name is “bs”. Run cmds below:

sudo apt install ntp # install ntp
sudo systemctl start ntp # start ntp

set client
Run cmd below on client that need update date and time.

sudo nano /etc/systemd/timesyncd.conf # you can also use vim or gedit

Edit like below

[Time]
NTP=bs 192.168.1.100
FallbackNTP=bs 192.168.1.100

Then save and close the file. Restart the timesyncd service with cmd below:

sudo systemctl restart systemd-timesyncd

Check that the NTP synchronization is working correctly with:

timedatectl status

If everything is going well, you will see System lock synchronized is yes. You can also run date and check whether the result is right.

16 Creating a Script to Run at Boot with Cron

  1. Prepare Your Script: Ensure your script in /usr/local/func/ is executable. If it’s not, make it executable:
sudo chmod +x /usr/local/func/myscript
  1. Edit the Crontab:
crontab -e

Add the following line to schedule your script to run at every system boot:

@reboot /usr/local/func/myscript

This line tells cron to run /usr/local/func/myscript every time the system boots up.

  1. Save and Exit: After adding the line, save and exit the editor. cron will automatically install the new crontab.

  2. Verify: To ensure your crontab is set up correctly, you can list your current user’s crontab entries:

crontab -l

This command will show all scheduled cron jobs, including your new @reboot job.

  1. Additional Considerations Environment: Cron jobs run with a limited environment, meaning many of the environmental variables available in a full shell session (like those started by logging in) may not be available. If your script relies on certain environment variables, you might need to define them in the script or in the crontab entry. Or you can run source /home/${user}/.bahsrc to make script run like in a terminal.

Logging: Since cron jobs don’t have a terminal, you might want to redirect your script’s output to a file for debugging:

@reboot /usr/local/func/myscript >> /var/log/myscript.log 2>&1

This setup ensures that your script runs at system boot without involving system-wide services or requiring systemd, fitting your preference for a simpler, less formal method.

17 split and convert string

#!/bin/bash

# colorful output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
NC='\033[0m'

if [ "$#" -eq 0 ]; then
  echo -e "${RED}Should contain project name.${NC}"
  exit
fi

# (absolute path (relative path))
SHELL_DIR=$(realpath $(dirname "${BASH_SOURCE[0]}"))

name=$1

# whether file exists
# if [ -e "$name" ]; then
#   echo -e "${BLUE}File ${name} exists, skip generatation.${NC}"
#   exit
# fi

# if [ -d "$name" ]; then
#     echo -e "${YELLOW}Directory ${name} exists, skip creation.${NC}"
#     exit
# fi

# Internal Field Separator
IFS='_' 
# Convert string into an array
read -ra PARTS <<< "$name"
# Reset IFS if needed
unset IFS

# STRING="part1_part2_part3"
# # Get specific parts
# FIRST_PART=$(echo "$STRING" | cut -d '_' -f 1)
# SECOND_PART=$(echo "$STRING" | cut -d '_' -f 2)
# THIRD_PART=$(echo "$STRING" | cut -d '_' -f 3)

# traverse an array 
for PART in "${PARTS[@]}"; do
  # echo "$PART"
  # Convert first character to uppercase
  Name=${Name}$(echo "$PART" | sed 's/./\U&/') 
  # convert all charactera to uppercase
  NAME=${NAME}${PART^^} 
  NA_ME=${NA_ME}${PART^^}"_"
done

mkdir -p $name
cd $name

echo $Name
echo $NAME
echo $NA_ME

# "eval" allows you to construct and execute commands dynamically.
eval "${NAME}VERSION=1.0"
echo "${COVFERVERSION}"

date "+%Y-%m-%d %H:%M" # use date as separator

var_name="${NAME}VERSION"
# Create the variable dynamically
declare "$var_name=2.0"
# Access the newly created variable using indirect reference
echo "${!var_name}"  # Correct way to reference it