2020年 · Linux · 点滴记录

linux批量监测IP端口存活

一、nc命令检测端口的用法
安装nc工具
yum install -y nc

# nc -v -w 10 -z 192.168.31.101 22 (nc命令用法)
-v 显示指令执行过程。
-w <超时秒数> 设置等待连线的时间。
-u 表示使用UDP协议
-z 使用0输入/输出模式,只在扫描通信端口时使用。

一、批量监测IP端口存活脚本 (命名 testip.sh)
cat testip.sh

#!/bin/bash

cat ip.txt | while read line
do
nc -w 5 -z $line > /dev/null 2>&1
if [ $? -eq 0 ]
then
echo -e “\033[32m $line:通 \033[0m”
else
echo -e “\033[31m $line:不通 \033[0m”
fi
done
二、文件内容cat ip.txt
192.168.31.101 22
192.168.31.102 22
192.168.31.103 22
三、将 testip.sh 和 ip.txt 放在同一个目录下
chmod +x testip.sh
运行脚本
sh testip.sh

2020年 · Linux · 点滴记录

confd template src格式和 templates 语法

Template Resources

Template resources are written in TOML and define a single template resource. Template resources are stored under the/etc/confd/conf.d directory by default.

Required

  • dest (string) – The target file.
  • keys (array of strings) – An array of keys.
  • src (string) – The relative path of a configuration template.

Optional

  • gid (int) – The gid that should own the file. Defaults to the effective gid.
  • mode (string) – The permission mode of the file.
  • uid (int) – The uid that should own the file. Defaults to the effective uid.
  • reload_cmd (string) – The command to reload config.
  • check_cmd (string) – The command to check config. Use {{.src}} to reference the rendered source template.
  • prefix (string) – The string to prefix to keys.

Notes

When using the reload_cmd feature it’s important that the command exits on its own. The reload command is not managed by confd, and will block the configuration run until it exits.

Example

[template]
src = "nginx.conf.tmpl"
dest = "/etc/nginx/nginx.conf"
uid = 0
gid = 0
mode = "0644"
keys = [
  "/nginx",
]
check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/service nginx restart"

Templates

Templates define a single application configuration template. Templates are stored under the /etc/confd/templates directory by default.

Templates are written in Go’s text/template.

Template Functions

map

creates a key-value map of string -> interface{}

  1. {{$endpoint := map “name” “elasticsearch” “private_port” 9200 “public_port” 443}}
  2. name: {{index $endpoint “name”}}
  3. private-port: {{index $endpoint “private_port”}}
  4. public-port: {{index $endpoint “public_port”}}

specifically useful if you a sub-template and you want to pass multiple values to it.

base

Alias for the path.Base function.

  1. {{with get “/key”}}
  2. key: {{base .Key}}
  3. value: {{.Value}}
  4. {{end}}

exists

Checks if the key exists. Return false if key is not found.

  1. {{if exists “/key”}}
  2. value: {{getv “/key”}}
  3. {{end}}

get

Returns the KVPair where key matches its argument. Returns an error if key is not found.

  1. {{with get “/key”}}
  2. key: {{.Key}}
  3. value: {{.Value}}
  4. {{end}}

gets

Returns all KVPair, []KVPair, where key matches its argument. Returns an error if key is not found.

  1. {{range gets “/*”}}
  2. key: {{.Key}}
  3. value: {{.Value}}
  4. {{end}}

getv

Returns the value as a string where key matches its argument or an optional default value. Returns an error if key is not found and no default value given.

value: {{getv "/key"}}

With a default value

value: {{getv "/key" "default_value"}}

getvs

Returns all values, []string, where key matches its argument. Returns an error if key is not found.

  1. {{range getvs “/*”}}
  2. value: {{.}}
  3. {{end}}

getenv

Wrapper for os.Getenv. Retrieves the value of the environment variable named by the key. It returns the value, which will be empty if the variable is not present. Optionally, you can give a default value that will be returned if the key is not present.

export HOSTNAME=`hostname`
hostname: {{getenv "HOSTNAME"}}

With a default value

ipaddr: {{getenv "HOST_IP" "127.0.0.1"}}

datetime

Alias for time.Now

# Generated by confd {{datetime}}

Outputs:

# Generated by confd 2015-01-23 13:34:56.093250283 -0800 PST
# Generated by confd {{datetime.Format "Jan 2, 2006 at 3:04pm (MST)"}}

Outputs:

# Generated by confd Jan 23, 2015 at 1:34pm (EST)

See the time package for more usage: http://golang.org/pkg/time/

split

Wrapper for strings.Split. Splits the input string on the separating string and returns a slice of substrings.

  1. {{ $url := split (getv “/deis/service”) “:” }}
  2. host: {{index $url 0}}
  3. port: {{index $url 1}}

toUpper

Alias for strings.ToUpper Returns uppercased string.

key: {{toUpper "value"}}

toLower

Alias for strings.ToLower. Returns lowercased string.

key: {{toLower "Value"}}

json

Returns an map[string]interface{} of the json value.

lookupSRV

Wrapper for net.LookupSRV. The wrapper also sorts the SRV records alphabetically by combining all the fields of the net.SRV struct to reduce unnecessary config reloads.

  1. {{range lookupSRV “mail” “tcp” “example.com”}}
  2. target: {{.Target}}
  3. port: {{.Port}}
  4. priority: {{.Priority}}
  5. weight: {{.Weight}}
  6. {{end}}

Add keys to etcd

  1. etcdctl set /services/zookeeper/host1 ‘{“Id”:“host1”, “IP”:“192.168.10.11”}’
  2. etcdctl set /services/zookeeper/host2 ‘{“Id”:“host2”, “IP”:“192.168.10.12”}’

Create the template resource

  1. [template]
  2. src = “services.conf.tmpl”
  3. dest = “/tmp/services.conf”
  4. keys = [
  5. “/services/zookeeper/”
  6. ]

Create the template

  1. {{range gets “/services/zookeeper/*”}}
  2. {{$data := json .Value}}
  3. id: {{$data.Id}}
  4. ip: {{$data.IP}}
  5. {{end}}

Advanced Map Traversals

Once you have parsed the JSON, it is possible to traverse it with normal Go template functions such as index.

A more advanced structure, like this:

  1. {
  2. “animals”: [
  3. {“type”: “dog”, “name”: “Fido”},
  4. {“type”: “cat”, “name”: “Misse”}
  5. ]
  6. }

It can be traversed like this:

  1. {{$data := json (getv “/test/data/”)}}
  2. type: {{ (index $data.animals 1).type }}
  3. name: {{ (index $data.animals 1).name }}
  4. {{range $data.animals}}
  5. {{.name}}
  6. {{end}}

jsonArray

Returns a []interface{} from a json array such as ["a", "b", "c"].

  1. {{range jsonArray (getv “/services/data/”)}}
  2. val: {{.}}
  3. {{end}}

ls

Returns all subkeys, []string, where path matches its argument. Returns an empty list if path is not found.

  1. {{range ls “/deis/services”}}
  2. value: {{.}}
  3. {{end}}

lsdir

Returns all subkeys, []string, where path matches its argument. It only returns subkeys that also have subkeys. Returns an empty list if path is not found.

  1. {{range lsdir “/deis/services”}}
  2. value: {{.}}
  3. {{end}}

dir

Returns the parent directory of a given key.

  1. {{with dir “/services/data/url”}}
  2. dir: {{.}}
  3. {{end}}

join

Alias for the strings.Join function.

  1. {{$services := getvs “/services/elasticsearch/*”}}
  2. services: {{join $services “,”}}

replace

Alias for the strings.Replace function.

  1. {{$backend := getv “/services/backend/nginx”}}
  2. backend = {{replace $backend “-“ “_” -1}}

lookupIP

Wrapper for net.LookupIP function. The wrapper also sorts (alphabeticaly) the IP addresses. This is crucial since in dynamic environments DNS servers typically shuffle the addresses linked to domain name. And that would cause unnecessary config reloads.

  1. {{range lookupIP “some.host.local”}}
  2. server {{.}};
  3. {{end}}

Example Usage

etcdctl set /nginx/domain 'example.com'
etcdctl set /nginx/root '/var/www/example_dotcom'
etcdctl set /nginx/worker_processes '2'
etcdctl set /app/upstream/app1 "10.0.1.100:80"
etcdctl set /app/upstream/app2 "10.0.1.101:80"

/etc/confd/templates/nginx.conf.tmpl

  1. worker_processes {{getv “/nginx/worker_processes”}};
  2. upstream app {
  3. {{range getvs “/app/upstream/*”}}
  4. server {{.}};
  5. {{end}}
  6. }
  7. server {
  8. listen 80;
  9. server_name www.{{getv “/nginx/domain”}};
  10. access_log /var/log/nginx/{{getv “/nginx/domain”}}.access.log;
  11. error_log /var/log/nginx/{{getv “/nginx/domain”}}.log;
  12. location / {
  13. root {{getv “/nginx/root”}};
  14. index index.html index.htm;
  15. proxy_pass http://app;
  16. proxy_redirect off;
  17. proxy_set_header Host $host;
  18. proxy_set_header X-Real-IP $remote_addr;
  19. proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  20. }
  21. }

Output: /etc/nginx/nginx.conf

  1.  worker_processes 2;
  2.  upstream app {
  3.  server 10.0.1.100:80;
  4.  server 10.0.1.101:80;
  5.  }
  6.  server {
  7.  listen 80;
  8.  server_name www.example.com;
  9.  access_log /var/log/nginx/example.com.access.log;
  10.  error_log /var/log/nginx/example.com.error.log;
  11.  location / {
  12.  root /var/www/example_dotcom;
  13.  index index.html index.htm;
  14.  proxy_pass http://app;
  15.  proxy_redirect off;
  16.  proxy_set_header Host $host;
  17.  proxy_set_header X-Real-IP $remote_addr;
  18.  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  19.  }
  20.  }

Complex example

This examples show how to use a combination of the templates functions to do nested iteration.

Add keys to etcd

  1.  etcdctl mkdir /services/web/cust1/
  2.  etcdctl mkdir /services/web/cust2/
  3.  etcdctl set /services/web/cust1/2 ‘{“IP”: “10.0.0.2”}’
  4.  etcdctl set /services/web/cust2/2 ‘{“IP”: “10.0.0.4”}’
  5.  etcdctl set /services/web/cust2/1 ‘{“IP”: “10.0.0.3”}’
  6.  etcdctl set /services/web/cust1/1 ‘{“IP”: “10.0.0.1”}’

Create the template resource

  1.  [template]
  2.  src = “services.conf.tmpl”
  3.  dest = “/tmp/services.conf”
  4.  keys = [
  5.  “/services/web”
  6.  ]

Create the template

  1.  {{range $dir := lsdir “/services/web”}}
  2.  upstream {{base $dir}} {
  3.  {{$custdir := printf “/services/web/%s/*” $dir}}{{range gets $custdir}}
  4.  server {{$data := json .Value}}{{$data.IP}}:80;
  5.  {{end}}
  6.  }
  7.  server {
  8.  server_name {{base $dir}}.example.com;
  9.  location / {
  10.  proxy_pass {{base $dir}};
  11.  }
  12.  }
  13.  {{end}}

Output:/tmp/services.conf

  1.  upstream cust1 {
  2.  server 10.0.0.1:80;
  3.  server 10.0.0.2:80;
  4.  }
  5.  server {
  6.  server_name cust1.example.com;
  7.  location / {
  8.  proxy_pass cust1;
  9.  }
  10.  }
  11.  upstream cust2 {
  12.  server 10.0.0.3:80;
  13.  server 10.0.0.4:80;
  14.  }
  15.  server {
  16.  server_name cust2.example.com;
  17.  location / {
  18.  proxy_pass cust2;
  19.  }
  20.  }
2020年 · Linux · 点滴记录

Linux shell ${}简单用法

为了完整起见,我这里再用一些例子加以说明 ${ } 的一些特异功能:
假设我们定义了一个变量为:
file=/dir1/dir2/dir3/my.file.txt
我们可以用 ${ } 分别替换获得不同的值:
${file#*/}:拿掉第一条 / 及其左边的字符串:dir1/dir2/dir3/my.file.txt
${file##*/}:拿掉最后一条 / 及其左边的字符串:my.file.txt
${file#*.}:拿掉第一个 . 及其左边的字符串:file.txt
${file##*.}:拿掉最后一个 . 及其左边的字符串:txt
${file%/*}:拿掉最后条 / 及其右边的字符串:/dir1/dir2/dir3
${file%%/*}:拿掉第一条 / 及其右边的字符串:(空值)
${file%.*}:拿掉最后一个 . 及其右边的字符串:/dir1/dir2/dir3/my.file
${file%%.*}:拿掉第一个 . 及其右边的字符串:/dir1/dir2/dir3/my

记忆的方法为:

# 是去掉左边(在鉴盘上 # 在 $ 之左边)

% 是去掉右边(在鉴盘上 % 在 $ 之右边)

单一符号是最小匹配﹔两个符号是最大匹配。

${file:0:5}:提取最左边的 5 个字节:/dir1
${file:5:5}:提取第 5 个字节右边的连续 5 个字节:/dir2
我们也可以对变量值里的字符串作替换:
${file/dir/path}:将第一个 dir 提换为 path:/path1/dir2/dir3/my.file.txt
${file//dir/path}:将全部 dir 提换为 path:/path1/path2/path3/my

2020年 · Linux · 点滴记录

Linux tcp状态 FIN_WAIT1、TIME_WAIT、ESTABLISHED

命令:
netstat -n | awk ‘/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}’
显示服务器各种状态及其数量

1、time_wait状态过多。
原因:最可能的原因是服务器keepalive没有开,导致每次请求都要建立新的tcp连接,请求完成以后关闭,增加了很多time_wait的状态。另,keepalive可能会增加一部分内存的开销,但是问题不大。

2、fin_wait1状态过多。fin_wait1状态是在server端主动要求关闭tcp连接,并且主动发送fin以后,等待client端回复ack时候的状态。fin_wait1的产生原因有很多,需要结合netstat的状态来分析。
netstat -nat|awk ‘{print awk $NF}’|sort|uniq -c|sort -n
上面的命令可以帮助分析哪种tcp状态数量异常
netstat -nat|grep “:80″|awk ‘{print $5}’ |awk -F: ‘{print $1}’ | sort| uniq -c|sort -n
则可以帮助你将请求80服务的client ip按照连接数排序。

回到fin_wait1这个话题,如果发现fin_wait1状态很多,并且client ip分布正常,那可能是有人用肉鸡进行ddos攻击、又或者最近的程序改动引起了问题。一般说来后者可能性更大,应该主动联系程序员解决。
但是如果有某个ip连接数非常多,就值得注意了,可以考虑用iptables直接封了他。
————————————————

转:https://blog.csdn.net/u012661496/java/article/details/80432499

2020年 · Linux · 点滴记录

Linux中删除特殊符号文件名文件

Linux 系统下的文件名长度最多可到256个字符。通常情况下,文件名的字符包括:字母、数字、“.”(点)、“_”(下划线)和“-”(连字符)。
Linux 允许在文件名中使用除上述符号之外的其它符号, 但并不建议用户这样做。 有些转意字符 (即该字符被系统借用,表示某种特殊含义) 在 Linux 的 命令解释器(shell)中有特殊的含义。这样的转意字符有:“?”(问号),“*”(星号), “ ”(空格), “$”(货币符), “&”, 扩号等等。在文件名中应尽量避免使用这些字符。文件名中可以有” “(空格),但建议用户用“_”(下划线)来替代。“/” 既可代表目录树的根也可作为路径名中的分隔符(类似DOS下的“\”),因此“/“不能出现在文件名中。和 DOS 一样,“.” 和 “..”分别用来表示“当前目录”和“父目录”, 因此它们也不能作为文件名。

1. 文件名含有特殊字符,直接使用 rm 可能删除不了,可以使用如下方法:
1) 使用 ls -i 查处该文件的 inode 号,假设为654321
2) 使用find命令删除
find ./ -inum 654321 -exec rm ‘{}’ \;

2. 如果文件名是以 – 连字符开头的,可以使用如下方法来删除,如删除 “-filename” 文件:
rm — -filename
rm ./-filename