Online troubleshooting , The following scenarios , Have you ever met ?

 

One , Understand the number of machine connections

problem :1.2.3.4 Of sshd The listening port of is 22, How to make statistics 1.2.3.4 Of sshd Service various connection states (TIME_WAIT/ CLOSE_WAIT/
ESTABLISHED) Number of connections for .

 

Common methods :

*
netstat -n | grep 1.2.3.4:22 | awk '/^tcp/ {++S[$NF]} END {for(a in S) print
a, S[a]}’

*
netstat -lnpta | grep ssh | egrep “TIME_WAIT | CLOSE_WAIT | ESTABLISHED”

*
n [ Alibaba cloud only ]

 

explain :netstat It is a common tool for tracing network connection problems , and grep/awk Combination is more a artifact , Of course, if it's on alicloud , There are more convenient ways .

 

Two , Query data from logs that have been backed up

problem : From backed up suyun.2019-06-26.log.bz2 In the log , Find the keywords 1.2.3.4 How many logs are there in .

 

Common methods :

*
bzcat suyun.2019-06-26.log.bz2 | grep '1.2.3.4' | wc -l

*
bzgrep '1.2.3.4' suyun.2019-06-26.log.bz2 | wc -l

*
less suyun.2019-06-26.log.bz2 | grep '10.37.9.11' | wc -l

 

explain : Online log files are generally bz2 Keep after compression , If you unzip the query , Very space and time consuming ,bzcat and bzgrep It's a tool for students to master .

 

Three , Backup service tips

problem : Pack and go backup /opt/web/suyun_web catalog , Exclude from catalog logs And contents , Packed files are stored in /opt/backup Directory .

 

Common methods :

tar -zcvf /opt/backup/shenjian.tar.gz \

    -exclude /opt/web/suyun_web/logs \

    /opt/web/suyun_web

 

explain : This command is frequently used online , When a project needs to be packaged for migration , It is often necessary to exclude the log directory ,exclude It's a parameter to master .

 

Four , Number of query threads

problem : Query the number of busses the server runs , When the number of machine threads exceeds the alarm threshold , Can quickly find out related process and thread information .

 

Reference answer :

*
ps -eLf | wc -l

*
pstree -p | wc -l

 

Five , Disk alarm , Empty max file

problem : Find the server , A running tomcat A large number of exception logs generated , Find the file , And free up space . Let's assume that the file contains log keyword , And greater than 1G.

 

Common methods :

Step 1 , Find the file

*
find / -type f -name "*log*" | xargs ls -lSh | more 

*
du -a / | sort -rn | grep log | more

*
find / -name '*log*' -size +1000M -exec du -h {} \;

 

Step 2 , Empty file

Assume the file found is a.log

The right way to do this is to :

echo "">a.log

File space will be freed immediately .

Many students will use :

rm -rf a.log

In this way, although the file is deleted , But because tomcat Service is still running , Space will not be released immediately , Restart required tomcat To free up space .

 

Six , Show files , Filter comments

problem : display server.conf file , Shield off # Comment line beginning with

 

Common methods :

*
sed -n '/^[#]/!p' server.conf

*
sed -e '/^#/d' server.conf

*
grep -v "^#" server.conf

 

Seven , disk IO Troubleshooting

problem : disk IO How to troubleshoot exceptions , Similar to slow write or high current usage , Please find out the cause of the disk IO Abnormally high process ID.

 

Common methods :

Step 1 :

iotop -o

View all processes currently writing to disk ID information .

Step 2 : If the writing indexes are very low at this time , Basically no large write operations , Check the disk itself . Can view system

dmesg

or

cat /var/log/message

See if there is any disk error , At the same time, it can be written to the slow disk touch Look at an empty file , Whether the disk fails to write .

I hope it can help the students who often operate online , If there is better practice , Welcome to share .

Technology
©2019-2020 Toolsou All rights reserved,
Bigtable Paper notes be based on springboot File upload for python Hundred lines of effective code to achieve Hanoi Tower Games ( Minimalist version )2019 Ladder race Lucky lottery (15 branch ) MySQL8.0MGR Single owner / Multi master installation and switching Java Function and use of reflection python Painting love A masterpiece to be handed down !Alibaba Interior products Redis In depth notes and source code data structure C Parking management of language JAVA Interview questions - Find the longest string that does not repeat ( Personal approach )