atop

Usage:CPU, Memory usage of process, Network Throughput, Disk Throughput
Commands:
  t(For next sample taken after 10 min), Shift+t(prev 10 min)

$ atop -r /var/log/atop/atop_20260302
$ atop

MEM | tot 32G | free 860.4M | cache 6.3G    // 6.3G still left to be used
SWP | tot 1G | free 590M | swcac 16.3M

Press C   // CPU View
PID       VGROW       RGROW       CPU       CMD
xx         45.0G      17.8G       30%       npraxy

Press M   // Memory View
PID       VDATA       VSIZE       RSIZE    VGROW   RGROW     SWAPZ    MEM    CMD
xx         44.3G      45.0G       17.8G    45.05G  17.8G     92.4M    57%   npraxy

Network Throughput
NET | eth0 -- | pcki 460986 | pcko 308929 | sp 0Mbps | si 20Mbps | so 36 Mbps | erri 0 | erro 0| drpi 20 | drpo 0 |
si = incoming
so = outgoing
      

curl

-H: Header
-x: serverIP:Port
-k: Insecure, ignore server cert errors
            

URL="https://dlptest.com/10MB-Test.docx"
CONCURRENCY=10    #Simultaneous requests
ITER=10           #How many time entire batch will repeat

for j in $(seq 1 "$ITER"); do
  echo "Iteration: $j"
  for i in $(seq 1 "$CONCURRENCY"); do
    # Send request to server:port
    # & means run in background without waiting for 1st to finish
    curl -H "Connection: close" -x serverIp:port "$URL" -k &

  done
  sleep 10
done
wait      # Do not ext script until all processes finish
echo "Done"
            

df

# df -h        //Disk Free, Shows local and network file system
Filesystem   Size     Used    Avail    Use%  Mounted on
/dev/map/root 11G    3.8G     6.0G     39%    /          // root file system mounted on "/" has only 6.0G available
devtmpfs     2.0G       0     2.0G      0%    /dev
tmpfs        2.0G       0     2.0G      0%    /dev/shm
tmpfs        2.0G    1.6M     2.0G      1%    /run
tmpfs        2.0G       0     2.0G      0%    /tmp
      

htop

# htop //Similar to top with more colourful, more graphic interface which gives you more control of display scrolling
# ps    //Reports snapshot of current processes.        //ps -aux    a:Displays all processes on a terminal.  u: Show user-name,  x; Lists all process(Including background processes)
    user  pid  %cpu %mem  vsz     rss  tty  state   start-time      command
    root    1    0.0       0.1    19404  832  ?     Ss   Mar02 0:01   /sbin/init
    root    2    78.0       0.0       0         0    ?     S    Mar02 0:00   [abc]
    root    3    0.0       0.0       0         0    ?     S    Mar02 0:00   [migration/0]
    States of process:  D(uninterruptible sleep),  R(Running), S(Interruptible sleep),  T(stopped by job control signal),  t(stopped by debugger during the tracing), X(dead), Z(defunct/Zombie process, terminated but not reaped by its parent)
      

lsof (list open files)

This command reports list of all open files(pipes, sockets, devices, and directories) and the processes that opened them.

$ lsof
COMMAND PID TID USER FD TYPE DEVICE SIZE/OFF NODE NAME
...

$ sudo lsof -p 1234   // List files opened by process ID (PID) 1234.

// Sort in decending order of opened descriptors
$ sudo lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head -n 10
 20500 process_name1
  7500 process_name2
  5000 process_name3
 ..

// Print type of Object opened by process in decending order
$ sudo lsof -p pid_process_name1 | awk '{print $5}' | sort | uniq -c
          

proc

// Open file descriptors
$ ls /proc/$pid/fd/ | wc -l
288

// Max fds a process can Open
$ sudo cat /proc/$(pidof process1)/limits
limits          soft limit    Hard limit    Units
Max open files  1024          1024213       

// total number of open files across the entire OS
$ sudo cat /proc/sys/fs/file-max
Very huge number (More than 9 billion)
          

tc (Traffic Control)

Simulate Traffic Instability by adding delay, packet loss by changing Linux kernel's packet scheduling.

qdisc (Queuing Discipline)

Series of commands that tells kernel to catch specific traffic, hold it for few milliseconds then release.

// View all rules of qdisc
$ tc qdisc show dev eth0
-> Look for netem that means some delay is added to traffic
$ tc filter show dev eth0
-> Look for match lines for IP address

// Reset. Clear any existing traffic control rules on eth0
$ tc qdisc del dev eth0 root

// priority scheduling would apply on eth0
// Create 3 Bands: 1:1, 1:2, 1:3.
$ tc qdisc add dev eth0 root handle 1: prio

// Add delay of 30ms +- 10ms(jitter) to traffic in band 1:3 with 50% delay in current packet wrt prev packet
$ tc qdisc add dev eth0 parent 1:3 handle 30: netem delay 30ms 10ms 50%

// We built a slow lane band 1:3 in above command
// Traffic headed to this IP(172.16.178.41) will be placed in slow lane(ie delayed)
// 
$ tc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dst 172.16.178.41 flowid 1:3
      

top

# top
up  1:09, 3 user,  load average: 0.36, 0.12, 0.11
Tasks:   228 total,   1 running, 227 sleeping,   0 stopped,   0 zombie
Cpu(s):  3.7% us,  1.9 sy,  1.2 ni, 91.8 id,  1.3 wa,  0.0 hi,  0.1 si,  0.0 st
Memory(MB):  32092 total,   321 free,   31483 used,   287 buff/cache
Swap(MB):   1024 total,  0.0 free,   1024 used.      200 avail Mem

//All processes Running on System
pid   user    pr(priority)  ni(nice) virt  res  shr(SharedMem) State  %cpu %mem   time    command
7801  root    20               0    59.6g  26.5g   3224        R       15   84.2  4:36.47  sssd