Best command line tools for linux performance monitoring

Sometimes a system can be slow and many reasons can be the root cause. To identify the process that is consuming memory, disk I/O or processor capacity you need to use tools to see what is happening in an operation system.

There are many tools to monitor a GNU/Linux server. In this article, I am providing 6 monitoring tools and i hope it will help you.

First install EPEL RPM extras repo:

# sudo yum install epel-release

Htop

Htop is an alternative of top command but it provides interactive system-monitor process-viewer and more user friendly output than top.
htop also provides a better way to navigate to any process using keyboard Up/Down keys as well as we can also operate it using mouse.

htop

# yum -y install htop

To start Htop simply type the following

# htop

dstat

Dstat is a versatile replacement for vmstatiostatnetstat and ifstatDstat overcomes some of their limitations and adds some extra features, more counters and flexibility. Dstat is handy for monitoring systems during performance tuning tests, benchmarks or troubleshooting.
Dstat allows you to view all of your system resources in real-time, you can eg. compare disk utilization in combination with interrupts from your IDE controller, or compare the network bandwidth numbers directly with the disk throughput (in the same interval).
Dstat gives you detailed selective information in columns and clearly indicates in what magnitude and unit the output is displayed. Less confusion, less mistakes. And most importantly, it makes it very easy to write plugins to collect your own counters and extend in ways you never expected.
Dstat’s output by default is designed for being interpreted by humans in real-time, however you can export details to CSV output to a file to be imported later into Gnumeric or Excel to generate graphs.

dstat

# yum -y install dstat

To start dstat simply type the following

# dstat

Saidar

Saidar is a curses-based application to display system statistics. It use the libstatgrab library, which provides cross platform access to statistics about the system on which it’s run. Reported statistics include CPUloadprocessesmemoryswapnetwork input and output and disks activities along with their free space.

saidar

# yum -y install saidar

To start saidar simply type the following

# saidar

Glances

Glances is a cross-platform curses-based command line monitoring tool writen in Python which use the psutil library to grab informations from the system. Glance monitoring CPULoad AverageMemoryNetwork InterfacesDisk I/OProcessesand File System spaces utilization.

Glances can adapt dynamically the displayed information depending on the terminal siwrize. It can also work in a client/server mode for remote monitoring.

glances

# yum -y install glances

To start glances simply type the following

# glances

Atop

Atop is an interactive monitor to view the load on a Linux system. It shows the occupation of the most critical hardware resources on system level, i.e. cpu, memory, disk and network. It also shows which processes are responsible for the indicated load with respect to cpu- and memory load on process level.

Disk load is shown if per process “storage accounting” is active in the kernel or if the kernel patch ‘cnt’ has been installed. Network load is only shown per process if the kernel patch ‘cnt’ has been installed.

atop

# yum -y install atop

To start Atop simply type the following

# atop

IPTraf

The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others. It can provide the following info in easy to read format:

  • Network traffic statistics by TCP connection
  • IP traffic statistics by network interface
  • Network traffic statistics by protocol
  • Network traffic statistics by TCP/UDP port and by packet size
  • Network traffic statistics by Layer2 address

iptraf

# yum -y install iptraf

To start IPTraf simply type the following

# iptraf

or

# iptraf-ng

Enjoy!

Backup Server using Mondo Rescue

Good day! Here is a good tool to backup your entire CentOS server called Mondo Rescue.

First step! Download MondoRescue Repository for your Linux OS distribution version using following command:

cd /etc/yum.repos.d/
wget ftp://ftp.mondorescue.org/rhel/6/x86_64/mondorescue.repo

After downloading MondoRescue Repository, run the following command to install MondoRescue.

# yum install mondo

After installing Mondo, Run “mondoarchive” command from “root” user. Then follow steps according to screenshot that shows how to create Backup/Clone in ISO based backup media of your full system.

# mondoarchive

First you need to choose the backup media and press Enter.

mondo-rescue-step-1

Please enter the full path name to the directory for your ISO Images and press Enter.

mondo-rescue-step-2

Now select the type of compression type like: bzip2, gzip, lzo & lzma and press Enter.

mondo-rescue-step-3

Now select the compression level and press Enter.

mondo-rescue-step-4

Please enter how large you want each ISO image in MB (Megabytes) and press Enter.

mondo-rescue-step-5

Please give the name of ISO image filename and press Enter.

mondo-rescue-step-6

Please add the filesystems to backup (separated by “|“). The default filesystem is “/” means full backup.

mondo-rescue-step-7

Please exclude the filesystem that you don’t want to backup like /tmp.

mondo-rescue-step-8

Please enter your temporary directory path or select default one.

mondo-rescue-step-9

Now enter your scratch directory path or select default one.

mondo-rescue-step-10

If you would like to backup extended attributes. Just press “enter“.

mondo-rescue-step-11

It will found full path name of your Kernel.

mondo-rescue-step-12

If you want to Verify your backup, then Click “Yes“

mondo-rescue-step-13

If you want to proceed the backup then click on “Yes”

mondo-rescue-step-14

Creating a catalog of “/” filesystem.

mondo-rescue-step-15

Dividing filelist into sets.

mondo-rescue-step-16

Follow up the other setup and build your backup. You may also take a look at the log file located here:

# /var/log/mondoarchive.log

NOTE : Your backup will be located in /var/cache/mondo/

I ran into a problem with one of my HP Server

Error: Package: perl-IO-Interface-1.05-2.el6.x86_64 (mondorescue)
Requires: perl(:MODULE_COMPAT_5.10.1)

5_10_missing

5.10… 😉 missing!

Have fun!

Midnight Commander in ClearOS

Install Midnight Commander on ClearOS 7+

GNU Midnight Commander (also known as mc) is a free and visual file manager, licensed under GNU General Public License. It is a clone of Norton Commander. It was started by Miguel de Icaza in 1994. It’s a feature rich full-screen text mode application that allows you to copy, move and delete files and whole directory trees, search for files and run commands in the subshell. Internal viewer and editor are included.

Features

Midnight Commander is quite powerful and boasts an extensive set of features:

  • Performs all the common file and directory manipulations such as copying, moving, renaming, linking, and deleting.
  • Allows manipulation of file and directory permissions.
  • Can treat remote systems (via FTP or SSH) as though they were local directories.
  • Can treat archive files (like .tar and .zip) as though they were local directories.
  • Allows creation of a user-defined “hotlist” of frequently used directories.
  • Can search for files based on file name or file contents, and treat the search results like a directory.

Install Midnight Commander (mc) On Centos / RHEL / Fedora Linux / ClearOS

# yum -y install mc

How do I use mc?

Simply type the following command:

# mc

Sample outputs:

NOTE : Midnight Commander also support your mouse device 😉

Enjoy!

Prevent new user to browse your CentOS server using sFTP

Good day! I run into a problem yesterday that allowed a new created account to navigate my entire CentOS server when connecting using sFTP protocol. Here is the step by step guide to fix this major problem in CentOS/Webmin/VirtualMIN.

    1. Create a new account in VirtualMIN / Ex. mynewwebsite.com
      After you created the new account you will have to set things up so that when this new account user connect to your sFTP (I use proFTPD) they will not be able to navigate your entire server.

NOTE: You will have to create a new UserGroup : sftpusers

In webmin, navigate to this section /Others/File Manager/ Once you are there select the new created account

/home/mynewwebsite

You will have to do 3 more steps

  1. CHOWN the new created directory to ROOT
    NOTE : Do not use recursive!
  2. You have to set now the directory permissions to 0755
  3. Run this command in Shell # usermod -G sftpusers USERNAME

proftpd-root-01

NOTE : This will add your new created account to the sFTPusers group!

After this your new created account will only have permission to navigate into the directory /home/mynewwebsite/ when connecting with sFTP.

Enjoy!

Are you having a hard time installing this device?

Wow! I am stuck now!!! Trying to install my new Intel 3160 WIFI

minipcie card into … with red hat! Updated Kernel too!

Will crack my head on this ~ one… & will keep you posted

You should stick with this card … for C/OS & PFS…

Qualcomm Atheros!

Cheers!

Setup…
IMG_4169

Test failed! kernel 4.2! + updated bios + firmware .14 no luck, Thanks for all the help out there.;.. result = Qualcomm Atheros! 100%

Just ordered this beauty!

thinkpenguin

https://www.thinkpenguin.com/gnu-linux/wireless-n-pci-express-dual-band-mini-half-height-card-tpe-nhmpcied2

Supported distributions:

*ClearOS 6.x , 7.x
Trisquel 4.5, 5, 5.5, 6, 6.01, 7

Ubuntu 10.04, 10.10, 11.04, 11.10, 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 15.10, 16.04
Lubuntu 10.04, 10.10, 11.04, 11.10, 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 15.10, 16.04
Kubuntu 10.04, 10.10, 11.04, 11.10, 12.04, 12.10, 13.04, 13.10, 14.04, 14.10, 15.04, 15.10, 16.04
Peppermint Linux OS 1, 2, 3, 4, 5, 6, 7
LinuxMint 9, 10, 11, 12, 13, 14, 14.1, 15, 16, 17, 17.1, 17.2, 17.3, 18
Fedora 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24
OpenSuSE 11.0, 11.1, 11.2, 11.3, 11.4, 12.1, 12.2, 12.3, 13.1, 13.2, Leap 42.1
Mageia 1, 2, 3, 4, 5, 6
Slackware 13.1, 13.37, 14, 14.1, 14.2
Debian 6, 7, 8
Tails 1.0, 1.0.1, 1.1, 1.1.1, 1.1.2, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.3, 1.3.1, 1.3.2, 1.4, 1.4.1, 1.5, 1.5.1, 1.6, 1.7, 2.0, 2.0.1, 2.3, 2.4
PCLinuxOS 2011.6, 2011.9, 2012.2, 2013.2, 2013.4, 2013.6, 2013.7, 2013.715, 2013.10, 2013.12, 2014.04, 2014.05, 2014.07, 2014.08, 2014.12, 2016.03
Red Hat Enterprise Linux (RHEL) 7 Series 7.0, 7.1, 7.2
Red Hat Enterprise Linux (RHEL) 6 Series 6.5, 6.6, 6.7, 6.8
CentOS 7 Series 7.0, 7.1, 7.2
CentOS 6 Series 6.5, 6.6, 6.7, 6.8
Scientific Linux 7 Series 7.0, 7.1, 7.2
Scientific Linux 6 Series 6.5, 6.6, 6.7, 6.8

* will test this out soon!

All test are done, this card work like a charm for ClearOS 7.x and the latest stable PFSENSE. The WI-FI signal is very strong depending of your antenna setup! I used for my test…,,, (2) 12DBI 3′ long antenna 😉

antenna-2

Cheers!

SFTP Connection Error Received unexpected end-of-file from SFTP server

proftpd

Here are the steps to fix the issue of sFTP – Received unexpected end-of-file – error when trying to connect to the sFTP Server.

First step is to edit your sshd_config file.

# nano /etc/ssh/sshd_config

Then scroll down until you see this line:

# Subsystem sftp /usr/lib/openssh/sftp-server

*  Do not worry if you don’t find this line, just add this code at the end of your sshd_config file.

SFTP Connection Error Received unexpected end-of-file
SFTP Connection Error Received unexpected end-of-file
Subsystem sftp internal-sftp

Now restart SSHD! and try to connect to your sFTP Server.

et voilà!

Configure SSH for high security RSA-4096 Bits

There are some steps to do after SSH is installed on a system and there is a old saying that says “A chain is only as strong as its weakest link” and if you are using a weak password for your root account (or any other account) then you are extremely vulnerable. It does not matter if the communication is secure when you are easily brute forced. All steps is used on a Ubuntu 11.10 but should be the same on OpenBSD, Debian, Linux Mint or any other Linux distribution with none or very few modifications.

We are going to do the following steps

  • Create certificate
  • Set correct credentials to .ssh folder and files
  • Shut down the possibility to log in with password
  • Prevent root to log in via SSH
  • Remove less secure encryption methods
  • Enable visual identification of the server fingerprint
  • Optional: Change SSH port (does really not not increase security)

Create certificate
We are going to use a RSA-key with a key length of 4096 bits. Open a terminal and enter the following “‘ssh-keygen -t rsa -b 4096”.  1024 bits key should be enough but better to be safe than sorry.

Generating public/private rsa key pair
ssh-keygen -t rsa -b 4096

Then you will be asked where to store the key. If you already got keys in id_dsa then you should enter another file name or your existing keys will be overwritten. If you are satisfied with the suggestion simply press enter.

Enter file in which to save the key (/home/accountname/.ssh/id_rsa)

It’s now time to enter a password. Use a strong password with big and small letters, numbers and symbols. The password should also be unique and stored on a secure place like in a encrypted container.

Enter passphrase (empty for no passphrase): 2sWf3+@/’?B>.%DpBU”r
Enter same passphrase again: 2sWf3+@/’?B>.%DpBU”r

Your identification has been saved in /home/accountname/.ssh/id_rsa.
Your public key has been saved in /home/accountname/.ssh/id_rsa.pub.

The key fingerprint is:
31:b0:be:0b:5b:7c:f1:79:65:e4:72:42:18:08:c4:8d
The key’s randomart image is:

+–[ RSA 4096]—-+
|     o++ ..o.          |
|      Eoo ..            |
|      . o   . .           |
|     .   o o +         |
|      . S   +           |
|     . o o o          |
|    . + o .            |
|     + o .             |
|    . .                  |
+—————–+

Enable the public key for authentication
The public key should be stored in ~/.ssh/authorized_keys and there can be more then one key for a single user. Just make a new row for each public key. If you key should be installed on the same system from where you just created the private key simply copy id_rsa.pub to authorized_keys

# ~$ cd ~/.ssh
# ~/.ssh$ cp id_rsa.pub authorized_keys

Set correct credentials to .ssh folder and files

Make sure that your working folder is your home folder, replace “john” with your username.

# ~/.ssh$ cd ~
# ~/.ssh$ sudo chown -R john:john .ssh
# ~/.ssh$ sudo chmod -R 600 .ssh
# ~/.ssh$ sudo chmod +x .ssh

Do a test log in to test the public key

# ~/.ssh$ ssh john@localhost
Enter passphrase for key ‘/home/john/.ssh/id_rsa’:

After you entered the private key password you should have access to your machine, if not you will have to look for errors in the logs but I will not cover this in this guide.

Configure sshd
The next step is to modify sshd. All settings we will change is in the file /etc/ssh/sshd_config. Start to make a backup of sshd_config just in case.

# john@john-laptop:/$ sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config_backup

Use desired editor to edit sshd_config. I prefer vi but I will use nano in this example

# john@john-laptop:/$ sudo nano /etc/ssh/sshd_config

The following lines is going to be added or altered:

PermitRootLogin yes
#PasswordAuthentication yes
Ciphers

PermitRootLogin no

root should never be used since it much more secure to use a regular user instead and then you need to perform a administrative task use the command sudo instead which gives you temporary administrative rights
We are also going to prevent the possibility to log in with password (you will be forced to use the private key). Find the rows which looks like  this:

PermitRootLogin yes

Modify it to look like this

PermitRootLogin no

Find the row which look like this

#PasswordAuthentication yes

Modify it to look like this

PasswordAuthentication no

At the end Cipers is going to be added and it may not apply never installations but the default ciphers has not always been the best choices and sshd should be forced to only use the strongest ones.

Ciphers aes128-ctr,aes256-ctr,arcfour256,arcfour,aes128-cbc,aes256-cbc

Verify these entries:

  • Protocol 2
  • UsePrivilegeSeparation yes
  • StrictModes yes
  • RSAAuthentication yes
  • PubkeyAuthentication yes

Save and exit

Restart to active the settings.

# ~/.ssh$ sudo service ssh restart
or
# systemctl restart sshd

Verified that SSHD is running

# systemctl status sshd

Enable visual identification of the servers fingerprint (Visual Host Key)
It’s not easy to verify and remember the fingerprint of a host since it’s a long hexadecimal string that may look like this one: ” 31:b0:be:0b:5b:7c:f1:79:65:e4:72:42:18:08:c4:8d” , some one may have altered the DNS record so that you in fact are trying to authenticate to a rouge server and to remember that string is near impossible. . It’s more easy to remember a visual fingerprint but it’s still not bulletproof. It’s absolute best to verify the exact string every time and that is done by most SSH clients and for example openssh stored them in ~/.ssh/known_hosts and gives you a warning if it has changed.

Do the following to enable visual host key

Edit eider /etc/ssh/ssh_config witch effects all users on the system or ~/.ssh/config to enable it for a single user.

Add the following lines (“Host * is already at top of ssh_config)

Host *
VisualHostKey yes

Test and verify
It’s now time to test and verify. You should not be able to log in without your private key and password authentication should been disabled. You should also see your visual finger print when you tries to log in.

Your SSH should be more safe now but remember that SSH probably was the most secure software from the beginning with default settings and MySQL, Apache or any other system also has to be secured.

Enjoy!

Virtualmin multiple account RSA-2048 Bits Key Setup

rsa-security

In the article I will eleborate how to install a 2048 Bits RSA Encryption Key for multiple Virtual Server on your Webmin/Virtualmin system.

Generate the RSA Key in your /home/accountname/.ssh

To do that, you can log in as the SSH user in question, navigate to the /home/accountname/.ssh folder, and run this command:

ssh-keygen -t rsa

That will generate (2) key files.
id_rsa and id_rsa.pub

id_rsa = Client
id_rsa.pub = Server

Next step will be to add or create a file called authorized_keys inside the  /home/accountname/.ssh

Open a text editor leave it blank and copy the content of the id_rsa.pub key. Save the file & upload it to your server into /home/accountname/.ssh

Now download your id_rsa key on your system in a safe place.

After those steps you will be able to connect using RSA 2048 Bits Key. The key is good for FileZilla, Putty or any other program that support KeyFile Login.

FILEZILLA
* Make sure to have the latest version. In older version you don’t have the option of using a KEY File !

filezilla-rsa-key

PUTTY

ras-putty

NOTE: Make sure those options are there in  /etc/ssh/sshd_config

# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile    .ssh/authorized_keys

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no
PasswordAuthentication no

# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes
ChallengeResponseAuthentication no

rsa-putty

Et voilà!

Install rkhunter (Rootkit Hunter)

rkhunter (Rootkit Hunter) is a Unix-based tool that scans for rootkits, backdoors and possible local exploits. It does this by comparing SHA-1 hashes of important files with known good ones in online databases, searching for default directories (of rootkits), wrong permissions, hidden files, suspicious strings in kernel modules, and special tests for Linux and FreeBSD.

Install RKHunter which is the Rootkit Detection tool

Install from EPEL

# yum – -enablerepo=epel -y install rkhunter

# nano /etc/sysconfig/rkhunter
Recipient address for report

MAILTO=root@localhost

If specified “yes”, scan more detaily

DIAG_SCAN=no

Update database

# rkhunter – -update

Update system file properties

# rkhunter – -propupd

Execute checking
–sk means sikpping to push Enter key.
If specified –rwo , display only warnings

# rkhunter – -check – -sk