Hardened Kernel Variables ( /etc/sysctl.conf )

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

#Prevent SYN attack
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_synack_retries = 2

# Disables IP source routing
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.eth0.rp_filter = 1

# Disable ICMP Redirect Acceptance
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

# Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.lo.log_martians = 1
net.ipv4.conf.eth0.log_martians = 1

# Disables IP source routing
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.eth0.rp_filter = 1

# Disable ICMP Redirect Acceptance
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

# Modify system limits for Ensim WEBppliance
fs.file-max = 65000

# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 15

# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 1800

# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0

# Turn off the tcp_sack ( Need to turn on for traffic to internet)
#net.ipv4.tcp_sack = 0

# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0

# Enable TCP SYN Cookie Protection
net.ipv4.tcp_syncookies = 1

# Set maximum amount of memory allocated to shm to 256MB
kernel.shmmax = 268435456

# Increase the maximum total TCP buffer-space allocatable
net.ipv4.tcp_mem = 57344 57344 65536

# Increase the maximum TCP write-buffer-space allocatable
net.ipv4.tcp_wmem = 32768 65536 524288

# Increase the maximum TCP read-buffer space allocatable
net.ipv4.tcp_rmem = 98304 196608 1572864

# Increase the maximum and default receive socket buffer size
net.core.rmem_max = 524280
net.core.rmem_default = 524280

# Increase the maximum and default send socket buffer size
net.core.wmem_max = 524280
net.core.wmem_default = 524280

# Increase the tcp-time-wait buckets pool size
net.ipv4.tcp_max_tw_buckets = 1440000

# Allowed local port range
net.ipv4.ip_local_port_range = 16384 65536

# Increase the maximum memory used to reassemble IP fragments
net.ipv4.ipfrag_high_thresh = 512000
net.ipv4.ipfrag_low_thresh = 446464

# Increase the maximum amount of option memory buffers
net.core.optmem_max = 57344

Understanding the SysRq Key – The Magic Key To Control Linux

Linux do crash. It is not so frequent, but it happens. You know what to do when an application refuses to obey, but if everything fails, the last solution is using the supreme power on Linux: the magic SysRq key. By using this key with a combination of “Alt” and another letter, you will be able to make your computer respond no matter what. It was already evoked, but a little explanation does not hurt, and the more you know, the more prepared you will be when facing a real problem. Also, understanding the individual effect is always better than just repeating a combo learned by heart.

As said before, the magic key is super handy. But first, you have to activate this functionality if it is not already done. To check, use the command

cat /proc/sys/kernel/sysrq

If it returns a 1, the Magic Key is working. If it is a 0, you will need to activate it. In most cases, if you want to activate it temporarily, you can try

sudo echo "1" > /proc/sys/kernel/sysrq

But this did not work for me. So under Archlinux, I used instead:

sudo sysctl kernel.sysrq=1

Note that this will be only temporary. If you want the Magic Key to work even after a reboot, the best way is to edit the file “/etc/sysctl.conf”, so that the line

kernel.sysrq = 0


kernel.sysrq = 1

The whole point of using the Magic SysRq key is to combine it with the “Alt” key and another letter in order to trigger a particular function, even if your computer seems frozen. To begin, you should know that this System Request key is usually sharing its location with the “Print Scrn” key at the top right of your keyboard. Above the “Insert” key, and left to “Scroll Lock.”

As you can guess, the effect of the combination depends on which letter key is used. Here is a non-exhaustive list of letter you can use to obtain an interesting result.

  • b – reboots the computer
  • e – ask all processes to terminate gracefully
  • f – to get rid of an Out Of Memory condition via oom_kills
  • i – to kill all processes immediately except init
  • k – to kill absolutely all processes, including X
  • m – to output the current memory information
  • o – to shut down the computer
  • r – very useful, to take the keyboard out of the X server control
  • s – to sync data from all mounted devices (avoid data loss in case of violent reboot)
  • t – to display a list of the current tasks
  • u – to remount all file system in read-only mode

Every Linux user should know these examples. If your X server is frozen and you cannot access any virtual console, using “Alt + SysRq + r” should take the keyboard away from the X server and let you perform “Ctrl + Alt + F1” in order to get to a terminal and kill the appropriate processes.

If you do not have the patience to do that, “Alt + SysRq + k” will kill everything in the same way as “Ctrl + Alt + Backspace” (but more violently).

If nothing responds at all, and the only way is to reboot your computer, the classical combo is “reisub” for “Raising Elephant Is So Utterly Boring”, or just remember the opposite of “busier“. Use each letter of this anagram with the magic key in order to:

  • Take the keyboard from X
  • Ask all programs to end gently
  • Kill the one who did not
  • Save the data from the cache to the hard drive
  • Remounts the file systems
  • Reboot

The idea is that the magic key combinations are generally super violent. You do not want anybody to use it on your computer while you are not looking. But as a last resort, it can become the ultimate tool. I rarely encountered a situation where SysRq did not respond. So when your Linux does not obey its master, you know what to do.

Disable Firewire in Centos 7

Open the file /etc/modprobe.d/blacklist
* Don’t worry if the file is empty just populate it!

# nano /etc/modprobe.d/blacklist


# sudo nano /etc/modproble.d/blacklist

~ add these lines:

blacklist firewire_core
blacklist firewire_ohci

Save and Reboot!

Install SysStat to enable system monitoring

Install SysStat
# yum -y install sysstat
# systemctl start sysstat
# systemctl enable sysstat

Check the status of SysStat
# systemctl status sysstat

Add SysStat into a Cron Job
# cat /etc/cron.d/sysstat

# Run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 1 1
# 0 * * * * root /usr/lib64/sa/sa1 600 6 &
# Generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A

Settings of SysStat


# sysstat-10.1.5 configuration file.

# How long to keep log files (in days).
# If value is greater than 28, then log files are kept in
# multiple directories, one for each month.


# Compress (using gzip or bzip2) sa and sar files older than (in days):


# Parameters for the system activity data collector
# which are used for the generation of log files

# *note


# *note : valid options

INT     ⇒  System Interrupts
DISK    ⇒  Block Devices
SNMP    ⇒  SNMP statistics
IPV6    ⇒  IPv6 statistics
POWER   ⇒  Power Management statistics
ALL     ⇒  All of the above
XDISK   ⇒  DISK + Partition statistics
XALL    ⇒  All of the above (ALL + XDISK)

Setup psacct to Monitor User Activity in Linux

It is very important to know what are the activities for applications and users in linux operating system. This will be very useful in later time or in case of problems. For this purpose, i would recommend psacct tools to be install. psacct is a free monitoring program to monitor users and applications activity on linux server. This program will display how long user accessing the server, what command are they issuing, how many processes and display logs for commands.

If you are runninng Linux CentOS or Redhat, you should use the following command to install pssacct :

[root@test ~]# yum install psacct -y

By default psacct is disabled on Linux. We should manually start it :

[root@test ~]# systemctl start psacct

To enable psacct on your next reboot use the following command :

[root@test ~]# systemctl enable psacct

Check the status of psacct

[root@test ~]# systemctl status psacct

Other usage from that come in psacct or acct package :

ac command prints the statistics of user logins/logouts (connect time) in hours.
lastcomm command prints the information of previously executed commands of user.
accton commands is used to turn on/off process for accounting.
sa command summarizes information of previously executed commands.
last and lastb commands show listing of last logged in users.

Auditd service

Reload systemd daemon to reload changes made in auditd service unit file:

systemctl daemon-reload

Restart the auditd service: service auditd restart


## Remove any existing rules

## Buffer Size
## Feel free to increase this if the machine panic's
-b 8192

## Failure Mode
## Possible values are 0 (silent), 1 (printk, print a failure message),
## and 2 (panic, halt the system).
-f 1

## Audit the audit logs.
## successful and unsuccessful attempts to read information from the
## audit records; all modifications to the audit trail
-w /var/log/audit/ -k auditlog

## Auditd configuration
## modifications to audit configuration that occur while the audit
## collection functions are operating.
-w /etc/audit/ -p wa -k auditconfig
-w /etc/libaudit.conf -p wa -k auditconfig
-w /etc/audisp/ -p wa -k audispconfig

## Monitor for use of audit management tools
-w /sbin/auditctl -p x -k audittools
-w /sbin/auditd -p x -k audittools

## special files
-a exit,always -F arch=b32 -S mknod -S mknodat -k specialfiles
-a exit,always -F arch=b64 -S mknod -S mknodat -k specialfiles

## Mount operations
-a exit,always -F arch=b32 -S mount -S umount -S umount2 -k mount
-a exit,always -F arch=b64 -S mount -S umount2 -k mount

## changes to the time
-a exit,always -F arch=b32 -S adjtimex -S settimeofday -S stime -S clock_settime -k time
-a exit,always -F arch=b64 -S adjtimex -S settimeofday -S clock_settime -k time

## Use stunnel
-w /usr/sbin/stunnel -p x -k stunnel

## cron configuration & scheduled jobs
-w /etc/cron.allow -p wa -k cron
-w /etc/cron.deny -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /etc/cron.daily/ -p wa -k cron
-w /etc/cron.hourly/ -p wa -k cron
-w /etc/cron.monthly/ -p wa -k cron
-w /etc/cron.weekly/ -p wa -k cron
-w /etc/crontab -p wa -k cron
-w /var/spool/cron/crontabs/ -k cron

## user, group, password databases
-w /etc/group -p wa -k etcgroup
-w /etc/passwd -p wa -k etcpasswd
-w /etc/gshadow -k etcgroup
-w /etc/shadow -k etcpasswd
-w /etc/security/opasswd -k opasswd

## monitor usage of passwd
-w /usr/bin/passwd -p x -k passwd_modification

#Monitor for use of tools to change group identifiers
-w /usr/sbin/groupadd -p x -k group_modification
-w /usr/sbin/groupmod -p x -k group_modification
-w /usr/sbin/addgroup -p x -k group_modification
-w /usr/sbin/useradd -p x -k user_modification
-w /usr/sbin/usermod -p x -k user_modification
-w /usr/sbin/adduser -p x -k user_modification

## login configuration and information
-w /etc/login.defs -p wa -k login
-w /etc/securetty -p wa -k login
-w /var/log/faillog -p wa -k login
-w /var/log/lastlog -p wa -k login
-w /var/log/tallylog -p wa -k login

## network configuration
-w /etc/hosts -p wa -k hosts
-w /etc/network/ -p wa -k network

## system startup scripts
-w /etc/inittab -p wa -k init
-w /etc/init.d/ -p wa -k init
-w /etc/init/ -p wa -k init

## library search paths
-w /etc/ld.so.conf -p wa -k libpath

## local time zone
-w /etc/localtime -p wa -k localtime

## kernel parameters
-w /etc/sysctl.conf -p wa -k sysctl

## modprobe configuration
-w /etc/modprobe.conf -p wa -k modprobe

## pam configuration
-w /etc/pam.d/ -p wa -k pam
-w /etc/security/limits.conf -p wa  -k pam
-w /etc/security/pam_env.conf -p wa -k pam
-w /etc/security/namespace.conf -p wa -k pam
-w /etc/security/namespace.init -p wa -k pam

## GDS specific secrets
-w /etc/puppet/ssl -p wa -k puppet_ssl

## postfix configuration
-w /etc/aliases -p wa -k mail
-w /etc/postfix/ -p wa -k mail

## ssh configuration
-w /etc/ssh/sshd_config -k sshd

## changes to hostname
-a exit,always -F arch=b32 -S sethostname -k hostname
-a exit,always -F arch=b64 -S sethostname -k hostname

## changes to issue
-w /etc/issue -p wa -k etcissue
-w /etc/issue.net -p wa -k etcissue

## this was to noisy currently.
# log all commands executed by an effective id of 0 aka root.
-a exit,always -F arch=b64 -F euid=0 -S execve -k rootcmd
-a exit,always -F arch=b32 -F euid=0 -S execve -k rootcmd

## Capture all failures to access on critical elements
-a exit,always -F arch=b64 -S open -F dir=/etc -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/sbin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/sbin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/var -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/home -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/srv -F success=0 -k unauthedfileacess

## Monitor for use of process ID change (switching accounts) applications
-w /bin/su -p x -k priv_esc
-w /usr/bin/sudo -p x -k priv_esc
-w /etc/sudoers -p rw -k priv_esc

## Monitor usage of commands to change power state
-w /sbin/shutdown -p x -k power
-w /sbin/poweroff -p x -k power
-w /sbin/reboot -p x -k power
-w /sbin/halt -p x -k power

## Make the configuration immutable
-e 2

P.S After a reboot, total crash with auditd, IP internal Conflict ! and total CSF block! Good luck with this one!

How to find Hardware information on Live (running) Servers & Systems

dmidecode is a tool for dumping a computer DMI (some say SMBIOS) table contents in a human-readable format. This table contains a description of the system hardware components, as well as other useful pieces of information such as serial numbers  and  BIOS  revision.  You can retrieve this information without having to probe for the actual hardware.  While this is a good point in terms of report speed and safeness, this also makes the presented information possibly unreliable.

The  DMI  table  does not  only describe what the system is currently made of, it also can report the possible evolutions (such as the fastest supported CPU or the maximal amount of memory supported).

SMBIOS stands for System Management BIOS, while DMI stands for Desktop Management Interface. Both standards are  tightly related and developed by the DMTF (Desktop Management Task Force).

# yum install dmidecode

# dmidecode –type bios [About BIOS Details]

# dmidecode –type system [About System Information]

# dmidecode –type baseboard [About NIC & Storage Card]

# dmidecode –type chassis [About Chassis Information]

# dmidecode –type processor [About Processor Information]

# dmidecode –type memory [About Memory Information]

# dmidecode –type cache [About Cache Information]

# dmidecode –type connector [About USB Information]

# dmidecode –type slot [About Slot Information]

How to read or view utmp, wtmp and btmp files in Linux

In Linux operating systems everything is logged some where. Most of the system logs are logged in to /var/log folder. This folder contains logs related to different services and applications. In this folder we have some files such as utmp, wtmp and btmp. These files contains all the details about login’s and logout’s which are from local as well as from remote systems and system status such as uptime etc.

utmp will give you complete picture of users logins at which terminals, logouts, system events and current status of the system, system boot time (used by uptime) etc.

wtmp gives historical data of utmp.

btmp records only failed login attempts.

Provide how logged in, when they logged in and when they logged out
# last

To open wtmp file and view its content use blow command
# last -f /var/log/wtmp

To see still logged in users view utmp file use last command
# last -f /var/run/utmp

To view btmp file use same command
# last -f /var/log/btmp

Disable USB Devices Hardening Centos 7

We would like to disable all USB devices connected to our HP Red Hat Linux based workstations. I would like to disable USB flash or hard drives, which users can use with physical access to a system to quickly copy sensitive data from it. How do I disable USB device support under CentOS Linux, RHEL version 5.x/6.x/7.x and Fedora latest version?

The USB storage drive automatically detects USB flash or hard drives. You can quickly force and disable USB storage devices under any Linux distribution. The modprobe program used for automatic kernel module loading. It can be configured not load the USB storage driver upon demand. This will prevent the modprobe program from loading the usb-storage module, but will not prevent root (or another privileged program) from using the insmod/modprobe program to load the module manually. USB sticks containing harmful malware may be used to steal your personal data. It is not uncommon for USB sticks to be used to carry and transmit destructive malware and viruses to computers. The attacker can target MS-Windows, macOS (OS X), Android and Linux based system.

usb-storage driver

The usb-storage.ko is the USB Mass Storage driver for Linux operating system. You can see the file by typing the following command:
# ls -l /lib/modules/$(uname -r)/kernel/drivers/usb/storage/usb-storage.ko

All you have to do is disable or remove the usb-storage.ko driver to restrict to use USB devices on Linux such as:

  1. USB keyboards
  2. USB mice
  3. USB pen drive
  4. USB hard disk
  5. Other USB block storage

BIOS option

You can also disable USB from system BIOS configuration option. Make sure BIOS is password protected. This is recommended option so that nobody can boot it from USB.


In linux it’s even more easily done, by unloading the usb_storage module: for disable :-

# modprobe -r usb_storage

for enable :-

# modprobe -i usb_storage

The easiest way to disable usb storage device in linux is create following file And add following line inside the file

# touch /etc/modprobe.d/no-usb

install usb-storage /bin/true

Lynis new version 2.5.0

Lynis Security Auditing

Lynis is our system and security auditing tool for Linux, Mac OS X, and UNIX-based systems.

It provides insights in how well a system is hardened and what you can do, to improve your security defenses.

The software is open source and free to use. It is updated on a regular basis, to keep up with new technologies.

Security should be simple, but it is definitely not. With Lynis you gain quick insights in how well you are protecting your crown jewels. From your personal notebook to surf the web, up to where your company’s biggest secrets are stored.

We suggest people using it daily & compare the results for example:

Installation steps:

cd /tmp
wget https://cisofy.com/files/lynis-2.5.0.tar.gz
tar xvfz lynis-2.5.0.tar.gz
mv lynis cd

Move all contents of /tmp/cd into /usr/local/lynis
* Make sure that  lynis file is 775 or else you will get a perm denied 😉

To scan the server first do a update!

# lynis update info

Then to actually scan the system:

# lynis audit system

Once the scan is over you will get a System Scan Summary
Note: This is the actual results of easy-admin.ca server

Lynis suggests also a very good things that might be tampered to make the system more secure, so using some of its output when I have time I’ll work out on hardening all servers.

Commercial support available

For companies who prefer additional support, we have also Lynis Enterprise. It uses Lynis as a client. On top of that, it has additional plugins, reporting, central management, a dashboard, and more guidance (e.g. hardening snippets). With Lynis in its core, you are assured of a stable piece of software, which is up-to-date.

Examples of plugins:

  • Compliance (e.g. HIPAA, PCI DSS, ISO27001)
  • Docker
  • File integrity
  • Systemd

Source: https://cisofy.com/lynis

# sysctl -a
# lynis show
# lynis –tests “SSH-7440”
# lynis show help
# lynis update info
# systemctl status -all