How to Secure Your Linux Server: The Complete Guide

Securing Linux Servers blog image

Linux server hardening means fortifying and securing a Linux server in order to protect it from vulnerabilities and threats. While total security will always remain a moving target in the security arms race, this article explores some important fundamental steps on how to secure your Linux server.

🥷 NinjaOne helps you manage, patch, and support Linux endpoints with its robust endpoint management tool.

Schedule your 14-day free trial today!

No security risk is too small to ignore

Unfortunately, even the smallest vulnerabilities can transform into serious security threats. It’s important to remember that any code running on your system could potentially contain flaws that allow intruders to access a command shell or legitimate unprivileged users to elevate their access to superuser rights. No matter how low the level of risk may be, it’s always best to address minor security risks before they become major cybersecurity threats by following an established server hardening process. With Linux server hardening, IT teams can fortify their defenses before any serious threats occur, which will help prevent any major issues from occurring in the first place. 

System updates and security patches

Regularly updating your Linux server is essential for maintaining its security and overall health, especially weak security controls and practices are routinely exploited by threat actors, according to the Cybersecurity & Infrastructure Security Agency (CISA). Software and operating systems are constantly improved and refined to address vulnerabilities and weaknesses. By applying updates promptly, you ensure that known security flaws are patched, reducing the risk of exploitation by cyber threats. Moreover, updates often include bug fixes and performance enhancements, leading to a more stable and efficient server environment. Failing to update regularly can expose the system to potential attacks and compromise its integrity.

The risks of unpatched software vulnerabilities include:

  • Malware: Malicious code, such as ransomware, is one of the top cybersecurity threats. According to Norton, global cybercrime is expected to grow by 15% annually, reaching $10.5 trillion by 2025.
  • Data Breaches: IBM’s Cost of Data Breach Report 2023 states that the global average cost of a data breach for that year was $4.45 million. Experts have stated that data breaches are one of the leading causes of security vulnerabilities in an IT network.
  • Compliance issues: Regular patching keeps you compliant with various regulatory requirements, including PCI-DSS, HIPAA, and GDPR.
  • Reduced productivity: Unpatched software could impact operational efficiency, in addition to exposing your organization to cyber threats. Patching not only addresses security vulnerabilities but also fixes bugs that could impact performance.

Choosing the right patching strategies and best practices for your server patch management is crucial for maintaining a secure Linux server. Current enterprise-grade Linux distributions generally all have automated patch management services of some kind, whether by paid subscription service or community efforts. The paid services often go beyond simply patching your system files for security; some of these services also include a solution to live-patch your running kernel, allowing your running systems to stay patched before you even need to reboot them. 

Introduction to package managers

Package managers differ across distributions but tend towards using two main package formats: DEB and RPM. DEB is used on Debian-based systems and is managed by apt; RPM stands for Redhat Package Manager and is managed through dnf or yum for Red Hat/Enterprise Linux-based systems or zypper for SUSE. All package managers facilitate the installation, update, and removal of software packages, in addition to resolving dependencies. Utilize package managers to manage software effectively and streamline the update process. All major package managers are also supported by remote repositories, allowing you to centralize control over updates and software rollouts.

Configuring automatic updates for critical security patches

Configuring your Linux server to automatically install critical security patches ensures that high-priority updates are applied promptly without manual intervention, minimizing the window of vulnerability and reducing the burden on administrators. Enterprise distributions typically allow this to be configured during installation. See the configuration packages below, however, should you wish to set this up on your existing system:

Ubuntu: sudo apt install unattended-upgrades

Red Hat/CentOS: sudo yum install dnf-automatic

SUSE: sudo zypper install yast2-online-update-configuration ->  open YaST, select Software -> Online Update Configuration, and configure to taste.

Exercise caution, however, when enabling automatic updates; these may impact system stability or require compatibility testing. Testing is especially important when using mass remote admin tools such as ansible or puppet to do server updates in a heterogeneous computing environment.

Configuring sudo elevation and shell access

Correctly configuring your user accounts and permissions is essential to having a secure system that still allows the right access. Permissions that were not configured properly can create some issues, so it’s best to ensure that they are set up correctly from the start.

Using the sudo command to its full potential with the right /etc/sudoers configuration lets you limit and permit passwordless root access rights on a per-app basis. Always use the visudo command to edit /etc/sudoers as this maintains the correct permissions on that critical file:

sudo visudo

Make changes similar to these, replacing ninja with your username and adjusting the comma-separated list of allowed commands to suit your use case:

someuser ALL=(ALL:ALL) ALL

to

someuser ALL = NOPASSWD: /bin/systemctl restart nginx, /bin/kill, /usr/sbin/reboot

Then save and exit. Note that sudo effectively reads the /etc/sudoers file from the bottom upwards, so if your changes don’t seem to be effective, try moving your edited line to be near the bottom of the sudoers file.

A word on shells

You can set a user’s default shell by running the chsh command as root, or en masse by editing the /etc/passwd file as root. This is useful and desirable in some cases, such as changing an FTP-only account’s default shell to /sbin/nologin — since such an account should never need to run any terminal commands. Removing shell access in such a case would shrink the system’s potential attack surface.

Securing SSH access with keys and disabling root login

Secure Shell (SSH) is an industry-standard network protocol that allows command-line level remote access to a system. It is a firmly held belief among tech folk that shell access should by design be equivalent to “as if you’re local”, so it’s crucial to limit SSH access to necessary users only, and also to make sure only the right users can do things as root.

Best practices for securing SSH access to Linux servers

It is generally a good idea to entirely disable both root login via SSH and password-based SSH access. While the password text in password-based authentication is transmitted in encrypted form by the SSH host keys that get exchanged during the pre-login SSH client/server handshake, the password is nevertheless transmitted across multiple networks in some fashion. This is another attack surface aspect— If your SSH server’s host keys were generated using an older/insecure algorithm, for instance, then your passwords can be snooped.

SSH key logins are not vulnerable to this sort of attack; your public keys are the only parts that get transmitted in “plaintext”, and they were designed for public distribution. As with your private key itself, the passphrase you use to unlock it during SSH key login is not transmitted over the network.

Use SSH keys for authentication

Note that the code snippets below are annotated with the author’s comments indicated by “##” for instruction’s sake.

  1. Generate an SSH key using the ssh-keygen command. By default, the industry-standard RSA encryption key format is used, but it is recommended to use the faster and more secure ed25519 elliptic-curve encryption standard.

    user@host:~> ssh-keygen -t ed25519   
    Generating public/private ed25519 key pair. Enter file in which to save the key (/home/user/.ssh/id_ed25519): 
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /home/user/.ssh/id_ed25519
    Your public key has been saved in /home/user/.ssh/id_ed25519.pub
    The key fingerprint is: SHA256:NjPk3GbSIL6wdITiSrEQ3U9uXqqaYJdtS62cwCahKoM user@host
    The key's randomart image is:
    +--[ED25519 256]--+
    |.. . |
    | .. ... |
    |... .+o o |
    |..o. o+=.+ |
    | +. ooooS = |
    |o.o.o++o B |
    |=o *.=.. |
    |E.+.* + |
    |o.o. = |
    +----[SHA256]-----+ 
    user@host:~>
  2. Copy your public key to your online hosting platform. If you still have password access to the server in question, you can use the ssh-copy-id command to automagically set up your remote account SSH config files with your newly generated SSH key copied into the right places:

    ssh-copy-id -i ~/.ssh/id_ed25519.pub remoteuser@remotehost
  3. Ways of Using SSH Keys to Login:A) Directly in the command line:

    ssh -i ~/.ssh/id_ed25519 remoteuser@remotehost:port

    OR by using the ~/.ssh/config file:

  4. Edit your file ~/.ssh/config with any text editor. Do this as the account/user who will be using the SSH config to connect remotely.
  5.  Add stanzas in the following format:

    Host server1   ## Arbitrary label/handle for easy usage
    Hostname server1.example.com   ## Real hostname/IP
    User remote_user_1  ## Remote Linux username
    Port 12345  ## Remote port, default 22
    IdentityFile ~/.ssh/id_ed25519  ## NB! Private key file
    ForwardX11 yes ## Optional, forwards remote GUI apps
    
    Host server2  
    Hostname server2.example.com
    User remote_user_2
    Port 34567 
    IdentityFile ~/.ssh/id_ed25519


    There should be vertical whitespace between stanzas.

  6. Now you can SSH simply by typing:

    ssh server1
    

    Since everything else is specified under that stanza label in your .ssh/config file. Protip — if you’ve embraced Linux Tab-completion (aka bash completion), you’ll be happy to know that your SSH config file automatically plugs into most modern Linux distros’ bash-completion systems.

Disable SSH root login and password authentication

  1. Make ABSOLUTELY SURE you can login in with your SSH key. You can potentially lock yourself out of your system if this isn’t working properly.
  2. Use your favorite editor as root to change the following lines in  /etc/ssh/sshd_config to the following settings:

    PermitRootLogin no
    
    ChallengeResponseAuthentication no
    
    PasswordAuthentication no
    
    KbdInteractiveAuthentication no
  3. Optionally make your SSH server run on a non-standard port:

    Port 12345
  4. Save this file and reload your SSH service:

    sudo systemctl restart sshd
  5. Now try to log in via SSH without the SSH key:

    ssh remote_user_1@server1 [-p port]
    

    If you cannot log in without specifying an SSH key, you’ve done this step correctly.

NinjaOne’s low-footprint agent allows you to monitor your Linux endpoints with its endpoint management tool.

Check it out here.

Implementing a firewall on Linux servers

If you’re operating a production Linux server and your distribution comes with a pre-configured firewall, then you’re off to a great start. Whenever data from a production server will be exposed to the internet, it’s best to protect it by implementing a firewall. Enterprise Linux/Red Hat variants tend to ship with firewalld, whereas Debian/Ubuntu ships with ufw (“Uncomplicated Firewall”). Check out man firewall-cmd or man ufw for details.

Configuring firewall rules and policies

Firewall rules and policies, once you get past the details, should really boil down to “block everything except what I specifically allow” — again, these are generally preconfigured to sane defaults on modern Linux systems, especially on Enterprise Linux variants. Any use cases not covered by these sane defaults fall outside the scope of this document.

Monitoring and managing the firewall

  • Both firewalld and ufw plug into the major logging systems journald and/or [r]syslog.
  • Aside from the command-line firewall management tools firewall-cmd and ufw,  other remote management frontends like Cockpit, WebMin, and YaST offer modules which allow for firewall management. Other remote management tools like ansible and puppet also have modules for automatic firewall configuration and management.

Securing network-facing services

Best practices for securing network-facing services:

  • Have as little code on your system as possible.
  • Run the absolute minimum possible amount of services.
  • Don’t run a public-facing service as root. 
  • Servers do not need a GUI. Extraneous code and running services = increased attack surface.
  • Linux services rely on your securing the system with encryption certificates. Certbot/Let’s Encrypt has the easiest process to generate free encryption certificates.
  • Most Linux services listen on separate ports for their secure and insecure connections; see the relevant services’ man pages for details on security and other configuration options.
  • You can use the command netstat -tulpen to see the ports your system is listening on (whether or not those ports are blocked by your firewall; this shows listening ports on a per-process basis.) 

Install only necessary third-party packages

Ensure that you limit the number of third-party packages or applications you install on your Linux server. As with any download, practice good cybersecurity hygiene habits to reduce the risk of threats and keep your server as healthy as possible.

A simple yet powerful tip is to install server packages from reputable sources and those that will be used regularly. The last point needs to be emphasized! Even if a package comes from a trusted vendor, it is still prone to risk if it is not regularly used (not to mention that it takes up space in your server).

Use fail2ban

fail2ban is a self-regulating security utility for Linux that automatically checks your server for any automated digital attacks and blocks them. fail2ban can be easily configured by:

  1. Type the command: $ sudo apt install fail2ban – y
  2. Copy this configuration file: $ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
  3. Restart fail2ban to ensure it works properly: $ sudo service fail2ban restart

You can then configure the utility as you prefer. Regardless, fail2ban alerts you of any multiple login attempts on your remote servers and then block these addresses. Over time, you can examine log files to see a full list of banned IT addresses.

Implement two-factor authentication

Two-factor authentication (2FA) is a must for securing your Linux servers. Essentially, it is a security system that verifies users twice before they are given access to a system or account.

You can set up 2FA on your Linux server by first installing an authentication package. Depending on which package you choose, the additional layer of security can be anywhere from a security key to biometric authentication factors.

Work with a trusted endpoint management company

You can always manually check your Linux server for any vulnerabilities. Still, it may be more cost-effective to work with a trusted endpoint management company like Ninjaone to monitor, manage, and support your Linux devices from a single pane of glass.

Minimalism is as important to Linux server security as action

No amount of actions after the fact can replace making good choices from the start — in today’s cloud-connected world, one’s choices of Linux server distribution and remote management solutions matter more than ever. NinjaOne’s embedded Linux remote management software gives you the freedom to use any endpoint OS platform while remaining secure and patched.

It would be safe to say that the majority of SME companies’ tech infrastructure includes some Linux servers as part of their critical infrastructure. If you have a website, chances are that it’s running on a Linux server; if you use online banking, odds are it depends on the security of dozens of Linux servers. Given that even Android itself is (grossly oversimplified) a Linux kernel with some GUI stuff on top if you’re reading this on a smartphone, then the issue of Linux security may literally be closer to your heart than you might think.

Next Steps

The fundamentals of device security are critical to your overall security posture. NinjaOne makes it easy to patch, harden, secure, and backup all their devices centrally, remotely, and at scale.

You might also like

Ready to become an IT Ninja?

Learn how NinjaOne can help you simplify IT operations.

Watch Demo×
×

See NinjaOne in action!

By submitting this form, I accept NinjaOne's privacy policy.

Start your 14-day trial

No credit card required, full access to all features

NinjaOne Terms & Conditions

By clicking the “I Accept” button below, you indicate your acceptance of the following legal terms as well as our Terms of Use:

  • Ownership Rights: NinjaOne owns and will continue to own all right, title, and interest in and to the script (including the copyright). NinjaOne is giving you a limited license to use the script in accordance with these legal terms.
  • Use Limitation: You may only use the script for your legitimate personal or internal business purposes, and you may not share the script with another party.
  • Republication Prohibition: Under no circumstances are you permitted to re-publish the script in any script library belonging to or under the control of any other software provider.
  • Warranty Disclaimer: The script is provided “as is” and “as available”, without warranty of any kind. NinjaOne makes no promise or guarantee that the script will be free from defects or that it will meet your specific needs or expectations.
  • Assumption of Risk: Your use of the script is at your own risk. You acknowledge that there are certain inherent risks in using the script, and you understand and assume each of those risks.
  • Waiver and Release: You will not hold NinjaOne responsible for any adverse or unintended consequences resulting from your use of the script, and you waive any legal or equitable rights or remedies you may have against NinjaOne relating to your use of the script.
  • EULA: If you are a NinjaOne customer, your use of the script is subject to the End User License Agreement applicable to you (EULA).