Here’s a quick guide on how to patch some of the many Linux distros against the Meltdown and Spectre vulnerabilities! After spending the week monitoring each distribution, and deciding the best time to patch (after waiting for results).

Don’t forget to reboot your machine/server after applying the updates!

CentOS 7

x86_64

$ sudo yum clean all && yum install kernel-3.10.0-693.11.6.el7.x86_64
$ sudo reboot

Patched Kernel : kernel-3.10.0-693.11.6.el7.x86_64

Source: https://lists.centos.org/pipermail/centos-announce/2018-January/022696.html

CentOS 6

x86_64

$ sudo yum clean all && yum install kernel-2.6.32-696.18.7.el6.x86_64
$ sudo reboot

Patched Kernel : kernel-2.6.32-696.18.7.el6.x86_64

i386

$ sudo yum clean all && yum install kernel-2.6.32-696.18.7.el6.i686
$ sudo reboot

Patched Kernel : kernel-2.6.32-696.18.7.el6.i686

Source: https://lists.centos.org/pipermail/centos-announce/2018-January/022701.html

Ubuntu

x86_64

$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo reboot

Patched 16.04 LTS Kernel : linux-image-4.4.0-108-generic

Source: https://usn.ubuntu.com/usn/usn-3522-1/

Debian

x86_64

$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo reboot

Patched Kernel : linux-image-4.9.0-5-amd64

Source: https://packages.debian.org/stretch/kernel/linux-image-4.9.0-5-amd64

CloudLinux 7

x86_64

$ yum clean all --enablerepo=cloudlinux-updates-testing && yum update linux-firmware microcode_ctl && yum install kernel-3.10.0-714.10.2.lve1.5.8.el7 --enablerepo=cloudlinux-updates-testing
$ reboot

Patched Kernel : kernel-3.10.0-714.10.2.lve1.5.8.el7

Source: https://www.cloudlinux.com/cloudlinux-os-blog/entry/intel-cpu-bug-kernelcare-and-cloudlinux

CloudLinux 6

x86_64

$ yum clean all --enablerepo=cloudlinux-updates-testing && yum install kernel-2.6.32-896.16.1.lve1.4.50.el6 --enablerepo=cloudlinux-updates-testing
$ reboot

Patched Kernel : kernel-2.6.32-896.16.1.lve1.4.50.el6

Source: https://www.cloudlinux.com/cloudlinux-os-blog/entry/cloudlinux-6-kernel-updated-1-5

OpenVZ

x86_64

$ yum install vzkernel-2.6.32-042stab127.2.x86_64
$ reboot

Patched Kernel : vzkernel-2.6.32-042stab127.2.x86_64.rpm

Source: https://openvz.org/Download/kernel/rhel6/042stab127.2

Warning

At the time of posting, the kernel versions were the latest. If a new kernel is available, use the latest version.

If in doubt:

(centos, rhel, fedora, oracle, scientific linux)

$ yum update all

or (debian/ubuntu)

$ apt-get update && apt-get dist-upgrade

I’ll be adding more as I find stable releases.

The problem?

For weeks, I’ve been battling with an issue with a new CentOS 7 template for cPanel and Plesk, I built for the OpenVZ hypervisor. Even when setting the HOSTNAME=<hostname> in the /etc/vz/<CTID>.conf the container still rebooted with the hostname which was used when the template was created. Meaning the new and correct hostname would never be remembered. Causing various issues with BIND, Apache etc.

Even trying to set the hostname with the following failed!

[email protected] /]# hostnamectl set-hostname server2.host.com
Could not set property: Activation of org.freedesktop.hostname1 timed out

Today it finally clicked! It all starts with the with the base template I downloaded from https://openvz.org/Download/template/precreated

It seems that embedded in the template is a Systemd script which sets the hostname when the container starts.

The solution?

The hostname is taken from /etc/sysconfig/ovzhostname within the container. So you have a few options.

1. Set the hostname in  /etc/sysconfig/ovzhostname
2. Disable the ovzhostname service

   [email protected] /]# systemctl disable ovzhostname.service

This will then hopefully mean that when the container restarts, the hostname will be persistent.

Further reading…

There isn’t a lot I can find about this, apart from: https://lists.openvz.org/pipermail/users/2016-November/007204.html

Here’s the ovzhostname script so you can see what it’s doing:

#!/bin/bash

OVZHOSTNAME=""

if [[ -f /etc/sysconfig/ovzhostname ]]; then
source /etc/sysconfig/ovzhostname
if [[ -n "${OVZHOSTNAME}" ]]; then
echo "${OVZHOSTNAME}" > /etc/hostname
hostname "${OVZHOSTNAME}"
hostnamectl set-hostname "${OVZHOSTNAME}"
fi
fi

After seeing apache using all it’s threads, and connections not timing out as they should, I looked at the apache error_log and found the following error.

Message: collection_store: Failed to write to DBM file "/var/cache/modsecurity/ip": Invalid argument

I not only saw this on cPanel servers, but on Plesk and plain LAMP (with mod_security and comodo waf installed).

It looks like Comodo somehow released a broken update, that caused the /var/cache/modsecurity/ip.pag to corrupt (that’s my guess).

The fix is rather simple. Update your Comodo WAF rules to version 1.142 or higher, and reset the /var/cache/modsecurity/ip.pag file.

Check the rules.dat file for the version number.

$ echo "" > /var/cache/modsecurity/ip.pag

Then restart apache, and this should fix you issue. You’ll also notice apache threads timing out faster.

Plesk Support Article: https://support.plesk.com/hc/en-us/articles/213368509-mod-security2-Failed-to-write-to-DBM-file-var-cache-modsecurity-ip-Invalid-argument

Comodo WAF: https://waf.comodo.com/

Another quick fix post!

Problem

A mailbox for a specific user was not receiving mail on a postfix & dovecot on a Plesk server which I managed. The following error messages were being shown in the mail log:

$ tail -f /var/log/maillog
Apr 25 15:04:34 server01 dovecot: service=imap, [email protected],
ip=[192.168.0.1]. Error:
/var/qmail/mailnames/exampledomain.com/dan/Maildir/dovecot.index:
modseq_hdr.log_offset too large

As you can see the the error is: “modseq_hdr.log_offset too large”.

I’m not sure what caused this (I think it’s related to the dovecot.index file becoming corrupted – but not 100% sure), but this quick solution fixed it.

Solution

To fix this, delete all dovecot files (config and index files) from the specific users mail directory:

$ find /var/qmail/mailnames/example.com/dan/Maildir -name "dovecot*" -delete

Restart Postfix & Dovecot (to rebuild the dovecot files):

$ service dovecot restart
$ service postfix restart

Warning: This fix removes the dovecot configuration and index files for that specific user. Make sure you back them up before running the above command! You may wish to restore them at a later date.

By magic the mailbox began to receive email!

After spending time over Christmas coding a tool to query Steam servers for information. I’ve now been taking the next steps… Sending data to a Steam server!

For this example, I’m going to be using a Rust Dedicated Server, to send a simple command, then in future posts show how I sent scheduled commands (such as adverts, messages and other routine tasks which you would expect a Rust server to run).

First things first, after learning the hard way; Don’t run your Rust server with rcon.web = 1. It’s a new protocol, and I found it almost impossible to get working. Maybe if I had more time and patience, I could have cracked it. However, using the old protocol (which I assume is steam’s rcon protocol) is much easier.

Here’s the startup command I used when starting the Rust server (notice “rcon.web 0”):

./RustDedicated -batchmode +server.ip 127.0.0.1 +server.port 28015 +server.tickrate 30 +server.hostname "Server Name" +server.identity "rust-server"  +server.maxplayers 250 +server.worldsize 3000 +server.saveinterval 300 +rcon.web 0 +rcon.ip 127.0.0.1 +rcon.port 28016 +rcon.password "Password" -logfile "gamelog-2017-01-22-22-47-05.log"

Code time!

Again I’m using xPaw’s SourceQuery. Here’s a simple example of how to send a command via RCON:

<?php 
require __DIR__ . '/SourceQuery/bootstrap.php'; 
use xPaw\SourceQuery\SourceQuery; 

define( 'SQ_TIMEOUT', 1 ); 
define( 'SQ_ENGINE', SourceQuery::SOURCE ); 

// Init SourceQuery 
$Query = new SourceQuery( ); 

// Connect to the server 
$Query->Connect("<IP>", "<PORT>", SQ_TIMEOUT, SQ_ENGINE);

// Auth
$Query->SetRconPassword( "<Password>" );

// Run the "status" command and return
var_dump($Query->Rcon( "status" ));

// Disconnect
$Query->Disconnect( );

Result!

Which outputs the following:

[email protected]:/var/www/zurk/scripts/rcon# php test.php
string(269) "hostname: Server Name
version : 1955 secure (secure mode enabled, connected to Steam3)
map     : Procedural Map
players : 0 (250 max) (0 queued) (0 joining)

id name ping connected addr owner violation kicks
"

In the next post, I’ll be writing about how I took this simple example, and made a scheduled messaging system. Until then!

This is a really small blog post, but it’s an issue I wanted to share – so hopefully anyone who comes across this issue themselves can avoid the mistake I made!

Basically, we noticed that one of our older Plesk servers seemed to have lost basic functionality like editing DNS zones, or accessing PHPMyAdmin or Webmail.

When going to Tools & Settings > Server Components we noticed that next to all the services (components), there was the message “The component was not installed“.

Interestingly, when accessing the server via SSH, the servers were installed, and functioning correct (websites were accessible, DNS was querying etc).

Sadly, I jumped straight to the wrong conclusion, and thought that a recent automatic update of Plesk had failed, and that the PSA services had been corrupted. I was wrong.

I ran the Plesk bootstrapper. If anyone is familiar with this, you’ll understand the pain I was going through. Two hours later, and the issue was not resolved. What do you try next?

Well, after the bootstrapper failed to fix the issue, I just tried running “yum update” to see if there were any package updates etc.

BOOM! There it was!

Thread died in Berkeley DB library
DB_RUNRECOVERY: Fatal error, run database recovery

Great! That’s a really easy fix.

rm -f /var/lib/rpm/__*
rpm --rebuilddb
yum clean all

Now, check back in Plesk, and all functionality and the Server Components list will be fixed!

So I have my JSON data which contains all the query information which was returned from the Source Query Protocol. I’m storing it as a flat JSON file – the reason being (as discussed in my last post) is simply due to speed (or the assumption it would be faster). I don’t want to be getting into caching database queries, optimising the indexes, normalisation etc… As this project is for fun, let’s just store the JSON on the file system, and pull the data out when we need it.

What should I do with this data?

Well, that’s easy! Let’s create a website, and display it! I admit, I’m no front-end developer. So I chose to use a simple PHP Framework I wrote about 8 years ago. It’s a very simple MVC Framework, which consists of 10 files.

Without going into too much detail (maybe this is a topic for another blog post), here’s the file structure:

/config - Configuration Files

/content - Views

/data - Storage (JSON in this example)

/includes - Controllers

/models - Models

Very simple, but why spend time setting up and configuring something like Laravel or CodeIgnitor, when I can copy/paste my old code, and get cracking? I wrote a controller to loop through my list of servers, pull the data from each of the JSON files, then pass it to the view to be displayed.

CSS & HTML Framework

I decided to use Good old faithful Bootstrap (version 3), and customised some of the basic elements. Again, nothing special, just a quick and easy setup, and throwing together code in minutes!

No npm, grunt, composer, gulp or any other technology! Just raw PHP, CSS and HTML.

Here’s the finished design.

As you can see, I’m displaying information such as the Name, IP, Players and giving the user options to connect (or purchase their own, wink, wink). It’s just a simple “col-md-6” <div>. Nothing fancy, but it means when I add more servers, they can just drop in, and look apart of the page. There’s a few situations, where the divs don’t line up – mainly when there are many players online. However I could resolve this by adding an ellipsis to the players text – one for another time!

What about the Graph?

Ok I added the data used in the graph after my previous post. I simply added integer variable to the foreach loop, which I collate the number of players on each server together. I’m then throwing this data into another flat file with a timestamp.

<datetime>,<number of players>
<datetime>,<number of players>

I then pull this data back in the front end via the controller, and use ChartJS to display it.

<script>
 var ctx = document.getElementById("myChart");
 var data = {
 labels: ["04:20","04:25","04:30","04:35","04:40","04:45","04:50","04:55","05:00","05:05","05:10","05:15","05:20","05:25","05:30","05:35","05:40","05:45","05:50","05:55","06:00","06:05","06:10","06:15","06:20","06:25","06:30","06:35","06:40","06:45","06:50","06:55","07:00","07:05","07:10","07:15","07:20","07:25","07:30","07:35","07:40","07:45","07:50","07:55","08:00","08:05","08:10","08:15","08:20","08:25","08:30","08:35","08:40","08:45","08:50","08:55","09:00","09:05","09:10","09:15",],
 datasets: [
 {
 label: "Players",
 fill: true,
 lineTension: 0,
 backgroundColor: "rgba(187,255,0, 0.09)",
 borderColor: "rgba(187,255,0,1)",
 borderCapStyle: 'butt',
 borderDash: [],
 borderDashOffset: 0.0,
 borderJoinStyle: 'miter',
 pointBorderColor: "rgba(187,255,0,1)",
 pointBackgroundColor: "#fff",
 pointBorderWidth: 0,
 pointHoverRadius: 5,
 pointHoverBackgroundColor: "rgba(187,255,0,1)",
 pointHoverBorderColor: "rgba(187,255,0,1)",
 pointHoverBorderWidth: 2,
 pointRadius: 0,
 pointHitRadius: 10,
 data: [2,2,2,2,2,2,2,2,2,2,4,4,4,4,4,4,4,5,5,8,4,4,4,4,4,4,4,4,5,6,6,6,6,6,6,6,6,4,4,5,5,5,5,5,4,4,5,6,5,4,5,6,7,8,8,8,9,8,8,8],
 spanGaps: false,
 }
 ]
 };
 var myChart = new Chart(ctx, {
 type: 'line',
 data: data,
 options: {
 legend: {
 display: false
 },
 responsive : true,
 scaleShowLabels : false,
 scales: {
 yAxes: [{
 display: true,
 ticks: {
 stepSize: 1,
 beginAtZero: true
 }
 }],
 xAxes: [{
 display: false
 }]
 }
 }
 });
</script> 

As the cron is running every 5 minutes, the graph is showing the number of players, every 5 minutes. Really simple, but a nice addition to the page.

I’d imagine, when I add more servers, I could add options to the graph for different time periods (hour, day, month). This will allow me to see the peak times players are online.

Around this page, I’ve started to build up other pages, and more elements I can use. I’ll leave this blog post here, and maybe continue in another.

It’s been a very fun experience so far, and even though I’m not using any bleeding edge/funky languages or technologies. It’s been nice to refresh my knowledge of Bootstrap, CSS, jQuery, and also learn how Source Query Protocol works!

Here’s where I’ve left the website so far:

Hopefully as time allows, I’ll begin to build it up, and add more pages and servers!

Over the Christmas Period, I decided to develop a website to monitor the multitude of gaming servers that I host. As a game server provider, I’ve often wondered how many players are actively playing across the network.

For this, I decided to use a simple LEMP (Linux, NGINX, MySQL, PHP) environment. Furthermore, I wanted this website to be heavily reliant on JSON data, and would try to avoid pulling data from MySQL where possible – as I’m assuming pulling from a JSON file (using caching), is faster than from MySQL.

The first task was to obtain a list of the game servers on our network. This was quite easy, due to the fact our billing system already contains this information.

I built a JSON string containing an array of the servers:

Example:

{"1":{"id": "1", "protocol":"steam", "game": "Rust","ip":"46.101.5.78", "port": "28015"},"2":{"id": "2", "protocol":"steam", "game": "Rust","ip":"178.62.55.51", "port": "28015"}}

Lovely and simple!

ID: Unique Primary Key
Protocol: Steam
Game: The game!
IP: Servers IP
Port: Servers Port

Now, I have my list of servers. Next it to query them for the data! To do this, we need to interrogate the Source Query Protocol. There’s many open source examples of this out there. I found one by xPaw at https://github.com/xPaw/PHP-Source-Query – It’s very well coded class, and rather than reinventing the wheel, I choose to use his class.

So, I create a cron, to query the server list on a predefined interval.

Here’s the code.

It’s a proof of concept, so don’t judge the coding standards!

<?php

require __DIR__ . '/SourceQuery/bootstrap.php';
use xPaw\SourceQuery\SourceQuery;

define('SQ_TIMEOUT', 1);
define('SQ_ENGINE', SourceQuery::SOURCE);

$servers = json_decode(file_get_contents('servers.json'));

foreach ($servers as $server) {
 switch ($server->protocol) {
 case 'steam':
 $data = fetchSteam($server->id, $server->ip, $server->port, $server->game);
 break;
 }
}

function fetchSteam($id, $ip, $port, $game)
{

 $Query = new SourceQuery();

 try {
 $Query->Connect($ip, $port, SQ_TIMEOUT, SQ_ENGINE);

 $data["info"] = $Query->GetInfo();
 $data["info"]["Game"] = $game;
 $data["info"]["IP"] = $ip;
 $data["info"]["Port"] = $port;
 $data["players"] = $Query->GetPlayers();
 $data["rules"] = $Query->GetRules();

 file_put_contents('servers/' . $id . '.json', json_encode($data));
 } catch (Exception $e) {
 echo $e->getMessage();
 } finally {
 $Query->Disconnect();
 }
}

I’m looping through the list of servers, querying each one – then storing the returned data in a separate JSON file (1.json, 2.json etc).

Why the switch statement? Well, not every game server we host is powered by Steam, eventually I plan to add more protocols…

So, what data can we pull back from the Source Query Protocol?

Well, here’s the JSON decoded (pretty format thanks to http://jsonviewer.stack.hu/)

$Query->GetInfo()

$Query->GetPlayers()

$Query->GetRules()

As you can see, we get quite a fair amount of information back from the Source Query Protocol. The only thing I’m struggling to get is the Players SteamID. However, I have no requirement for it, so it’s not a pressing issue – If I wanted to display or track players in more depth across the network, I would need the SteamID.

I’ll leave this blog post here for now, and write Part 2 with the details of what I did next (The interface & graphing).

I’ve been going through the multiple languages I know recently, and refreshing my knowledge. After working with Python for a few months, I found going back to C# very easy.

I’ve noticed many similarities except for a few; In my opinion the ternary operator slightly differs from each language. So I’d thought I’d write these findings down for future use.

C# & C++

variable = (input > 0) ? 'true' : 'false';

Python

variable = 'true' if input > 0 else 'false'

PHP

$variable = ($input > 0) ? 'true' : 'false';

I encountered this issue on the R1Soft Web Interface last week, which I had to open a support ticket for with R1Soft’s brilliant support.

When trying to run tasks like a backup, run, or restore I would encounter the error messages as seen above.

The error was down to the maximum amount of memory which was assigned to the Java Heap and PermGen. After contacting R1Soft’s support, I was pointed to a config file where you can adjust the java heap and permgen values.

To increase the java heap:

# Edit /usr/sbin/r1soft/conf/server.conf

1. Locate the following line at the top: “compute.maxmemory=true” and change it to… “compute.maxmemory=false“
2. Locate the following line: “#maxmemory=” and remove the hash symbol. Set it to use 8192 (in MB). It should now read… “maxmemory=8192“
3. Save the server.conf file.

# Now restart the CDP Server Service.

1. /etc/init.d/cdp-server stop
2. /etc/init.d/cdp-server restart

To increase the PermGen Space:

# Edit /usr/sbin/r1soft/conf/server.conf

1. Locate the following line “additional.10=-XX:MaxPermSize=256m” and change it to… “additional.10=-XX:MaxPermSize=1024m“
2. Save the server.conf file.

# Now restart the CDP Server Service

1. /etc/init.d/cdp-server stop
2. /etc/init.d/cdp-server restart