Showing 11 result(s)
black trash bin with full of trash

NPM: Maximum call stack size exceeded

You may have cloned a project and tried running ‘npm install’ only to find the following error message:

error Maximum call stack size exceeded

There’s multiple ways I’ve found to fix this:

The Safe ways:

Force clear the NPM cache:

npm cache clean --force
npm install

Rebuild:

npm rebuild
npm install

Remove node_modules:

rm -r node_modules
npm install

The Nuclear way:

rm package-lock.json
rm -r node_modules
npm install

As always, I hope this fixes your problem!

Some thoughts on pairing during software development! (Part 1)

Over the past few months, I’ve been pairing (pair programming) with other Software Engineers when working on stories during our sprints. It’s been fun, and the first time in a while I’ve enjoyed and felt rewarded while ‘pairing’. While I’m sitting in the sun, on a beach in Wales enjoying our holiday with my family, I look back and consider it hasn’t all been plain sailing…

My first exposure to pair programming came as I was working on a team that was writing API’s for some of our business services. It was a relatively new formed team with a range of mid to junior level Software Engineers, a Business Analyst, a Product Owner, a QA tester and a Scrum Master.

We’re sitting around a table in the office’s shared kitchen (there’s never a decent sized meeting room when you need one) for our bi-weekly retrospective. The lead Software Engineer had joined us, and we are discussing the usual ups and downs of the last sprint.

We had discovered that the current block of work we had was a set of rather similar tasks. Each developer was picking up a new story ready for development that would be a small task, but all on a similar part of the micro-service. It was found that during committing the code into source control, that software engineers were overwriting or change the same chunk of code. This meant lots of merge conflicts and code reviews taking longer.

So trying to address this issue, the team had decided in the Retrospective that developers should be teaming up with each other and working on the chunk in parallel.

Ok let’s give pair programming a go now the team is mature enough” says the Lead Software Engineer.

Over the next few weeks and months, we attempted to introduce pair programming into the team, with some challenges. As I’ve later moved onto other teams, I’ve seen some sort of pattern to how teams adopt pair programming.

There’s some thoughts I’ve had on pairing which may help:

Pairing should not be forced. Processes, routines or schedules don’t work.

So it’s week one of our team attempting to pair programme. How should we approach this? “We have four developers on our team, so let’s rotate” one engineer suggests.

Ok, but when do we know when to change pairs?” asks another.

After a relatively lengthy discussion, it was decided that the following ‘rules/guidelines/suggestions’ would be introduced.

  • Developers should switch pairs once a story is completed.
  • If a pair has been together for more than two days, they must switch.
  • One developer must remain on the story from start to finish.
  • Nobody should work on a story on their own.

Ok, it was a little flakey. But it offered what we thought some ground rules for pairing to take off and work… How very wrong we were!

It turns out that even though we liked the idea of pairing when it came down to it, the first few weeks were incredibly hard. We had been used to independently working on our separate tickets. When ‘forced’ to work with another human being with a separate way of working, a different way of tackling a problem or even an alternative biological clock (toilet breaks, and caffeine pitstops) it is incredibly hard.

I personally found keeping my train of thought almost impossible. With many interruptions and events during the day which would take our focus off pressing keys and coding.

It wasn’t working! Not only was it rather frustrating for the developers, but the ‘rules’ we had come up with were not working. The nature of different stories being completed at separate times meant some developers were waiting around. Waiting for another pair to finish and unable to pick up a new story to work on – We had created dependencies between stories without realising it!

We tried to adapt the rules and even printed out a schedule that hung above our Kanban board. We could see which pairs were currently together, and developers waiting for a new partner.

Eventually, due to frustration and general lack of interest, the pairing stopped, and we naturally drifted back into flying solo! Much to the Scrum Master’s disappointment. In a last ditched effort, the Scrum Master regularly walked around the office trying to find developers not pairing. ‘Strongly’ encouraging them to relocate to another developers desk and pair. Which then started to cause other pain points, like feeling Micromanaged.

Forcing pair programming doesn’t work. If anything it turns the developers against the idea and we will then try our utmost to avoid it.

Pairing works best when it is natural, encouraged but not mandatory

If you’re reading this thinking ‘how should I introduce pairing into my team‘ this is my suggestion.

Don’t have any rules/suggestions/enforced ways of working… Let it be natural. Takedown the printed schedule. Delete the list of ‘rules’ you have. Certainly do not walk around the office asking why a developer is not in a pair!

Just go ahead, walk up to another developer and say something along the lines of “Is it OK if I sit with you and learn about what you’re working on?

I’m speaking for myself, but Developers are very proud of their work (I know we shouldn’t be protective of our code, but show me a developer who isn’t proud of their work, and I’ll eat my keyboard).

Asking the above will hopefully put the other developer into a mode where they want to show you what they’ve been working on, and open the door to further conversation. Sit with them. Don’t try and take over, kindly ask them to explain what they’re doing. Why they decided to take that approach, and what their train of thought is.

Hopefully, after a while, they may feel comfortable with you being there, and you may be able to offer suggestions, praise their work or help with that question they just Googled. But remember, you are in their territory. Don’t take over, and certainly don’t say they are doing something wrong. You will just rattle their cage and will not gain their trust (Yes this is like taming a Lion).

Eventually, you should have built up that level of trust with the developer that you can ask the following question, something along the lines of “I really enjoyed working alongside you with that Story, could we possibly try to work on the next one together?” You should offer to ‘drive’ this time.

Over time, you should find it easier to work alongside each other. So you can start to build on those foundations. Offer to take turns coding, writing a test, or contacting someone to find an answer to a question. I find writing tests together is really beneficial. You can help each other to spot the pitfalls or that bit of information missing from the Story.

When you feel like you’re walking, it’s then time to run… If you’ve been sharing a computer and taking turns driving, you may want to take it up a gear and install some sort of screen sharing software (like slack, teams etc). Basically a piece of software that you can see each other’s screens and control if needed. In the remote world we now exist in (COVID-19), this is must for pairing, but back in a physical office space, it might not be always needed (remember to do whatever feels natural).

Based off experience, both members of the pair need to offer to take turns. Don’t just sit back and watch the other developer. Get involved, even if you don’t know how to write the test, refactor the method, or create that builder pattern – offer to take control and work together to achieve the goal.

I’m not saying this will lead to amazing results, or it will work for all developers, but it’s worked for me. Especially when you join a new team, or start a new piece of work. It can work with a junior or senior developer, as long as you are humble and accept it may take time to form that trust in the pairing relationship.

You’ll encounter problems, you’ll get frustrated that the other developer may not have understood you. You will get annoyed when your idea isn’t implemented as you’d have liked. This is all part of the process, and as you keep pairing, you’ll overcome these little problems.

What you’ll get is a high performing, focused pair, taking stories across the board in record time. When the story first gets picked up, it will be understood quicker, code reviews will be quicker (as the story has technically been under review the whole time as the developers discuss the implementation) and will move to ‘Done’ faster.

In part two, I’ll talk about:

  • Pairing works when both engineers are not multitasking or context switching and can focus 100% of their time to the task.
  • Both Engineers should start and finish the task together. Breaks in between are of course beneficial.
  • Engineers can join the pair, but not replace anyone from the original pair.
  • Both engineers should be given equal credit.
  • Pairing remotely can seem hard, but is actually feasible and often works better than physical pairing.
  • Is it just Software Engineers who can pair? No.

[Link to Part Two when it’s finished here]

Penny Saving Challenge using Starling Bank, Lambda and Node

I came across a money-saving Challenge called the ‘1p Challenge‘. Basically you save an amount of money based on the day number in the year.

For example, on January the 1st, you would save 1p. Towards the end of the year, say day 365, you would save £3.65

As you can imagine, this starts saving small but eventually ramps up so you will save around £667 over the year.

The problem is, it’s incredibly hard to manage yourself. Finding the time or remembering to transfer funds into another account every day is a challenge. So, like most things in my life, I decided to automate it.

I recently moved over to Starling Bank. It’s an online-only bank which has an amazing app to control your money. It also allows access for developers to an API.

Inside the Starling App, there is a concept of ‘Spaces’. These are pots of money you can use to save for things (Holidays, Life Events etc). Basically, when you move money into a ‘Space’, it’s removed from your current balance, and added to the ‘Space’ – it’s like a separate savings account. This is perfect, as I can set up a ‘Space’ called the ‘Penny Challenge’ and move money into each day…

Now, people who know me closely will know when it comes to technology, I will automate anything that I don’t like doing (lazy)! So, this is how I automated saving money…

Setting up a Starling Bank Account

If you don’t have a Starling Bank, there are obviously many other banks out there. But I chose Starling due to them providing a usable API. I know Monzo and other banks also offer APIs, so feel free to shop around.

Once I set up the account (and started using it as my main current account), I set up a new space called ‘Penny Challengee’

Starling Developer Account

To access the Starling API, you’ll need a Developer Account. It’s a pretty straight forward signup process. Once you’ve signed up, make sure you link the Developer Account to your Starling Bank account (Settings -> Accounts)

Next, you’ll need to get a Personal Access Token. This is basically your authentication token (AccessToken) when accessing the API.

When choosing the permissions, make sure you select the following:

  • savings-goal-transfer:create

Creating the application

Luckily enough, the developers at Starling provide a nice Developer SDK for you to use out of the box – hosted on GitHub. This was a great place to start.

Using Node, you can easily get started with just the following:

const Starling = require('starling-developer-sdk');

const client = new Starling({
  accessToken: '<INSERT TOP SECRET TOKEN HERRE>',
});

client.account.getAccounts(); // Lists all available accounts.

After playing around for a while, seeing what information I could get from the API. I started to create something which would move money from my current balance into the ‘Penny Challenge Space’.

Here’s the method which performs that move.

client.savingsGoal.addMoneyToSavingsGoal(
    {
      accountUid,
      savingsGoalUid,
      transferUid,
      amount,
      currency: 'GBP',
    },
  )
    .then(({ data }) => console.log(data))
    .catch((err) => console.log(err));

You’ll notice there’s a few parameters which are required.

  • accountUid : The Account UUID (from .getAccounts())
  • savingsGoalUid : The Savings Goal UUID (from .getSavingsGoals())
  • transferUid : Randomly generated UUID (I used uuid package v4)
  • amount : The amount in minor units (1 = 1p, 100 = £1)
  • current : The currency your account is using.

Getting the current day and working out the amount to save

Ok great, using the Starling API I can move money into the ‘Penny Challenge Space’. Now, I need some logic to work out the amount which needs to move, based on the day of the year.

I spent more time than I want to admit creating a method which took the current date, compared it with the 1st January, and returned the number of days in between. It was horrible and hacky, before even mentioning it didn’t take into consideration the fact that it was a leap year… Who knew working with dates is so hard?

So why reinvent the wheel? It turns out momentjs have already figured this issue out and solved it! I’ll use their package…

const moment = require('moment');

const amount = moment().dayOfYear();

This returns a value between 1 – 366. This is perfect as I can use that value as the amount (minor unit). I’ll then pass this to the addMoneyToSavingsGoal() method.

Perfect! That works, it moves the amount into the correct SavingsGoal based on the day of the year. Now I need some way to trigger it to run each day automatically…

Lambda

I decided to use AWS Lambda to trigger the money transfer each day. I could have easily used something like a Cron Job to achieve this. But hey… AWS Provides a Free Tier, so I might as well use it!

Therefore, I created a new Lambda Function. You’ll need to package your node application into a ZIP file and upload it using the AWS-CLI. Here’s a nice guide on how to do that.

Make sure you wrap your code to export the event:

exports.handler = async (event) => {
// Code here
};

CloudWatch

To trigger the Lambda script, you will need to set up a CloudWatch Rule using a Schedule. I personally set a cron expression, but you can also use the ‘Fixed rate of’ setting. Both achieve the same outcome.

So that’s it! Every morning at 9 am, CloudWatch will trigger the Lambda script to move money into the ‘Penny Challenge Space’ in Starling based on the current day of the year. Job done!

Putting it all together

Here’s the code for the app.

Setup Node:

$ npm init
$ npm install starling-developer-sdk
$ npm install uuid
$ npm install moment
$ touch index.js

index.js

const Starling = require('starling-developer-sdk');
const uuidv1 = require('uuid/v4');
const moment = require('moment');

const accountUid = '<UUID for Account>';
const savingsGoalUid = '<UUID for Savings Goal>';

const client = new Starling({
  accessToken: '<TOP SECRET PERSONAL TOKEN>',
});

exports.handler = async (event) => {
  const amount = moment().dayOfYear();
  const transferUid = uuidv1();

  return client.savingsGoal.addMoneyToSavingsGoal(
    {
      accountUid,
      savingsGoalUid,
      transferUid,
      amount,
      currency: 'GBP',
    },
  )
    .then(({ data }) => ({
      statusCode: 200,
      body: JSON.stringify(data),
    }))
    .catch((err) => ({
      statusCode: 500,
      body: JSON.stringify(err),
    }));
};

ZIP & Deploy to AWS Lambda

$ zip functionName.zip .
$ aws lambda update-function-code --function-name MyLambdaFunction --zip-file fileb://functionName.zip

As always, my blog posts are just brain dumps of things I’ve been working on in my spare time. But I hope this is useful for those looking to achieve the same thing.

I have to give credit to Andrew Barber (a fellow Scouter) – as I was scrolling through his GitLab when I came across this ‘1p Challenge’ – He’s written his own version which uses a cron job! We both seem to have gone about solving the problem in a similar way (using moment).

Thanks,

Dano!

Scraping Imgurl.com for images!

Like most of the code I write these days in my spare time, It’s usually for a little bit of fun! This time, I just wanted to get a random bunch of images from Imgurl.com to collect some “memes”. I’ll warn you now, the internet is not a nice place. People upload all sorts of random stuff to Imgurl. Be warned!

Let’s get technical!

Firstly, I decided to use Python 3 on an EC2 micro (AWS virtual server). Secondly, it was written in about 10 minutes. I’m sharing it for future use (if anyone dares find a legitimate reason for using it). Plus I just wanted a reason to embed GitHub’s Gist into my blog!

I use a random string generator function (id_generator) to create a new ID for the imgurl path, then use urllib to create an http request and read the data from that path. I then pass this object which I’ve just read into Pythons Image library and do a comparison on the image size (test.size) and to see if the image is remove.png or not (the image displayed when no image has been uploaded, or the image has been removed). If the image is removed.png, I do not print the results, but move on to another ID. When I find an image that doesn’t match remove.png size, I output the ID, image size and image format.

I added a try/catch to deal with Keyboard Interrupts. You can press CTRL+C and the while loop will exit gracefully (well as gracefully as it can).

Code:

This will output the image IDs which are valid in the console, along with the image size and format. You can then, of course, use the ID as you wish (open them in a browser or embed in HTML etc).

That’s as far as I got, before I realised it was a terrible project and moved onto something else…

Centovacast – Getting Listener Statistics via MySQL

We ran into a little issue with our Centovacast installation last month. It turns out, that if you have a few large radio stations using the same server, the MySQL Database tables get rather full (20 million rows), and when trying to pull the data back into the Centovacast interface was causing some issues (timeout and 500 errors etc). Which ultimately meant our customers could not retrieve the statistics they needed.

So, the only solution was to manually query the tables, and generate our own statistics to provide to our customers. I wrote a little PHP/HTML interface for this, however you can easily do this via an MySQL Client.

Here’s what the customers were requesting, and the SQL queries to get them!

SQL Time!

Replace the accountid=269 in the queries below with the account you wish to get the data for.

You can get the accountid from the acounts table:

SELECT * FROM centovacast.accounts;

# Total sessions

SELECT COUNT(*) AS sessions FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# Total listener seconds

SELECT SUM(duration) as duration FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# Average session length (seconds)

SELECT AVG(duration) as seconds FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# Total data transfer (KB)

SELECT SUM(bandwidth) as bandwidth FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# Average transfer (KB)

SELECT AVG(bandwidth) as bandwidth FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# Unique Listeners

SELECT COUNT(DISTINCT ipaddress) FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# Unique Countries

SELECT COUNT(DISTINCT country) FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;

# ASCAP music sessions

SELECT COUNT(*) FROM centovacast.playbackstats_tracks WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269 GROUP BY DAY(starttime), HOUR(starttime), name;

 

With most of my blog posts, I just write this stuff down for future reference. However I hope it’s helped at least someone in the same situation!

Basic Steam RCON Example (Rust)

After spending time over Christmas coding a tool to query Steam servers for information. I’ve now been taking the next steps… Sending data to a Steam server!

For this example, I’m going to be using a Rust Dedicated Server, to send a simple command, then in future posts show how I sent scheduled commands (such as adverts, messages and other routine tasks which you would expect a Rust server to run).

First things first, after learning the hard way; Don’t run your Rust server with rcon.web = 1. It’s a new protocol, and I found it almost impossible to get working. Maybe if I had more time and patience, I could have cracked it. However, using the old protocol (which I assume is steam’s rcon protocol) is much easier.

 

Here’s the startup command I used when starting the Rust server (notice “rcon.web 0”):

./RustDedicated -batchmode +server.ip 127.0.0.1 +server.port 28015 +server.tickrate 30 +server.hostname "Server Name" +server.identity "rust-server"  +server.maxplayers 250 +server.worldsize 3000 +server.saveinterval 300 +rcon.web 0 +rcon.ip 127.0.0.1 +rcon.port 28016 +rcon.password "Password" -logfile "gamelog-2017-01-22-22-47-05.log"

Code time!

Again I’m using xPaw’s SourceQuery. Here’s a simple example of how to send a command via RCON:

<?php 
require __DIR__ . '/SourceQuery/bootstrap.php'; 
use xPaw\SourceQuery\SourceQuery; 

define( 'SQ_TIMEOUT', 1 ); 
define( 'SQ_ENGINE', SourceQuery::SOURCE ); 

// Init SourceQuery 
$Query = new SourceQuery( ); 

// Connect to the server 
$Query->Connect("<IP>", "<PORT>", SQ_TIMEOUT, SQ_ENGINE);

// Auth
$Query->SetRconPassword( "<Password>" );

// Run the "status" command and return
var_dump($Query->Rcon( "status" ));

// Disconnect
$Query->Disconnect( );

Result!

Which outputs the following:

[email protected]:/var/www/zurk/scripts/rcon# php test.php
string(269) "hostname: Server Name
version : 1955 secure (secure mode enabled, connected to Steam3)
map     : Procedural Map
players : 0 (250 max) (0 queued) (0 joining)

id name ping connected addr owner violation kicks
"

In the next post, I’ll be writing about how I took this simple example, and made a scheduled messaging system. Until then!

Querying Source Servers with PHP : Part 2

So I have my JSON data which contains all the query information which was returned from the Source Query Protocol. I’m storing it as a flat JSON file – the reason being (as discussed in my last post) is simply due to speed (or the assumption it would be faster). I don’t want to be getting into caching database queries, optimising the indexes, normalisation etc… As this project is for fun, let’s just store the JSON on the file system, and pull the data out when we need it.

What should I do with this data?

Well, that’s easy! Let’s create a website, and display it! I admit, I’m no front-end developer. So I chose to use a simple PHP Framework I wrote about 8 years ago. It’s a very simple MVC Framework, which consists of 10 files.

Without going into too much detail (maybe this is a topic for another blog post), here’s the file structure:

/config - Configuration Files

/content - Views

/data - Storage (JSON in this example)

/includes - Controllers

/models - Models

Very simple, but why spend time setting up and configuring something like Laravel or CodeIgnitor, when I can copy/paste my old code, and get cracking? I wrote a controller to loop through my list of servers, pull the data from each of the JSON files, then pass it to the view to be displayed.

CSS & HTML Framework

I decided to use Good old faithful Bootstrap (version 3), and customised some of the basic elements. Again, nothing special, just a quick and easy setup, and throwing together code in minutes!

No npm, grunt, composer, gulp or any other technology! Just raw PHP, CSS and HTML.

Here’s the finished design.

As you can see, I’m displaying information such as the Name, IP, Players and giving the user options to connect (or purchase their own, wink, wink). It’s just a simple “col-md-6” <div>. Nothing fancy, but it means when I add more servers, they can just drop in, and look apart of the page. There’s a few situations, where the divs don’t line up – mainly when there are many players online. However I could resolve this by adding an ellipsis to the players text – one for another time!

What about the Graph?

Ok I added the data used in the graph after my previous post. I simply added integer variable to the foreach loop, which I collate the number of players on each server together. I’m then throwing this data into another flat file with a timestamp.

<datetime>,<number of players>
<datetime>,<number of players>

I then pull this data back in the front end via the controller, and use ChartJS to display it.

<script>
 var ctx = document.getElementById("myChart");
 var data = {
 labels: ["04:20","04:25","04:30","04:35","04:40","04:45","04:50","04:55","05:00","05:05","05:10","05:15","05:20","05:25","05:30","05:35","05:40","05:45","05:50","05:55","06:00","06:05","06:10","06:15","06:20","06:25","06:30","06:35","06:40","06:45","06:50","06:55","07:00","07:05","07:10","07:15","07:20","07:25","07:30","07:35","07:40","07:45","07:50","07:55","08:00","08:05","08:10","08:15","08:20","08:25","08:30","08:35","08:40","08:45","08:50","08:55","09:00","09:05","09:10","09:15",],
 datasets: [
 {
 label: "Players",
 fill: true,
 lineTension: 0,
 backgroundColor: "rgba(187,255,0, 0.09)",
 borderColor: "rgba(187,255,0,1)",
 borderCapStyle: 'butt',
 borderDash: [],
 borderDashOffset: 0.0,
 borderJoinStyle: 'miter',
 pointBorderColor: "rgba(187,255,0,1)",
 pointBackgroundColor: "#fff",
 pointBorderWidth: 0,
 pointHoverRadius: 5,
 pointHoverBackgroundColor: "rgba(187,255,0,1)",
 pointHoverBorderColor: "rgba(187,255,0,1)",
 pointHoverBorderWidth: 2,
 pointRadius: 0,
 pointHitRadius: 10,
 data: [2,2,2,2,2,2,2,2,2,2,4,4,4,4,4,4,4,5,5,8,4,4,4,4,4,4,4,4,5,6,6,6,6,6,6,6,6,4,4,5,5,5,5,5,4,4,5,6,5,4,5,6,7,8,8,8,9,8,8,8],
 spanGaps: false,
 }
 ]
 };
 var myChart = new Chart(ctx, {
 type: 'line',
 data: data,
 options: {
 legend: {
 display: false
 },
 responsive : true,
 scaleShowLabels : false,
 scales: {
 yAxes: [{
 display: true,
 ticks: {
 stepSize: 1,
 beginAtZero: true
 }
 }],
 xAxes: [{
 display: false
 }]
 }
 }
 });
</script> 

As the cron is running every 5 minutes, the graph is showing the number of players, every 5 minutes. Really simple, but a nice addition to the page.

I’d imagine, when I add more servers, I could add options to the graph for different time periods (hour, day, month). This will allow me to see the peak times players are online.

Around this page, I’ve started to build up other pages, and more elements I can use. I’ll leave this blog post here, and maybe continue in another.

It’s been a very fun experience so far, and even though I’m not using any bleeding edge/funky languages or technologies. It’s been nice to refresh my knowledge of Bootstrap, CSS, jQuery, and also learn how Source Query Protocol works!

Here’s where I’ve left the website so far:

Hopefully as time allows, I’ll begin to build it up, and add more pages and servers!

Querying Source Servers with PHP : Part 1

Over the Christmas Period, I decided to develop a website to monitor the multitude of gaming servers that I host. As a game server provider, I’ve often wondered how many players are actively playing across the network.

For this, I decided to use a simple LEMP (Linux, NGINX, MySQL, PHP) environment. Furthermore, I wanted this website to be heavily reliant on JSON data, and would try to avoid pulling data from MySQL where possible – as I’m assuming pulling from a JSON file (using caching), is faster than from MySQL.

The first task was to obtain a list of the game servers on our network. This was quite easy, due to the fact our billing system already contains this information.

I built a JSON string containing an array of the servers:

Example:

{"1":{"id": "1", "protocol":"steam", "game": "Rust","ip":"46.101.5.78", "port": "28015"},"2":{"id": "2", "protocol":"steam", "game": "Rust","ip":"178.62.55.51", "port": "28015"}}

Lovely and simple!

ID: Unique Primary Key
Protocol: Steam
Game: The game!
IP: Servers IP
Port: Servers Port

 

Now, I have my list of servers. Next it to query them for the data! To do this, we need to interrogate the Source Query Protocol. There’s many open source examples of this out there. I found one by xPaw at https://github.com/xPaw/PHP-Source-Query – It’s very well coded class, and rather than reinventing the wheel, I choose to use his class.

So, I create a cron, to query the server list on a predefined interval.

Here’s the code.

It’s a proof of concept, so don’t judge the coding standards!

<?php

require __DIR__ . '/SourceQuery/bootstrap.php';
use xPaw\SourceQuery\SourceQuery;

define('SQ_TIMEOUT', 1);
define('SQ_ENGINE', SourceQuery::SOURCE);

$servers = json_decode(file_get_contents('servers.json'));

foreach ($servers as $server) {
 switch ($server->protocol) {
 case 'steam':
 $data = fetchSteam($server->id, $server->ip, $server->port, $server->game);
 break;
 }
}

function fetchSteam($id, $ip, $port, $game)
{

 $Query = new SourceQuery();

 try {
 $Query->Connect($ip, $port, SQ_TIMEOUT, SQ_ENGINE);

 $data["info"] = $Query->GetInfo();
 $data["info"]["Game"] = $game;
 $data["info"]["IP"] = $ip;
 $data["info"]["Port"] = $port;
 $data["players"] = $Query->GetPlayers();
 $data["rules"] = $Query->GetRules();

 file_put_contents('servers/' . $id . '.json', json_encode($data));
 } catch (Exception $e) {
 echo $e->getMessage();
 } finally {
 $Query->Disconnect();
 }
}

I’m looping through the list of servers, querying each one – then storing the returned data in a separate JSON file (1.json, 2.json etc).

Why the switch statement? Well, not every game server we host is powered by Steam, eventually I plan to add more protocols…

So, what data can we pull back from the Source Query Protocol?

Well, here’s the JSON decoded (pretty format thanks to http://jsonviewer.stack.hu/)

$Query->GetInfo()

$Query->GetPlayers()

$Query->GetRules()

As you can see, we get quite a fair amount of information back from the Source Query Protocol. The only thing I’m struggling to get is the Players SteamID. However, I have no requirement for it, so it’s not a pressing issue – If I wanted to display or track players in more depth across the network, I would need the SteamID.

I’ll leave this blog post here for now, and write Part 2 with the details of what I did next (The interface & graphing).

Shorthand If/Else (ternary) Statements – C#, C++, Python & PHP

I’ve been going through the multiple languages I know recently, and refreshing my knowledge. After working with Python for a few months, I found going back to C# very easy.

I’ve noticed many similarities except for a few; In my opinion the ternary operator slightly differs from each language. So I’d thought I’d write these findings down for future use.

C# & C++

variable = (input > 0) ? 'true' : 'false';

Python

variable = 'true' if input > 0 else 'false'

PHP

$variable = ($input > 0) ? 'true' : 'false';

Knapsack Problem (0-1 solution) – Dynamic Algorithm

I’ve recently dug up old code from my University days, which I thought I’d share for the benefit/misfortune of others.

There’s a common problem in programming called the knapsack problem. Here was my solution based on the dynamic algorithm.

# Math is used to floor/round the floats to an interger and back!
import math
# Total allowed weight
totalWeight = 2392;

# Define the items name, Weight, Profit
items = (("Weapon and Ammunition",    4.13, 1.4),
         ("Water", 2.13, 2.74),
         ("Pith Helmet", 3.03, 1.55),
         ("Sun Cream", 2.26, 0.82),
         ("Tent", 3.69, 2.38),
         ("Flare Gun", 3.45, 2.93),
         ("Olive Oil", 1.09, 1.77),
         ("Firewood", 2.89, 0.53),
         ("Kendal Mint Cake", 1.08, 2.77),
         ("Snake Repellant Spray", 3.23, 4.29),
         ("Bread", 2.29, 2.85),
         ("Pot Noodles", 0.55, 0.34),
         ("Software Engineering Textbook", 2.82, -0.45),
         ("Tinned Food", 2.31, 2.17),
         ("Pork Pie", 1.63, 1.62))

# Get the total of bagged items.
def totalvalue(comb):
    totwt = totval = 0
    for item, wt, val in comb:
        totwt  += wt
        totval += val
    return (totval, -totwt) if totwt <= totalWeight else (0, 0)

# The knapsack 0-1 Dynamic Programming Algorithm.
def knapsack(items, limit):
    table = [[0 for w in range(limit + 1)] for j in xrange(len(items) + 1)]

    for j in xrange(1, len(items) + 1):
        item, wt, val = items[j-1]
        wt = int(wt * 100)
        val = int(val * 100)
        for w in xrange(1, limit + 1):
            if wt > w:
                table[j][w] = table[j-1][w]
            else:
                table[j][w] = max(table[j-1][w],
                                  table[j-1][w-wt] + val)

    result = []
    w = limit
    for j in range(len(items), 0, -1):
        was_added = table[j][w] != table[j-1][w]

        if was_added:
            item, wt, val = items[j-1]
            wt = int(wt * 100)
            val = int(val * 100)
            result.append(items[j-1])
            w -= wt

    return result

# Bag the items
bagged = knapsack(items, totalWeight)

# Print the results
print("Bagged the following items\n  " +
      '\n  '.join(sorted(item for item,_,_ in bagged)))
val, wt = totalvalue(bagged)

print("Total Value: %f - Total Weight: %f" % (float(val), float(-wt)))