Like most of the code I write these days in my spare time, It’s usually for a little bit of fun! This time, I just wanted to get a random bunch of images from Imgurl.com to collect some “memes”. I’ll warn you now, the internet is not a nice place. People upload all sorts of random stuff to Imgurl. Be warned!
Let’s get technical!
Firstly, I decided to use Python 3 on an EC2 micro (AWS virtual server). Secondly, it was written in about 10 minutes. I’m sharing it for future use (if anyone dares find a legitimate reason for using it). Plus I just wanted a reason to embed GitHub’s Gist into my blog!
I use a random string generator function (id_generator) to create a new ID for the imgurl path, then use urllib to create an http request and read the data from that path. I then pass this object which I’ve just read into Pythons Image library and do a comparison on the image size (test.size) and to see if the image is remove.png or not (the image displayed when no image has been uploaded, or the image has been removed). If the image is removed.png, I do not print the results, but move on to another ID. When I find an image that doesn’t match remove.png size, I output the ID, image size and image format.
I added a try/catch to deal with Keyboard Interrupts. You can press CTRL+C and the while loop will exit gracefully (well as gracefully as it can).
This will output the image IDs which are valid in the console, along with the image size and format. You can then, of course, use the ID as you wish (open them in a browser or embed in HTML etc).
That’s as far as I got, before I realised it was a terrible project and moved onto something else…
We ran into a little issue with our Centovacast installation last month. It turns out, that if you have a few large radio stations using the same server, the MySQL Database tables get rather full (20 million rows), and when trying to pull the data back into the Centovacast interface was causing some issues (timeout and 500 errors etc). Which ultimately meant our customers could not retrieve the statistics they needed.
So, the only solution was to manually query the tables, and generate our own statistics to provide to our customers. I wrote a little PHP/HTML interface for this, however you can easily do this via an MySQL Client.
Here’s what the customers were requesting, and the SQL queries to get them!
Replace the accountid=269 in the queries below with the account you wish to get the data for.
You can get the accountid from the acounts table:
SELECT * FROM centovacast.accounts;
# Total sessions
SELECT COUNT(*) AS sessions FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# Total listener seconds
SELECT SUM(duration) as duration FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# Average session length (seconds)
SELECT AVG(duration) as seconds FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# Total data transfer (KB)
SELECT SUM(bandwidth) as bandwidth FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# Average transfer (KB)
SELECT AVG(bandwidth) as bandwidth FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# Unique Listeners
SELECT COUNT(DISTINCT ipaddress) FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# Unique Countries
SELECT COUNT(DISTINCT country) FROM visitorstats_sessions WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269;
# ASCAP music sessions
SELECT COUNT(*) FROM centovacast.playbackstats_tracks WHERE starttime > '2017-12-25' AND starttime < '2017-12-31' AND accountid=269 GROUP BY DAY(starttime), HOUR(starttime), name;
With most of my blog posts, I just write this stuff down for future reference. However I hope it’s helped at least someone in the same situation!
After spending time over Christmas coding a tool to query Steam servers for information. I’ve now been taking the next steps… Sending data to a Steam server!
For this example, I’m going to be using a Rust Dedicated Server, to send a simple command, then in future posts show how I sent scheduled commands (such as adverts, messages and other routine tasks which you would expect a Rust server to run).
First things first, after learning the hard way; Don’t run your Rust server with rcon.web = 1. It’s a new protocol, and I found it almost impossible to get working. Maybe if I had more time and patience, I could have cracked it. However, using the old protocol (which I assume is steam’s rcon protocol) is much easier.
Here’s the startup command I used when starting the Rust server (notice “rcon.web 0”):
So I have my JSON data which contains all the query information which was returned from the Source Query Protocol. I’m storing it as a flat JSON file – the reason being (as discussed in my last post) is simply due to speed (or the assumption it would be faster). I don’t want to be getting into caching database queries, optimising the indexes, normalisation etc… As this project is for fun, let’s just store the JSON on the file system, and pull the data out when we need it.
What should I do with this data?
Well, that’s easy! Let’s create a website, and display it! I admit, I’m no front-end developer. So I chose to use a simple PHP Framework I wrote about 8 years ago. It’s a very simple MVC Framework, which consists of 10 files.
Without going into too much detail (maybe this is a topic for another blog post), here’s the file structure:
/config - Configuration Files
/content - Views
/data - Storage (JSON in this example)
/includes - Controllers
/models - Models
Very simple, but why spend time setting up and configuring something like Laravel or CodeIgnitor, when I can copy/paste my old code, and get cracking? I wrote a controller to loop through my list of servers, pull the data from each of the JSON files, then pass it to the view to be displayed.
CSS & HTML Framework
I decided to use Good old faithful Bootstrap (version 3), and customised some of the basic elements. Again, nothing special, just a quick and easy setup, and throwing together code in minutes!
No npm, grunt, composer, gulp or any other technology! Just raw PHP, CSS and HTML.
Here’s the finished design.
As you can see, I’m displaying information such as the Name, IP, Players and giving the user options to connect (or purchase their own, wink, wink). It’s just a simple “col-md-6” <div>. Nothing fancy, but it means when I add more servers, they can just drop in, and look apart of the page. There’s a few situations, where the divs don’t line up – mainly when there are many players online. However I could resolve this by adding an ellipsis to the players text – one for another time!
What about the Graph?
Ok I added the data used in the graph after my previous post. I simply added integer variable to the foreach loop, which I collate the number of players on each server together. I’m then throwing this data into another flat file with a timestamp.
<datetime>,<number of players>
<datetime>,<number of players>
I then pull this data back in the front end via the controller, and use ChartJS to display it.
As the cron is running every 5 minutes, the graph is showing the number of players, every 5 minutes. Really simple, but a nice addition to the page.
I’d imagine, when I add more servers, I could add options to the graph for different time periods (hour, day, month). This will allow me to see the peak times players are online.
Around this page, I’ve started to build up other pages, and more elements I can use. I’ll leave this blog post here, and maybe continue in another.
It’s been a very fun experience so far, and even though I’m not using any bleeding edge/funky languages or technologies. It’s been nice to refresh my knowledge of Bootstrap, CSS, jQuery, and also learn how Source Query Protocol works!
Here’s where I’ve left the website so far:
Hopefully as time allows, I’ll begin to build it up, and add more pages and servers!
Over the Christmas Period, I decided to develop a website to monitor the multitude of gaming servers that I host. As a game server provider, I’ve often wondered how many players are actively playing across the network.
For this, I decided to use a simple LEMP (Linux, NGINX, MySQL, PHP) environment. Furthermore, I wanted this website to be heavily reliant on JSON data, and would try to avoid pulling data from MySQL where possible – as I’m assuming pulling from a JSON file (using caching), is faster than from MySQL.
The first task was to obtain a list of the game servers on our network. This was quite easy, due to the fact our billing system already contains this information.
I built a JSON string containing an array of the servers:
ID: Unique Primary Key Protocol: Steam Game: The game! IP: Servers IP Port: Servers Port
Now, I have my list of servers. Next it to query them for the data! To do this, we need to interrogate the Source Query Protocol. There’s many open source examples of this out there. I found one by xPaw at https://github.com/xPaw/PHP-Source-Query – It’s very well coded class, and rather than reinventing the wheel, I choose to use his class.
So, I create a cron, to query the server list on a predefined interval.
Here’s the code.
It’s a proof of concept, so don’t judge the coding standards!
As you can see, we get quite a fair amount of information back from the Source Query Protocol. The only thing I’m struggling to get is the Players SteamID. However, I have no requirement for it, so it’s not a pressing issue – If I wanted to display or track players in more depth across the network, I would need the SteamID.
I’ll leave this blog post here for now, and write Part 2 with the details of what I did next (The interface & graphing).