Home » Computing » Archive by category "Programming"

The “Bite Me” Minetest Server

For those who do not know, Minetest is an open-source free-to-play and free-to-modify alternative to Minecraft. It’s Free as in Freedom – and as in Free Lunch.

For the past few days I had been running a Minetest server – it was fun whilst it lasted, but I have been extremely busy with work and it turns out I need to liberate that server node for something more productive… so for now, Bite Me, and its villages, are defunct.

Intent

Originally the idea was to run a Minetest server with a difference – during the week, an easy setup would have allowed players to create whatever they wanted in the world; settle villages and such, without any aggressive mobs or PvP.

On Friday nights however the world would be backed up, and NSSM would be turned on, PvP would be enabled, and random protector blocks would be deleted.

On Monday, the world would be reverted to its state as registered on Friday.

It would have been oh so fun…

I did keep a backup of the world data though, and all tools I had written to monitor and manage the server. It’ll be back online some day, but with a little bit more pre-prep; I’ll post some of the tools on my github page eventually, including the items allowing swithcing between the week-day safe mode, and the week-end massacre mode…..

Specs

For those of you who are curious about stats and requirements, I was running this on a Ubuntu 16.04 server with 512 MB RAM and 1 GB swap, 20 GB storage (more than enough) and a single CPU at 2.3 GHz, courtesy of digitalOcean.

At peak, I think I had about 10 players all playing simultaneously, with a good few off exploring different caves and causing the map gen to work in several locations siultaneously, and the server was handling fairly well I believe.

I expect if you want to run a properly specced server, 2GB RAM, 2 CPUs and 2GB swap would be a better bet. I might use that in future.

Examples

Here’s some screenshots from when I was exploring as admin:

Exhibit 1 – some weird shadows from the clouds. These shadows were persistent (never changed location), and dark enough that stone monsters would spawn in them….! You can see a htop report showing server stats too

Exhibit 2 – I was using maikerumine’s esmobs mod to generate some difficult mobs. For some reason, they would hardly ever spawn in my main village – but they did everywhere else and my goodness were they a handful… see the log how much damage they would have been doing if I hadn’t my admin shield equipped! (probably from 3d-Armor mod, made easily accessible from the Unified Inventory)

Exhibit 3 – Sokomine’s mg and mg_villages mods, combined with VanessaE’s moretrees mod produce some superb settings…

Mods

Here’s the full list of mods I was using:

https://github.com/minetest-mods/biome_lib
https://github.com/minetest-mods/moretrees
https://github.com/minetest-mods/moreores
https://github.com/minetest-mods/homedecor_modpack
https://github.com/minetest-mods/stamina
https://github.com/minetest-mods/city_block
https://github.com/Sokomine/locks
https://github.com/Sokomine/travelnet
https://github.com/PilzAdam/nether
https://github.com/stujones11/minetest-3d_armor
https://github.com/minetest-technic/unified_inventory
https://github.com/dmonty2/minetest-u_skinsdb
https://github.com/tenplus1/protector
https://github.com/Uberi/MineTest-WorldEdit
https://github.com/Sokomine/mg
https://github.com/Sokomine/mg_villages
https://github.com/Sokomine/cottages
https://github.com/tenplus1/farming
https://github.com/minetest-mods/playeranim
https://github.com/BlockMen/cme
https://github.com/maikerumine/esmobs
https://github.com/NPXcoot/nssm
https://github.com/Sokomine/mobf_trader

Posted on May 29, 2016 by Tai Kedzierski, Reply

The Power of #!/bin/bash

The pictured snippet is probably one of the dirtier pieces of code I have had the misfortune of needing – and not being able to refactor to anything cleaner.

I am trying to make bash my main programming language, and to this goal I have created a number of tools to aid me on my way: a library of code snippets that anybody can re-use, a packing tool to create executables, a pre-compilation tool to add compiler directives to bash, a build tool to pull all these together, and a make/release tool to manage versions. And I’m not yet done.

Of the main efforts here is the library of scriptlets.

Once of the main attractions of one language over the other is the availability of a large amount of re-usable code released as libraries and which are, pretty much, taken for granted.

Python can get modules from pip, there are Java libraries in JAR files around the web and even JavaScript has such libraries as jQuery to help to write terser, more manageable code more easily.

bash has none of this. And quite a few quirks. You have to contend with the very string-y way of passing data around (arrays are a bit of a nightmare until you get used to the arcane notations), and nearly every non-trivial operation you want to do is a command, an external process.

The inset image depicts some code I wrote so that assigning variables

But that is also exactly what makes so appealing – any language, and its associated libraries, can become your library!

For starters, python, perl and PHP can all be leveraged for their respective strengths in specific situations, and it is possible to write and store additional scripts such as long MySQL tasks in their own files. Use a perl script to write change on the fly, and pipe to MySQL.

Consider the following silly example:

controlapp -getusers |
    perl alter_sql.pl "$SCENARIO/mysqltemplate.sql" |
    mysql -u "$sqluser" -p"$sqlpass" "$mydb" |
    php to_xml.php |
    controlapp -dostuff

It looks quite awful at first glance, but consider the power of perl to perform text processing, keeping your mysql files ordered in a directory-based hierarchy, and passing the result to PHP which will handle outputting XML much more ergonomically than PHP.

So whilst bash does have its odd and at times even infuriating quirks, I am learning to love it.

It is now after all the de facto language of DevOps. It seems even Windows can no longer do without it 🙂

Posted on April 26, 2016 by Tai Kedzierski, Reply

Installing SliTaz GNU/Linux

Recently I’ve been playing with SliTaz GNU/Linux, a tiny distro that can be made to operate even the oldest of PCs!

This article is a short bootstrap to get you started with the essentials on SliTaz 4.0

What is SliTaz?

SliTaz is an independant GNU/Linux distribution, with its own package manager, repositories and kernel, focused on providing an OS for embedded devices and computers with extremely small amounts of power and storage.

It is extremely lightweight: its standard ISO is about 35 MiB in size, botting from cold to desktop in as fast as 15 seconds, and ab initio takes up 32 MiB RAM with the Gtk openbox desktop and no extra applications.

Whilst it can be used as a lightweight desktop environment, its main application would more likely be for use as

an embedded Linux
an SSH gateway
an easily reloaded web kiosk
a portable PC troubleshooting/rescuing disk
such uses where slick features are shunned in favour of lightness and efficiency.

A GUI desktop environment is provided for those who are afraid of being in full-on command line mode, but to maintain its lightness, there are no traces of such heavy packages as LibreOffice or Firefox.

Out of the box you get

the lightweight Leafpad text editor (if you’re not content with nano or vi!)
the Midori web browser
the sakura terminal emulator
and mtPaint if you need to edit pictures…

and apart from that, not excessively more. That’s all you really need.

There’s a web-based GUI manager running on localhost for managing the computer, but make no mistake – this systems is more appropriate for seasoned Linux hobbyists who are OK filling in the documentation gaps…

There is even an Raspberry Pi version of SliTaz available to squeeze the most performance out of your Pi.

GUI install

On the LiveCD, to configure SliTaz, boot into the “Gtk” version; then open the web browser go to http://tazpanel:82 and enter the root login details. By default, the user is “root” and the password is “root”.

Once in TazPanel, you can manage the system, upgrade packages, install new software – an more!

Go to the final menu entry labelled “Install” and choose to “Install SliTaz”

For the purposes of this guide, we are just going to do a very simple install. If you’re comfortable with partitioning, go wild.

You will be offered the opoortunity to launch GParted – click that button. You will be shown a map of the first hard drive. If you have multiple hard drives, BE CAREFUL with which one you choose – the operation we are about to perform will erase the disk you operate on.

Choose the disk in the disk chooser menu in the top right – probably /dev/sda if you have only one disk. Again CHOOSE WISELY.

Once a disk is chosen wisely, use the Device menu and choose to Create Partition Table

Then choose Partition menu: New partition

Leave the defaults and hit OK, then click the Apply button to apply the changes. At this point the disk is erased and a new partition is set up. You have been warned!

Exit GParted, scroll down and choose to “Continue installation”

Most options should be fairly self explanatory.

Under Hard Disk Drive, choose the drive you just erased in GParted (/dev/sda for example) and choose to format the partition as “ext4”

Set the host name to whatever you want.

Change the root password to something sensible.

Set the default user as desired.

Remember to tick “Install Grub bootloader” or you’ll have a non-bootable system…

Click “Proceed to SliTaz installation”. After a few seconds… SliTaz is installed. Reboot!

You’ll have to set up your locale and keyboard just once more and voila, a desktop Linux that boots in seconds!

Command line install

Here’s the simple recipe for installing SliTaz from the command line. Note that even if started from the LiveCD headless, this install will install a full GUI and take up around 100MB of space.

The first thing to know is that the installer is invoked by the command tazinst.

The second thing to know is that you need to create a configuration file for it.

The third thing you need to know is that you need to partition your disk first. Naturally, this is what we’ll do first.

WARNING – PARTITIONING ERASES THE DISK THAT YOU ARE PARTITIONING

Type these keys in order to erase /dev/sda and set up a single partition. If you have never done this before…. read up on partitioning with fdisk. It’s a whole topic on its own! Hit return for each new line of course.

fdisk -l
fdisk /dev/sda
o
n
p
1
1
 (just hit return here to accept the default)
a
1
w

Great, you have a new partition on /dev/sda1. Now create the config file.

tazinst new configfile

Once you have created the configuration file, edit it.

vi configfile

Three key things you need to change are as follows:

TGT_PARTITION – the partition you will be installing on – in our case, /dev/sda1 or whichever you configured earlier
TGT_FS – the file system you want to use – for example, ext4
TGT_GRUB – “yes” unless you intend on installing Grub manually afterwards.

Finally, run

tazinst install configfile

After a few second, the install will be finished and you can reboot.

Post-install customizations

SliTaz is very light. Extremely light. You might even say it’s missing some packages you would expect as standard. You should think about doing some initial setup…

su
tazpkg -gi vim
tazpkg -gi htop
tazpkg -gi tmux
tazpkg -gi sudo
tazpkg -gi iptables # ...and whatever else you want...
#one tazpkg per item to install

/etc/init.d/dropbear start # SSH server
 vim /etc/rcS.conf

# add dropbear SSH server to startup
 %RUN_DAEMONS=" ...dropbear"

vim /boot/grub/menu.lst

# change timeout
 %timeout 2

visudo
 # add your own users to sudo location

And that’s about it. Some extra commands that may be different from what you may know from elsewhere:

poweroff # instead of shutdown

tazpkg recharge # sync package list
tazpkg info (package)
tazpkg description (package)
tazpkg search (string)
tazpkg get-install (package name) # install from repo
tazpkg get (package name) # download from repo
tazpkg install (TGZ file) # install from local file

Bonus – tpgi

Instead of directly using the restrictive tazpkg, try using my wrapper 🙂

Switch to root and run the following

tazpkg -gi git
tazpkg -gi bash
git clone git://github.com/taikedz/handy-scripts
cp handy-scripts/bin/tpgi /usr/bin/

This will set up the tpgi command which you can use to make life with tazpkg a little easier… run the command without arguments for help. Try:

tpgi install htop vim tmux sudo

Now you can install multiple packages from one line….!

tpgi search and gcc 3.4

Searches for packages containing the term “gcc” then filters the results for version 3.4

Posted on May 13, 2015 by Tai Kedzierski, Reply

Watcher-RSS : Your own, Personal, Feeder

I finally got around to putting together some initial code for that thing I wanted – a script to detect changes in a page and produce an RSS entry as an outcome.

“watcher-rss” is just that – a simple script that can be called by cron to check a page for a significant area, and generate an RSS “feed” in response.

It’s designed such that you need to define a bash handler script that sets the required variables; after that it can generate an RSS entry in response to anything. Read more

Posted on December 12, 2014 by Tai Kedzierski, Reply

Checking for a Mac resource fork

Following a mishap with a backup some time ago in which I lost the contents of my text clippings, I have started investigating resource fork detection for my rsync pre-processor.

The pre-processor will eventually be python-based, with a couple of extra scripts for platform-specific operations that arise. I need it to be able to backup using rsync or some Windows backup program with equivalent functions, to a FAT32 target, without any data loss.

Pre-processing the backup source directory includes:

split files greater than 4 GB into individual chunks – for backing up to FAT32 drives
- mark chunks for use in restore process
- use an exclusion for file sizes greater than 4 GB
- remove chunks post sync
OS X specific – detect files with resource forks
- determine if a child of a grouped directory, if so bubble up to grouped directory and tar-gz
- else tar-gz the file itself
- add the original file/directory to rsync exclusion list
- remove the tar-gz post-sync
build a database of current tree state – to “detect” moved files and reduce transfer time
- build hardlink repository of all files in the source directory
- replace previous state database with current tree state and remove hardlink repo post-sync

From the restoration side, re-building the chunks and unpacking the tar’d files need to be handled.

For detecting resource forks on Mac, this script should work:

MYFILE="$1/..namedfork/rsrc"
if [ "old"$(sw_vers | grep -Po "10\.[0-6]\.") = "old" ]; then
    MYFILE="$1/rsrc" # different format pre-10.7
fi

FOUNDRES=yes$(ls -l "$MYFILE" | grep -Po "0\s+[0-9]+\s+[a-zA-Z]");
# resource fork found to be zero length - returns string

# FOUNDRES is exactly "yes" if resource is non-zero
if [ "$FOUNDRES" = "yes" ]; then
    echo "RSRC present on $MYFILE"
else
    echo "RSRC absent from $MYFILE"
fi

Use at will.

Posted on June 9, 2014 by Tai Kedzierski, Reply

Sony Walkman MTP workarounds…

This post answers the questions:

What is MTP (Media Transfer Protocol)?
How can I manage files on my media device by simply copying files?
How do I manage a MTP USB device through the file manager on Linux?
How do I use jmptfs?

So… I’ve just gone and bought a little Sony Walkman Series B and was looking forward to trying it out on my Xubuntu 14.04 install with Clementine.

The Walkman Series B comes with an integrated USB connector for transfer and recharging, which I plugged into a USB 3.0 port. Alas, rather than using a common interface such as the block file system, the device uses the Media Transfer Protocol only interface.

Media Transfer Protocol is a way that a device such as a phone, camera or music player can show itself to the computer, and generally requires special software to add/remove files to. To wit:

Problem #1 – after double-clicking the desktop icon, it took a few seconds to mount, during which I double-clicked it again (thinking my taps on the touchpad may have been too light…) and was presented with a “could not mount” error. Ouch. But it mounted after that.

Problem #2 – it was not mounted as a block device (read: regular hard disk) but as a MTP transfer. Maybe it’s Thunar, or maybe it’s an inherent limitation in the protocol, but this did not allow me to move/add/delete files at all.

Problem #3 – After firing up Clementine, it seems the device was detected as practically full. I had a number of Clementine crashes over my various attempts at copying files from my library over as well. Ghastly procedure.

Workaround – after doing a little bit of reading up, I found that the most seamless solution was to mount the device as a block file system by way of an added library called “jmtpfs” installed following the incantation

sudo apt-get install jmtpfs

Now you can use the command line to mount and unmount the first MTP device found using various commands. I opted to create a script to do this for me: I created a script in my ~/bin directory (which is already in my path) called “mmtpfs” containing:

#! /bin/bash

MTPMAIN=~/mmtpfs
MTPDIR=$MTPMAIN/$2

if [ "${1}k" = '-mk' ]; then
    mkdir -p $MTPDIR
    jmtpfs $MTPDIR
elif [ "${1}k" = '-uk' ]; then
    fusermount -u $MTPDIR
    rmdir $MTPDIR # only removes end directory if empty
    rmdir $MTPMAIN
else
    echo Help:
    cat << EOT
Mount the first MTP device to ~/media/MOUNTDIR

Only specify the name of a directory as MOUNTDIR

Mount device:
    $0 -m MOUNTDIR

Unount device:
    $0 -u MOUNTDIR
EOT
fi

Now I can just open the terminal and type

mmtpfs -m sony

To mount the device or

mmtpfs -u sony

to unmount the device. I might make a Zenity script and a .desktop entry to make it even easier to manage should I have to add this to a user’s configuration.

Posted on May 7, 2014 by Tai Kedzierski, Reply

About that: getting out of walled gardens by using Blockchain?

ReadWrite is runnning a piece touting Blockchain as the panacea to solving the problem of Walled Gardens (because these in themselves are somehow stifling innovation).

The article does a poor job as far as I can tell, from reading it and from seeing the comments, of linking the two aspects, and I had to read a bit further to understand why this is potentially a game changer. Personally, I’m not sure it is. Below is the comment I added to the article:

Blockchain is a protocol that ensures identification and integreity of a piece of data and its iteration in time.

Apps are created and delivered in a walled garden. Where’s the connection?

I did go and read some of the linked articles and I think I can see where this was meant to be going:

The idea is that we want to develop apps that can leverage the power of “App+Could” type systems – that is, having an app that accesses your data online – without seeing your data being siloed into that company’s server forever more.

For example, a notepad app, with lots of your notes in it, accessible from all of your devices (desktop, laptop, tablet, phone, watch, fridge [hey, you might have a shopping list note]). Your data is stored somewhere online, on some company’s cloud infrastructure, and all these apps query that company’s server.

Now if your data is in a peer-to-peer store, using a blockchain-encoded transaction to modify it, then its data is also distributed around the Internet, with changes in the data recognized and stored by the network. I guess the goal is to move the data to a truly disembodied cloud, and not just pigeon holes.

So you change a line in your shopping list on your fridge. The entire network validates this, changes the corresponding peered data, and that change is spread, verifying against the blockchain for the latest status. Your data is stored peer to peer, potentially anonymized and encrypted (ha!), and is resilient to deletion.

However we still have walled gardens where the actual apps have restrictions on their on-device functionality, and we cann’t necessarily control what servers they actually talk to. The reason apps are more interesting than mobile browsers is the interaction with the hardware on the device.

So it’s an interesting start, but so long as there are practical advantages of using an app, we won’t escaped walled gardens on mobile devices.

In the mean time, if you’re really concerned about walling, try open source, try full computers, and try the real world. We’re all over here making stuff. Innovating freely.

Posted on April 15, 2014 by Tai Kedzierski, Reply

About that: why are we still writing programs using text-based languages?

PCPro.co.uk has an article on graphical programming

Programming is a fairly specialist activity, requiring a different manner of thinking to operate in. Some programming languages try to be friendly in terms of words, symbols and grammar used to write the code (syntax) – but it still tends to be an initial hurdle to get past.

An asker on Slashdot had the following question today:

“…why are we still writing text based code? Shouldn’t there be a simpler, more robust way to translate an algorithm into something a computer can understand? One that’s language agnostic and without all the cryptic jargon? It seems we’re still only one layer of abstraction from assembly code. Why have graphical code generators that could seemingly open coding to the masses gone nowhere?”

The /. editor added:

Of interest on this topic, a thoughtful look at some of the ways that visual programming is often talked about.

Here are my thoughts:

For the TL/DR: graphical programming is inefficient, and error prone; better methods of viewing source code during read-back is more interesting.

Posted on February 8, 2014 by Tai Kedzierski, Reply

Gaining the first popularity

After being led to the Ada Programming Language Wikipedia entry (and being impressed by what it offers and promises – it is designed for fail-safeness and robustness, and as such is used in satellites, military jets and orbital satellites), I wondered, why isn’t the language more popular, like the less effective and more error-prone C and C++ which are used for coding our operating systems, or Java/PHP/Ruby which power our interwebz?

A few forum posts exist about this, and numerous explanations abound, from petty whining (“everybody who tries it it hates it”) to accusatory (“it’s designed by a committee, and like Esperanto, nobody wants to use that”) through conspiracy theory (“it’s in the interests of repair & diagnostic/IDE vendors to make sure crappy languages persist”) and even via circular reasoning (“it’s not gained popularity because other languages are more in demand”). I kid you not.

In all of that, I only once noted one answer which hit a sensible mark: in a time when there wasn’t a prominent single language, due to the closedness of computer systems, hobbyists could only work with what was lying around, and there was no global applicability in any real sense [citation needed…], C came along and offered what nobody else was: a high-level language for low-level computing, a free compiler, and an operating system to use it on, in an ecosystem that promoted (at the time) code sharing. It was this seed traction that led to C being still the most popular language for system programming today.

Other languages have since risen, but the most popular, with the largest communities, still are application-level and web-oriented – not system level. In the midst of this, Ada still retains (bless or curse) its original image as the brainchild of the US Department of Defense’s brainchild, with little push to make it a business-systems or high-availability systems contender.

The onus then is on the existing Ada community to start building tools and libraries for newcomers to use. Newcomers like to feel invited in by pioneers.

Posted on June 18, 2013 by Tai Kedzierski, Reply

Scrambling for notepads