Blog

These started out as notes for me regarding hard to find stuff on the Internet. I have now added things that I found interesting or thought provoking. 

Mick

All your base are belong to us.


Ubuntu Removing Unwanted Kernels

If you have ever struggled with Ubuntu Server and the /boot/ partition being filled up as a result of ‘apt upgrades’ then I’m with you. 

For me until recently the process I use is as follows:

sudo apt update
sudo apt autoremove
sudo apt upgrade -y

Whilst autoremove gets rid of old Linux kernels, it will try to purge all but the latest two versions. However it won’t remove anything from version currently running. 

Previously I took the approach of running apt upgrade -y before the (reboot then) purge, but I have tended to run out of disk space in /boot/ due to the number of updates to the operating system. Hence adding apt autoremove before hand.

Whilst have a mooch around the system I discovered that the following file, controls which versions of Linux are permitted to be purged, during autoremove:

    /etc/apt/apt.conf.d/01autoremove-kernels

The file should not be manually edited, it is maintained by calling:

   sudo /etc/kernel/postinst.d/apt-auto-removal

So, whilst it doesn’t solve all the problems with disk space and running apt. The process I use for updating the system is as follows:

sudo apt update
sudo apt autoremove -y
sudo apt update -y
.. if all goes well and reboot the system ..
sudo /etc/kernel/postinst.d/apt-auto-removal
sudo apt autoremove -y

This leaves the system running from the latest version of Linux, and only the previous one left in the /boot/ partition.

One last thing, following the apt autoremove, sometimes I get asked to reboot the system one final time. I’m guessing that this is some final housekeeping that the system thinks it needs.

I hope that this has made some sense, and you’ve gained from it.

Okay Pipe Drums - Awesome

Studying and spotted this on Youtube … it’s incredible this is all coming from this guy and his pipes !!



Addictive.. 

There are others. Just Google 'playing music with pipes’.

Enjoy…


Updated; this appears to be the guys homepage: https://pipeguy.bandcamp.com/ 

Installing MySQL Server as a Docker image

For my PhD project I have elected to deploy a Mysql server to store data, but I don’t want to install it directly on my Mac. Various reasons for this but mainly as I need to be able remove it cleanly when I finished. I’ve used Docker before as it allows service daemons to be deployed and remove very easily - no worries about the correct versions of supporting libraries … bliss 

Unless you’ve got Docker installed on your computer then, you'll need to download and install it. A lot has happened, politically, since I originally did this so I’m just going to refer to the download location that I tend to use: https://www.docker.com/community-edition

Follow your normal processes to download and install the software, and ensure that it is running.

When installing new containers; you ask Docker to run them, it checks to see if there is a local image and if not it downloads it. It then starts the image on your host computer. 

When installing the MySQL Server Docker I would not recommend running on the default port (3306) as it may get in the way if either you do install a full MySQL server on your computer or deploy multiple instances. In this example I will use port 6603 as I can easily remember it, I think that the default port used by the Docker installer, is 33060. 

To install I used the following:

$ docker run —name mysql -e MYSQL_USER=user -e MYSQL_PASSWORD=pass \
    -e MYSQL_DATABASE=db1 -p 6603:3306 -d mysql/mysql-server

This will download the latest Docker image for mysql/mysql-server, into the local repository. It will start up the instance, create the database 'db1' and setup the user 'user' with a password of 'pass’. To access the server you need to either install the mysql client software, or a client application, and connect to TCP port 6603 (or whatever you selected, on ‘localhost' or ‘127.0.0.1’. 

If you have SQL scripts I would suggest installing the command line version of MySQL client. That was my approach, but I would also suggest looking at an application, as they tend to let you tinker with the databases in a more intuitive way.

My preferred application is 'Sequel Pro’ - it was easy to get used to and has the best combination of capabilities. That said, I did stuggle with creating SSL/TLS connections … 


Python3 Setup on OSX/MacOS High Sierra

By default MacOS has Python 2.7 installed out of the box, however for my PhD project I am using a number of libraries that require Python 3.x.x. There are numerous approaches to deploy multiple versions of Python to MacOS, so I’ve documented the approach I took.

A big thanks to Chris Mendez, as my approach is based on his article: Installing Multiple versions of Python on Mac using Home-brew

I already had Homebrew installed, but incase you haven’t there is a tutorial on Installing Homebrew for Mac. I’ve tried a number of approaches using VirtualEnv and found that PyEnv seems to do the job much better. The steps as follows:

brew install pyenv
echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
. ~/.bash_profile

I then looked for the lastest version of python (version 3.6.5) and installed it using PyEnv, and switched the local and global environmenal settings to point to it.

pyenv install 3.6.5

The next problem was to install all the modules currently loaded on the MacOS provided version of Python (version 2.7.10) the local and global environmenal settings to point to it. So before switching to the new version, I used 'pip' to extract the list to a file, '/tmp/2.7-module.list’. I then edited the file, to remove any version 2 specific modules, also I removed 'MySQL-python’ as it has been replaced with a nice shiny pure python MySQL client library ‘PyMySQL’, and I’d had problems with it.

pip list > /tmp/2.7-module.list
- edit the file to remove unwanted modules -
pyenv global 3.6.5

for l in `cat /tmp/2.7-modules.list | awk '{ print $1 }'`;do
 pip install $l;
done

Make note of which modules were installed sucessfuly and if there are any errors.

Just a quick note on ‘PyMySQL’. I have installed both client and server on my Mac, so I did’t experence library dependencies. The reason was that I'd previously had great fun trying to install ‘MySQL-python’ without MySQL server installed locally, and in the end gave and installed it. For this reason I’m guessing I didn’t have any problems installing ‘PyMySQL’.

I ran into a problem and originaly needed to use 'sudo pip' to install modules for the system version of Python, as the libriaries were installed in the system library. This is no longer required with PyEnv as all the libraries as installed under ‘~/.pyenv/‘ tree.

Quick reference:

pyenv versions         - lists the versions under its control
pyenv global system - switches back to the system version (2.7.10 for my system)
pyenv global 3.6.5    - switches to version 3.6.5


A big thanks to the authors of the software I’ve used. 

Python    - https://www.python.org/
PyEnv     - https://github.com/pyenv/pyenv
PyMySQL - https://github.com/PyMySQL/PyMySQL

mod_rewrite in .htaccess

In the process of migrating the website I needed to catch old URLs and map them to the locations of the documents on the new website. Deploy a .htaccess file in the root of the website and introduce Redirect 301’s.

Simples…

Well for the static mappings yes.

e.g. 

Redirect 301 /2009/ /blog/

However to avoid having to implement lines for every single permutation mod_write allows to regex matching and re-writing.

Firstly it is neccessary to enable mod_rewrite which for me on Ubuntu means:

sudo a2enmod rewrite
sudo service apache2 restart

Don’t forget to insert ‘AllowOverride All’ in the '<Directory nn>' section. Then it is possible to add the following to the .htaccess file.

e.g. 

RewriteEngine on
RewriteRule ^20[01][0-9]/$ /blog/ [R=301]

In this case the important thing to remember is that the initial slash for the webpage is removed, so don’t match on ‘^/blah/$’ but instead ‘^blah/$’. This took a little while to figure out what was going on when I first implemented this. Also logging has changed since installed a web server from scratch, ‘RewriteLog’ has been depricated, you must now use ‘LogLevel alert rewrite:trace3’ or whichever level you require. Which can be applied to the '<Directory nn>' section.


More details can be found on the apache website:


ErrorDocument Pages and Sandvox

An iritating problem with Sandvox is that is doesn’t appear to allow a page to be setup with absolute references pointing at the local site. This causes a problem when create DocumentError pages for Apache, because the current references for a page aren’t changed with an error is generated.

For example:

  • Assume ErrorDocument 404 /404_errorpage.html in the .htaccess file, causing the page /404_errorpage.html to be opened in the event of a page not found error.
  • If a page /non_existent_folder/non_existent_page.html/non_existent_folder/ is selected by the browser, causing a 404 error to be generated. All relative references in the error page, would contain /non_existent_folder/ before the relative reference.


This means that if a style sheet or page menu links don’t have an absolute reference then they’re all broken.

Sandvox only appears to provide relative links in any page it controls. So either you create static pages, and manually maintain them or …  well I created a clunky workaround.

HTML has a handy capability to allow a page to be automatically refreshed to an alternative location, using a meta tag that is placed in the header of a HTML file. 

For example:

<html>
<head>
<meta http-equiv=“refresh” content=“0; URL=/redirect_page.html”>
</head>
</html>

Note: just in case the browser doesn’t interpret this tag, it is advisable to include a <body> with some useful information.


So the fix for my problem with Sandvox is to create a jump page for each of the error conditions that need to be trapped. This jump page is then redirects to the full URL of the associated page controlled by Sandvox. Hey presto all the links work. 

As I said it was clunky but it does appear to work.

Site moved from GoDaddy

With the aim of pulling my research efforts together I have moved my website away from a Wordpress site hosted on GoDaddy to a flat HTML site hosted on a Virtual Private Hosting. I’m currently using Sandvox to author this site, and whilst quite quirky (and buggy) I’ve managed to pull this site together. I’ve looked at other software and so far it is doing what is required. I will keep looking for other software but for now I’ll stick to this one.

Limited edn Millennium Falcon

I wandered into the LEGO shop at Disney Springs and they’ve got a limited edition Millennium Falcon (LEGO part code 75192) for $799.99. It is like rocking horse droppings and considerably more expensive in the Uk. I asked one of the shop assistants about shipping to the Uk, they politely declined and suggested bringing it through as hand luggage … this box is absolutely massive!

IMG_4313


 

And near the exit to the shop, in a glass display case, they have one built up … I think this is the largest LEGO kit I have found to date. I think the Deathstar (LEGO part code 75159) is smaller.

IMG_4317


Too big for my collection but I can dream a little...


Erwin Schrodinger lived here 1940 - 1956

I was taking a walk along a riverside path heading toward St Pancras in London and spotted this plaque.

IMG_4226


I was definately there ...


© Mick Vaites 2018