Arduino Libraries uploaded to Github

I was introduced to blink and the Arduino ecosystem a few years ago, by a friend who was, at that time, building an engine management system for his kit car. Since then he’s moved on to building a car in a box, which is based on a Peugeot, and provides a platform that can be used to hack the multitude of computers that now make up a car. He has made extensive use of Arduino microprocessors to simulate and detect CAN bus messages in his pursuit.

In my earlier life I have developed realtime systems, and since my introduction to Arduino, it has often proven easier to simulate what I want to do, with an Espressif ESP8266 or ESP32 using the Arduino IDE, than to deploy to a RaspberryPi. (And ESP32 uses FreeRtos).

I have always meant to upload and share the code I write, the problem has always been to make it useful before doing so, which takes time.  Building code as part of a complex system, or hacking something together is not very useful.

I have now started uploading code from a couple of projects; Solar, Home Automation and something I’m currently looking at, managing a Bitcoin AntMiner.

Projects

ESP32_GetSolaxRealtimeData – monitors a Solax Inverter and controls remote power switches.

ESP8266_TimerPowerSwitch – a decentralised power switch system, using the SONOFF basic hardware (ESP8266).

Arduino Classes

SolaxInverter – monitors a Solax inverter.

AntminerS9 – monitors the status of an AntMiner S9.

 

 

Posted in Diary, Projects | Comments Off on Arduino Libraries uploaded to Github

Pipe Guy first person view

Posted in Music, Pipe Guy | Comments Off on Pipe Guy first person view

Flush DNS cache on MacOS

For most versions of OSX and macOS I have found the following works

sudo killall -HUP mDNSResponder

I have seen notes to indicate that 

dnscacheutil -flushcache 

or

discoveryutil udnsflushcaches

are needed for  Yosemite and older versions.

Thanks to: https://help.dreamhost.com/hc/en-us/articles/214981288-Flushing-your-DNS-cache-in-Mac-OS-X-and-Linux amongst other sites.

Posted in Apple, Hints and Tips | Comments Off on Flush DNS cache on MacOS

All your base are belong to us!

Posted in Stuff | Comments Off on All your base are belong to us!

Ubuntu Removing unwanted kernels

If you have ever struggled with Ubuntu Server and the /boot/partition being filled up as a result of ‘apt upgrades’ then I’m with you. 

For me until recently the process I use is as follows:

sudo apt update
sudo apt autoremove
sudo apt upgrade -y

Whilst autoremove gets rid of old Linux kernels, it will try to purge all but the latest two versions. However it won’t remove anything from version currently running. 

Previously I took the approach of running apt upgrade -y before the (reboot then) purge, but I have tended to run out of disk space in /boot/ due to the number of updates to the operating system. Hence adding apt autoremove before hand.

Whilst have a mooch around the system I discovered that the following file, controls which versions of Linux are permitted to be purged, during autoremove:

    /etc/apt/apt.conf.d/01autoremove-kernels

The file should not be manually edited, it is maintained by calling:

   sudo /etc/kernel/postinst.d/apt-auto-removal

So, whilst it doesn’t solve all the problems with disk space and running apt. The process I use for updating the system is as follows:

sudo apt update
sudo apt autoremove -y
sudo apt update -y
.. if all goes well and reboot the system ..
sudo /etc/kernel/postinst.d/apt-auto-removal
sudo apt autoremove -y

This leaves the system running from the latest version of Linux, and only the previous one left in the /boot/ partition.

One last thing, following the apt autoremove, sometimes I get asked to reboot the system one final time. I’m guessing that this is some final housekeeping that the system thinks it needs.

I hope that this has made some sense, and you’ve gained from it.

Posted in Hints and Tips, Linux | Comments Off on Ubuntu Removing unwanted kernels

Installing MySQL Server as a Docker image

For my PhD project I have elected to deploy a Mysql server to store data, but I don’t want to install it directly on my Mac. Various reasons for this but mainly as I need to be able remove it cleanly when I finished. I’ve used Docker before as it allows service daemons to be deployed and remove very easily – no worries about the correct versions of supporting libraries … bliss 

Unless you’ve got Docker installed on your computer then, you’ll need to download and install it. A lot has happened, politically, since I originally did this so I’m just going to refer to the download location that I tend to use: https://www.docker.com/community-edition

Follow your normal processes to download and install the software, and ensure that it is running.

When installing new containers; you ask Docker to run them, it checks to see if there is a local image and if not it downloads it. It then starts the image on your host computer. 

When installing the MySQL Server Docker I would not recommend running on the default port (3306) as it may get in the way if either you do install a full MySQL server on your computer or deploy multiple instances. In this example I will use port 6603 as I can easily remember it, I think that the default port used by the Docker installer, is 33060. 

To install I used the following:

$ docker run —name mysql -e MYSQL_USER=user -e MYSQL_PASSWORD=pass \
    -e MYSQL_DATABASE=db1 -p 6603:3306 -d mysql/mysql-server

This will download the latest Docker image for mysql/mysql-server, into the local repository. It will start up the instance, create the database ‘db1‘ and setup the user ‘user‘ with a password of ‘pass’. To access the server you need to either install the mysql client software, or a client application, and connect to TCP port 6603 (or whatever you selected, on ‘localhost‘ or ‘127.0.0.1’. 

If you have SQL scripts I would suggest installing the command line version of MySQL client. That was my approach, but I would also suggest looking at an application, as they tend to let you tinker with the databases in a more intuitive way.

My preferred application is ‘Sequel Pro’ – it was easy to get used to and has the best combination of capabilities. That said, I did stuggle with creating SSL/TLS connections … 

Update: Sequel Pro has no support for ‘caching_sha2_password.so’, which appears to be the default in MySQL 8.0 and onwards. I have now switched to using TablePlus which does not have this restriction.

Posted in Apple, Diary, PhD | Comments Off on Installing MySQL Server as a Docker image

Python3 setup on OSX/MacOS High Sierra

By default MacOS has Python 2.7 installed out of the box, however for my PhD project I am using a number of libraries that require Python 3.x.x. There are numerous approaches to deploy multiple versions of Python to MacOS, so I’ve documented the approach I took.

A big thanks to Chris Mendez, as my approach is based on his article: Installing Multiple versions of Python on Mac using Home-brew

I already had Homebrew installed, but incase you haven’t there is a tutorial on Installing Homebrew for Mac. I’ve tried a number of approaches using VirtualEnv and found that PyEnv seems to do the job much better. The steps as follows:

brew install pyenv
echo ‘eval “$(pyenv init -)”‘ >> ~/.bash_profile
. ~/.bash_profile

I then looked for the lastest version of python (version 3.6.5) and installed it using PyEnv, and switched the local and global environmenal settings to point to it.

pyenv install 3.6.5

The next problem was to install all the modules currently loaded on the MacOS provided version of Python (version 2.7.10) the local and global environmenal settings to point to it. So before switching to the new version, I used ‘pip‘ to extract the list to a file, ‘/tmp/2.7-module.list’. I then edited the file, to remove any version 2 specific modules, also I removed ‘MySQL-python’ as it has been replaced with a nice shiny pure python MySQL client library ‘PyMySQL’, and I’d had problems with it.

pip list > /tmp/2.7-module.list
– edit the file to remove unwanted modules –
pyenv global 3.6.5

for l in `cat /tmp/2.7-modules.list | awk ‘{ print $1 }’`;do
 pip install $l;
done

Make note of which modules were installed sucessfuly and if there are any errors.

Just a quick note on ‘PyMySQL’. I have installed both client and server on my Mac, so I did’t experence library dependencies. The reason was that I’d previously had great fun trying to install ‘MySQL-python’ without MySQL server installed locally, and in the end gave and installed it. For this reason I’m guessing I didn’t have any problems installing ‘PyMySQL’.

I ran into a problem and originaly needed to use ‘sudo pip‘ to install modules for the system version of Python, as the libriaries were installed in the system library. This is no longer required with PyEnv as all the libraries as installed under ‘~/.pyenv/‘ tree.

Quick reference:

pyenv versions         – lists the versions under its control
pyenv global system – switches back to the system version (2.7.10 for my system)
pyenv global 3.6.5    – switches to version 3.6.5

A big thanks to the authors of the software I’ve used. 

Python    – https://www.python.org/
PyEnv     – https://github.com/pyenv/pyenv
PyMySQL – https://github.com/PyMySQL/PyMySQL

Posted in Apple, PhD | Comments Off on Python3 setup on OSX/MacOS High Sierra

ErrorDocument pages and Sandvox

An iritating problem with Sandvox is that is doesn’t appear to allow a page to be setup with absolute references pointing at the local site.

This causes a problem when create DocumentError pages for Apache, because the current references for a page aren’t changed with an error is generated.

For example:

  • Assume ErrorDocument 404 /404_errorpage.html in the .htaccess file, causing the page /404_errorpage.html to be opened in the event of a page not found error.
  • If a page /non_existent_folder/non_existent_page.html/non_existent_folder/ is selected by the browser, causing a 404 error to be generated. All relative references in the error page, would contain /non_existent_folder/ before the relative reference.

This means that if a style sheet or page menu links don’t have an absolute reference then they’re all broken.

Sandvox only appears to provide relative links in any page it controls. So either you create static pages, and manually maintain them or …  well I created a clunky workaround.

HTML has a handy capability to allow a page to be automatically refreshed to an alternative location, using a meta tag that is placed in the header of a HTML file. 

For example:

<html>
<head>
<meta http-equiv=“refresh” content=“0; URL=/redirect_page.html”>
</head>
</html>

Note: just in case the browser doesn’t interpret this tag, it is advisable to include a <body> with some useful information.

So the fix for my problem with Sandvox is to create a jump page for each of the error conditions that need to be trapped. This jump page is then redirects to the full URL of the associated page controlled by Sandvox. Hey presto all the links work. 

As I said it was clunky but it does appear to work.

Posted in Apache, Apple, Diary | Comments Off on ErrorDocument pages and Sandvox

mod_rewrite in .htaccess

In the process of migrating the website I needed to catch old URLs and map them to the locations of the documents on the new website. Deploy a .htaccess file in the root of the website and introduce Redirect 301’s.

Simples…

Well for the static mappings yes.

e.g. 

Redirect 301 /2009/ /blog/

However to avoid having to implement lines for every single permutation mod_write allows to regex matching and re-writing.

Firstly it is neccessary to enable mod_rewrite which for me on Ubuntu means:

sudo a2enmod rewrite
sudo service apache2 restart

Don’t forget to insert ‘AllowOverride All’ in the ‘<Directory nn>’section. Then it is possible to add the following to the .htaccess file.

e.g. 

RewriteEngine on
RewriteRule ^20[01][0-9]/$ /blog/ [R=301]

In this case the important thing to remember is that the initial slash for the webpage is removed, so don’t match on ‘^/blah/$’ but instead ‘^blah/$’. This took a little while to figure out what was going on when I first implemented this. Also logging has changed since installed a web server from scratch, ‘RewriteLog’ has been depricated, you must now use ‘LogLevel alert rewrite:trace3’ or whichever level you require. Which can be applied to the ‘<Directory nn>’ section.

More details can be found on the apache website:

Posted in Apache, Hints and Tips | Comments Off on mod_rewrite in .htaccess

Pipe Drums – Awsome

Studying and spotted this on Youtube … it’s incredible this is all coming from this guy and his pipes !!

This is totally wicked …

.. and there are others.

Just Google ‘playing music with pipes’

Updated; this appears to be the guys homepage: https://pipeguy.bandcamp.com/

Posted in Music, Pipe Guy | Comments Off on Pipe Drums – Awsome