Accessing an AWS EC2 Instance via Macfusion

As part of a drive to improve security we are moving more of our AWS EC2 development instances behind a VPN. This is fine for most things but the code editor we use, ShiftEdit, won’t see the servers behind the VPN as they are not publicly assessable over SSH. This means I was looking for a different solution. Ideally I wanted a code editor that had native SSH support using private keys or agent forwarding and had a tabbed interface. Unfortunately none of major players seemed to have what I was looking for.

There are a number of free editors that have sprung up recently including Adobe Brackets, Microsoft’s Code and Atom from GitHub. I really like Atom and I noticed that there were some plug-ins that purportedly supported SSH so I gave a couple of those a go but none seemed to work very well. One connected but wouldn’t show any files and the other wouldn’t connect at all.

Then I stubbled upon MacFuse, a Google project that allows you to mount various file systems as virtual drives on a Mac. This sounded ideal but the project had been discontinued a while ago but a quick search led me to one that was spawned from MacFuse that is still supported – FUSE for OSX.

It took me a while to get my head around how to connect FUSE for OSX to an EC2 instance so here is how I did it.

Installing the Required Components

First of all download and install FUSE for OSX from here: https://osxfuse.github.io/. Installation is mainly a case of clicking through the installer but when you get to the option make sure that you select “MacFUSE Compatibility Layer”:

Install_FUSE_for_OS_X__OSXFUSE__and_FAQ_·_osxfuse_osxfuse_Wiki

Next download and install Macfusion from here: http://macfusionapp.org/. Then start Macfusion.

Configuring Macfusion

The first time you run you will see the following screen. Click Start and select the option to start at Login.

Macfusion_and_Evernote_Premium

Next click the + at the bottom to add a new destination and select SSHFS from the menu:

New_filesystems_and_Macfusion_and_Some_Answers_-_Google_Groups_🔊

Now enter your instance details:

Macfusion_and_Some_Answers_-_Google_Groups_🔊

It isn’t necessary to make any changes on the other tabs.

Making the Link to Your Key

Before you can connect to your instance you have to let Macfusion know where your key is held. To do this you need to create a text file called “config” in ~/.ssh. Into this file you need to put the following (changing “your-instance-key.pem” for the name of your key file).

IdentityFile ~/.ssh/your-instance-key.pem

You can add as many keys here as you need.

Connecting to Your Instance

Now you have everything set-up you can go back to Macfusion and click on the Mount button:

Macfusion_and_Evernote_Premium

All being well the icon should turn green and say Mounted, as follows:

Macfusion_and_Evernote_Premium

Viewing Your Files

Finally go to  Finder and when you look at your mounted devices you should see your SSH drive. When you double click it will list the file on the remote drive and you can use them just as you would any other local file.

Neil’s_iMac

I was able to select a folder as a project in Atom and edit just as if the files were local.

When you are done unmount the drive.

It’s all relatively straight forward, let me know how you get on.

Bringing Handwriting to the 21st Century

Over the years I have had all manner of tablets and writing devices but I have never come across one that did the trick for me. Most styluses either have a big fat end which is fine for replicating your finger but no good for emulating handwriting. Or there are the more “active” styluses that only work with certain apps, or have calibration issues, or have terrible palm rejection. In most cases they suffered from all three issues.

And then along came the Apple Pencil

I have to admit that I was pretty skeptical about how accurate the Pencil would be after so many missed attempts in the past and the claims made by Apple were, frankly, ludicrous based on my prior experiences. “Highly responsive. Virtually no lag.”, “Draw lines of any weight. Just apply pressure.” and most unbelievably “Go ahead, rest your palm on the display.”

But it really is all true. The Pencil is super accurate with no discernible lag whatsoever meaning that it is suitable for handwriting and it seems to work in every application so all those notebook apps you have are now properly usable. Better still is that the palm rejection really does work you can lean on the iPad and write and only the writing shows up.

It’s not all sunshine and roses though. You pay a lot for that level of accuracy and whether it is worth it will depend on your workflow. Also you are going to want to find a way of not losing that expensive Pencil because Apple have provided no smart (or dumb for that matter) way of attaching it to the iPad so you need to carry it round in a pocket or get one of these great holders from Quiver. Either way it is an expensive loss so it is slightly disappointing that there isn’t a better solution from Apple themselves.

So, in conclusion, ff you like writing or drawing and are happy to do so on your glassed screen iPad then the Apple Pencil is for you.

Pros

  • Very accurate
  • no palm rest problems

Cons

  • Typical Apple pricing
  • No way to attach it to the iPad
  • iPad Pro only

Adding MicroSD Storage to a Laptop

I have, amongst other things, an Acer Chromebook which I really love but it is a bit tight on storage at 16GB. I know that Google wants me to store everything in the cloud but that just isn’t possible when you are disconnected and want to watch a movie.

My requirement was to have something with a reasonable amount of storage but didn’t stick out too far from the machine so I could leave it connected all the time. I somehow stumbled upon this
Micro SD USB Card Reader adaptor which pretty much does what it says on the tin.

I shoved in an existing 64gb micro SD card that I had and put it into my laptop and was able to play movies and music without any issues and not dropping or lag. I know that there are USB drives which are similar in size and capacity but if you have a Micro SD card kicking around unused then this could be just what you are looking for.

UPDATE! SInce I wrote this I notice that 7dayshop is selling this 32gb card complete with a USB 3.0 card reader for only £12.99. So if you want to try this out now might be the time.

wh2-lsdmi32gbbeu633r_01

Simple Bug Tracking with Evernote

I have been starting on a side project with my son this week where I am developing a small web app for him. He has left home a while ago so we needed to find a way to collaborate on the project without too much effort and without interfering with our day jobs. So, for example, we went through the requirements while travelling on a train up to London.

When it comes to testing and bug tracking we again needed something quick and simple. In the past I have used both The Bug Genie for work and Trello for smaller side projects but both seemed too much for this project. This is when I decided to use Evernote which I have had a long love affair with.

For this to work I created a series of tags as follows. I created Bug and Enhancement for the ticket types and High, Medium and Low for the ticket severity. Finally I created one further tag called Fixed. Now when a note is created we can add the relevant tags along with a title and a description of the issue.

To retrieve the tickets we can make use of Evernote’s search grammar. So to find all notes that are in the ProjectX notebook that are enhancements of high priority but not fixed we could search for the following:

Notebook:ProjectX Tag:Enhancement Tag:High -Tag:Fixed

Similarly to find all bugs that have raised in the last couple of days we would search for:

Notebook:ProjectX Tag:Bug -Tag:Fixed created:day-2

Once you have completed a search you can save it for quicker access in future. On a Mac you can find this on the Edit > Find menu as shown below. For other platforms see here.

Save a Search on Evernote

When you click in the search field the save searches appear below the search history:

Evernote_Premium

Finally you can also drag the search from the drop down list to the Shortcuts for even faster access.

A simple way to keep a track of issues on our side project.

Pebble Health Review

As I have said a couple of times before I use a Fitbit to record my steps and have toyed with Misfit on my Pebble when that became available. My issue was that the gap in number of steps recorded was too great. So when Pebble Health was released I was keen to see whether this might finally be a replacement for my faithful Fitbit.

Pebble Health is a native watch app that does all the same step and sleep tracking as other fitness trackers such as Fitbit and the on-Pebble Misfit. Difference being that as this is developed by Pebble themselves it looks and feels very much like the rest of the system with the quirky graphics which I really like.

How does it compare?

Once again I ran the Thompson’s completely unscientific test to see how close the step counting was between the Fitbit flex, Misfit (as an app on the Pebble) and Pebble Health. Pebble don’t recommend running multiple fitness trackers on the watch simultaneously but in my testing this didn’t make any difference either to the steps counted or to the battery life.

The graph below shows the number of steps counted over approximately a one month period and, as you can see, they all followed the same trend but the total number of step recorded was highest on the Fitbit, followed by Pebble Health and bringing up the rear, Misfit. So in terms of reaching my daily goal of 10,000 steps it is definitely easier to do with the FitBit.

Initially I was surprised that there was a difference between Pebble Health and Misfit given that they are using the same accelerometer but apparently each software has its own algorithm that is used to calculate the steps.

Pebble_vs_Misfit_vs_Fitbit_-_Google_Sheets

The Health app on the watch has several useful screens including showing you the number of steps today and a comparison against what is typical, another showing the last weeks steps which you can step through (pun intended) to get the total for each day. There are also screens for sleep tracking but given I don’t use that I can’t add much.

One of the great features of the Fitbit is that when you meet your tracking goal it lights up like a Christmas tree and I get a warm glow inside. The Misfit app on the Pebble also displays a trophy when you meet your goal. Pebble Health? Nothing. Nada. Occasionally I get some words of encouragement but I can only remember it happening three times in the whole time I have been using Pebble Health and that’s a shame.

Get me out of here

One of the things I like about the Fitbit is that I can connect it to other services and record my data and keep it forever. So I send data to Exist.io which looks for correlations with other data sent to it and to Evernote as a permanent record. With Pebble Health the data is (currently) locked tightly away on the watch and it is (difficult) to get out. The caveats in the previous statement are because there is an api and with a separate watch app you can send the data to a third party source. It works but it is a pain to do as you have to remember to select the watch app to send the data and that isn’t going to happen. No doubt at some point someone will develop an app that syncs the data to a third party source (hopefully Pebble themselves) but right now it is too much of a Frankenstein solution to work reliably.

Conclusions

A health app developed by Pebble itself is an exciting prospect and on the watch itself it works well but, for me, it still doesn’t offer everything that I want in order for it to replace my Fitbit but that day is edging closer.

Pros

  • looks and feels like the rest of the watch app
  • designed and built by Pebble so should be maintained going forward

Cons

  • algorithm records lower steps than Fitbit so have to do more to reach 10,000 each day (you might see this as an encouragement)
  • only very occasionally see the messages of encouragement or message telling me I have reached my goal
  • data locked on device

 

Raspberry Pi GPS tracker – Connecting Without a Network

All articles to date have been about getting the GPS working with the Raspberry Pi and converting the original Python code to PHP. All of this works well (for me) but what if you want to make changes to the settings or download the log file while you are out and about and away from your home network?

Normally what happens is that you configure your device to connect to a known network but as you are going to be using the GPS/Pi combination away from known networks you need a way to access anywhere without having to connect a screen and keyboard to the Pi. To do this you need to configure peer-to-peer networking. What this means is that you are creating a machine to machine connection without the router that would traditionally be required. This allows you to connect in any way you would over a normal network, so over http or ssh, for example.

To setup peer-to-peer networking you need to be able to connect your Pi to the internet to install the software that you will need. The rest of this post assumes that you have already done this but if not see here to connect a wireless dongle or hook up a USB Ethernet dongle.

NOTE: If you complete these steps your Pi won’t have internet access at the end over wifi. If you want to connect again you will have to change the interface back or connect over ethernet.

Install a DHCP Server

So first we have to install a DHCP server which will be used to allocate an ip address in the same range as the Pi to any device that connects to it. To do this you first need to update apt and then do the install itself:

sudo apt-get update
sudo apt-get install isc-dhcp-server

The last few lines of the DHCP install should look as follows:

Generating /etc/default/isc-dhcp-server...
Job for isc-dhcp-server.service failed. See 'systemctl status isc-dhcp-server.service' and 'journalctl -xn' for details.
invoke-rc.d: initscript isc-dhcp-server, action "start" failed.
Processing triggers for systemd (215-17+deb8u2) ...

The service has failed to start as you need to setup the config file as follows. Edit the dhcp.conf file that has been created:

sudo nano /etc/dhcp/dhcpd.conf

The file will have a lot in it already, mostly commented out, but we are going to replace it all with the following which will allocate addresses in the range .10 – .100 and .150 – .200:

default-lease-time 600;
max-lease-time 7200;
authoratative;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.1.255;
option routers 192.168.1.254;
option domain-name-servers 192.168.1.1, 192.168.1.2;
option domain-name "mydomain.example";

subnet 192.168.1.0 netmask 255.255.255.0 {
 range 192.168.1.10 192.168.1.100;
 range 192.168.1.150 192.168.1.200;
}

Once you save the file you should now be able to start the DHCP service successfully:

sudo service isc-dhcp-server start

Change the wifi interface

Now we need to change the interfaces file to give the Pi a static IP address in the same range as the DHCP service is serving:

sudo nano /etc/network/interfaces

If you have already setup wireless networking then you will have already made changes to this file. We are going to replace the contents of this file so it’s a good idea to take a backup first. Then replace the content with the following:

auto lo
iface lo inet loopback
iface eth0 inet dhcp

auto wlan0
iface wlan0 inet static
  address 192.168.1.1
  netmask 255.255.255.0
  wireless-channel 1
  wireless-essid RPiAdHocNetwork2
  wireless-mode ad-hoc

The value you give wireless-essid will be what is shown when you search for the network so you can change it to something more memorable if you wish.

For the changes to take effect you need to restart the network service but the easiest way to achieve this is to restart the Pi itself:

sudo reboot now

Now when you search for wireless networks you should see your Pi network appear similar to as shown below on an iPhone:

2016-02-24 19.52.21

You can select and use this exactly as you would any other network so, for example, here I have connected over SSH to my Pi and, if you have installed it, you could also access a web browser.

unspecified

If you complete all the above you will be able to access your Pi no matter where you are and make changes on the fly without having to change connection details for each wifi network you want to connect to. In a later post I will show how you can use of this alongside the GPS.

Raspberry Pi GPS tracker – Converting Code to PHP – Part 2

In the last post I looked at converting the original Python code to PHP. This all ran without issue but I quickly found that because the Pi wasn’t connected to the internet the date and time of the device never got updated. This meant that the log files always had the wrong timestamp when they were created making it difficult to find the one I needed.

Turns out that there is a simple answer to this problem. As the GPS satellites include the current date and time as part of the detail that is sent along with the location we can extract this information and then update the Pi using the Linux date command. The code for this is below.

    $resp = explode(",", $line);
    if ($first_time){
        // set the pi time
        $dt = substr($resp[9],4,2).'-'.substr($resp[9],2,2).'-'.substr($resp[9],0,2).' '.
              substr($resp[1],0,2).':'.substr($resp[1],2,2).':'.substr($resp[1],4,2);
        if ($set_time) exec('date -s "'.$dt.'"');
        $first_time = FALSE;
    }

You only need to do this once on start-up hence the $first_time boolean.

This is part of the code, the rest of which you can find on my Github page here.

Next up I will be looking at how you can access your Raspberry Pi from another device without having to connect it to the Internet.

Raspberry Pi GPS tracker – Converting Code to PHP – Part 1

Last week I looked at getting the hardware up and running for a Raspberry Pi GPS tracker. However, as I said I was using some Python code and I don’t speak Python so I wanted to convert it to PHP. So this week I am going to look at what I did.

Before I could even begin to look at the converting the code I had to see if it was even possible for PHP to access the serial port. Turns out this is exactly what Direct IO (dio) is for but it isn’t included as standard so you have to download and install as follows:

First you need the prerequisites which are pear and php5-dev which is required for compiling additional modules:

sudo apt-get install php-pear php5-dev -y

Next download and compile dio:

sudo pecl install channel://pecl.php.net/dio-0.0.7

This will show the following output:

Build process completed successfully
Installing '/usr/lib/php5/20131226/dio.so'
install ok: channel://pecl.php.net/dio-0.0.7
configuration option "php_ini" is not set to php.ini location
You should add "extension=dio.so" to php.ini

Pay particular attention to the last line!

Reading from the serial port is pretty easy and very similar to reading from a file, so to open the port you do the following:

if ( !$t = fopen('dio.serial:///dev/ttyACM0','r+b',false, $c) )  echo 'Failed to open ttyACM0';

Then to read from the port:

$line=fgets($t,1024);

In our case we want to continuously cycle reading the port looking for valid GPS location records.

// cycle round forever looking for the correct GPS record
while(true) {
    // get a record from the GPS
    $line=fgets($t,1024);
    // we are only interested in GPMRC records
    if (!empty($line)){
        if ( strpos($line,"GPRMC")){
            // break up the string
            $resp = explode(",", $line);
            // we are only interested in valid GPS records
            if ($resp[2] == "A") {
                $lat_decimal = degrees_to_decimal($resp[3], $resp[4]);
                $lon_decimal = degrees_to_decimal($resp[5], $resp[6]);
                file_put_contents($logs.date('Ymd').'-simple-log.txt', $resp[9].",".$resp[1].",".$lat_decimal.",".$lon_decimal.PHP_EOL,FILE_APPEND);
            }
        }
    }
}

Rather than reproduce all the code here which will become out of date as I improve it I have published it to Github here.

Next time I will look at some interesting quirks of this and how to improve the code’s usefulness.

Raspberry Pi GPS tracker – Getting it Together

Having secured a Pi Zero from the cover of a the magazine MagPi I thought I would start out simple by trying one of the projects shown there. The one that caught my eye and required no soldering was to build a GPS tracker.

Getting hold of a cheap USB unit was pretty easy but make sure it is Linux compatible. I bought this one from eBay.

The first thing I did when I got the unit was to try it out on the Pi and while it was working I could see that it had got a satellite fix the record containing the actual location wasn’t populated and showed as invalid.

Satellite fixes in Ublox U-center software

Satellite fixes in Ublox U-center software

Assuming that it was a an issue with the unit I downloaded ublox software on a Windows machine and tried from there. Again the same thing. So I contacted the company that sold me the unit and they came back with this very full explanation.

Invalid records in Ublox U-center software

Invalid records in Ublox U-center software

The first time these devices are powered up in a new location, they firstly locate as many satellite signals as possible, then download the almanac data (future predicted satellite positions), whilst trying to calculate the current position.

If the signals aren’t strong enough, the satellites can be seen and identified by number, however the position cannot be calculated and the almanac data cannot be downloaded.

When I test all of these modules, I do so outside with a laptop, a clear view of the sky, and a hot mug of coffee (as each module can take 5-15mins the first time depending on satellite positions).

The device will likely output the current satellite time with a signal which is much weaker than the signal level required to get a location fix.

Whilst the device may give a satellite lock indoors after the initial location fix, you are unlikely to get a first location fix at all indoors, believe me I’ve tried this without much success on cold days to avoid going outside, ultimately they need a clear view of the sky to work.

So I put the kettle on and set off outside with the GPS unit once again connected to the Pi. Almost immediately it got a fix and the green light started to blink. Success!

The next step was putting it all together with some code to record the GPS records as I moved around. For this I used the recommended Python script, GPS Experimentation, which was easy to setup and configure. Unfortunately I don’t speak Python so one of my first tasks will be to see if I can re-write it in PHP.

And so I set off out with my Heath Robinson GPS tracker. As you can see from the images below the combined battery, wires and Pi resembles a cinematic bomb so I spent quite a bit of time searching for the perfect container in which to house it. In the end Poundland came up trumps with a box of screen wipes and it was cheap too (a pound, obviously).

The first surprise was that the battery,  a freebie 2000mAh usb power bank, lasted a full 10 hours just driving the Pi and GPS unit. That was seriously impressive. The second was just how accurate the tracker was when out in the open. As you can see when I plotted the data in GPSPrune.

A meander around Reading Town

A meander around Reading Town

However, there were times when things were not all that they seemed. The image below shows the hour I spent sat in a coffee shop working and highlights the small variations on GPS location that are returned. I think one improvement I will make to the code is to stop tracking when the speed is zero, or very close to it.

An hour sat in the same place

An hour sat in the same place

And at one point the tracker lost the plot completely and decided that I was in France which led to this anomaly!

A day trip to France

A day trip to France

 

On the whole this was an easy and fun project to put together and all relatively cheap. The next step is to convert the code to PHP and start to make some improvements. Check back next week for an update.