Visualizing Data

We need a way of presenting the temperature data that has been saved to the database in the last tutorial. This tutorial shows how we can create a simple representation of the data and is only intended to as a starting point for the display of data and can be taken much further. For the purposes of this course our data display techniques will remain simple and could become more advanced if you were to take another course like the Making Data Interactive INTD319 course. In the mean time, enjoy this simple effort.

The Code

The following code is a PHP file that we can place on our Raspberry Pi’s web server (in the /var/www directory) that will allow us to view all of the results that have been recorded in the temperature directory on a graph.

Download the following s_temp.php.zip file and decompress it.

Place the file in a folder in your var/www/ directory and view the file by typing localhost/folder_name/s_temp.php. If all goes well, you should be able to see a graph that will look a little like this (except your data will be different).


Simple Line Graph of Temperature

VNC Resolution

By now you’ve probably tried to connect to your Pi with VNC. In VNC Viewer on your laptop the resolution of the your Pi is set low by default. To change this default we’ll need to change a setting in raspi-config.
To use raspi-config, access Terminal and run raspi-config. Then navigate to Advanced Options > Resolution. Change the setting to Mode 16 1024 x 768.

Recording Data

To record data from a sensor we will use a Python program that checks the sensor every minute and writes the temperature (with a time stamp) into our MySQL database.

Database preparation

First we will set up our database table that will store our data.

Using the phpMyAdmin web interface (http://localhost/phpmyadmin) that we set up, log on using the administrator (root) account and create a database called ‘measurements’.

Enter in the name of the table and the number of columns that we are going to use for our measured values. In the screenshot above we can see that the name of the table is ‘temperature’ (how imaginative) and the number of columns is ‘2’.

We will use two columns so that we can store a temperature reading and the time it was recorded.

Once we click on ‘Go’ we are presented with a list of options to configure our table’s columns. Don’t be intimidated by the number of options that are presented, we are going to keep the process as simple as practical.

For the first column we can enter the name of the ‘Column’ as ‘dtg’ (short for date time group) the ‘Type’ as ‘TIMESTAMP’ and the ‘Default’ value as ‘CURRENT_TIMESTAMP’. For the second column we will enter the name ‘temperature’ and the ‘Type’ is ‘FLOAT’ (we won’t use a default value).

Configure the MySQL Table Columns

Scroll down a little and click on the ‘Save’ button and we’re done.

Why did we choose those particular settings for our table?

Our ‘dtg’ column needs to store a value of time that includes the date and the time, so either of the types ‘TIMESTAMP’ or ‘DATETIME’ would be suitable. Either of them stores the time in the format ‘YYYY-MM-DD HH:MM:SS’. The advantage of selecting TIMESTAMP in this case is that we can select the default value to be the current time which means that when we write our data to the table we only need to write the temperature value and the ‘dtg’ will be entered automatically for us. The disadvantage of using ‘TIMESTAMP’ is that it has a more limited range than DATETIME. TIMESTAMP can only have a range between ‘1970-01-01 00:00:01’ to ‘2038-01-19 03:14:07’.

Our temperature readings are generated (by our sensor) as an integer value that needs to be divided by 1000 to show degrees Centigrade. We could therefore store the value as an integer. However when we were selecting the data or in later processing we would then need to do the math to convert it to the correct value. It could be argued (successfully) that this would be a more efficient solution in terms of the amount of space taken to support the data on the Pi. However, I have a preference for storing the values as they would be used later and as a result we need to use a numerical format that supports numbers with decimal places. There are a range of options for defining the ranges for decimal numbers, but FLOAT allows us to ignore the options (at the expense of efficiency) and rely on our recorded values being somewhere between -3.402823466E+38 and 3.402823466E+38 (if our temperature falls outside those extremes we are in trouble).

Record the temperature values

The following code (which is based on the code that is part of the great temperature sensing tutorial on Adafruit) is a Python script which allows us to check the temperature reading from the sensor approximately every 10 seconds and write it to our database.

The full code can be found here(pi_temp.py).

#!/usr/bin/python
# -*- coding: utf-8 -*-

import os
import glob
import time
import MySQLdb as mdb

os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')

base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')[0]
device_file = device_folder + '/w1_slave'

def read_temp_raw():
f = open(device_file, 'r')
lines = f.readlines()
f.close()
return lines

def read_temp():
lines = read_temp_raw()
while lines[0].strip()[-3:] != 'YES':
time.sleep(0.2)
lines = read_temp_raw()
equals_pos = lines[1].find('t=')
if equals_pos != -1:
temp_string = lines[1][equals_pos+2:]
temp_c = float(temp_string) / 1000.0
return temp_c

while True:

try:
pi_temp = read_temp()
con = mdb.connect('localhost', \
'pi_insert', \
'xxxxxxxxxx', \
'measurements');
cur = con.cursor()
cur.execute("""INSERT INTO temperature(temperature) \
VALUES(%s)""", (pi_temp))
con.commit()

except mdb.Error, e:
con.rollback()
print "Error %d: %s" % (e.args[0],e.args[1])
sys.exit(1)

finally:
if con:
con.close()

time.sleep(10)

Additionally, if you’d like to use the Grove Pi, I’ve altered the grove_sound_sensor.py script to store the data values to our Measurements database.

#!/usr/bin/env python
import time
import grovepi
import MySQLdb as mdb

# Connect the Grove Sound Sensor to analog port A0
sound_sensor = 0

# Connect the Grove LED to digital port D5
led = 5

grovepi.pinMode(sound_sensor,"INPUT")
grovepi.pinMode(led,"OUTPUT")

def read_sensor():
        # Read the sound level
        sensor_value = grovepi.analogRead(sound_sensor)
        print("sensor_value = %d" %sensor_value)
        return sensor_value

while True:

    try:
        sensor_value = read_sensor()
        connection = mdb.connect('localhost',
                          'root',
                          'yourPassword',
                          'measurements');
        cursor = connection.cursor()
        cursor.execute("""INSERT INTO temperature(temperature)
                       VALUES(%s)""", (sensor_value))
        connection.commit()

    except mdb.Error, e:
        connection.rollback()
        print "Error %d: %s" % (e.args[0],e.args[1])
        sys.exit(1)

    finally:
        if connection:
            connection.close()

	time.sleep(10)

Setting up Apache, PHP and MySQL

Web Server and PHP

Because we want to be able to explore the data we will be collecting, we need to set up a web server that will present web pages to other computers that will be browsing within the network (remembering that this is not intended to be connected to the internet, just inside your home network). At the same time as setting up a web server on the Pi we will install PHP.

At the command line run the following command;

sudo apt-get install apache2 php5 libapache2-mod-php5

Your Pi will advise you of the range of additional packages that will be installed at the same time (to support those we’re installing (these additional packages are dependencies)). Agree to continue and the installation will proceed. This should take a few minutes or more (depending on the speed of your Internet connection).

Once complete we need to restart our web server with the following command;

sudo service apache2 restart

We can now test our web server from our laptops by opening up a web browser and type in the IP address of our Pi into the URL bar at the top. Hopefully we will see…

Tweak permissions

The default installation of the Apache web server has the location of the files that make up our web server owned by the ‘root’ user. This means that if we want to edit them we need to do so with the permissions of the root user. This can be easily achieved by typing something like:

sudo nano /var/www/index.html

but if we want to be able to use an editor from our GUI desktop, we will need the permissions to be set to allow the ‘pi’ user to edit them without having to invoke sudo.

We’re going to do this by making a separate group (‘www-data’ will be its name) the owners of the /var/www directory (where our web files will live) then we will add the ‘pi’ user to the ‘www-data’ group.

We start by making the ‘www-data’ group and user the owner of the /var/www directory with the following command;

sudo chown www-data:www-data /var/www

Then we allow the ‘www-data’ group permission to write to the directory:

sudo chmod 775 /var/www

Then we add the ‘pi’ user to the ‘www-data’group;

sudo usermod -a -G www-data pi

This change in permissions are best enacted by rebooting the Raspberry Pi;

sudo reboot

Now the ‘pi’ user has the ability to edit files in the /var/www directory without problem.

MySQL Database

We will use a MySQL database to store the information that we collect. We will install MySQL in a couple of steps and then we will install the database administration tool phpMyAdmin to make our lives easier.

From the command line run the following command;

sudo apt-get install mysql-server

You will be prompted (twice) to enter a root password for your database. Note it down somewhere safe.

Once this installation is complete, we will install a couple more packages that we will use in the future when we integrate PHP and Python with MySQL. To do this enter the following from the command line;

sudo apt-get install mysql-client php5-mysql python-mysqldb

Agree to the installed packages and the installation will proceed fairly quickly.

That’s it! MySQL server installed. However, it’s not configured for use, so we will install phpMyAdmin to help us out.

phpMyAdmin

phpMyAdmin²⁸ is free software written in PHP to carry out administration of a MySQL database installation.

To begin installation run the following from the command line;

sudo apt-get install phpmyadmin

Agree to the installed packages and the installation will proceed. You will receive a prompt to ask what sort of web server we are using:

Select ‘apache2’ and tab to ‘Ok’ to continue.

We will then be prompted to configure the database for use with phpMyAdmin. We want the program to look after it for us, so select ‘Yes’ and continue. We will then be prompted for the password for the administrative account for the MySQL database.

This is the root password for MySQL that we set up earlier. Enter it and tab to ‘Ok’ to continue.

We will then be prompted for a password for phpMyAdmin to access MySQL.

The installation should conclude soon.

Once finished, we need to edit the Apache web server configuration to access phpMyAdmin. To

do this execute the following command from the command line;

sudo nano /etc/apache2/apache2.conf

Get to the bottom of the file by pressing ctrl-v a few times and there we want to add the line;

Include /etc/phpmyadmin/apache.conf

Save the file and then restart Apache2;

Now if we go to our browser on our laptops (or on the Pi) and enter the IP address followed by /phpmyadmin (in the case of our example 10.1.1.8/phpmyadmin) it should start up phpMyAdmin in the browser.

If you enter the username as ‘root’ and the MySQL root password that we set earlier, it will open up the phpMyAdmin interface.

Allow access to the database remotely

It might seem a little strange to say that we want to allow access to the database remotely since this would appear to be part of the grand plan all along. But there are different types of access and in particular there may be a need for a remote computer to access the database directly.

This direct access occurs when (for example) a web server on a different computer to the Raspberry Pi wants to use the data. In this situation it would need to request access to the database over the network by referencing the host computer that the database was on (in this case we have specified that it is on the computer at the IP address 192.168.1.74).

By default the Raspberry Pi’s Operating System is set up to deny that access and if this is something that you want to allow this is what you will need to do.

On the Raspberry Pi we need to edit the configuration file ‘my.cnf’ in the directory /etc/mysql/. We do this with the following command;

sudo nano /etc/mysql/my.cnf

Scroll down the file a short way till we find the section [mysqld]. Here we need to edit the line that reads something similar to;

bind-address = 127.0.0.1

This line is telling MySQL to only listen to commands that come from the ‘localhost’ network (127.0.0.1). Essentially only the Raspberry Pi itself. We can change this by inserting a ‘#’ in front of the line which will turn the line into a comment instead of a configuration option. So change it to look like this;

# bind-address = 127.0.0.1

Once we have saved the file, we need to restart the MySQL service with the following command;

sudo service mysql restart

RaspberryPi 05: Remote Desktop

UPDATE:

This article is out of date, RealVNC is now installed on Raspbian, see this article for more info

(excerpt from page 27 of MRE-RPi book)

Often all you need to control your Raspberry Pi is an SSH connection, however sometimes it is useful to be able to remotely control your RPi using the mouse and seeing just what you would see on the screen of your RPi.

VNC (Virtual Network Connection) is a standard for doing just this. To use it, you have to install some software on your Pi. There are a number of VNC server applications, and the one we are going to use is called “tightvnc”.
We can install the VNC server software using the SSH connection that we established earlier.

**sudo apt-get install tightvncserver**
“vncserver :1 -geometry 1024×768 -depth 24”

on mac: open screen sharing
“vnc://192.168.10.70:5901”

Starting TightVNCServer on Boot

Having a remote desktop is extremely useful, but if we need to run the tightvncserver program on the Raspberry Pi each time we want to use the remote desktop, it would be extremely inconvenient. What we will do instead is set up tightvncserver so that it starts automatically each time the Raspberry Pi boots up.

To do this we are going to use a little bit of Linux cleverness. We’ll explain it as we go along, but be aware, some will find the explanations a little tiresome (if you’re already familiar) but I’m sure that there will be some readers who will benefit.

Our first task will be to edit the file /etc/rc.local. This file can contain commands that get run on start-up. If we look at the file we can see that there is already few entries in there;

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will “exit 0” on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
# Print the IP address

_IP=$(hostname -I) || true
if [ $_IP” ]; then
printf “My IP address is %sÄn” “$_IP”
fi

exit 0

We will add the following command into rc.local;
su – pi -c ‘/usr/bin/tightvncserver :1’

To do this we will edit the rc.local file with the following command;
sudo nano /etc/rc.local

RaspberryPi 03: Connecting RPi to WiFi

Overview

For these tutorials we will not use the Emily Carr Wifi network as it is problematic, instead we’ll be using my own network for this course, here is the login:

  • SSID: wifi network name is ecu-ixd
  • Password: wifi password is babelfish67

To connect to this network from your Raspberry Pi, we’ll need to edit the /etc/network/interfaces file.

We’re going to use nano to do this. Nano is a text editor for Unix-like computing systems or operating environments using a command line interface.

To edit the file type:
sudo nano /etc/network/interfaces

Modify the file so it looks the same as below:

auto lo
 
iface lo inet loopback
iface eth0 inet dhcp
 
allow-hotplug wlan0
auto wlan0
 
 
iface wlan0 inet dhcp
        wpa-ssid "ecu-ixd"
        wpa-psk "babel67"

Now save the file by using the key combination ctl+O (press the control button and the o button at the same time) Press ‘return’ to confirm the file name. Finally, to exit nano press ctl+X.

We’ll need to restart the network interface in order for it to connect to the network. We can do that with:
sudo /etc/init.d/networking stop

and then
sudo /etc/init.d/networking start

Now that we’re connected to Wifi let’s connect our computer to the RPI

RaspberryPi 02: Exploring Command Line Basics

The command prompt

Now that you’ve logged in you’ll see a command prompt with:

pi@raspberrypi ~ $

What does it mean?

  1. The pi section is your username followed by @.
  2. The raspberrypi (the default hostname), the ~ is your current working directory.
  3. The ~ symbol is short hand for your home directory. When you log into your Raspberry Pi, you first land in your home directory.
  4. The tailing $ is your shell prompt, anything you type to the right of it gets executed as commands.

Poking around the home directory

When you first login, you’re in the home directory. The home directory is represented as ~ on the command prompt. The ~ character can be used as a shortcut to jump into your home directory.

To see what’s inside your home directory use the ls command.

Other Directories

Your home directory has the following path /home/pi. There are other folders/directories above your home directory.

Don’t believe me? We can ask Linux to tell us what our current directory is by running:

pwd

you should see the following output:

/home/pi

pwd stands for print working directory.

Changing directory

To change directory use the cd command. What did you think cd stands for?

Moving up a directory

To move up a directory from pi’s home directory to the /home directory use the following command:

cd ..

Now when you print the current working directory, you should see.

/home

To go back to your home directory you would use the command:

cd pi however the following commands could also do the trick:

  • cd /home/pi (specifying the full path)
  • cd ~ (using the ~ shorthand)
  • cd (even typing just cd takes you back to your home directory, the maker of cd sure hates to type extra characters)
  • cd - (the - argument takes us to the directory we were immediately previously in, so this command would also work)

Other directories

Of course there’s more to your Pi than the /home directory. Let’s have an explore.

The folder structure on your Raspberry Pi looks something like this (with your home directory highlighted).’

folder-structure

Directory Description
/ The upper most of your hard disk (or SD Card)
/bin Programs and commands that all users can run
/boot All the files needed for booting your Raspberry Pi
/dev Files that represent devices on your Raspberry Pi
/etc Configuration files
/etc/init.d Scripts to start, stop and otherwise command services
/etc/X11 X11 Configuration Files
/home All the user home directories (except for root’s)
/home/pi The Pi user’s home directory
/lib This is where the Kernel modules / drivers live.
/media There is where hard disks, SD Cards and other removable media are mounted
/proc Virtual directory that provides details of running processes
/sbin Programs primarily used for systems maintenance
/sys Another special folder that represents hardware devices
/tmp A space for temporary files
/usr Programs and data usable by every user
/usr/bin Most programs you’ll run live in here
/usr/lib Libraries to support programs and programming languages
/usr/local By convention software specific to a machine goes here.
/usr/sbin More system software
/usr/share Supporting files that aren’t specific to chip architecture
/usr/src Source code!
/var System logs
/var/backups Typically compressed copies of system logs
/var/cache apt-get and other programs keep their data here
/var/log/ All the system and service logs
/var/mail This is where your mail goes (if you’ve set it up)
/var/spool Data that is waiting to be processed or dealt with lives here (e.g. mail or print jobs)

Ok, enough exploring let’s see what other commands we can do on the Pi!

RaspberryPi 01: Console Cable

Overview

Despite having four full sized USB ports and one micro USB port (for power), you can’t connect to your Raspberry Pi from your computer via USB (without additional hardware).

You can however connect to your Raspberry Pi via a “USB to TTL Serial Cable”, also sometimes called a Console Cable. This cable allows you to access the command line of your Raspberry Pi, by plugging the USB end into your computer and the other end into specific serial GPIOs on the Raspberry Pi.

For the most current information on Connecting with the Console look here. http://elinux.org/RPi_Serial_Connection

Note for RaspberryPi 3:
Add the following two lines in config.txt file on your RPi SD Card:
dtoverlay=pi3-disable-bt
enable_uart=1

What you’ll need

For this exercise you’ll need a console cable:

Caveat: Make sure your computer delivers at least 500 mA (milliamps) at 5 V (Volts). This will assure you that your RPi will receive the power it needs otherwise there’s a chance you’ll fry your Pi!

Software Installation (Mac)

Mac OS X includes a Terminal (inside /Applications/Utilities/), so the only other software you need to install is the drivers for the cable.

You can download the Mac drivers here.

Software Installation (Windows)

Windows does not include a Terminal application that you can use to connect over serial.

For this workshop we’ll use Putty. You can get the “putty.exe” app from the puttywebsite.

Download the one called “PuTTY” under For Windows on Intel x86 (direct link).

Putty downloads as an uncompressed binary.

Now that you’ve got Putty, you’ll also need to install the cable drivers here.

Software Installation (Linux)

Assuming you’re running a v2.6 kernel or later you shouldn’t need to install drivers for the cable. If you find that your distribution doesn’t have the drivers installed you can download the drivers from the FTDI website.

You may also need to install screen. The screen command (some Linux distributions such as Ubuntu 12.10 don’t include the screen software). To see if you’ve got screen installed type screen and if you get an error install screen by running: sudo apt-get install screen.

Connecting the cable

The console cable handily provides the name of the line on the side of the terminal header.

Connect the console cable to the following pins:

  • DC → RPi Pin 02 (DC Power 5V)
  • GND → RPi Pin 06 (Ground)
  • RXD → RPi Pin 08 (Data Transmit)
  • TXD → RPi Pin 10 (Data Receive)

Connecting via the Shell

Bringing up the Shell On Mac / Linux

TL;DR:

If you’re running Mac OS X all you need to do is open a Terminal window and run screen in the command line using the command:

screen -L /dev/tty.usbserial 115200

screen is the command. /dev/cu.usbserial is the path to your serial device and 115200 is the baud rate (the speed at which we send data across the serial interface).

If you’re on Linux you’ll need to escalate your privileges by putting sudo in front of the command.

E.g:

sudo screen /dev/cu.usbserial 115200

Bringing up the Shell on Windows

If you’re running Windows, you’ll need to find out what com port the console cable is using.

  1. Search for Device Manager
  2. Expand the Ports (COM & LPT) section

You now need to open putty

  1. Select Serial as the connection type
  2. Enter the name of the COM port you’ve found
  3. Set the speed to 115200
  4. Click Open and you should see the terminal

Logging In

Once you’ve brought up the shell, you’ll be faced with a login prompt. You can log into your Raspberry Pi with the following credentials:

  • username: pi
  • password: raspberry

Ok we’re in! Let’s have a look around in the next tutorial.

Raspberry Pi 00: Introduction & Flashing SD Card

The Raspberry Pi is an incredible little computer made up of a components and connectors on an unfeasibly small printed circuit board that is approximately the dimensions of a credit card. Depending upon your intended use, your Rasberry Pi may have a keyboard, monitor and mouse added. Regardless of your intended use, you are going to need an SD (Secure Digital) card on which to install an operating system.

This tutorial describes how to install an operating system onto an SD card for use in your Raspberry Pi.

What You Need to Do and Why

In order to get up and running with your Raspberry Pi, you will first need to flash an SD card. Whilst your Mac, or PC, uses a magnetic hard disc drive (HDD) or a Solid State Drive (SSD), your Raspberry Pi has no internal storage of its own.

Instead, the Raspberry Pi uses a Secure Digital (SD) card, such as is common as storage in many digital cameras.

This means that the Raspberry Pi will run from whatever operating system you install on an SD card. You can do from your main computer regardless of whether you are using Apple OS X, Linux or Microsoft Windows.

For the purposes of this tutorial I will be downloading Raspian, for Raspberry Pi.

The download will be compressed as a .zip file. You will need to extract to .img or .iso file before proceeding.

Tip: You can download a number of different operating systems, including RISC OS, Raspian, Raspbmc, Openelec and pidora, from the Raspberry Pi downloads site or you can download the NOOBS, New Out Of Box Software

Good Installation Guide:

https://www.raspberrypi.org/documentation/installation/installing-images/README.md

The easiest method on the Mac is to install Apple Pi Baker, it’s got a simple user interface so you don’t have to muck about with command lines.

Now it’s time to connect all the peripherals and boot our Raspberry Pi up.