Building a Generic Ubuntu LAMP Server¶

These instructions assume that you have already installed VirtualBox and downloaded the latest Ubuntu LTS ISO. Note that the process is similar for other releases and for other distributions. The commands shown below, however, have only been tested/verified on Ubuntu 18.04 LTS and 20.04 LTS. It is expected that these will be similar for Ubuntu 22.04, the latest version as of 7/2022. Check with your Project Manager as to which version to use!

The host computer can be Windows, Linux or Mac. All of this works the same regardless of the host computer details which is part of the power of using virtual machines (VM's).

Create the Virtual Machine¶

From the VirtualBox GUI, create a new machine. When prompted, choose Linux for the type and Ubuntu 64bit for the operating system. Take the next few defaults. For disk space provide at least 12GB, preferably 14GM and if you have room, then allow 16GB - more if you have plans for an application that requires a lot of disk space. You CAN add disk space later. This will be explained later during the Operating system installation.

Once you have created the virtual machine, you essentially have your own personal computer that happens to "live inside" of your PC.

Before "booting" your PC, we need to adjust some settings.

System - set the RAM to 2GB, higher if you have memory to spare. Anything beyond 4G will be wasted for most virtual machines.
Display - select the maximum video ram. Do not select 3D or 2D graphics.
Network - if creating a virtual machine to act as a server, then you will need a predefined IP address. Note that the Design Lab has reserved a set of IP addresses. See Mark Anderson BEFORE you use any of these! For this case, choose "Bridged". For personal use only, NAT is the correct choice and does not require a pre-allocated IP address.
Serial Ports - no changes
USB - no changes
Shared folders - no changes
User Interface - no changes

Operating System Installation¶

As you would for any new PC, the operating system must now be installed. Go to the Storage section of your new virtual machine and click on the Empty item for the IDE controller. Then, using the pull down on the right, select the disk icon next to the Removable Drive and from there, Choose Optical Disk file. Locate and select the previously downloaded ISO file for the operating system you wish to install. This is equivalent to putting a disk in the DVD drive on a real PC. Select Live CD because that is how the Linux distribution media works. Note that the Design Lab standard is Ubuntu, latest version for new virtual machines.

Now that your virtual PC is configured, you may click on the Start arrow in the GUI to "boot" your machine. It will now load your live disk allowing you to either try or install the operating system. In this case, we wish to Install.

Ubuntu will now present it's current set of installation questions. The defaults will mostly work fine. One exception is that we recommend the use of LVM for the disk setup. This makes it very easy to add a second disk later if your VM needs to expand.

All VM's built by the Design Lab will include an account for either Mark Anderson or Junichi Kanai or both. All VM's built by students will have one or both of these accounts added as a condition of hosting. Mark and Junichi will have administrator access so that we can provide support as needed.

All VM's should use a team-wide account to ensure that work done on the server is accessible by the entire team and later by the Client. The team-wide account information must appear in your team's wiki with both user name and password.

The team-wide account will generally be given administrator privileges so that you can install software as needed on your VM.

It is highly recommended that you take a snapshot at this point! You can then revert back to this in case a future step does not work as expected.

It is highly recommended that you install the Guest Editions. You need to open the terminal prompt and run

sudo apt install y build-essential tar bzip2

.
This installs a pre-requisite for the Guest Editions.

On the Virtual Machine's menu bar, choose Devices, then Insert Guest Editions CD. A window should open with the files on this virtual CD and the installation should automatically start. If not, you can use the Run Software button OR you can right click on the "autorun.sh" file and select "Run as Program". Once that is started it should yield a message to reboot. When that completes, you can double click on the CD image in the left side menu and start over with the Guest Editions installation. You must reboot the virtual machine to complete the installation. After it reboots, you can go to the menu and select Devices, Shared Clipboard and choose Bidirectional. Then you can copy from Linux and paste to Windows and vice versa. The other key benefit of the Guest Edition is that you can make your virtual machine full screen. If this does not work, please see your PE for help. This can be done later.

At this point you will have a fully functioning virtual machine. It will NOT yet have the fixed IP address as that must be manually configured and for NAT it may have access to the network. If you chose NAT for use in a personal machine, the network may have to be enabled. If you are using Bridged to provide a server at a fixed IP address, then additional configuration is required before activation of the network.

Network Configuration - NAT¶

The network should already working and require no further attention. Note - there was apparently a one-time issue, possibly caused by an early release of Ubuntu 20 as this "fix" has not been needed.

Must edit "/sysconfig/???"

change to "on"
restart networking

Network Configuration - Bridged / Fixed IP¶

Note that you must change the network interface from NAT to Bridged to complete the configuration.

See https://tecadmin.net/how-to-configure-static-ip-address-on-ubuntu-22-04/

MQTT¶

IoT applications may elect to use the MQTT approach. To install this software:

# Install Mosquitto Broker and clients
sudo apt update
sudo apt -y upgrade
sudo apt install -y mosquitto mosquitto-clients

Note that this does not install / enable any security for MQTT. That can be added at a later time.

Use of MQTT does NOT require LAMP installation.

Another typical method used for IoT applications is to use Django (see below). IoT applications can use the RESTful API implemented by Django to have their devices interact with a server. This approach is beneficial when one or more web based apps are required. Note that MQTT and Django can also both be used.

LAMP Software Installation¶

(turn on shared clipboard bidirectional)

NOTE - this page is revised as/when installation changes are identified, e.g. due to Ubuntu operating system updates. It represents the best information we currently have as of the last update to this page. If you learn / discover something that requires a change, please notify Mark Anderson, the page maintainer.

With Ubuntu installed we have the "L" in LAMP. Now install Apache web server, MySQL. If PHP is needed, it can be installed (not always needed but standard to "LAMP"). We will not perform all the steps to create a very secure server - only the minimal steps to create a working one.

# make sure we have the latest of everything for this version of this release of Ubuntu
# do these two commands one at a time. Can skip them if you just did them as part of the previous step.
sudo apt update
sudo apt -y upgrade

# the A in LAMP
sudo apt install -y apache2
# ON the virtual machine, opening the browser and going to
# http://localhost will bring up an Apache default page, showing that this worked.

# now the M
sudo apt install -y mysql-server
################################################################################
# This is for Ubuntu 22.04 because it has a more recent version of MySQL.
# This version has changed the security methods which broke the
# "mysql_secure_installation" command! So we have to do a workaround first.
# If the "mysql_secure_installation" command fails on your installation, you need this workaround.
# The following password can be anything but should be long enough for security purposes.
# On a personal machine / project you will want to choose your own.
# MUST do one line at a time
sudo mysql
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password by '!gao_24x!mpo';
exit
################################################################################

# The mysql_secure_installation command will ask several questions - so do not paste it in as part of a block!
# now it must be configured for the first time
sudo mysql_secure_installation
# can say no to validate password component used to enforce long passwords
# be sure to document in your team wiki whatever password you use for the root MySQL account.
# Prebuilt VM's will have the unsecure value of "password"
# note that this is NOT the same as the operating system's root account
# If you set it per the above work around then keep the existing root password
# or set it here, remove the default user, do not allow root to log in over the network
# remove the test database, reload the privilege table
# done with mysql install

# If your project needs it, install the P -- the PHP language.
sudo apt install -y php

You can access MySQL as root with a password using:

sudo mysql -u root -p

with the password used when setting up the database. Now that MySQL is configured to require a root password you can no longer use the old

sudo mysql

command.

This completes the essential completion of a LAMP server, ready for other application software as needed.

Application Software Installation¶

There are some common utility applications any many more that might be project specific. This section will include some of each from which you can elect items to include.

Productivity Tools¶

Network Tools¶

These tools will be pre-installed on Design Lab provided VM's.

# provide tools such as ifconfig
sudo apt install -y net-tools
# MANDATORY !!! for remote logins
sudo apt install -y openssh-server

Miscellaneous¶

This text editor will be pre-installed on Design Lab provided VM's.

# Geany text editor. 
# Similar to Notepad++ on Windows. Personal preference.
sudo apt install -y geany libvte9

This utility is not pre-installed:

# for command line, ability to show a directory tree
sudo apt install -y tree

This utility is not pre-installed:

# lets you interact with MySQL database via a GUI
# on Ubuntu <= 18:
#sudo apt install -y mysql-workbench
# to run it on Ubuntu 18, the command is: mysql-workbench
# on Ubuntu 20-22
sudo snap install mysql-workbench-community
# let the password manager do it's job!
sudo snap connect mysql-workbench-community:password-manager-service :password-manager-service
# to run it on Ubuntu 20-22, the command is now: mysql-workbench-community

Subversion and Client - RabbitVCS¶

RabbitVCS works in a manner similar to TortoiseSVN on Windows. It is accessed by right clicking within the Nautilus file manager. RabbitVCS and the command line interface for Subversion are pre-installed on Design Lab VM's:

sudo apt -y install subversion rabbitvcs-nautilus

Note that this may already installed on Ubuntu 20. It is harmless to try to install a package if it's already installed.

At the command line type "svn help" for guidance.

Django Web application framework¶

As the Design Lab has selected Django as our standard web framework, it is pre-installed on Design Lab provided VM's. You do not need to remove it if your team's project does not require it as it has very little impact on the system.

Do NOT use old instructions, such as those relating to Ubuntu 16.04, as there are enough differences to make it fail. For example, for Python 3, be sure to use venv, not the older virtualenv command.

The following commands can be copied from here and pasted into a shell script file for later edit & execution. Note that these instructions use the anderm8 account which must be changed for your individual project usage. Do not simply copy these into the command line as all of the commands involving the folder names will fail!

System Software Setup¶

These things are done once for the system.

sudo apt update
# Apache extras first
sudo apt install -y apache2 apache2-utils apache2-dev

# IF this was (accidentally) installed, it provides support for Python 2.x, not 3.x
# so remove it!
# This will indicate if it was installed or not.
sudo apt remove libapache2-mod-python libapache2-mod-wsgi
# this version is required for Python 3.x
sudo apt install -y libapache2-mod-wsgi-py3
# note that wsgi may be enabled by the above command. this just makes sure.
sudo a2enmod wsgi

python3 -V
sudo apt install -y python3-pip
sudo apt install -y python3-venv
# to access MySql from Python3 needs some more software installed
sudo apt install -y python3-dev default-libmysqlclient-dev
sudo apt install -y python3-dev libmysqlclient-dev
sudo apt install -y pkg-config

The above sets up only the software environment. The following would need to be done for each user account.

Virtual Python Environment¶

Our standard is to create a virtual Python environment. EVERYONE needs to do this on their own development environment! This only needs to be done once on a server (Design Lab VM)!

We use the "venv" command to create the virtual environment. This Python tool allows the user to create one or multiple separate environments, each potentially containing a different Python version and associate packages. Using a virtual environment, even when there is only a single one as there is for Capstone, allows the user to keep this Python configuration separate from the main system-wide. Thus anything else on your virtual machine or computer will not be impacted by your Capstone work.

# create the location, so that everyone has the same folder structure
mkdir -p ~/Documents/django/project/
cd ~/Documents/django
# Create the environment. The name does not matter, so here we are using "env".
python3 -m venv env

# every time we want to use it, must use the source command. Note that the first part of the path is the name of the virtual environment just created, e.g., "env".

source env/bin/activate

# now install django IN this environment (one time)
# these MUST be done after you activate this virtual environment!
pip install django
# and confirm it
django-admin --version
# additional needed items
pip install wheel
pip install mod_wsgi
pip install mysqlclient

Folder Structure and Configuration for Each Project¶

Since not all teams have a need for an example project, these steps have not been performed on the standard Design Lab VM. With an existing Django project, you will need to create the folder path as shown but instead of using "example" you must use the name of the Django project as found in your repo. You then check out your Django project from the repo into that folder.

These things must be done by each user, for each separate Django project they wish to create. Here, our Django project will be called "example". Do NOT call your project example! DO figure out a meaningful name from the start as it is NOT easy to change it later.

These commands will create some of the required folder structure.

cd ~/Documents
mkdir -p django/project/example
# set up a place for the static files to be served from
mkdir -p django/project/example/static
cd django

For ongoing projects, the above should already have been done on your Design Lab VM for the shared account.

On your personal virtual machines, you will need to obtain the current Django project by checking it out of your project's repository.

In some circumstances you may want to or need to create a completely new Django project, such as to experiment with programming ideas before adding them to the team-wide Django project. In this case, the following steps can be used to create a new Django project on your development VM. Please do not create additional projects on the Design Lab provided server VM.

# ONLY FOR NEW PROJECTS
# now can create projects...
# for THIS example, we'll use example. 
# Be sure to use a meaningful name for YOUR project instead
# and then adjust these instructions accordingly.
django-admin startproject example

# IF you are working with an ongoing project, you will use subversion to check out a copy of the project into the
# ~/Documents/django/project/ folder.
# Moving forward we'll assume that either the new or existing project was named "example" 

cd example
# FOR NEW PROJECTS manually edit the example/settings.py file
# For existing projects, this should have already been done! But you can double check!

# ALLOWED_HOSTS = []
# line to allow any host -> change [] to be ['*'], i.e.
# ALLOWED_HOSTS = ['*']
# this can be done with nano from the command line
# this change is not required to connect from inside the VM with the localhost

# create a MySQL database and user for this project
# assuming the root password is just password. Note the required use of sudo for this
# You can use other usernames - just make note of them as you'll have to tell Django this information to gain access to the database
sudo mysql -u root
CREATE DATABASE iot;
CREATE USER 'iot_user'@'localhost' IDENTIFIED BY '#!iot!#';
GRANT ALL PRIVILEGES ON iot.* TO 'iot_user'@'localhost';
exit

# with the database in place, you can "migrate" the data models into the actual database
# This sets up the database to match the latest definitions of the Django project's models
python manage.py makemigrations
python manage.py migrate

# with a database defined it is now possible to define one or more superusers
# The Django superuser can log in as an admin and then has access to some of the models and other admin functions.
# I used capstone / password and told it to accept the weak password
# also created one for anderm8
python manage.py createsuperuser

# with the above steps completed, it is now possible to start your Django project
#run server specifying permission for any host at port 8000. If this port is in use by a team mate, you can use a higher number, ex: 8001, 8002, etc.
python manage.py runserver 0.0.0.0:8000
# should now be able see Django screen on a web browser by visiting: http://localhost:8000

The default database for Django is sqlite. Now you must manually edit the example/settings.py file to select MySQL instead of sqlite and to set the db_name, db_user, and db_password to match the above.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'iot',
        'USER': 'iot_user',
        'PASSWORD': '#!iot!#',
        'HOST': '127.0.0.1',
    }
}

You will then need to redo the migrations:

python manage.py makemigrations
python manage.py migrate

to set things up in the MySQL database.

You will also need to recreate the superusers as their info is stored in the active database.

python manage.py createsuperuser

Database Note¶

The account above is only given access to the database intended for Django usage. That user does NOT have the required privileges needed to create a new table for any other purposes.

To create other databases, you need to have root privileges as shown above. It is recommended that you create other different database user names to access any other non-Django databases to help keep things modular.

Serving the Django Website from Apache¶

While the "runserver" approach documented above works for development purposes, the "production" version of the website should be set to run under Apache.

To serve the website available from Apache requires manual editing of the virtual host configuration file
found at

 /etc/apache2/sites-available/000-default.conf

You may have to make multiple edits to that file depending on where you placed your Django project and the specific folder structure you used.

A sample file is attached to this page that serves the sample application created above. 000-default.conf

Using your New Virtual Machine / LAMP Based Django Server¶

When you create the environment folder, keep it OUTSIDE your project folder. DO NOT commit the environment to your repository! That just wastes space. It takes only minutes to regenerate a fresh environment as needed - so no need to store it!

Be sure to "source" your environment each time you start work. If you try to run python manage.py commands and they fail - it's probably because you forgot to do this and as a result it's attempting to use Python 2.x - which will not work with your Django Python 3.x environment.

Be sure to "svn update" your working copy before you start work so that you have the most recent team work.

As a team, figure out what your project / app naming conventions will be ahead of time. Use meaningful names. Do NOT simply copy the above process so that all future Django projects are named iot_rpi! It is NOT easy to change project or app names later!

Do NOT work on your project as the root user! Be in "root" mode ONLY to make operating system changes such as installing new programs or editing the Apache site configuration file.

Additional Resources¶

For more info on using / working with this framework, please visit the Django page.

The following is a partial list of websites that were helpful in creating these instructions. Do NOT FOLLOW the instructions from these other sites but instead directly follow the instructions detailed on this page. All the required information is on this page and this page alone! The links shared here are only for reference purposes.

https://www.digitalocean.com/community/tutorials/how-to-install-the-django-web-framework-on-ubuntu-18-04

Files (1)

Project

General

Profile

Capstone Support

Wiki