Etherbox

These notes are based on a setup used by the "SOftware observatory" workshop, Constant Michael Murtaugh (discussion) 30 janvier 2018 à 10:48 (CET)

Initial image + setup

Downloaded from https://www.raspberrypi.org/downloads/raspbian/

Based on the "lite" image (zip or torrent). As of 20 Jan 2018, this is Raspian "Stretch" lite.

The **lite** image has no desktop / windows session.

Based on 2017-04-10-raspian-jessie-lite.zip

unzip -p 2017-04-10-raspbian-jessie-lite.zip | pv | sudo dd of=/dev/sdc bs=4M

You could use Etcher.io as well

SSH is no longer on by default! So need to connect with a screen first time and turn this on.

sudo raspi-config

Enable ssh under connectivity.

You can also enabled SSH by default after creating the boot SD Card, you just need to create a empty file named SSH and save it the root of your SD Card.

Bring the rest of the software up to date.

sudo apt-get update
sudo apt-get upgrade

How to find the IP address of the pi

Simplest way is to connect on a wired network (plug the pi and your laptop into a router with ethernet cables), then type:

   ping raspberrypi.local

And you should be able to see the IP address.

Then you can conenct with ssh with:

   ssh pi@raspberrypi.local

or with the IP address in place of "raspberrypi.local" if you are on the wifi.

with the default password "raspberry"

Make it easier to login, with an ssh key

Starting from your laptop (open a new Terminal session if you are connected to the pi):

   ssh-keygen

Choose the defaults. This generates an "ssh key" pair.

Use the ssh-copy-id utility to send it to the pi.

   ssh-copy-id pi@raspberrypi.local

Setup apache to serve the root with custom header + readme's

sudo apt-get install apache2
cd /etc/apache2/sites-available
sudo nano 000-default.conf

    ServerAdmin webmaster@localhost
    # DocumentRoot /var/www/html
    DocumentRoot /home/pi
    <Directory /home/pi>
           Options Indexes FollowSymLinks
           AllowOverride none
           Require all granted
    </Directory>

    HeaderName /include/HEADER.shtml
    ReadmeName README.html

NB: Sets the HeaderName and ReadmeName directives (part of mod_autoindex).

sudo service apache2 reload

droptoupload.cgi

sudo a2enmod cgi
sudo service apache2 restart

Placed 'droptoupload.cgi' in /usr/lib/cgi-bin and tried running it with:

./droptoupload.cgi

Like this is just outputs an HTML form. Looking at http://etherbox.local/cgi-bin/droptoupload.cgi should also display an upload form.

The HEADER.shtml includes a link to this cgi.

  sudo chmod +x droptoupload.cgi

HEADER.shtml

Sample Header that adds javascript to:

• Move the README.html to the TOP of the page
• Includes the "droptoupload.cgi" https://gitlab.constantvzw.org/aa/etherbox/raw/master/usr/lib/cgi-bin/droptoupload.cgi

<script src="/cgi-bin/droptoupload.cgi"></script>
<style>
body {
background: #38b8e9;
color: black;
}
a {
color: white;
}
#logo {
white-space: pre;
font-family: monospace;
}
</style>
<div class="links" style="margin-bottom: 1em">LOCAL:
<a href="/">&nbsp;/&nbsp;</a>
<a href="/home/pi/">home</a>
<a href="/home/pi/etherdump/">etherdump</a>
PUBLIC:
<a href="http://constantvzw.org/site/-The-Technogalactic-Software-Observatory-.html">constant</a>
<a href="https://gitlab.constantvzw.org/observatory">gitlab</a>
</div>
<style>
.links {
font-family: monospace;
text-transform: uppercase;
</style>
<script>
document.addEventListener("DOMContentLoaded", function () {
  var p = document.querySelectorAll(".top"),
      t = document.getElementsByTagName("table")[0];
  for (var i=0, l=p.length; i<l; i++) {
    document.body.insertBefore(p[i], t);
  }
});
</script>

Better permissions with facl

setfacl

sudo addgroup pi www-data

sudo setfacl -Rm g:www-data:rwX /home/pi
sudo setfacl -d -Rm g:www-data:rwX /home/pi

Unfortunately, I had problems then with permissions on the .ssh folder (preventing keys to be used). To remove the fact on just this folder:

sudo chmod g-w /home/pi

Install etherpad

And the version of "nodejs" is now 0.10.29~dfsg-2. So let's try it with etherpad...

sudo apt-get install npm git

sudo ln -s /usr/bin/nodejs /usr/bin/node

cd /opt
sudo git clone https://github.com/ether/etherpad-lite.git
sudo mv etherpad-lite etherpad

# TODO: don't create home folder! ... find option
sudo adduser --system --home=/opt/etherpad --group etherpad
sudo chown -R etherpad:etherpad etherpad

Les fichiers se trouvent dans opt/etherpad/var sous forme d'un seul fichier "dirtyDB" - > You should use a dedicated database such as "mysql", if you are planning on using etherpad-in a production environment.

sudo apt-get install mysql-server

Create your setting file.

sudo cp settings.json.template settings.json

then, desactivate the dirty.db file and configure the mysql database

sudo nano settings.json

Find and change this part :

  //The Type of the database. You can choose between dirty, postgres, sqlite and mysql
  //You shouldn't use "dirty" for for anything else than testing or development
 /* "dbType" : "dirty",
  //the database specific settings
  "dbSettings" : {
                   "filename" : "var/dirty.db"
                 },
*/
  //An Example of MySQL Configuration
   "dbType" : "mysql",
   "dbSettings" : {
                    "user"    : "etherpaduser",
                    "host"    : "localhost",
                    "password": "etherpadpass",
                    "database": "etherpad",
                    "charset" : "utf8mb4"
                  },

Set up Mysql

First thing to do is to install MySQL :

sudo apt-get install mysql-server

Then create the database, for this we need to login with the Root user (super-user) :

sudo su

Run mysql  :

mysql

Then create the database and the user 'etherpaduser' with the password 'etherpadpass' :

create database etherpad;
grant all on etherpad.* to 'etherpaduser'@'localhost' identified by 'etherpadpass';

Just to test if it works :

mysql -u etherpaduser -p etherpad

Run etherpad for the first time as the etherpad user...

cd /opt/etherpad
sudo --user etherpad bin/run.sh

Following the first recipe on this page about deploying etherpad as a systemd service

Setup etherpad to start as a service

sudo nano /etc/systemd/system/etherpad.service

[Unit]
Description=Etherpad-lite, the collaborative editor.
After=syslog.target network.target

[Service]
Type=simple
User=etherpad
Group=etherpad
WorkingDirectory=/opt/etherpad
ExecStart=/usr/bin/nodejs /opt/etherpad/node_modules/ep_etherpad-lite/node/server.js
Restart=always

[Install]
WantedBy=multi-user.target

After this,

sudo systemctl start etherpad

And to start on boot:

sudo systemctl enable etherpad

etherdump

System wide installation of etherdump

Install deps:

sudo apt install python-pip python-dev
sudo pip install python-dateutil jinja2 html5lib

Install from repo:

git clone http://murtaugh@gitlab.constantvzw.org/aa/etherdump.git
cd etherdump
sudo python setup.py install

Setup the folder

cd /home/pi
mkdir etherdump
cd etherdump
etherdump init

Type in:

http://etherbox.local:9001/

And paste the API key. (Look at: http://etherbox.local/opt/etherpad/APIKEY.txt)

styles.css + versions.js

scp styles.css versions.js pi@etherbox.local:etherdump/lib

The URLs of these files are options to the etherdump pull command and should match.

etherdump.sh + cron

Make the script that runs automatically.

nano etherdump.sh

#!/bin/bash
# PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
cd /home/pi/etherdump
etherdump pull --all --pub /home/pi/etherdump --css lib/styles.css --script lib/versions.js
etherdump index *.meta.json > index.html

And set it to run every 5 minutes

crontab -e

PATH=/home/pi/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# m h  dom mon dow   command
*/5 * * * * /home/pi/etherdump.sh > /home/pi/cron.log.txt 2>&1

The PATH is important. It can also be in the etherdump.sh but basically should match what you see when you "echo $PATH" (for the script to run in the same way as for the pi user).

Other software

sudo apt-get install emacs-nox screen pdftk pandoc texlive-latex-recommended texlive-fonts-recommended
sudo pip install csvkit

To install

• screen
• pandoc + latex
• pdftk
• csvkit

What about

• texlive-xetex texlive-luatex pandoc-citeproc etoolbox

The current version of pandoc in this raspbian is 1.12.4.2~dfsg-1+b3 We will use latex for PDF generation (via pandoc)

(which is way better than 1.9 of the previous raspian, and even beats the instructions for compiling 1.11.1)

MORE

sudo apt-get install pandoc texlive-latex-recommended texlive-fonts-recommended 

Was able to:

pandoc --from markdown hello.markdown -o hello.pdf

Access point

Taken from this "ultimate" guide

apt-get install dnsmasq wireless-tools hostapd

# the next wasn't necessary for jessie, but for completeness..
RPI3 broadcom chip 
apt-get install firmware-brcm80211
rmmod brcmfmac
modprobe brcmfmac

Give fixed IP to wlan0 interface, edit /etc/network/interfaces switch off the built in stuff and add (section 2):

auto eth0
allow-hotplug eth0
iface etho inet dhcp

#################################
# 1. ORIGINAL settings... use wpa_supplicant for client mode
#allow-hotplug wlan0
#iface wlan0 inet manual
#    wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf
#
#################################
# 2. Fixed IP address (for hotspot / hostapd)
auto wlan0
iface wlan0 inet static
address 10.9.8.7
netmask 255.255.255.0
#################################

Replace /etc/dnsmasq.conf with:

interface=wlan0
dhcp-range=10.9.8.10,50.9.8.254,12h
address=/#/10.9.8.7
no-resolv

Edit /etc/hostapd/hostapd.conf file (adjust depending on driver/hardware)

interface=wlan0
driver=nl80211
ssid=WiFeels
hw_mode=g
channel=6

Edit /etc/default/hostapd and add

DAEMON_CONF="/etc/hostapd/hostapd.conf"

Make hostapd start at boot

update-rc.d hostapd defaults

Reboot.

Makeserver + etherpad (experimental!)

Ingredients

• Etherdump's pad.html with <rel> links including LIVE EDIT URLs
• THIS should replace/complement makeserver's EDIT button

Key question: Makeserver as a separate view ?! (probably)

TRY as 2 separate things ... basically AS IS...

Install:

cd /home/pi/software
git clone http://murtaugh@gitlab.constantvzw.org/aa/makeserver.git
cd makeserver
git submodule init
git submodule update
sudo pip install twisted jinja2

BUGFIX with twisted / SSL issues:

sudo pip install twisted[tls]

DIDN"T FIX sudo pip install twisted==16.0.0

Seems to work!

TODO

• Why are the links hardcoded long form in etherdump index (fails then via makeserver based in home)?

Extra

Changed dnsmasq.conf

interface=wlan0
dhcp-range=10.9.10.50,10.9.10.254,12h
#address=/#/10.9.10.7                                                                                                           # no-resolv

AND added to /etc/hosts

10.9.10.7 etherbox.local

and this seems to then work over the direct hotspot connection.