Daily Python

Thursday, August 18, 2016

Raspberry Pi Experiments - Remote Web Cam

One of my initial ideas for using Raspberry Pi was to remotely view my sons activities when ever I was away from him. So as a start I checkout my old web cam (which is a Zebatronics Ecam 2009 model) it was working and finally followed the steps in the below link to setup the webcam in my raspberry pi.
https://www.raspberrypi.org/documentation/usage/webcams/

Here I will give a brief rundown of steps that I did to setup the Web Cam.

1. Check if your web cam is a USB Webcam and the make, mine was Zebatronics Ecam 2009 model
2. Connect it to the Raspberry Pi USB port and see if the lights of the Web Cam are glowing.
3. Check if the Web Cam is detected using lsusb For me it shows the below for the webcam:

pi@raspberrypi:~ $ lsusb
Bus 001 Device 004: ID 0c45:614a Microdia

Note: The trick to know that if webcam is connected is to lsusb before and after connecting the webcam if its detected an additional USB device will show. Additionally in /dev folder check if there any device like /dev/video0

pi@raspberrypi:~ $ ls /dev/video*
/dev/video0

4. Install fswebcam and check if the webcam capture works sudo apt-get install fswebcam

pi@raspberrypi:~/webcam $ fswebcam image.jpg
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
Adjusting resolution from 384x288 to 352x288.
--- Capturing frame...
Captured frame in 0.00 seconds.
--- Processing captured image...
Writing JPEG image to 'image.jpg'.

5. Next I developed a script to capture the image and save to a folder /home/pi/webcam

mkdir /home/pi/webcam

Created a script webcam.sh

#!/bin/bash

DATE=$(date +"%Y-%m-%d_%H%M")

fswebcam -r 1280x720 --no-banner /home/pi/webcam/$DATE.jpg

 6. Then setup cron to capture images every minute:

crontab -e

Added this line in the editor:

* * * * * /home/pi/webcam.sh 2>&1

7. The output can be seen as below where the script runs every one minute:

pi@raspberrypi:~ $ ls ~/webcam2016-08-18_1644.jpg 2016-08-18_1707.jpg 2016-08-18_1730.jpg2016-08-18_1645.jpg 2016-08-18_1708.jpg 2016-08-18_1731.jpg2016-08-18_1646.jpg 2016-08-18_1709.jpg 2016-08-18_1732.jpg2016-08-18_1647.jpg 2016-08-18_1710.jpg 2016-08-18_1733.jpg2016-08-18_1648.jpg 2016-08-18_1711.jpg 2016-08-18_1734.jpg2016-08-18_1649.jpg 2016-08-18_1712.jpg 2016-08-18_1735.jpg2016-08-18_1650.jpg 2016-08-18_1713.jpg 2016-08-18_1736.jpg2016-08-18_1651.jpg 2016-08-18_1714.jpg 2016-08-18_1737.jpg2016-08-18_1652.jpg 2016-08-18_1715.jpg 2016-08-18_1738.jpg2016-08-18_1653.jpg 2016-08-18_1716.jpg 2016-08-18_1739.jpg2016-08-18_1654.jpg 2016-08-18_1717.jpg 2016-08-18_1740.jpg2016-08-18_1655.jpg 2016-08-18_1718.jpg 2016-08-18_1741.jpg2016-08-18_1656.jpg 2016-08-18_1719.jpg 2016-08-18_1742.jpg2016-08-18_1657.jpg 2016-08-18_1720.jpg 2016-08-18_1743.jpg2016-08-18_1658.jpg 2016-08-18_1721.jpg 2016-08-18_1744.jpg2016-08-18_1659.jpg 2016-08-18_1722.jpg 2016-08-18_1745.jpg2016-08-18_1700.jpg 2016-08-18_1723.jpg 2016-08-18_1746.jpg2016-08-18_1701.jpg 2016-08-18_1724.jpg 2016-08-18_1747.jpg2016-08-18_1702.jpg 2016-08-18_1725.jpg 2016-08-18_1748.jpg2016-08-18_1703.jpg 2016-08-18_1726.jpg 2016-08-18_1749.jpg

8. Finally I created a script to generate a time lapse video of these images, for this first we need to install avconv which is a fork of ffmpeg.

pi@raspberrypi:~ $ sudo apt-get install avconv
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package avconv

Then created a script conv2vid.sh to generate the time lapse video from the images and flush the images once the video is created:

#!/bin/bash
DATE=$(date +"%Y-%m-%d_%H%M")
mv /home/pi/webcam/*.jpg /home/pi/webcam/video
cat /home/pi/webcam/video/*.jpg | avconv -r 10 -f image2pipe -codec:v mjpeg -i --pix_fmt yuvj420p -r 30 -c:v libx264 -vf scale=480:300 -y /home/pi/webcam/video/$DATE.mp4
rm /home/pi/webcam/video/*.jpg

Please find the below video for the Time lapse video

Monday, August 8, 2016

Raspberry Pi 3 Tricks - Connecting USB

Raspberry PI with 32GB Sandisk USB Stick

One of the first use cases that I found after setting up my Raspberry PI was connecting my USB Sticks. Now in home I want to access my USB flash storage stick anytime and without any issue and Raspberry PI seems to be a great way of doing it. The short steps are give below:

1. For using any standard USB stick to Raspberry Pi's USB socket
2. install pmount on Raspberry Pi using sudo apt-get install pmount
3. Use sudo fdisk -l to find your USB drive, it may be /dev/sda1
4. Mount the USB to your OS using sudo pmount /dev/sda1 it shoud mount under /media/sda1
5. After you have verified the mount and used you can unmount the USB sudo pumount /dev/sda1

Note: Before getting the USB stick out make sure to unmount the drive.

I tried loading a 1TB hard disk but I guess it will need its own dedicated power, will post it in another post.

Terminal Cast showing how to Mount USB

Sunday, August 7, 2016

Raspberry pi 3 mod b unboxing and demo

In my latest experiment, I bought my first Raspberry PI 3 Mod B, and tried setting it up. Additionally I got a transparent casing and 16GB Sandisk Micro SD with the OS installed.

Now coming to the unboxing and setting up Raspberry PI always precision screw driver helps which I additionally bought and for power I reused one of my 5V Android mobile chargers, this type of charger is specifically suitable for Raspberry PI power requirements but 5.25 V micro usb charger is recommended. The particular case that I got from Amazon was very helpful as it was transparent and all the ports in Raspberry were readily accessible and was very easy to setup.

When I put the micro SD for some reason it did not start, hence I downloaded the latest Raspbian lite OS SD image from Raspberry website and flashed it to the Micro SD with the help of Win32Image Write on Windows. Initially I used Wired LAN to connect to Raspberry, the Wireless setup for my Raspberry I will share in a separate post. The whole process is captured in my below video.

Useful links:
Source for Raspbian OS
https://www.raspberrypi.org/downloads/raspbian/

Source for power
https://www.raspberrypi.org/help/faqs/#power

Friday, July 15, 2016

IOT Experiments with QEMU, ARM, Debian & Docker

Background

I wanted to explore how can we run docker on Raspberry PI as I myself do not have a physical PI device. I checked out how can we emulate it in my Windows PC. Subsequently by googling I found about this link which explains about hypriot. These folks are really great folks as they have done a very great thing on ARM systems which was previously missing, getting to run docker on ARM devices. They have already provided .img images that can be cloned onto a SD card and can be run on Raspberry PI.

But as I wanted to emulate it I cam across this link that talked about a beta program from docker where you can directly run ARM docker images on Windows using “Docker for Windows” which uses exactly the same Alpine Linux in a Hyper-V virtual machine. And to run the hypriot docker image you need QEMU. As I wanted to avoid using an additional layer of VM. I directly wanted to try running the hypriot raw image on Windows using QEMU. But while searching for any good resources for this requirement I did not find any consolidated answer and in the end did a one week of R&D to figure out how to achieve it. This Blog post is a consolidations of those experiments.

P.S. Rest of my blog post provides the steps and the resources that I explored to successfully run this experiment and is heavily inspired from blog post by Franck Besnard titled "ARMv7 Docker emulator in QEMU" . This blog post talks in great detail on how to setup Docker on a Cortex-A15 processor using QEMU using Debian kernel on linux. My post will focus on how to how I did the same thing in Windows.

Prelude - My experiments Hypriot Image

1. I downloaded QEMU for windows from https://qemu.weilnetz.de/w32/ I downloaded the latest version and installed it.

2. I downloaded the latest version of http://blog.hypriot.com/downloads/ where I downloaded the image for Version 0.8.0 Barbossa hypriotos-rpi-v0.8.0.img.zip and unzipped to a local directory and renamed the .img file to hypriotos.img.

3. Then I downloaded the latest kernel-qemu from https://github.com/dhruvvyas90/qemu-rpi-kernel , this is required to boot the hypriotos image in Qemu. Since the latest version of hypriotos is based on Debian Jessie hence this link has some useful kernel-qemu kernels for Debian Jessie and Wheezy. Please check the tools folder in the above link if you want to build your own kernel-qemu. Next I put these kernel files in the same folder as the hypriotos.img file.

4. I triggered qemu with the below command.

qemu-system-arm.exe -M versatilepb -cpu arm1176 -kernel kernel-qemu-4.1.13-jessie -hda 2016-05-27-raspbian-jessie-lite.img -m 256 -append "root=/dev/sda2" -net nic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::22280-:80

But I ended with the below case where OS was running in emergency mode and docker was not running

4. While exploring for the fix I came across "http://besn0847.blogspot.in/2016/01/armv7-docker-emulator-in-qemu.html" and hence thought to give it a try.

Setting the Stage - So What is required

This is my Setup :

1. I have Windows 7, 8 GB RAM, 500 GB Disk Laptop.
2. QEMU emulator (personally i have version 2.3.0)

3.Few files from a debian mirror site (see below)

Step 1: Downloading the Images

1.Collect the kernel and boot files (vmlinuz & initrd) needed to launch a network install (you can also download via a browser)

curl -O http://ftp.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/initrd.gz

curl -O http://ftp.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/vmlinuz

curl -O http://ftp.nl.debian.org/debian/dists/jessie/main/installer-armhf/current/images/device-tree/vexpress-v2p-ca15-tc1.dtb

2.Create a disk which will host your filesystem where there will be 3 partitions : /boot, / and swap

qemu-img create -f qcow2 debian-jessie-armhf.img 8G

Step 2: Install the Operating System

1. Start the QEMU emulator of the Cortex-A15 chip

qemu-system-arm -m 2047M -M vexpress-a15 -dtb vexpress-v2p-ca15-tc1.dtb -cpu cortex-a15 -kernel vmlinuz -initrd initrd.gz -sd debian-jessie-armhf.img -append "root=/dev/ram console=ttyAMA0 earlycon"

Few things about the options here:
-cpu cortex-a15 represents the Cortex-A15 chip, it is still an ARMv7 architecture which is the same as the Raspberry Pi 2 and there is no better option in QEMU AFAIK.

-kernel vmlinuz -initrd initrd.gz - represent the kernel & ram disk for the initial boot which we had downloaded earlier.

-append "root=/dev/ram console=ttyAMA0 earlycon" - Here root=/dev/ram means that the boot will happen from the ram disk for the installations & console=ttyAMA0 means that the boot screen will show in serial0 screen on QEMU

2. Proceed to the Debian install - please note this takes few hours to complete; here are my basic settings

Use the whole disk with default install settings
Create a debian user
Also set the root user & host details
Just install SSH and the minimal system tools if possible or install the defaults as directed during the installation.
I had few screens where the default screen was No, to change to yes I used ctrl-alt-h which magically simulated the tab.

After the installation ideally you should see the below screen, after this its better to restart and follow the next steps:

3. Now the operating system has been installed it is now time to finalize the boot part by extracting the vmlinuz and initrd files from the SD card image

On windows I had a particular issue that I was not able to mount partitions from debian-jessie-armhf.img and copy the vmlinuz & initrd files. Hence I installed Peazip from http://www.peazip.org/. One of the advantages of this utility is the ability to extract data from many types of image formats like (ISO, DMG, UDF, VMDK, WMI...).

The files can be seen below

We need to extract vmlinuz-3.16.0-4-armmp-lpae & initrd.img-3.16.0-4-armmp-lpae

4. Restart the emulator with the new boot files with the below command

qemu-system-arm -m 1024M -M vexpress-a15 -dtb vexpress-v2p-ca15-tc1.dtb -cpu cortex-a15 -kernel vmlinuz-3.16.0-4-armmp-lpae -initrd initrd.img-3.16.0-4-armmp-lpae -sd debian-jessie-armhf.img -append "root=/dev/mmcblk0p2 console=ttyAMA0" -net nic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::22280-:80

let me explain some of these options:

-kernel vmlinuz-3.16.0-4-armmp-lpae -initrd initrd.img-3.16.0-4-armmp-lpae , this line represents ths kernel & the init ram disk we have extracted from sd image debian-jessie-armhf.img

-append "root=/dev/mmcblk0p2 console=ttyAMA0" - Defines that the system's boot file system is /dev/mmcblk0p2 present in debian-jessie-armhf.img and not /dev/sda2

-net nic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::22280-:80 this is to map 22 port in QEMU VM to 2222 port of host system for ssh and map port 80 of QEMU to 22280 port of host system for http.

You should be able to login with the username and password set during the installations. Please check the image below:

Step 3: Install docker

Please note that all the actions below are performed as root.

1. First update your Debian operating system

apt-get update && apt-get upgrade

2. Then install the Docker 1.9.1 package from Hypriot (it will return errors but don't worry at this stage)

curl -O http://downloads.hypriot.com/docker-hypriot_1.9.1-1_armhf.deb
dpkg -i docker-hypriot_1.9.1-1_armhf.deb

3. The vanilla Jessie kernel does not support the Overlay filesystem, so just change the following line in /etc/default/docker :

From : DOCKER_OPTS="--storage-driver=overlay -D"

To : DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"

4. You also need to change the Docker service file in /lib/systemd/system/docker.service :

From : ExecStart=/usr/bin/docker daemon -H fd:// $DOCKER_OPTS

To : ExecStart=/usr/bin/docker dameon -H unix:///var/run/docker.sock $DOCKER_OPTS

5. Add your debian user to the docker group

e.g usermod -a -G docker pi

6.Restart Docker after reloading and enabling

systemctl daemon-reloadsystemctl enable docker.servicesystemctl start docker.service

Now you should be right there :

# docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES

7. Try out docker run

Just type docker run into the terminal of your Raspberry Pi:

docker run -d -p 80:80 hypriot/rpi-busybox-httpd

This command will download and start the Docker image hypriot/rpi-busybox-httpd which contains a tiny web server. Once an image is started it is called a container.

An image can also be used to start multiple containers. You can check if your container is running by typing

docker ps

The output should be something like below:

# docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

8ccb6f25d559 hypriot/rpi-busybox-httpd "/bin/busybox httpd -" 6 minutes ago Up 6 minutes 0.0.0.0:80->80/tcp condescending_panini

While checking on the host system it should look like this:

Conclusion

In retrospect it was a learning experience and brings me closer to running hypriotos on QEMU while I will revisit in my next blog post.

Few references that will be helpful:

Very good blog post which is the inspiration for this experiment: http://besn0847.blogspot.in/2016/01/armv7-docker-emulator-in-qemu.html
Useful Kernel QEMU images and process to how to build them https://github.com/dhruvvyas90/qemu-rpi-kernel
Reference site about installing Debian Jessie on ARM : http://bit.ly/1S093w8
Reference site to download QEMU: https://qemu.weilnetz.de/w32/
Reference site to Peazip http://www.peazip.org/iso-image-files.html
Reference site to download hypriot os images, docker deb package & docker images http://blog.hypriot.com/downloads/
Finally this blog post how to run docker on ARM: http://blog.hypriot.com/getting-started-with-docker-on-your-arm-device