Messed up Thunderbird folders – sharks circling

We’ve all done it – messed up something so badly while trying to do something clever we’d sell our first-born just to get back to where we were (see And if we succeed at merely restoring the status quo, we’re pitifully grateful.

It all started with trying to find some lost photos that Shotwell couldn’t find the originals of. Presumably they had been linked to only and then the originals deleted leaving only the reference and the thumbnail behind. So here was the plan:

  1. Identify all email attachments which are images
  2. See if any of them have the same name as the missing images
  3. Open email based on date and sender to recover image

The good news is the plan worked for lots of the missing photos. Thanks to Python3, import mailbox, and import email. The bad news was when I opened Thunderbird the next day. The folder I had been working on was missing. So in addition to my missing photos I also had seemingly lost 1.8GB of emails.

tldr; 1) Close TB; 2) rename the missing folder in your file system and delete the .msf version; 3) Open TB; 4) Close TB; restore original name; 5) Open TB – success? Inspired by

Now back to the original problem. But I should probably run a full backup first.

Installing wireless USB modem driver on Ubuntu 15.10

It really was this simple:

How to install D-Link DWA-182 Wireless AC1200 Dual Band USB Adapter on Linux Ubuntu

Download zip from here:

rtl8812AU_8821AU_linux on GitHub

Unzip folder e.g. as “/home/mythbuntu/rtl8812AU_8821AU_linux-master”

cd "/home/mythbuntu/rtl8812AU_8821AU_linux-master"
make clean
sudo make uninstall
sudo make install

Restart and enable wireless and enter password when prompted (after selecting your own ssid).

Trouble copying audio file – until rdd-copy

I had trouble copying a audio file (wav) from a CD to my computer – the copy process was always getting stuck at exactly the same point 153MB in. And it didn’t matter whether I was using sound-juicer or nautilus. The answer was to install rdd (sudo apt-get install rdd). rdd copes with errors by supplying blanks (assuming multiple careful attempts to read the data have all failed) rather than halting.

The required command was rdd-copy src dest. But what to supply as src? I tried /media and similar but no luck. The final answer was ‘/run/user/1000/gvfs/cdda:host=sr0/Track 3.wav’. But how to find it? Just drag the file from nautilus to the terminal and see what is displayed there. The following worked even though it took a long time to get the file:

rdd-copy '/run/user/1000/gvfs/cdda:host=sr0/Track 3.wav' /home/g/Desktop/track3.wav

Simple flask app on heroku – all steps (almost)

Note – instructions assume Ubuntu Linux.

See Getting Started with Python on Heroku (Flask) for the official instructions. The instructions below tackle things differently and include redis-specific steps.

Don’t need postgresql for my app even though needed for heroku demo app. Using redis for simple key-value store.

Main reason for each step is indicated in bold at start. There are lots of steps but there are lots of things being achieved. And each purpose only requires a few steps so probably hard to streamline any further.

    >> sudo apt-get install python3 python3-pip python-virtualenv git ruby redis-server redis-tools
    Get free heroku account
    Install heroku toolbelt Heroku setup. Sets up virtualenvwrapper for you too (one less thing to figure out)
    Do the once-ever authentication
    >> heroku login
  5. APP
    Make project folder e.g.
    >> mkdir ~/projects/myproj
  6. APP
    >> cd ~/projects/myproj
    >> echo “web: python” > Procfile
    >> git init
    >> mkvirtualenv sticky

    So requirements for specific project can be separated from other project – lets heroku identify actual requirements. Normally “workon sticky” thereafter; deactivate to exit virtual env

  10. APP
    >> pip3 install flask
    Note – installed within virtualenv
  11. HEROKU
    Save the following as requirements.txt – needed by heroku so it knows the dependencies. Update version of redis as appropriate. gunicorn is a better approach than the flask test server
  12. HEROKU
    So we can use Python 3.4 instead of the current default of 2.7:
    >> echo “python-3.4.3” > runtime.txt
  13. APP & HEROKU

    Make a toy app to get started from.

    Note – modify the standard demo flask app to add a port to ease eventual heroku deployment. Otherwise the app will fail because of a problem with the port when running

    heroku ps:scale web=1

    Starting process with command `python`
    Web process failed to bind to $PORT within 60 seconds of launch

    Here is an example (may need updating if flask changes):

    import os
    from flask import Flask
    app = Flask(__name__)

    def hello():
        return "Hello World!"

    if __name__ == "__main__":
        port = int(os.environ.get("PORT", 33507))'', port=port)

    >> deactivate
  15. Make a module to make it easier to work with redis – let’s call it

    import os
    import urllib
    import redis

    url = urllib.parse.urlparse(os.environ.get('REDISTOGO_URL',
    redis = redis.Redis(host=url.hostname, port=url.port, db=0,

    We can then use redis like this:
    from store import redis

  16. APP
    Keep building app locally. The following is good for redis: Redis docs. And flasks docs are always good: Flask Docs – Minimal Application

    Before deploying to production:

    1. Update git otherwise you’ll be deploying old code – heroku uses git for deployment
    2. set app.debug to False (although no rush when just getting started and not expecting the app to get hit much)
    3. probably switch to gunicorn sooner or later (will need to change ProcFile to
      web: gunicorn main:app --workers $WEB_CONCURRENCY
    4. Example nginx.conf:

      # As long as /etc/nginx/sites-enable/ points to
      # this conf file nginx can use it to work with
      # the server_name defined (the name of the file
      # doesn't matter - only the server_name setting)
      # sudo ln -s /home/vagrant/src/nginx.conf ...
      #     ... /etc/nginx/sites-enabled/
      # Confirm this link is correct
      # e.g. less /etc/nginx/sites-enabled/

      server {
          listen 80;
          server_name localhost;

          location /static { # static content is

              # handled directly by NGINX which means
              # the nginx user (www-data) will need
              # read permissions to this folder
              root /home/vagrant/src;


          location / { # all else passed to Gunicorn to handle
              # Pass to wherever I bind Gunicorn to serve to
              # Only gunicorn needs rights to read, write,
              # and execute scripts in the app folders

    5. Example gunicorn.conf
      import multiprocessing

      bind = "" # ensure nginx passes to this port
      logfile = "/home/vagrant/gunicorn.log"
      workers = multiprocessing.cpu_count() * 2 + 1

  18. HEROKU
    >> heroku create

    Should now be able to browse to the url supplied as stdout fom command e.g. Note – not working yet – still need to deploy to new app

    >> git push heroku master

    Must then actually spin up the app:

    >> heroku ps:scale web=1

    A shortcut for opening is

    >> heroku open

  19. HEROKU
    Add redis support (after first deployment – otherwise

    ! No app specified.
    ! Run this command from an app folder or specify which app to use with --app APP.
    >> heroku addons:create redistogo

    Note – need to register credit card to use any add-ons, even if free ones. Go to

Some other points: when developing on a different machine, I needed to supply my public key to heroku from that other machine (Permission denied (publickey) when deploying heroku code. fatal: The remote end hung up unexpectedly).

heroku keys:add ~/.ssh/

And the full sequence for upgrading your app after the prerequisites have been fulfilled is:

  1. git commit to local repo
  2. Then git push to heroku
  3. Then run heroku ps:scale web=1 again

And I had a problem when I switched from Python 2 to 3 with redis – my heroku push wouldn’t work. By looking at the logs (>> heroku logs –tail) I found that import imap wouldn’t work and searching on that generally found I needed a newer version of redis than I had specified foolishly in requirements.txt.

F-spot vanished in Ubuntu 15.04 (Vivid)

F-spot has been removed from Ubuntu Vivid (15.04).

Dependency is not satisfiable: liblcms1 (>= 1.15-1)

None of the data for f-spot was gone, just the ability to run the application – probably something to do with mono library deprecation.

~$ find / -name f-spot 2>/dev/null

1.3MB in /home/g/.config/f-spot/photos.db

Anyway, opened Shotwell, “Import from Application”, “Import media from: F-Spot”, moving on.

Printer driver for Canon MG-7100 on Ubuntu

Once again I was solving a computer problem for my parents-in-law ;-). This time, it was yet another new printer they had bought – a Canon MG7100. Usually, I have had a really good experience with modern Ubuntu and popular printers. They Just Work. And this time it seemed things had gone well again. And they almost had. Except the colours were a bit off. On the Ubuntu printer test page Magenta was brown, bright green was instead a darker green, and yellow was very muddy – more like taupe. I wasted a lot of time cleaning ink nozzles etc etc but the actual solution was to choose a slightly different driver manually from the Canon list. There were two v4.0 options and it was the second that worked.

Category Good setting Bad setting
Job ID Canon-MG7100-2-671 Canon-MG7100-665
Driver CNMG7100.PPD STP00541.PPD
Driver Version 1.0 5.2.10-pre2
Description Canon MG7100 Canon MG7100
Driver Version Canon MG7100 Canon MG7100
Make and Model Canon MG7100 series Ver.4.00 Canon MG7100 series – CUPS+Gutenprint v5.2.10
Printer Canon-MG7100-2 Canon-MG7100

Hope this helps someone else.

Listing Python scripts changed most recently

find / -name *.py -type f -mtime -60 -printf '%TY-%Tm-%Td %TT %p\n' 2>/dev/null | sort -r

Command/options Explanation
find Not as fast as locate but has many advantages. See locate vs find: usage, pros and cons of each other
-name *.py Only find python scripts
-mtime -60 -mtime is days
-printf ‘%TY-%Tm-%Td %TT %p\n’ Displays output with easy-to-read dates
2>/dev/null Pumps all the annoying output about lacking permissions into /dev/null i.e. ignores them
| Pipes output through to be sorted
sort -r Sort so most recent are at top

Also see
How to find recently modified files on Linux

Complex good – complicated bad

In the Zen of Python we are taught that complex is better than complicated. Which is fair enough if we understand the terms as follows:

It is ok if something is complex so long as it is not complicated.

complex: composed of many interconnected parts; compound; composite

complicated: difficult to analyze or understand

Complex vs Complicated

Any decent web framework is going to be a bit complex becauise of all the moving parts it has to handle. But it should make sense and be logically structured enough to avoid being overly complicated.

Eclipse and PyDev on Utopic

I upgraded to Utopic (Utopic Unicorn a.k.a 14.10) and eclipse wouldn’t complete loading anymore. Solution:

Download latest plain vanilla Eclipse from the standard downloads page. And feel free to donate something too.

sudo su

chown -R root:root /home/username/eclipse && mv /home/username/Downloads/eclipse /opt

ln -s /opt/eclipse/eclipse /usr/local/bin/eclipse && exit

Start by running:


It didn’t even break PyDev so my luck’s finally turning ;-).