Saddest Programming Concept Ever

Python has spoiled me for other languages – I accept that – but I still wasn’t fully prepared for some of the horrors I discovered in Javascript. Which made the satiric article by James Mickens, “To Wash It All Away“, all the more enjoyable. Here is a slice I especially liked:

Much like C, JavaScript uses semicolons to terminate many kinds of statements. However, in JavaScript, if you forget a semicolon, the JavaScript parser can automatically insert semicolons where it thinks that semicolons might ought to possibly maybe go. This sounds really helpful until you realize that semicolons have semantic meaning. You can’t just scatter them around like you’re the Johnny Appleseed of punctuation. Automatically inserting semicolons into source code is like mishearing someone over a poor cell-phone connection, and then assuming that each of the dropped words should be replaced with the phrase “your mom.” This is a great way to create excitement in your interpersonal relationships, but it is not a good way to parse code. Some JavaScript libraries intentionally begin with an initial semicolon, to ensure that if the library is appended to another one (e.g., to save HTTP roundtrips during download), the JavaScript parser will not try to merge the last statement of the first library and the first statement of the second library into some kind of semicolon-riven statement party. Such an initial semicolon is called a “defensive semicolon.” That is the saddest programming concept that I’ve ever heard, and I am fluent in C++.

Nice deal on programming books

First a disclosure – I will be getting two free e-books for promoting the Packt Sale ;-). But I wouldn’t bother writing this up unless I thought Packt books would be of some value to me – so that makes it a genuine endorsement. For reference, here are the three I’m weighing up:

  1. Learning IPython for Interactive Computing and Data Visualization
  2. Git: Version Control for Everyone
  3. Responsive Web Design with jQuery

Buy One - Get One Free

Apparently the deal is:

  • Unlimited purchases during the offer period
  • Offer is automatically applied at checkout

I procrastinated a bit so there are only a couple of days left (ends 26th March). So be fast! Here’s the promotion link

Installing iNZight on Ubuntu (13.10)

Existing documentation for installing iNZight on Ubuntu covers a wide range of possible scenarios in Step 1. For my own specific case, however, I needed to translate it into a series of precise step-by-step instructions.

So here are the instructions that worked for me for Step 1. Once that is done, just follow steps 2 onwards in the official docs.

STEP 1 STEP-BY-STEP

It is probably best you don’t already have R already installed. And if you’re using iNZight, you probably won’t. But if you do, purge it! Installation may or may not work depending on the version of R you install so remove one source of difference from what the support people will have tested.

  1. ctrl-alt-t to open terminal window
  2. sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9
  3. sudo gedit /etc/apt/sources.list

    and then append

    deb-src http://cran.stat.auckland.ac.nz/bin/linux/ubuntu saucy/

    Note: must have trailing slash (/) at end
  4. Click on “Save” button on toolbar or enter ctrl s on keyboard to save
  5. sudo apt-get update
  6. sudo apt-get install r-base
  7. Open R by typing R into terminal
  8. From R prompt:
    update.packages(ask = FALSE)

    y to everything

  9. q("no")

    to quit the R prompt.

  10. Then follow steps 2 onwards in the official instructions …

USB 2 on VirtualBox

Get the appropriate version of the VirtualBox extension pack installed – I found the correct version in here: Download VirtualBox (Old Builds): VirtualBox 4.2

Add self to vboxusers group as per Set up USB for Virtualbox. Must reboot for change to take effect.

sudo usermod -aG vboxusers

Then follow the additional steps. For Windows XP at least, must add a filter before opening so that the USB is recognised by the File Manager and displayed as an extra drive.

Dual boot Samsung 700 Ubuntu / Win8

My Mother found Windows 8 very bewildering (actually, so did I) and asked me to install Ubuntu (13.10) instead on her Samsung 700 tablet/notebook. I thought I’d set it up dual boot so I could occasionally use Windows 8 for testing purposes and in case there was any sofware that we needed in the future that was Windows only.

  1. Power off
  2. Put bootable Ubuntu USB in
  3. Boot but hold down F2
  4. In the BIOS config (it was touchscreen which was quite nice):
    • Advanced:
      • Fast BIOS Mode: Off
    • Boot:
      • Secure Boot: Off
      • OS Mode Selection: CSM and UEFI OS
      • PXE boot: On
  5. Save>Yes
  6. Reboot but hold down F2
  7. Boot to Device now showed: UEFI: DSE MicroDrive 2GB 1.00 (or however your USB identifies itself)
  8. Boot into Ubuntu installer
  9. Shrink Windows partition (the NTFS one)
  10. Add a large Ext4 partition for Ubuntu and a smaller swap partition
  11. Install Ubuntu onto the new Ext4 partition with / as mount point
  12. Reboot and set everything up. Success :-).

My favourite eclipse settings

Preferences>PyDev>Editor>Code Style>Code Formatter – Tick “Right trim lines?”

Preferences>PyDev>Editor>Typing – untick After ‘(‘ indent to its level (indents by
tabs if unchecked)

Number of indentation levels to add: 1

Switch pylint off so pydev code analysis can do its job:

Preferences>PyDev>Editor>PyLint Untick “Use pylint?”.

Preferences>General>Editors>Text Editors>Annotations>Occurences (PyDev) set to #6BF459 so bright enough to see when very small but can read text on top.

Preferences>General>Editors>Text Editors>

Displayed tab width: 4

Tick “Insert spaces for tabs”
Tick “Show print margin” and set to 80.

Tick “Show line numbers”.

Rotate video (and cope with frame rate problem)

The problem – I had taken a video on a smart phone upside down and needed to rotate it (Note – rotating is not the same as flipping). Here is the successful command I used on Ubuntu:

avconv -i "my_input_video.mp4" -r 25 -vf transpose=1,transpose=1 my_output_video.mp4

Note – I needed to rotate it so chaining two 90 degree transpose commands (transpose=1 is a single 90 degree transposition) worked (see How can I rotate video by 180 degrees with avconv).

Note the -r 25. I don’t pretend to understand all the options of ffmpeg/avconv but the -r 25 forced the conversion to use a manageable number of frames. Without it, converting one particular video was failing, thrashing the CPU, and taking forever. There was a message in the ffmpeg/avconv output about MB rate > level limit. Another video worked fine without -r 25 but if you have the same problem with MB rate give -r 25 a try. Or better yet, learn more about what you’re doing ;-). I didn’t have the time or inclination so tried to do the least I could to get a good result.

Quick MPG4 to AVI conversion

The problem – my wife had an mpg4 but it wouldn’t play on the machine upstair. Needed an AVI instead. I couldn’t be bothered learning all the various settings of ffmpeg (libav is a fork) but found an acceptable (albeit inefficient) way of getting the same quality in the AVI as in the MPG4. OK, the file size was 3-4x larger, so I would recommend optimising things if doing multiple conversions, but this is a good and simple answer for the occasional one-off.

ffmpeg -i original.mp4 -sameq converted_version.avi

Misc Moonpig Provocation

Mark Dominus writes a thought-provoking and entertaining piece on Moonpig. Here are some snippets I found especially interesting and some brief comment:

On ORMs and relational databases:

Right now the principal value of ORM software seems to be if your program is too fast and you need it to be slower; the ORM is really good at that. Since speed was the only benefit the RDB was providing in the first place, you have just attached two large, complex, inflexible systems to your program and gotten nothing in return.

Can’t say I agree, as I find a lot of value in relational databases irrespective of speed (transactions, incredible flexibility of analysis), but still worth mulling over.

On built it yourself versus reuse:

What we should have done, instead of building our own object store, was use someone else’s object store. KiokuDB is frequently mentioned in this context. After I first gave this talk people asked “But why didn’t you use KiokuDB?” or, on hearing what we did do, said “That sounds a lot like KiokuDB”. I had to get Rik to remind me why we didn’t use KiokuDB. We had considered it, and decided to do our own not for technical but for political reasons. The CEO, having made the unpleasant decision to have me and Rik write a new billing system, wanted to see some progress. If she had asked us after the first week what we had accomplished, and we had said “Well, we spent a week figuring out KiokuDB,” her head might have exploded. Instead, we were able to say “We got the object store about three-quarters finished”. In the long run it was probably more expensive to do it ourselves, and the result was certainly not as good. But in the short run it kept the customer happy, and that is the most important thing; I say this entirely in earnest, without either sarcasm or bitterness.

(On the other hand, when I ran this article by Rik, he pointed out that KiokuDB had later become essentially unmaintained, and that had we used it he would have had to become the principal maintainer of a large, complex system which which he did not help design or implement. The Moonpig object store may be technically inferior, but Rik was with it from the beginning and understands it thoroughly.)

Interesting to note there is a genuine trade off with uncertainty in it. If you rely on something external, you may spend longer trying to find the right input to your project and it may be abandoned after you have heavily invested in it. A lot depends on the reusable component.

And finally, on inheritance:

  1. Object-oriented programming is centered around objects, which are encapsulated groups of related data, and around methods, which are opaque functions for operating on particular kinds of objects.
  2. OOP does not mandate any particular theory of inheritance, either single or multiple, class-based or prototype based, etc., and indeed, while all OOP systems have objects and methods that are pretty much the same, each has an inheritance system all its own.
  3. Over the past 30 years of OOP, many theories of inheritance have been tried, and all of them have had serious problems.
  4. If there were no alternative to inheritance, we would have to struggle on with inheritance. However, Roles are a good alternative to inheritance:
    • Every problem solved by inheritance is solved at least as well by Roles.
    • Many problems not solved at all by inheritance are solved by Roles.
    • Many problems introduced by inheritance do not arise when using Roles.
    • Roles introduce some of their own problems, but none of them are as bad as the problems introduced by inheritance.
  5. It’s time to give up on inheritance. It was worth a try; we tried it as hard as we could for thirty years or more. It didn’t work.
  6. I’m going to repeat that: Inheritance doesn’t work. It’s time to give up on it.

I have used inheritance successfully to prevent repeated code but it has only really fitted a few (important) places. Delegation is more flexible than inheritance, but inheritance can be a natural fit for some problems.

Anyway, an interesting read with lots more in it than I’ve commented on (e.g. avoiding floating point rounding errors etc). Read it yourself.

Don’t need no stinkin’ GROUP_CONCAT

One of the things I loved about MySQL was GROUP CONCAT. A very, very useful function. I thought it was something I was going to have to learn to live without having shifted to PostgreSQL. But PostgreSQL has STRING_AGG() (string aggregate) which does much the same thing – who knew?! Others have also been surprised..

Here is an example:

SELECT country, string_agg(city, ‘, ‘ ORDER BY city DESC)
FROM cities
WHERE country = ‘New Zealand’
GROUP BY country;

>> “New Zealand”, “Whangarei, Whanganui, Wellington, Tauranga, Taupo, Rotorua, Palmerston North, New Plymouth, Nelson, Napier, Invercargill, Hamilton, Gisborne, Dunedin, Christchurch, Auckland”

Note the odd location of ORDER BY, and the absence of a comma between the ORDER BY and the previous argument! See Aggregate Expressions