Rotate video (and cope with frame rate problem)

Posted on by

The problem – I had taken a video on a smart phone upside down and needed to rotate it (Note – rotating is not the same as flipping). Here is the successful command I used on Ubuntu:

avconv -i "my_input_video.mp4" -r 25 -vf transpose=1,transpose=1 my_output_video.mp4

Note – I needed to rotate it so chaining two 90 degree transpose commands (transpose=1 is a single 90 degree transposition) worked (see How can I rotate video by 180 degrees with avconv).

Note the -r 25. I don’t pretend to understand all the options of ffmpeg/avconv but the -r 25 forced the conversion to use a manageable number of frames. Without it, converting one particular video was failing, thrashing the CPU, and taking forever. There was a message in the ffmpeg/avconv output about MB rate > level limit. Another video worked fine without -r 25 but if you have the same problem with MB rate give -r 25 a try. Or better yet, learn more about what you’re doing ;-). I didn’t have the time or inclination so tried to do the least I could to get a good result.

Update – needed to alter aspect ratio as per:

avconv -i "my_image.MOD" -aspect 0.487 -r 25 -vf transpose=1,transpose=1,transpose=1 -strict -2 "my_image.mp4"

Quick MPG4 to AVI conversion

Posted on by

The problem – my wife had an mpg4 but it wouldn’t play on the machine upstair. Needed an AVI instead. I couldn’t be bothered learning all the various settings of ffmpeg (libav is a fork) but found an acceptable (albeit inefficient) way of getting the same quality in the AVI as in the MPG4. OK, the file size was 3-4x larger, so I would recommend optimising things if doing multiple conversions, but this is a good and simple answer for the occasional one-off.

ffmpeg -i original.mp4 -sameq converted_version.avi

Misc Moonpig Provocation

Posted on by

Mark Dominus writes a thought-provoking and entertaining piece on Moonpig. Here are some snippets I found especially interesting and some brief comment:

On ORMs and relational databases:

Right now the principal value of ORM software seems to be if your program is too fast and you need it to be slower; the ORM is really good at that. Since speed was the only benefit the RDB was providing in the first place, you have just attached two large, complex, inflexible systems to your program and gotten nothing in return.

Can’t say I agree, as I find a lot of value in relational databases irrespective of speed (transactions, incredible flexibility of analysis), but still worth mulling over.

On built it yourself versus reuse:

What we should have done, instead of building our own object store, was use someone else’s object store. KiokuDB is frequently mentioned in this context. After I first gave this talk people asked “But why didn’t you use KiokuDB?” or, on hearing what we did do, said “That sounds a lot like KiokuDB”. I had to get Rik to remind me why we didn’t use KiokuDB. We had considered it, and decided to do our own not for technical but for political reasons. The CEO, having made the unpleasant decision to have me and Rik write a new billing system, wanted to see some progress. If she had asked us after the first week what we had accomplished, and we had said “Well, we spent a week figuring out KiokuDB,” her head might have exploded. Instead, we were able to say “We got the object store about three-quarters finished”. In the long run it was probably more expensive to do it ourselves, and the result was certainly not as good. But in the short run it kept the customer happy, and that is the most important thing; I say this entirely in earnest, without either sarcasm or bitterness.

(On the other hand, when I ran this article by Rik, he pointed out that KiokuDB had later become essentially unmaintained, and that had we used it he would have had to become the principal maintainer of a large, complex system which which he did not help design or implement. The Moonpig object store may be technically inferior, but Rik was with it from the beginning and understands it thoroughly.)

Interesting to note there is a genuine trade off with uncertainty in it. If you rely on something external, you may spend longer trying to find the right input to your project and it may be abandoned after you have heavily invested in it. A lot depends on the reusable component.

And finally, on inheritance:

  1. Object-oriented programming is centered around objects, which are encapsulated groups of related data, and around methods, which are opaque functions for operating on particular kinds of objects.
  2. OOP does not mandate any particular theory of inheritance, either single or multiple, class-based or prototype based, etc., and indeed, while all OOP systems have objects and methods that are pretty much the same, each has an inheritance system all its own.
  3. Over the past 30 years of OOP, many theories of inheritance have been tried, and all of them have had serious problems.
  4. If there were no alternative to inheritance, we would have to struggle on with inheritance. However, Roles are a good alternative to inheritance:
    • Every problem solved by inheritance is solved at least as well by Roles.
    • Many problems not solved at all by inheritance are solved by Roles.
    • Many problems introduced by inheritance do not arise when using Roles.
    • Roles introduce some of their own problems, but none of them are as bad as the problems introduced by inheritance.
  5. It’s time to give up on inheritance. It was worth a try; we tried it as hard as we could for thirty years or more. It didn’t work.
  6. I’m going to repeat that: Inheritance doesn’t work. It’s time to give up on it.

I have used inheritance successfully to prevent repeated code but it has only really fitted a few (important) places. Delegation is more flexible than inheritance, but inheritance can be a natural fit for some problems.

Anyway, an interesting read with lots more in it than I’ve commented on (e.g. avoiding floating point rounding errors etc). Read it yourself.

Don’t need no stinkin’ GROUP_CONCAT

Posted on by

One of the things I loved about MySQL was GROUP CONCAT. A very, very useful function. I thought it was something I was going to have to learn to live without having shifted to PostgreSQL. But PostgreSQL has STRING_AGG() (string aggregate) which does much the same thing – who knew?! Others have also been surprised..

Here is an example:

SELECT country, string_agg(city, ‘, ‘ ORDER BY city DESC)
FROM cities
WHERE country = ‘New Zealand’
GROUP BY country;

>> “New Zealand”, “Whangarei, Whanganui, Wellington, Tauranga, Taupo, Rotorua, Palmerston North, New Plymouth, Nelson, Napier, Invercargill, Hamilton, Gisborne, Dunedin, Christchurch, Auckland”

Note the odd location of ORDER BY, and the absence of a comma between the ORDER BY and the previous argument! See Aggregate Expressions

Canonical – ignore the hate and make your bet

Posted on by

Does Canonical deserve the hate that is being directed at it by some people? I doubt it – who could? Canonical deserves criticism for some things, of course, but not the ongoing and disproportionate spite some technical people direct at it. The only explanation I can come up with is that when some people love or believe in something strongly, and then are disappointed in that love, they can feel very, very angry. And as with romantic love, nothing will make them feel better till they have a new love to take the place of the old one. So I’m not sure there is much Canonical can do to appease them. Instead, the focus should be on the big bets it has to make to secure a future. The following comment (on Slashdot) captured the issue especially well so I quote it in full:

It’s not rocket science. Rightly or wrongly Canonical has decided that the future of general computing is in the mobile space and they are working on getting Ubuntu there and bridging the gap between the mobile computing experience and the desktop computing experience.

In simplest of terms, they’re trying to make a distro that can be both a phone and a desktop all in the same device. Again — rightly or wrongly — they have decided that they needed to move certain things in house to best accomplish that goal (Mir) and needed a specific interface they were in control of to scale between display form factors (Unity).

If you are a person that thinks this direction is wrong and will hurt Linux in the long run, then you belong in the “bad for Linux” category. I’m a person that thinks this is absolutely the best way for Linux to finally have its “year of the desktop” similar to how Apple made their comeback but with a twist — by providing a compelling mobile experience with a device that just so happens to be able to double as someone’s desktop when they want a bigger screen.

Pay attention to plunging desktop sales numbers. As people find ways to make mobile devices and tablets their only computing devices, this strategy will start to look smarter and smarter. Whatever else you think of Canonical (and by extension Ubuntu), this will either make them or break them.

Ubuntu good for linux?

As with most big bets, there are no guarantees of success. And Canonical will make mistakes. But I fully support their attempt and wish them well.

Kernel panic fixed on mother’s laptop (Ubuntu 13.10)

Posted on by

My mother’s laptop started displaying an alarming message:

(3.347276) Kernel panic - not syncing: VFS: Unable to mount root fs on unknown block (0, 0)

The good news is that this seemingly terminal problem has proved easy to fix (so far anyway 😉 ).

  1. Hold Down the Shift key when booting to see the GRUB menu displayed
  2. Press the Escape key to get to the menu
  3. Select a different kernel to boot into instead of the one that has been panicking
  4. Do whatever needs to be done as per [SOLVED} kernel panic-not syncing: VFS: unable to mount root fs on unknown block(0,0)

In my case, I removed numerous kernels (Note – leave a couple behind including the two most recent)
sudo apt-get remove linux-image ...
sudo apt-get autoremove
and was advised by the terminal to run
sudo dpkg --configure -a

Python tuples are immutable right?

Posted on by

Tuples are immutable, unchangeable right? Well yes – sort of. The issue is easier to illustrate than describe so here goes. If we have a list, we can add new items. E.g.

>>> l = ["apple", "banana", "cucumber"]
>>> l
['apple', 'banana', 'cucumber']
>>> l.append("date")
>>> l
['apple', 'banana', 'cucumber', 'date']

If we have a tuple, we can’t change which objects are contained in the tuple:

>>> t = ("apple", "banana", "cucumber")
>>> t
('apple', 'banana', 'cucumber')
>>> t[3] = "date"

You get TypeError: ‘tuple’ object does not support item assignment

So once you have a tuple, nothing will change, right? It is even called immutable so end of story right? Not quite.

Well try this:

>>> a = ["apple",]
>>> b = ["banana",]
>>> c = ["cucumber",]
>>> t = (a,b,c)
>>> t
(['apple'], ['banana'], ['cucumber'])
>>> b.append("A new banana in my immutable tuple! WAT?!")
>>> t
(['apple'], ['banana', 'A new banana in my immutable tuple! WAT?!'], ['cucumber'])

From the official documentation:

“Objects whose value can change are said to be mutable; objects whose value is unchangeable once they are created are called immutable. (The value of an immutable container object that contains a reference to a mutable object can change when the latter’s value is changed; however the container is still considered immutable, because the collection of objects it contains cannot be changed. So, immutability is not strictly the same as having an unchangeable value, it is more subtle.)” (Data Model) [emphasis added]

Subtle – yes – well said. Which objects are contained in a tuple is fixed – but what those objects are is not. In practice, most tuples are containers of numbers and strings and this potential confusion never arises. Incidentally, even though tuples are immutable, they can’t be used dictionary keys if they contain mutable objects. Try it! E.g.

d = {((1,2), (3,4)): 100}
vs
d = {((1,2), [3,4]): 100}

For more useful explanation of how Python handles variables, objects etc check out: Drastically Improve Your Python: Understanding Python’s Execution Model. It sometimes introduces too many new ideas at once in the example code but is very helpful if you focus on the salient parts of the examples only.

[Added later] See Python tuples: immutable but potentially changing

Picasa 3.9 + F-spot together on Ubuntu

Posted on by

Some people want to use F-Spot and Picasa together – F-spot to import and tag photos; and Picasa to do minor editing ready for printing. Here’s their desired workflow:

  1. attach camera
  2. import into F-Spot
  3. do editing in Picasa
  4. want any changes to also be the same in F-spot.

This should be manageable given that Picasa can work on existing files, without having to separately import them to a different place on the file system. So the photo files are where F-Spot put them, and, irrespective of how Picasa organises them in its own “virtual” folders, changes made to the photo in Picasa alter the original photo in its F-spot location. So far, so good.

But there is one issue. If you install the last version of Picasa made for Linux, it is significantly out-of-date. It works as described with Picasa, but it is old. And if you install the latest version of Picasa (3.9) under WINE (instructions), a problem occurs when bringing all the photos into Picasa. Instead of merely pointing to the file locations for the photos, Picasa actually separately makes copies of them. Which means that when you edit them, the original in F-spot is unaltered.

Fortunately, there is a workaround and it involves changing the way Picasa watches folders. It requires some simple edits to the watchedfolders.txt file in: /home/username/.wine/dosdevices/c:/users/username/Local Settings/Application Data/Google/Picasa2Albums

Note: even though you might be using Picasa 3.9, the path is Picasa2Albums.

Make the following alterations as appropriate, depending on which folders you have photos in:

C:\users\username\My Pictures\ –>

/home/username/Pictures

/home/username/Pictures\

C:\users\username\My Documents\ –>

/home/username/Documents

/home/username/Documents\

C:\users\username\Desktop –>

/home/username/Desktop

/home/username/Desktop\

Note that both versions seem to be needed (see also How can I get Picasa to view my second HDD?).

[UPDATE]

Here are some extra issues I encountered:

  • One huge gotcha – Picasa cropping must be Saved to change the file on disk. Clicking on “Apply Change” is not enough to change the file on disk even though the change persists within Picasa. If it isn’t saved, the change is only visible when looking at the image in Picasa – it won’t show up when looking at the file otherwise (e.g. using ImageViewer) and that includes F-Spot. Can’t really blame F-spot for that.
  • F-Spot doesn’t automatically know about files put into its folder structure e.g. if you manually insert an image into /home/username/Photos/2014/03/03 f-spot won’t display it.
  • Picasa only grabs images from the watched folders if they have an image extension e.g. .jpg. Otherwise it ignores them.

The net result is that if you want a simple workflow where you view and tag photos in F-Spot and modify images using Picasa you should follow these instructions:

WORKFLOW INSTRUCTIONS

  • Only put images into F-Spot using import. Don’t manually put images into the standard folders F-Spot uses e.g. “Photos/2014/03/03/”.

    A simple workflow is to have a standard folder under Photos e.g. “Photos/0 Upload These” where you put copies of photos whether you get them from a camera or from another source e.g. as email attachments. You can give these human-readable names such as me_at_beach.jpg rather than IMG9805.JPG etc before importing them. Once imported into F-Spot (and thus copies put into the date-driven folder locations), empty the “Photos/0 Upload These” folder.

  • Save changes in Picasa – don’t just Apply Changes.
  • If you want multiple versions of an image, make them in F-Spot so F-Spot knows about them, then edit the extra versions in Picasa.

    To create new versions in F-Spot File, select a photo, then Photo>Create New Version in the menu.

Ubuntu, new programs, and opening files with them

Posted on by

First an observation – it should NOT be this hard to associate applications with file types in Ubuntu – there has to be a better way. And eventually these rough edges will be removed I expect. But in the meantime: Ubuntu Fix: Add Program to List of Applications in “Open With” When Right-Clicking Files in Nautilus and How do I set the default file association in Ubuntu 13.10 Nautilus/Files with Unity to something that is not listed in the Open With tab?. Re: the latter, you’ll find programs in /usr/bin/ but don’t forget to let the display load all the applications in the folder – it takes a while.

Eclipse + PyDev problems on Ubuntu 13.10

Posted on by

I wanted the latest eclipse + the latest PyDev. The problem was that PyDev wouldn’t install properly or show up. And no amount of using the Install New Software GUI path helped ;-). The solution was to manually unpack PyDev and put it in the appropriate folder. Should have blogged on this at the time (a week ago) because now I can’t remember all the details. But the following helped: PyDev not showing up in Eclipse