I spent a few hours today dicking with my N900 and thought I’d write up some of the things I dealt with.

For a long time I’ve been using Dropbear SSH client/server on my phone, due to an alleged less-memory-usage. (When your phone starts swapping, it sucks big time.) Dropbear even supports serving SCP, but does not support SFTP. This prevents you from using any relatively-nice "file transfer over SSH" GUI, such as Nautilus’s "ssh" support or gFTP. (I think Konqueror’s fish mechanism would still work, but that is of limited utility to me right now.) It may be possible to use the sftp from OpenSSH with dropbear, but since the Dropbear packages conflict with the OpenSSH packages in the Maemo repository, that’s not especially on an N900. In fact, dropbear-scp conflicts with openssh-common (both provide /usr/bin/scp, which I think is silly, but there you are).

Of course, if you insist on using Dropbear, you can use Bluetooth to copy files over Obexftp (which Nautilus supports nicely). But since this requires Bluetooth hardware to be powered on both the laptop and the phone, I decided to replace Dropbear with OpenSSH.

Installing OpenSSH server on your N900 forces you to change your root password (the default is "rootme"), whether you’ve already changed it or not. Kind of annoying. The user account by default "doesn’t have a password", which I think means all password access is disabled. Folk wisdom suggests that giving a password to the user account "could" cause problems, but I think this is based on an (incorrect) belief that the default password is "blank" (in fact, it’s invalid, meaning there is no phone software that relies on using a password to switch to the user account, so there should be no problem with granting a password). Nevertheless I decided to just drop in a key using authorized_keys. But if you don’t set a password, OpenSSH won’t let you log in (even using publickey access); the log messages will tell you that your account is "locked". The reason is that OpenSSH looks at /etc/passwd to decide whether to let you in using any access methods at all; since the password hash is "!", it locks you out.

This page shows how to fix the "locked account" status.

I just learned of the passing, last summer, of Sam Shapiro, half of the dynamic duo behind Force Monkeys. His family has been erecting memorials: samshapiro.org and Saving Ghandiah (Ghandaiah being his best-known handle). Here is some of his writing:

WITHOUT PERSONAL EXPRESSION, WE BECOME STATISTICS. WE ARE NO LONGER PEOPLE, BUT INSTEAD MACHINES, BODIES MOVING ABOUT AND FUNCTIONING ONLY TO ASSURE OUR BASIC SURVIVAL.

WITHOUT PERSONAL EXPRESSION, WE LOSE PART OF OUR HUMANITY, AND WE LOSE PART OF WHAT MAKES US SENTIENT.

FOR THIS REASON, WITHOUT PERSONAL EXPRESSION, NOTHING ELSE IS POSSIBLE. AS AUTOMATONS WE WOULD LOSE OUR HAPPINESS. AS AUTOMATONS WE WOULD SHED ALL VARIETY AND BLUR TO A SINGLE SHADE OF MONOTONOUS GRAY.

PEOPLE CHERISH THEIR DIFFERENCES, AND YET, SIMOLTANEOUSLY [SIC], TAKE ADVANTAGE OF THEM. WE OFTEN DISCARD THE OPINIONS OF OTHERS AS INSIGNIFICANT MERELY BECAUSE THEY ARE NOT OUR OWN.

IF PEOPLE REALIZED THE IMPORTANCE OF SELF EXPRESSION, THEN PERHAPS CONFLICT WOULD NOT BE SO COMMON.

A lot of his art is really stunning. Here is one that I’m fond of. Look at the eyes in particular.

I saw Emacs Got Git, sometimes called "Egg", listed on the list of git-related software on the git wiki (although now taken down) and of course on the EmacsWiki. I decided I’d give it a whirl. The most recent version I could find was the one listed on the EmacsWiki, the version from bplayer, which seems to still be actively developed. The incumbent here is either magit, which is by far the best user interface I’ve ever seen for any version control anywhere, or VC mode, which was written once to support SCCS and has largely survived unchanged since then.

A brief digression about magit and vc-mode. Magit is a little bit of a challenge to pick up: you actually have to read the manual. But the short version is: M-x magit-status to open a view of your repository, and then TAB things open and closed. You can press "s" to stage files, hunks, or even "highlight" lines using the region and stage only those. "u" to unstage; "c" to start a commit, and then C-c C-c to make the commit. You can create a commit that amends the previous commit by pressing C-c C-a in the log message buffer. It probably offends some that there are already conventions here for VC system integration, notably vc-mode. But vc-mode takes a file-based view of version control, has no support for staging hunks, and in general just doesn’t feel good to use. magit is much better — so much better that it is easily worth the break in convention.

So I thought I’d check out Emacs Got Git, to see if it was any better than magit. This isn’t a detailed analysis — actually I’ve probably spent longer writing this post than I did looking at Egg.

I find a screenshot is worth a thousand words. On my ~/etc repository, magit looks like this:

Magit highlights all the important details: which files are changed? Which are untracked? What commits exist locally that don’t exist on the remote?

On the same repo, Egg looks like this:

This is what we call the "angry fruit salad" school of UI design. Also, it doesn’t have a section for "unpushed" commits.

I’m going to be sticking with magit for the forseeable future.

Seen on LWN, an article about porting an application from C# to Java. Punchline: automated translation. Quote:

The inspiration for this was an article about Boeing and automatic conversion. Well we thought "if Boeing can do it so can we". Sounds stupid? Well it is. Luckily for us we did not think that at the time.

I’ve been spending a bit of time steeping myself in EmacsLisp these last few days. I’ve been looking for information on elisp "best practices" — specifically, is it OK to rely on (require 'cl)?

Here’s one page wondering the same thing. There’s always a ton of interesting stuff whenever you go poking at emacs packages; most surprising to me this time around was ELPA, the Emacs Lisp Package Archive. Perl has CPAN, Python has PyPI, Ruby has Rubygems.

I also found the blog emacs-fu pretty interesting looking — approximately one post a week, I think. Lots of stuff I wish I could absorb better.

Emacs Coding Conventions from the Elisp manual is also pretty helpful. To this point (about CL), it says:

Please don’t require the cl package of Common Lisp extensions at run time. Use of this package is optional, and it is not part of the standard Emacs namespace. If your package loads cl at run time, that could cause name clashes for users who don’t use that package.

However, there is no problem with using the cl package at compile time, with (eval-when-compile (require 'cl)). That’s sufficient for using the macros in the cl package, because the compiler expands them before generating the byte-code.

For me, this is enough, because I want to use dolist. But there are programmers out there like David O’Toole, who writes in his interactive guide to the GNU Emacs CL package:

Despite what people say about still being able to use the macros while complying with the policy, in my opinion the policy is still a discouragement. You have to memorize which of its features you must abstain from using (and therefore lose the benefit of those features) if you are to have any hope of someday contributing Lisp code to GNU Emacs.

I think the GNU Emacs maintainers are hesitant to allow use of a package, like cl, which isn’t "namespaced". I bet if all the functions in cl were prefixed with cl-, nobody would mind…

[Update, 2010-Apr-27: From an email on the magit email list:

There's also the small matter that many of the function implementations in cl, striving for the full generality of Common Lisp (much of which is completely useless in Emacs), turn out to be horrible.

E.g., for a fun time, dig down through

(find-if pred list :from-end t),

and look at what it ACTUALLY does when you finish macroexpanding everything. It tests every element of the list against the predicate, not just the rightmost ones stopping when it finds the first match. Once it determines the rightmost match, it then retains NOT the element itself, but its ordinal position N, which then gets used in (elt list N), meaning ANOTHER listwalk, just to get the element back in order to return it. Nor is the byte-compiler anywhere near smart enough to optimize this away (I'm not sure any compiler would be...)

I'll grant cl has some useful macros in it, but it comes bundled with a lot of crap and you need to be really careful about what you use. For many things, you're better off rolling your own functionality using the standard routines available (e.g., while, mapcar, and reverse are all written directly in C).

And you most definitely do NOT want to be foisting the crap on everybody else, hence the need to keep it out of the runtime.

Thanks!]

Subtitle: poulsbo and why you should avoid it.

I recently bought an MSI Wind U110. Linux suffers on this platform as a result of its GMA 500 aka "Poulsbo" graphics hardware. Wikipedia: "GMA 500 support on Linux is not optimal." It’s possible to run Ubuntu 9.10 Netbook Remix, which is what I’m doing now, but it’s not optimal in the following ways:

1. The wireless is disabled in fascinating ways until you press Fn-F11. Once you do that, it should work out of the box.

2. You absolutely need to get the psb driver. Without it, you can’t even drive the LCD at native resolution, and life just utterly sucks. I did this by dumping a bunch of PPAs into my /etc/apt/sources.list.d using add-apt-repository. The Ubuntu-Mobile PPA doesn’t seem to have packages for karmic right now; I found relevant packages in the PPA of kalon33: Nicolas DERIVE and the gma500 PPA of lucazade. I’m currently running the packages built by Nicolas DERIVE.

3. You need to "fix" the driver in some obscure way so that it doesn’t crash when you modprobe it. Bug 406529 documents the fix — see comment 6. Be sure to uninstall the package psb-modules when you rebuild, because it has priority. The psb-kernel-source package unpacks into /usr/src.

4. After you do the updated-packages dance, the "home screen" thing that lets you select packages doesn’t come up by default, but it does show up if you click the Ubuntu logo in the upper-left. Not sure what’s going on here.

5. Suspend doesn’t work by default — the backlight doesn’t come on during resume. I’ve successfully suspended from the command line, both by sudo pm-suspend --quirk-vbestate-restore and using the uswsusp package using a command like sudo s2ram --force --vbe_save. It seems that by default, HAL is adding a bunch of quirks that interfere with backlight coming back on. HAL is in "maintenance mode" and may be being replaced by DeviceKit in the future, but for now we have to work with it.

   If you close the lid, you’ll see the quirks that are being added in /var/log/pm-suspend.log. On my machine, these were: --quirk-dpms-on --quirk-dpms-suspend --quirk-vbe-post --quirk-vbemode-restore --quirk-vbestate-restore --quirk-vga-mode-3. It’s --quirk-vbe-post that really messes everything up, but even when I disabled that one, I still had problems with the screen randomly not coming back (but with the backlight on), so I decided to disable all of these quirks except --quirk-vbestate-restore.

   There are at least two ways to do this. One way involves adding a pm-utils script to remove these quirks from pm-suspend calls. The other way involves telling HAL to get rid of these quirks.

   The first way is in some ways simpler, so let’s start there.

   1. Add a file to /etc/pm/sleep.d/05_remove-annoying-quirks with these contents:

      #!/bin/sh
      
      . "${PM_FUNCTIONS}"
      
      remove_quirks(){
        remove_parameters --quirk-dpms-on \
              --quirk-dpms-suspend \
              --quirk-vbe-post \
              --quirk-vbe-post \
              --quirk-vga-mode3 \
              --quirk-vbemode-restore
      }
      
      case "$1" in
              suspend|hibernate) remove_quirks;;
      esac
      

   2. sudo chmod a+x /etc/pm/sleep.d/05_remove_annoying_quirks

   Here’s how you disable the quirks in HAL, which may or may not be a more elegant solution.

   1. lshal | less to see what HAL thinks of your system. You should see, close to the top, something like power_management.quirk.vbe_post = true, and similar for all the other above quirks.

   2. sudo pm-suspend --quirk-vbestate-restore --store-quirks-as-fdi, which writes a file in /etc/hal/fdi/information/99local-pm-utils-quirks.fdi, storing the quirks you just used. On my machine, that looks like this:

      <?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->
      <!-- Created by pm-utils -->
      <deviceinfo version="0.2">
        <device>
          <match key="system.hardware.vendor" string="Micro-Star International">
            <match key="system.hardware.product" string="U110">
              <match key="system.firmware.version" string="AN012IMS Ver1.0E">
                <match key="system.hardware.primary_video.vendor" int="0x8086">
                  <match key="system.hardware.primary_video.product" int="0x8108">
                    <merge key="power_management.quirk.vbemode_restore" type="bool">true</merge>
                    <merge key="power_management.quirk.vbestate_restore" type="bool">true</merge>
                  </match>
                </match>
              </match>
            </match>
          </match>
        </device>
      </deviceinfo>
      

   3. See the <merge> lines? Add some more that match the other quirks, to turn them off.

      <?xml version="1.0" encoding="ISO-8859-1"?> <!-- -*- SGML -*- -->
      <!-- Created by pm-utils -->
      <deviceinfo version="0.2">
        <device>
          <match key="system.hardware.vendor" string="Micro-Star International">
            <match key="system.hardware.product" string="U110">
              <match key="system.firmware.version" string="AN012IMS Ver1.0E">
                <match key="system.hardware.primary_video.vendor" int="0x8086">
                  <match key="system.hardware.primary_video.product" int="0x8108">
                    <merge key="power_management.quirk.vbe_post" type="bool">false</merge>
                    <merge key="power_management.quirk.dpms_suspend" type="bool">false</merge>
                    <merge key="power_management.quirk.dpms_on" type="bool">false</merge>
                    <merge key="power_management.quirk.vga_mode_3" type="bool">false</merge>
                    <merge key="power_management.quirk.vbemode_restore" type="bool">true</merge>
                    <merge key="power_management.quirk.vbestate_restore" type="bool">true</merge>
                  </match>
                </match>
              </match>
            </match>
          </match>
        </device>
      </deviceinfo>
      

   4. sudo restart hal and then lshal | less again. You should now see power_management.quirk.vbe_post = false, etc.

6. I stuck with uswsusp because resume is a lot faster using it. To enable uswsusp in pm-utils (which is what gnome-power-manager uses), edit /etc/pm/config.d/00sleep_module and add SLEEP_MODULE="uswsusp".

7. Sound is broken out of the box. The fix is documented on Linux Tipps:

   Sound does not always work without problems. In Karmic 9.10 you have to comment out

   #options snd-hda-intel power_save=10 power_save_controller=N
   

   in /etc/modprobe.d/alsa-base.conf.

   options snd-hda-intel position_fix=1 bdl_pos_adj=64
   

   instead helped me as well.

   I’ve been playing with enabling/disabling this option and haven’t seen any clear gains or losses, although I have had sound cut out after resume both ways.

8. The machine can drive external monitors at 1024×768, 800×600, 640×480, and 720×480, which is about what you would expect from hardware like this.

9. Like the man says, webcam works fine once you press Fn-F6.

10. I’m finding that when I try to record audio, I get a massive amount of static and very little input volume. Still working on this one..

11. Bluetooth seems to work out-of-the-box, but you have to Fn-F11 to toggle it on. (This seems to be a matter of some contention for some people, but on my machine Fn-F11 cycles through both Bluetooth and wireless off, to Bluetooth off and wireless on, to Bluetooth on and Wireless off, to both on, and then back. (00, 01, 10, 11.)

[I'll continue to update this as I discover things about this machine.]

Otherwise, the keyboard’s pretty good and the battery life seems pretty good (as you might expect from the big-ass 9-cell battery).

As far as I can tell, it’s not really possible to run the Ubuntu Lucid alpha on this hardware, and probably won’t be for a while, because Xorg 1.7 is incompatible with the closed-source psb driver. I wrote a post on the thread on Intel’s forum about Linux support for GMA500 summarizing the various-driver situation; it’s possible that in a month, when Lucid finally comes out, there’ll be a better driver situation, or that in six months, Intel will release a better driver of their own. Do people still reverse engineer graphics drivers? Would anyone be interested in doing so for a free U110 or two of their own?

Knowing the things that I know, I’d probably advise against this netbook for anyone hoping to run Linux on it. In today’s day and age, it just doesn’t make sense to buy open-source-unfriendly hardware, even for the sake of a 10-inch screen. Then again, it’s hard to know what hardware to support. The Dell Mini 10, for example, has an Intel NM10 Express graphics card: what does that even mean? Does it have support? Wikipedia has a comparison of netbooks page, but there’s no column for "Linux support", and in general comparing multi-dimensional things like hardware is a real challenge. Anyone have any tips?

[Update: I have been tinkering with the different quirks for suspend. Right now I'm using SLEEP_MODLE="kernel" and removing both quirks --quirk-vga-mode3 and --quirk-vga-mode-3 (one is specified by HAL, and another by the man page..). However, I still haven't solved the problem that periodically my machine will still just fail to suspend. My impression is that SLEEP_MODULE="uswsusp" takes about half the time to suspend and resume, but that it's more unstable, where the failure mode is failing to resume. I don't know what else to try so for now I'm leaving it alone.]

Seen on Planet Debian: ‘Can you get cp to give a progress bar like wget?’ The solution starts:

#!/bin/sh
cp_p()
{
   strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
...

The author notes in the comments:

If you feel the need to point out an alternative solution, then you have missed the entire point by a wide margin.

—lamby

At PyCon, I saw a lightning talk about scrape.py, a lightweight Python library for parsing webpages/interacting with them programmatically. For example, finding page elements:

>>> from scrape import *
>>> s.go('http://zesty.ca/')
<Region 0:17780>
>>> d = s.doc
>>> t = d.first('title')
>>> t
<Region 247:258 title>
>>> t.tagname
'title'
>>> t.text
u'Ka-Ping Yee'

The presentation I saw focused on the use case of testing your website. This is definitely a pain point for me personally: I currently either grep the HTML with regexes or I parse the whole thing using ElementTree and use XPath. But there’s still a couple of problems: 1. JS isn’t usually testable this way; 2. you often have to construct your HTML with an eye towards testability. For example, to test pagination, you might need to add a class or id specifying that this is the pagination section and that these pages link to pagination things.

Via Suzanne, who writes "this is bad news for computer scientists".

A new paper in Medical Hypotheses journal posits that carpal tunnel syndrome can be caused by sexual intercourse. From the abstract:

It is proposed that carpal tunnel syndrome can develop during sexual intercourse when the hands become repeatedly extended while under pressure from the weight of the upper body. Of the eight risk factors associated with non-occupational carpal tunnel syndrome, age, marital status, pregnancy and use of hormonal agents can be explained by changes in the frequency of sexual intercourse. On the other hand, obesity, macromastia and large chest circumference can be explained by the increased pressure imposed on the wrists by the heavier upper body associated with such conditions. The bilaterality of carpal tunnel syndrome can be explained by the fact that both hands are needed to support the upper body during sexual intercourse. A parallel decrease in the frequency of sexual intercourse and the incidence of carpal tunnel syndrome between the sixth and the seventh decades of life suggests a possible cause and effect relationship between sexual intercourse and carpal tunnel syndrome.

Also: if you do push-ups, you might invest in getting some push-up bars. See also JWZ’s experiences with his wrists.

Thanks Adam — how to safely store a password.

Use bcrypt. Use bcrypt. Use bcrypt. Use bcrypt. Use bcrypt. Use bcrypt. Use bcrypt. Use bcrypt. Use bcrypt.

Good to know!