May 13, 2011

Using Chef to write out a munin.conf file which uses the Chef Server node name to label clients

Pretty much what it says on the tin. If you configure your munin server using Opscode chef, and you're looking at EC2/Opscode nodes, it's advantageous to label the munin clients in the munin conf using the Opscode node name so that you can quickly match them to the proper records on Opscode. Also, the Opscode name has the EC2 instance ID suffixed to it, which makes it easy to find the node on EC2 as well.

In addition, you want to group your nodes into reasonable groups! So let's say you have project/environment/type attributes which describe each node (like, for example, myapp-staging-app).

Pass an array of nodes into the template as @munin_nodes. Inside the template erb file which you use to write out the munin.conf file, use the following:

# a simple host tree
<% @munin_nodes.each do |system| -%>
[<%= system[:project]] %>-<%= system[:environment] %>-<%= system[:type] %>;<%= %>]
    address <%= system[:ipaddress] %>
    use_node_name yes

<% end -%>
This will give you a nice conf file, with each node configured to be part of a node group and each node configured to have munin talk to it using its IP address rather than your label as a DNS name.

This is pretty straightforward, except for that '' call. Although I had gotten used to pulling any attribute I wanted out as symbols (node[:symbol]) it turns out there is no symbol that links to the Chef server node name - that attribute is at a 'higher level' than the attributes you can get to with symbols. So you have to use (here to get to it.

Posted by jbz at 6:06 PM | Comments (0) | TrackBack

January 26, 2011

Opscode Chef LWRPs and notifications

Wow, so @kallistec is my hero. I was trying to write a LWRP for my rsyslog cookbook in my Chef repo, and it WASN'T WORKING. The purpose of the LWRP was to allow other cookbooks to declare that they had text-only logfiles, and have the rsyslog LWRP then generate the appropriate imfile configuration stanza in /etc/rsyslog.d.

All well and good. However, this meant that inside my provider file, I had the following: (from rsyslog/providers/imfile.rb):

# Create new rsyslog snippet to configure imfile to follow a text log
action :create do
  template "/etc/rsyslog.d/#{new_resource.tag}.conf" do
    source "imfile-stanza.erb"
    cookbook "rsyslog"
    owner "root"
    group "root"
    mode 0644
      :location => new_resource.location, 
      :tag => new_resource.tag, 
      :facility => new_resource.facility, 
      :priority => new_resource.priority, 
      :label => new_resource.label)
    notifies :reload, "service rsyslog", :delayed
...because in the default recipe for the rsyslog cookbook (/rsyslog/recipes/default.rb) I had:
  service "rsyslog" do
    supports :restart => true, :reload => true
    action  :enable, :start
...but this DIDN'T WORK. I kept getting errors that looked like this:
ERROR: Exception handlers complete
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:53:in `run_action': undefined method `run_action' for "service rsyslog":String (NoMethodError)
...when the chef client ran on the appropriate node. After much patience with me on IRC, @kallistec explained that the problem I was having was that when an LWRP is compiled and executed during a chef run, it has its own resource collection *distinct* from the 'larger' resource collection. As a result, the service which was getting declared in the default.rb wasn't getting pulled in to the resource collection during the run. At his suggestion, I put a service definition into the action provider action :create stanza, so I then had this in /providers/imfile.rb:
# Create new rsyslog snippet to configure imfile to follow a text log
action :create do
  service "rsyslog" do
    supports :restart => true, :reload => true
    action :nothing

  template "/etc/rsyslog.d/#{new_resource.tag}.conf" do
    source "imfile-stanza.erb"
    cookbook "rsyslog"
    owner "root"
    group "root"
    mode 0644
      :location => new_resource.location, 
      :tag => new_resource.tag, 
      :facility => new_resource.facility, 
      :priority => new_resource.priority, 
      :label => new_resource.label)
    notifies :reload, "service rsyslog", :delayed
But that didn't work either! Finally, @kallistec advised that I change the notification syntax in the provider stanza back to the *old* (pre-chef-0.9.10) syntax. That meant changing:
    notifies :reload, "service rsyslog", :delayed
    notifies :reload, resources(:service => "rsyslog"), :delayed
AND THAT WORKED! Whew. Thanks again to @kallistec for patience, expertise and helpfulness. Also, thanks to @fujin for help getting the LWRP together in the first place!

Posted by jbz at 6:18 PM | Comments (1) | TrackBack

August 5, 2010

Dealing with stale cached apt manifests in Chef-run apt transactions

I have a 'base' Chef recipe whose job is to do global basic bootstrapping tasks to EC2 slices. One of the first things that this recipe does on Ubuntu slices is update the apt caches using 'apt get update.' Without that step, it generally fails, because the AMIs I use don't have complete and up-to-date apt repo information in them.

Well, today, I ran the recipe on a new slice and it tanked. apt was exiting with status 100, and when I checked, it was producing the following error:

 IP ADDRESS apt-get update returned 100, expected 0

...and running apt-get update manually returned an error complaining that it couldn't update the repo information. It complained:

W: Failed to fetch  (the problem archive file): Hash Sum mismatch

The problem here, it turns out, is that the archive in question I was attempting to get updated manifests for had been updated - but the checksum (Hash Sum) for that archive had been *cached* by something in between my slice and the archive (likely some form of proxy). As a result, they didn't match, and bam.

The Fix: The fix came from here. Essentially, mod your apt-get command to include an option for 'no-cache', either during the call:

apt-get update -o Acquire::http::No-Cache

...or by putting the option into your apt config (see the post linked above for how-to).

Whew. Another day, another thing.

Posted by jbz at 2:40 PM | Comments (0) | TrackBack

July 28, 2010

Chef cookbook attribute files

Learned today the hard way: Don't use set_unless in a Chef cookbook's attribute file unless there is a really good reason. And corollary: there's never a really good reason. At least, not that I've found.

Why? Here's what happened. I started hacking Chef recipes by looking at examples from the Opscode and 37Signals git repos, probably like many folks. In many of those examples, attributes are set using things like

set_unless[:cookbook][:varname] = "some_value"

The problem is that what set_unless means here is that whenever this cookbook is run, Chef checks to see if the variable in question already has a value...any value. If so, it skips the set statement. See where I'm going?

Yeah. So I had an attribute with a really long list in it that was going to be executed by a stanza in my main recipe file. To test it, I created a three-item list in the attribute file. It worked fine on my test node. So I changed the attribute to have the full 21-item list, and then re-upped the cookbook to the Chef server, and re-ran the client on the node. It diligently went and got the updated attribute file and downloaded it to the cache, then performed the chef run...with the same 3-item list.

Because I'd already set that variable on that node, the set_unless wouldn't fire. The only way I got this to work was to go replace the set_unless statements with plain ol' set statements. Then, when it ran, it happily overwrote all the variables with the new values.

Does this waste time/resources? Yes, but probably not as much as a set_unless_equals sort of test would (not that that test exists now).

Posted by jbz at 6:58 PM | Comments (0) | TrackBack

July 12, 2010

Amazon EC2 ami manifests and architecture mismatches

So I'm muddling my way through various means of managing an AWS infrastructure. We're currently using Opscode's chef and their online SaaS implementation (experimentally). When using the chef tool 'knife' to boot up an EC2 server using one of our custom AMIs, I kept getting an error:


`request': InvalidParameterValue => The requested instance type's architecture
(i386) does not match the architecture in the manifest for ami-xxxxxxxx (x86_64)


...after mucking about for a bit, I have discovered that AMIs whose declared arch is x86_64 (and, presumably, whose arch2 is amd64) *cannot* be used to instantiate running instances whose type are m1.small or m1.large. This matters because while some DSL tools (*cough* chef *cough) have the means to specify and pass thru an instance 'flavor' to the EC2 toolset, they can't/don't/won't accept or pass thru an 'arch' value. So the error message is a bit of a red herring. In my case above, I changed the instantiation command from specifying '-f m1.large' to '-f m1.xlarge' and...voila, it worked.

Posted by jbz at 2:23 PM | Comments (0) | TrackBack

June 9, 2010

Resolving an nginx and passenger problem

Was working on remote-configuring a server which runs a Ruby app using nginx and Phusion Passenger. Despite having everything configured properly (I was convinced it was, at least) I kept getting the following error on nginx startup:

Alert: could not send password to the passenger helper server: write() failed (32: Broken pipe)

nginx would refuse to start. It wouldn't even write anything to the error log. Argh.

To make a long story short, there were *two* reasons I was getting this error. Here they are, and here are the fixes.

Reason 1: Bad nginx.conf

In my nginx.conf, I was specifying the location of the passenger_root and of the passenger_ruby. In both cases, I had an incorrect path due to us having changed versions of ruby and the passenger gem - it had been updated, so its path changed.

Reason 2: Bad HelperServer

We (I) had installed nginx via the command passenger-install-nginx, included with the passenger gem. However, the version of passenger had been updated (from 2.2.5 to 2.2.14, as if it matters) and we (I) had not re-run that install process. I originally didn't think it mattered. However, apparently (and I'm not positive of this) there is a HelperServer that is compiled at install time which points to a particular passenger instance, so if you upgrade passenger, you need to rerun the passenger-install-nginx command. It won't stomp on any configs or anything.

Anyhow, after fixing those problems, whee, nginx and passenger started fine. I hope this helps someone, because I was looking for exactly this post frantically during the Google phase of debugging, and didn't find it. I found pieces of info, but not all of it.

Posted by jbz at 1:10 PM | Comments (2) | TrackBack

April 6, 2010

iTunes 9.1, ePubs and cover art

Well, iTunes does handle ePubs, but it seems sorta b0rked. Here's how.

I took a set of ten or fifteen epubs that I have, all non-DRMed, and dropped them onto iTunes. It imported them without error messages. I have no way to test if they're actually OK because I don't (yet) have an iPad to sync them to. But anyway, to continue.

Two of them have cover art in iTunes. The rest do not. Huh?

If I look at them in Calibre, they all have cover art. Furthermore, something odd - if I go back into iTunes and look at one of the ones whose cover art shows up in the 'Summary' section of Get Info and in the cover view, although the cover shows up in the 'Summary' section there's nothing in the 'Album Artwork' section. I cannot copy or paste the cover in the 'Summary' section (they're greyed out, and I tried).

Going back to the ePubs as it was saved to disk by calibre, I can compare a working ePub and a non-working ePub. They both have cover art in their folders, and both are jpgs. Opening them in preview shows both of them are identically sized; both are using the same (RGB) colorspace. They look identical. But one of them imports when you drop the folder of the ePub on iTunes, and one does not.

If I paste the cover art into the non-working ePub's 'Album Artwork' pane in iTunes, it shows up as having a cover, and that cover shows up in the Album Artwork pane. The one which worked originally still doesn't have anything there.

So if I had to hazard a guess, I'd say there is still something screwy with iTunes' ePub import function; something which screws up some cover art. Great. I don't want to have to go manually re-fix all my cover art on my 200+ books.

Posted by jbz at 10:51 PM | Comments (2) | TrackBack

October 22, 2009

Files disappearing on Desktop on OS X 10.5.8

Ran into an interesting problem. Using OS X 10.5.8, whenever I dropped a file or folder onto the desktop (from a subfolder, or from Entourage, whatever) it would just ...vanish. Not to be found. Checking in a windowed list view, I found that the files were there, they were just apparently being placed on the Desktop at coordinates that were outside my viewable area. I don't know why - I'm not running VNC or desktop sharing, and I don't have another monitor attached (and never do, to this machine) - but there you go. Essentially, the default file location on the desktop was outside the frame, or viewable area.

I eventually 'fixed' the problem by selecting the Finder, clicking the desktop, and then selecting 'View->Show View Options' from the menus. Then I moved the Icon grid spacing slider, released it (and watched the icons move) and then moved it back.

That appears to have 'reset' whatever preference was borked. Now, when I drop stuff on the desktop, it shows up in the next unused slot on the right side, just like it always used to.

Hope that helps some poor soul who is Googling as frantically as I was.

Posted by jbz at 1:09 PM | Comments (0) | TrackBack

October 8, 2009

I want my Warp Drive, damn it

Apple's already got a standard technology named Time Machine. I just want to register my hope that their new Light Peak tech gets called Teleporter. I don't think they'd call it Transporter due to the baggage.

Posted by jbz at 5:03 PM | Comments (0) | TrackBack

September 23, 2009

Hooray, it was a bug not a feature!

As maw pointed out, Apple seems to have agreed it was a bug, not a half-finished feature:

iTunes 9.0.1 provides a number of important bug fixes, including:
  • Addresses an issue with the Zoom button not switching to Mini Player.

And hooray, the zoom button now goes to/from MiniPlayer! Yay!

Posted by jbz at 12:23 PM | Comments (0) | TrackBack

September 9, 2009

Epic. iTunes. Fail.

iTunes 9 upgraded itself fine on my work Mac (I don't trust iTunes enough to install new versions on the Mac with my main library until it's been thoroughly tested). But, EPIC GODDAMN FAIL: clicking the green window button NO LONGER sends iTunes into 'MiniPlayer' mode. No, now there's a whole new #$&)(@# command for that, 'Switch To Miniplay' in the View Menu, with hotkey Cmd-Shift-M.

The window buttons still appear on the miniplayer. But this change means that if you have iTunes minimized into the MiniPlayer, and click the green button, it expands to fullscreen just like it used to. Great. But if you click it again, it instead behaves *differently* and sends iTunes to a different window size (presumably one you used prior) rather than back to the damn MiniPlayer.


Posted by jbz at 5:21 PM | Comments (1) | TrackBack

August 21, 2009

Losing and recovering your iTunes library redux

So. Let's get this out of the way. I'm a Mac (not Apple) fanboy. But iTunes, for all its handiness and utility as a media player, is not a good media storage application. Let's be honest: it's frigging terrible. It doesn't handle failure well at any level, it doesn't offer any form of resiliency, its required datastores are both fragile *and* brittle, and in general if anything goes wrong you're screwed even if you have a full backup. I didn't have a full backup. Here's some things that helped (but didn't totally solve) my backup nightmare.

The Situation

As mentioned in earlier posts, my iTunes library lived on an external drive on my iMac, along with another partition containing my Time Machine and SuperDuper backups. My reasoning at the time was thus - I had three classes of information in terms of backup strategy. My primary data was Class I, and I placed that on the internal drive of my iMac. My media (iTunes library and movies, which I keep separate) was Class II. My backups themselves were Class III, under the assumption that if I lost a backup without losing the primary associated with it that was no big deal - I could just replace the drive and generate a new one. I'm not worried about keeping versioning outside of those apps which handle that themselves like source control systems.

So. Class I was internal. I put Class II on the external, on its own partition, as I reasoned that I didn't want the disk ops for that media contributing to my primary (Class I) disk MTBF or entropy. So if Class II failed, it wouldn't take Class I with it. I placed the backup partitions (Class III) on the same physical external device as Class II.

No, I didn't think about it in this mind-numbing level of detail - this was my practice as had evolved over many years of storing data and having drive (and tape, and CD) failures.

Anyway. My Class II and Class III storage both tanked as the common hardware under them died. This meant I could still operate my iMac for primary (critical) tasks - all my non-media data was still intact (and backed up elsewhere, like absolutely critical docs are in cloud storage, etc.)

I had a backup of my Class II data. Since it was Class II, it wasn't a live drive backup - it was a directory on my fileserver, which held that backup and some additional Class II data (movies) which would be 'nice to get back' but were huge and not critical. The problem was that my scripted rsync autoupdate of the server volume of my iTunes library never worked right; I would run the script 'when I got around to it'.

So the last time I'd run the script was May 9.

Unfortunately, after I copied that version of my media folder back to my iMac, I found a few problems. To wit:

  • Apparently, the last *successful* backup had been sometime in October 2008, because music files which I'd added since that date just weren't present in the backup volume, despite 'modification times' which seemed to indicate that they should have been.
  • The iTunes *library* file I had (the database) was the *current* iTunes library, which lived with Class I data on my iMac. I had had Time Machine backups of it, but those had gone with the external drive.

Getting the backup back

So. My first problem was that since I'd kept all my media on a separate volume, the path information for both my iTunes library path preference *and* (as I found out) embedded in all my track information was incorrect. I couldn't put the files back where they 'had been' because I no longer had a drive volume named 'Jukebox'. I had the files in my ~/Music/iTunes/iTunes Media/ folder, which is where iTunes assumes they live if you haven't told it differently. So I went into iTunes and changed the 'Library location' param to the proper place and started it up.

It couldn't see any media files.

Hm. Okay. I went into the iTunes directory. There's two files in there that matter - "iTunes Library" (which is a binary database) and "iTunes Library.xml". The problem is that while you can manually edit the XML version of the library file that doesn't help you - the XML is generated *from* the binary as a matter of convenience for non-iTunes apps to be able to see your library information and iTunes itself ignores the file. However, looking at it can tell you in plain XML what iTunes *thinks* the situation is inside the binary DB.

I wasn't sure how, but although the library parameter was properly set in the XML (to the right path) the *files* all had pathnames starting with "file:///Volumes//Users//Music/iTunes/iTunesS%20Music/..." which was *not* where the tracks had once lived. And no matter what I did, I couldn't get iTunes to change that. Even if I changed the library location and restarted iTunes, causing it to say 'updating library,' when it was done the pathnames all stayed there although the library location parameter would update. Thus, I was thwarted trying to place my tree of music (some 85GB) in a spot of my choosing.

Finally, I put the music where iTunes thought it should be, which was in that above path. Then iTunes found the tracks. But it had huge gaps, files with the dreaded (!) next to them that means iTunes can't find the referenced track. And not just the most recent few months of files - they were scattered everywhere.

It turns out that I, being anal, continually tweak my music library to ensure consistent and correct tagging and filing of my music. The problem was that any track I'd touched in the past nine or ten months, since the cutoff, was now (in the tree) back in its original place and state, whereas the iTunes database thought it should be where I'd corrected/moved it. So all of those were out of sorts.

What I ended up Doing

The first thing I did was make a backup of my iTunes/ folder, including all data files. DO NOT FAIL TO DO THIS. It will save your ass. Trust me.

The first problem, getting the library to come up at all, I covered above. Moving the library into iTunes' "default path" of /Volumes/Users//Music/iTunes/iTunes Media/ fixed that problem. Sure, it's possible I could have figured out what DB tech iTunes was using for its main DB (SQLite? db4/5?) and gone after the pathnames in there, but...meh. iTunes has made it clear to me it REALLY DOESN'T LIKE stoopid hoomans messing around inside its data files. I'm willing to muck around with XML files since I trust my regexps, but not with database files I can't really *see* when I don't know how the app using them really works.

This solution is suboptimal. It violates my Class I/Class II separation, as all media disk ops now affect my Class I internal drive. Once I am satisfied that my tweaking of the library is done, I will probably investigate getting it to move properly to Somewhere Else (where is not totally clear).

Getting my music back

The first problem was all those tracks that I'd added since the cutoff date that no longer existed. I really didn't want to lose their DB entries, mostly because I didn't want to have to manually go fix all the playlists that referred to them and didn't want to lose my ratings. Didn't care about play counts.

Well, for stuff I bought from the ITMS, there's good and bad news, as anyone who has been here knows. Good news: If you go to the ITMS and add a song to your cart you've already purchased in the past, it will pop up a warning about that fact - so you can tell if that's where you actually got the track. This is important because if iTunes can't find the actual file, it won't give you critical info like what file type it is, which is how you tell where you got it).

Bad news: It will still make you pay full freight to re-buy those tracks. There is no feature for 'I LOST MY STUPID LIBRARY PLZ LET ME REDOWNLOAD KTHX.' Apple: You're fucking morons. This would remove SO MUCH of the objections to digital media 'ownership.' I had a backup of my library, but even the files that hadn't changed required non-trivial (from a Mac owner perspective) fucking around to get iTunes to recognize. In other words, if I was Joe Random User, there's a good chance I wouldn't have been able to get iTunes to recognize my tree of files because the DB and the media tree were out of sync - and if they live on different drives and I lose my backups, that's gonna happen.

Good news: For tracks you bought in reg'lar ITMS which have been made available in Itunes Plus - Apple's 256kpbs DRM-free format - you can download the higher-rez open track for the *difference* in price, usually $0.30, per track. That's a lot, but it's better than the $99 replacement cost, that's for sure. And there is a page on the iTMS called 'Upgrade to iTunes Plus' (look for it on the front page of the store, under 'Quicklinks' on the right) which has a list of all the albums and individual songs you've bought and will let you (on a one-by-one basis) pay the upgrade fee and download the new version of the track. Best of all, even if the file doesn't exist on your machine, iTunes will recognize the (broken) database entry and simply replace the track info, so all your metadata will still be there. Expensive but maximally convenient way to go.

iPhone tracks

Handily for me, a large number of the most recent tracks I'd imported were synced to my iPhone. If you want, you can get music back off the iPhone, although the way I did it was maximally manual in order not to step on my fragile iTunes database.

  1. jailbreak the iphone. Mine already is. If yours is not, look up the Dev Team blog and find out how.
  3. Make sure ssh and rsync are installed on your iPhone. Make sure SSH is running.
  4. From a terminal, copy your music over using something like the following:

    iMac:SomeTempDir user$rsync -av -e ssh --delete root@:/private/var/mobile/Media/iTunes_Control/Music/ .

    ...note there is no 'z' option; compression makes this more than twice as slow. Also note the underscore in iTunes_Control. This will copy your music files to SomeTempDir on your mac.

  5. What it will create is a directory of subdirs that looks sort of like this: F00
    F02 get the idea.

    Inside these Fxx dirs are all your media files. They have been renamed to three- or four-digit keys with their extensions (.mp3, .m4a for AAC, .m4p for DRMed-AAC, .mp4 for video) intact.

  6. If you double-click each of these items, they will usually (if they already exist in the iTunes library) just start that version playing, without copying a new version. If they do copy it in, you know it's a file you didn't have, *or* that it is a file that was missing. If your 'keep library organized' settings have changed, it may copy the files in if the newly frobbed 'correct name' is different from the ones that it was synced to the phone with. But anyway, they'll play fine, even the protected ones, since they were synced using your iTMS keys.

Anyway, that's how I got some of it back. I hunted around for the CDs I'd used for a bunch more of it. Some of it was freely downloadable, so I just went through and red/l-ed it (thanks DJ Earworm, thanks Pheugoo!) The last of it, the stuff I'd bought via iTMS and didn't have a backup of and that I couldn't get by 'upgrading'...well, I saved off the list of them to work on getting hold of later, and deleted the DB entries.

The next problem was the stuff that had moved. iTunes was pretty good about figuring out all the problem entries and tagging them with a (!). You can force it by just selecting the top entry and then using the down arrow to zip through the library on repeat - it checks the file validity every time the track is *selected*, not played, so it'll tag the ones that are broken. I found that in 99% of the cases, I could find the track back in its original location in my media tree. Since most of the corrections involved adding album data or switching 'compliations' to 'real albums' and etc, it was pretty trivial. The only crappy bit was that the only way to fix these tracks was to double-click *each track* and then navigate the file selection dialog to relocate them. Ugh.

Once you've done so, be warned - it doesn't always move the track back to the right place. I don't know what the algorithm is. However, if you've selected 'Keep library organized,' once it's properly found the track (i.e. it'll play if you doubleclick it) you can rename the album or artist and then move it back to the right name (my database entries were newer, thus retained the 'proper' data) and when you make *changes* it'll refile the track properly.

So there you have it. A fuckton of work, but that's because I"m anal about my music library in a way I am not about anything else.

I think I got back all 18K+ tracks, with the exception of around 150 tracks I didn't have backups for.

Then I Time Machined my internal drive to my (replacement) external HD, which is now all one volume - backup only - and told it to include my iTunes library.


Posted by jbz at 3:06 PM | Comments (0) | TrackBack

August 10, 2009

Benthic Randites

My devious brother has gifted me with a copy of Bioshock for my Xbox. On first play, I got sucked in for around four hours and had to drag myself, shuddering, up into reality in time to get *some* sleep before my next day's work.

This bodes well.

Posted by jbz at 4:18 PM | Comments (0) | TrackBack

More telephony options

Sweet! A beta invite to Google Voice just hit my inbox. Now to find a Cool Number(tm).

Posted by jbz at 4:15 PM | Comments (0) | TrackBack

June 26, 2009

Dear Microsoft Mac Business Unit

When will you deign to make your fucking Office suite work properly with Spaces on Mac OS X? As in, not have the goddamn floating palette windows randomly change spaces, and then have them lose track of which space they're in causing your whole desktop to swoop to another Space when you click on your Office document? It's really getting fucking old.


Posted by jbz at 3:35 PM | Comments (1) | TrackBack

March 2, 2009

Safari 4 on OS X and Windows AD authentication

After a frustrating couple of days, I think I've figured out what bit me. Safari 4, or rather, a feature therein.

Background: At work, I use my iMac on our corporate network. This network uses Active Directory for all manner of tasks. At the time, my Mac was using local account authentication; however, I was running MS Office with Entourage, which was of course authenticating to our Exchange server. Our network had recently installed a web proxy filter, which required periodic (once per session, or less frequently) authentication to access the outside world. I'm also doing sub-rosa stuff with corkscrew to get SSH access out through the proxy server. So far, so good; all this stuff has been working for months.

Then I started getting locked out of my AD account. At first, this was hard to pin down, because everything I was using would cache credentials, so I would discover the next morning or after coming back from a system upgrade restart that I couldn't get mail. Highly annoying.

The AD server logs claimed that the auth request was coming from my machine's IP address, and was coming from MICROSOFT_AUTHENTICATION_PACKAGE or something close to that. Also, there would seem to be a string of failures in a row, resulting in a lockout - something between five and twenty tries, sometimes all within a second.

After a few days of fruitless troubleshooting (not using Entourage, not using my corkscrew hack, etc. etc.) I gave up and tried for a clean slate.

I switched my iMac over to authenticating against the AD domain, and as a result (because my local account had a different username than my domain account) I ended up with a new, clean account. All my work data was living in a fileserver folder anyway. I moved over a few plists in order to retain configs and license from some apps (BBEdit, Terminal, yadda). Just to be sure, I created a brand new Entourage profile (not copying over any office prefs) and put only my work email (not my personal accounts) on it.

Nope. I kept getting locked out. This was more annoying, because usually I'd figure this out when trying to get out of my (locked) screen saver - and it wouldn't exit, because the password had locked down.

Aha. I tried turning off the screen saver. Nope.

I went through my Apple keychain and deleted every entry that was from before the 'new account' switchover, and all entries with 'Microsoft' anywhere on them.

Nope. Still locked me out.

Then I thought really hard about what had changed on my computer. Yep. I'd installed the Safari 4 Beta.

So I downloaded the uninstaller (it comes with the installer) and backed out to Safari 3.

A couple of hours later, no lockouts.

Here's what I think was going on. Safari 4 had dutifully built me one of those 'Top Sites' walls, even though I told it I didn't want to see it unless I asked. However, I think Safari was still updating my various (ten to twenty) URLs from my Top Sites wall. And to do that, it needed to get through my corporate proxy authentication. As far as I could tell, it had not stuffed that credential anywhere into my main keychain - so it's possible it was storing it locally somewhere in the Safari app prefs.

It trying to update twenty 'Top Sites' would explain why sometimes the stupid lockout logs showed twenty password failures in close succession.

I can't prove this is what's going on. However, after dumping back to Safari 3, the problem seems to have gone away.

I would suggest, on the incredibly long chance anyone from Apple reads this, that Safari be smart enough to update its Top Sites in serial if there is an auth credential stored, and if it fails, to not try to update each site - because it will definitely lock you out. My AD server is set to five tries before lockout, within a short timeframe, I think.

Posted by jbz at 5:40 PM | Comments (0) | TrackBack

February 7, 2009

I Iz an Op

So I've been running a personal mail server on them there internetz for a few years. I run the Courier-IMAP daemon (with SSL, natch) for mail retrieval. I've always had trouble getting my install of it to handle certs properly - I had made my own CA in order to generate a signed CA and import the root CA into my client. But that never seemed to work, so I just became resigned to clicking the 'Accept' button on various email clients.

I finally got annoyed enough tonight to actually go buy a cert. I installed it. Postfix happily accepted it for SMTP-TLS. But Courier...when I ran my email client, it kept bitching about self-signed certs.


After literally 45 minutes of poking around, I discovered that I had (lo those years ago) installed a Courier-IMAP package from source. Yep, remembered that.

Then I'd been editing the conf files for the damn distro version of Courier full, in /etc/courier. Um.



I looked at the source init scripts and decided it wasn't worth my time to re-fucknicate all the conf files, so I eventually just found out where the installed init scripts were expecting to find the cert files and linked those to my new cert.



Hey...I wonder if that's why Entourage can't see new mail...the damn TRAVERSE_FULL setting is..HEY!

Posted by jbz at 3:34 AM | Comments (0) | TrackBack

January 28, 2009


Okay, I can officially no longer deal with Bloglines. Now to train myself to use Google Reader instead. Step one: rename all my linkbar aliases...

Posted by jbz at 3:37 PM | Comments (0) | TrackBack

January 27, 2009

Choking under the weight

I've been a faithful user of bloglines for quite some time now. But recently I've been feeling it might be time to give in to the mighty Goog and switch to Google Reader. I have around 175 feeds in my blogroll. Bloglines, starting a couple of months ago, started doing noticeably wacky stuff - not linked, AFAICT, with my adding any more feeds, just out of the blue. For example, when I go to the Bloglines page, there's now a 20% chance that it will load everything *but* my feeds and then claim all is peachy (there's just nothing in that part of the left frame). Or I'll click on a category and the page will load perhaps 40% of the elements and then just sit there forever, loading - unless I click the category again, when it will just load. Or (my personal favorite) a random category which I've caught up in will suddenly claim it has N new entries, with N being something like 500 or 838...but when I go to the category, no new entries come up. Finally, the most annoying of all - it's started losing track of my reading, so I'll come to the page, read a couple of dozen entries spread across my categories, reload the page, see that there's nothing new, and leave. An hour later, when I come back, voila, the same categories have the same new items in them again.


Posted by jbz at 10:07 PM | Comments (1) | TrackBack

December 17, 2008


Um, yeah, so I remain unimpressed with our web filtering solution. (pops soda, kicks back with gootube)

Posted by jbz at 6:04 PM | Comments (0) | TrackBack

Misdirected Productivity

Well, my company decided to put a seriously draconian web filter/firewall in place which affects, for the first time, our internal IT and R&D networks. Everything (address/port) seems to be default blocked, with the exception of http/https to non-blacklisted IPs which requires authentication. Sigh. I suppose this means I need to devote some time tonight to breaking through it again.

Posted by jbz at 4:56 PM | Comments (0) | TrackBack

December 15, 2008

Overengineering, underengineering

I was overengineering of course, as I discovered, there is an app in the Cydia iPhone installer named 'FileViewer' which...does just that. It doesn't suffer from the Safari patch's problem with the /var tree permissions, and it understands html well enough to render frames and navigate among docs in my ebook tree. Plus it reads .TXT docs as well. But, this solution is underengineered. Unfortunately, while it tries to render PDF, large docs seem to cause it to puke, and it doesn't allow bookmarking. This means I have to laboriously type in '/var/mobile/media/dir/index.html' (switching keyboards to hit the slashes) every time I boot it. Suck suck suck.


Posted by jbz at 4:45 PM | Comments (0) | TrackBack

December 14, 2008

Caching eBooks for offline use on the iPhone

The esteemed fellow Ximianite Dave Camp pinged me to offer a solution to the problem I was kvetching about in my last writeup. He noted the following solution, which requires a web server to work - but if you're using a Mac, you can start up web sharing and hit it locally over your wifi. Anyway, in a folder on the web server, place an index file (let's say, index.html) with links to the various ebook files. These files need to be contained within the same folder (although they can be in subdirectories). At the top of the file, in the html tag, place the following:
<html manifest="books.offlineManifest">
  <!--- your page's html--->
Now create a file named (you guessed it) books.offlineManifest. Place this file in the same directory (at the same level) as index.html. Now the tricky part - modify your web server configuration so that files with the suffix ".offlineManifest" get served not as text/html but as text/cache-manifest. If you're using apache, the command you need to put into the configuration file looks like this:

AddType text/cache-manifest .offlineManifest

Restart the server if necessary to reload the config.

Now, inside the books.offlineManifest file, place the following:


Note that the book-title-1.html and the like are, of course, the location of the files that are pointed to by links on your index.html page. These files can be in subdirectories, but be careful - the cache manifest should not contain lines describing the subdirectories, just the files. So for example, do not do the following>

...the 'books/' line will cause the caching process to puke and stop. I mention this because, of course, I did it, being lazy and using 'find .' to get my tree info. DON'T BE ME.

All right, once you've got all these files saved to your webserver, use your iPhone and navigate to the index page. Once you've loaded it, click the '+' in Safari and seelct 'Add to Home Screen.' This will create an applet on the iPhone which loads that index. Quit Safari and then click that applet. It will reload Safari and bring up your index page. It will also start traversing the file list in books.offlineManifest and caching them onto your iPhone! Once it has done that, you'll be able to navigate to them using Safari even if the iPhone is not on the network, because Safari will realize it has them cached.

The down sides:

Well, yeah, there are some. For one, it means stuffing an awful lot of things into your cache. I don't know how that will affect usability. This feature is a) intended for smallish javascript files and the like, not megabyte+ book files, so YMMV. Also, as far as I know, it's an undocumented feature in Safari, so expect that Apple can change how it works or the fact that it's there at any time.

The most annoying thing, for me, is that it picks some strange order of files in my (really long, because I use exploded-by-chapter HTML books) file list. And, it never seems to finish loading my manifest - usually, the phone sleeps, or something silly, or it just stalls - and when it restarts (because I re-navigate to the page) it... starts loading them all over again. It might be most effective to have a manifest and applet per book or per author rather than having everything stuffed into one file. I found that when I limited the manifest to around twenty files, they all loaded fine. There is, however, no real way of telling what's loaded and how far it's gotten, because it's supposed to be invisible to the user (I figured it out by tailing the logs on my webserver while it was caching).

So there you have it. Thanks Dave Camp! Until the Safari file:// patch works on the 2.2 firmware, this is what I'm going to have to do.

Posted by jbz at 12:58 AM | Comments (0) | TrackBack

December 12, 2008

iPhone and eBooks, again

I recently upgraded JohnnyJohnny (my iPhone. If you don't know the reference, tough) to the 2.2 firmware. I hadn't jailbroken him since before the 2.0 release, and I was getting really, really sick of not having my eBooks locally stored, so I downloaded QuickPwn and went to work.

QuickPwn did exactly what it advertised, and I soon had a jailbroken-but-still-perfectly-functional iPhone. The only problem, I realized, was that the 2.2 firmware upgrade had brought a new bugbear - that of the Disappearing Wi-Fi. I found that I couldn't get the phone to stay on my wifi network, despite the fact that it hadn't been changed since I had been on 2.1. I tried everything - forgetting the network, bouncing into Airplane Mode, etc. - and no matter what, the damn thing would join my wifi net for maybe 30 seconds and then drop off. Bam. Even when I was sitting 7 feet from the antenna with nothing intervening.

I nuked the phone and reloaded a fresh 2.2 firmware, no jailbreak.

Nope. Problem still there. Damn it.

So I re-jailbroke it. No change, but at least I could try to work on my primary issue. I have maybe 500MB of eBooks, and in the 1.x days I had them all available on the phone itself thanks to two things - SSH/SCP, and the Safari file:// patch. Using those two tools, I could finally read my books on the subway. So using Cydia, I installed OpenSSH, MobileTerminal, and the Safari file:// patch.


Apparently MobileTerminal just...doesn't work. It runs fine, and give me a completely black screen, no matter what I do. It's like it just isn't running a shell. Don't know why. I could SSH into the phone with no trouble, though, so I did that and then SCP-ed my eBook directory into /var/mobile/Media/.

Nope. No worky. See, the new Safari file:// patch has a problem - No matter what you do, if you try to access a file that's in the /private/var tree (which is aliased to /var) you get a 'permission denied' error trying to look at the file. If you move the file into the / tree, it works fine - but / has only around 50 MB of space available and that's where the OS lives, so bad idea.

So I'm still screwed. I can't get MobileTerminal to work, and I can't get file:// to work in Safari properly. I know some people will just dismiss this with 'get a real eBook reader!' but phooey on them - I have books in .txt, .html and .pdf format. Some are in html directories (my favorite format, thanks Baen/Webscriptions!) so they need to be linked. I haven't found a product for the iPhone which will let me read all of these - except Safari, when the damn file:// space works.

So for the moment, I can still now only read books if I have network connectivity. Which is just completely @#&()(@# stupid.

Posted by jbz at 12:39 PM | Comments (2) | TrackBack

October 7, 2008

Frakking Spammers. Frakking zombie nets.

This is what happens when your primary email address somehow pops up onto the spammers' radar:

frakking spammers

Posted by jbz at 3:50 PM | Comments (0) | TrackBack

October 4, 2008

iPhone ringtone annoyance

Okay, so, there's two things that annoy me about iPhone ringtones.

One, you can't use them for SMS message alerts. WHY, APPLE?!?

Two, if you make a ringtone out of a song file (see bottom for how to do this) and that song file still exists in iTunes - as far as I can tell, if a songfile with the same name and/or metadata exists, that is - then iTunes will refuse to import the ringtone, and just play its existing song when you double-click on the ringtone file.

If you manage to get a short song into iTunes as a ringtone, and the actual song exists on your iPhone (say, as part of an album) then the ringtone won't sync over from iTunes, even if it shows up in the Ringtones section.


Oh yeah, the howto.

How To Make iPhone Ringtones

  1. Find a nice clip you'd like to ringtone. If it's not the right length, use a sound editor to snip it to the right length. Let's assume it's mp3, which is easiest to find an editor for.
  2. Once you've got the mp3, get it into iTunes (drop it in, it should import) and then right-click it. Select 'Make AAC version' from the pop-up menu.
  3. Take the new AAC version and drag it back onto the desktop. Once you've verified it's there, delete both versions out of iTunes and have it trash the files. You'll still have the AAC file where you just dropped it, and (if you were s-m-r-t) you'll have the original mp3 file somewhere still as well.
  4. Change the extension of the AAC version on your desktop from .m4a to .m4r.
  5. Double-click the .m4r version. It should bring up iTunes, and bring up the 'Ringtones' section (if you didn't have one until now, it'll pop up) with your file in it. You can play it from here or whatever. Make sure its entry is checked.
  6. Dock your iPhone. Go to the iPhone in your device list. One of the tabs for sync items should now be 'Ringtones.' Make sure your new ringtone is selected.
  7. Sync.
  8. et voilà. Your ringtone should show up in the iPhone's 'select ringtone' menu under the 'custom ringtones' section. Whee!
Posted by jbz at 12:29 AM | Comments (0) | TrackBack

September 9, 2008

Skype is made of fail.

Why on earth can't a SkypeOut account (like I have) connect to conference bridges which use toll-free numbers? That's just inexcusable. Every time I try, the Skype call gives me the 'call finished' message immediately after the normal 'wait to connect' time. What the hell is the point of this, then?

Posted by jbz at 2:57 PM | Comments (0) | TrackBack

August 22, 2008

Infrastructure Design Test

Can you spot the failure in this notice?

I'll wait.

Did you say 'RedHat let their servers be penetrated?' If so, good try, but no.

Did you say 'RedHat let their signing servers be penetrated?' Again, good try, but no.

Hopefully, you said 'Wait, their signing servers are accessible via network? WTF?' Because if you did, congratulations, you too saw the Epic Op Fail.

Posted by jbz at 1:12 PM | Comments (0) | TrackBack

August 18, 2008

Fuck Microsoft in the Goat Ass.

Why the hell is Microsoft Office the only program I use which has terrible problems with Spaces on OS X? I mean, I know Spaces isn't all that well-designed, but I start Word. Then I use Spaces to move the main Word window to another Space so that it coexists with the Powerpoint window I'm working on and I can flip between the two. Well, problems. Then every time I do anything, the damn Formatting pane and other toolpanes always manifest on some other Space, meaning that the whole screen goes zooming over to some other place entirely. I move the new pane back to the proper space, and ten seconds later, ZOOOP I'm somewhere else because some other damn pane popped up. Honestly, it feels like if you start a MS Office app in a particular space, anything you do involving new windows or showing/hiding panes will zip you back to that original Space, even if you've moved all the windows for the app over to another one.


Posted by jbz at 11:54 AM | Comments (0) | TrackBack

Why I use linux for my online presence

top - 11:11:00 up 789 days, 23:57,  1 user,  load average: 0.03, 0.03, 0.00

'nuff said.

Posted by jbz at 11:12 AM | Comments (0) | TrackBack

Paranoia and OS Security

This blog entry from ZDNet bemoans the fact that Apple employees ask you for your administrator password when you bring them your Apple computer for service or assistance. I agree that this is a bad practice in that they should never ask you for your administrator password. However, I think it's perfectly reasonable, if you're asking them to troubleshoot your computer, that you give them administrator access - because if you don't, essentially you're asking them to either spend the time and resources to break into it or to skip software troubleshooting and data preservation entirely and just reformat the hard drive.

There are options, people.

One: FileVault. I know it's been put down and bitched about, but generally, it's a good option in this case. You should configure a master password (so that someone else can't set FileVault and lock you out) as well. If you've done this, the only way that someone with administrator access should be able to gain access to your data is by changing your login password, so at least you'll have warning if they try.

Two: Encrypted Disk Images. If you're really worried about this, create (using Disk Utility) an ecrypted disk image and store your private data there. Don't put the access code in your Keychain. If you do this, even if someone has administrator access to your Macintosh, they won't be able to open your encrypted image no matter whether they copy it off the computer or try locally (unless they crack your password, of course).

Essentially, while I think it's a bad policy on Apple's part to try to get their users to hand over an administrator password to their current image, let's not get overwrought - it's not a good idea to ask someone to troubleshoot your machine without giving them access as well.

If your hard drive has gone bad, or is going bad, you should be able to format it before bringing it in - if you can't, it's unlikely anyone else is going to recover data off it either. In any case, if you're bringing in the machine for a flaky drive, then you should be wanting them to nuke/replace it.


Posted by jbz at 10:54 AM | Comments (0) | TrackBack

August 7, 2008

iPhone Game I Want To See

I realized that one of the best games I could think of to make the jump to iPhone is an old Macintosh favorite - Spaceward Ho! That would be incredible, and there's already a mobile version for Palm OS - although that version is slightly simplified, which is annoying.

I emailed the author idly and asked if it would ever be ported, noting that I'd happily pay $20 or so.

His half-encouraging answer: "Someday."

Ah well. Guess I should just look forward to The Force Unleashed then.

Posted by jbz at 2:17 PM | Comments (0) | TrackBack

July 16, 2008

Entourage/Exchange II

Bleah. So it works from his other computer. Looks like it's the local store on his Macbook Air, so time to nuke and rebuild. Oy. I was hoping it'd be a more fun answer.

Posted by jbz at 11:54 AM | Comments (0) | TrackBack

July 15, 2008

A Diverting Entourage/Exchange problem

A user here at work ambled into the geek cave with a problem. We use Exchange here at work, and several people (myself included) access our Exchange using Entourage, the Mac OS X client. His problem was as follows.

He has a Macbook Air which he uses as his primary computer. In addition, he'd just gotten his original iPhone upgraded to firmware 2.0 and set it to talk to our Exchange server. As a result, he realized that the calender information he could see on the iPhone wasn't the same as that on his Macbook. He logged onto the server using our Outlook Web Access (OWA) URL, and found that the iPhone and the OWA system (hence, the server) agreed on his calendar information - but that changes that had been made on his Macbook weren't syncing up to the server.

We began to test.

If I sent him an invitation to a meeting and he accepted it using OWA, it would appear on his OWA calendar and after short delay appear on his Macbook and iPhone.

If I sent him an invitation and he accepted it on his Macbook, it would immediately appear on the Macbook's Entourage calendar but not appear on the OWA (or iPhone) calendars.

If I sent him an invitation and he checked his iPhone, the iPhone would immediately put in a 'tentative' meeting at the proper time. If he then 'accepted' the meeting from the iPhone, it would change to 'confirmed' - but if we checked on the iPhone again after a few minutes, it would have changed back to 'tentative' but the invitation was still gone. It would not show up in OWA.

He can send and receive mail from any of the three clients - Macbook Air, iPhone or OWA. That works fine.

If he sent an invitation to me using OWA and I accepted it using my Mac (and Entourage) it showed up in my OWA and on my Mac.

If he sent an invitation to me using his Macbook and I accepted it using my Mac it would show up in my OWA and on my Mac.

So at this point, it appeared there was something different about his Entourage setup or his Macbook or his Exchange store. Note that I had changed my Entourage account settings to mirror his, and made sure my Mac was on the same wireless network as his prior to the above tests.

Next we created an account for me on his Entourage on his Macbook Air, using my account information. It took around 20 minutes to sync initially (arg) but eventually everything was there. I accepted an invitation using his Macbook Air and his Entourage but my account configuration - and lo and behold, it showed up on my OWA and on my own Macintosh.

At this point, we seem to have narrowed the problem down to either his local datastore on his Macbook Air or his Exchange store on the servers. We discovered that his store was 1.7 GB in size, and that the store that it was residing in has a policy set such that over 1.5 GB it 'blocks send and receive.' However, he can still send and receive mail fine either via WebDAV (Entourage/iPhone) or via OWA.

His store has a large number of items in it, and we noticed that although the appointments did show up on my account (which has many many fewer items, but still several hundred to a thousand) it took some twenty to thirty seconds to sync. We're going to see if his appointments have properly synced by tomorrow. If they have, the next step will be to try to prune his account and see if they sync faster. If they haven't shown up tomorrow, we're going to try to prune his store below 1.5 GB (the limit observed on the server) and see if that allows calendaring as well as mail to work.

Confused, but making progress.

Posted by jbz at 7:57 PM | Comments (0) | TrackBack

March 21, 2008

iTunes iPhone update server

This is info that is available elsewhere but it took me a while to figure it out from behind a work firewall. If you need to exclude the iTunes iPhone software update server from a block list, say on a work proxy server, use the following address:

Posted by jbz at 2:31 PM | Comments (0) | TrackBack

March 17, 2008

Glad to see I'm not the only one

...having trouble with Apple's 'Spaces' feature. While my frustration doesn't derive from anything as complex as that writer's ongoing requirements, it's the little things that drive me bats. My primary kvetch at the moment is the following. I have Entourage and Safari and Word in separate spaces. Entourage and Safari have been 'designated' spaces; Word I just 'opened' from another empty space. I only have one Word window open (well, two if you include the formatting palette) - so only one document.

As I'm typing in the word doc, the damn space will slide over to Entourage whenever the 'Activity' window pops up indicating that it's doing network tasks like getting mail. It will pull the Word doc *with* it. So suddenly, without any action on my part, I have the sliding animation, Entourage pops into the foreground, and my Word doc is now in the background on a different space. If I manually ctrl-# back to the Word space, it's now *empty* - because the doc got pulled over and shoved behind Entourage. It's impossible to get any work done.

Posted by jbz at 3:11 PM | Comments (1) | TrackBack

February 15, 2008

Stupid Entourage/Spaces behavior

I have no idea whether to mark this down as a Stupid Entourage Trick or a Stupid Spaces Trick. The problem, when using Entourage and OS X 10.5's 'Spaces' workspace manager, is as follows: I have bound Entourage to a particular workspace, so I can just hop over to it with a keypress when I need it. That's fine. However, I use an outbound mailserver which has an SSL cert that Entourage will not accept, no matter how many times I follow their stupid help tip and try to import the cert into my keychain (it's there. Entourage just won't see it). In any case, what this means is that the first time I send a message through that server during any Entourage session, it pops up an application-modal alert box warning that the cert is bad and requires me to click a 'confirm' button to allow the message through.

That's acceptable.

What's not acceptable is this. If I go to the Entourage Space, write a message and hit 'send' and then immediately (as I'm used to) jump to, say, my Safari space and continue working, I will get a 'bong' a few seconds later as Entourage pops up the dialog. I go back to the Entourage space...and it's not there. I search madly throughout every Space I can find. It turns out that it places the dialog *behind* the Safari window I was working on when it notified me - and bringing Entourage to the front, rather than popping that dialog, instead shifts me to the Entourage workspace, guaranteeing I won't find it. Of course, no Entourage window will respond to focus, since the application is modal and waiting on that (now hidden) dialog. The only way to get Entourage back is to locate the dialog by using Expose on the proper workspace, or by manually shoving around all my windows until I see it, and then dismissing it.


Posted by jbz at 1:23 PM | Comments (0) | TrackBack

November 5, 2007

Leopard Gripes III - The Disappearing Drive

Leopard seems to have installed with tolerable success on my iMac (i.e. nothing huge broke) so it's time to install it on my MacBook Pro. Okay. Fire up the DVD, reboot into the installer, select 'English,' agree to licensing terms and...

...there are no hard drives in the drive selection window. WTF?

Try running disk utility. It finds my internal drive but not the partition. This freaks me out. I exit Disk Utility without touching anything. The Macbook Pro was running beautifully on Tiger 1 minute prior.

Search the web. Find this thread at Apple's support forums. Wait, let me get this straight - during an OS upgrade/install, one of the more stressful and careful-tippytoe-making times of a computer user's experience, the damn installer just won't bother showing the drive until it's completed a drive check it doesn't tell you about?

What the fuck?

So I did as recommended. I got to the Drive Selection window and I waited.

Ten minutes later, my internal HD popped up in the pane.

That's...just...Apple, that's just fucking stupid. No warning at all that the thing is busily checking the drive integrity? No notice that it knows the drive exists at all? I'm so lucky I didn't panic in Drive Utility and decide to repartition or some such to get the thing to recognize it. (Well, okay, I'm not lucky, I know better, but a lot of users might panic...)


Posted by jbz at 12:04 AM | Comments (0) | TrackBack

November 2, 2007

Leopard Gripes II - a partial retraction

Well, I retract any assertion that Apple had deliberately hosed my Shares behavior. See, I'd tried the 'show connected shares on desktop' pref in the Finder Preferences (which somehow had gotten silently switched to 'off' during the upgrade). Didn't do anything. However, another reader on Macintouch assured me it worked, and an install to another machine showed he was correct. I did a full disk verify and repair on my main machine, then cycled the Finder, and then changed the setting - and there we are.

I hadn't done a full verify/repair during the install, only a perms check - so let that be a lesson to me.

On the other hand, I stand by my kvetching about interfaces. Also, as another friend pointed out to me and I verified, X11 is now limiting X11 windows to the machine's main monitor, which is...annoying.

Time Machine, after spending four hours or so doing its first backup, hung my machine when I came back the next day. It was hung during what looked to be a later backup, because instead of the 1.8MM+ files in the status window, it said something like 31k; however, it wasn't doing anything. It also wouldn't let me do anything related to disk access. Nor would Finder relaunch. On hard-cycling it after letting it sit there for 45 minutes just in case it was stuck on some big file, I found that Time Machine thought that there was no backup on the disk I'd designated - and my main drive was suddenly a Time Machine drive according to its icon and the fact that I had no write perms to it, although Time Machine's prefs showed nothing of the sort. A full reboot 'fixed' that.


Posted by jbz at 3:32 PM | Comments (0) | TrackBack

Leopard Gripes I

Wow, this whole Mac OS X 10.5 thing is causing me serious agitas. Major meshugah. Let's see, where to start?

  1. What the fuck, Steverino, is this whole thing about not being able to make aliases of network shares anymore? They now (AFAICT) are only accessible via a Finder window, from a section in the sidebar. The server shows up as one object in the sidebar, and clicking on it brings up a list of shares that it offers. I can apparently no longer choose not to connect to certain of those shares; nor can I move those share folders anywhere else or make aliases of them? What. The. Fuck. Serious dropoff in functionality, smotchkiss. I liked having aliases on my desktop.
  2. Which leads us to the whole base problem. I'm one of those retrograde users that really really like the spatial Finder. You know, the one they killed years ago but are now exorcising from the current finder piece by painful piece. The desktop is slowly becoming a place where I can't determine what lives there - only Steverino and company can. Fuck that. I want share aliases, and I want to not connect to ALL THE AVAILABLE SHARES ON A GODDAMN SERVER by connecting to one, and I want those share aliases on my goddamn desktop.
  3. The new Dock? Ugly as ass. Even when it's on the sides of the screen, where instead of a functionally useful area of the screen it now has been massaged to look just like an iPhone/iPod Touch (dark work area, rounded shiny edges). Ooooooh, how clever, I don't fucking think. And God, let's not even go into what it looks like on the bottom. Thank god Ars Technica came out with the wayback prefs tweak, instanter.
  4. Translucent menu bar? Nope. Fugly.
  5. Finder view prefs. Big, big, big, big fuckup. See Ars Tech/Siracusa for a detailed explanation. Essentially, 'view as' prefs are now fucking GLOBAL, with individual location overrides. But in order to set those, you have to go everywhere. They didn't preserve 'em. And it's being made real clear to me, from the number of times I've opened windows to find they've suddenly gone back to a list display, that my preferred spatial icon mode is no longer cool (assholes). I. Cannot. Fucking. Stand. Browser mode. If I wanted a goddamn file browser, I'd use Windows, or wait for some geek to write a useful one (which Finder really fucking isn't).
  6. Good christ, what a clusterfuck. I haven't even gone into Time Machine, because I haven't figured out how to get it to configure without first enthusiastically wiping all my non-Time Machine backups on the other partitions of the drive I want to use - and I don't want to give up my Tiger image yet, this is too damn scary. Plus, Spotlight is taking forever and a year to reindex everything.
Posted by jbz at 1:40 AM | Comments (0) | TrackBack

October 29, 2007

Entourage 'Unknown Error (170)'

If you're like me and spend a lot of time roadwarrioring outside your corporate Exchange network, as well as spend as much time as possible working on non-MS machines, you might find this cropping up in your Entourage one day when you try to get mail. I don't know what, precisely, 'Unknown Error (170)' means or why MS thinks that's helpful, but I can tell you this: in my case, it meant my domain password had expired, and I hadn't been online on a Windows box (or other piece of software capable of handling the password change) recently enough to get the expiration warning. ,p> Logging in to my corporate VPN using Parallels provided me with a password change dialog, and once that new password was entered into Entourage it became all happy again. I had to quit/restart it after entering the p/w in order to force it to refresh the mailboxes properly, but it did so fine after the restart.

Posted by jbz at 11:52 AM | Comments (0) | TrackBack

September 14, 2007

"Secure Connection Truncated" errors using Subversion + DAV over SSL

I was recently tasked with setting up a Subversion server using DAV+SSL, authenticating to a Windows domain controller. I fumbled about and did so, and it appeared to work okay, but when checking large trees in or out Bad Things would happen. Specifically, errors that looked like this:

svn: REPORT request failed on '/svn/repo/!svn/vcc/default' 
svn: REPORT of '/svn/repo/!svn/vcc/default': Could not read response body: Secure connection truncated ( 

...or like this, apparently random choice as to which:

svn: REPORT request failed on '/svn/repo/!svn/vcc/default' 
svn: REPORT of '/svn/repo/!svn/vcc/default': Could not read chunk delimiter: Secure connection truncated ( 

I had taken the following steps to try to ameliorate the problem already:

  • increased the timeout to 1000 seconds (which fixed an earlier connection issue)
  • tried disabling apache worker mpm and relying on prefork, as per a suggestion for someone losing connections using svn+ssh. That didn't work; re-enabled.
  • disabled http compression (by commenting out the loading of mod_deflate). Left that disabled just in case.
  • enabled keepalive and upped the requests per connection. No change.

Then, after investigating and watching apache while the checkout was running, I noted that the working apache process would grow during the checkout to consume appro. 27m of resident memory and then suddenly vanish when the error occured, briefly showing up in the process list as httpd <defunct>. Closer examination of the apache logs showed that these httpd children were indeed segfaulting.

This is a RHEL5 system using Subversion 1.4.5 compiled from source, being served by the distro's Apache (httpd-2.2.3-7.el5) over SSL using the distro's neon-0.25.5-5.1 at runtime AFAICT (although subversion was compiled with neon source in the tree, from subversion-deps). It is a fsfs repository on an ext3 filesystem (local disk) using mod_auth_kerb to auth to a windows DC.


So someone smarter than I continued where I left off and discovered that I was mostly right, there was a memory leak. But when I had tumbled off to get sleep, exhausted, he soldiered on, and tracked that leak to the DAV module (mod_dav_svn) in Apache. The problem, apparently, is that the DAV module was performing the authentication steps for every directory, every time it was accessed, and the leak was leaking during the auth process. Thus, in a large tree, the dozens or hundreds of auth steps would end up leaking the module into instability. The solution was to tell the DAV module not to reauthenticate for every path underneath the main one once the user had been authenticated into the repository. To do this, add the following statement to the apache config (in the vhost entry for the repo, in our case):

SVNPathAuthz off

...and in our case at least, all was well.

Posted by jbz at 2:29 PM | Comments (1) | TrackBack

August 8, 2007

MacLockPick and professional paranoia

Browsing around, I came across this How To Defend Yourself Against MacLockPick article. It seemed, in its flood of professional paranoia, to have missed a bet. If you're willing to discommode yourself to that degree, why not take a page from the NSA themselves and just disable USB storage on your Mac? That should prevent the USB-loaded tool from intruding in the first place, at least long enough for the intruders to have to figure out what you did and then load the drivers back into place using a CD or wireless - which means they'd have to perform a full single-user re-password of your Mac anyway, or do something that would take even longer, precisely what the MacLockPick is intended to avoid.

I'm not saying I'd do this permanently. But a quick script to swap those kexts out somewhere and reboot, then (if I needed to use USB storage) swap them in? Probably worth it if I was already going to all that trouble.

Posted by jbz at 11:18 AM | Comments (0) | TrackBack

June 20, 2007


Apparently my home IP address is listed on the DSBL blackhole list. I first discovered this when a friend who runs his own mailserver helpfully informed me (during a test we were running) that mail I sent him via my own offsite mail server or my work Exchange server got junkfiltered by his SpamAssassin setup; poking about, it turns out it's because my dynamic IP is on DSBL.

This confused me a bit, since I don't send mail directly. Both of those messages were sent through SMTP servers, one I run privately from a colo facility and one run by my employer in the NY Metro area. As far as I could tell after checking, neither of those servers was listed on the DSBL. However, the SA module that does the checking apparently checks the first hop in the mail headers, and sure enough, there's the IP that the message originates at in both of them, and it's my Comcast IP.

So. Doesn't matter that I'm sending through SMTP servers, I'm blacklisted. How to get removed? Ah, there's the rub. DSBL will remove you if you can respond to a confirmation email they will send to you. That email can be sent to one of four possible email addresses, and their site tells you flatly that there are no exceptions. Those addresses are:

  • postmaster@(your reverse dns FQDN)
  • postmaster@(your toplevel reverse dns) - in my case,
  • abuse@(your FQDN)
  • abuse@(your reverse DN) - in my case,

Now, obviously, the addresses are useless to me. The problem is that the others, which accurately point out the IP address of my cable modem, are equally useless. This is because I don't run an SMTP mail server out of my house. Even if I did, Comcast, in its attempt to help prevent SPAM, actively blocks that port on my segment or perhaps modem, I'm not sure (I use secure SMTP on a different port to reach my outbound servers). So what the DSBL is saying, essentially, is that the only thing I can do is to have their system generate an 'annoyance' email to comcast's abuse address which will be ignored.

I'm generally a proponent of blackhole lists. However, I'm also a proponent of responsible administration of them. The problem here is that (as DSBL acknowledges) the range of IP addresses that mine falls into is dynamic and public access. This means it is both a high-threat range and one which churns, meaning that harsh measures taken against addresses in the range will in time impinge on other, innocent customers as the addresses 'turn over.'

Realizing I might be overreacting, I went to check the report on my IP address to determine what, in fact, it was being blacklisted for.

Turns out that there was an open SOCKS4 proxy on my IP sometime back in November of 2005, approximately seven months before I was assigned this IP. Um. Okay. So here's my question - why is there no way to retest - or request a retest - of an IP? I understand that an automated retest would simply invite gaming. I also understand (and have some sympathy) with the notion of 'let the customer tell the ISP to fix itself.' However in this case, the ISP isn't the problem; the problem is what some unknown person did with this IP before I had it, and there's no way for the confirmation process to reach me because I don't control the DNS or the abuse/postmaster accounts for this domain. I do control the IP address, but I don't (and probably can't) run a mailserver on it. Hence I can't receive an email addressed to its specific reverse FQDN, because there is no internet mail service at that address.

Admittedly, part of my particular problem seems to be that the site I'm sending to is penalizing me for being on the 'single hop' DSBL list when I'm clearly not sending single-hop - I'm sending through SMTP servers. I'm not on the multi-hop list, although I am on the 'unconfirmed' list. However, I know for a fact that site is using standard SpamAssassin filtering, which means whatever it's doing can't be that esoteric.

So what am I to do?

The fucking system's broken, boys.

Posted by jbz at 2:01 AM | Comments (0) | TrackBack

April 27, 2007

The iPhone and Mobipocket titles?

I'm quite looking forward to the iPhone, indeedy-deed. I use a Treo 650 for my personal phone and a Blackberry 8700 for my work phone, and I have serious hatred issues with both of them. The Berry, while satisfying my 'always on the web' cravings tolerably well (I don't consider EDGE too slow for handheld use) suffers from not having a media slot and being an absolutely abysmal phone. I want my cell to be a phone, damn it. My Kyocera 7135 was a phone first and a Palm second, and while that gave its Palm side short shrift sometimes, that's what I wanted. If I was more concerned about carrying portable computing power I'd carry a frigging OQO.

The Treo 650 is still soldiering on after two years of hard use, for which I must give it full marks. It's scarred and beaten, but unbowed. It has a completely useless camera and a more useful MiniSD slot, big plus on the latter. The reason for my approval here is that the Treo also serves, in around 95% of its non-phone use, as my pocket book reader using Mobipocket's software.

I buy my e-books mostly from Baen Books' Webscriptions site - DRM-free Mobipocket format sci-fi. I have maybe 135 titles in my Palm at any given time, pretty much guaranteeing that if I find myself stuck in an airport or train station for twenty minutes, I can at least zone out to a fun book.

The iPhone, it appears, has little chance of gaining Mobipocket functionality. If Apple is being as stingy with dev kits as is being whispered, that doesn't bode well. So what to do? This is sort of a big deal for me. As I see it so far, I have a few options.

  • Hope that a) there will be an Apple-approved ebook reader and that b) I can convert the titles if need be.
  • Find a hack.

Well, I'll be ecstatic if a) comes to pass, but I've been noodling about b) anyway. It occurs to me that at the most industrious end of the scale, I could try to get a Mobipocket-format capable reader that is implemented as a Java Applet and which would run inside Safari (although, I must admit, I have no idea if there will be a full JVM on the iPhone). That leaves only the problem of where to store the books, at least, such that the applet could read them.

I might mail them to myself, embedded in HTML. I wonder if the iPhone would cache them in and then open them with a reader? Perhaps I could render them as PDFs, first. That might help.

Hmf. It all seems so convoluted.

Posted by jbz at 10:39 PM | Comments (1) | TrackBack

April 23, 2007

How to receive nervous looks on the train.

I have found a new and effective method of garnering furtive looks of horror aboard public transportation. It involves, as did my previous efforts, my trusty Powerbook. My favorite used to be the watching of abysmally bad movies, complete with occasional happy chuckle despite headphones; luring seatmates into covertly looking over and seeing their faces freeze as they recognized Freejack or John Carpenter's Vampires is worth the temporary loss of brain function that watching said movies will cause. Besides, when trapped on an Acela or 737, sometimes that loss of function is desirable anaesthesia.

But no, I have a new passion.

Its name is DEFCON.

Introversion Software, who brought us the game of hacking called Uplink several years ago and more recently the critically acclaimed vector Real-Time Strategy title Darwinia, have come up with this gem. Reusing a great deal of their simple but effective artwork from Uplink, they have created an icon of my 1980s nightmares and dreams.

Everybody Dies.

This is an exquisitely succinct summation of the game itself. The game, as can be deduced from its title, is no less than a playable simulation of your favorite and mine, Global Thermonuclear War. What makes it brilliant is the visual design and the playability. Let me take you through a quick notional game by way of explanation.

The Setup

You choose a Bloc to play. You can play North America, South/Middle America, Europe, The Soviet Bloc, Africa, or Southeast Asia. Up to six players at once can thus face off. Choose a color. As the game begins, the world is at DEFCON 5; a timer indicates the time until DEFCON 4 is reached. The assumption, I presume, is that those damn diplomats have screwed everything up and it's up to you (the warfighter) to make good on their threats. Operational diplomacy is possible; Diplomacy (the game) style maneuverings are encouraged, as alliance blocs can be joined and broken during play! Ah, backstabbing.

Enemy Launch Detected!

The Pieces

In the interest of avoiding information overload, there are really only a few play pieces, which you must place] somewhere in your territory at the start of play (during the DEFCON 5 period). On land, you can situate radar stations, airbases and silos; at sea, fleets consisting of battleships, carriers and/or submarines. Airbases can launch either fighters or bombers. Carriers can do the same, as well as go into antisubmarine mode. Silos can operate in either air defense or ICBM launch mode. It takes time to switch between modes, and units have a limited number of expendables - fighters, bombers, or ICBMS. SAMs are infinite and simply have an interval timer, and fighters will slowly regenerate. Submarines can operate in passive sonar mode, where they merely move from place to place and try to avoid being seen; active sonar mode, where they can (poorly) attack surface vessels, and SLBM launch mode where they can loft their limited number of nukes.

Launching anything, naturally, makes your unit visible to early warning systems, even if it's not currently within radar range of an enemy unit.

ICBM launch mode: 128 seconds

Gameplay Gameplay moves in phases, each phase being a different DEFCON level. At DEFCON 5 ("peace") all you can do is place your units and give mobile units movement orders. At DEFCON 4, you can invade enemy territory/airspace, and units will automatically engage each other; at 3, you can launch bomber strikes, etcetera. A large and foreboding timer scrolls down at the center bottom of the screen: DEFCON 4 in 13:45. This is a RTS, so units move continuously; you can change their orders at any time by simply clicking and dragging. If you are playing solo, you can change the time compression using a standard set of arrow controls (one, two, three or four arrows). If you are playing in a network game, you can requesta time compression change, and the other players will be given a chance to aquiesce or refuse.

I haven't yet had the opportunity to, entice any of my friends into purchasing the game and having a go 'round, so I can't tell you how well the alliance/diplomacy bits work. If they work as well as the rest of it, I'd say I'll be quite happy with the mechanics. The interface is smooth; there are a few UI design issues which are eminently fixable. For example, the game field (a Mercator projection world map) scrolls smoothly about as you move your cursor near to the edges. However, the time compression controls are at the top of the screen, so moving to them causes the field to scroll as you approach. You can zoom out to fullscreen to prevent this, but it's annoying. It's also possible to have the unit info window (fixed at the bottom right) actually overlay units on the map. Still, these are minor quirks; the interface is generally only noticeable in those few moments when its deficiencies become jarring. The rest of the time, it just is, which is what you want.

There are various modes of play. There is a standard game; there is a 'quick' game, compressed even further. There's one I'm dying to try, called 'office,' which is played over a network with friends in realtime over the course of a day - missiles take ~20 minutes to traverse their paths, and you have time to sweat and watch everything go to pieces in realtime slow motion. The network game is played by a player creating a master server and others joining. Thanks to Ambrosia Software, there is now a Macintosh OS X version of the game (which I'm using) in addition to the original Windows version; Ambrosia also promises a patch to allow Linux users to get in on the megadeaths. It's OpenGL and plays smooth as butter on my 1 GHz Powerbook G4, and is distributed as a Universal binary for Macs.

Beograd hit: 1.4M dead


SImple. As the game says: Nobody wins - but maybe you can lose the least. You can try to play a full counterforce game, but the game is scored in terms of how many kills you get, how many civilians you lose, and how many nukes you have left at the end. Once DEFCON 1 is reached (and the nukes really start flying) it's not about saving 'em for later, trust me.

The design is really excellent. It's simple strategy at the outset during placement - pick your enemies and place your units to face them; choose a defense-in-depth versus a frontal wall, etc. During the game, it's simple maneuver - if you're careful, you can maximize your units' placement - but it's not a true maneuver wargme, because the timers ensure you don't have the game length to really move units around the board. You have the time to adjust your units' initial placement, and that's all. Finally, during the Global Thermonuclear War phase, you get to push the button and see how well you set up your armageddon.

Incoming Nuclear Weapons: Impact in 348 Seconds

Look and Feel

This is the genius bit. The entire game is built using vector graphics, brightly lit on a black field. The look of it closely hews to the displays prominently featured in the iconic '80s movie Wargames - if you can remember at all the scenarios that Joshua was spinning out on the displays in Cheyenne Mountain, then you know pretty much exactly what DEFCON looks like on your computer. The same hazy glows to indicate unknown territory; the same angular bright line icons to indicate units; the same sprouting flowers of arcs to show the moment of truth when the missiles climb out. That's what always arrests the attention of the passenger next to me - that impossible to miss image of a world map drawn in glowing blue-white lines, with green and red bullet shapes climbing slowly out from the continental US and Russia, leaving arcing lines behind them as I frantically click around the Atlantic ocean, moving submarines and carriers into position for the followup strike.

The only thing missing is the haunting music.

Oh wait; no it's not. The game has that too. Turn your speakers up.

Introversion Software ( - Windows Version
Ambrosia Software ( - Mac OS X / Linux(?) Versions
Shareware - Playable Demo (limited to 1 AI player and 1 game mode, only 1 demo player per network game) downloadable; $25.00 for full license code.

Posted by jbz at 8:17 PM | Comments (0) | TrackBack

January 9, 2007

Linden Labs, Free Software, DRM and the state of the world

Jacked in yesterday and saw that Linden Labs had GPLed the source to the Second Life client. Via Luis Villa, I meandered over to sogrady's take on the subject and started musing. I'm not a Second Life fan, mostly for the reasons that others have articulated with much more clarity than I could - it bores me, it crashes, and I refuse to put all my eggs in someone else's basket. But there's something here which does interest me.

Linden may be running up against something which I and several of my friends spent a great deal of time thinking about.

If you have a virtual environment, that is, one which is compelling and pervasive - or if you're going to have one, you're going to need a draw. You're going to need a reason for it to be there, and a reason for people to come play there. The web came into its own not because it was technically sweet, not because it allowed people to share knowledge (which was its original purpose) but because of commerce. A persistent multiple user virtuality is a much more complex undertaking. I posit (and freely admit that indeed, my PoV is colored by my upbringing and background) that here, too, you will need commerce and value in order to bring people and organizations into this brave new world.

Linden knows that. That's why they have an economy, after all.

Here comes the problem. In order to have property, you have to have property rights. In order to have property rights in a digital environment, you need to have digital rights management. This is not news, zillions of other people have said the same thing. I am struck, though, at how few decent answers to this conundrum there seem to have been presented, and how many of the 'copyfight crew' seem to also be ardent pushers of the Second Life world. The take-no-prisoners fight against DRM as a technology, a concept and a practice runs directly counter to one of the most powerful motivating factors of there ever being a worthy persistent and pervasive virtuality that we can all play in.

It's fairly easy to rail against DRM on the Web, because the use of said information occurs in a space other than on the Web itself - consumption of digital media by a consumer occurs in a private space, usually, since 'use' on the web is akin to 'publishing' - there's no interaction. Plagiarism, after all, is much more clearly identifiable as 'bad use.' But in a persistent virtuality, a user might 'use' a piece of information in much the same way as they do in meatspace - by listening to music, or watching flat video - for their personal consumption, making the question much much hazier. In other words, the existence of electronic data inside this virtual environment is suddenly directly problematic, as opposed to the web, where it was mostly a publishing and transport problem. Now, it's not just publishing and transport (which is fairly easy to litigate, regulate, monitor and lock down either legally or technologically). It's the very existence and utilization of this material at the endpoint which matters.

So. Where does this leave me, in my rambling state? I guess what I'm trying to say is that while the uses DRM is being put to may suck just as badly as those decrying it say it does, and while DRM may be just as technologically futile in its present forms as they say, the very existence and goals of DRM may not be unremittingly evil (as it's being presented). I think what worries me is that at some level, the cry against DRM, and the copyfight movement in general, taken to extremes, can be seen as a cry against the notion of property rights in general when applied to electronic media. That's a problem for me, because in my vision of the future there is, in fact, a persistent and pervasive virtuality that I want to play in, and it's supported and built because various people and entities think there's value to be had there. That value is realized based on the notion of the existence of property rights in a virtual space. If the current fashionable trend of the decrying of copyright protection continues, those very property rights, which might attract the investment and effort required to make that playground a reality, may be threatened.

There are indeed, in my opinion, massive problems with how copyright law and DRM is being used and interpreted today. The DMCA is a prime example. However, trying to 'fix' the problem by attacking the very concept itself is, in my opinion, counterproductive. Working to fix the law, and working to fix the abuses, is better and safer; while using extreme responses to an extreme problem is always tempting, it runs the risk of creating situations just as hard to recover from later. If we end up with a world with no RIAA, but where no corporate entity is willing to invest in a virtual world because everyone using it thinks CopyBot is perfectly normal behavior, what then will we do?

Posted by jbz at 10:56 AM | Comments (1) | TrackBack

January 5, 2007

Google and the Reality Filter

This is interesting. I just went to Google News and saw a big white area on the page. Investigation showed that my 'World News' section was...blank. I opened the prefs and told it I wanted 9 stories in the World section, refreshed...nothing, still empty. I clicked 'World' on the left hand bar, and got a full-page of the 'World News' feed...and it was blank, empty.

Bemused, I logged out of the current Google account I had logged in in my browser for GMail access.

The World news reappeared.

What the hell?

I have no idea what's going on. My prefs on that user (in Google News) all show that I'm asking for World news to be my first (after the Top) visible section, and that I want 9 stories. The only thing that strikes me is that there's a notice which says 'New! Search History now includes News Items' or some such, implying that my search history might be used to select News of interest or vice versa, and that it's a new feature. I can imagine that a bug in that algorithm might cause a user profile from a GMail account I use for specific purposes (read: Google Analytics and some website registration) to show up as 'completely disinterested in world news.' I suppose.


Posted by jbz at 4:59 PM | Comments (0) | TrackBack

December 4, 2006

Microsoft and Massachusetts, redux

Around a year ago, I blogged about a colleague's experiences watching Microsoft move in on the Massachusetts ODF adoption process, and the tactics they used. Apparently, it was even worse than we knew.

Posted by jbz at 3:58 PM | Comments (0) | TrackBack

October 13, 2006

OMG! I lost my RPM Database!

Yeah, so what do you do then? All the sources say...pray. Or reinstall. Or take some convoluted set of steps that are akin to dancing thrice round a cattail thrush on the third full moon of the year while wearing the ooze of a great-grandfather snail and chanting the nine billion names of God.

Well, here's what I did.

First, the situation: I have a Red Hat Enterprise Linux 3 machine. For reasons we won't go into, the RPM database was entirely erased. Not just corrupted; it was hurt so badly that a new, blank one was put in its place, and rpm happily claimed the machine had no packages on it. For other reasons we won't go into, reinstalling on this machine was possible but not desirable; the machine is remote (in a colo) and is running a role that would have required the building of a substitute box, switchover, travel, reinstall, etc.

No, it's not a critical role.

Anyhow, the machine in question hadn't been updated for several months - it was this overdue update that had caused the freakout. I was using the Ximian/Novell rug/rcd product to update it - this system front-ends RPM, though. It had resolved what needed to be done, resolved dependencies, downloaded the requisite rpm files for the update into a cache dir, and called RPM to remove the conflicting (old) RPM files when Something Went Wrong.

After thrashing around trying to rebuild the db using the tools (failed), trying to create a new db by figuring out what files I had on the system (bwahahahah), and looking at the originally-installed rpm db state file in /usr/lib/rpmdb (as opposed to the /var/lib/rpm/rhel-3as-i386/redhat location of the running database) I came within a hair of giving up.

Then I found out that rpm sticks a cron job into RHEL installations (and FC installations?) that simply dumps the output of rpm -qa to a logfile (/var/log/rpmpkgs) once a night. That logfile, thanks to logrotate, is rotated once a week, so although it had been several days since the incident and the main file had been overwritten with the blank rpm dump, I grabbed /var/log/rpmpkgs.1.

Inside that file was a one-per-line list of everything that had been installed on my system.

I took that file to a machine which had the full RHEL3 current package set mounted to a directory. I then created a new, blank RPM database:

rpm --dbpath /home/jbz/newrpmdb/ --initdb

Then I used the mounted copy of the package repository and xargs to do a series of rpm -i --justdb commands, which 'install' rpm packages but only perform the db modification steps:

cat rpmpkgs.1 | xargs --replace={} rpm -ivh --force --nodeps --justdb --dbpath /home/jbz/newrpmdb/ /(path-to-rhel-3as-i386-packagerepo-rpms)/{} > install.log 2>&1

The --force and --nodeps were required because I was installing one rpm at a time (via xargs) to a blank database, so all dep checks would fail, but I didn't care - the packages were already installed on my machine, I just needed the DB to reflect that.

Once that was complete, I had a directory containing my new rpmdb file in /home/jbz/newrpmdb. I copied that to the affected machine, put it into /var/lib/rpm, and then performed an rpm --rebuilddb just to be safe. That completed without a hitch.

At this point, I had a mostly-complete installation. There were a couple of problems, though. The main one was that since several packages had been updated since the machine had last been rereshed, some package names in the package repository didn't match the package names in the rpmpkgs.1 file. As a result, the logfile (install.log) contained a bunch of lines showing errors where rpm couldn't find the file it had been asked to install since the current repository had new file names.

Note: This is because Red Hat (and others?) when updating the packages apparently change the package name by infixing versions. For example, suppose aspell- is updated; the new package will have its true name, and that version might be renamed aspell-2: I presume this is in case there are dependencies which rely on the earlier version of the package; it's kept in the update channels but marked up so that the updaters understand that it isn't the current version. This is a guess, though.

In any case, I had several packages which hadn't been installed, spread throughout this massive logfile. I used the following to get a simple list of which ones got missed:

grep "No such" install.log | cut -c 70- - | cut -d " " -f 1 > missingrpms.txt

Note that the '70-' reflects the fact that the pathname of the repository I was using was 70 characters long, so cutting the first 70 characters off the line produced output beginning with the filename in question. Your mileage will vary. The second cut statement sets the delimiter to a space, then discards all text after the space, leaving only the filename.

I then got lazy and, since my list was only a few lines long, manually copied those RPMs over to the affected machine and force-installed them using --just-db. You may need/want to script this update.

At that point, I ran rpm -aV to verify the installation. I came up with approximately forty lines of variance, mostly permissions variance, which is not at all out of whack for a 350-day uptime, 2.5yr install server. No major issues have cropped up. Looking through the variances, they all appear to involve config file changes, permissions on doc files, or (in five or six cases) missing files from packages I have customized. I will mark this 'successful' and count myself lucky.

WARNING: Your mileage WILL vary. Messing with your RPM database is NOT FUN and should NOT BE DONE on a running system unless there is NO OTHER OPTION. I had no other option, so that's why I did it. I take no responsibility for any damage you may do to your system or data in trying any of this; I offer it purely as a last-ditch recovery option, just in case you find yourself with nowhere else to go.

Good luck.

Posted by jbz at 3:50 PM | Comments (0) | TrackBack

September 11, 2006

A Technology I Mused About

A while back, I mused about how to secure a video stream against tampering or modification. I was concerned with the production end; that is, with making changes to the actual video which would provide proof of integrity as well as other metadata for legal use later.

Others who are actually qualified have been working on this problem. This gentleman has been approaching from the other end, that of forensic examination of unmodified data; at the moment, his work involves still imagery but he appears to be moving into video.

So there is interest in proving video (un)modified. I will acknowledge that the applications are much larger when applied to 'standard' video. Still, I can't help thinking that if there is enough doubt to create this field of study, then a 'verified video stream' might be worthwhile.

Posted by jbz at 1:16 PM | Comments (0) | TrackBack

July 24, 2006

Um, yeah, so, comments work again.

...because, like, I'm a moron, and just assumed that nobody was interested in leaving comments. Which might still be true, but someone actually told me IRL that "hey moron, comments aren't working." So I checked, and sure enough, I'd locked 'em out with an overaggressive mod_security setting. Note to self: poor self-image is a hindrance to proper spamfiltering.

If you didn't notice or don't care, please cheerfully ignore this message.

Posted by jbz at 10:27 PM | Comments (1) | TrackBack

July 19, 2006

Coalesce a little faster

Ooooh, I love it when vaporware shows signs of life.

Man, I still remember QuakeWorld TF2 as the best team shooter ever. If that is actual engine footage, this has the potential to be a complete friggin' hoot. Cannot wait. Wheeeeeeee!

Posted by jbz at 11:30 PM | Comments (0) | TrackBack

April 9, 2006

Apple Boot Camp Gamer Glitch #2

This one is waaaay more annoying. Once in a while, my iMac just loses its USB keyboard. This doesn't appear related to games, either - it has happened while sitting there looking at a blank Windows XP desktop, doing nothing else, no other devices on the bus but my Apple keyboard and Mighty Mouse - just *BONG* and then nothing, no keyboard. On all but one or two occasions, the mouse itself still worked (plugged into the keyboard hub) but the keyboard did not. The CAPS LOCK key was lit (despite my not having hit it) and wouldn't un-light.

Worse, it happens frequently enough that a good solid game of Battlefield: Vietnam guarantees it happening at least twice.

When it happened in Windows alone, I went into the Device Manager and looked, and there was indeed a USB Device in the Human Interface category that had the deadly yellow question mark on it. However, when I unplugged the keyboard and reconnected it, causing it to come back, that device was still there - and when it happened another time, Uninstalling that device and then performing a scan didn't make things better. It looks like the keyboard itself is actually crashing.

This happened three times before I managed to install any software into my Windows XP image, so I don't think it's something I did. Bugger.

Update: Well, hm. I moved the keyboard to a powered USB hub so as to avoid having to stand up and fumble behind the iMac and scratch up its Shiny Plasticky Goodness every time this happened. This changed things slightly; every few times the glitch occurs, the unplug-replug-keyboard trick doesn't work, and can only be remedied by actually unplugging the hub. When this happens, though, Windows goes through its whole "Oooooh! New hardware, goody goody gumdrops, I get to make you reboot now!" routine (although it just puts up a nagging dialog, it doesn't force an immediate reboot, presumably since it's just a USB hub). The interesting thing though is that when I do reboot, it says it can't because the program 'AppleCDEject' is Not Responding.

Interesting. Presumably whatever bit o' software Apple uses to watch for that handy Eject button on the keyboard is tanking. I don't know if that's causal or symptomatic, though.

Update Update: On the advice of a colleague, I spent some time trying to figure out how to prevent AppleCDEject from loading at startup. It's not in Startup Items, nor is it obvious in the Services that are coming up (although I did gleefully kill off stuff like the Theme Manager and other useless frippery). As a test, though, I booted XP, went to the Process Manager and killed it manually, then gamed for two hours without a single keyboard error - so things are looking up. One place I haven't looked yet - there is a single 'Apple Drivers' service loading, I didn't carefully look to see if there are arguments to that service which might be telling it to load the AppleCDEject deal.

I should also try a non-Apple USB keyboard (gotta find one first) and then look to see if it loads.

Update Update Update: I still haven't checked to see if there are keys in the registry that govern the startup of the AppleCDEject program, but as a stopgap measure, I just moved the executable out of the C:\windows\system32\ directory into a temp dir, and it doesn't execute on startup. I notice that I still do get the occasional 'hiccup' in the USB drivers (loss of input, multiple BONGs, input comes back) but they seem to pass within five to ten seconds and the keyboard doesn't crash anymore - so it's still an improvement.

Posted by jbz at 4:50 PM | Comments (2) | TrackBack

April 7, 2006

Apple Boot Camp Gamer Glitch #1

So aside from the spineshivering freakazoidicalness of watching blocky, hideous Redmondoid WinInstaller text scroll around the panel of my shiny new iMac, Apple's Boot Camp public beta seems to work quite well. There are several limitations made known to you in advance: you must have your hard drive partitioned in the default manner (i.e. a single Mac HFS+ Journaled partition, nothing else) at the start, and if you use NTFS for the Windows filesystem, OS X can't by default read your Windows partition. The clock in the iMac isn't exposed properly by their BIOS emulator, so unless you use a network time server, it will keep coming up wacky when you boot into Windows.

I found a couple of my own, some more annoying than others. Here's what I have so far.

  • The ATI display drivers always want to warn me about 'disabling TV display mode' when the XP partition boots, no matter that the drivers don't seem to think I have a TV connected or enabled or that I tell it to not warn me about such things, save, and reboot. Still comes up.
  • Getting CDs out of the drive, until you have installed the Mac driver disk and can use the keyboard eject button, is tiresome. Thankfully the MightyMouse 'right button' does work since it seems to be a hardware function of the mouse itself.
  • Here's the Gamer Glitch: The SigmaTel audio drivers didn't understand I'd plugged headphones into the back until I went and told them so manually, and most problematic they don't seem to be able to shut off the internal speakers at all. No matter what I do, the internal speakers remain on. Manually specifying that I'd plugged in headphones got the headphones themselves to work, but the speakers work as well...which makes gaming late at night not too cool given that I have light-sleeping neighbors. Sigh.
More as I find it. Still, this is a tickle. It irritates me to hear all these analysts go on about how 'this new approach won't help Apple' when in fact Apple has a long history of offering means to run Windows/x86 software, albeit usually pricier ones - remember the DOS Compatibility Card?

Posted by jbz at 2:34 AM | Comments (0) | TrackBack

February 16, 2006

For a Clue as to Why Our State Needs Open Source

...look no further.*

* Okay, this isn't totally fair. Every state in the union has a Springfield. But still. State government IT. Shudder. At least ours is making the effort.

Posted by jbz at 2:24 PM | Comments (0) | TrackBack

December 7, 2005


I know I sound like a shill. But if you use OS X and are a lazy sot about backups (like me) then get this. NOW. It rules. I have two physical drives in my Mac, set to mirror nightly, so that if one of them dies I don't even have to 'recover the array' or anything, I just remove the bad one and boot off the other. I have used it around ten times to restore my system from the 'backup' drive to a preinstall state after removing iffy software I was testing. All for $28. If you don't have multiple drives, it's marginally less safe but easier to just use multiple partitions on a drive - doesn't protect you against physical failure of the drive itself, but does protect you against corruption of the drive partition. Combine with cron for nightly happy. (Update: the newest version has prefs for automatic runs, and will insert itself into crontab without the need to resort to add'l software or editors. W00t.) Simple software ftw.

Posted by jbz at 10:39 PM | Comments (0) | TrackBack

November 21, 2005

The Tribulations of Gir

Hmf. Well, as far as I can tell, at least one of Gir's problems is that drive sector 0 is bad. This is Not Good(tm), as this is where the MBR and hence partition map are written. Thus, whether I succeed in marking the *other* bad sectors as such in the FAT or not, the likelihood that he'll remain stable is quite low. I may be wrong about this, because sometimes it *does* write the MBR, just after much click-thrashing and the like. One problem I have is that the MacOS is too damn enthusiastic about automounting and/or autotwiddling any volumes which 'appear' - like, when he's plugged in. I haven't been brave enough to just go kill automountd (and launchd) in order to prevent this. That would be a logical next step.

If, however, I'm going to start mucking around with my Mac at the process level as well as on the disk bus, I need a better backup than the one I have now - or to at least disconnect my media drive first.

Posted by jbz at 10:02 PM | Comments (0) | TrackBack

November 18, 2005

Sharp Tools Too

Okay. So if sector zero a.k.a. the FAT on disk3s1 (disk 3,, partition 1) is damaged, then the obvious hack is to move sector zero of partition 1 somewhere else on the disk. The partition table appears to remain undamaged amongst all this hackery. So. After trying multiple methods and means of tweaking the partition table (Disk Utility=not enough control, diskutil same thing, fdisk complains about resource busy, desperately trying dd if=/dev/zero of=/dev/rdisk3 bs=1024 count=3 results in no write) I discover that...

OS X is trying to background fsck the stupid partition, despite the fact that it doesn't work.

Kill that process. Try again w/fdisk. Ah! Okay, now partition one is offset by a few (disk) sectors. Now run Disk Utility again, and erase the volume as MS-DOS...meh. Well, it appears to be working, but very slowly, with that wonderful 'CLIK' noise of 'bad read/write.' Hm! On the other hand, the log window of Disk Utility hands me all sorts of wonderful info, like the backup sector location, sector numbers...the stuff that the code earlier was hardcoding.

Gack. I wonder if the iPod firmware stuff is going to assume disk sectors and/or nuke all this.

Meh. Gotta sleep. More later.

Posted by jbz at 2:08 AM | Comments (0) | TrackBack

Sharp tools in truth. Ow.

So li'l Gir died. So sad. I replaced him. However, it continues to rankle that an otherwise perfectly good, complex device like a 40G iPod has to be called 'dead' and discarded/recycled because the HD is acting up. When this first started happening, I tried to figure out if there was a way to just map out the bad sectors on the drive, since there did appear to be particular sectors that caused the poor thing to freak out when hit. Unfortunately, I couldn't figure out a way, because I'm not as smart as yer average bear.

Idly trolling the net, however, I found this - where someone else with a similar problem apparently had the same thought! And unlike me, knew enough C to make a dangerous stab at it!

Intrigued, I wished to subscribe to his newsletter. Er, no. That is to say, I downloaded the code, untarred it onto my Mac, and had a look. Apparently, his solution to the 'I'm not sure how the heck bad sector tables or their equivalent work in HFS+' problem that forced me to give up was much more elegant - he simply formatted his iPod as a FAT32 volume. The iPod doesn't care, and will function as either on a Macintosh - and there's much more info available on FAT32 FAT muckery. In particular, rather than telling the drive that the sector was bad, this gent proposed to tell the FAT itself that the sector was no good.

The downsides of this approach are obvious from reading his page - the modified FAT, with the bad sectors marked, wouldn't survive a reformatting/re-iPod-firmwaring of the device, since those would write a new FAT to the disk. The page itself came about because after several months of using his 'hacked' 'pod, he himself had to flash it, and was forced to go revisit his quick hack to determine what it did and how so that he could do it again. This time around, he figured why not document it (sorta) on the web?

Anyway, I find myself with a few terminal windows open, mucking around inside typedefs and big-endian->little-endian macros, blithely reading and writing raw bytes from and to li'l Gir. Haven't fixed anything yet, but I'm learning. After having the code tell me that my FAT copies weren't in agreement, and then refuse to make any changes, I've discovered that the author (apparently) hardcoded in certain values like the number of sectors per FAT and the number of FAT copies - values which differ depending on the size of the disk, and Gir isn't the same size as his 'pod, apparently.

While there's code that's ostensibly supposed to figure out some of this info by dumping it from the MBR/boot sector, it's not in the same executable, so i can't be sure that things which share var names across the executables really mean the same thing...esp. when they're derived in the first case from data off the drive, and simply declared for use in a different manner in the second. Replacing the values in the hardcoded versions with information taken from my runs of the dump code doesn't seem to work - it changes what sectors the code attempts to muck with, but they still come up as inconsistent (the FAT and backup FAT, that is) which prevents attempts to change it.

Now I just noticed that the dump code is warning that there's a sector 0 read error...and while the second FAT copy typedef's hexdump has interesting stuff in it, including what looks like packed bytes and a 'NON-BOOTABLE DISK' text message, the first FAT copy seems to be all zeros.


Oh, suck. If sector 0 is tanked, then the primary FAT isn't working?

Hm, but Disk Utility claims it managed to properly initialize the volume, which doesn't sound like the primary FAT is corrupted. More likely, the place it's looking for the FAT is wrong, given the disk size difference.

ARGH. I don't know enough yet. More research.

Posted by jbz at 1:14 AM | Comments (0) | TrackBack

October 21, 2005

Healthy Suspicion

Time for a PSA, folks. The season's first few 'internet chain letters' have started trickling in - kids are back at school, which may or may not have something to do with it; holidays are approaching, who knows. But speaking as your friendly neighborhood spam, network administrator, please, please, PLEASE apply this one simple rule when you read email. Even when it comes from friends. ESPECIALLY when it comes from friends.

If the email message was forwarded, and it asks you to either send information or money somewhere OR to send out email or postal mail, check carefully on its pedigree. Note that I don't just mean be sure your friend sent it. They probably did. But check that the 'story' in the email is true and verifiable. If it's for a cause that's in the news, verify that the organization making the appeal sent it. If it is for an 'unnamed project' or 'unnamed child' or sympathy-jerking cause, be very suspicious.

Here's a good place to look: Hoaxbusters at CIAC - the U.S. Department of Energy's Computer Incident Advisory Capability website. Or just Google for some representative terms from the email - like, say "sick child wish chain letter internet" and see what you get. Dollars to doughnuts you get verbatim text from your email in the results - and now you know.

You're not done, though. Responsibility demands you consider your actions carefully. Blowing a 'reply-all' out to every email on the damn email would really only compound the problem. Better to pick the last few people, the one(s) you know personally, and write them an email with a *different* subject, alerting them to the chain letter (preferably with a link to an internet citation of its hoax status) and a quick suggestion (politely phrased, it's not THEIR fault, remember?) that they check first in future.

Hopefully, next time, you won't get the email.

And we'll all save time.

Thanks. This concludes this particular Nannygram.

Posted by jbz at 3:58 PM | Comments (0) | TrackBack

September 20, 2005

Postfix, Red Hat Enterprise Linux and a Dell PowerEdge 2650

We run our mailman server on a Dell PowerEdge 2650. A while back, we migrated to a new distro (RHEL4) and hence a new mailserver and mailman version. We were using the postfix which shipped with RHEL4 - postfix-2.1.5-4.2 - and began having problems. Messages sent to the lists were not arriving. Not all of them, but a significant portion. So the game begins.

Upon investigation, we found that in all cases, the messages were showing up in mailman's list archives, indicating that they had been received by postfix on the list server, passed to mailman, processed by mailman, and added to the list archives in a timely fashion. So the hangup was somewhere either in mailman's outbound message processing or afterwards.

Looking in mailman's queue, we found that it was running and contained only seven or eight messages. Nope.

Moving along, we checked postfix on the list server. Whoa. Nine thousand messages. Using

showed us all of them. Although a couple of thousand of them were in various states which indicated 'normal' deferral, such as 'Connection timed out' or 'Host not found' or 'Mailbox full', several thousand had the following error message:
status=deferred (delivery temporarily suspended: unknown mail transport error)

Um. WTF?

Checked the process table. Postfix had multiple smtp agents running. Tailed the logs, and things were happening...but as we watched, messages flew by, deferring with that same error. We reloaded postfix, and confirmed that it reloaded its configuration - but no difference, same behavior. Okay, so postfix was functional, as far as it could tell us, but something was definitely wrong. Corrupt files? Perhaps.

We stopped postfix and had it run a check. It helpfully found the five or six known corrupt message files that it had already quarantined, announced that all of its permissions were correct, and returned. We started it back up. This time, mail began to flow properly. Looking in the mailqueue, however, many of the messages were still marked as deferred with that error. We manually flushed the queue (postqueue -f) and, watching the logs, saw them flood back into active status. A few minutes later, our list mail folders filled with backlogged list mail.

A few hours later, it happened again.

Then a couple of hours later, again.

I investigated further. Looking in the logs, I found the moment the first message came back as deferred. A few seconds prior to this, I found a warning:

postfix/qmgr 9083: warning: premature end-of-input on private/smtp socket while reading input attribute name

Going back to the prior incidents, I found similar but not identical errors a few seconds prior to them as well. In one case, I found a panic:myfree error warning of corrupt or unallocated memory. In each incident, however, what apparently happened was that one of the smtp agents had crashed and the qmgr (queue manager) had caught wind of that fact. The process ID number given in the warning message by the qmgr (9083 in the example above) was the proc number of the agent that had died, so by backtracking in the logs I could figure out which message ID the agent in question had been handling when it stopped reporting.

The problem was that when I restarted postfix, I found that in every case, those same messages that were in process during the crashes seemed to go out just fine. So it wasn't a case of a malformed or corrupt message, which is what had always been the cause of such behavior in my experience with postfix up to this point.

At this point, I had a problem. Well, two problems, potentially three. Problem one: an unknown factor was causing the intermittent but frequent crash of smtp agents on my lists machine. This by itself was merely annoying, given that postfix's master process was properly figuring this out, reporting it, and spawning new agents. However, it was rendered critical by problem two: Postfix, for unknown reasons of its own, upon discovering the crash of one of these agents, would start marking large numbers of messages in the queue as 'undeliverable.' Potential problem three: As far as I could tell, there wasn't any attempt later on to revisit those messages marked as such.

I'm not going to even try to address why postfix was behaving that way. Suffice to say it pissed me off immensely.

I attempted the almighty Google search. I found a couple of hits on the errors messages I was receiving. The answers, from the Postfix coxers themselves, seemed to fall into two categories. One: this kind of problem could be introduced when using mail scanner or routing scripts that directly touched the mail queue. Okay, not me; I'm using mailman but only in a manner prescribed, i.e. postfix delivers to mailman and mailman uses postfix's available tools to inject messages back into the queue. Two: Memory or hard drive troubles on the server. Okay, fair enough, I'll check.

Ran full hard drive checks. Nothing. Just for giggles, took the system drive of the server out of mirror mode, ran it on each of the mirrors individually. Showed the problem both times. Put it back into (PERC hardware) mirror. I consider that to come as close as I can to eliminating hard drive corruption as a cause. To be safe, I disabled swap on the machine in case there was a problem with the swapfile, and ran it on physical RAM. Nope, problem still showed up. Okay. Problem isn't in swap disk.

What about RAM? Downed the server, and ram memtest86+. Note that this program doesn't really like PowerEdge 2650s - it shows a constant error on one address in high RAM which I suspect is used by the ServerWorks management hardware. Other than that...nothing, really. Decided to be safe. Ordered new DIMMs from Dell. Installed. Nope. Same problem. I consider that, again, to essentially rule out the 'bad RAM' problem.

Finally, I was forced to do what I didn't want to do. I upgraded postfix past the official Red Hat release, to 2.2.5.

It's been 12 hours, and it hasn't crashed an agent once, nor marked any of the mail as temporarily suspended.

I'm a little annoyed about this, and here's why.

First of all, the postfix crew were adamant that this wasn't postfix's fault. I understand their bias, but, um, heh. I'll let that one go. I wasn't using the 'current' version anyway.

What really pisses me off: Why the hell did postfix 'handle' a crashed smtp agent like that anyway? How am I to know it won't *still* do that? Please don't tell me to go read the code. I'm an op. I'm not a software engineer. If the only way I can be reasonably sure your software will work the way it should is to read the code, because testing a prior version has resulted in behavior I can't explain and this behavior hasn't been addressed in docs, then there's a problem. Back to the point: Why does a crashed smtp agent result in messages in the queue being flagged as undeliverable? The entire reason for there being a watchdog process and multiple smtp agents (well, one reason) is so that one agent dying shouldn't be able to tank mail delivery as a whole.

Anyway, just another day as an Op.

Posted by jbz at 1:20 PM | Comments (2) | TrackBack

August 1, 2005

Cory Doctorow misses the point.

Mr. Doctorow appears to be on the verge of having a full-blown snit over Apple Computer's use of TCPA/TPM (Palladium?) DRM inside the kernel in the Mac OS X for Intel developer kits that have so far been released. I (personally) think he's (again) entirely missing the point, in a frenzy of Copyfighter Open-My-Information kneejerk.

Allow me to state for the record: I think DRM is a bad thing. Especially when applied to anything in my data. Especially when applied to anything without my permission. But in this case, take a careful look at what's going on.

Apple is a company that has always made its money on hardware. Sure, people may buy the hardware because of the OS - but the margins are on the hardware, and the higher-end, the better. While this may change in the future, and may be changing as I write this, it's still true. If it weren't true, I don't think Apple would be bothering to produce Intel-based Macs - they'd be producing Mac OS X for your PC. The problem is that that's what Steve Jobs' last company did - remember NeXTStep? Sure you do. Look what happened to them? They had to get bought out by a company that made hardware (Apple) because they fell into the ground trying to survive only making software.

No, I'm not claiming the companies are in similar situations. However, they are both companies with 'different' OSes, headed by Steve Jobs, who doesn't forget things. Apple is spending a great deal of time and money to build a Macintosh product line that runs on Intel chips - to the point of being rumored to be trying to poach Sony engineers to help them build out Intel-based sexy products. So they're going the hardware route.

Given that, anyone running OS X on a non-Apple box is cutting into their survival. They're denying Apple the profit margin on an Intel-based Mac (Note, Mr. Doctorow - I don't say 'stealing'). This isn't something that Apple thinks is illegal, or should be legislated against, or should be punished - yet - but it's something they would like to prevent if possible. So they have started taking steps.

Let's have a look at precisely what, so far, that TCPA/TPM stuff is being used for. As far as I could tell by reading the forum thread that Mr. Doctorow references in his post, the stumbling block to those folks attempting to run their versions of the Mac OS X for Intel Developer's CD on non-Apple hardware have come up against is that the OS seems to be using TPM when instantiating the Rosetta kernel.

The Rosetta kernel is the piece of code that performs on-the-fly translation between PPC binaries and the x86 instruction set. It was (IIRC) a piece of commercial software that Apple purchased via the acquisition of the firm that produced it. Ergo, it's fully theirs. So the TPM is being used to protect the running of the Rosetta code.

This, in turn, is a problem because the GUI for OS X (the ATSServer) apparently hasn't been ported to x86, and is a PPC binary. The GUI code from OS X (at least in the Dev kits) lives in PPC, and thus requires the Rosetta kernel to start up in order to run. No Rosetta, no Mac OS X interface. Thus, if your Mac OS X Intel can't properly authenticate against the TCPA/TPM hardware on the motherboard, it won't be able to start Rosetta, which means it won't be able to start up its GUI, which means - bam.

Mr. Doctorow states that this is an unacceptable foray by Apple into DRM, and goes on to explain that he worries about his data, stored in proprietary programs like Apple's and the like, and that this use of TCPA/TPM means that he will no longer be recommending or purchasing Apple's products.

I wish him well.

I'm personally unsure what has him in such a twist. If, in fact, we find out that the TCPA/TPM is being used anywhere in the OS which handles the loading and unloading of actual userspace data, or in the filesystem routines, or in any place in fact where user data or user code might run up against it, then I could see and possibly agree with his concern. But not at this point. Here's why.

Rosetta is proprietary software, and it is being protected against execution. As far as I can tell from reading the thread, the TCPA/TPM is only being invoked in the specific bit of loader code that invokes Rosetta for the purpose of translating PPC code to x86. NOTE: I am not a coder. I may well be wrong. Ergo, there doesn't seem to be any way (to me) for that to be used to encrypt or otherwise 'lock in' a user's data. The only way it would prevent users from accessing data is if they have data written by a PPC-only program which they then move to an Intel-based mac and cannot access - but that is a compatibility issue, not a DRM issue, since there is no way to retroactively 'lock down' that user's data. It simply means that they cannot transfer it to the new system (voluntarily) and have it work. If they copy it back over, it'll work fine. The data has not been encrypted, and they are free to write an OS X/Intel version of the program which can read it.

Moving on the fact that Apple is locking the GUI of an OS 'based on free software' up behind DRM - yes, so what? I recall quite clearly some diagrams of the structure of OS X. Darwin (the Open Source base of OS X) does not include the GUI. The GUI is proprietary Apple software, just like Rosetta. It is why people want to buy Mac OS X, rather than download Darwin (which they have been able to do for x86 for years).

If we discover that Apple is putting routines inside OS X that allow the TCPA/TPM to be accessed from inside Quicktime, or from inside filesystem modules - then, yes, I would be concerned. I am not even saying we won't discover these routines. I'm not defending Apple. What I am saying is that the fight against DRM is a serious one, and that knee-jerking responses to it weaken our ability to make reasoned arguments against its use. Mr. Doctorow's throwing up his hands at the very mention of TCPA/TPM and DRM and the word 'kernel' and threatening to never buy an Apple product again are a perhaps laudable statement of his commitment, but they are also an easy way for those in the industry to dismiss his statements as hyperbole or ignorance or just plain rabble-rousing. Whether I approve of him or not, he has significant influence in the Copyfighter community - and I would hate to see that influence wasted by firing shots too early, without proper information, on issues that sidetrack from the real problem.

Posted by jbz at 12:10 PM | Comments (0) | TrackBack

May 5, 2005

(Solaris) Ten Zen

For reasons best left unstated, I was attempting today to rectify the fact that a recently installed Solaris 10 (Intel) machine was refusing to admit that it had a network interface. In fact, it had three it could have picked from, one of which was a PCI 3Com 3C905-TXM which it clearly says it supports, but that's besides the point. In any case, I was logged into the Java Desktop environment as root, so after much pointless thrashing on a terminal in a vain attempt to either a) recall my much-out-of-date Solaris-fu or b) apply it to this situation, I decided to make-like-monkey and use the GUI. Click 'Launch'...aha! 'System Preferences.' Under that...'Network Configuration!' Perfect. Click.

A dialog box comes up. It wants information. This is hopeful. There is a Java logo...


It's asking me for an IP address.

For a server from which it can download the Network Configuration Agent.


Posted by jbz at 11:36 PM | Comments (0) | TrackBack

April 25, 2005

Bad reviews suck

...and they suck even more when not only do you yourself agree with many of their points (despite their somewhat dubious origins), but when some of their most telling front-line factual dings (see six paragraphs down) are ones that you and many others inside your company been shouting about for over eight months prior to release. Not only was the problem well-known by several but, more importantly (and something I hadn't known) apparently the documentation 'dealt with it' by simply claiming it couldn't be fixed. So rather than fixing it, or even offering the workarounds...sigh.

Some days...

Posted by jbz at 9:44 AM | Comments (1) | TrackBack

March 11, 2005

...and while you're at it, a shoeshine would be nice.

First, read this. I'll wait.

Okay. I have a bit of skin in this game, because I work for some highly GNOME-invested people here at the corp. Furthermore, I think I qualify (in Eugenia's books, at least) as an Evil Conspirator - I spent a great deal of time a year or so ago convincing Dave Camp that spatial navigation in a file manager was a really, really good thing (Red Hat unilaterally decided to implement spatial nautilus? It wasn't the Nautilus maintainer? Hm, well, okay) so I may up for some of this venom.

But I gotta take exception to a whole bunch of stuff said here. Let me be frank; I just think it's sort of stupid, and although I can be this blunt because this is my own blog, I just see no point in sugarcoating. The writer seems to feel that OSS developers and projects have a responsibility to the world to behave like large corporations and perform user-centric market research for feature implementation, else be punished by users (like the writer) deciding not to use their software. The problem seems to stem from the fact that these developers and projects appear to not care if these users use their software.


Big fat logical flaw, right there. But let's ignore that, for the moment. How, precisely, are these uncompensated OSS dev/projects supposed to do this? Eugenia, at the beginning of the article, notes that Novell/Sun/Redhat brushed off the problem as only applicable to their own market research and their own customers, and stated that that was fair enough because 'they had a business to run.' Well, OSS folk (sometimes) have lives - and time spent writing that software you're using has to come out of those. Coders are in fact usually the easiest to find to commit time to projects, because usually they are utilizing skills and resources they either use to support themselves anyway, or skills that they utilize for fun in their spare time and hence would be doing anyway.

Market research, though? Organized feedback and project planning? Those require a vastly different set of skills and organizational roles and resources, which are much harder to come by - and cost, either in coin or otherwise. Many fewer people (I would think) perform market surveys as a hobby. Fewer people (than code) solicit feedback on other people's work for fun. Fewer people than code on something that interests them are willing to organize groups of other people who are working for fun and try to coerce them into doing something else, without resources to offer them or means of coercion. That's called management, and precious few people can do that effectively even when given all the tools a corporate position and an orgchart make available; how many do we think have the ability to nerdherd for fun in the wild?

Some can. I've seen it. But it's not nearly as directed as some seem to imagine. It's a reality distortion game of motivation and imagination, a crowd manipulation game. It's not a project management task. That's the part that's best left to paid company employees to handle, and is one of the reasons that it's so exciting that companies like ours have actually been willing to jump into this scary arena.

In the end, what the writer of that article seems to me to be saying is that 'gee, GNOME is nice, but you people who write it aren't listening to us people who have decided to use it, and there isn't a mechanism in place whereby I can make you listen to me, and I'm going to threaten to stop using it, and hope that the threat makes you worried that you're losing 'traction' and hence pay more attention to my demands.' It's a sometimes effective tactic, I guess. But it's really whiny. Hence the title of this entry.

As others commenting on that article have noted, there are vastly more efficient and less needy ways to motivate people. Have you considered the bounty system? Money works, and if enough users want something, well, gee, put your money where your mouth is. As the article points out, 'vast numbers of users...' wanted something and it didn't get done, or hasn't gotten done, and people are willing to pony up money for it. Okay, then harness the hobby coder and put up a bounty site. Organize those rogue developers with cold hard cash. Compete with Novell/Sun/Red Hat for those coders' time and attention not with threats of firing but with the lure of lucre. Play the game. If you don't mind that those companies are in the marketplace (and you don't seem to, from your comment) then you shouldn't mind playing on their playing field, and you seem to think there are enough of you with demands to match their resources...especially if your demands aren't in conflict with theirs.

Or are your demands really all that widespread?

Posted by jbz at 4:57 PM | Comments (1) | TrackBack

December 31, 2004

Ops, Time Management, and the proper use of laze

This is actually about Operations, not Code. I apologize for offkilter categories. I'm trying to resist the temptation to balloon the number of categories out to fit what I feel they should be, as I think that would approach roughly the number of posts in the blog.

Hi ho.

Anyhow, on the meat of it.

I was recently involved in a bit of a disagreement with a cow orker ( moo) about the proper way to go about 'fixing a problem' at work. I am an Op. What does that mean? Well, that's part of the problem. Since this is my blog, I get to pontificate for a moment. When I say I'm an Op, I mean I am responsible for Operations, in the computer and information technology sense, for my firm. I do not mean simply that "I work for IT." That's not specific enough. I do a strange mix of jobs that doesn't fit well in a large corp, and fits much more neatly at a small dotcom - I do whatever is required to keep things running at the corporate level if it involves computers and infrastructure. While this can include fixing the CEO's secretary's laptop, you'll have to convince me that the CEO's secretary's laptop is damn well mission critical, unless I signed on to do end-user support when you hired me...and believe me, you're paying me more if I did.

The problem with being an Op is that it is extremely difficult to explain to those who are not Ops what I do with my time, and how I prioritize it and parcel it out. Please note: this is not an attempt to pretend that I am not, at some level, lazy. Far from it. In fact, I am strongly of the belief that at the core of every truly inspired Op is a tightly held and well-managed streak of pure slothfulness, which when properly channeled can produce genius of time-fu. I shall explain. And no, I don't have it, really; I can only approximate it.

I spend my work time performing three types of task - or rather, in three states of work. I shall call them

  • Routinized
  • Structured
  • Chaotic
Routinized tasks are pre-planned, scheduled tasks. Maintenance, in other words; expected, allocated time, used to perform regular tasks in a (usually) repetitive cycle. Backups, security audits, user account maintenance, inventory checks, system management e.g. OS upgrade installations (as opposed to testing), etc. etc. Anything and everything that you know needs to be done in advance, you've scheduled time for, and you can check a box off your task list for the day/week/month/whatever when you're done. Whack-a-mole type stuff.

These tasks are usually boring, but important. Stuff that cannot get skipped, or things stop working. It is the goal of all good Ops to attempt, on a continuous basis, to minimize the amount of time they spend on these tasks - through automation, through process management, and through careful sanity checking on their task lists. The reasons are manifold, but here are some of the most important. First of all, the more of these tasks you have to do, the more time each routine 'cycle' takes - and the less time you have for other types of tasks, as I'll outline below. Second, the more tasks there are, and the more complex they are for humans to carry out (i.e. the more manual they are) the easier it is for mistakes to be made, steps to be forgotten, or subtasks to be missed - resulting in bad state which then causes Problems. Problems are Not Good, and result in Chaotic Tasks. Third, we're Ops, and we're lazy - we don't like working on boring things. It's not fun and makes us irritable, and you won't like us when we're irritable.

With me so far? Okay, good. This brings us to our next type of task - the Scheduled task, which occupies scheduled time. These are 'one off' tasks which are not routine, but are usually preplanned, known outcome tasks with deadlines. While they can be interrupted, doing so exacts a cost in both 'shutdown' and 'startup' costs for the task, as well as in resources (lab space, spare/swap machines, etc.) needed to maintain potentially fragile 'middle states' of the task. Some examples of this include hacking/scripting to develop new automation, configuring machines to perform new duties, configuring new machines to replace older servers in a preplanned switchover, implementing new services on existing servers, implementing test plans for new services/software, etc. In short, the expected use of time to carry out more demanding work with less tolerance for interruption, but which does not represent a 'regular task list.' Time spent in Routinized tasks, naturally, cannot be used for Scheduled tasks, and vice versa. So this is a primary reason to avoid overburdening Ops with Routinized task loads.

Now, if these were all an Op had to worry about (like, say, if s/he were a coder, or a web developer, perhaps) then things wouldn't be too bad. Nice Gantt Charts could be drawn, timeslices could be set up, and spreadsheets could be done showing managers precisely what was being done when. The problem is that this isn't what Ops do. This brings us to the third type of task and time.

Sometimes, a server gets compromised at 2 am. Sometimes the mailserver eats its disk array at noon on saturday. Sometimes for whatever reason, somebody needs a new public-facing machine up in 5 hours, and even the Op is forced to agree they may have a point. This is a chaotic task. I call it this because even in the case of building a new machine, being forced to do so with no advance warning usually introduces enough uncertainty (what machine are we using? Do we have enough RAM for that? Are the disks OK? Do we have drivers for that? Well, will it fit that rack? Oh, they want SLES, but it's a Dell 2650? Well, we have to run RHEL then, because we don't have RAID drivers for SLES for that box...etcetera, etcetera) that claiming these are preplanned processes is a bit of a joke. In the case of a full recovery, making time predictions may even be criminally stupid.

So there's the rub. You have three different types of demands. You want to make sure that when chaotic events, or tasks, arise, they'll get done - because they are almost always mission-critical for somebody or other. You need to make sure that the routinized tasks get done without too much interruption, or things start breaking. You need to make sure that the scheduled tasks get done in some reasonable timeframe, or you start falling behind in your ability to implement services and become a purely maintenance organization (and a poorly performing one, at that).

This will mean people will come in and want to know why you're reading mail rather than implementing the new mail server. One answer: Because I don't have enough of a scheduled time block to do so at the moment. Another: I'm on call at the moment, and can't get too deep into something that I can't drop.

Whatever the reason, it's true, there are limits. They're different for people and places, and they're hard to know without knowledge of the organization and people in question. But the next time you're told, as an Op, that you can easily fix something with a minor little tweak that requires semi-regular touching by a human, remember - if you say yes, you just added something small but recurring to your routinized workload - another thing that might get forgotten or missed if time runs short, and another thing that steals cycle time on a regular basis. Posted by jbz at 1:37 AM | Comments (0) | TrackBack

December 1, 2004

Mea culpa spamorica.

I must offer my apologies to the Novell spam gateway system. As I have been informed by numerous cow orkers (moo) there is, in fact, no need to deal with messages the gateway traps. All messages are default flushed unless 'preserved' by their recipient, so the message digests are in fact means of watching for false *positives.*
Posted by jbz at 1:08 AM | Comments (2) | TrackBack

November 23, 2004

Spamcans of Hanoi

Quick calculation re: corporate spamfiltering. Novell uses a central spamfilter, which reports spam that it has trapped at regular intervals via email digests. Those emails contain lists of the subject lines and senders of said spam, along with links for each to release the message or delete the message. If you choose to delete the message, the resulting webpage offers a link which will add the sender to a personal blacklist, 'preventing further spam from that sender.' Um, okay. Let's have a quick look at that.

I am fortunate that my Novell account does not (as yet) receive an enormous quantity of spam. Probably because I don't use it much outside of the company. My Ximian account, however, does - and I am migrating to my Novell address as per the Integration Marching Orders received from on high. On average, I have found that I receive something like thirty to forty messages per day which make it past our currently-outdated SpamAssassin+Bogofilter install on the Ximian mailserver. I have noted similar messages making it into the Novell 'You've got Spam!' mail alerts. Assuming (for the nonce) that I end up with a roughly similar spam load, let's see what that will do to me.

I just got an alert. I clicked on the 'Delete Message' link. A browser window popped up, and churned for twenty=nine seconds (at 0921 Eastern, when most folks in Provo - where the servers are - aren't working yet). Then it gave me the 'add sender to blacklist' option. I clicked that. That churned for thirty-three seconds before returning. One minute two seconds, then, not counting browser launch time and reading time for the digest, to deal with one spam message. Even if I ignore the blacklist option (a good bet, since the 'from' addresses are usually one-time-spoofs) that's still a half-minute-per) then I have fifteen minutes of browser churn to cope with a day's average spam load, as opposed to maybe ten seconds of select-and-delete in a mail client.

I fail to see the point.

Posted by jbz at 9:26 AM | Comments (2) | TrackBack

November 3, 2004

Of Dell, dkms and dimwittery

Dell distributes machines which they support Linux on, and now they support us too. Hooray! However, they haven't quite gotten the whole Thing, yet. Ran into the following today, which caused some eyebrows to rise in the office since we do, after all, write ZenWorks Linux Management (nee Red Carpet).

Dell, you see, has this problem. They sell boxes which have RAID hardware controllers in them which require drivers. These drivers (under Linux) are dependent on the particular version of the kernel which is presently running. This is not surprising, as they are kernel modules. In any case, I spent some time yesterday assembling a system comprised mostly of a PowerEdge 2650 with a PeRC 3/DC (a badged MegaRAID card) and a PowerVault 2205 RAID shelf. Since we purchased the various components of this system separately, it turned out that we needed to upgrade the drivers for the MegaRAID card after I'd installed linux on the PowerEdge. No problem, Dell distributes drivers as RPM files...I located the driver files and downloaded the requisite rpms, after assuring their website that I was not a terrorist nor their butler.

I ended up with two rpms: dell-perc-2.10.yada.i386.rpm (okay, no mystery there) and dkms-yada-noarch.rpm. Hah? Well, these came in a tarball amongst other stuff, including a file named Said file was nought more than:


rpm -Uvh dkms-yada-noarch.rpm
rpm -Uvh dell-perc-2.10-yada.i386.rpm

Um, okay. Feeling a tad cavalier, I run that. dkms (whatever that is) installs, and then the fun starts. It turns out that dkms stands for Dynamic Kernel Management System, or some such fuckery. As rpm starts to load the dell-perc drivers, it copies over a bunch of files (fine) and then starts running some strange script. The script informs me that it is checking its prebuilt version against the running kernel; that the check failed; and that finally because I don't have the current kernel source rpms installed, it can't build and tag a new version of the driver or make a new initrd, and hence it's not recommended that I reboot the machine. It says all this very fast, in that wonderful I'm-scrolling-past-you-isn't-it-fun-reading-at-115200-baud? sort of way (said Pooh). Then it stops. Apparently, the script has exited successfully, because rpm now is convinced that the dell-perc drivers are installed.

However, they're not.

See, I don't have the kernel sources. And Dell apparently relies on this dkms thing to build them at install time (or, upon close examination, even at machine startup maybe...there's this evil looking init script, see...) and then make a new initrd for the machine with said drivers in it. I'm fortunate; the RAID volume I'm trying to see does not contain my system's running system drive. If it did, then any of the prior oopses would have rendered my system unable to boot.

This, of course, is a problem for our system management product, which relies on rpm. See, as far as RPM is concerned, everything is hunky-dory. However, dkms has failed to properly compile/make the drivers and initrd. If this was a system volume, and if this upgrade had been triggered by (let us say) a dependency check due to the installation of a new kernel version by zlm/red carpet, well, then, as soon as the box cycled, pssssht that's it, no more RAID volume.

I'm not entirely sure what the answer is. One possibility is that Dell should have the dell-perc driver actually have an rpm-level dependency on the kernel-sources. The problem with this is that the driver rpm doesn't know in advance which version of the kernel will be running on the machine, and I don't know if you can have a dependency on a package whose identity depends on the running kernel. If so, problem partially solved; if not, this doesn't help.

Even if so, however, we're not out of the woods. Just having the kernel sources doesn't guarantee that the additional steps of building the driver and/or initrd will be successful, but as far as I can tell, Dell has put all the additional hoo-hah into the postinstall script in the RPM - which means there's no way for that script to return a fail and have that result reflected in the rpm transaction.

I have nightmares about things like this, as an which are designed to allow me to manage servers remotely, including box OS updates and reboots, ending up hosing something critical like, say, the RAID shelf holding the box OS and data. Things that purport to make the box safer and more reliable ending up serving as a point of vulnerability - all because somebody made a dumb design decision as to how to distribute their drivers.

Personally? I would either avoid the entire 'on the fly fuckery' of dkms, or, if that's not viable (and I'm not qualified to say it is or isn't) then force manual driver installation in order to avoid luring the user into upgrade practices which could kill their installation due to remote procedures which aren't safe.

Posted by jbz at 4:25 AM | Comments (0) | TrackBack

September 5, 2004

RSS Feeds and Entry Length

I've received feedback from various monkeys and others that my entries in this blog (what the hell, people read this?!) are frustrating to those who use RSS aggregators. This is due to the fact that Moveable Type tends to limit its RSS feed to 40-word snippets of each entry, which isn't too useful. After noodling around inside the Templates for the blog, I have discovered that buried inside both the RSS 1.0 and RSS 2.0 templates is the MT tag $MTEntryExcerpt; I have replaced it in both cases with $MTEntryBody. We'll now just see if that, in fact, does place the entire entry into the feed.
Posted by jbz at 2:14 AM | Comments (0) | TrackBack

June 18, 2004

Hyper-Spatial Kvetching

All right; I admit it. I'm to blame. Dave Camp came to see me several months ago on one of his frequent, death-metal-fuelled wanderings around the office (he claims that this is how and when he thinks, but most of us think he needs to walk while listening to that stuff or he'll involuntarily punch out his flatpanel one day). At any rate, he solicited my opinion on the upcoming redesign of the Nautilus File Manager.

Fine, so he didn't ask me how it should look, or how it should act. He did, however, if my flaky memory serves, ask me about my preferences regarding file management and the various philosophies then available. I told him that as far as I was concerned, Mac OS 9's Finder was the then- acme of filesystem management. He nodded, indicated in a particular grunting sort of Dave Camp way that he'd heard this particular opinion before, and told me that while that was interesting, he'd really like to know why I felt this way.

Poor boy.

So, despite the fact that he is without a doubt much more educated about, informed on, experienced with and generally understanding of this topic than I, I plunked him down in a chair in the Ops Pit and fired up a Blue and White Macintosh G3 with which to demonstrate my upcoming argument. I told him about a great many things, in poorly-chosen words, but probably the most important single thing I told him was this:

In the Macintosh Finder, the objects you see are not representations of objects on your filesystem. They are the objects on your filesystem.

This is no doubt going to cause all manner of howls from many of my nonexistent readers, but let me try to explain myself better. The problem I had and have with what I have since learned is called the browser model of file management is that what the user is seeing is a loose representation of the truth. The user can 'break' the relationship between the viewed object and the file or data represented fairly easily, in a number of ways. For example, the simplest way to break the relationship in the Windows explorer is to bring up another window, browse the same directory as the first window, and delete something from within it. It won't go away in the first window until that window has a reason to refresh.

This is wrong to me, because it makes the following demands on the user. It requires the user (as opposed to the computer - or software, really) to maintain state information in his or her head, or at the very least, perform an action (window refresh) to be 'sure' that what he or she is seeing is the 'current truth.' Bzzzt. That's not what I want. I don't want my file manager to be a tool that shows me most of the state of my filesystem, or a mostly current state. I don't want to have to remember for myself which window is my active working one; I don't want to have to clutter up my working brain with metadata like this which the computer can and should remember for me.

I prefer what I have since learned is called the spatial model or object model depending on if you work for Ars, in which the metaphor of physicality is extended as far as is practically possible inside the file manager. Not necessarily the aspects of physicality such as relative size, or mass, or smell (heh). The important bits, though - relative position of the object to other objects and directories on the filesystem, and the inability of a physical object to exist in two places at once. This latter is key for the easy and secure manipulation of files without having to maintain state information in ones head.

I showed Dave how the Finder will rigorously enforce the one-to-one rule - one object gets one view. A folder on the hard drive can only be viewed in one open window at once. If you double click on the folder again, rather than open an additional window displaying its contents, the Finder will 'promote' the existing window view - moving it to the front, or enlarging/unhiding it if it has been parked somewhere for convenience. There is no way in the Finder to trip oneself up by having 'ghosts' of objects hanging around after operations - at least, assuming things are working and there is no corruption.

This does not mean, as some who have yelled at me for this preference seem to think, that there *must* be a bazillion open windows after the user has browsed down several levels into the filesystem. Nope. The default behavior in the Finder, in fact, is to be parsimonious about opening new windows at all; double-clicking an object should, in fact, open up a view of its contents inside the existing window, such that the window is essentially 'replaced' by a window showing the new information. At that point, double-clicking the original object can and should bring up a new window, because doing so does not violate the singularity rule - there is still only one view of that object's first level visible (by object, I mean anything with contents - volumes, folders, files with Finder previews, etc. etc.)

Anyway. There are several excellent discussions and articles online that do a much better job than I of explaining this and setting out normative battle lines. John Siracusa at Ars Technica has a much more detailed, logical, sensible and just plain correct opinion piece on why the Mac OS X Finder sucks, which makes much the same argument. The GNOME and Nautilus crew had some lively discussions about it too.

In any case, I then performed all manner of unfair acts designed to take advantage of my own, superior, privileged position. I gave Dave a Mac to play with. I continued to fire incoherent opinion pieces at him by mail. I bought him beer, which was perhaps the most effective. I threatened his net access and his hard drive, since I was and am his sysadmin. In short, I abused every possible avenue to ensure that we would have at least one sensible file-management app out there, since Avie Tevanian and the rest of the NeXTies seem to have finally taken over Apple to the point where Steve Jobs' insane love for brushed metal and 'simplicity' seems to be more than enough reason to do things like, I dunno, lay off the entire Human Interface Guideline group and knife their baby by putting a craptastically 'browser model' Finder into Mac OS X - extending it past the outpost of shitty that Microsoft had staked out by including the NeXT 'columns' view. I don't know about you, but when I'm forced to use that thing I have these awful feelings that I'm being shown a bunch of paper lists with the pages overlapping and being told I'm looking at a marvelous new paradigm in information management.

OK, I'm done. Really.

Posted by jbz at 1:44 AM | Comments (0) | TrackBack

March 23, 2004

Objects, Navigation, Long Noses

Long noses because I plan on indulging in my longtime hobby of sticking mine into places it has no qualifications for nor business being, but about which I have an opinion nonetheless. First things first: This post is about a discussion which happened some time ago amongst the worthy folk whose effort brings us the Nautilus file browser on Linux. Late last year, the project maintainer decided to announce that Nautilus would be pursuing a new direction, of sorts - that of presenting an object metaphor for the navigation of files and directories. Here's the thread.

The thread offers some excellent views on why this is a good or bad idea (or both). I happen to work with a couple of the folks on that list, and reading the thread for the first time recently I was inspired to rethink my own user interface preferences and philosophy.

To make one thing perfectly clear, I was a long-time old-style Macintosh OS user. This interface is mentioned in the thread as embodying the 'object' or (as Ars Technica calls it) the 'spatial' model. In it, each discernable screen object (icon, window, etc.) does more than just represent something on the computer - it behaves more as if it was that thing. For example, an icon doesn't just represent a file - it is a file. Double-clicking it opens the file. If it is a folder icon, double-clicking it opens a window which shows us the contents of the folder. Not only that, however, but there can only be one window open on the screen showing us that particular folder's contents at any one time. This strengthens the tie between that picture of a folder on the screen and the notion that manipulating that picture is manipulating that folder.

There are many other characteristics of an object system. My favorite? That position matters (this is why it is also called a spatial system). If you put something in a folder, and close the folder and then open it again, that object is painted at the same place you dropped it, relative to other objects in the folder. When you open an object and get a window, that window will always open on the screen to the same spot it was in when it was last closed. All this and the many more things that make up an object system are there to do a specific thing - they are there to allow (and encourage) us to take advantage of the human mind's ability to organize objects spatially when using the computer. The mind has been conditioned since birth (in sighted individuals, at least) to use spatial recollection and placement to impart meaning. Important stuff? Closer. Center. Less important? Fringe. Far. Ordering? How about top to bottom, or left to right, or bottom left far to top right near. Whatever we've decided on the spur of the second is easiest or most effective for us. This is why it can be such a powerful organizing tool, and is the reason so many otherwise-low-tech Mac users would reach for hatchets whenever anyone talked about taking away their Macs (for more on why Windows never did this right, see the thread).

There are, of course, limitations and disadvantages. It's difficult to perform the magic that the computer, a mechanism for abstraction, lets you, when you're miring yourself in a physical-representation interface. It's very difficult to handle scripting, and complex selection, and searching - for the very reason that there are no physical representation for those tasks. For these tasks, some argue, a navigator metaphor - a viewer representation that shows you what is available, as in the Windows file explorer or the Nautilus navigator window - is better, because since it is more abstract, it is easier to abstract out past its visual limitations. The current maximum abstraction is the command line; maximal learning curve, minimal representative binding, but maximal power to perform actions with minimal user physical effort.

I know, I know. This is all in the thread. What the hell are you saying, J.B.?

I suppose I was intending to weigh in with this notion. The limitations of the physical representative interface are not necessarily inherent limitations of the approach. In many cases, they are in fact limitations of the technology (both software and hardware) and resources (ditto) available to the user. It is easy to say that one simply can't represent a search action on an object interface, but this may not be true - rather, it may just be that there aren't any available standard methods for doing so, and not enough horsepower to do it convincingly.

If this is the case, if in fact the limitations are limitations of the present rather than of the logic, then don't bind us to an interface (the navigator) simply because it can be made more powerful with today's technology. Of course, offer it as an option for the technically astute. However, given technology's curve, it is becoming increasingly difficult to claim that the 'average user' doesn't have the horsepower in CPU or GPU to represent a fully object interface in complete Virtual Reality, even - the limitation is that there is no standard mechanism for handling VR available to the coder.

Always, always make the 'reach for the sky' solution a possibility, even if you have to cover your bets with the currently possible. Open source is one of the few places in the world right now where this approach is possible, much less lauded - but here's a perfect example. When these VR methodologies, and interfaces, and systems, are invented, coded and built - let's not have them shot down because people have convinced themselves that 'the Navigator metaphor is more powerful than they are.' Sure, it might be, initially. It might always be, because it will always be easier to increase capabilities via increasing abstraction than by building the elegant machine to take The Rest Of Us (thank you Apple) there. But someday, that elegant machine will come - and an Object metaphor implemented now will give all of us an easier step and path to it when it arrives.

Oh yeah, there's another reason. Heh heh.

As it stands, our current user interfaces are all attempts to some degree or other to produce a working object model (thank you, Xerox PARC). The Navigator model came about in order to cover technology deficits in implementing an actual object model. However, we still have 'desktops' and 'windows' and files are still icons that look like pieces of paper, or like pictures, or what-have-you. Simply put - don't let these interfaces remain half-assed. So much of the confusion in current computer interfaces is at that point where the object model craps out and the navigator model intrudes; at that meniscus of interpretation where understanding curves, like light going from water to air, and we lose track of what we're doing.

Let's push that point outwards by building on the object. Push it away from us. Work for consistency in the presentation. This isn't to say abolish the navigator - just try to build, for us dunces, a consistent world option. There's no way to make a navigator model 'consistent and complete' really, because the first thing it asks you to do is give up some of your understanding of reality in favor of its own. The object model, in contrast, asks you to retain your understanding of reality and then endeavors to provide as few rude shocks as possible. The second approach is closer to the seamless experience we want, especially as newbies; indeed, once we are of a technical mind to use the navigator, we're mostly to the CLI since so many navigators, these days, rely on typing arbitrary stuff into search boxes and address widgets.

Posted by jbz at 11:59 PM | Comments (0)

Ahhh, Office 2004...FOR ME TO POOP ON!!!

It’s just not that fucking hard, really. I mean, let’s be clear here. Microsoft wants me to buy their new attempt at productivity suite monopoly - I acknowledge this. I already use the present version. There is one small problem with the present version that I have been waiting to hear that they have fixed. See, they have this email app called Entourage, which is actually a fairly functional app, save for this one, small, HIDEOUSLY STUPID lack of functionality.

It does most of what an email app/PIM is supposed to - handle multiple email accounts, of various types, using folders, yada, yada. However, (and here’s the killer) it does not interface with the OS X built-in address book or calendar. Nope. Apple goes and adds an LDAP-compatible address book system to OS X, provides us with iCal (which is good, because Entourage has so/so calendaring) …and Entourage DOESN‘T TALK TO THEM.

This is hideously stupid.

So now we have Office 2K4, in which the previews are all gushing “multiple additional functionality areas have been added!!” But do they tell me if it does that one, simple, no-brainer thing? Nope. Which leads me to believe that it doesn’t. This is why Microsoft needs to die. In order to use Entourage, see, I now need to give up on using any other application on OS X which needs/uses address information, because those apps (sensibly) use the OS-level services for such info. But Entourage doesn’t. So never the twain shall speak. This, by the way, is the reason I don’t use Entourage, but force myself to utilize OS X’s Mail.App - strictly for OS level Address book compatibility. See, if I use that, I can have my address book backed up automatically by Backup. I can have my Palm sync directly to the OS Address Book and not have to worry about it. I can use other, small apps and applets that tweak that datastore, and know that it will all be OK. I can iSync my address book information up to Apple’s servers, or between my computers, without thought.

Not if I use Entourage, though. And from the looks of it so far, despite having an entire major rev cycle to fix this stupid design issue, Microsoft hasn’t done it. I hope I’m wrong, and that the dumbshit here is merely one of a marketing nature - i.e. that Entourage has been fixed but that they don’t consider it important enough to trumpet in the ‘preview’ snippets they’ve been handing out. I don’t know. But I’m not holding my breath ‘till I turn blue.

Posted by jbz at 11:43 AM | Comments (0)

February 5, 2004

Speed bleeding feed

I am fascinated with this, which is a neato Java-applet-cum-literary-oasis-avec-fromage-and-digerati. Essentially, it is Cory Doctorow's new book Eastern Standard Tribe, presented to you the net reader one word at a time at user-selectable speed. The ultimate in lazy-ass infoslurping. The epitome of 'feed'. Ideal.

I find myself reading comfortably one click below the max setting. It's sort of like being stoned somewhere you're really not supposed to be, having conversations - you have no way of referring to the previous words, or sneaking peeks at the paragraph shapes in front of you. You just have the flow, which firehoses into your eyes and occasionally leaks in sprays out your ears. All your thoughts about the story have to take place in parallel with the unstopping work of deciphering it, holding the sentence structures in place in your brain's equivalent of an input buffer. I can't shake the image of being buffer overflowed by a frenzied, coffee-pumped cyberwhatsis author.


It has possibilities.


Posted by jbz at 11:55 PM | Comments (0)

January 28, 2004

VPN = Vehemently anti-Penguin Network

One of the latest problems we've run into in our quest to become a truly enterprise-class Linux company is the face that we have a lot of remote workers. Folks who hack from the comforts of their homes and offices around the planet, from places strange and wondrous all call themselves monkeys. This, of course, means that they need access to resources which live on our network.

Easier said than done.

At first, the BigCorp crew were fairly confident. "VPN," they intoned, invoking mystical TLAs with supreme confidence. "Simple software client. Choice of flavors."

However, as it turned out, all was not well. See, many of our remote hackers live on dynamic IP addresses. Most use some form of NAT on their home networks. The client available for our Nortel Contivity extranet switches is called NetLock, and it's now made by a company called Apani. On the surface, it's just what the doc ordered - a cross-platform piece of software (Windows, Mac OS X, Linux) which allows the user to tunnel into the corporate net.

Problem: This piece of software, despite being distributed teasingly as an SRPM, is in fact a big chunk of binary nastiness hidden inside some wrapper code to talk to the kernel. But it has to be compiled into a kernel module (two, actually) in order to function. And therein lies the rub. It only works with an extremely short list of Linux distros, and then only if you are using bog-standard kernels (it doesn't even like RedHat errata kernels, the silly thing).

Given that our remote workers are developers and testers, they end up running all manner of weird distros and kernels. And some not so weird, but which just didn't work anyway. No go.

After unsuccessfully perusing the list of options amongst the commercial VPN client market (zero) we turned, naturally, to the Open Source software world. OpenVPN, which uses SSL encryption, was a possibility; so was FreeS/WAN. OpenVPN was non-responsive, and while FreeS/WAN could be enticed to work in most cases, it suffered from several problems: one, it was management-intensive (since it couldn't auth using the Corp's native auth solution and hence required X.509 cert generation, distribution, installation and tracking) and two, it simply wouldn't operate from behind a NAT gateway.

bzzzzzzzzt. Thank you for playing. Would you like a nice copy of our OFFICE GAME?

Ah, I hear people shouting, what about Super FreeSWAN? Huh? Hunh?

Well, yes. Except that to work in NAT traversal, it, too, requires a specific kernal patch and recompile. Bzzzzzzzzt.

So there we have it. At present, we're looking at just buying hardware endpoints (little SOHO routers with Contivity logic built in), but those present their own problems. Namely, I can't let people directly in to the network from those, because any time someone opens an 802.11 net in their house behind one of these things, their neighbors can happily surf into my network. Nopenopenopenope.

At the moment, we're looking at setting up a separate DMZ for the VPn endpoint on the corporate end, and then installing a bridge box which users can SSH tunnel through to get to the resources they need - a compromise between access (better than the 'nothing' they have now) and security (that box is still vulnerable, but only to the smaller subset of people who compromise a remote network with one of these endpoints installed).

I'm not sure why this is such an issue, although I have my suspicions. I think that, in point of fact, there aren't that many organizations doing remote linux development that have access requirements for their remote users than go beyond simple SSH tunneling (*cough* NFS sucks*cough*). Or they are willing to accept vulnerabilities or workflow disruptions that we're not. I'm sure many have done this before, but I just can't see how they'd do it without having to make the sort of compromises we are.

Not that the compromises make the task undoable; it's just aggravating. On the other hand, better to find these sorts of problems while dogfooding than when selling the product to the public.

Posted by jbz at 2:53 AM

December 18, 2003

Portscans and packetsprays, Oh my!

Hm, this is an interesting one. We received a complaint today from some user about one of our servers. His IPCop/ Snort IDS had reported that one of our machines was portscanning his network. Naturally, this is a Bad Thing(tm), so we jumped into the OpMobiles and hit the OpPhones for some Late Night Colo Action(r)! (To clarify: I went to the home office. Mike went to the colo, the lucky bugger.)

...however, on first (and second) examination, the machine in question looks to be fine. We've checksummed all manner of binaries against known goods, checked the logs, etc. etc. To be safe, we downed and isolated the machine and sucked up the downtime while bringing up a warm backup.

My firewall shows numerous SYN FLOOD attacks against our network right around the time that the complaint scans occurred, so my first (well, second, after "FUCKITY FUCK") thought was that we've been hit with some manner of reflective attack or spoof, such as idlescan. That would be a bear, since there's no real easy way to prove that's what happened.

Next step: See if the 'easy script kiddie' version of idlescan works using us as a zombie. It shouldn't; we run current versions of the TCP/IP stack and custom-compiled versions of our servers, and I can say with confidence that we do not use sequential fragment/packet ID numbers, which renders us fairly useless as a zombie for idlescan. Of course, idlescan isn't new, and there's always the chance that the blackhats have figured out a new, snazzy means of doing it that I can't verify with a bog-standard Nmap and my own devious brain.

I have another look at the logs from the complainant. I take the time to curse the fact that we have been given super-basic info (essentially, 'detected a portscan from your host:port 80 to our host: 21 ports scanned in x seconds'). Now, a third possibility has occurred to my sleep-fogged brain: This could, in fact, be purely legitimate traffic.

Here's the scoop. The machine he's complaining about serves as, among other things, a public webserver for Ximian Red Carpet. It spends (some of) its time serving http requests for lots and lots of linux distro and software rpms that come in from machines running the Red Carpet client. This client is fairly quick, uses some really fast http client libraries, and can multithread.

Posit the following situation. I know nothing about this guy's network, but I can probably assume that he put his IDS somewhere near a network bottleneck (like his gateway) so as to catch the maximum number of packets. Now, if there are even a few machines on that network running a Red Carpet update simultaneously, then it is quite plausible that our server will be spraying large numbers of http packets back at his net. Let's look at the complaint again. In all cases, there seem to be '21 ports on 1 host' scanned. That number is significant. I don't know the defaults for Snort off the top of my head, but scanlogd assumes that accesses are portscans if 7 privileged ports or 21 unprivileged ports (or some weighted combo of the two) are accessed in a row with three seconds or less between accesses.

So it seems safe to assume, given that similar number of 21 ports, that whatever his IPCop is doing, it's using similar trigger numbers. The problem is this: let's suppose he has, somewhere on his network, several machines behind a NAT gateway. These machines are all running Red Carpet (for example, suppose they've all been set up to auto-update themselves late at night - maybe even at the same time). From his IDS' point of view, there will be a flurry of packets originating from our machine's http port, addressed to various high-numbered ports on the NAT gateway - and there will easily be more than the 21-in-63-seconds-on-different-ports required to trip the 'portscan' alert.

Well, that's it for now. I'm going to request additional data from the complainant. This is just my thought process during a potentional intrusion; I dunno if it's interesting or even relevant to anyone else, but it's useful for me to record for later perusal and reference.

Posted by jbz at 1:14 AM | Comments (0)

December 8, 2003

My CUPS Runneth over

This is a silly tech tip, one that I’m placing here pretty much because I forget it every time I need it. When using CUPS (Common Unix Printing System) it may happen that a user’s printer simply disappears out of the ‘Printers window (this is in Nautilus, on linux). The reason is usually that the partition where CUPS keeps its spool has filled up (typically /var) and needs to be cleaned out. CUPS will ‘helpfully’ remove the printer icon to let you know that it is unusable. It’d be nice if it also told you what the problem was, but, you know, we take what we can get.

Posted by jbz at 2:50 PM | Comments (0)

November 23, 2003

That neglected category

Originally, this blog was s'posed to be a place where I could reflect, rant, rave and otherwise blurt about developments in the world of Linux which I get to see from a rarified (albeit not unique) perspective.

Of course, there's a problem with this - notably, people in the linux world are typically more gregarious than I, and blog the hell outta what's going on anyway. Given that some of them are the ones actually writing the software, and arguing over what it will look like and why, my contribution would be somewhat pointless.

I guess it might be interesting to some small crew what it looks like for a small but loudmouthed linux shop to be glommed by a big and old-world corporation - but I'm not sure if I can write down the really good bits anyway.

So here we are. I guess I leave this space for my own personal views on where Linux is and goes - much less point and meaning, but there you have it, this is a blog after all and I'm not one of the movers and shakers. So whaddya want for nothing, a rubber biscuit?

I use linux daily at work. I don't actually use any other OS; I don't have a Windows machine at my desk or even readily available. This leads to my current, major gripe with Linux desktop software. I'm a Network Admin (at least, I like to style myself as such) and diagrams are my life and bread. However, there really is no good way to diagram a network in Linux. The GNOME crew will pipe up with "Dia!" except that, as a user of software, Dia is a dysfunctionally idiotic piece of software. Who ever heard of a diagramming program that wouldn't let you rotate an object?

I'm gonna use the awful word - yep, Visio. Visio, for all of its own dimwittedness and the layers of dysfunction that its evil step-parent has laid on it, is still the Gold Standard for doing infrastructure diagramming.

There were better packages for doing this sort of stuff - my favorite used to be something called SmartDraw, which was around $20 and was optimized for doing network diagrams. Then, of course, the behemoth bought Visio and used it to club the other contestants into the alleyway. OOo is doing an excellent job of becoming a MSO replacement...and it'll need this functionality.

More later.

Posted by jbz at 2:59 PM | Comments (0)