Sharp Tools

Down at the Pub, Afloat

I am tickled by the notion that in the US Navy the ships USS Stout and USS Porter can pass by each other on the high seas and their captains agree to meet for a beer later.

Losing and recovering your iTunes library redux

So. Let's get this out of the way. I'm a Mac (not Apple) fanboy. But iTunes, for all its handiness and utility as a media player, is not a good media storage application. Let's be honest: it's frigging terrible. It doesn't handle failure well at any level, it doesn't offer any form of resiliency, its required datastores are both fragile *and* brittle, and in general if anything goes wrong you're screwed even if you have a full backup. I didn't have a full backup. Here's some things that helped (but didn't totally solve) my backup nightmare.

The Situation

As mentioned in earlier posts, my iTunes library lived on an external drive on my iMac, along with another partition containing my Time Machine and SuperDuper backups. My reasoning at the time was thus - I had three classes of information in terms of backup strategy. My primary data was Class I, and I placed that on the internal drive of my iMac. My media (iTunes library and movies, which I keep separate) was Class II. My backups themselves were Class III, under the assumption that if I lost a backup without losing the primary associated with it that was no big deal - I could just replace the drive and generate a new one. I'm not worried about keeping versioning outside of those apps which handle that themselves like source control systems.

So. Class I was internal. I put Class II on the external, on its own partition, as I reasoned that I didn't want the disk ops for that media contributing to my primary (Class I) disk MTBF or entropy. So if Class II failed, it wouldn't take Class I with it. I placed the backup partitions (Class III) on the same physical external device as Class II.

No, I didn't think about it in this mind-numbing level of detail - this was my practice as had evolved over many years of storing data and having drive (and tape, and CD) failures.

Anyway. My Class II and Class III storage both tanked as the common hardware under them died. This meant I could still operate my iMac for primary (critical) tasks - all my non-media data was still intact (and backed up elsewhere, like absolutely critical docs are in cloud storage, etc.)

I had a backup of my Class II data. Since it was Class II, it wasn't a live drive backup - it was a directory on my fileserver, which held that backup and some additional Class II data (movies) which would be 'nice to get back' but were huge and not critical. The problem was that my scripted rsync autoupdate of the server volume of my iTunes library never worked right; I would run the script 'when I got around to it'.

So the last time I'd run the script was May 9.

Unfortunately, after I copied that version of my media folder back to my iMac, I found a few problems. To wit:

• Apparently, the last *successful* backup had been sometime in October 2008, because music files which I'd added since that date just weren't present in the backup volume, despite 'modification times' which seemed to indicate that they should have been.
• The iTunes *library* file I had (the database) was the *current* iTunes library, which lived with Class I data on my iMac. I had had Time Machine backups of it, but those had gone with the external drive.

Getting the backup back

So. My first problem was that since I'd kept all my media on a separate volume, the path information for both my iTunes library path preference *and* (as I found out) embedded in all my track information was incorrect. I couldn't put the files back where they 'had been' because I no longer had a drive volume named 'Jukebox'. I had the files in my ~/Music/iTunes/iTunes Media/ folder, which is where iTunes assumes they live if you haven't told it differently. So I went into iTunes and changed the 'Library location' param to the proper place and started it up.

It couldn't see any media files.

Hm. Okay. I went into the iTunes directory. There's two files in there that matter - "iTunes Library" (which is a binary database) and "iTunes Library.xml". The problem is that while you can manually edit the XML version of the library file that doesn't help you - the XML is generated *from* the binary as a matter of convenience for non-iTunes apps to be able to see your library information and iTunes itself ignores the file. However, looking at it can tell you in plain XML what iTunes *thinks* the situation is inside the binary DB.

I wasn't sure how, but although the library parameter was properly set in the XML (to the right path) the *files* all had pathnames starting with "file:///Volumes//Users//Music/iTunes/iTunesS%20Music/..." which was *not* where the tracks had once lived. And no matter what I did, I couldn't get iTunes to change that. Even if I changed the library location and restarted iTunes, causing it to say 'updating library,' when it was done the pathnames all stayed there although the library location parameter would update. Thus, I was thwarted trying to place my tree of music (some 85GB) in a spot of my choosing.

Finally, I put the music where iTunes thought it should be, which was in that above path. Then iTunes found the tracks. But it had huge gaps, files with the dreaded (!) next to them that means iTunes can't find the referenced track. And not just the most recent few months of files - they were scattered everywhere.

It turns out that I, being anal, continually tweak my music library to ensure consistent and correct tagging and filing of my music. The problem was that any track I'd touched in the past nine or ten months, since the cutoff, was now (in the tree) back in its original place and state, whereas the iTunes database thought it should be where I'd corrected/moved it. So all of those were out of sorts.

What I ended up Doing

The first thing I did was make a backup of my iTunes/ folder, including all data files. DO NOT FAIL TO DO THIS. It will save your ass. Trust me.

The first problem, getting the library to come up at all, I covered above. Moving the library into iTunes' "default path" of /Volumes/Users//Music/iTunes/iTunes Media/ fixed that problem. Sure, it's possible I could have figured out what DB tech iTunes was using for its main DB (SQLite? db4/5?) and gone after the pathnames in there, but...meh. iTunes has made it clear to me it REALLY DOESN'T LIKE stoopid hoomans messing around inside its data files. I'm willing to muck around with XML files since I trust my regexps, but not with database files I can't really *see* when I don't know how the app using them really works.

This solution is suboptimal. It violates my Class I/Class II separation, as all media disk ops now affect my Class I internal drive. Once I am satisfied that my tweaking of the library is done, I will probably investigate getting it to move properly to Somewhere Else (where is not totally clear).

Getting my music back

The first problem was all those tracks that I'd added since the cutoff date that no longer existed. I really didn't want to lose their DB entries, mostly because I didn't want to have to manually go fix all the playlists that referred to them and didn't want to lose my ratings. Didn't care about play counts.

Well, for stuff I bought from the ITMS, there's good and bad news, as anyone who has been here knows. Good news: If you go to the ITMS and add a song to your cart you've already purchased in the past, it will pop up a warning about that fact - so you can tell if that's where you actually got the track. This is important because if iTunes can't find the actual file, it won't give you critical info like what file type it is, which is how you tell where you got it).

Bad news: It will still make you pay full freight to re-buy those tracks. There is no feature for 'I LOST MY STUPID LIBRARY PLZ LET ME REDOWNLOAD KTHX.' Apple: You're fucking morons. This would remove SO MUCH of the objections to digital media 'ownership.' I had a backup of my library, but even the files that hadn't changed required non-trivial (from a Mac owner perspective) fucking around to get iTunes to recognize. In other words, if I was Joe Random User, there's a good chance I wouldn't have been able to get iTunes to recognize my tree of files because the DB and the media tree were out of sync - and if they live on different drives and I lose my backups, that's gonna happen.

Good news: For tracks you bought in reg'lar ITMS which have been made available in Itunes Plus - Apple's 256kpbs DRM-free format - you can download the higher-rez open track for the *difference* in price, usually $0.30, per track. That's a lot, but it's better than the $99 replacement cost, that's for sure. And there is a page on the iTMS called 'Upgrade to iTunes Plus' (look for it on the front page of the store, under 'Quicklinks' on the right) which has a list of all the albums and individual songs you've bought and will let you (on a one-by-one basis) pay the upgrade fee and download the new version of the track. Best of all, even if the file doesn't exist on your machine, iTunes will recognize the (broken) database entry and simply replace the track info, so all your metadata will still be there. Expensive but maximally convenient way to go.

iPhone tracks

Handily for me, a large number of the most recent tracks I'd imported were synced to my iPhone. If you want, you can get music back off the iPhone, although the way I did it was maximally manual in order not to step on my fragile iTunes database.

1. jailbreak the iphone. Mine already is. If yours is not, look up the Dev Team blog and find out how.
2. CHANGE YOUR iPHONE ROOT PASSWORD.
3. Make sure ssh and rsync are installed on your iPhone. Make sure SSH is running.
4. From a terminal, copy your music over using something like the following:

   iMac:SomeTempDir user$rsync -av -e ssh --delete root@:/private/var/mobile/Media/iTunes_Control/Music/ .

   ...note there is no 'z' option; compression makes this more than twice as slow. Also note the underscore in iTunes_Control. This will copy your music files to SomeTempDir on your mac.

5. What it will create is a directory of subdirs that looks sort of like this: F00
   F01
   F02
   ...you get the idea.

   Inside these Fxx dirs are all your media files. They have been renamed to three- or four-digit keys with their extensions (.mp3, .m4a for AAC, .m4p for DRMed-AAC, .mp4 for video) intact.

6. If you double-click each of these items, they will usually (if they already exist in the iTunes library) just start that version playing, without copying a new version. If they do copy it in, you know it's a file you didn't have, *or* that it is a file that was missing. If your 'keep library organized' settings have changed, it may copy the files in if the newly frobbed 'correct name' is different from the ones that it was synced to the phone with. But anyway, they'll play fine, even the protected ones, since they were synced using your iTMS keys.

Anyway, that's how I got some of it back. I hunted around for the CDs I'd used for a bunch more of it. Some of it was freely downloadable, so I just went through and red/l-ed it (thanks DJ Earworm, thanks Pheugoo!) The last of it, the stuff I'd bought via iTMS and didn't have a backup of and that I couldn't get by 'upgrading'...well, I saved off the list of them to work on getting hold of later, and deleted the DB entries.

The next problem was the stuff that had moved. iTunes was pretty good about figuring out all the problem entries and tagging them with a (!). You can force it by just selecting the top entry and then using the down arrow to zip through the library on repeat - it checks the file validity every time the track is *selected*, not played, so it'll tag the ones that are broken. I found that in 99% of the cases, I could find the track back in its original location in my media tree. Since most of the corrections involved adding album data or switching 'compliations' to 'real albums' and etc, it was pretty trivial. The only crappy bit was that the only way to fix these tracks was to double-click *each track* and then navigate the file selection dialog to relocate them. Ugh.

Once you've done so, be warned - it doesn't always move the track back to the right place. I don't know what the algorithm is. However, if you've selected 'Keep library organized,' once it's properly found the track (i.e. it'll play if you doubleclick it) you can rename the album or artist and then move it back to the right name (my database entries were newer, thus retained the 'proper' data) and when you make *changes* it'll refile the track properly.

So there you have it. A fuckton of work, but that's because I"m anal about my music library in a way I am not about anything else.

I think I got back all 18K+ tracks, with the exception of around 150 tracks I didn't have backups for.

Then I Time Machined my internal drive to my (replacement) external HD, which is now all one volume - backup only - and told it to include my iTunes library.

Whew.

Justifying console gaming

JC Fletcher over at Joystiq has an excellent closing thought on a post regarding the delay of Battlefield 1943 for PCs - he advises PC users who really don't want to wait to just, you know, buy a console, and recommends:

If you find the idea of spending that much on a console distasteful, just tell yourself it's a really big video card.

Heh. I thought about it. 'S true, that - an Xbox360 Pro costs approximately as much as a high-end gamer video card. And comes with a computer wrapped around it, to boot.

BUNNIEZ IN ACTION

...at my desk.

Stupid hard drives. Stupid iTunes.

Spent five or six hours last night trying to resurrect my iMac. Well, wrong word - the iMac was fine. I'd lost my external drive with my backup and my iTunes library. Don't care about the former, so long as I can get a backup on the replacement without anything happening to the Mac, but the latter..yeah.

I had a backup of the media directory on my server, but due to perms craziness, the automatic rsync update no worky and I'd been doing it when I remembered to. The 'last modified date' on the backup tree was 5/9/09, which wasn't too bad - but I discovered while looking that in fact the most recent actual file synced to the backup was from 10/8/2008. Guh. So everything I've put into iTunes since then - *pffft*.

Worse yet - I couldn't get iTunes to find the restored files. The problem was that since they had lived on a separate volume, the path my iTunes library DB was set to reflected that. However, iTunes lets you change the path to the library, right? Okay. So I copied the rest of my library back into a local directory, and changed the library location.

Nope. Couldn't find the files. Even through they were there. When I clicked a file, it would give it the '!' mark and give me the message ("Do you want to locate it?") When I located it, it was right where the prefs file said it should be, and then it would play - but I have over 18K items in my library. Even after the loss. Not viable.

Okay, time to make a backup copy of my iTunes library directory (the databases, not the media) and start fucking around. BBEdit let me open iTunes Library.xml. In there, I found both the prefs setting for the library location and the listings for all the tracks. Interestingly, the library location was *different* from the path in all the track entries - and the track entries' paths were all set to the 'default' of ~/Music/iTunes/iTunes Music. Lovely.

Okay. Changed all the paths of all the entries using grep. Saved. Restarted iTunes.

Nope. Apparently, the XML file is an *export* - the controlling file is the iTunes Library.itdb or whatever, which is a binary database file. Fuckers.

After a couple of hours, I determined that if I moved all the files into the actual ~/Music/iTunes/iTunes Music/ path, and then changed the prefs in iTunes to reflect that, it would in fact find the music. Why it wouldn't find it if I put the music anywhere *else* and changed the prefs to point *there,* I have no idea - but I have a guess; I'll guess that since the file loc paths in my XML file didn't agree with the library location pref, I'm going to guess that iTunes was not or was refusing to properly update the file locations when the location pref was changed - maybe because it was already screwed - i.e. they didn't match.

I don't know how that happened, but I can guess - I'm going to guess that when my original drive died, my Mac (which was running at the time) did *something* involving iTunes (I may even have left it open) and when it realized it couldn't see the volume the library was set to, it changed all the file paths back to the default location - even though the files weren't there. That library file was the one I was still using - because my backup of the library database had been on the other partition of the dead drive, in the backup volume.

Fuck fuck fuck.

So anyway, after moving everything back into the 'default' location, iTunes found it all.

At which point I can determine that 353 items that *had* been in my library no longer exist, although their DB entries still do. Ah well, at least I have a chance to go re-acquire all those tracks and re-enter them, since their data is still there - playcounts, ratings, playlist inclusion, etc. And looking at them, I know that at least 200 of 'em are tracks I *got* but never played with, so meh.

Bother and other words

I went out of town for two days, and during that time a) I had a vicious gout attack and b) my iMac ate its external hard drive. That drive contained my Time Machine/SuperDuper backups (okay, not terrible, the *original* is still working) and my iTunes music files. FRAK. The iTunes library DB is still sitting on my internal drive, but all the tunes & movies...yeah.

On the plus side, I have also maintained a backup of that volume on a server.

On the minus side, the automagic rsync+ssh never worked right, so I would do it manually when I got around to it. Last time? May 9th, 2009.

On the meh side, I don't think I added more than maybe ten or fifteen tracks to my iTunes library since May 9th.

Sigh.

On the minus side, now I gotta shell out not only for a new external drive, but for the USB<->SATA dock and pair of bare drives I kept *meaning* to buy as my 'backup backup.'

Shame, degradation, and responsibility

I can't believe this.

Lynndie England is without question a symbol, known internationally, of the worst components of the American character. The absolute best that can be said about her involvement in Iraq is that she was guilty of willful and reckless ignorance - ignorance of ethics, ignorance of the potential damage her actions could do, and ignorance of basic human values.

Worse things that could be said? That she, in writing and promoting this book, is guilty of deliberate profiting on her degradation of others, publicly and indefensibly.

And a Veteran's Association is sponsoring this? At the Library of Congress? For fuck's sake, people, this woman dishonored the uniform you wore or wear. Out of willful ignorance at best, malice aforethought at worst.

So now you're going to help her push her story of victimization.

Get a fucking grip.

Best use of word 'mishegoss' ever

Ladies and gentlemen, the incomparable Walter Matthau.

For those who don't know, the quote is from the original version of The Taking of Pelham One Two Three.

Benthic Randites

My devious brother has gifted me with a copy of Bioshock for my Xbox. On first play, I got sucked in for around four hours and had to drag myself, shuddering, up into reality in time to get *some* sleep before my next day's work.

This bodes well.

More telephony options

Sweet! A beta invite to Google Voice just hit my inbox. Now to find a Cool Number(tm).

PlasmaCon

Hm. I notice, looking over my electricity bills, that they seem to have jumped $40/month starting the first full month I had my plasma TV and Xbox360. And then another 25/month when air conditioner season started. Really? That TV and gimped PC use more power monthly than my antiquated air conditioners? Huh.