I'll wait.
Did you say 'RedHat let their servers be penetrated?' If so, good try, but no.
Did you say 'RedHat let their signing servers be penetrated?' Again, good try, but no.
Hopefully, you said 'Wait, their signing servers are accessible via network? WTF?' Because if you did, congratulations, you too saw the Epic Op Fail.
FAIL.
top - 11:11:00 up 789 days, 23:57, 1 user, load average: 0.03, 0.03, 0.00
'nuff said.
There are options, people.
One: FileVault. I know it's been put down and bitched about, but generally, it's a good option in this case. You should configure a master password (so that someone else can't set FileVault and lock you out) as well. If you've done this, the only way that someone with administrator access should be able to gain access to your data is by changing your login password, so at least you'll have warning if they try.
Two: Encrypted Disk Images. If you're really worried about this, create (using Disk Utility) an ecrypted disk image and store your private data there. Don't put the access code in your Keychain. If you do this, even if someone has administrator access to your Macintosh, they won't be able to open your encrypted image no matter whether they copy it off the computer or try locally (unless they crack your password, of course).
Essentially, while I think it's a bad policy on Apple's part to try to get their users to hand over an administrator password to their current image, let's not get overwrought - it's not a good idea to ask someone to troubleshoot your machine without giving them access as well.
If your hard drive has gone bad, or is going bad, you should be able to format it before bringing it in - if you can't, it's unlikely anyone else is going to recover data off it either. In any case, if you're bringing in the machine for a flaky drive, then you should be wanting them to nuke/replace it.
Perspective.
I emailed the author idly and asked if it would ever be ported, noting that I'd happily pay $20 or so.
His half-encouraging answer: "Someday."
Ah well. Guess I should just look forward to The Force Unleashed then.
He has a Macbook Air which he uses as his primary computer. In addition, he'd just gotten his original iPhone upgraded to firmware 2.0 and set it to talk to our Exchange server. As a result, he realized that the calender information he could see on the iPhone wasn't the same as that on his Macbook. He logged onto the server using our Outlook Web Access (OWA) URL, and found that the iPhone and the OWA system (hence, the server) agreed on his calendar information - but that changes that had been made on his Macbook weren't syncing up to the server.
We began to test.
If I sent him an invitation to a meeting and he accepted it using OWA, it would appear on his OWA calendar and after short delay appear on his Macbook and iPhone.
If I sent him an invitation and he accepted it on his Macbook, it would immediately appear on the Macbook's Entourage calendar but not appear on the OWA (or iPhone) calendars.
If I sent him an invitation and he checked his iPhone, the iPhone would immediately put in a 'tentative' meeting at the proper time. If he then 'accepted' the meeting from the iPhone, it would change to 'confirmed' - but if we checked on the iPhone again after a few minutes, it would have changed back to 'tentative' but the invitation was still gone. It would not show up in OWA.
He can send and receive mail from any of the three clients - Macbook Air, iPhone or OWA. That works fine.
If he sent an invitation to me using OWA and I accepted it using my Mac (and Entourage) it showed up in my OWA and on my Mac.
If he sent an invitation to me using his Macbook and I accepted it using my Mac it would show up in my OWA and on my Mac.
So at this point, it appeared there was something different about his Entourage setup or his Macbook or his Exchange store. Note that I had changed my Entourage account settings to mirror his, and made sure my Mac was on the same wireless network as his prior to the above tests.
Next we created an account for me on his Entourage on his Macbook Air, using my account information. It took around 20 minutes to sync initially (arg) but eventually everything was there. I accepted an invitation using his Macbook Air and his Entourage but my account configuration - and lo and behold, it showed up on my OWA and on my own Macintosh.
At this point, we seem to have narrowed the problem down to either his local datastore on his Macbook Air or his Exchange store on the servers. We discovered that his store was 1.7 GB in size, and that the store that it was residing in has a policy set such that over 1.5 GB it 'blocks send and receive.' However, he can still send and receive mail fine either via WebDAV (Entourage/iPhone) or via OWA.
His store has a large number of items in it, and we noticed that although the appointments did show up on my account (which has many many fewer items, but still several hundred to a thousand) it took some twenty to thirty seconds to sync. We're going to see if his appointments have properly synced by tomorrow. If they have, the next step will be to try to prune his account and see if they sync faster. If they haven't shown up tomorrow, we're going to try to prune his store below 1.5 GB (the limit observed on the server) and see if that allows calendaring as well as mail to work.
Confused, but making progress.
nwk.itunes.com 17.149.160.45
As I'm typing in the word doc, the damn space will slide over to Entourage whenever the 'Activity' window pops up indicating that it's doing network tasks like getting mail. It will pull the Word doc *with* it. So suddenly, without any action on my part, I have the sliding animation, Entourage pops into the foreground, and my Word doc is now in the background on a different space. If I manually ctrl-# back to the Word space, it's now *empty* - because the doc got pulled over and shoved behind Entourage. It's impossible to get any work done.
That's acceptable.
What's not acceptable is this. If I go to the Entourage Space, write a message and hit 'send' and then immediately (as I'm used to) jump to, say, my Safari space and continue working, I will get a 'bong' a few seconds later as Entourage pops up the dialog. I go back to the Entourage space...and it's not there. I search madly throughout every Space I can find. It turns out that it places the dialog *behind* the Safari window I was working on when it notified me - and bringing Entourage to the front, rather than popping that dialog, instead shifts me to the Entourage workspace, guaranteeing I won't find it. Of course, no Entourage window will respond to focus, since the application is modal and waiting on that (now hidden) dialog. The only way to get Entourage back is to locate the dialog by using Expose on the proper workspace, or by manually shoving around all my windows until I see it, and then dismissing it.
Irritating.
...there are no hard drives in the drive selection window. WTF?
Try running disk utility. It finds my internal drive but not the partition. This freaks me out. I exit Disk Utility without touching anything. The Macbook Pro was running beautifully on Tiger 1 minute prior.
Search the web. Find this thread at Apple's support forums. Wait, let me get this straight - during an OS upgrade/install, one of the more stressful and careful-tippytoe-making times of a computer user's experience, the damn installer just won't bother showing the drive until it's completed a drive check it doesn't tell you about?
What the fuck?
So I did as recommended. I got to the Drive Selection window and I waited.
Ten minutes later, my internal HD popped up in the pane.
That's...just...Apple, that's just fucking stupid. No warning at all that the thing is busily checking the drive integrity? No notice that it knows the drive exists at all? I'm so lucky I didn't panic in Drive Utility and decide to repartition or some such to get the thing to recognize it. (Well, okay, I'm not lucky, I know better, but a lot of users might panic...)
Sigh.
I hadn't done a full verify/repair during the install, only a perms check - so let that be a lesson to me.
On the other hand, I stand by my kvetching about interfaces. Also, as another friend pointed out to me and I verified, X11 is now limiting X11 windows to the machine's main monitor, which is...annoying.
Time Machine, after spending four hours or so doing its first backup, hung my machine when I came back the next day. It was hung during what looked to be a later backup, because instead of the 1.8MM+ files in the status window, it said something like 31k; however, it wasn't doing anything. It also wouldn't let me do anything related to disk access. Nor would Finder relaunch. On hard-cycling it after letting it sit there for 45 minutes just in case it was stuck on some big file, I found that Time Machine thought that there was no backup on the disk I'd designated - and my main drive was suddenly a Time Machine drive according to its icon and the fact that I had no write perms to it, although Time Machine's prefs showed nothing of the sort. A full reboot 'fixed' that.
Ew.
svn: REPORT request failed on '/svn/repo/!svn/vcc/default' svn: REPORT of '/svn/repo/!svn/vcc/default': Could not read response body: Secure connection truncated (https://svn.myco.com)
...or like this, apparently random choice as to which:
svn: REPORT request failed on '/svn/repo/!svn/vcc/default' svn: REPORT of '/svn/repo/!svn/vcc/default': Could not read chunk delimiter: Secure connection truncated (https://svn.myco.com)
I had taken the following steps to try to ameliorate the problem already:
Then, after investigating and watching apache while the checkout was running, I noted that the working apache process would grow during the checkout to consume appro. 27m of resident memory and then suddenly vanish when the error occured, briefly showing up in the process list as httpd <defunct>. Closer examination of the apache logs showed that these httpd children were indeed segfaulting.
This is a RHEL5 system using Subversion 1.4.5 compiled from source, being served by the distro's Apache (httpd-2.2.3-7.el5) over SSL using the distro's neon-0.25.5-5.1 at runtime AFAICT (although subversion was compiled with neon source in the tree, from subversion-deps). It is a fsfs repository on an ext3 filesystem (local disk) using mod_auth_kerb to auth to a windows DC.
THE ANSWER
So someone smarter than I continued where I left off and discovered that I was mostly right, there was a memory leak. But when I had tumbled off to get sleep, exhausted, he soldiered on, and tracked that leak to the DAV module (mod_dav_svn) in Apache. The problem, apparently, is that the DAV module was performing the authentication steps for every directory, every time it was accessed, and the leak was leaking during the auth process. Thus, in a large tree, the dozens or hundreds of auth steps would end up leaking the module into instability. The solution was to tell the DAV module not to reauthenticate for every path underneath the main one once the user had been authenticated into the repository. To do this, add the following statement to the apache config (in the vhost entry for the repo, in our case):
SVNPathAuthz off
...and in our case at least, all was well.
I'm not saying I'd do this permanently. But a quick script to swap those kexts out somewhere and reboot, then (if I needed to use USB storage) swap them in? Probably worth it if I was already going to all that trouble.
This confused me a bit, since I don't send mail directly. Both of those messages were sent through SMTP servers, one I run privately from a colo facility and one run by my employer in the NY Metro area. As far as I could tell after checking, neither of those servers was listed on the DSBL. However, the SA module that does the checking apparently checks the first hop in the mail headers, and sure enough, there's the IP that the message originates at in both of them, and it's my Comcast IP.
So. Doesn't matter that I'm sending through SMTP servers, I'm blacklisted. How to get removed? Ah, there's the rub. DSBL will remove you if you can respond to a confirmation email they will send to you. That email can be sent to one of four possible email addresses, and their site tells you flatly that there are no exceptions. Those addresses are:
Now, obviously, the @comcast.net addresses are useless to me. The problem is that the others, which accurately point out the IP address of my cable modem, are equally useless. This is because I don't run an SMTP mail server out of my house. Even if I did, Comcast, in its attempt to help prevent SPAM, actively blocks that port on my segment or perhaps modem, I'm not sure (I use secure SMTP on a different port to reach my outbound servers). So what the DSBL is saying, essentially, is that the only thing I can do is to have their system generate an 'annoyance' email to comcast's abuse address which will be ignored.
I'm generally a proponent of blackhole lists. However, I'm also a proponent of responsible administration of them. The problem here is that (as DSBL acknowledges) the range of IP addresses that mine falls into is dynamic and public access. This means it is both a high-threat range and one which churns, meaning that harsh measures taken against addresses in the range will in time impinge on other, innocent customers as the addresses 'turn over.'
Realizing I might be overreacting, I went to check the report on my IP address to determine what, in fact, it was being blacklisted for.
Turns out that there was an open SOCKS4 proxy on my IP sometime back in November of 2005, approximately seven months before I was assigned this IP. Um. Okay. So here's my question - why is there no way to retest - or request a retest - of an IP? I understand that an automated retest would simply invite gaming. I also understand (and have some sympathy) with the notion of 'let the customer tell the ISP to fix itself.' However in this case, the ISP isn't the problem; the problem is what some unknown person did with this IP before I had it, and there's no way for the confirmation process to reach me because I don't control the DNS or the abuse/postmaster accounts for this domain. I do control the IP address, but I don't (and probably can't) run a mailserver on it. Hence I can't receive an email addressed to its specific reverse FQDN, because there is no internet mail service at that address.
Admittedly, part of my particular problem seems to be that the site I'm sending to is penalizing me for being on the 'single hop' DSBL list when I'm clearly not sending single-hop - I'm sending through SMTP servers. I'm not on the multi-hop list, although I am on the 'unconfirmed' list. However, I know for a fact that site is using standard SpamAssassin filtering, which means whatever it's doing can't be that esoteric.
So what am I to do?
The fucking system's broken, boys.
The Treo 650 is still soldiering on after two years of hard use, for which I must give it full marks. It's scarred and beaten, but unbowed. It has a completely useless camera and a more useful MiniSD slot, big plus on the latter. The reason for my approval here is that the Treo also serves, in around 95% of its non-phone use, as my pocket book reader using Mobipocket's software.
I buy my e-books mostly from Baen Books' Webscriptions site - DRM-free Mobipocket format sci-fi. I have maybe 135 titles in my Palm at any given time, pretty much guaranteeing that if I find myself stuck in an airport or train station for twenty minutes, I can at least zone out to a fun book.
The iPhone, it appears, has little chance of gaining Mobipocket functionality. If Apple is being as stingy with dev kits as is being whispered, that doesn't bode well. So what to do? This is sort of a big deal for me. As I see it so far, I have a few options.
Well, I'll be ecstatic if a) comes to pass, but I've been noodling about b) anyway. It occurs to me that at the most industrious end of the scale, I could try to get a Mobipocket-format capable reader that is implemented as a Java Applet and which would run inside Safari (although, I must admit, I have no idea if there will be a full JVM on the iPhone). That leaves only the problem of where to store the books, at least, such that the applet could read them.
I might mail them to myself, embedded in HTML. I wonder if the iPhone would cache them in mail.app and then open them with a reader? Perhaps I could render them as PDFs, first. That might help.
Hmf. It all seems so convoluted.
But no, I have a new passion.
Its name is DEFCON.
Introversion Software, who brought us the game of hacking called Uplink several years ago and more recently the critically acclaimed vector Real-Time Strategy title Darwinia, have come up with this gem. Reusing a great deal of their simple but effective artwork from Uplink, they have created an icon of my 1980s nightmares and dreams.
DEFCON
Everybody Dies.
This is an exquisitely succinct summation of the game itself. The game, as can be deduced from its title, is no less than a playable simulation of your favorite and mine, Global Thermonuclear War. What makes it brilliant is the visual design and the playability. Let me take you through a quick notional game by way of explanation.
The Setup
You choose a Bloc to play. You can play North America, South/Middle America, Europe, The Soviet Bloc, Africa, or Southeast Asia. Up to six players at once can thus face off. Choose a color. As the game begins, the world is at DEFCON 5; a timer indicates the time until DEFCON 4 is reached. The assumption, I presume, is that those damn diplomats have screwed everything up and it's up to you (the warfighter) to make good on their threats. Operational diplomacy is possible; Diplomacy (the game) style maneuverings are encouraged, as alliance blocs can be joined and broken during play! Ah, backstabbing.
Enemy Launch Detected!
The Pieces
In the interest of avoiding information overload, there are really only a few play pieces, which you must place] somewhere in your territory at the start of play (during the DEFCON 5 period). On land, you can situate radar stations, airbases and silos; at sea, fleets consisting of battleships, carriers and/or submarines. Airbases can launch either fighters or bombers. Carriers can do the same, as well as go into antisubmarine mode. Silos can operate in either air defense or ICBM launch mode. It takes time to switch between modes, and units have a limited number of expendables - fighters, bombers, or ICBMS. SAMs are infinite and simply have an interval timer, and fighters will slowly regenerate. Submarines can operate in passive sonar mode, where they merely move from place to place and try to avoid being seen; active sonar mode, where they can (poorly) attack surface vessels, and SLBM launch mode where they can loft their limited number of nukes.
Launching anything, naturally, makes your unit visible to early warning systems, even if it's not currently within radar range of an enemy unit.
ICBM launch mode: 128 seconds
Gameplay Gameplay moves in phases, each phase being a different DEFCON level. At DEFCON 5 ("peace") all you can do is place your units and give mobile units movement orders. At DEFCON 4, you can invade enemy territory/airspace, and units will automatically engage each other; at 3, you can launch bomber strikes, etcetera. A large and foreboding timer scrolls down at the center bottom of the screen: DEFCON 4 in 13:45. This is a RTS, so units move continuously; you can change their orders at any time by simply clicking and dragging. If you are playing solo, you can change the time compression using a standard set of arrow controls (one, two, three or four arrows). If you are playing in a network game, you can requesta time compression change, and the other players will be given a chance to aquiesce or refuse.
I haven't yet had the opportunity to coerce...er, entice any of my friends into purchasing the game and having a go 'round, so I can't tell you how well the alliance/diplomacy bits work. If they work as well as the rest of it, I'd say I'll be quite happy with the mechanics. The interface is smooth; there are a few UI design issues which are eminently fixable. For example, the game field (a Mercator projection world map) scrolls smoothly about as you move your cursor near to the edges. However, the time compression controls are at the top of the screen, so moving to them causes the field to scroll as you approach. You can zoom out to fullscreen to prevent this, but it's annoying. It's also possible to have the unit info window (fixed at the bottom right) actually overlay units on the map. Still, these are minor quirks; the interface is generally only noticeable in those few moments when its deficiencies become jarring. The rest of the time, it just is, which is what you want.
There are various modes of play. There is a standard game; there is a 'quick' game, compressed even further. There's one I'm dying to try, called 'office,' which is played over a network with friends in realtime over the course of a day - missiles take ~20 minutes to traverse their paths, and you have time to sweat and watch everything go to pieces in realtime slow motion. The network game is played by a player creating a master server and others joining. Thanks to Ambrosia Software, there is now a Macintosh OS X version of the game (which I'm using) in addition to the original Windows version; Ambrosia also promises a patch to allow Linux users to get in on the megadeaths. It's OpenGL and plays smooth as butter on my 1 GHz Powerbook G4, and is distributed as a Universal binary for Macs.
Beograd hit: 1.4M dead
Scoring
SImple. As the game says: Nobody wins - but maybe you can lose the least. You can try to play a full counterforce game, but the game is scored in terms of how many kills you get, how many civilians you lose, and how many nukes you have left at the end. Once DEFCON 1 is reached (and the nukes really start flying) it's not about saving 'em for later, trust me.
The design is really excellent. It's simple strategy at the outset during placement - pick your enemies and place your units to face them; choose a defense-in-depth versus a frontal wall, etc. During the game, it's simple maneuver - if you're careful, you can maximize your units' placement - but it's not a true maneuver wargme, because the timers ensure you don't have the game length to really move units around the board. You have the time to adjust your units' initial placement, and that's all. Finally, during the Global Thermonuclear War phase, you get to push the button and see how well you set up your armageddon.
Incoming Nuclear Weapons: Impact in 348 Seconds
Look and Feel
This is the genius bit. The entire game is built using vector graphics, brightly lit on a black field. The look of it closely hews to the displays prominently featured in the iconic '80s movie Wargames - if you can remember at all the scenarios that Joshua was spinning out on the displays in Cheyenne Mountain, then you know pretty much exactly what DEFCON looks like on your computer. The same hazy glows to indicate unknown territory; the same angular bright line icons to indicate units; the same sprouting flowers of arcs to show the moment of truth when the missiles climb out. That's what always arrests the attention of the passenger next to me - that impossible to miss image of a world map drawn in glowing blue-white lines, with green and red bullet shapes climbing slowly out from the continental US and Russia, leaving arcing lines behind them as I frantically click around the Atlantic ocean, moving submarines and carriers into position for the followup strike.
The only thing missing is the haunting music.
Oh wait; no it's not. The game has that too. Turn your speakers up.
DEFCON
http://everybody-dies.com
Introversion Software (http://introversion.co.uk) - Windows Version
Ambrosia Software (http://ambrosiasw.com) - Mac OS X / Linux(?) Versions
Shareware - Playable Demo (limited to 1 AI player and 1 game mode, only 1 demo player per network game) downloadable; $25.00 for full license code.
Linden may be running up against something which I and several of my friends spent a great deal of time thinking about.
If you have a virtual environment, that is, one which is compelling and pervasive - or if you're going to have one, you're going to need a draw. You're going to need a reason for it to be there, and a reason for people to come play there. The web came into its own not because it was technically sweet, not because it allowed people to share knowledge (which was its original purpose) but because of commerce. A persistent multiple user virtuality is a much more complex undertaking. I posit (and freely admit that indeed, my PoV is colored by my upbringing and background) that here, too, you will need commerce and value in order to bring people and organizations into this brave new world.
Linden knows that. That's why they have an economy, after all.
Here comes the problem. In order to have property, you have to have property rights. In order to have property rights in a digital environment, you need to have digital rights management. This is not news, zillions of other people have said the same thing. I am struck, though, at how few decent answers to this conundrum there seem to have been presented, and how many of the 'copyfight crew' seem to also be ardent pushers of the Second Life world. The take-no-prisoners fight against DRM as a technology, a concept and a practice runs directly counter to one of the most powerful motivating factors of there ever being a worthy persistent and pervasive virtuality that we can all play in.
It's fairly easy to rail against DRM on the Web, because the use of said information occurs in a space other than on the Web itself - consumption of digital media by a consumer occurs in a private space, usually, since 'use' on the web is akin to 'publishing' - there's no interaction. Plagiarism, after all, is much more clearly identifiable as 'bad use.' But in a persistent virtuality, a user might 'use' a piece of information in much the same way as they do in meatspace - by listening to music, or watching flat video - for their personal consumption, making the question much much hazier. In other words, the existence of electronic data inside this virtual environment is suddenly directly problematic, as opposed to the web, where it was mostly a publishing and transport problem. Now, it's not just publishing and transport (which is fairly easy to litigate, regulate, monitor and lock down either legally or technologically). It's the very existence and utilization of this material at the endpoint which matters.
So. Where does this leave me, in my rambling state? I guess what I'm trying to say is that while the uses DRM is being put to may suck just as badly as those decrying it say it does, and while DRM may be just as technologically futile in its present forms as they say, the very existence and goals of DRM may not be unremittingly evil (as it's being presented). I think what worries me is that at some level, the cry against DRM, and the copyfight movement in general, taken to extremes, can be seen as a cry against the notion of property rights in general when applied to electronic media. That's a problem for me, because in my vision of the future there is, in fact, a persistent and pervasive virtuality that I want to play in, and it's supported and built because various people and entities think there's value to be had there. That value is realized based on the notion of the existence of property rights in a virtual space. If the current fashionable trend of the decrying of copyright protection continues, those very property rights, which might attract the investment and effort required to make that playground a reality, may be threatened.
There are indeed, in my opinion, massive problems with how copyright law and DRM is being used and interpreted today. The DMCA is a prime example. However, trying to 'fix' the problem by attacking the very concept itself is, in my opinion, counterproductive. Working to fix the law, and working to fix the abuses, is better and safer; while using extreme responses to an extreme problem is always tempting, it runs the risk of creating situations just as hard to recover from later. If we end up with a world with no RIAA, but where no corporate entity is willing to invest in a virtual world because everyone using it thinks CopyBot is perfectly normal behavior, what then will we do?
Bemused, I logged out of the current Google account I had logged in in my browser for GMail access.
The World news reappeared.
What the hell?
I have no idea what's going on. My prefs on that user (in Google News) all show that I'm asking for World news to be my first (after the Top) visible section, and that I want 9 stories. The only thing that strikes me is that there's a notice which says 'New! Search History now includes News Items' or some such, implying that my search history might be used to select News of interest or vice versa, and that it's a new feature. I can imagine that a bug in that algorithm might cause a user profile from a GMail account I use for specific purposes (read: Google Analytics and some website registration) to show up as 'completely disinterested in world news.' I suppose.
Er?
Well, here's what I did.
First, the situation: I have a Red Hat Enterprise Linux 3 machine. For reasons we won't go into, the RPM database was entirely erased. Not just corrupted; it was hurt so badly that a new, blank one was put in its place, and rpm happily claimed the machine had no packages on it. For other reasons we won't go into, reinstalling on this machine was possible but not desirable; the machine is remote (in a colo) and is running a role that would have required the building of a substitute box, switchover, travel, reinstall, etc.
No, it's not a critical role.
Anyhow, the machine in question hadn't been updated for several months - it was this overdue update that had caused the freakout. I was using the Ximian/Novell rug/rcd product to update it - this system front-ends RPM, though. It had resolved what needed to be done, resolved dependencies, downloaded the requisite rpm files for the update into a cache dir, and called RPM to remove the conflicting (old) RPM files when Something Went Wrong.
After thrashing around trying to rebuild the db using the tools (failed), trying to create a new db by figuring out what files I had on the system (bwahahahah), and looking at the originally-installed rpm db state file in /usr/lib/rpmdb (as opposed to the /var/lib/rpm/rhel-3as-i386/redhat location of the running database) I came within a hair of giving up.
Then I found out that rpm sticks a cron job into RHEL installations (and FC installations?) that simply dumps the output of rpm -qa to a logfile (/var/log/rpmpkgs) once a night. That logfile, thanks to logrotate, is rotated once a week, so although it had been several days since the incident and the main file had been overwritten with the blank rpm dump, I grabbed /var/log/rpmpkgs.1.
Inside that file was a one-per-line list of everything that had been installed on my system.
I took that file to a machine which had the full RHEL3 current package set mounted to a directory. I then created a new, blank RPM database:
rpm --dbpath /home/jbz/newrpmdb/ --initdb
Then I used the mounted copy of the package repository and xargs to do a series of rpm -i --justdb commands, which 'install' rpm packages but only perform the db modification steps:
cat rpmpkgs.1 | xargs --replace={} rpm -ivh --force --nodeps --justdb --dbpath /home/jbz/newrpmdb/ /(path-to-rhel-3as-i386-packagerepo-rpms)/{} > install.log 2>&1
The --force and --nodeps were required because I was installing one rpm at a time (via xargs) to a blank database, so all dep checks would fail, but I didn't care - the packages were already installed on my machine, I just needed the DB to reflect that.
Once that was complete, I had a directory containing my new rpmdb file in /home/jbz/newrpmdb. I copied that to the affected machine, put it into /var/lib/rpm, and then performed an rpm --rebuilddb just to be safe. That completed without a hitch.
At this point, I had a mostly-complete installation. There were a couple of problems, though. The main one was that since several packages had been updated since the machine had last been rereshed, some package names in the package repository didn't match the package names in the rpmpkgs.1 file. As a result, the logfile (install.log) contained a bunch of lines showing errors where rpm couldn't find the file it had been asked to install since the current repository had new file names.
Note: This is because Red Hat (and others?) when updating the packages apparently change the package name by infixing versions. For example, suppose aspell-0.33.7.1-25.3.i386.rpm is updated; the new package will have its true name, and that version might be renamed aspell-2:0.33.7.1-25.3.i386.rpm. I presume this is in case there are dependencies which rely on the earlier version of the package; it's kept in the update channels but marked up so that the updaters understand that it isn't the current version. This is a guess, though.
In any case, I had several packages which hadn't been installed, spread throughout this massive logfile. I used the following to get a simple list of which ones got missed:
grep "No such" install.log | cut -c 70- - | cut -d " " -f 1 > missingrpms.txt
Note that the '70-' reflects the fact that the pathname of the repository I was using was 70 characters long, so cutting the first 70 characters off the line produced output beginning with the filename in question. Your mileage will vary. The second cut statement sets the delimiter to a space, then discards all text after the space, leaving only the filename.
I then got lazy and, since my list was only a few lines long, manually copied those RPMs over to the affected machine and force-installed them using --just-db. You may need/want to script this update.
At that point, I ran rpm -aV to verify the installation. I came up with approximately forty lines of variance, mostly permissions variance, which is not at all out of whack for a 350-day uptime, 2.5yr install server. No major issues have cropped up. Looking through the variances, they all appear to involve config file changes, permissions on doc files, or (in five or six cases) missing files from packages I have customized. I will mark this 'successful' and count myself lucky.
WARNING: Your mileage WILL vary. Messing with your RPM database is NOT FUN and should NOT BE DONE on a running system unless there is NO OTHER OPTION. I had no other option, so that's why I did it. I take no responsibility for any damage you may do to your system or data in trying any of this; I offer it purely as a last-ditch recovery option, just in case you find yourself with nowhere else to go.
Good luck.
Others who are actually qualified have been working on this problem. This gentleman has been approaching from the other end, that of forensic examination of unmodified data; at the moment, his work involves still imagery but he appears to be moving into video.
So there is interest in proving video (un)modified. I will acknowledge that the applications are much larger when applied to 'standard' video. Still, I can't help thinking that if there is enough doubt to create this field of study, then a 'verified video stream' might be worthwhile.
If you didn't notice or don't care, please cheerfully ignore this message.
Man, I still remember QuakeWorld TF2 as the best team shooter ever. If that is actual engine footage, this has the potential to be a complete friggin' hoot. Cannot wait. Wheeeeeeee!
Worse, it happens frequently enough that a good solid game of Battlefield: Vietnam guarantees it happening at least twice.
When it happened in Windows alone, I went into the Device Manager and looked, and there was indeed a USB Device in the Human Interface category that had the deadly yellow question mark on it. However, when I unplugged the keyboard and reconnected it, causing it to come back, that device was still there - and when it happened another time, Uninstalling that device and then performing a scan didn't make things better. It looks like the keyboard itself is actually crashing.
This happened three times before I managed to install any software into my Windows XP image, so I don't think it's something I did. Bugger.
Update: Well, hm. I moved the keyboard to a powered USB hub so as to avoid having to stand up and fumble behind the iMac and scratch up its Shiny Plasticky Goodness every time this happened. This changed things slightly; every few times the glitch occurs, the unplug-replug-keyboard trick doesn't work, and can only be remedied by actually unplugging the hub. When this happens, though, Windows goes through its whole "Oooooh! New hardware, goody goody gumdrops, I get to make you reboot now!" routine (although it just puts up a nagging dialog, it doesn't force an immediate reboot, presumably since it's just a USB hub). The interesting thing though is that when I do reboot, it says it can't because the program 'AppleCDEject' is Not Responding.
Interesting. Presumably whatever bit o' software Apple uses to watch for that handy Eject button on the keyboard is tanking. I don't know if that's causal or symptomatic, though.
Update Update: On the advice of a colleague, I spent some time trying to figure out how to prevent AppleCDEject from loading at startup. It's not in Startup Items, nor is it obvious in the Services that are coming up (although I did gleefully kill off stuff like the Theme Manager and other useless frippery). As a test, though, I booted XP, went to the Process Manager and killed it manually, then gamed for two hours without a single keyboard error - so things are looking up. One place I haven't looked yet - there is a single 'Apple Drivers' service loading, I didn't carefully look to see if there are arguments to that service which might be telling it to load the AppleCDEject deal.
I should also try a non-Apple USB keyboard (gotta find one first) and then look to see if it loads.
Update Update Update: I still haven't checked to see if there are keys in the registry that govern the startup of the AppleCDEject program, but as a stopgap measure, I just moved the executable out of the C:\windows\system32\ directory into a temp dir, and it doesn't execute on startup. I notice that I still do get the occasional 'hiccup' in the USB drivers (loss of input, multiple BONGs, input comes back) but they seem to pass within five to ten seconds and the keyboard doesn't crash anymore - so it's still an improvement.
I found a couple of my own, some more annoying than others. Here's what I have so far.
* Okay, this isn't totally fair. Every state in the union has a Springfield. But still. State government IT. Shudder. At least ours is making the effort.
If, however, I'm going to start mucking around with my Mac at the process level as well as on the disk bus, I need a better backup than the one I have now - or to at least disconnect my media drive first.
diskutil same thing, fdisk complains about resource busy, desperately trying dd if=/dev/zero of=/dev/rdisk3 bs=1024 count=3 results in no write) I discover that...
OS X is trying to background fsck the stupid partition, despite the fact that it doesn't work.
Kill that process. Try again w/fdisk. Ah! Okay, now partition one is offset by a few (disk) sectors. Now run Disk Utility again, and erase the volume as MS-DOS...meh. Well, it appears to be working, but very slowly, with that wonderful 'CLIK' noise of 'bad read/write.' Hm! On the other hand, the log window of Disk Utility hands me all sorts of wonderful info, like the backup sector location, sector numbers...the stuff that the code earlier was hardcoding.
Gack. I wonder if the iPod firmware stuff is going to assume disk sectors and/or nuke all this.
Meh. Gotta sleep. More later.
Idly trolling the net, however, I found this - where someone else with a similar problem apparently had the same thought! And unlike me, knew enough C to make a dangerous stab at it!
Intrigued, I wished to subscribe to his newsletter. Er, no. That is to say, I downloaded the code, untarred it onto my Mac, and had a look. Apparently, his solution to the 'I'm not sure how the heck bad sector tables or their equivalent work in HFS+' problem that forced me to give up was much more elegant - he simply formatted his iPod as a FAT32 volume. The iPod doesn't care, and will function as either on a Macintosh - and there's much more info available on FAT32 FAT muckery. In particular, rather than telling the drive that the sector was bad, this gent proposed to tell the FAT itself that the sector was no good.
The downsides of this approach are obvious from reading his page - the modified FAT, with the bad sectors marked, wouldn't survive a reformatting/re-iPod-firmwaring of the device, since those would write a new FAT to the disk. The page itself came about because after several months of using his 'hacked' 'pod, he himself had to flash it, and was forced to go revisit his quick hack to determine what it did and how so that he could do it again. This time around, he figured why not document it (sorta) on the web?
Anyway, I find myself with a few terminal windows open, mucking around inside typedefs and big-endian->little-endian macros, blithely reading and writing raw bytes from and to li'l Gir. Haven't fixed anything yet, but I'm learning. After having the code tell me that my FAT copies weren't in agreement, and then refuse to make any changes, I've discovered that the author (apparently) hardcoded in certain values like the number of sectors per FAT and the number of FAT copies - values which differ depending on the size of the disk, and Gir isn't the same size as his 'pod, apparently.
While there's code that's ostensibly supposed to figure out some of this info by dumping it from the MBR/boot sector, it's not in the same executable, so i can't be sure that things which share var names across the executables really mean the same thing...esp. when they're derived in the first case from data off the drive, and simply declared for use in a different manner in the second. Replacing the values in the hardcoded versions with information taken from my runs of the dump code doesn't seem to work - it changes what sectors the code attempts to muck with, but they still come up as inconsistent (the FAT and backup FAT, that is) which prevents attempts to change it.
Now I just noticed that the dump code is warning that there's a sector 0 read error...and while the second FAT copy typedef's hexdump has interesting stuff in it, including what looks like packed bytes and a 'NON-BOOTABLE DISK' text message, the first FAT copy seems to be all zeros.
Hm.
Oh, suck. If sector 0 is tanked, then the primary FAT isn't working?
Hm, but Disk Utility claims it managed to properly initialize the volume, which doesn't sound like the primary FAT is corrupted. More likely, the place it's looking for the FAT is wrong, given the disk size difference.
ARGH. I don't know enough yet. More research.
If the email message was forwarded, and it asks you to either send information or money somewhere OR to send out email or postal mail, check carefully on its pedigree. Note that I don't just mean be sure your friend sent it. They probably did. But check that the 'story' in the email is true and verifiable. If it's for a cause that's in the news, verify that the organization making the appeal sent it. If it is for an 'unnamed project' or 'unnamed child' or sympathy-jerking cause, be very suspicious.
Here's a good place to look: Hoaxbusters at CIAC - the U.S. Department of Energy's Computer Incident Advisory Capability website. Or just Google for some representative terms from the email - like, say "sick child wish chain letter internet" and see what you get. Dollars to doughnuts you get verbatim text from your email in the results - and now you know.
You're not done, though. Responsibility demands you consider your actions carefully. Blowing a 'reply-all' out to every email on the damn email would really only compound the problem. Better to pick the last few people, the one(s) you know personally, and write them an email with a *different* subject, alerting them to the chain letter (preferably with a link to an internet citation of its hoax status) and a quick suggestion (politely phrased, it's not THEIR fault, remember?) that they check first in future.
Hopefully, next time, you won't get the email.
And we'll all save time.
Thanks. This concludes this particular Nannygram.
Upon investigation, we found that in all cases, the messages were showing up in mailman's list archives, indicating that they had been received by postfix on the list server, passed to mailman, processed by mailman, and added to the list archives in a timely fashion. So the hangup was somewhere either in mailman's outbound message processing or afterwards.
Looking in mailman's queue, we found that it was running and contained only seven or eight messages. Nope.
Moving along, we checked postfix on the list server. Whoa. Nine thousand messages. Using
mailqshowed us all of them. Although a couple of thousand of them were in various states which indicated 'normal' deferral, such as 'Connection timed out' or 'Host not found' or 'Mailbox full', several thousand had the following error message:
status=deferred (delivery temporarily suspended: unknown mail transport error)
Um. WTF?
Checked the process table. Postfix had multiple smtp agents running. Tailed the logs, and things were happening...but as we watched, messages flew by, deferring with that same error. We reloaded postfix, and confirmed that it reloaded its configuration - but no difference, same behavior. Okay, so postfix was functional, as far as it could tell us, but something was definitely wrong. Corrupt files? Perhaps.
We stopped postfix and had it run a check. It helpfully found the five or six known corrupt message files that it had already quarantined, announced that all of its permissions were correct, and returned. We started it back up. This time, mail began to flow properly. Looking in the mailqueue, however, many of the messages were still marked as deferred with that error. We manually flushed the queue (postqueue -f) and, watching the logs, saw them flood back into active status. A few minutes later, our list mail folders filled with backlogged list mail.
A few hours later, it happened again.
Then a couple of hours later, again.
I investigated further. Looking in the logs, I found the moment the first message came back as deferred. A few seconds prior to this, I found a warning:
postfix/qmgr 9083: warning: premature end-of-input on private/smtp socket while reading input attribute name
Going back to the prior incidents, I found similar but not identical errors a few seconds prior to them as well. In one case, I found a panic:myfree error warning of corrupt or unallocated memory. In each incident, however, what apparently happened was that one of the smtp agents had crashed and the qmgr (queue manager) had caught wind of that fact. The process ID number given in the warning message by the qmgr (9083 in the example above) was the proc number of the agent that had died, so by backtracking in the logs I could figure out which message ID the agent in question had been handling when it stopped reporting.
The problem was that when I restarted postfix, I found that in every case, those same messages that were in process during the crashes seemed to go out just fine. So it wasn't a case of a malformed or corrupt message, which is what had always been the cause of such behavior in my experience with postfix up to this point.
At this point, I had a problem. Well, two problems, potentially three. Problem one: an unknown factor was causing the intermittent but frequent crash of smtp agents on my lists machine. This by itself was merely annoying, given that postfix's master process was properly figuring this out, reporting it, and spawning new agents. However, it was rendered critical by problem two: Postfix, for unknown reasons of its own, upon discovering the crash of one of these agents, would start marking large numbers of messages in the queue as 'undeliverable.' Potential problem three: As far as I could tell, there wasn't any attempt later on to revisit those messages marked as such.
I'm not going to even try to address why postfix was behaving that way. Suffice to say it pissed me off immensely.
I attempted the almighty Google search. I found a couple of hits on the errors messages I was receiving. The answers, from the Postfix coxers themselves, seemed to fall into two categories. One: this kind of problem could be introduced when using mail scanner or routing scripts that directly touched the mail queue. Okay, not me; I'm using mailman but only in a manner prescribed, i.e. postfix delivers to mailman and mailman uses postfix's available tools to inject messages back into the queue. Two: Memory or hard drive troubles on the server. Okay, fair enough, I'll check.
Ran full hard drive checks. Nothing. Just for giggles, took the system drive of the server out of mirror mode, ran it on each of the mirrors individually. Showed the problem both times. Put it back into (PERC hardware) mirror. I consider that to come as close as I can to eliminating hard drive corruption as a cause. To be safe, I disabled swap on the machine in case there was a problem with the swapfile, and ran it on physical RAM. Nope, problem still showed up. Okay. Problem isn't in swap disk.
What about RAM? Downed the server, and ram memtest86+. Note that this program doesn't really like PowerEdge 2650s - it shows a constant error on one address in high RAM which I suspect is used by the ServerWorks management hardware. Other than that...nothing, really. Decided to be safe. Ordered new DIMMs from Dell. Installed. Nope. Same problem. I consider that, again, to essentially rule out the 'bad RAM' problem.
Finally, I was forced to do what I didn't want to do. I upgraded postfix past the official Red Hat release, to 2.2.5.
It's been 12 hours, and it hasn't crashed an agent once, nor marked any of the mail as temporarily suspended.
I'm a little annoyed about this, and here's why.
First of all, the postfix crew were adamant that this wasn't postfix's fault. I understand their bias, but, um, heh. I'll let that one go. I wasn't using the 'current' version anyway.
What really pisses me off: Why the hell did postfix 'handle' a crashed smtp agent like that anyway? How am I to know it won't *still* do that? Please don't tell me to go read the code. I'm an op. I'm not a software engineer. If the only way I can be reasonably sure your software will work the way it should is to read the code, because testing a prior version has resulted in behavior I can't explain and this behavior hasn't been addressed in docs, then there's a problem. Back to the point: Why does a crashed smtp agent result in messages in the queue being flagged as undeliverable? The entire reason for there being a watchdog process and multiple smtp agents (well, one reason) is so that one agent dying shouldn't be able to tank mail delivery as a whole.
Anyway, just another day as an Op.
Allow me to state for the record: I think DRM is a bad thing. Especially when applied to anything in my data. Especially when applied to anything without my permission. But in this case, take a careful look at what's going on.
Apple is a company that has always made its money on hardware. Sure, people may buy the hardware because of the OS - but the margins are on the hardware, and the higher-end, the better. While this may change in the future, and may be changing as I write this, it's still true. If it weren't true, I don't think Apple would be bothering to produce Intel-based Macs - they'd be producing Mac OS X for your PC. The problem is that that's what Steve Jobs' last company did - remember NeXTStep? Sure you do. Look what happened to them? They had to get bought out by a company that made hardware (Apple) because they fell into the ground trying to survive only making software.
No, I'm not claiming the companies are in similar sit