Virtual Illusions

So one of my favorite blogs is a blog for GMs called Treasure Tables (http://www.treasuretables.org), it is one of the few blogs which strikes a perfect pose of being player centered and system neutral. It’s generated some great ideas and mad me think about GMing and gaming in general in new ways.

Sadly the gentleman who writes has declared himself burned out and is taking an indefinite hiatus. Wanting to make sure the content didn’t simple disappear I fired up wget and downloaded the site. 800+ posts, 8000+ comments. wget set it up all nice and clean, but there was a problem.

In the scheme of things a corpus that size doesn’t count as large, but it is big enough that the search feature of the site is more than just a mere convenience. Treasure Tables had doled out search to google, and since I was operating under the idea that I’d burn the site to a CD, I certainly couldn’t burn google to disk.

So I was looking for a client side search mechanism. After a bit of searching I found reference to jsFind. There is actually a pretty good write up of its working at Linux Journal (http://www.linuxjournal.com/node/6932/print). You may notice that the article is from 2004. All the resources point to www.elucidsoft.net/projects/jsfind which fell off the Internet in 2005.

Spending far too long searching I couldn’t find an opensource replacement for jsFind or anywhere that jsFind is currently available. There was some perl code on CPAN but not only did it not seem to work for me it was mostly a Btree population system it didn’t seem to have a simple method to call passing in a path to create an index. I was feeling stymied.

So went to the way back machine (http://www.archive.org/index.php) and pulled the original code back from the abyss. I spent Sunday night getting swish-e (http://swish-e.org/) working and writing some python to export the index into the form desired by jsFind. Before going to be I fire up mkIndex.pl which seems to work fine and I have a nice btree.

To be continued.

Tech

This is a contribution to the Internet Knowledge base.

So about a week ago my installation of Squirrelmail (a web based imap client) went screwy. Specifically every time I went the compose page firefox wanted to know what program it should use to run php files or if I’d prefer to save it to disk. At first it was only replies that would cause the problem, then it was all attempts to go to the compose page.

First point is that the ‘how should I run/save this file’ is caused by the script returning a 0 length file when it runs. Since content-type isn’t being set firefox doesn’t know how to handle it. A 0 length file clearly means that the php is throwing an error before spitting out any text and probably any headers. Obviously the cause of the error is not returned to the browser. As it happens neither is it logged in the apache error log. I had to edit php.ini, which on my machine is stored in /etc/php4/apache2/php.ini. I had to set log_errors to On.

Then I got an error I was familiar with in the apache’s errorlog.

Allowed memory size of 8388608 bytes exhausted (tried to allocate 44 bytes)

PHP has a memory ceiling (8mb default), if the script tries to consume more than that PHP bails. I have no idea why the compose page wants that kind of memory, I assume it is loading messages from the inbox for some reason. It would seem unnecessary, but beyond the scope of getting my compose page to work again.

In the php.ini file is a memory_limit entry set to 8M. Changing that value to 16M ups the memory usage to 16mb which is enough so the compose page will function. You need to reload/restart apache after making this change.

I suppose this means I should delete some old/useless email messages, but that seems like a lot of effort and I’d have to make decisions.

Tech

Remember the cyberpowerpc issue (http://nephlm.livejournal.com/84068.html) back in April? I finally got my money out of them. It seems that after the Fedex processed the claim and they promised my refund would be processed the next day they ‘misplaced’ my refund for two weeks until I started calling and yelling at them.

When they tried to tell me no refund was due and I started insisting on talking to a supervisor they started hanging up on me. Four times in a row they hung up on me. Only when I threatened them with lawyers if I was hung up on again did they decide to talk to me.

Finally done with them.

Justice, Life, Tech

This is a double event with matching subject lines.

So the AC at work has been set to 35 Kelvin or something. I seriously considered bringing my winter coat into work today. it wouldn’t have really helped because it does nothing for the numb fingers or the ears/nose freezing off problem. I’m exaggerating but not as much as you think. Yesterday I had to take a walk outside every 30 minutes to an hour to warm up my fingers and allow me to think straight again.

In related news my AC didn’t turn on this season. I tried various things to try and figure out what was going on but no luck. Called a guy in, he took one look at it and said that I was leaking coolant and I was going to have to get a new unit. He could recharge it, but it was just going to leak out again, maybe in a couple of days, maybe a couple of months, but it would just need recharging again. Since this would have been the second time I’d run out of coolant his argument was persuasive.

So yesterday 3 guys came over and ripped out my central air and put a new one in its place. It’s interesting that they don’t actually replace the indoor unit so much as rip out the coils of the existing housing a replace them with new ones which they rip out of a different housing.

The whole thing only took a couple of hours but took place right in the middle of the day so I took it off. In case your wondering, it costs $3200 to get a new central air installed in my townhouse, so if you see me buy me a beer.

Tangentially related (and mentioned because it doesn’t deserve a mention of its own) is the fact that my dryer fried its terminal block and cord. Would have fixed it myself, but 220v and why did it do that? Answer was that someone tried to repair it in the past and it arced. 220v arcing melts wires (and is potentially fatal, don’t muck with 220v).

So if you’re keeping track at home in the last two months I’ve had a hot water heater rupture ruining my basement carpet, central air needed to be replaced, dryer broken and HTPC died. In MMO lingo this would be *burst* damage. Good news is that everything except the HTPC and carpet have been fixed. Got measurements for the carpet on Tuesday. So I could likely start that process soon, but I’m doing a bit of a construction project in the basement and was hoping ot hold out until it is done before traipsing plaster all over the new carpet. I’ll tell you about construction later.

Life, Tech

Attempts to get Knoppmyth installed on the new box have failed so far. Tried installing R5B7 version which installed fine, and would begin recording but about an hour in IVTV would stop passing data and nothing would get it back. Tried a couple of times, and did a search but all found was stuff on the mythtv developer list and I’m not quite ready to begin hacking mythtv itself.

So I burned a copy of R5A30 and I’m trying that, hopefully it has a different version fo IVTV which might not have the same problem. It’s installing as we speak, or really finishing installing. There’s the menu now.

Life, Media, Tech

In case you were wondering the offset command in postgres does not operate in constant time.

For example:
select * from table limit 100 offset 1000;

is fast (about 75 ms). While:
select * from table limit 100 offset 11000000;

takes just shy of 3 minutes.

The reason for this is that postgres actually retrieves all 11,000,100 rows and just ignores the first 11,000,000. The upshot of this is if you’d like to page through 11.000,000 records it will start out nice and fast. You do the calculations and you figure it’ll take about 90 minutes. But when you get back into work the next day it is still running. In fact the average speed is about 200 records a second and dropping.

If you need the offset functionality on large dataset you want to give the dataset its own indexed sequential id and use that in the where clause to do limit/offset stuff.

For example:
select * from table where id > 11000000 and id < 11000100;

That query always takes double digit ms instead of growing as you get further into the dataset. You’re kind of S.O.L. if you need to sort the whole dataset. Best bet would be to copy the table into a temp table in the right order and use the above technique, buy come one do you really need to sort 14,000,000 records. It’ll take forever.

In a similar vein, if you flush your cache after each record, your process will take much longer and if you load all 14,000,000 records into memory it’ll swell memory and generally slow things down. Just thought I’d share.

This post brought to you by fun with large datasets and the leter O.

Tech

So on April 10th, I got frustrated with the fact that my MythTV box decided to do its disk system cough again and I said screw it time to throw money at the problem. I decided to get a new computer and make this problem go away. And so I went to my traditional supplier CyberPower and ordered myself an AMD 3200 with dual 200Gb Sata drives.

And I dutifully watched as it went from non-existent to built to tested to shipped. I watched as Fedex moved it moved from Pomona, CA to Bloomington,CA then across the country in two days to Hagerstown, MD and finally to the local facility in Beltsville, MD. It arrived there at 5am on Saturday April, 22nd. And it sat there.

No delivery attempt or other scan that day, or Sunday. I figured, weekend, no delivery, how annoying. Then no delivery attempt on Monday. Now I’m vexed. I call them on Sunday and then spend the next three days arguing with various employees of Fedex, until they admit that it isn’t on any truck or in the Beltsville facility. Luckily they’ll keep monitoring for 14 days. Oh, joy.

At this point I contact CyberPower assuming they will start building me a new computer and ship it out to me at the earliest possible date, while they are following up with Fedex and filing a claim and all that crap. That would have been the correct thing to do to retain a customer. Instead they told me the claim should be processed sometime next week and I should call back then to figure what to do. They made it quite clear that my satisfaction as a customer was not at all an interest of theirs.

Almost every computer I’ve bought since I’ve been buying computers has been bought from them. Lets call it about 15 machines over the years. All it would have taken was for them to have started building me a new computer while the followed up on the claim process with Fedex and I would have been happy. Instead they really went out of their way to antagonize me.

And so now I have no computer and the bastards have my $800. Now I’m both in the market for a new vendor, (and it’s a pain to find a good cheap one with the degree of customization I like from a vendor) and I need to somehow get my money out of these people. I suspect I’ll need to contract the services of a lawyer to write a letter on my behalf.

More the point I’ve been without home theater PC for like 3 weeks now. My 106″ screen hasn’t gotten its proper use.

The important thing to remember is don’t buy from CyberPower and if you can avoid it don’t use Fedex as a shipper. Their reliability and customer service leave something to be desired.

Justice, Tech

Innate, adj. Natural, inherent - as innate ideas, that is to say, ideas that we are born with, having them previously imparted to us. The doctrine of innate ideas is one of the most admirable faiths of philosophy, being itself an innate idea and therefore inaccessible to disproof [....] Among innate ideas may be mentioned the belief in one’s ability to conduct a newspaper, in the greatness of one’s country, in the superiority of one’s civilization, in the importance one one’s personal affairs and in the interesting nature of one’s diseases.

–Ambrose Bierce, The Devil’s Dictionary, 1906(?)

Tech

Everyone will be happy to know that with 1.5 gigs of ram CoV runs decently again. Yeah.

Gaming, Tech

I’m having two minor issues I hope the UPS man delivered the solution to one.

City of Villains: On my windows machine I can’t seem to play CoV with any pep. During beta I figured, “Hey, its beta,” but the released version seems to have the same problems. The main one is about zone loading taking forever rending the game fairly unfun to play. I’m going to have to guess that it’s the half gig ram that is the problem. Everything else seems up to spec: Athlon 2800+ XP, GeForce 6600 video card, clean OS install, etc.

Well anyway I got a gig of ram to add, Hopefully that will fix the problem.

Xine: On my Home Theater PC since I installed the new version of Knoppmyth, xine has had these little sound skips every few seconds. Last night around 2am it worked fine. I’m pretty sure that was because nothing was being recorded or commercial flagged. This problem doesn’t occur when playing the video with mplayer or ogle but they have other issues that make them less attractive for playing dvds.

CPU is around 20% and spikes to about 50% so it doesn’t seem to be CPU bound. Most likely scenario is that it is IO bound. Something is touching the dvd causing a delay or the dvd is trying to use the hardware en/decoder which is now busy with the recording of TV shows and it causes “issues.”

Not really sure how to tackle this. First step is to make sure xine is using software decoding, there’s probably a switch to force that somewhere. After that I’m not sure. The dvd and the harddrive of on different IDE buses so I’m not sure what else I do about IO bound problem.

It is possible that mythtv is checking to see if someone has changed the dvd every few second and that is causing the problem. I’ll have to make sure that option is disable. I thought that was a front end option and I’ve run tests with the front end disabled, so I’m not hopeful.

Anyway I’m going to see what 1.5 gig does for CoV now.

Media, Tech