Virtual Illusions

In case you were wondering the offset command in postgres does not operate in constant time.

For example:
select * from table limit 100 offset 1000;

is fast (about 75 ms). While:
select * from table limit 100 offset 11000000;

takes just shy of 3 minutes.

The reason for this is that postgres actually retrieves all 11,000,100 rows and just ignores the first 11,000,000. The upshot of this is if you’d like to page through 11.000,000 records it will start out nice and fast. You do the calculations and you figure it’ll take about 90 minutes. But when you get back into work the next day it is still running. In fact the average speed is about 200 records a second and dropping.

If you need the offset functionality on large dataset you want to give the dataset its own indexed sequential id and use that in the where clause to do limit/offset stuff.

For example:
select * from table where id > 11000000 and id < 11000100;

That query always takes double digit ms instead of growing as you get further into the dataset. You’re kind of S.O.L. if you need to sort the whole dataset. Best bet would be to copy the table into a temp table in the right order and use the above technique, buy come one do you really need to sort 14,000,000 records. It’ll take forever.

In a similar vein, if you flush your cache after each record, your process will take much longer and if you load all 14,000,000 records into memory it’ll swell memory and generally slow things down. Just thought I’d share.

This post brought to you by fun with large datasets and the leter O.

Tech

So I’ve cracked the seal on the Nephilim Song manuscript and begun reworking it based on what I learned during the last two campaigns. The changes I’m proposing are somewhat radical. I was hoping to do a quick edit and fire up a new campaign, but I’m looking at changing several fundamental things about how powers work.

My thought is to change everything to a maneuver. A character can use some number of maneuvers in a given round (probably soul +1). I’ll probably take the levels away from maneuvers, though certain maneuvers may count as multiple maneuvers to activate or stack with themselves for increased benefit. Probably also have some maneuvers with prerequisites.

Changed the way armor worked, multiplied the relevant def times hits to provide a pool of armor points that avoids the division in combat. So that mental and physical combat contribute to each other I’m instituting a blow through system where for every 10 points of armor damage, 1 point of real hit damage is done to the character.

Probably doing other stuff but that probably are the biggest things.

Gaming

So on April 10th, I got frustrated with the fact that my MythTV box decided to do its disk system cough again and I said screw it time to throw money at the problem. I decided to get a new computer and make this problem go away. And so I went to my traditional supplier CyberPower and ordered myself an AMD 3200 with dual 200Gb Sata drives.

And I dutifully watched as it went from non-existent to built to tested to shipped. I watched as Fedex moved it moved from Pomona, CA to Bloomington,CA then across the country in two days to Hagerstown, MD and finally to the local facility in Beltsville, MD. It arrived there at 5am on Saturday April, 22nd. And it sat there.

No delivery attempt or other scan that day, or Sunday. I figured, weekend, no delivery, how annoying. Then no delivery attempt on Monday. Now I’m vexed. I call them on Sunday and then spend the next three days arguing with various employees of Fedex, until they admit that it isn’t on any truck or in the Beltsville facility. Luckily they’ll keep monitoring for 14 days. Oh, joy.

At this point I contact CyberPower assuming they will start building me a new computer and ship it out to me at the earliest possible date, while they are following up with Fedex and filing a claim and all that crap. That would have been the correct thing to do to retain a customer. Instead they told me the claim should be processed sometime next week and I should call back then to figure what to do. They made it quite clear that my satisfaction as a customer was not at all an interest of theirs.

Almost every computer I’ve bought since I’ve been buying computers has been bought from them. Lets call it about 15 machines over the years. All it would have taken was for them to have started building me a new computer while the followed up on the claim process with Fedex and I would have been happy. Instead they really went out of their way to antagonize me.

And so now I have no computer and the bastards have my $800. Now I’m both in the market for a new vendor, (and it’s a pain to find a good cheap one with the degree of customization I like from a vendor) and I need to somehow get my money out of these people. I suspect I’ll need to contract the services of a lawyer to write a letter on my behalf.

More the point I’ve been without home theater PC for like 3 weeks now. My 106″ screen hasn’t gotten its proper use.

The important thing to remember is don’t buy from CyberPower and if you can avoid it don’t use Fedex as a shipper. Their reliability and customer service leave something to be desired.

Justice, Tech

So on the way home from work today I caught an NPR bit on fact that some latino musicians have translated the national anthem into Spanish. The general subtext of the piece which we are supposed to assume without question is that this is somehow disrespectful. I have never understood that way of thinking. The anthem, like the flag, is nothing but a symbol and the only power or place it has is its ability to convey an idea.

Incorporating the symbol into a new version assists it having meaning for more people, it isn’t disrespectful, quite the opposite, it gives it new life and power. Jimi Hendrix gave it new power when he remade it in the image of his generation. It is an honorific to use the symbol in a new way.

I just don’t understand the people who believe symbols shouldn’t be used. Up on a shelf they have no power and serve no function it is only with their use and periodic re-invention that they continue to have meaning to actual people rather than staid institutions.

News, Politics

Since I’ve gone to the effort of posting I should probably do an update dump sort of thing that I seem to do occasionally as if someone is listening. On the other hand more people might listen if I’d do them the courtesy for updating regularly.

Since in general I only post when stressed, upset, alienated or in some other way need a release one could interpret my silence for the last month as “wow it’s nice to know Todd is doing so well”. This would probably be the incorrect way of interpreting the event.

So on St. Patrick’s Day I turned 35. I didn’t really have the emotional resources to deal with it. I’ve been running on low if not empty for a long while now. I managed to arrange a work happy hour down at the Bricks because the idea of being alone was just too depressing. It was a decent enough time, even got Doug out (and the bastard bought me a lemon drop in the name of tradition). Got way too drunk. And there’s photographic evidence, very disturbing.

Had my party on the 25th and everything went according to plan. It was an enjoyable enough time all said but for me personally I contained a number of disappointments. In order:

A few hours before the beginning the party my friend of 5+ years who I was kissing in January and decided we couldn’t date (See February Post) (Here after referred to as M) emailed me to say she couldn’t make it with no elaboration. You will recall the tangible, measurable version of her promise to continue to be my friend took the form of promising to come to my party. Her choice not to come in all likelihood spells the practical end of our friendship. Not a big fight end, but an allowing it die on the vine. Our friends in common in the local area have dwindled to near nothing. Since she won’t accept invitations to talk to her in the hopes of normalizing our friendship, and she won’t come to parties as she’s done since I’ve known her, I can’t guess when or if I’ll ever see her again. I sort of think I should write a goodbye letter, but so far I’ve resisted. I did respond to her email that I was disappointed that she couldn’t make it. I wanted to write that I was disappointed that she chose not to come, but that still seemed like a tactical error.

At the party itself this was certainly the first party with no representation from either The Motley Fool era of my life or the AU era of my life (discounting Bo). I seriously considered ending the institution, but I don’t think I have the heart for it. The whole thing made me feel old, tired and alone.

I don’t want to take away from those who did show up we had a lovely time but the above meta-issues had me too close to collapse throughout the whole night. I stayed remarkably sober, because I lacked the strength to be drunk, it probably would have been ugly.

Bo left for El Salvador Sunday morning at like 5am or something. I should email him about the hot water heater exploding. Not sure that it significantly affects him but it seems appropriate.

Only other interesting thing that I can recall is that my cherry blossom tree (or close enough) started dropping its petals a couple of days ago. My street is lined with them and with them all falling it has a very snowfall like effect.

Life

I was playing some WoW last night and decided I needed a beer. So I head down to the basement where all the good beer is, and as I approach the bottom of the stairs I hear the telltale sounds of running water. Then as my I put my foot down on the floor it makes a squishing sound.

Turns out my hot water heater exploded. I spent the rest of Friday night moving everything that one person could move out of the basement. Slept for three hours at got up before 6am. Picked up some day laborers and spend the morning destroying my pride and joy, the basement bar. It only took 4 hours to completely destroy it.

Carpet ripped up and thrown out the back, all the furniture up on stilts with bunches of fans doing the air circulation thing. Plus no hot water until Monday (assuming no complications, heh). Oh, and I’m leaving town on Tuesday and need to show up at work sometime Monday.

My patience is being sorely tried.

On the upside 24 hours later I’ve finally got that beer and my mother is thrilled, she’s wanted me to get new carpet for years.

Life