A better version of Silent Night

TWAS THE NIGHT BEFORE CHRISTMAS, HE LIVED ALL ALONE,

IN A ONE BEDROOM HOUSE MADE OF PLASTER AND STONE.

I HAD COME DOWN THE CHIMNEY WITH PRESENTS TO GIVE,

AND TO SEE JUST WHO IN THIS HOME DID LIVE.

I LOOKED ALL ABOUT, A STRANGE SIGHT I DID SEE,

NO TINSEL, NO PRESENTS, NOT EVEN A TREE.

NO STOCKING BY MANTLE, JUST BOOTS FILLED WITH SAND,

ON THE WALL HUNG PICTURES OF FAR DISTANT LANDS.

WITH MEDALS AND BADGES, AWARDS OF ALL KINDS,

A SOBER THOUGHT CAME THROUGH MY MIND.

FOR THIS HOUSE WAS DIFFERENT, IT WAS DARK AND DREARY,

I FOUND THE HOME OF A SOLDIER, ONCE I COULD SEE CLEARLY.

THE SOLDIER LAY SLEEPING, SILENT, ALONE,

CURLED UP ON THE FLOOR IN THIS ONE BEDROOM HOME.

THE FACE WAS SO GENTLE, THE ROOM IN SUCH DISORDER,

NOT HOW I PICTURED A UNITED STATES SOLDIER.

WAS THIS THE HERO OF WHOM I’D JUST READ?

CURLED UP ON A PONCHO, THE FLOOR FOR A BED?

I REALIZED THE FAMILIES THAT I SAW THIS NIGHT,

OWED THEIR LIVES TO THESE SOLDIERS WHO WERE WILLING TO FIGHT.

SOON ROUND THE WORLD, THE CHILDREN WOULD PLAY,

AND GROWNUPS WOULD CELEBRATE A BRIGHT CHRISTMAS DAY.

THEY ALL ENJOYED FREEDOM EACH MONTH OF THE YEAR,

BECAUSE OF THE SOLDIERS, LIKE THE ONE LYING HERE.

I COULDN’T HELP WONDER HOW MANY LAY ALONE,

ON A COLD CHRISTMAS EVE IN A LAND FAR FROM HOME.

THE VERY THOUGHT BROUGHT A TEAR TO MY EYE,

I DROPPED TO MY KNEES AND STARTED TO CRY;

THE SOLDIER AWAKENED AND I HEARD A ROUGH VOICE,

“SANTA DON’T CRY, THIS LIFE IS MY CHOICE;

I FIGHT FOR FREEDOM, I DON’T ASK FOR MORE,

MY LIFE IS MY GOD, MY COUNTRY, MY CORPS.”

THE SOLDIER ROLLED OVER AND DRIFTED TO SLEEP,

I COULDN’T CONTROL IT, I CONTINUED TO WEEP.

I KEPT WATCH FOR HOURS, SO SILENT AND STILL

AND WE BOTH SHIVERED FROM THE COLD NIGHT’S CHILL.

I DIDN’T WANT TO LEAVE ON THAT COLD, DARK, NIGHT,

THIS GUARDIAN OF HONOR SO WILLING TO FIGHT.

THEN THE SOLDIER ROLLED OVER, WITH A VOICE SOFT AND PURE,

WHISPERED, “CARRY ON SANTA, IT’S CHRISTMAS DAY, ALL IS SECURE.”

ONE LOOK AT MY WATCH, AND I KNEW HE WAS RIGHT.

“MERRY CHRISTMAS MY FRIEND, AND TO ALL A GOOD NIGHT.”

Ubuntu and compiz

Suck it windows / mac users!

Linux krogebry-laptop 2.6.32-25-generic #45-Ubuntu SMP Sat Oct 16 19:48:22 UTC 2010 i686 GNU/Linux

krogebry@krogebry-laptop:~$ cat /proc/cpuinfo

processor : 0

vendor_id : GenuineIntel

cpu family : 6

model : 23

model name : Intel(R) Core(TM)2 Duo CPU T6400 @ 2.00GHz

stepping : 10

cpu MHz : 1200.000

cache size : 2048 KB

physical id : 0

siblings : 2

core id : 0

cpu cores : 2

apicid : 0

initial apicid : 0

fdiv_bug : no

hlt_bug : no

f00f_bug : no

coma_bug : no

fpu : yes

fpu_exception : yes

cpuid level : 13

wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm

bogomips : 3990.13

clflush size : 64

cache_alignment : 64

address sizes : 36 bits physical, 48 bits virtual

power management:

processor : 1

vendor_id : GenuineIntel

cpu family : 6

model : 23

model name : Intel(R) Core(TM)2 Duo CPU T6400 @ 2.00GHz

stepping : 10

cpu MHz : 1200.000

cache size : 2048 KB

physical id : 0

siblings : 2

core id : 1

cpu cores : 2

apicid : 1

initial apicid : 1

fdiv_bug : no

hlt_bug : no

f00f_bug : no

coma_bug : no

fpu : yes

fpu_exception : yes

cpuid level : 13

wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm

bogomips : 3990.88

clflush size : 64

cache_alignment : 64

address sizes : 36 bits physical, 48 bits virtual

power management:

00:02.1 Display controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 07)

I smell another tribunal coming!

This weeks topics for discussion:

Education is a major social institution and agent of socialization. As such, various interest groups try to influence what is taught in schools, or at least they criticize what is taught, especially in public schools. With that in mind, let's discuss why it is that education is seen as a contested social institution. Use the sociological terms and concepts in the current and past chapters and the sociological theories to analyze the hypothesis that education performs important functions for the stability of society.

Political and moral leaders frequently are heard making statements about family values. Using the sociological terms and concepts in the current and past chapters and the sociological theories to analyze the question: "should government act to promote the nuclear family model or should it give equal support to all types of familes, including single-parent households and families headed by gay and lesbian parents?"

This is gonna be FUN!!

Physics question.

I'm wondering if some of you Twilight fans can help me with something. If Edward is a vampire, and vampire's don't have reflections ( one of the few facts that hack of an author didn't butcher ), then what would his pornoscanner image look like? Would he be unscanable? Would the TSA be required to provide a special vampire agent to violate his person administer a freedom pat?

My new favorite game.

My new favorite game: finding the oldest nuggets of bug documentation on DeVry's student portal site.

"11/01/00 BugTar 1614 tomf" is the current winner!!

That's right, a bug was fixed in 2000. For some reason beyond me the developers are actually leaving javascript comments in the HTML stream. They don't bother putting the JS/CSS code on some kind of static content server, no, instead they have large chunks of very old javascript inline with the html, which is also very old.

The more I found out about this school the more I want to just cry.

DeVry student portal

I found this gem in the HTML source of the email form:

//11/21/00  bugtar 991
if (document.SendEmail.ATTACHMENT.value != "") {
  document.StatusUser.submit();
}
//11/21/00  end
selectAllOptions(document.SendEmail.EMAIL, 0);
document.SendEmail.submit();

Wow, is that really 2000-11-21? Seriously, 10 f'ing years?!? ARE YOU KIDDING ME?!?

More fun with DeVry.

Get this, in DeVry's infinite stupidity they actually thought that using your birth date as your password was s good idea. Well, it's actually worse then that. They use the form mmyy, so as long as someone knows your d-number, approximate age and what month you were born, they could get into your account.

So let's see here, in every class they pass around a piece of paper with everyone's d-numbers, so that's not a problem. After you have a camera pic of everyone's d-number it's a matter of guessing their age which is about 10 years ( between 20-30 ), and a month, which is any of 12 possibilities. I know their site doesn't have a max attempt hook on it, so it's a matter of brute force.

The sad part is that they don't make any attempt to make it harder. They don't force you to change your password when you first login, and most of their systems don't use any kind of central auth mech. If a persons birth day is posted anywhere where someone else at school can see it ( facebook friends? ), then you're screwed.

I really have to wonder how DeVry can square itself with this and still call itself a viable source of higher education.

Azure

I just told someone that I thought Azure was awesome. I feel a little dirty now.

Social stratification

I wanted to share an absolutely fascinating conversation I just had with two rather high level Microsoft folks this morning. As it turns out Microsoft ( I’m sure everyone else is working on the same thing ) is developing software and infrastructure to allow the consumers of the Internet to be the brokers of their own e-DNA. What this means is that you can tell everyone out there ( google, facebook, you name it ) that they can have your information - your electronic DNA - for a price which you decide.

This is relevant to the concept of social stratification in so much as this has the potential of creating tiered levels of economic gain. For example, if I am a very popular person and I have several hundred followers on twitter, am an active shopper on froogle.com, and participate in community organizations, then my social data might be worth more then someone that has a relatively low “footprint” in the community. What this could mean is that those who protest about Internet privacy, and fight to keep themselves private and “under the radar” will end up loosing out. However, those who embrace openness, and encourage social interactions will end up gaining more.

What would your e-DNA cost you?

Ruby love

Ohh ruby how I miss you. I never realized how much I truly loved you until you were ripped from my loving arms. How I long to once again snuggle with you and MongoDB in a hot, naughty threesome of pure joy.

I remember when we were young, when the days were long and we sat on my workstation and you allowed me to poke at your insidey parts. Do you remember that warm summer afternoon when you let me write the first few iterations of Heylu? Ohh how I long for those days again. Days of field enforcement, key :love,Integer, { :default: 10000000000 }. Ohh how I miss you!

Another rant about DeVry

It is shocking to me that people in the CIS graduation program can't figure this out. If you don't believe me, go grab a copy of firebug, or use chrome's element inspector and find out for yourself. EVERYTHING I know about this issue was gathered from using firebug and looking at the HTTP headers. If that's illegal, then so is every single web browser since the days of mosaic.

I personally agree with the idea of regulation on the Internet, however, I would put an emphasis on corporate responsibility. In the case of DeVry, I would charge $1000 per violation of their own student privacy policies, and leave the option open for a personal lawsuit. Enforcing a sanction on the person that finds the problem makes absolutely no sense to me, but does act as a reminder for everyone out there to keep things silent; it makes more business sense to eliminate the person screaming about the problem then actually fix the problem.

How does it feel to know that your information is exposed to everyone in the world and there isn't a dam thing DeVry is going to do about it but punish the people that expose the problem? Feel safe? You shouldn't, because you aren't. Remember, the only laws that exist are those that can be enforced, if you don't know the rules are being broken, how are you going to enforce them? Do you really think anyone is checking the server log files? They can't even enable SSL, I seriously doubt they're doing anything about anything else security related.

One final note about the subject: which would you rather have on your side, a Jedi knight with next to no social skills, or a Sith lord hell bent on the destruction of the world? Think about the next time you get the urge to go tattle to the dean. ;)

Traffic predictor

Demo

The idea with this little project is to predict what the traffic for a given road segment is going to be.

I'm scrapping the data from WSDOT's rss feed into a MongoDB collection, then aggregating the data to make things faster.

The problem that I'm having now is determining the color for any given segment, more on that later.

Lots of work left to do on this little project, but I'm really enjoying myself on this one. It's quite a fun little puzzle.

Why I love Zenoss

In the kitchen of the sysadmin you'll generally see some mix of tools like Nagios, Cacti, collectd, maybe even some custom stuff in there as well. In most cases people are looking to fulfill two core requirements: monitoring of services, and trend analytics.

In the past I've found that Nagios + Cacti was a fantastic mix to satisfy these requirements, however, I have recently found that Zenoss to be a much more satisfying tool.

The problem with Nagios is that you end up with lots of configuration files. Granted, most good admins will have these organized in a way that makes sense which ultimately makes them easy to manage and maintain. However, with Zenoss you have no configuration files ( at least not in the sense of host/services/etc... ). This is nice since I don't have to restart the service if I add a host, nor do I end up having to edit anything on the server itself. Tracking changes via svn/git is nice and all, but having all of the change log information in the interface is even better.

As for Cacti, I've found it to be rather prickly to get setup, and doesn't seem to work all that well in large scale environments.

Zenoss combines both of these tools into one tool and adds some very nice polish to the entire process. For example, if I want to ensure that Zenoss is monitoring any service on any host that matches: /^thin.*[0-9]{4}$/ ( thin server port 6900 ), I can add a service rule. This service rule then watches the process table on each host and will 'catch' any process matching my regex.

This has several benefits:
* The monitoring of the process is automatically picked up, if it crashes, an alert will be sent out.
* Along with the state monitoring, Zenoss will also start profiling this process as far as memory and cpu usage.
* I created one object, and that object was automatically propagated to all hosts.

The last point there is the most important. If I had a mix of hosts and services I can still get the trending and monitoring regardless of the role for any given host.

For example, let's say you have a mix of memcache instances, some that run on m1.small instances in AWS, and some that run as bare metal in a datacenter. In this case, as long as Zenoss can touch the snmp port on all instances, it can watch for any service matching something like /^memcache/. Again, regardless of the role for the given memcache instance, it'll be picked up and monitored automatically without you having to configure anything beyond the initial service.

Once I have this item configured and running, I can 'lock' the service, and any changes made from that point on are tracked by the system. So, if someone ( perhaps a Jr. Admin ) goes and fat fingers my regex, I'll know who did it and when. This is slightly more convenient then having to dig through commit logs.

Nginx + Sinatra + MongoDB

I created a document to help explain the how and why of my web app setup.

Document

I use this in production, so I guess you could say that I eat my own dog food on this one.

Gotta love optParse

I use a very simple deployment system which goes like this:

* export code base at HEAD
* tar export
* scp tar to host server(s)
* create directory for this version of the payload
* unlink 'current' directory ( /var/www/[project]/current )
* unspool tarball into newly created revision directory
* link 'current' to newly created directory

In this way I can roll the entire code base back using a simple unlink/link-to-old-build-id command. This keeps my environments nice and tidy, and safe from evil things like version conflicts.

To this day I don't know why anyone would actually think that 'svn update' is an OK thing to do in a live production environment. Ehh...guess that's just me being all anal again.

My favorite part about this is the deployment script, dead simple, and I did the opsParse thing to remind me of how it all works:

krogebry@krogebry-desktop:~/aws/deployment$ ./deploy.rb -h
Usage: deploy.rb --project [project] --environment [stage|prod] --verbose
-v, --verbose Run verbosely
-p, --project PROJECT Project name
-e, --environment ENVIRONMENT Environment name ( stage | prod )
-h, --help Show this message

EBS logging VS Scribe...

I was running some numbers trying to figure out which would be the better solution between using EBS volumes on the web front ends for logging versus using something like scribe. In either case the log data would end up in a database for analytics.

In my mind the choice is pretty dead simple, however, I've recently run into a case where and admin had put together the most bizarre little hack for dealing with logging data. They were writing the log data to an attached volume, then reading the data off the volume with a series of nearly indecipherable spaghetti code written entirely in bash. This meant that the web front end were horribly bloated with all of this extra work, not to mention the sheer fragility of it all.

At any rate, I was thinking about how much this is costing them to do as opposed to doing something more elegant like using Scribe...

Assuming 10 web servers taking an average rate of 5 hits per second, and assuming that each hit is logged to an EBS volume, then read from the same volume later on into a database:

5 hps * 10 servers = 50 hps
50 writes and 50 reads = 100 total IO operations per second.
360,000 ops/hr = 8,640,000 ops/day

Amazon charges $0.10 per 1M operations, so that works out to $0.664 / day = $25.92 per month.

So, at the end of the day they end up spending a very small amount for this. Even if you piled the expense of the web server instances themselves it probably won't add up to much more then $50/month. That's not bad at all, but it's still %50/month they don't have to be spending when they could be getting a better service for free. Call me overly pedantic, but I just hate spending money I don't have to, especially when I have a lot riding on a newly funded startup. Every penny counts right?

New updates.

I just pushed a new build up to production. This build shows off some new features, and begins to show some of the onclick actions for the name plates.

More tanking effectiveness.

Now with event goodness!

The little red x is a death event. Eventually you'll be able to hover over it and see who died and when.

The blue horizontal lines are bone spikes. I haven't quite worked out all of the details on this yet, but it's getting closer!

RR: Tanking effectiveness

Tanking Effectiveness is basically a comparison between damage input on the tanks, and healing output on the tanks from the healers.

There are 20 ticks on the page that represent the 20 samples taken from this encounter. The white dots on the top show the total amount of healing done to the tanks by the dedicated healers. The red dots on the bottom show the amount of damage taken by the tanks from all sources.

There are 3 things missing from this graph that I am working on now:
* Connecting the white dots together.
* Connecting the red dots together.
* Build the difference line.

The difference line shows the difference between the healing output and the tanking input. The thinking here is that you would use the difference line to get a sense of how effective your healers and tanks are. If the line is above the center line, then the output is being mitigated by the healer output ( which is good ), if it's below the line, then the damage input isn't being mitigated by the healing output ( bad ).

Fun with WoWCombatLog and mongoDB

I wrote a very basic parsing utility to take data from a WoWCombatLog.txt file and put it into a mongoDB document. Now I can use mapreduce to create a summary report of the data:

res.drop()
function fMap(){
emit( this.sourceName, {
cnt: 1,
sum: this.amount,
cntNormal: (this.critical == false ? 1 : 0),
sumNormal: (this.critical == false ? this.amount : 0),
cntCritical: (this.critical == true ? 1 : 0),
sumCritical: (this.critical == true ? this.amount : 0)
});
}

function fReduce( key,values ){
var r = { cnt: 0, sum: 0, cntNormal: 0, sumNormal: 0, cntCritical: 0, sumCritical: 0};
for ( var i=0; i r.cnt += values[i].cnt;
r.sum += values[i].sum;

r.cntNormal += values[i].cntNormal;
r.sumNormal += values[i].sumNormal;

r.cntCritical += values[i].cntCritical;
r.sumCritical += values[i].sumCritical;
}
return r;
}

function fFinalize( sourceName,stats ){
stats.avg = ( stats.sum / stats.cnt );
stats.avgNormal = ( stats.sumNormal / stats.cntNormal );
stats.avgCritical = ( stats.sumCritical / stats.cntCritical );
return stats;
}

res = db.raid_data.mapReduce( fMap,fReduce,{
query: { raid_id: "4b7c64da756f4e5799048fca", eventType: /_DAMAGE$/, sourceGUID: /^0x0600/ },
sort: "sourceName",
finalize: fFinalize
});
res.find();

Which produces:

{ "_id" : "Badjoojoo", "value" : { "cnt" : 9, "sum" : 66260, "cntNormal" : 9, "sumNormal" : 66260, "cntCritical" : 0, "sumCritical" : 0, "avg" : 7362.222222222223, "avgNormal" : 7362.222222222223, "avgCritical" : NaN } }
{ "_id" : "Endofire", "value" : { "cnt" : 2618, "sum" : 11396761, "cntNormal" : 1478, "sumNormal" : 3797148, "cntCritical" : 1140, "sumCritical" : 7599613, "avg" : 4353.231856378915, "avgNormal" : 2569.1123139377537, "avgCritical" : 6666.327192982456 } }
{ "_id" : "Guslado", "value" : { "cnt" : 5146, "sum" : 13370673, "cntNormal" : 2921, "sumNormal" : 3690768, "cntCritical" : 2225, "sumCritical" : 9679905, "avg" : 2598.2652545666538, "avgNormal" : 1263.5289284491612, "avgCritical" : 4350.519101123596 } }
{ "_id" : "Jenix", "value" : { "cnt" : 3705, "sum" : 7205291, "cntNormal" : 2376, "sumNormal" : 2719061, "cntCritical" : 1329, "sumCritical" : 4486230, "avg" : 1944.7479082321188, "avgNormal" : 1144.3859427609427, "avgCritical" : 3375.6433408577877 } }
{ "_id" : "Kaymaira", "value" : { "cnt" : 16, "sum" : 91501, "cntNormal" : 13, "sumNormal" : 86764, "cntCritical" : 3, "sumCritical" : 4737, "avg" : 5718.8125, "avgNormal" : 6674.153846153846, "avgCritical" : 1579 } }
{ "_id" : "Mistyayn", "value" : { "cnt" : 2545, "sum" : 10732921, "cntNormal" : 1597, "sumNormal" : 4329100, "cntCritical" : 948, "sumCritical" : 6403821, "avg" : 4217.257760314342, "avgNormal" : 2710.7701941139635, "avgCritical" : 6755.085443037975 } }
{ "_id" : "Rhyea", "value" : { "cnt" : 9568, "sum" : 7576176, "cntNormal" : 7236, "sumNormal" : 5245271, "cntCritical" : 2332, "sumCritical" : 2330905, "avg" : 791.8244147157191, "avgNormal" : 724.8854339414041, "avgCritical" : 999.5304459691253 } }
{ "_id" : "Silverrend", "value" : { "cnt" : 7351, "sum" : 13101369, "cntNormal" : 4238, "sumNormal" : 3451620, "cntCritical" : 3113, "sumCritical" : 9649749, "avg" : 1782.256699768739, "avgNormal" : 814.4454931571496, "avgCritical" : 3099.8230003212334 } }
{ "_id" : "Stabback", "value" : { "cnt" : 7685, "sum" : 12463405, "cntNormal" : 3117, "sumNormal" : 3976589, "cntCritical" : 4568, "sumCritical" : 8486816, "avg" : 1621.783344176968, "avgNormal" : 1275.7744626243182, "avgCritical" : 1857.8844133099824 } }
{ "_id" : "Zuulbash", "value" : { "cnt" : 4501, "sum" : 6388398, "cntNormal" : 3373, "sumNormal" : 3449622, "cntCritical" : 1128, "sumCritical" : 2938776, "avg" : 1419.3285936458565, "avgNormal" : 1022.7162763118886, "avgCritical" : 2605.2978723404253 } }

Awesome. Basically I'm combining a view and a stored procedure, but unlike your standard SQL server, I get to use javascript for the processing language.