phirate

Contains caffeine, guarana and traces of nuts
 

I'm awful, and I take full responsibility.

This post is inspired by, and related to, the following blog posts and news items. if you haven't been keeping up because you have a life outside of reading the internet, you might be confused. http://skepchick.org/2011/12/reddit-makes-me-hate-atheists/
http://kateharding.info/2011/12/29/you-are-awful-too/
http://www.spiegel.de/international/world/0,1518,806386,00.html

To summarise, it won't come as much of a surprise to anyone that people make judgements of others based on their appearance, their gender etc. What might be confusing is the number of people who agree that it is wrong, then promptly turn around and encourage the *victim* to change their behaviour.

The last article I linked above may have resulted in a bit of an eye blink reaction. The first two relate to typical semi-adolescent male behaviour towards females in what they choose to treat as a boys club. The last is about religious fundamentalists harassing women.

Firstly, I would like to say I'm Awful. That is, I may not be awful in the sense outlined by Kate Harding, but I guarantee I am still awful somehow. The chances of me actually not having a harmful opinion are infinitesimally small. Maybe the Dalai Lama might be able to claim that one but honestly he probably wonders sometimes.

Interestingly, in this way I am of agreement with a number of Ultra-Orthodox Jews and commenters on blogs - people are awful, unavoidably awful. They exist, they are practically unstoppable and sometimes they're even me.

Some people don't believe this. We'll call this group the "I'm not awful" group. These people believe that it is possible for humans to be devoid of opinions that are a net social negative. This is a denial of reality as far as I'm concerned, I'm gonna ignore them.

So instead we have lots of people pointing out reality, "I am awful". Why then is there such a divergence in the ways we choose to respond to it? why does the action of the Ultra Orthodox, in which they defend women from the awful predetations of men by … harassing them … seem so off base? surely this is simply realistic?

The answer is in the second part of my statement. I am awful, and I take *full responsibility*. What you see in the harassment, what you see in defenses like "don't reveal your gender online" is a belief that not only are we awful, but that we shouldn't have to take responsibility for that. We should be protected from it, actions that might result in us revealing how awful we are should be avoided at all costs so that at no point could someone come along and point out "hey, you're awful".

This is where I disagree. In my mind, a male who lacks the self control to act in an adult manner around women should be given every opportunity to demonstrate it, so that he can be judged and shamed. There is no other way to learn. Protecting us from our faults only gives them legitimacy. Worse, to be protected leads us to believe that we are in fact not at fault. Denial is something we already excel at, it doesn't need any help. It is only when we see people we respect accepting responsibility for their awful behaviour that we start to believe we may have to do the same. There are very few events in someones life that can change their beliefs like respecting someone, agreeing with an opinion they hold, and then having that person offer a mea culpa regarding that opinion and changing their position.

What the Ultra-Orthodox, and the commenters who say "if you don't want to be commented on based on your gender/appearance, then don't show it" don't understand is that every person who follows their advice generates a feedback effect, whereby those who are awful are never held to account, never forced to confront their beliefs - and so they go on to live their life like that, be awful like that, and hand that awfulness down to their children.

I don't want to live in a world like that, full of weak-willed sheeple. It would be awful. So I have a simple message:

Accept some fucking responsibility chumps, and clean your goddamn act up.

Posted December 30, 2011
// 0 Comments
 

What "HashDoS" is / is not

Lots of people running around in a fuss about the whole HashDoS thing. For those not familiar with the issue, let me explain. No, there is too much. Let me sum up:

Imagine you are being given parcels randomly, each of them numbered between 1 and 10,000, and at any time someone might come and ask for a particular parcel. In the first instance you just stack them all on top of each other, but then when someone asks for #888 and it's at the bottom of the pile it takes a long time to get to it to hand it over. So you think for a bit and you split it up into 10 piles. you put anything numbered 1-1000 in the first pile, 1001-2000 in the second etc.

This makes it quicker to get to a given number. Someone asks you for #888 now, you're only looking through an average of 1/10th of the number of items.

In computing terms, the ten boxes and their number guides are called a "hash function" and it forms the basis of many data types. It turns out there's a bit of a headache case though. Imagine that the person giving you the boxes really wants to make your day suck, and imagine that they also know how you've laid out your boxes. All they need to do is keep giving you fake parcels numbered between 1 and 1,000. Those keep going into the same box and it gets harder and harder for you to get anything out of that bin anymore - eventually the pile grows so high that asking for #888 could take you all day.

Getting back on point, the problem this whole "HashDoS" thing talks about is that when you make a web request, you often do the following:

http://foo.bar.com/query?parcel9682=1&parcel1837=2

Those bits at the end, parcel9682=1 and parcel1837=2, are usually stored in the computer memory in a set of boxes using a hash function. The big fuss is that someone has pointed out that if you choose particular values for those keys (imagine "parcel0001", "parcel0002", "parcel0003" etc) the computer has to work very hard to store and retrieve them - for the same reason you would have above.

This isn't a security issue, it's what we call a force-multiplier. On the internet it's a given that any attacking computer can knock any defending computer off the internet, if the attacking computer has more bandwidth (this is a vast oversimplification but you get the picture, the attacker just stuffs the pipe full). However there are a number of force-multipliers that allow an attacking computer with less bandwidth to overload a defending computer with more. The traditional way of doing this is to have the attacker ask the defender to do something very bandwidth-intensive, or very computationally difficult.

Lets imagine for example that you offered a website that lets people on the internet test prime numbers. I could send you a number, and you'd reply telling me whether it was a prime number or not. It turns out that the effort involved in me asking about any given number is enormously less than the effort required by you to determine the answer. This means I can trivially saturate your computer by asking about lots and lots of big numbers - My work is easy, your work is hard.

Anyway, the interesting bit about HashDoS is not that it's a new thing to discover that you can make web servers work hard. Almost any web application has this problem. The interesting bit is how early in the process this happens. Well engineered web applications attempt to prevent attackers from making expensive calls using various practices and defenses, but the HashDoS problem in many frameworks occurs very early in the processing, often before the web application itself gets involved. As a result many applications that might otherwise not be terribly vulnerable to force-multipliers may be vulnerable to denial of service using this technique.

That said, the number of people who care about this is actually going to be very small. In practice the defense for the purpose of the web framework is trivially simple - a HashDoS attack has to deliver a truly ridiculous number of parcels before it makes any obvious difference to the request processing, something that can be easily limited without any impact on the normal operation of the service. More importantly, the vast majority of web applications are already utterly vulnerable to denial-of-service attacks. There is almost no focus on avoiding force-multipliers for any but the most obvious target sites, and the majority of the sites you use on a daily basis offer attackers force-multipliers that are an order-of-magnitude more effective than a HashDoS.

In other words, this is mostly a storm in a teacup. It will result in various web frameworks getting their shit together on this topic, and it may also result in some languages adopting more defensive measures for their hashing functions (perl, for example, turned out to be harder to attack because it semi-randomises its hashing function every time it starts. Historically however, this has only ever granted a platform a temporary reprieve. Nothing short of a truly secure hash function is actually immune to this attack).

Posted December 28, 2011
// 0 Comments
 

New Pledge

@GarethMP wanted to pledge to Kiwis, and I thought that was kinda cool so I came up with a pledge for him and the rest of the MPs to recite in parliament.

 

As an MP I swear:

To serve all the people of Aotearoa New Zealand as equal individuals, regardless of how much I like or dislike them, how much money they have or do not have, whether they voted for me, live near me, look like me, suffer from a physical or mental illness that dramatically impacts on their lives, or have made and continue to make decisions that are short-sighted.

To not shout, yell, boo, or generally behave like a pre-schooler at any time, most particularly when the school children of our country are visiting parliament to see how their elected representatives and leaders serve their interests.

To retain my faith in the people I represent, ensuring that all information regarding upcoming decisions is available to them so that they can determine for themselves whether their interests are best served by the path I intend to take.

To advocate strongly and effectively for the policy I believe in, and was voted in to achieve - but never to forget that it is the whole of our country and everything in it that is my trust now, not my voters or those policies.

To act with long sight, understanding that short term gain for long term suffering serves only myself and my political interests rather than the people.

To at all times put the maintenance of freedom and democracy as my highest calling, knowing that to fail in this is to act against all of us.

To be courageous when surrounded by fear, cautious in the face of confidence and forgiving when the people are lost to hate.

And finally to remember at all times: each person is a whole person who has lived, loved, felt joy and suffering, and that they deserve my utter respect. Even the media.

Should I fail in this sacred charge, I swear I will wear a neon clown suit for the rest of my days.

 

Honestly, in writing this although it's pretty much just a long-winded joke I admit I wouldn't mind seeing a citizen-created pledge that MPs would be required to recite along with the regular one, particularly focused on behaviour and relevant high ideals. No doubt it'd turn into a morass of political BS tho.

Hell, for all I know we already have one, but if we do then it's not sinking in.

Posted November 30, 2011
// 0 Comments
 

Make your interview work

Hosting a public interview can be a tricky thing, regardless of whether you're doing it online or face to face.

At buzzumi we've done a few of those now and I figured we'd share some of the stuff we've learned as a result. All of this is aimed at the buzzumi video chat platform but most of it is probably relevant no matter how or where you're doing the interview online.

Preparation

I cannot stress this enough - preparation is absolutely key. This is especially true when there is any kind of technology involved.

When we're looking at running an interview, we make sure we get the whole gig set up well in advance - days if possible. We set up the environment (on buzzumi you can use relevant wallpaper with logos etc), we get the timing all nailed down, we organise a general pattern of promotion and most importantly we get the interviewer and the interviewee into the interview space and we make damn sure everything works.

For any kind of web video/audio broadcast there are a number of subtle things you need to make sure you get right:

  1. Ensure you have sufficient reliable bandwidth for the interviewer and interviewee.
  2. Ensure you have a good position for you and the camera, so that you aren't back-lit, don't have your laundry in the background etc.
  3. Make sure you're somewhere *quiet*. There is no noise cancellation that will deal effectively with a cafe or an open plan office.
  4. Ensure your camera and microphone work, that the voice pickup is good etc. If not, go get stuff that works.

When you do this test, it is best to do it at the same time of day, and preferably wearing (roughly) the same clothing as you will for the interview. The position of the sun can dramatically alter lighting conditions, and a bright white shirt has a whole different effect than a dark red one.

And of course on top of this there are all the usual things - make sure you know basically what you're going to be talking about, have backup points in case things are running short, make sure the interviewee has a basic understanding of the topics etc so they can prepare.

In action

With all the preparation, you're now in a good position to make the interview a success, but Murphy is still looking for an excuse to make your life miserable. The most important bit is that the interviewer and interviewee are in the space *early*. Nothing looks less professional than arriving late, and worse if there are any last-second technical problems it's impossible to address them without time.

You also want a private channel for the interviewer, interviewee and technical support. In the event that there are problems, you don't want to be talking about it in the public channel - get it sorted in private.

Lastly, make sure that both the interviewer and interviewee close down everything else on their system prior to the interview (at least everything that makes noise). Preferably do a fresh reboot before start. Constant beeps from various software does nothing to enhance the experience, and some apps may interfere with the camera or audio at the wrong moment.

Ok, all the tech is working, you start your interview, it's going great and then ... someone asks a question in the text chat.

Interaction

Interaction in an online interview, particularly on something like buzzumi that gives participants a text chat space to work with, can be a delicate balance. Much depends on the interviewee and the topic - there is no question that a comment at the wrong time can derail a really interesting answer.

As a general rule of thumb, the interviewee should probably not be responding to participant questions - the interviewer should act as a moderator in a sense, picking up questions from the text chat and posing them to the interviewee at an appropriate moment. Don't get too fixated on this however - some interviewees simply love audience interaction and will be eager to engage directly.

In some cases, the questions can present an interesting opportunity to bring a third participant in for a moment. In one particularly memorable case in a recent interview one of the viewers wanted to ask a question about a newspaper article. Using buzzumi we were able to temporarily enable their camera and let them show the article to everyone. The lack of preparation for a random participant means that it may not work, and the interviewer should keep that in mind and be ready to move on promptly if it fails, but the added dimension it can add to an interview should not be discounted.

Closing up

Once the official interview is over, it is worth considering sticking around for a few minutes to see whether further discussion evolves from the participants. While it's possible to do the old "And now we'll take questions" format, this really isn't necessary in an online, asynchronous environment - people with questions can pose them at any time while others are being discussed.

Once this is over, remember to close up cleanly - close the cameras first and then leave the chat space open for a while to allow any post-event chatter to come to a conclusion (If it doesn't look like it's going to finish you can always leave the chat space open). Once it's done, close the room completely.

Hopefully, at this point you get to feel that lovely glow when something done on-the-fly goes right. Remember to thank your interviewee!

Posted November 26, 2011
// 0 Comments
 

When Skype is just Too Hard

Sometimes we just have to talk face to face. When there's too much emotion involved, or it's basically a chat where the words matter less than your ability to decide whether someone is trustworthy/friendly, the presence of audio and video in realtime can make all the difference.

Skype and other real-time chat solutions went a long way to making this a reality over the internet. The video and audio quality is sufficient to substitute (mostly) for a physical meeting. Unfortunately, sometimes it's just a pain.

If you've used Skype for this kind of problem you're familiar with the Skype Dance. This is the one where you suggest meeting on Skype, the other person asks what your skype ID is, you tell them, they find you and request approval, you authorize them, they call you and finally you get to talk.

That's what happens on a good day anyway. On a bad day, you or they have several skype accounts and are logged into the wrong one, or got the skype ID wrong, or one of you doesn't even have skype installed and the delays just stack up.

The project I'm working on at the moment solves this (and several other) problems, reducing the time-to-face dramatically. What follows is a simple tutorial to get you signed up with Buzzumi and into a One-to-one chat.

The key to the buzzumi way of doing things is that one person - you - is the Host. Only the host needs an account with buzzumi. Guests don't need an account so there's no need for them to sign up or even to have visited the site before.

First lets look at getting you a Host account. This will only take a few moments. First jump to buzzumi.com and click the (nice yellow friendly) signup button.

From here, signup should be straightforward. The first field is your username - pick this with care, because it will be your "profile page" as well - ie buzzumi.com/username - and shows up in your chat URLs as well. Your email address is just for the welcome email and password recovery.

Once you're signed up you're taken straight to your new profile page. You can customise things here - add details about yourself, your websites, blogs, twitter etc. For the moment we'll skip that and go straight to the fun bit - creating your chat.

Lets do this real quick - click "launch chat" on your profile, then select one-to-one for chat type, and lastly hit "create" (You could do lots more using the rest of the tabs but..later). That's it, you're done and now you're in your new chat space. We just have one more task - invite your friend! there are two simple ways to do this.

If you're chatting with your friend via another medium - say twitter or facebook - you can simply copy & paste the link from your URL bar to them. When they follow it, buzzumi will ask for their name and that's it - no signup or anything - and then they'll be straight in.

If your friend is on email we have a simple shortcut - just hit settings and put their email address in the invite box and click invite and buzzumi will send them a friendly message straight away with a link.

Once you're both in the chat space you can chat using text, or click enable camera for the full video/audio experience. Chat away and when you're done just close the window - no fuss, nothing to install!

And in the future, you can chat with anybody just by sending them a link. No Skype dance for you!

Posted November 21, 2011
// 1 Comment
 

I won a Kindle from @tokbox for this idea

OpenTok, a service that allows you to embed realtime video chat into your web app, recently decided to run a competition for great application ideas based on their platform. The prize was an Amazon Kindle.

Back in 2005, a short blog article entitled "Ideas are just a multiplier of execution" was posted by Derek Sivers, and it has recently done the rounds again thanks to a repost on Daring Fireball. I agree with the statement, but I think it's worth saying more.

Ideas and *timing* are multipliers of execution. Ideas are easy to come up with. Good ideas are somewhat rarer, and good ideas at the right time are rarer still.

So write the damn things down.

The idea you think of today may not be right for now. But the future is rather unpredictable and honestly it's difficult to say what might have value at a given moment. I keep a big list of ideas for this reason - ideas are cheap, but having the right one at the right time can make a big difference.

Of course, having all these ideas and being able to execute on the ones that are timely would require that I have a lot of time - which I don't. I can only do a few things at a time and I'm a busy guy. When the competition came up I looked at my list, grabbed the best relevant idea I could find and threw it out there.

The concept is simple - turn any old laptop or system with a webcam into an instant, no-download webapp-based security camera. Using the OpenTok features and a bit of ingenuity it should be possible to record, do motion detection, and then add on SMS and audio alerting, logging etc. In a sense an idea I had 4 years ago is now much much more practical - the time has (probably) come.

It turns out the peeps at TokBox really loved the idea and gave me the prize. There were other great ideas submitted, you can see a list by searching the #opentokideas hashtag.

But...I don't have time to do it. I'm busy with Buzzumi and my other work, adding another project just isn't possible right now.

I think the world would be a better place with a simple, no-effort security camera solution like the above. It would be useful for all kinds of scenarios, not just security. I'd really like someone to do it.

I'm probably not the first to think of it, but I'm taking the opportunity to make clear that I want you to take this idea, and make a fortune from it. I don't want a penny, I don't want credit. Regardless of the fact that I have no legal right to it anyway, I want anyone who is thinking "Man, I could do that.." to be absolutely free of any moral qualms. Take the idea and be awesome and I will be happy.

In addition, since it's a service I'd love to see: If you write up a *good* Kickstarter or whatever and send me the link, I will contribute. If you build the service, I will be your first paying customer.

Andrew Sell from Hipcycle did a great interview on Buzzumi today and afterwards I took a look at his Twitter feed. His most recent tweet was the following:

"Far and away the best prize that life offers is the chance to work hard at work worth doing." -Teddy Roosevelt

Among the many projects out there worth doing, I think this is one. Got some time on your hands? why not give it a shot.

Posted November 17, 2011
// 0 Comments
 

We've been alone too long

Humans are social creatures. To be alone is a pathological state for us, leading to delusions and self-destructive behavior. We use isolation as a punishment.

What is true for individual humans is true for us as a species as well. The loneliness of our existence was held at bay for much of our history by the pressing needs of survival, but this time has now passed.

We are unable to distract ourselves any longer. We have no-one to love but ourselves, and we have become narcissistic. We have no-one to fear but ourselves, and we have become self-hating. Our motivations and dreams are diminished by the understanding that there is no-one to show them to.

The effects are everywhere: with no greater goals we obsess about relatively meaningless things, granting them importance simply because there is nothing else. We are in a constant state of bike-shed design. We argue bitterly and incessantly about minor variations in economy, health care, social support etc, because there is nothing pressing that would motivate us to simply solve the damn problem and move on.

We have nobody to look good to, nobody who will judge us for the way we treat our own and say "You guys are jerks", nobody who would take advantage of our sloppy behavior.

We need someone else. We have to find them or create them, and we have to hurry, because I don't think we can be alone with ourselves much longer without going mad.

Posted October 8, 2011
// 0 Comments
 

How to send me design mockups

As a web developer I get my fair share of mockups from a designer who has been working in photoshop or similar. If this is you, I honestly don't mind these, but there are a small number of rules that make my life a lot easier:

1. If you use a non-standard font, make sure it can be retrieved from Typekit, specify a fallback, and YOU get to explain to the client why it won't look like your design on browsers that don't support it. If it can't be retrieved from typekit (or otherwise is properly licensed for web use), DON'T EVEN THINK OF USING IT.

2. Any graphic elements should be supplied both as part of the mockup, but also in separate files. These files should be in PNG format, they should contain the minimum piece of the element that is necessary (if you have a white triangle in a dark-grey rounded box, just send the triangle, I'll do the box in CSS), and they MUST have a transparent background.

3. Try and be consistent about your spacing, and when you aren't, tell me. If you have one item aligned at 8px and another below at 10px and there's no obvious reason for it, I'm likely to put them both in the same style so they'll both end up at either 8 or 10. I don't pretend to be a graphic designer but many of your profession seem a little bit lazy when it comes to precise layout in your mocks, so if you need something weird like that, make sure it's clear.

4. My life is easier if you provide a measure layer (lines with text indicating how wide each element should be). Not only is it easier, but the result will be truer to your vision.

5. One more thing about fonts. There is no way you can use a custom font that doesn't have a rough analogue in the standard font set. If you choose a custom font that is really narrow, for example, then we're screwed because the fallback won't be that narrow and it will bust the layout.

I can make CSS do some pretty fancy things in a fashion that degrades well, but I need your help.

Posted July 17, 2011
// 0 Comments
 

The state of Javascript

Web programming is going fairly javascript crazy at the moment, and for good reason. As the only universally supporting in-browser scripting language it has a monopoly in its niche few other languages can match, and the popularity and increasing demands placed on it have led to some truly impressive optimisation work by highly motivated and well resourced organisations.

As the complexity of the client side of the web application equation has increased, the desire to be able to share code (and programming load) between the client and server has become increasingly desirable, leading directly to server-side offerings such as Node.js (not the first server-side JS, but certainly the first to really gain traction).

A number of serious issues still remain in the path of serious javascript adoption. Some are merely a matter of time, others are so endemic now it seems difficult to imagine how they might be overcome.

Unlike most mainstream modern languages, Javascript suffers from an almost non-existent standard library. This lack was less obvious when we were just using it to update elements in a browser window, but with developers replacing entire stacks with JS code it is reaching a critical stage. While a wide range of developers are providing various bits of commonly-used functionality, their implementations differ dramatically and often overlap. Perhaps a real standard might emerge over time but at this point it doesn't look hopeful.

Similarly the language is suffering from a lack of standardisation across basic patterns. Even something as simple as creating an object is rife with debate. Install any 3 node modules and open them up and you'll find three completely different approaches to defining objects (and inheritance or mixins per the authors preference). In and of itself this isn't an issue as long as the interface is stable, however it leads to a much larger required body of knowledge. If I choose to inherit/extend a given class/object, I need to understand the particular method the developer has used for it, as well as the appropriate method of inheriting/extending it.

This gets even worse when it comes to things like singletons, modules, events etc. CommonJS are trying but even there the results have often been complex and difficult to understand as simple solutions are stymied by the client/server/sync/async mix.

Finally there is the async issue. In essence, Javascript is not an async language. It does not offer any of the standard features of languages designed for asynchronous operation such as continuations, co-routines, promises, or even threads. Some of these have been hacked on in various ways (promises/futures and WebWorkers for example) but again the enormous diversity of approaches has created a huge learning problem for a developer. Some asynchronous functions support event broadcast, others defers, still others continuation-calling style. Try to use the wrong pair of async libraries and you're left juggling two different promise implementations, plus handing callbacks off into what passes for the node stdlib.

A number of other popular languages suffer the same lack of async support, but almost none of them are used almost exclusively in an asynchronous fashion. All browser work is async, and if you use NodeJS pretty much everything you do is async. It turns into a morass of callbacks.

This is a serious problem. As it stands, even with nice simple synchronous server approaches we as an industry suck at producing robust, readable code. Add in an unavoidable layer of what genuinely amounts to bullshit (it's not *necessary*, we have demonstrated you can do async without callbacks everywhere) and we're faced with a programming nightmare - when a PHP process hangs doing something stupid, one client is affected. When a node.js callback graph does the same thing, the entire server instance grinds to a halt. Don't even get me started on the stack traces that result.

Worst of all, it seems there is genuinely no solution. JS is a shit language not because of the language but because it does not match the problem space well (anymore), yet it is the only option - nobody has successfully introduced a second language into the browser stack, not ever, and they have tried.

Various courageous people are attempting to deal with some of these issues. CoffeeScript is a language transform that has a lot going for it - not the least of which is that it attempts to reduce the number of different ways you can do classes and prototypes to just two - but the stdlib remains empty, tool support is still abysmal and it offers no solution to the callback problem. Various overlays trying to "fix" the callback issue are failing hard, just the same as they did for python - it seems you just cannot put this stuff in later, it has to go in on day one otherwise you can't wrap enough of the other shit the coder has to work with to be useful.

Perhaps the most interesting concept thus far has been simply to throw hands in air on the server-side and run a pool of WebWorker threads just like python, rails etc do now and offer a purely sync form of web dev for the most part, with the developer able to occasionally foray into async at the front when they need to write a chat client or stream a file. At this point, this approach in combination with CoffeeScript looks like the only real way that server-side JS is going to be able to compete with the existing platforms in code maintainability and ease of construction.

I suppose it's too much to ask for a miracle, some kind of new language shoulder-barged in by Google that offers a nice clean syntax with native async, the equivalent of PEP-8 and a solid standard library.

Yeah, that's what I thought too :(

Posted June 19, 2011
// 0 Comments
 

You're laughing at the labour web guy now..

So, you saw the video, you laughed your ass off, you joked with your buddies about what an idiot they were, or whatever.

What you didn't stop to think about is just how easy this kind of error is. Yes, there were files containing vital information just sitting there out in the open for anyone to grab, indexed by search engines. This will lead to an easy compromise of the information at hand.

Do you know how many other ways the same information could be compromised? you're laughing now but the chances are excellent that, assuming you're a web developer, you have at least one app out there with your name on it that has an information disclosure vulnerability of that degree or worse. You don't even know it.

  • Perhaps you never upgraded the framework you built it on
  • Perhaps you trusted the session engine that came with that framework (do you have any idea how many of those things have holes you could drive a truck through?)
  • Perhaps you echo'd a single unvalidated, unescaped piece of data back to the user, somewhere, anywhere in your app.
  • Perhaps you used SHA1 to sign a piece of data without using the right mode, or forgot to put a return statement after an auth check
  • Maybe you used a rewrite rule to redirect people to HTTPS but only did it on the landing page
  • Maybe you have one tiny piece of content on your secure page that is sourced from a non-https URL
  • Maybe you're running event logging on your app to track down a rare bug without realising it is logging credit card numbers in the clear
  • Maybe you trusted some third party library to auth via facebook, twitter, google, send email, do crypto, etc and somewhere, on a mailing list you never subscribed to, is a notice you have never seen about a security issue
  • Maybe you save a user-provided header such as user-agent to your logs without encoding or filtering terminal commands, or worse you read them in a web browser
  • Maybe you accidentally typed your ssh key password into a terminal once, and it is still sitting there in .bash_history waiting for someone who nicks off with your laptop
  • Maybe you think hashing your user passwords is enough to keep them safe in the event of a compromise

Are you paranoid yet? if not you're either not a web developer, or you're actively participating in the kind of delusion that will only go away when you finally get owned.

Are you reading the above list and thinking "How does that make me vulnerable?". Welcome to the club, the club of people who don't know what can hurt them. Population: everyone.

A web application is an enormously complex system, stuffed full of leaky abstractions designed primarily so that we can actually get anything done at all. Do you think the guy who put the backups there was aware that directory listings were turned on and that the page would inevitably be indexed? almost certainly not. Generally speaking, humans are not actually stupid - we simply act without all the information that is relevant to the decision.

This is why the job of security revolves almost entirely around defense in depth. The objective is to behave in a way that reduces the risk, you don't put backups in a publicly accessible dir on the website, ever. You don't do it not because it isn't safe (hell, stick it in a randomly numbered dir, randomly named file, no directory index and never ever put a link to it), but because it only takes one tiny thing to go wrong and bang, the whole of the internet is laughing at you for getting caught.

Instead, you put the thing outside the web root. You public-key encrypt it when you generate it. You put it in a non-obviously named directory and you make sure it's owned by a different user and not accessible by the web user. You do all of this because there are any number of ways that each of those defenses could fail by accident, and it's only the combination of all of them that gives you a fighting chance.

Oh, you don't do all those things? Even if you do, you're only less stupid than everyone else by degree. You are still not smart enough to understand the system entirely, not smart enough to predict all the ways in which your security could fail, not smart enough to even be sure you've covered *your* part of the job, without thinking about what everyone else has done.

So quit laughing, unless you're laughing at all of us poor monkeys stumbling around in a morass of systems and data we're far far too stupid to understand, pretending like we know what's going on.

Posted June 13, 2011
// 1 Comment