• François (unregistered)

    So basically if you wanted to rob the place, you just had to show up in the middle of the night and wait for a click.

  • SteveC (unregistered) in reply to Former Jr. Programmer
    Former Jr. Programmer:
    They WERE queued, but there should never have BEEN queueing.

    The delay built up from having to both read the now-enormous log AND all the swipes.

    I'm guessing that queuing was not a design decision. One processwould be appending the latest data from the card reader into the log file, and a second process is looking for new events that will release the locks. This approach would be one way to asynchronously manage the comms between the reader and the lock, while offering a log file at the same time.

    I've seen a few 'front desk' access applications that have a scrolling display of entry and exit events. They usually sit on the reception desk until the novelty wears off or the furniture needs moving and then get consigned to the floor or somewhere out of the way, living out their simple DOS/Win3.11/WinNT lifestyle in the dust. As long as the doors keep opening, nobody cares, and the security concerns that first justified the system are long forgotten.

    It probably ran like a rocket when it was first installed. I don't think the original developer would have thought what might happen with many years of log files (or what might happen if the logrotate process failed), nor that multiple swipes would be significant in the operation of the system.

  • Unit 73 (unregistered) in reply to whicker

    Or, just maybe, the application crashes next time because it can't open the log file to append to. And then, as soon as our clown walks out the door, there's no way of getting back in (other than through the walls, roof or floor.

    He took action when he didn't understand the consequences - that's foolish.

  • iMalc (unregistered)

    Magstripe is stone-age stuff, and that system takes the cake!

    I'm a developer working for a large company that makes Access Control systems. Perhaps your company would be interested in upgrading to the latest in contactless Mifare technology?

    We try our best not to give burgulars a helping hand!

  • (cs) in reply to iMalc
    iMalc:
    Magstripe is stone-age stuff, and that system takes the cake!
    The cake is a lie.
  • captain obvious (unregistered) in reply to OzPeter
    OzPeter:
    For changing a critical piece of security infrastructure with out any authorization or approval? I don't see him as a hero but as more of a vigilante.

    A hero would have done an investigation without changing anything and then made a clear and precise report the next day as to what was causing the heartache. At which point a considered response could be made which took the overall situation into account. After all he was new there, so there is no way that he would have been able to take the entire companies position into consideration.

    In fact he did not actually change anything, all he did was fix a symptom. I would guess that as the current log file filled up the system would start to behave in the same way again. So all he achieved was to give himself a virtual wank and a smug look when his coworkers next discussed the improved performance.

    Dude, shut up... this was a system that effectively did not work, anyone could come and wait for a backlogged access. I'm sure writing that gave you much more of a virtual wank than to this guy.

  • Tom (unregistered)

    Nice targeted advertising for "biometric fingerprint door locks"...

  • James (unregistered)

    Amazing the number of people who criticize this guy for not going through all eight layers of management to fix this problem. Not only was it a serious security issue, but it impeded work, too.

    "Oh, but what if X happened ..."

    We're not talking about prison door access or medical equipment here.

    How about what if one of the outsourced janitors noticed that the doors opened easily all night every night for them? How much equipment could they walk away with?

    The guy showed some initiative and a bunch of you want to squelch it. Lame.

  • Jeremy (unregistered)
    • It ran on Windows 3.1 (or 3.11). This was about 1996.
    • There were three or four doors - one from the lift lobby into reception (where the PC was), and two or three from reception into other areas. Constant traffic through all doors all day.
    • I thought about the possible consequences and took a conscious decision on what action to take, and what to do if something went wrong. That may well have constituted the most due diligence ever performed in that company, apart from when it was wound up.
    • The next day I informed both my manager and the most senior admin person I could find. Did they show signs of caring? Good job Alex! J
  • Stu (unregistered)

    The last company I worked for was very hot on timekeeping - no flexibility was allowed - and hence people generally arrived at 8:58 and left at 5:32. Unfortunately, it was discovered that the swipe card system only had limited resolution on the time, and the time was used as primary key in the database. Swipe in too soon after someone, and you wouldn't be logged, and hence you'd receive an email from HR.

    Once that was discovered, a new company policy was instituted - wait five seconds after the last person. Hence at five to nine there would be a nice queue of coders, any one of whom could have fixed the real issue, counting "one elephant...two elephant...five...swipe" under their breath.

  • Todd (unregistered) in reply to OzPeter
    OzPeter:
    For changing a critical piece of security infrastructure with out any authorization or approval? I don't see him as a hero but as more of a vigilante.

    A hero would have done an investigation without changing anything and then made a clear and precise report the next day as to what was causing the heartache. At which point a considered response could be made which took the overall situation into account. After all he was new there, so there is no way that he would have been able to take the entire companies position into consideration.

    In fact he did not actually change anything, all he did was fix a symptom. I would guess that as the current log file filled up the system would start to behave in the same way again. So all he achieved was to give himself a virtual wank and a smug look when his coworkers next discussed the improved performance.

    So you would let the doors unlock themselves all night? GREAT security!!!!

  • (cs) in reply to OzPeter
    OzPeter:
    For changing a critical piece of security infrastructure with out any authorization or approval? I don't see him as a hero but as more of a vigilante.

    Let me guess. You work in upper level management? Maybe even the PHB himself?

    Grow a sense of humor.

  • (cs) in reply to Guest
    Guest:
    The question is : How is it possible he could pass doors that where locked, and only responded to key-card swipes made hours before ?

    <quote>He examined the card reader and swiped his card. With a typical delay of 15 seconds, it unlocked.</quote> <quote>new swipes were being queued. So many were queued up that it was still processing swipes from 8:00 AM the previous morning,</quote> So it responded to his swipe (allmost) directly, but he machine was still processing swipes of the previous morning ?

    Something stinks here, and it aint my socks ...

    This is really too complex for you?

    It didn't respond to his swipe at all, but to a swipe made the day before that was the next item in the queue.

    I see you're "Guest". Perhaps this site is too advanced for you, and you should try somewhere else.

  • (cs) in reply to Patrick
    Patrick:
    I'm curious how it's possible that the swipes would process at different rates. If the swiping process and the door unlocking process are as disconnected as they seem to be here, wouldn't the door always be unlocking at the rate heard late at night?

    No, because at night when no one is swiping to get in, the system only has to deal with unlocking the door. During the day, it has to both unlock the door and deal with adding new swipes to the end of the queue.

    Obviously, if the system is as poorly written as it is, it isn't good enough to combine the two operations, and is opening and closing the entire file each time, plus doing the positioning to EOF to append the new swipe to the queue.

  • (cs) in reply to OzPeter
    OzPeter:
    Zylon:
    OzPeter:
    he didn't fix it .. he fixed the symptom. All he did was clear out the log file. In X number of days time the new log file will have filled up and you would have exactly the same problem
    Then the solution is to institute a scheduled archiving of the log file. E.G.-- make what he did into official policy.

    What are you, simple or something?

    from the story:

    "The following morning the buzz around the water cooler among those that noticed the improvement was that some anonymous hero had fixed the doors. Others couldn't put their finger on what it was exactly, but they knew something was right with the world that morning."

    So it looks like he did not tell anyone about his heroics. So his "fix" (which could have broken any number of other things - thats what investigations are for) seems like it was never rolled into policy.

    He just shot from the hip and got lucky

    No, he just hadn't yet talked to anybody about the issue.

    I think you'd have to vastly improve yourself to even be considered simple. You're a moron, and lack the ability to think.

    Wait! I know! You're an upper level manager in the government, right?

  • Mr Oli (unregistered) in reply to OzPeter
    ...done an investigation without changing anything and then made a clear and precise report the next day...considered response could be made which took the overall situation into account...no way that he would have been able to take the entire companies position into consideration...all he did was fix a symptom...

    You're new here, aren't you?

    Unless you're in the very rare position of working permanantly on the head of a cvs tree, or even a branch, you'll be in The Real World, which is all about fixing a symptom ahead of getting the buy-in and sign-off required to fix the problem.

    Would your boss, sorry, professor decide your uni should spend £5000 developing a new system, interfacing to the old (or new!) database to do the door swipes, when they have a perfectly good working one (provided the logs are cleared down every now and again)?

    They didn't even want to invest in fixing the quirky old one!

    The world just doesn't work the way you want it to dude. One day you'll learn that, then you'll be less stressed. Picture you're down to your last fiver. You have a new girlfriend coming round for dinner, no food in the house and a broken loo seat. What do you spend your fiver on? Now translate that to a company with, like, responsibilities towards paying their employees and stuff...

  • (cs) in reply to Marvin the Martian
    Marvin the Martian:
    I can't really believe the hypothesis that the unlocking is in response to requests made 24hours earlier: the requests would not spread evenly enough to let access be between 10 and 60 seconds, that just doesn't make sense. There must be a way that e.g. 1 request in 5 is honoured and the rest put in backlog (an advantage to the multi-swipers), or only one request per minute per door or so?

    Can nobody here think?

    It's not hard to believe, given the obvious quality of this system, that the following could happen:

    Someone swipes their card several times, because they're impatient. All are queued; each time, the log file is opened, a seek is done to EOF, the new scan is added, and the log is closed.

    Since the software is so crappy, it takes 10 or 15 seconds to get to the next item in the queue (the first original swipe). The log is opened, the first line is read, and the entire file from the second line to the end is rewritten. The log is closed.

    As soon as it closes the log, someone else swipes their card. The system opens the log, seeks EOF, and appends the new swipe. It then closes the log.

    Now it hits the loop to open the door. It opens the log, reads the first item (the second of the original swipes), writes the entire log from item 2 to EOF, and closes the log. Etc, etc.

    On a slow older machine, with a slow OS, processor, and hard disk, this sequence could easily start to back up. Intensify that by crappy software that doesn't reject duplicate key numbers in rapid sequence, and I could easily see this happening.

    And, from the number of people here who can't seem to figure this out, I'd bet that software's author is here somewhere. Maybe the whole team.

  • Martin (unregistered)

    Really nice story, very entertaining;), thank you for this one.

    Martin

  • (cs) in reply to D. T. North
    D. T. North:
    Anon:
    The other real WTF is that nobody has mentioned Irish Girl yet.

    I know...this is a fun game of mine now. I actually come into these response strings and do a search on "irish" just to see all the fun posts. It's almost more fun than the on-topic discussions.

    I have an idea for the both of you.

    Start your own Irish girl site. You could link to photos and everything, and make your juvenile, obviously virgin nerd comments there instead of in every single post here. You could attract more virgin nerds, and all talk to each other online. Wouldn't that be more fun?

  • NXavier (unregistered) in reply to KenW
    KenW:
    Can nobody here think?

    LMAO! Get 'em KenW, you're on a roll!

    I think we both need a vacation, Ken.

    /sigh

  • Dave (unregistered) in reply to ebs2002
    ebs2002:
    I'm actually pleasantly surprised. Normally I read these with bated breath, but this one was quite good.

    What - articles on this site?

  • (cs) in reply to RandomWTF
    RandomWTF:
    NameNotFoundException:
    The Real WTF is that nobody has yet commented on the complete and utter stupidity of queuing card swipes. Doors are meant to be instantly accessible. That means, while a card swipe is being processed it makes absolutely no sense to queue up more. Somebody really screwed up this one.

    Not necessarily. At my previous job the door swipe records were also used to compare against our reported hours. They were also used to record who was coming and going.

    As such it was official policy that we must swipe our cards coming and going, even if there was already someone in front of us who had the door opened. "Tailgating", as it was called, was expressly forbidden.

    In that context it makes sense to process and record each and every swipe, even if the door may already be unlocked at the time of certain swipes.

    But in that context it wouldn't be queued, would it? Just logged; no need for adding to the queue.

  • Dave (unregistered) in reply to OzPeter
    OzPeter:
    For changing a critical piece of security infrastructure with out any authorization or approval? I don't see him as a hero but as more of a vigilante.

    A hero would have done an investigation without changing anything and then made a clear and precise report the next day as to what was causing the heartache.

    Sometimes there's an argument for that, yes. Othertimes it's obviously wrong and you need a good argument for leaving things the way they are, given that they're so obviously broken.

  • (cs) in reply to Franz Kafka
    Franz Kafka:
    By the way, notepad can't deal with 300M files.

    Notepad works fine with a 300MB file on my machine, running XP SP2 with 2GB RAM.

    Theoretically (I haven't tried it), Notepad should open a file up to 2GM in size (albeit slowly).

  • Avalanche (unregistered) in reply to iMalc
    iMalc:
    Magstripe is stone-age stuff, and that system takes the cake!

    I'm a developer working for a large company that makes Access Control systems. Perhaps your company would be interested in upgrading to the latest in contactless Mifare technology?

    We try our best not to give burgulars a helping hand!

    This is the Real WTF, since Mifare has been seriously hacked (causing some panic in the Dutch government, because Mifare cards have been used extensively for 'securing' government and military facilities here). The fact that Mifare is contactless only makes it easier to hack.

    See als http://www.wtop.com/index.php?nid=108&sid=1363664

  • (cs) in reply to He Who Is Looking For Trouble
    He Who Is Looking For Trouble:
    WTF. didn't anyone notice that it's clicking all the time during the day?

    Right. I work in a fairly large office with 50 other people. There are people going in and out the security doors all day (the bathrooms are outside the secured area so guests can get to them), phones are ringing, people are talking, printers are grinding out pages, copiers are scanning documents and producing copies, keyboards are clattering, file cabinets are being opened and closed, the microwave is beeping, sodas and snacks are dropping from the vending machines, water is running in the sink in the kitchen area, desk drawers are opening and closing... Yeah, I notice every time the security locks click when someone enters their numeric code. Right.

    Do you even bother to think before posting nonsense? Somehow I doubt it.

  • Geezer coder (unregistered)

    Let's not trouble ourselves over the real WTF here, which is a security system that is accessible to a random employee.

  • (cs) in reply to DHager
    DHager:
    Strilanc:
    If the delay was measured in days, or even hours, people would notice right away. If you ever were the first to open a door in a day, it would take hours for it to unlock.
    OriginalPseudonym:
    Eh, I call shenanigans. If it really was still queuing swipes from the day before, what do people do on Monday?

    I hope neither of you guys are writing multithreaded programs for consumer use. :P

    I hope neither of those guys are writing any programs for consumer use. In fact, I hope they're not writing programs, and use their computers for browsing the web exclusively.

  • Dave (unregistered) in reply to KenW
    KenW:
    Theoretically (I haven't tried it), Notepad should open a file up to 2GM in size (albeit slowly).

    2 GigaMegs? That's a pretty big file...

  • JRock (unregistered)

    Wouldn't the problem still exist?

    'The problem was that cards had been swiped faster than the system could handle, so new swipes were being queued'

    Jeremy shouldn't get too big a head.

    Knowing that the system isn't adequate and security is at stake should be reason enough to get a new one.

    The hero worship can begin when Jeremy talks management into getting a new card reader.

  • Jay (unregistered) in reply to OzPeter
    OzPeter:
    For changing a critical piece of security infrastructure with out any authorization or approval? I don't see him as a hero but as more of a vigilante.

    A hero would have done an investigation without changing anything and then made a clear and precise report the next day as to what was causing the heartache. At which point a considered response could be made which took the overall situation into account. After all he was new there, so there is no way that he would have been able to take the entire companies position into consideration.

    In fact he did not actually change anything, all he did was fix a symptom. I would guess that as the current log file filled up the system would start to behave in the same way again. So all he achieved was to give himself a virtual wank and a smug look when his coworkers next discussed the improved performance.

    Jeremy: Mr OzPeter, Mr OzPeter! The building is on fire! Quick, we've got to evacuate! Call the fire department! Where's the fire extinguisher?!

    OzPeter: Now now, those actions would just address a symptom of the problem, not the root cause. We have to determine what caused the fire, and then prepare a full report, taking into account all relevant company policies, before we take any action. You begin work on the report and I'll schedule a meeting for next Thursday to discuss the appropriate response. In the meantime, I don't want to hear any more of this "fire extinguisher" nonsense. I expect you and all the other employees to remain at your desks working as usual until management can formulate suitable policies to deal with the situation.

    Seems to me the only part OzPeter left out of his original post was the need to consult with the company lawyers to make sure that Jeremy's actions could not be construed as sexism or creating a hostile work environment for the handicapped.

  • Ken B (unregistered) in reply to g
    g:
    so why was it slow? How can processing a card swipe take that long? I assume it was something to do with the size of the log file or something?
    My guess is that the logic went something like this:
    • Open the log file.
    • Read the log file until EOF.
    • Append new line to the log file.
    • Close the log file.

    After several years (?), the log file would probably get quite large.

  • Kuba (unregistered) in reply to Thief^
    Thief^:
    My guess is that there was a roughly constant one minute delay on processing any swipe, thanks to stupid processing of the log file. Swiping a lot would cause the door to lock and unlock about once a minute until the number of swipes had happened. With so many people using the door and getting pissed at the one minute delay and swiping a lot, it got to a queue over a day long. If it unlocked in less than a minute then it was responding to someone else's swipe from earlier.

    Over the weekend it would catch up again and take a minute to respond to the first person arriving to work on Monday. This would be longer than when the queue was backlogged (which would average 30 seconds), so they would definitely swipe several times to be sure, and set the whole thing going again.

    This is actually the most correct analysis, IMHO.

  • Kuba (unregistered) in reply to JimM
    JimM:
    Thief^:
    My guess is that there was a roughly constant one minute delay on processing any swipe...
    Can't be that - when Jeremey tested it after clearing the log file it responded instantly.

    Jeez, it was slow because the log was long. That constant delay was probably O(log file size). Small log = small delay.

  • Laie Techie (unregistered)

    Many card swipe systems are used to track people, not just unlock doors. The system should have had two processes: one for appending the swipe in the access log, and another to send an open command to the door (real time, no queue). The access log should be rotated.

    I half-expected the dust on the relic machine to somehow be responsible.

  • Aare (unregistered)

    Ha-ha! I just love the solution of unix hackers to write log files to infinity with absolutely no concerns about where it can lead in a long run. If they would plan so far ahead, this incredible story would never had happened. Must give them credit for that. :)

  • Ostadan (unregistered)

    I propose The Curious Incident of the Door in the Night as a better title.

  • (cs)

    One of my first thoughts was that there was no security at all. Since they knew their system didn't work they just unlock doors randomly or for example every two minutes, so noone has to wait for too long. :-)

  • DOA Is Right (unregistered) in reply to Jeroen

    I agree with DOA. The only way the system would be usable would be if the logging and the unlocking subroutines were separate. The delays could be explained by the significant CPU load created by the logging subroutine trying to manage a 300MB file. However, this theory doesn't hold water either because the doors were unlocking long after the people had used them. Certainly such a scenerio would cause some rarely used rooms to be completely inaccessible and a service call would have certainly been made long before a work-a-holic found the issue.

  • Security Concious (unregistered) in reply to Neil
    Neil:
    And the WTF was? That a system went unchecked and unmaintained for a long time?

    No. The WTF is that the computer that controls the doors was unlocked and outside of the secure area. If he was able to modify the log file, he could have done anything to that machine. And, by hacking the door machine someone could gain access to any part of the facility.

  • eryn (unregistered) in reply to Rik
    Rik:
    Carl:
    If the delay was only about 1.5 days, then what happened on the weekends? If no one swiped their card on Saturday or Sunday, then the door would not open when swiped on Monday morning until mid-day Tuesday, right?

    Nonsense. The queue would either be gone, resulting in a pretty quick result, or it would still be busy, opening and closing. The delay isn't 1.5 days, the queue could very well be.

    And renaming a logfile without an investigation what else could be using that file is not only just fixing a symptom rather then the problem, it possibly breaks things.

    I think everybody is working themselves up here, maybe the "processing requests from yesterday 8 a.m is a bit of an embellishment", perhaps the system dropped a request if it noticed a previous one for the same door, or the problem wasn't with the log file itself but more the contention for the piece of code writing to the file... maybe some sort of timeout, or it cleared all requests for a particular card once an entry for it was processed. maybe it felt random because in the morning it was instant and during the course of the day it got progressively slower because the log was flushed on a set time or a maximum size.

    What I object to though is everybody saying it felt random because the system responded to a previous request sent a day or two ago, that's complete rubbish.

  • eryn (unregistered) in reply to KenW
    KenW:
    Marvin the Martian:
    I can't really believe the hypothesis that the unlocking is in response to requests made 24hours earlier: the requests would not spread evenly enough to let access be between 10 and 60 seconds, that just doesn't make sense. There must be a way that e.g. 1 request in 5 is honoured and the rest put in backlog (an advantage to the multi-swipers), or only one request per minute per door or so?

    Can nobody here think?

    It's not hard to believe, given the obvious quality of this system, that the following could happen:

    Someone swipes their card several times, because they're impatient. All are queued; each time, the log file is opened, a seek is done to EOF, the new scan is added, and the log is closed.

    Since the software is so crappy, it takes 10 or 15 seconds to get to the next item in the queue (the first original swipe). The log is opened, the first line is read, and the entire file from the second line to the end is rewritten. The log is closed.

    As soon as it closes the log, someone else swipes their card. The system opens the log, seeks EOF, and appends the new swipe. It then closes the log.

    Now it hits the loop to open the door. It opens the log, reads the first item (the second of the original swipes), writes the entire log from item 2 to EOF, and closes the log. Etc, etc.

    On a slow older machine, with a slow OS, processor, and hard disk, this sequence could easily start to back up. Intensify that by crappy software that doesn't reject duplicate key numbers in rapid sequence, and I could easily see this happening.

    And, from the number of people here who can't seem to figure this out, I'd bet that software's author is here somewhere. Maybe the whole team.

    ken, maybe u should let ur brain catch up with ur mouth.

  • John K. (unregistered)

    Best WTF I've read in a while, though the stargate one comes in a close second.

  • Sasquatch (unregistered) in reply to Jeroen

    Why would he have access to the log?...

  • Anonymous (unregistered) in reply to Chez

    I heard some of the brand new OSs have a really cool feature where you can open a file, and then write only the new stuff to it, and it will keep all the old stuff there AND not make you take forever and a day to write the old stuff back out.

    I think it's called "append mode". Definitely a WTF.

  • (cs) in reply to Anonymous
    Anonymous:
    I heard some of the brand new OSs have a really cool feature where you can open a file, and then write only the new stuff to it, and it will keep all the old stuff there *AND* not make you take forever and a day to write the old stuff back out.

    I think it's called "append mode". Definitely a WTF.

    What, we're not talking about a LispMachine here? Where does this "write the old stuff out" concept come from?

    The general consensus seems to be that this doomed program opens, searches, appends (and preferably closes) the txt file each time a card is swiped. Why? Well, search me. Appalling design would seem to be the culprit. I'd assume that the thing reads in the entire file in one go (as per Perl <>, Notepad, or even Emacs for that matter) and then goes back one line (or more than one, if security is important). Even opening a file and going to the end using, eg, fseek, may be an expensive effort. Going backwards from there would require the programmer to understand that a txt file can be treated as a non-structured byte sequence, and I doubt that this was the case.

    Brand new OSs (ie anything since 1970) are not the issue. Designer and programmer stupidity are the issue.

  • tibbe (unregistered) in reply to Jeroen
    Jeroen:
    DOA:
    I must be missing something. If the system was still processing swipes from yesterday, how come people were still able to use the doors. Wouldn't they have to wait a day for their swipe to open the door?
    No I think people just swiped their card and then the door would open because of a swipe made hours/days earlier. That's why the time it took seemed random.

    Well, actually, why would that make the time required to open the door random? As far as I could tell, it hadn't time shifted, it was just very slow. One could think that it would always take about the same time to process one swipe - causing the door to open with a fixed interval, not random interval (if it had queued swipes).

    Ah, perhaps the "random" part came from the fact that if the door opened once a minute and they came there just before that it'd open fast and if they swiped right after the door had previously opened the door if would seem slow...

  • VP (unregistered) in reply to OzPeter
    OzPeter:
    A hero would have done an investigation without changing anything and then made a clear and precise report the next day as to what was causing the heartache. At which point a considered response could be made which took the overall situation into account. After all he was new there, so there is no way that he would have been able to take the entire companies position into consideration.
    Man you sound like the IT-Architects at the corp. i work at... Way to overdo a problem. In most cases adHoc solutions are preferable and the more cost-efficient and well efficient thing to do.
  • fred (unregistered)

    who gives a crap...can't believe I wasted time reading this drivel, and even worse writing a comment....

  • Martin (unregistered)

    The ad below the article was for a biometric door unlocker.

Leave a comment on “The Haunted Door”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article