• (disco)

    :interrobang:

  • (disco)

    I coulda sworn I had a comment on this topic…

  • (disco) in reply to RaceProUK

    Paula created a duplicate.

  • (disco)

    Sounds like a quite nice architecture to me, compared to all the WTFs I have seen and read about.

    Why didn't they just change the column to a BIGINT instead of truncating the table?

  • (disco) in reply to ChrisH
    ChrisH:
    Why didn't they just change the column to a BIGINT instead of truncating the table?

    My thoughts exactly. Also, it seems like a bit of an overreaction to quit your job just because a single column is int instead of bigint...

  • (disco)

    My feeling too: some architect weasel words, some truth, some dumb implementation, but no WTF. Only the HOS (Hourly O Shit). Then ending might have been added for dramatic effect, though. No WTF story is complete without the hunt for a new job. Where do all these people go, I wonder.

  • (disco) in reply to ChrisH
    ChrisH:
    Why didn't they just change the column to a BIGINT instead of truncating the table?

    Yeah, I don't get why they didn't just do that. But then again I've worked in places that use char instead of varchar and were too afraid to just change it.

  • (disco)

    Danny shook the intern briskly

    And was later served with a lawsuit for assaulting a coworker.

  • (disco) in reply to Vault_Dweller

    Yeah, I'm not sure I'd quit my job for that. It doesn't seem to bad to either change the column to a bigint, or to periodically truncate the table (or delete old data and reset the ID counter when it wraps if their DB engine supports that). That seems like a minor change to me.

    The management seem to have been behind Danny as well, and didn't seem to be blaming him - so why quit? Maybe that's TRWTF?

  • (disco) in reply to Hanzo
    Hanzo:
    No WTF story is complete without the hunt for a new job. Where do all these people go, I wonder.

    To write articles here would be my guess. Either that or to go around Spain looking for anybody who even checked in anything in any code base they've worked on.

  • (disco) in reply to BobbyTables
    BobbyTables:
    go around Spain looking for anybody who even checked in anything in any code base they've worked on.

    What? :raising_hand:

  • (disco)

    Just goes to show, Hell has weather just like any other place. Sometimes it's a bit colder, and all the denizens will say 'Wow, it's a cold day in hell today'.

    Too bad for Roger's monolith.

  • (disco) in reply to Eldelshell

    http://thedailywtf.com/articles/The-Professionals

  • (disco)

    2147483647 monitoring records ought to be enough for anybody.

  • (disco)

    http://what.thedailywtf.com/t/the-monolith/47593/6?u=eldelshell

  • (disco)

    Roger: "Listen you morons, my design is scalab..."

  • (disco) in reply to Eldelshell
    Eldelshell:
    using a RDBMS to store logs

    Why would you do that?

  • (disco) in reply to dkf
    dkf:
    Why would you do that?

    Yeah, they should append to a log.

  • (disco) in reply to dkf

    Analytics.

  • (disco) in reply to dkf
    dkf:
    Why would you do that?
    With a sensible archival strategy, why not?
  • (disco) in reply to blakeyrat
    blakeyrat:
    Analytics.

    But why would you keep that live as part of the application? Analytics is meta-processing, and can be shuffled off to a dedicated server if necessary since it's quite likely to involve unpleasant things like ad hoc queries…

  • (disco) in reply to dkf

    I didn't build their fucking system, I don't know. I was just answering your dumbass question.

  • (disco) in reply to dkf
    dkf:
    But why would you keep that live as part of the application?

    You can't do it off line, or by the time you do, the tinfoil hat brigade will have slipped through your fingers.

  • (disco) in reply to boomzilla
    boomzilla:
    You can't do it off line, or by the time you do, the tinfoil hat brigade will have slipped through your fingers.

    If you're worried about that, just use a 16-bit integer for your ID column. Like that there'll be almost nothing happening until you've had a chance to analyse it!

  • (disco) in reply to blakeyrat

    At which point I could answer the question with....

    Frog tongue juice.

    I don't know, I didn't build their system.

  • (disco) in reply to dkf
    dkf:
    If you're worried about that, just use a 16-bit integer for your ID column. Like that there'll be almost nothing happening until you've had a chance to analyse it!

    But I want to track those guys and offer them ads for the same brand as they make their hats out of. Geez, that's the whole freakin' reason for analytics!

  • (disco) in reply to boomzilla
    boomzilla:
    But I want to track those guys and offer them ads for the same brand as they make their hats out of. Geez, that's the whole freakin' reason for analytics!

    That's how the Very Big Corporation of America discovered the true Meaning of Life: That people are not wearing enough hats.

  • (disco) in reply to EatenByAGrue
    EatenByAGrue:
    That's how the Very Big Corporation of America discovered the true Meaning of Life: That people are not wearing enough hats.

    And that's where @abarker comes in

  • (disco)
    Danny shook the intern briskly to help restore his composure
    I, too, find that shaking interns briskly helps to restore my composure. I highly recommend it to any and all.
  • (disco) in reply to RogerC

    Yay ambiguity!

    Filed Under: inb4 lojban

  • (disco) in reply to Hanzo
    Hanzo:
    No WTF story is complete without the hunt for a new job. Where do all these people go, I wonder.

    Each one just moves into the empty position created at the end of the previous WTF. It's like Hilbert's Grand IT Department, only with no room service.

  • (disco) in reply to dkf

    It depends on what you want to log.

    Many logging frameworks allow you to send log entries to a DBMS.

    We send simple log entries to our DB; there is an ID for the log entry, an ID for the user (also special IDs for servers) that caused the action, an enum ID for the action, the type of log entry (error, action, etc.), the date and time, and the type of main and secondary objects the entry related to (customer, order, etc).

    Then the log entry in an actual text log also contains the ID from the DB.

    So now a person can search the DB for a specific log entry for a given type of entry, in a given date/time range for a given "user" and then correlate that with the entry in the log file. Eventually we will write an app (or find one) that can do the searching from a user friendly app. This is mostly for the help desk, QA and dev. to work with.

  • (disco) in reply to ChrisH

    Because it would take an hour or so for the alter table to run and the web site would be down for that time and unlike the initial problem Management would know exactly who to blame. I've worked in places where taking the website off line for a hour for some annual maintenance was considered a grater evil than letting the database crash once a year and spending half a day trying to fix it.

  • (disco) in reply to John_Imrie
    John_Imrie:
    grater evil

    http://drawception.com/pub/panels/2012/6-7/scnmwSOc5g-6.png

  • (disco) in reply to John_Imrie

    I think you're too optimistic. How many of those 2 billion rows were still in the table? The ALTER TABLE could take a lot longer than one hour. And if it dies because the transaction log fills up while attempting that, how many hours will go by before the transaction rollback finishes and you're right back where you started? I don't think I've ever worked for management that would be happy with that.

    Rename the original table and create a new one with a BIGINT primary key, starting at 2.2 billion. Set up a job to copy records from the original table into the new one in small batches, limiting the rows/minute to something that won't stress the system. Stop it when it has copied enough recent history to make everyone comfortable (last week, month?). Then drop the original table. Alternatively, skip the copying and keep the original table around for a while. Drop it when it is no longer relevant.

  • (disco)

    So I'm only an Ahole DBA and not a jerk face developer, but there might have been a few options to keep the old records and keep the system live. Creating the table with a different name, putting a view on top it excluding the ID column, and then renaming the two tables could work live depending on the architecture. Then they would keep their old data and still be able to query it.

    They could just rename the old table in a script right after creating a new one with BIGINT as the ID column while keepign everything live. Yes BigInt will take up more space that's true, but they don't seem too terribly concerned about space and there's good ways of dealing with that.

    Truncate is just so.... brutal and non reversible live esp. if you aren't in a emergency to get usable DB space. Truncate makes me sad.

    EDIT: Caveman ANGRY! Caveman truncate table with big club. Caveman now happy he took care of problem. Ooo ooo ! That's my view when someone immediately in a panic suggests to truncate a table in an emergency. Not this particular story.

  • (disco)

    As others have said, logs in an RDBMS are really nice for QA/quick problem resolution (well, as quick as the queries run), etc. Just don't expect 'em to scale to a large number of requests per time period.

    For an app whose traffic averages less than a request per second, I've got just such an infrastructure in place. When we start serving 100x that number of requests, we'll figure out something else.

    For another app that I've worked on, peak traffic is around 70 requests per second. I probably wouldn't recommend this architecture for them. Throw logs into a database? Sure...just not a relational one. Cassandra maybe?

  • (disco)

    I have a hard drive named "The Monolith". It was an external drive, and came in a sleek, black Lacie case.

    https://www.lacie.com/imgstore/product_intro/hd_hddesignbyneilpouton_triple.jpg

  • (disco) in reply to iansltx

    shhhh... don't say the "No" word or all the i-relational ones will jump at you.

  • (disco) in reply to MRAholeDBA

    But, but BAD ARCHITECT! He designed this WRONG! :crying_cat_face: :crying_cat_face: :crying_cat_face:

    And he's on a beach :crying_cat_face: :crying_cat_face: :crying_cat_face:

  • (disco) in reply to Eldelshell
    Eldelshell:
    shhhh... don't say the "No" word or all the i-relational ones will jump at you.

    NoSQL?

    is good at solving one class of problem, is utter crap at solving most of the problems people ask it to solve.

  • (disco) in reply to accalia
    accalia:
    NoSQL?

    is good at solving one class of problem, is utter crap at solving most of the problems people ask it to solve.

    It's just the latest Free LunchSilver Bullet.

  • (disco)

    How long has The Architect been on sabbatical? Long enough for the volume of requests to increase 100x, apparently.

  • (disco) in reply to antiquarian

    yep.

    I love NodeJS... i use it all the time myself for personal projects, it's a fun language and the design makes it easy to fall into a pit of success for certain classes of problems.... but it's no silver bullet.

    and neither is NoSQL. @todo uses nosql for storage and i'm still not sure i havent made a serious error there...

  • (disco) in reply to iansltx

    It depends what your definition of large is. Using proper data warehousing techniques you could get nice high concurrent queries against a table with billions upon billions of rows. You'd want to utilize techniques such as indexed views, compression, partitioning, and highly tuned queries/indexes. There's a lot of considerations with these NoSQL solutions but they are very exciting IMO, I would just suggest not worrying so much about size but the entire app lifecycle personally.

  • (disco) in reply to dkf

    I'll bite. Personally I'd store the logs in a RDBMS in a few scenarios: 1- It's highly joinable. Do I have other tables that need to join this data? If so, are they reasonably wide rows where having deduplicated copies of them make sense? Let's not forget the 'magic sauce' of many NoSQL solutions for 'web scale' is that they keep it in memory. Keeping highly redundant data in memory could be a bad design.

    2- Does it need to be transaction controls and ACID compliance? Maybe the app has government compliance or PCI tier 1 requirements/etc. etc. where you have to enforce this level of compliance on your logging data. Perhaps you can get this with many NoSQL solutions but this is one area I'd look at.

    3- You have sufficient in house experience and investment in hardware/software/support and you don't in the other areas nor are you really motivated to get it.

    Meh those are the ones off the top of my head. Maybe someone will find them useful.

  • (disco) in reply to Captain
    I have a hard drive named "The Monolith". It was an external drive, and came in a sleek, black Lacie case
    This, together with the article, opens the way to interpretations of the Space Odyssey hitherto unbeknownst to mankind.
  • (disco)

    TRWTF is Roger being able to retire on his Sabbatical.

  • (disco) in reply to John_Imrie

    The site was down anyway while they were trying to reach the architect on his seagull-infested island.

  • (disco)

    The real wtf here is of course tcp/db.

Leave a comment on “The Monolith”

Log In or post as a guest

Replying to comment #:

« Return to Article