• Shoko (unregistered)

    No comment on this page

  • Dave C. (unregistered)

    And so once again we learn: Check up often on the new guy.

  • (cs)

    Monitor the new hire's source control entries people. Seriously, write this down. If the new guy screws it all up it's also your fault for not checking up on him.

  • (cs)

    No fair, I was fist, but before I could post Hammerhead Bob came up and carried on a pointless, circular monologue for 10 minutes.

  • Steve (unregistered)

    If you let the new guy work on a simple project for 3 weeks without glancing at his code, this might happen.

  • Big Bad Bear (unregistered) in reply to Dave C.
    Dave C.:
    And so once again we learn: Check up often on the new guy.
    Or to put it another way: design reviews, design reviews, design reviews!

    What? No design to review? FAIL! Prove to me you understand the problem and have a usable solution, or BEGONE!

    Forcing the goat of our story to explain how he intended to solve the problem would have snuffed that hideous, convoluted, EVIL flowchart in the cradle.

  • (cs)

    oh come on, we've all done that! :) i remember writing an unholy mess which used 3 different recordsets to spew out pagination, all because i was trying to second guess windows indexing service logic.. those were the days

    actually i think that's still in production somewhere

  • @Deprecated (unregistered) in reply to DOA
    DOA:
    Monitor the new hire's source control entries people. Seriously, write this down. If the new guy screws it all up it's also your fault for not checking up on him.

    No kidding. Perhaps if Bruno had shown that spiffy flowchart to Greg, it would not have taken 3 weeks. Questions to ask new guy:

    • Any problems?
    • Let's see what you've got so far

    or in this case

    • WTF is taking three weeks?

    OTOH, Greg implemented that crazy workaround in ONE DAY!

  • (cs)

    Here's The Real WTF:

    The logic used to calculate distances, for instance, was handled by the front end, as well as real-time checks for vacancies would affect the sorting and remove some results from whatever was returned from the database.

    Any time your webapp features the term "logic handled by the front-end," you're probably doing something wrong. I don't see any reason why their server couldn't have done these things. The new guy should have slapped Bruno before starting the project.

  • Downfall (unregistered)

    Does Bruno have any tips for how to make text fit inside the shapes that comprise a flowchart? If so, you should hit him up.

  • (cs)
    items[] = loadAllItemsFromDatabase() items[] = filter ( items[] )
    This disturbs me. Was there really no way to filter the items during the query? If, based on the desired location as specified by the user, it's possible to filter out the hotels by distance after they're returned, then why can you not simply create a query that has the desired location as a parameter and returns only the appropriate hotels, ordered by distance?
  • Philip Skinner (unregistered)

    Who returns all results from a database when you're not going to show them all?

    Fail.

  • (cs)

    Let me boil it down for you.

    Bruno writes web app that doesn't query or process data in the most efficient manner. New guy implements new feature. New guy feels like the greatest developer ever. Bruno rewrites feature to solve issues caused by new guy's poor code and misunderstanding of the application framework. New guy gets fired. New guy feels about three inches tall. Bruno feels like the greatest developer ever.
  • Ben (unregistered)

    2nd real wtf on frontend processing...

  • Anonymous (unregistered)

    It's clear that Greg wasn't worth his paycheck but it's also clear that the application was pretty damn shoddy itself if the front-end had to do "some light massaging of the list in PHP code". We all know perfectly well that in software development the term "massaging" is a synonym for "hacking the shit out of it to make it work in any manner possible".

  • (cs) in reply to Outlaw Programmer
    Outlaw Programmer:
    Here's The Real WTF:
    The logic used to calculate distances, for instance, was handled by the front end, as well as real-time checks for vacancies would affect the sorting and remove some results from whatever was returned from the database.

    Any time your webapp features the term "logic handled by the front-end," you're probably doing something wrong. I don't see any reason why their server couldn't have done these things. The new guy should have slapped Bruno before starting the project.

    By "Front End" I'm going go say he meant "Their PHP Code" not "Some Javascript in the webbrowser"

    so yes, their server is handling it.

    not everyone has the same exact definition of "Front end" and "back end". in much of web development the "back end" = storage/db and "front end" = your PHP, JSP, ASP.NET, whatever code.

  • (cs)
    ps aux:
    http://php.about.com/od/mysqlcommands/g/Limit_sql.htm

    items = pageNumber * 10;

    LIMIT items, 10

    It's (should be) faster to store the results in the session if each record is small - no need to have the database scan a non-random-access* collection for matching results, discard the first X each then return the next Y each time you flip the page when you can store your entire result set in a random-access collection.

    but that's just my high-falutin university education speaking :P

    [edit]

    • yes i know in general the collection IS random access - if you place no constraints on the data returned. when you have to search for results matching a pattern you effectively loose the random access because you don't have an ordered index listing ONLY entries that match that criteron.
  • Mike Foster (unregistered)

    "Bruno was a little hesitant to give such a simple task to a developer with Greg's qualifications."

    This is the root of the problem. Bruno doesn't check up on new guy because he doesn't want to make new guy think that he thinks he is incompetent. New guy doesn't ask Bruno for clarification because he doesn't want Bruno to think he is incompetent. It takes time and effort to get comfortable in a new situation and that usually results in failed communication early on.

  • leppie (unregistered)

    I think I worked with Greg before...

  • Downfall (unregistered) in reply to Kazan
    By "Front End" I'm going go say he meant "Their PHP Code" not "Some Javascript in the webbrowser"

    so yes, their server is handling it.

    not everyone has the same exact definition of "Front end" and "back end". in much of web development the "back end" = storage/db and "front end" = your PHP, JSP, ASP.NET, whatever code.

    I interpreted it this way too. The point remains, though, that "front end"/"back end" is one of those terms like "business logic"; it's vague enough that using it requires explanation, and therefore should be avoided whenever possible to avoid exactly this type of confusion. I wonder how many programmer-hours have been lost because somebody gave a vague direction along the lines of, "make sure that <x> is processed in the front end."

  • (cs) in reply to Kazan

    Well I suppose that's a little better. Not sure how things are in the PHP world, but if I ever put logic like that in our JSPs I'd definitely get a paddlin'.

  • (cs) in reply to Outlaw Programmer
    Outlaw Programmer:
    Well I suppose that's a little better. Not sure how things are in the PHP world, but if I ever put logic like that in our JSPs I'd definitely get a paddlin'.

    which logic is that? the one that calls an external webservice to determine room availability*? in a situation like the code described above there are going to have to be some code filters on the list - things that their database may not have the most up to date data on and the only sane way of updating that data is on-demand [and not bother storing it in the DB at all] since it can change in real time and you're not the data source: an external entity is the data source.

    if you've never worked on a system that couldn't store everything in the DB then you've never worked on a truly complex web system.

    *that's an educated guess that is how they were getting that data.

    [edit] OH that's right.. in JSP the .jsp is just the presentation logic. in PHP it's ALL .php, though best practices you'll have separate presentation php and logic code.

    .php is ALWAYS like JSP in that text not between PHP start/end tags is just echo'ed straight through. much of the time though someone writing php will take a template file for the HTML and construct the HTML programatically.

    i've even known some people doing MVC designs to write the DOM programatically then just render the DOM.

  • Observer (unregistered)
    ps aux:
    http://php.about.com/od/mysqlcommands/g/Limit_sql.htm

    items = pageNumber * 10;

    LIMIT items, 10

    For whatever reason, they had to cull out unavailable hotels after they retrieved a list from the DB. Also, hotels needed to be sorted by distance which could not be a part of the SQL query, so getting 10 results at a time would not work. The system needed ALL the results so it could process the list and THEN paginate.

  • (cs) in reply to Observer
    Observer:
    ps aux:
    http://php.about.com/od/mysqlcommands/g/Limit_sql.htm

    items = pageNumber * 10;

    LIMIT items, 10

    For whatever reason, they had to cull out unavailable hotels after they retrieved a list from the DB. Also, hotels needed to be sorted by distance which could not be a part of the SQL query, so getting 10 results at a time would not work. The system needed ALL the results so it could process the list and THEN paginate.

    damn. and i wasted time talking about program data access efficiency and TOTALLY overlooked that aspect of it.

  • mr_smith (unregistered) in reply to Kazan
    Kazan:
    ps aux:
    http://php.about.com/od/mysqlcommands/g/Limit_sql.htm

    items = pageNumber * 10;

    LIMIT items, 10

    It's (should be) faster to store the results in the session if each record is small - no need to have the database scan a non-random-access* collection for matching results, discard the first X each then return the next Y each time you flip the page when you can store your entire result set in a random-access collection.

    but that's just my high-falutin university education speaking :P

    [edit]

    • yes i know in general the collection IS random access - if you place no constraints on the data returned. when you have to search for results matching a pattern you effectively loose the random access because you don't have an ordered index listing ONLY entries that match that criteron.

    So what if you are using a high falutin framework like ASP.NET and your session identifier is being stored in a cookie and you want to support a user doing multiple searches in different tabs/windows of the same browser?

    I'd just use limit in the database unless you run into problems unless testing for the load you have doesn't work out so well.

  • (cs)

    for all of you guys saying "why are you returning all results for a search instead of having the db filter it"... how exactly are you going to do "order by driving distance desc" in a db? driving distance is a non trivial calculation. The best I can think of is to throw in a stored proc that does as-the-crow-flies distance calculation and uses that to sort/limit the results, and then further filter in the frontend/business logic layer to do actual driving distance. Of course, I'm no GIS expert... so I might be completely wrong. But you can't ALL be GIS experts.

  • St Mary's Hospital for the Kazakh National Pride (unregistered)

    Am I the only one who thought of Brüno?

  • (cs) in reply to mr_smith
    mr_smith:

    So what if you are using a high falutin framework like ASP.NET and your session identifier is being stored in a cookie and you want to support a user doing multiple searches in different tabs/windows of the same browser?

    I'd just use limit in the database unless you run into problems unless testing for the load you have doesn't work out so well.

    give each search an identifier and then have a collection of search result collections.

    eg

    searchresults.asp?srchid=1235234635&page=0

    i'll leave how to generate search IDs as an exercise for the user

    hint: they only need to be session-unique.

  • shimon (unregistered) in reply to kastein
    for all of you guys saying "why are you returning all results for a search instead of having the db filter it"... how exactly are you going to do "order by driving distance desc" in a db? driving distance is a non trivial calculation.

    For crow-fly distances, WGS84 formula is easily implementable in SQL, albeit not very straightforward, but hey, not all programming is about basic math.

    Then, you can always look up the driving distance between cities. In the city itself, calculate the crow-fly distance. In most of the cases, you will end up with fairly accurate distances.

  • AdT (unregistered) in reply to Observer
    Observer:
    Also, hotels needed to be sorted by distance which could not be a part of the SQL query, so getting 10 results at a time would not work.

    SELECT blah blah FROM hotels WHERE blah blah ORDER BY SQUARE(hotels.x - @locationX) + SQUARE(hotels.y - @locationY) ASC

    QED (What to do if there is no SQUARE function is left as an exercise to the reader.)

  • shimon (unregistered) in reply to AdT
    ORDER BY SQUARE(hotels.x - @locationX) + SQUARE(hotels.y - @locationY) ASC

    QED (What to do if there is no SQUARE function is left as an exercise to the reader.)

    And now I have a piece of breaking news for you: the Earth, as the recent researches have shown, is not flat at all!

    And it's enough for the endpoints to be two or three states apart for cartesian distance to go from simply inaccurate to f-cking wildly inaccurate.

  • Stig (unregistered)

    OK web warriors, put your tommy tippy sippy cups down and listen up...

    Program it like a real man....in C cgi....forget all your namby-pamby "We can't handle real code so we'll swaddle it in layers of fluffy shash" web frameworks.

    Time to take the training wheels off

  • (cs) in reply to Observer
    Kazan:
    ...and the only sane way of updating that data is on-demand [and not bother storing it in the DB at all] since it can change in real time
    The hotel location changes in real time?
    Observer:
    Also, hotels needed to be sorted by distance which could not be a part of the SQL query
    Why not? What information do they have in the front end that can't be included in the query?
  • shimon (unregistered) in reply to Stig
    Program it like a real man....in C cgi....forget all your namby-pamby "We can't handle real code so we'll swaddle it in layers of fluffy shash" web frameworks.

    Time to take the training wheels off

    Better yet, roll your own webserver, CGI is for weenies.

    In the machine code.

    Typed in with a Morse key.

  • My Name (unregistered) in reply to Kazan
    Kazan:
    hint: they only need to be session-unique.

    You should also include the full query, to make it easier to share the link. What if I wanted to visit a city with my friends, and wanted to share the link to the search results? Unless the search id is unique, and you store it for a long time, this won't work.

    The best choice (IMHO) would be to include both a search id and all query parameters. If the search id is an active search, get the results from the session. If not, query the database and build the results again.

    People like to share links. The worst example of this, is a book store I once saw. Each link was actually a javascript postback call, and the link to the book was not a nice semantic URL, but a releative referance. The link was searchresults.aspx?item=2 (where 2 was the position of the book in the search...)

  • (cs) in reply to Outlaw Programmer
    Outlaw Programmer:
    Here's The Real WTF:
    The logic used to calculate distances, for instance, was handled by the front end, as well as real-time checks for vacancies would affect the sorting and remove some results from whatever was returned from the database.

    Any time your webapp features the term "logic handled by the front-end," you're probably doing something wrong. I don't see any reason why their server couldn't have done these things. The new guy should have slapped Bruno before starting the project.

    I completely agree. Why the fuck is he passing data to the client to be parsed instead of handling that on the backend, which would send a lot less data to the client and make the app faster.

  • Stig (unregistered) in reply to shimon
    shimon:
    Better yet, roll your own webserver, CGI is for weenies.

    In the machine code.

    Typed in with a Morse key.

    Attaboy. That's the fightin' spirit we need ;)

  • (cs) in reply to shimon
    shimon:
    ORDER BY SQUARE(hotels.x - @locationX) + SQUARE(hotels.y - @locationY) ASC

    QED (What to do if there is no SQUARE function is left as an exercise to the reader.)

    And now I have a piece of breaking news for you: the Earth, as the recent researches have shown, is not flat at all!

    And it's enough for the endpoints to be two or three states apart for cartesian distance to go from simply inaccurate to f-cking wildly inaccurate.

    I've never been in US but I guess most of the cities can be considered as flat(resonable small curvature can be locally mapped to flat - hence the human mistakes about flatness of earth/spacetime). Those which don't usually needs to be separated - why would I want a hotel in another state?

  • shimon (unregistered) in reply to Kazan
    in a situation like the code described above there are going to have to be some code filters on the list - things that their database may not have the most up to date data on and the only sane way of updating that data is on-demand [and not bother storing it in the DB at all] since it can change in real time and you're not the data source: an external entity is the data source.

    I would better cache it in the database once per 30 minutes and not bother.

    Pros: it becomes naturally easy to filter the results by the query alone.

    Cons: the data might be outdated in those loooong 30 minutes. Oh wait, it might get as well outdated as soon as the availability information travels from the vacancy webservice to the website and from there to the client! So who will give a shit?

    Check the most recent information only when the client selects a particular hotel, don't do it for all of them. You can then even update the availability data in the database without waiting for every-30-min-job. The bonus track is that the hottest hotels have the most accurate information.

  • Stig (unregistered) in reply to amischiefr
    amischiefr:
    I completely agree. Why the fuck is he passing data to the client to be parsed instead of handling that on the backend, which would send a lot less data to the client and make the app faster.

    Since the PHP is all server-side, obviously nothing is being processed in the client. The term 'front end' here is clearly being used to represent the outermost(ish) layer of the service.

  • shimon (unregistered) in reply to uzytkownik
    Those which don't usually needs to be separated - why would I want a hotel in another state?

    Why would I need a hotel in a city where I have a house/apartment I already pay for?

  • (cs) in reply to shimon
    shimon:
    In the city itself, calculate the crow-fly distance. In most of the cases, you will end up with fairly accurate distances.
    You must work in a midwest city yes? Lots of cities elsewhere have odd restrictions in where you can drive (one way systems, odd road blockages, you name it) which mean that the cartesian distance has nothing to do with how far it is on the ground. You might get a better estimate for some cities by treating them as a normed vector space using the Manhattan norm, but you need to know the orientation of the axes to make that work right and too often those aren't uniform across metro areas.

    Or you can do the sane thing and delegate the whole problem to some expert piece of software written by people who have to actually do this for a living. Whoops! That's outside the DB...

  • shimon (unregistered) in reply to dkf
    Or you can do the sane thing and delegate the whole problem to some expert piece of software written by people who have to actually do this for a living. Whoops! That's outside the DB...

    Well you can cache that.

    And, after all, it is approximate. It is supposed to be approximate.

  • (cs) in reply to Code Dependent
    Code Dependent:
    Kazan:
    ...and the only sane way of updating that data is on-demand [and not bother storing it in the DB at all] since it can change in real time
    The hotel location changes in real time?
    If you're California and the Big One hits, for sure…
  • Stig (unregistered)

    If you can't handle your own data by maintaining a vast multi-dimensional array of char* pointers, by all means buy a COTS DBMS, but please drop your testicles off by the door as you leave.

  • (cs) in reply to shimon
    shimon:
    Or you can do the sane thing and delegate the whole problem to some expert piece of software written by people who have to actually do this for a living. Whoops! That's outside the DB...
    Well you can cache that.
    Depends if you're only doing this for a small set of locations, or going the Web2.0 way and allowing it from any point of a map of the city. Caching only works for the first way. Users want the second.
  • Mayhem (unregistered) in reply to Code Dependent
    Code Dependent:
    Kazan:
    ...and the only sane way of updating that data is on-demand [and not bother storing it in the DB at all] since it can change in real time
    The hotel location changes in real time?
    Observer:
    Also, hotels needed to be sorted by distance which could not be a part of the SQL query
    Why not? What information do they have in the front end that can't be included in the query?

    I expect at least the availability and quite probably the price can change in real time.

    Location is probably fixed, (although I hear NZ just moved closer to Australia due to an earthquake) but I can imagine that you wouldn't be asking the user for their own location as part of the initial search parameters, only what city they want to see a list of hotels for.
    Once you have that dataset, any further filtering by distance/price/quality etc is almost certainly better done outside the DB otherwise you'd have to do another round trip to the DB every time the user changed their mind about what they are filtering on...

  • (cs) in reply to shimon
    shimon:
    ORDER BY SQUARE(hotels.x - @locationX) + SQUARE(hotels.y - @locationY) ASC

    QED (What to do if there is no SQUARE function is left as an exercise to the reader.)

    And now I have a piece of breaking news for you: the Earth, as the recent researches have shown, is not flat at all!

    And it's enough for the endpoints to be two or three states apart for cartesian distance to go from simply inaccurate to f-cking wildly inaccurate.

    Huh... and here I thought the article was talking about the results being based on a single city... Unless your city is 100 miles wide?

    As to the "street-level" distance, who the heck cares? It's not driving directions, it's just a fairly raw distance to figure out what is "closer"... 99% of the time the street directions, etc. will make even less difference than a gnat fart in a hurricane.

  • shimon (unregistered) in reply to dkf
    or going the Web2.0 way and allowing it from any point of a map of the city. Caching only works for the first way. Users want the second.

    Then you have no choice but actually purchase some data sets from those who are doing it for living.

    Another way that I see it is

    1. make your database PostgreSQL
    2. write a stored procedure DRIVING_DISTANCE in pl/Python, which will call a 3rd party and return a number
    3. still use pure SQL to get your data set

    It will add no performance, but hey, getting a huge list of location and show 10 of them bloody every time, that's kinda sick.

  • JoLoCo (unregistered) in reply to dkf
    dkf:
    shimon:
    In the city itself, calculate the crow-fly distance. In most of the cases, you will end up with fairly accurate distances.
    You must work in a midwest city yes? Lots of cities elsewhere have odd restrictions in where you can drive (one way systems, odd road blockages, you name it) which mean that the cartesian distance has nothing to do with how far it is on the ground. You might get a better estimate for some cities by treating them as a normed vector space using the Manhattan norm, but you need to know the orientation of the axes to make that work right and too often those aren't uniform across metro areas.

    Or you can do the sane thing and delegate the whole problem to some expert piece of software written by people who have to actually do this for a living. Whoops! That's outside the DB...

    Hmm good point, but usually when booking a hotel people want one that is within X miles of a certain point (city centre, concert venue, etc.), the nearer the better, right? I reckon a crow-flies radius from the desired point would work just fine for searching for a hotel. You can do the driving directions once they've selected a hotel.

Leave a comment on “Pagination Consternation”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article