- Feature Articles
- CodeSOD
- Error'd
- Forums
-
Other Articles
- Random Article
- Other Series
- Alex's Soapbox
- Announcements
- Best of…
- Best of Email
- Best of the Sidebar
- Bring Your Own Code
- Coded Smorgasbord
- Mandatory Fun Day
- Off Topic
- Representative Line
- News Roundup
- Editor's Soapbox
- Software on the Rocks
- Souvenir Potpourri
- Sponsor Post
- Tales from the Interview
- The Daily WTF: Live
- Virtudyne
Admin
select * from death
Admin
Yes, placing the pages inside the same barn would be really helpful!
Admin
Something to do with fraud detection, maybe? We have a query about two pages long that chews recursively through member accounts and activities doing multiple deep joins and counts to find webs of collusion / fraudulent activity, and assign scores to each potential network.
Admin
Admin
Use htsql to collapse the query. htsql is a 5th generation language, an abstraction on top of SQL. http://htsql.org
Admin
Pfft... right here in front of me sits a query with 25 unions in it. It is ~1800 lines long, making it 37 printed pages in Calibri 10.
Beat that.
Admin
I'm looking at my monster query here: 92 joins over 25 unions.
Admin
Really nasty piece of SQL code, definitely not for human-based processing. What do you think about tools that may decipher and visualize such complex queries in a more structured way, like DBClarity (http://www.microgen.com/dbclarity/)? Have you been using something similar recently?
Admin
Edit with TABs and LFs... half work done.
Admin
you're probably right in that no human wrote that
Admin
What was he coding: the world?
Admin
This must be a generated Query from MS Access.
Admin
I knew a COBOL programmer who wrote a major application program that used DMS1100 and which had one period after the period at the end of the PROCEDURE DIVISION. header.
No, unfortunately I don't have a copy of the code-list.
Admin
the his
Admin
6 pages with lots of formatting and whitespace is ok.
6 pages with no line breaks, formatting or whitespace is blurgh.
Admin
Probably took longer to run than debug
Admin
Try with DISTINCT or change all for a "HELLO WORLD" (as many Microsoft good solutions)
Admin
Oh. Dear. FAIL.
You wrote a query to extract a ton of data, most of which you weren't interested in, and sent it over the wire (slow), in order to run your own unproven search algorithm against it.
You have in one moment;
Removed the ability of the solution to scale. The amount of data sent across the wire will determine the fastest speed of the query - and as that base set of 'relevant data' likely grows as a multiple of the desired result set, that query is likely slowing down.
Removed the ability of the solution to scale. If the data exceeds your available RAM allocation, you're now writing to disk anyway. A disk on a web app platform that is probably already busy enough. Even slower.
Removed the ability of the solution to scale. That 'relevant data' set is being sent over the wire every time the query is run. A web app you say? I hope you won't be surprised when your user base grow and your network grinds to a halt. Really slow.
Removed the ability of the solution to scale. A good SQL engine optimises it's query execution plan according to the underlying data. It will use indexes when it makes sense, cache results when it makes sense, rewrite sub-queries into joins when it makes sense. As the underlying dataset changes, so does the optimisation. You're now at the mercy of your algorithm, your memory allocation system, your network bandwidth.
I suggest you read the "Art of SQL" and get a clue. There were obviously some things wrong with the system before you touched it (eg schema flaws, or sql engine configurations, either of which could prevent optimisation), but you didn't fix them. You applied a band aid that will fail.
Some people are best not allowed to do anything with a relational database.
Admin
All the fields from every joined table are written out in the SELECTs. Temporarily simplifying that to check out the WHERE joins or qualifiers would be useful in debugging.
Admin
I couldn't agree more. I was debugging an app once (it was taking 8 hours to pull up a weeks payroll, and 50% of the time it failed) and isolated the problem down to a query. The query looked something like this printout.
When I requested time to study the query, I was told the query could not be the problem because the guy that wrote it had been a genius. As I studied the query, I realized the problem was that his "hand crafted" query, had been pulling all the records from the database, running them through a Java filter, and writing them back to the database. I rewrote the query in my spare time (with a where clause to only look at the current pay period), it was a fraction of the size, formatted to be readable: it took less than 2 seconds to run. I quit the next day with the statement "filter early, filter often".
When I found out the original author was working for my prior employer, I phoned them and recommended immediate termination.
Admin
If he is able to find the line of code causing the error I would have to say he has a unique skill that is very marketable.
Admin
Liz, is that you?
Admin
Last place I worked had a 76 page long stored procedure.... added one entry to the database. Not computer generated, but rather, shoddy offshore contractor generated.
Admin
That is amazing. I am sure we will never know how that came to be.
Admin
That's an SQL query, which should be declarative - in other words describe what you want to get out of the system. They are usually short. This query seems to go over 10 foolscap pages. In short, it's a nightmare - so bad that I wonder how the query engine even parsed it, let alone ran it...
Admin
Well at least the pages are numbered. I rarely use abbreviations but for this I make the exception. OMG
Admin
It is fun way to demonstate to outsiders how tough your work is.
As employer who debug such queries almost every day I can say only "Deal with it!"
Admin
Looks bad and I feel sorry for him. Such things usually end up on my desk too. I would start to get some structure into it by using a SQL formatting tool. If he is lucky there might be a data model that helps to make some sense out of the join conditions. Slowly and systematically break it up into chunks. Not an easy and sexy task but achievable. Maybe the error message gives a hint too.
Admin
Query shmeery. I want to see the execution plan.