The Daily WTF: Curious Perversions in Information Technology

2010-12-07 Reply Admin

Anonymous:
Anonymous:
Sutherlands:
Anonymous:
I needed the underlying SQL to comprise of the logic ((A and B) or (C and D)). That is not a complicated query by any stretch of the imagination but it is physically impossible to express with lambda statements. [...] Is that specific enough for you?
No, because
IEnumerable<Tuple<bool, bool, bool, bool>> list = new List<Tuple<bool, bool, bool, bool>>
             {new Tuple<bool, bool, bool, bool>(false, false, true, true)};
foreach(var newVal in list.Where(val => (val.Item1 && val.Item2) || (val.Item3 && val.Item4)))
{
  //Breakpoint reached
}
I must be missing something important.
You are missing something important (the keywords you missed were "build up a query sequentially") but I'm just about to go home (UK here) so bear with me and I'll follow up tomorrow.

[OK, forget about this bit, I concede I didn't read it properly and didn't realise you were just setting up some test data. I apologise, so let's not get off-topic]

But hey, it's a completely moot point because that abomination won't actually work in my scenario. Have a good evening, check back tomorrow.
OK, I'm back! My siginificant other has abandoned me in favour of her sewing machine so I've had some time to write up an example scenario for you (in my spare time, aren't I nice?!). I'm very much looking forward to you proving me wrong about lambdas because I'd love to clean up my code.

You said you must be missing something important and you were. I specifically said I needed to build up a query sequentially, although maybe I should have said dynamically. Anyway, the point is that you don't always have all your filter criteria up-front to build a single lambda statement with. Very often you need to build up a statement piecemeal based on various conditional logic. This is where lambdas fall down, because using the syntactic sugar only lets you build statements in one hit, there is no good way to progressively build up a statement as you go.

So I've produced a dramatically simplified example for you based loosely on my scenario. The table you are querying with LINQ to SQL consists of 4 columns; SourceId, SourceName, TargetId and TargetName. Your query method accepts a criteria object that has properties matching the columns, so we have Criteria.SourceId, Criteria.SourceName, Criteria.TargetId and Criteria.TargetName. Any of these may be null but any that are provided should be used to filter on the appropriate column. The critieria object contains an addtional property - Criteria.TwoWaySource. If this is true, you need to return all the rows where the source criteria values appear in either the source columns or the target columns. In other words, you need to return all the rows where ((SourceId = Critieria.SourceId [AND SourceName = Criteria.SourceName]) OR (TargetId = Criteria.SourceId [AND TargetName = Criteria.SourceName])). The bits in angle brackets are just there to indicate that you may have to additionally filter on these properties if they are provided, but as I already said, any of these properties could be null in which case that property should be ignored.

Please satisfy these requirements using lambda expressions. Don't even think of resorting to a separate "if" statement for every single possible combination of criteria states because in the real thing there are way more values than this so an if statement to cover every single permutation is totally unrealisitic, it would span thousands of lines and be completely unmaintainable. You're going to have to query each property and dynamically build up your statement so that any possible permutation generates the right SQL. This is just some pseudocode to give you an idea:
if (Criteria.SourceId != null)
{
  // SQL needs to be SELECT * FROM table WHERE SourceId = Criteria.SourceId.
  // Easy with a lambda: t => t.SourceId == Criteria.SourceId
}

if (Criteria.SourceName != null)
{
  // SQL now needs to be whatever we previously defined but with the addition of
  // of AND SourceName = Critieria.SourceName. We need to take the existing query
  // (if any) and AND it with a new query but this is not possible with lambdas.
}

// So on for TargetId and TargetName as necessary

if (Criteria.TwoWaySource)
{
  // Oh shit, source values are two way! Now we need to take the existing query, 
  // whatever the hell that is, and include OR TargetId = Criteria.SourceId,   
  // which may also need to be ANDed with TargetName = Criteria.SourceName. 
  // This can't be done with lambdas either!
}
Well, I've given you more than enough information on the requirements, let's see you achieve this using nothing but lambda expressions.

I don't know about C# lambda expressions, but in SQL you can easily handle this problem with a static query. I think this does what you want:

SELECT *
FROM table
WHERE (
COALESCE(:SourceId, SourceId) = SourceId
AND COALESCE(:SourceName, SourceName) = SourceName
)
OR (
:TwoWaySource = true
AND COALESCE(:SourceId, TargetId) = TargetId
AND COALESCE(:SourceName, TargetName) = TargetName
)

Criteria.SourceId is bound to the parameter :SourceId, and so on. If your database doesn't have COALESCE, "(:SourceId is null OR :SourceId = SourceId)" is equivalent to "COALESCE(:SourceId, SourceId) = SourceId" for this example. (NOTE: I assume all database columns are NOT NULL.)

You should be able to construct a similar query in LINQ by incorporating Criteria into the where clause. Lambda expressions are closures, right?

In practice, you should almost never need to generate a query dynamically unless you are writing a developer tool or your database sucks.

Sutherlands · 2010-12-07 Reply Admin

Anonymous:
...

I've come up with something - it's not building a query sequentially, as I have no experience with building queries that way. It is, however, a lambda expression which follow the logic that you described. If this doesn't do what you need just because you specifically need to do it piece by piece, I probably don't have the time right now to learn expression trees. However, if this is a valid way of doing things, but just slightly off the mark, let me know what's off.

var testList = new List<TestClass>
                  {
                    new TestClass{SourceId = 1, SourceName = "Name1", TargetId = 5, TargetName = "Name5"},
                    new TestClass{SourceId = 1, SourceName = "Name1", TargetId = 6, TargetName = "Name6"},
                    new TestClass{SourceId = 1, SourceName = "No Name", TargetId = 6, TargetName = "No Name"},
                    new TestClass{SourceId = 2, SourceName = "No Name", TargetId = 6, TargetName = "Name6"},
                    new TestClass{SourceId = 5, SourceName = "Name3", TargetId = 1, TargetName = "No Name"}
                  };
Criteria criteria = null;
var enumerable = from t in testList
where
(
(criteria.SourceId == null || criteria.SourceId == t.SourceId)
&&
(criteria.SourceName == null || criteria.SourceName == t.SourceName)
&&
(criteria.TargetId == null || criteria.TargetId == t.TargetId)
&&
(criteria.TargetName == null || criteria.TargetName == t.TargetName)
)
||
(
criteria.TwoWay
&&
(criteria.SourceId == null || criteria.SourceId.Value == t.TargetId)
&&
(criteria.SourceName == null || criteria.SourceName == t.TargetName)
&&
(criteria.SourceId != null || criteria.SourceName != null)
)

select t;
criteria = new Criteria(false, 1, 6, null, null);
if(enumerable.Count() != 2) throw new Exception("Test 1");
criteria = new Criteria(false, 1, 6, "Name1", null);
if (enumerable.Count() != 1) throw new Exception("Test 2");
criteria = new Criteria(true, 1, null, null, null);
if(enumerable.Count() != 4) throw new Exception("Test 3");
criteria = new Criteria(false, null, null, null, null);
if(enumerable.Count() != 5) throw new Exception("Test 4");
criteria = new Criteria(true, null, null, null, null);
if (enumerable.Count() != 5) throw new Exception("Test 5");
criteria = new Criteria(true, null, 5, null, null);
if(enumerable.Count() != 1) throw new Exception("Test 6");

2010-12-08 Reply Admin

Anonymous:
Sutherlands:
Coyne:
When designing things (such as, for example, parseint) the default behaviors should always be the thing most commonly required and/or that which is least likely to be the wrong thing.
As a C# dev, I can say that the compiler should never guess what you want.
As another C# dev, I agree with you completely. The single worst thing about C# compared to Java is the wholesale shift in mindset of the people writing the language. The Java mantra is "what you see is what you get". The C# mantra is "stuff as much compiler guesswork and syntactic sugar in there as possible, so what you see is only a tiny part of what you actually get".

Well, I don't know if you're right about C#, but never fear, Java is long since over that "what you see is what you get" phase.

Every time I get a new CDI/Weld injection error I hear a little voice in my head: "Yo Dog! I put a proxy on your proxy, so you can proxy while you proxy".

Every once in a while I look around at where JEE is heading and wonder if I'm the only one with any common sense, or if I've just lost it.

2010-12-08 Reply Admin

frits:
boog:
Alexis:
Except he went on to defend C#'s API...
Saying that something is fine is hardly a defense, but point taken. However, if I may be serious for a moment, I'd like to point out that the following comment he made deserves more attention than it's getting:
Sutherlands:
I'm curious as to what you think sucks and what your required features are.
In other words, how exactly does one go about measuring "sucky-ness" with respect to a given language's Date API (or any API for that matter)?

An API method should require as close to one line of code with very few supplemental parameters as possible for routine tasks. One line of code with one parameter and one return value (when appropriate) is optimal. The further from this ideal the suckier the API is. In other words, don't make have to construct an API for your API.

'Sup dawg, I herd U liek embedded programming, so I put a API in ur API. That way U can generate code while you program.

2010-12-08 Reply Admin

Sutherlands:
Anonymous:
...
I've come up with something - it's not building a query sequentially, as I have no experience with building queries that way. It is, however, a lambda expression which follow the logic that you described...
<Snipped some unholy abomination of a lambda that DIDN'T follow the logic I described>

Err... you're kidding me, right? Not only is this a god-awful mess of code (I only gave you 4 columns to search on, the real app has 20+ so don't think for a second this is going to scale) but you haven't addressed the requirement to generate the right underlying SQL. All I want from the SQL is to filter on the required columns in the most efficient manner. This mess generates a ridiculously convoluted SQL statement, seriously, attach a logger to your datacontext then try telling me that's acceptable SQL - my DBA would have an absolute fit if he saw this crap come up in his profiler, it would never be accepted.

What you've done is used lambdas to search an object collection which gives you way more flexibility than using lambdas in LINQ to SQL because when you're writing LINQ to SQL you need to ensure that the underlying SQL is actually efficient and doesn't include all sorts of superfluous checks. Have you ever actually used LINQ to SQL or did you simply assume you could use the same convoluted lambdas as you do with LINQ to objects? I made the LINQ to SQL requirement perfectly clear when I wrote my original missive, as too the logic I was trying to create in the underlying SQL.

And besides this, you have totally failed in another core requirement - to build up the query sequentially. I said the example was dramatically simplified and it is because in the example you have the whole criteria object up-front. In the real app you don't this luxury. This is exactly why I told you, several times, that the query had to be built up sequentially based on conditional logic. You have failed to do this and your argument is "I have no experience building expression trees this way". Er, no, you can't build expression trees this way, ie. by using lambda statements. That was the entire point of my original post and exactly the issue you were trying to contest!!! Yet now you tell me you don't actually know how to? So why the hell are you trying to contest my point when you don't even know how??!! I made the requirements perfectly clear, you obviously misunderstood them and it is obvious you were flat out wrong in your assertion that my requirements could be fulfilled with lambdas. Or maybe you weren't, but either way you have no clue how to do it.

Sorry my friend, I mean you no disrespect but the simple fact of the matter is that you have TOTALLY failed to meet my requirements and dispute my orginal point. I also think it's cute how you say "I have no experience building expression tress..." because this was exactly my original point about syntactic sugar!!! You can't physically do it with the syntactic sugar and you don't know how to do it any other way because you don't understand the underlying classes. I, on the other hand, do, which is why I can easily fulfil these requirements through direct manipulation of the underlying classes. Score one for actually understanding the classes and a big fat zero for syntactic sugar.

If you want to keep this going I'm happy to oblige but so far you've only managed to prove your ignorance. I mean no offense by that, just stating a fact.

frits · 2010-12-08 Reply Admin

I've never used lambda expression in the way you're describing, but they are quite useful for inlining anonymous functions for IO and simple multi-threading applications. Also, making use of local variables is a neat trick.

Sutherlands · 2010-12-08 Reply Admin

Hmmm, ok let's go over the conversation, and I'll dumb down how it went for you.

You: YOU CAN'T DO (A AND B) OR (C AND D) WITH LAMBDA STATEMENTS! ("That is not a complicated query by any stretch of the imagination but it is physically impossible to express with lambda statements.") Me: Yes you can (show example code). I must be missing something. You: OMG I SAID SEQUENTIALLY, AND YOUR CODE IS TEH SUX0R. BESIDES I WANT TO USE CLASSES! Me: It's test code, here's some other test code that uses classes. You: YOU NEED TO DO THIS!!! ("Please satisfy these requirements using lambda expressions") Other guy: You can do this in SQL by doing -X- Me: I don't have experience with expression trees, but here is the logic requirements -X- using lambdas You: OMG THE SQL IS HORRIBLE, DIDN'T YOU ATTACH A LOGGER FOR THIS TEST EXAMPLE CODE YOU CREATED FOR MEEEE?! YOUR CODE SUX0RS AND YOU SUX0R.

The point I was trying to contest was actually the "you can't do (A and B) or (C and D) with lambda statements." I was trying to give you a working solution using lambdas, and I have. The fact that you think they don't follow the logic requirements is unfortunate, because someone else came up with the same solution in SQL. You obviously are too proud to be on the internets, so I'll leave you be.

Anonymous:
Sorry my friend, I mean you no disrespect

Forgive me if I don't believe you.

2010-12-08 Reply Admin

Sutherlands:
<Snipped a rather childish admission of failure which seemed to completely ignore the fact that I did stated my requirements clearly and concisely from the outset>

Translation: "Your original missive was absolutely correct, these requirements cannot be fulfilled with lambdas. Sorry for wasting your time."

No problem mate, glad you learnt something today. If you ever actually figure out how to do this I'd love to hear back from you but trust me you won't, because it simply isn't possible with the syntactic sugar. Which is, of course, exactly the point I was making in my original post.

No hard feelings, have a good one.

2010-12-08 Reply Admin

Anonymous:
Sutherlands:
Anonymous:
...
I've come up with something - it's not building a query sequentially, as I have no experience with building queries that way. It is, however, a lambda expression which follow the logic that you described...
<Snipped some unholy abomination of a lambda that DIDN'T follow the logic I described>

Err... you're kidding me, right? Not only is this a god-awful mess of code (I only gave you 4 columns to search on, the real app has 20+ so don't think for a second this is going to scale) but you haven't addressed the requirement to generate the right underlying SQL. All I want from the SQL is to filter on the required columns in the most efficient manner. This mess generates a ridiculously convoluted SQL statement, seriously, attach a logger to your datacontext then try telling me that's acceptable SQL - my DBA would have an absolute fit if he saw this crap come up in his profiler, it would never be accepted.
What you've done is used lambdas to search an object collection which gives you way more flexibility than using lambdas in LINQ to SQL because when you're writing LINQ to SQL you need to ensure that the underlying SQL is actually efficient and doesn't include all sorts of superfluous checks. Have you ever actually used LINQ to SQL or did you simply assume you could use the same convoluted lambdas as you do with LINQ to objects? I made the LINQ to SQL requirement perfectly clear when I wrote my original missive, as too the logic I was trying to create in the underlying SQL.

And besides this, you have totally failed in another core requirement - to build up the query sequentially. I said the example was dramatically simplified and it is because in the example you have the whole criteria object up-front. In the real app you don't this luxury. This is exactly why I told you, several times, that the query had to be built up sequentially based on conditional logic. You have failed to do this and your argument is "I have no experience building expression trees this way". Er, no, you can't build expression trees this way, ie. by using lambda statements. That was the entire point of my original post and exactly the issue you were trying to contest!!! Yet now you tell me you don't actually know how to? So why the hell are you trying to contest my point when you don't even know how??!! I made the requirements perfectly clear, you obviously misunderstood them and it is obvious you were flat out wrong in your assertion that my requirements could be fulfilled with lambdas. Or maybe you weren't, but either way you have no clue how to do it.

Sorry my friend, I mean you no disrespect but the simple fact of the matter is that you have TOTALLY failed to meet my requirements and dispute my orginal point. I also think it's cute how you say "I have no experience building expression tress..." because this was exactly my original point about syntactic sugar!!! You can't physically do it with the syntactic sugar and you don't know how to do it any other way because you don't understand the underlying classes. I, on the other hand, do, which is why I can easily fulfil these requirements through direct manipulation of the underlying classes. Score one for actually understanding the classes and a big fat zero for syntactic sugar.

If you want to keep this going I'm happy to oblige but so far you've only managed to prove your ignorance. I mean no offense by that, just stating a fact.

If what you want is the most efficient sql, then you do not want a sql condition of where ( (SourceId = Critieria.SourceId [AND SourceName = Criteria.SourceName]) OR (TargetId = Criteria.SourceId [AND TargetName = Criteria.SourceName]) ) at all. You want to "select where the first condition ... union select where the second condition". Trust me, it will run faster. And if you're doing something that is really hard to do in Linq, why not just do dynamic sql in your database itself? It can be a reasonable solution in the few exceptional circumstances where it's actually needed. Most of the time it's used is a mis-use, but there are legitimate cases. At no point did your requirements mention that this sequential query building be done in .net at all, so why are you using the tool this way?

2010-12-08 Reply Admin

Anonymous:
Sutherlands:
<Snipped a rather childish admission of failure which seemed to completely ignore the fact that I did stated my requirements clearly and concisely from the outset>
Translation: "Your original missive was absolutely correct, these requirements cannot be fulfilled with lambdas. Sorry for wasting your time."
No problem mate, glad you learnt something today. If you ever actually figure out how to do this I'd love to hear back from you but trust me you won't, because it simply isn't possible with the syntactic sugar. Which is, of course, exactly the point I was making in my original post.

No hard feelings, have a good one.

Awww is it over already? But I just refilled my popcorn! I've been there and I feel your pain with L2S but don't be too hard on him, he's probably never used it. There's a lesson here about spouting off on things you never used but its easy to think you can just lambda it up until you realise the shit sql its coming out with underneath. We're currently porting a L2S app to L2E (don't laugh, we're using v4 at least) and you wouldn't believe the shit i've waded throguh. Not fully related but just recently I found a single query that went to the database 5 times, this is one query. The idiot that wrote it didn't realise that calling ToList on the query would send it to the DB, he was just stringing them together then wondered why it was slow as hell. All good fun.

2010-12-08 Reply Admin

Brian White:
If what you want is the most efficient sql, then you do not want a sql condition of where ( (SourceId = Critieria.SourceId [AND SourceName = Criteria.SourceName]) OR (TargetId = Criteria.SourceId [AND TargetName = Criteria.SourceName]) ) at all. You want to "select where the first condition ... union select where the second condition". Trust me, it will run faster. And if you're doing something that is really hard to do in Linq, why not just do dynamic sql in your database itself? It can be a reasonable solution in the few exceptional circumstances where it's actually needed. Most of the time it's used is a mis-use, but there are legitimate cases. At no point did your requirements mention that this sequential query building be done in .net at all, so why are you using the tool this way?

Yes, I know exactly how best to structure this query in SQL. But the point is I'm not writing the SQL directly, I'm using LINQ to SQL which is essentially an ORM layer that uses LINQ as the query language and generates the SQL for you. The only problem is structuring the LINQ so as to generate efficient SQL.

What I'm doing is not hard to express in LINQ, in fact it is very, very easy - the diffcult part is getting that LINQ to write decent SQL!! I admit it was my choice to use LINQ to SQL because I wanted to get my hands dirty with a new technology but I don't for one second think it was an inappropriate choice of technology for the job. This was a really trivial little database app that should have been a breeze to implement with LINQ to SQL. This is exactly what LINQ to SQL was designed for so why on Earth would I not expect it to be an adequate solution to the problam at hand?

2010-12-08 Reply Admin

Kaizer:
Awww is it over already? But I just refilled my popcorn! I've been there and I feel your pain with L2S but don't be too hard on him, he's probably never used it. There's a lesson here about spouting off on things you never used but its easy to think you can just lambda it up until you realise the shit sql its coming out with underneath. We're currently porting a L2S app to L2E (don't laugh, we're using v4 at least) and you wouldn't believe the shit i've waded throguh. Not fully related but just recently I found a single query that went to the database 5 times, this is one query. The idiot that wrote it didn't realise that calling ToList on the query would send it to the DB, he was just stringing them together then wondered why it was slow as hell. All good fun.

Sorry to waste your popcorn but I think we're all done here - I don't think he's coming back for another round. And thank God because I really hate arguing with the uninformed.

2010-12-08 Reply Admin

Anonymous:
Anonymous:
Sutherlands:
Anonymous:
I needed the underlying SQL to comprise of the logic ((A and B) or (C and D)). That is not a complicated query by any stretch of the imagination but it is physically impossible to express with lambda statements. [...] Is that specific enough for you?
No, because
IEnumerable<Tuple<bool, bool, bool, bool>> list = new List<Tuple<bool, bool, bool, bool>>
             {new Tuple<bool, bool, bool, bool>(false, false, true, true)};
foreach(var newVal in list.Where(val => (val.Item1 && val.Item2) || (val.Item3 && val.Item4)))
{
  //Breakpoint reached
}
I must be missing something important.
You are missing something important (the keywords you missed were "build up a query sequentially") but I'm just about to go home (UK here) so bear with me and I'll follow up tomorrow.

[OK, forget about this bit, I concede I didn't read it properly and didn't realise you were just setting up some test data. I apologise, so let's not get off-topic]

But hey, it's a completely moot point because that abomination won't actually work in my scenario. Have a good evening, check back tomorrow.
OK, I'm back! My siginificant other has abandoned me in favour of her sewing machine so I've had some time to write up an example scenario for you (in my spare time, aren't I nice?!). I'm very much looking forward to you proving me wrong about lambdas because I'd love to clean up my code.

You said you must be missing something important and you were. I specifically said I needed to build up a query sequentially, although maybe I should have said dynamically. Anyway, the point is that you don't always have all your filter criteria up-front to build a single lambda statement with. Very often you need to build up a statement piecemeal based on various conditional logic. This is where lambdas fall down, because using the syntactic sugar only lets you build statements in one hit, there is no good way to progressively build up a statement as you go.

So I've produced a dramatically simplified example for you based loosely on my scenario. The table you are querying with LINQ to SQL consists of 4 columns; SourceId, SourceName, TargetId and TargetName. Your query method accepts a criteria object that has properties matching the columns, so we have Criteria.SourceId, Criteria.SourceName, Criteria.TargetId and Criteria.TargetName. Any of these may be null but any that are provided should be used to filter on the appropriate column. The critieria object contains an addtional property - Criteria.TwoWaySource. If this is true, you need to return all the rows where the source criteria values appear in either the source columns or the target columns. In other words, you need to return all the rows where ((SourceId = Critieria.SourceId [AND SourceName = Criteria.SourceName]) OR (TargetId = Criteria.SourceId [AND TargetName = Criteria.SourceName])). The bits in angle brackets are just there to indicate that you may have to additionally filter on these properties if they are provided, but as I already said, any of these properties could be null in which case that property should be ignored.

Please satisfy these requirements using lambda expressions. Don't even think of resorting to a separate "if" statement for every single possible combination of criteria states because in the real thing there are way more values than this so an if statement to cover every single permutation is totally unrealisitic, it would span thousands of lines and be completely unmaintainable. You're going to have to query each property and dynamically build up your statement so that any possible permutation generates the right SQL. This is just some pseudocode to give you an idea:
if (Criteria.SourceId != null)
{
  // SQL needs to be SELECT * FROM table WHERE SourceId = Criteria.SourceId.
  // Easy with a lambda: t => t.SourceId == Criteria.SourceId
}

if (Criteria.SourceName != null)
{
  // SQL now needs to be whatever we previously defined but with the addition of
  // of AND SourceName = Critieria.SourceName. We need to take the existing query
  // (if any) and AND it with a new query but this is not possible with lambdas.
}

// So on for TargetId and TargetName as necessary

if (Criteria.TwoWaySource)
{
  // Oh shit, source values are two way! Now we need to take the existing query, 
  // whatever the hell that is, and include OR TargetId = Criteria.SourceId,   
  // which may also need to be ANDed with TargetName = Criteria.SourceName. 
  // This can't be done with lambdas either!
}
Well, I've given you more than enough information on the requirements, let's see you achieve this using nothing but lambda expressions.

This is something I fixed up. I know there's probably a bug with something. I have not TESTED it, and it's 3 AM so I wont.

I'm 100% sure there's a better solution for this... I'm just too tired.

Anyway, I've created two methods, one with a union and one with a Where clause. I've got a list of Criterias (yeah...) and I'm checking it with SomeCriteria.

public class Criteria
    {
            public int? SourceID { get; set; }
            public string SourceName { get; set; }
            public int? TargetId { get; set; }
            public string TargetName { get; set; }
    }

    public class stuff
    {
        public IQueryable<Criteria> getQueryWhereSolution(Criteria someCriteria)
        {
            IQueryable<Criteria> listOfStuff = new List<Criteria>().AsQueryable();
            Func<Criteria, bool> WhereClause = null;

            if (someCriteria.SourceID.HasValue || !string.IsNullOrEmpty(someCriteria.SourceName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.SourceID.HasValue || c.SourceID == someCriteria.SourceID)
                             && (someCriteria.SourceName == null || c.SourceName == someCriteria.SourceName);
                };
                WhereClause = p;
            }

            if (someCriteria.TargetId.HasValue || !string.IsNullOrEmpty(someCriteria.TargetName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.TargetId.HasValue || c.TargetId == someCriteria.TargetId)
                            && (someCriteria.TargetName == null || c.TargetName == someCriteria.TargetName);
                };
                if (WhereClause == null)
                {
                    WhereClause = p;
                }
                else
                {
                    WhereClause = (Criteria c) => { return WhereClause(c) || p(c); };
                }
            }

            if (WhereClause != null)
            {
                listOfStuff = listOfStuff.Where(s => WhereClause(s));
            }

            return listOfStuff;
        }

        public IQueryable<Criteria> getQueryUnionSolution(Criteria someCriteria)
        {
            var OrigOfStuff = new List<Criteria>();
            IQueryable<Criteria> query = new List<Criteria>(OrigOfStuff).AsQueryable();
            List<Func<Criteria, bool>> unionClauses = new List<Func<Criteria, bool>>();

            if (someCriteria.SourceID.HasValue || !string.IsNullOrEmpty(someCriteria.SourceName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.SourceID.HasValue || c.SourceID == someCriteria.SourceID)
                             && (someCriteria.SourceName == null || c.SourceName == someCriteria.SourceName);
                };
                unionClauses.Add(p);
            }

            if (someCriteria.TargetId.HasValue || !string.IsNullOrEmpty(someCriteria.TargetName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.TargetId.HasValue || c.TargetId == someCriteria.TargetId)
                            && (someCriteria.TargetName == null || c.TargetName == someCriteria.TargetName);
                };
                unionClauses.Add(p);
            }

            if (unionClauses.Count > 0)
            {
                query = query.Where(a => unionClauses[0](a));
                for (int i = 1; i < unionClauses.Count; i++)
                {
                    query = query.Union(OrigOfStuff.Where(a => unionClauses[i](a)));
                }
            }
            return query;
        }
    }

BTW There is a way to dynamically combine Labmdas.

You can use something like Func<T,TResult>.Combine(Func1,Func2), I'm not sure how it'll work out though, it'll probably blow up.

You can also try something like this:

Func<Something,bool> f1= (Something s) => s.foo == bar Func<Something,bool> f2= (Something s) => f1(s) && s.bar == baz

Hope this was a helpful first post...

2010-12-09 Reply Admin

linkgoron:

This is something I fixed up. I know there's probably a bug with something. I have not TESTED it, and it's 3 AM so I wont.

I'm 100% sure there's a better solution for this... I'm just too tired.

Anyway, I've created two methods, one with a union and one with a Where clause. I've got a list of Criterias (yeah...) and I'm checking it with SomeCriteria.

public class Criteria
    {
            public int? SourceID { get; set; }
            public string SourceName { get; set; }
            public int? TargetId { get; set; }
            public string TargetName { get; set; }
    }

    public class stuff
    {
        public IQueryable<Criteria> getQueryWhereSolution(Criteria someCriteria)
        {
            IQueryable<Criteria> listOfStuff = new List<Criteria>().AsQueryable();
            Func<Criteria, bool> WhereClause = null;

            if (someCriteria.SourceID.HasValue || !string.IsNullOrEmpty(someCriteria.SourceName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.SourceID.HasValue || c.SourceID == someCriteria.SourceID)
                             && (someCriteria.SourceName == null || c.SourceName == someCriteria.SourceName);
                };
                WhereClause = p;
            }

            if (someCriteria.TargetId.HasValue || !string.IsNullOrEmpty(someCriteria.TargetName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.TargetId.HasValue || c.TargetId == someCriteria.TargetId)
                            && (someCriteria.TargetName == null || c.TargetName == someCriteria.TargetName);
                };
                if (WhereClause == null)
                {
                    WhereClause = p;
                }
                else
                {
                    WhereClause = (Criteria c) => { return WhereClause(c) || p(c); };
                }
            }

            if (WhereClause != null)
            {
                listOfStuff = listOfStuff.Where(s => WhereClause(s));
            }

            return listOfStuff;
        }

        public IQueryable<Criteria> getQueryUnionSolution(Criteria someCriteria)
        {
            var OrigOfStuff = new List<Criteria>();
            IQueryable<Criteria> query = new List<Criteria>(OrigOfStuff).AsQueryable();
            List<Func<Criteria, bool>> unionClauses = new List<Func<Criteria, bool>>();

            if (someCriteria.SourceID.HasValue || !string.IsNullOrEmpty(someCriteria.SourceName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.SourceID.HasValue || c.SourceID == someCriteria.SourceID)
                             && (someCriteria.SourceName == null || c.SourceName == someCriteria.SourceName);
                };
                unionClauses.Add(p);
            }

            if (someCriteria.TargetId.HasValue || !string.IsNullOrEmpty(someCriteria.TargetName))
            {
                Func<Criteria, bool> p;
                p = (Criteria c) =>
                {
                    return (someCriteria.TargetId.HasValue || c.TargetId == someCriteria.TargetId)
                            && (someCriteria.TargetName == null || c.TargetName == someCriteria.TargetName);
                };
                unionClauses.Add(p);
            }

            if (unionClauses.Count > 0)
            {
                query = query.Where(a => unionClauses[0](a));
                for (int i = 1; i < unionClauses.Count; i++)
                {
                    query = query.Union(OrigOfStuff.Where(a => unionClauses[i](a)));
                }
            }
            return query;
        }
    }

BTW There is a way to dynamically combine Labmdas.

You can use something like Func<T,TResult>.Combine(Func1,Func2), I'm not sure how it'll work out though, it'll probably blow up.

You can also try something like this:

Func<Something,bool> f1= (Something s) => s.foo == bar Func<Something,bool> f2= (Something s) => f1(s) && s.bar == baz

Hope this was a helpful first post...

Hi linkgoron and welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas, as long as everybody keeps their head (fair warning: most people don't!!). Thanks very much for your input, I appreciate your taking the time to put this together. I haven't had the opportunity to try out some of your ideas but I think in many respects you've come to a similar solution that the one I ended up with.

I notice you're working directly with predicates in the form Func<T, bool>. This is similar to the approach I came up with (using some guidance from the excellent PredicateBuilder class, google it if you want to take a look). By working directly with predicates and the Expression class, I was able to quite easily create AND and OR relationships between existing queries, similar to this:

// Start with an empty expression for both src and tgt that just returns 
// everything (because they're always true)
Expression<Func<QueryObj, bool>> exprSrc = (t => true);
Expression<Func<QueryObj, bool>> exprTgt = (t => true);

// Build up the expressions based on the criteria
if (!Criteria.SourceId == null)
{
  // AND the existing expression with this criteria
  exprSrc = this.LambdaAnd(exprSrc, t => t.SourceId == Criteria.SourceId;

  // If TwoWaySource is true we need to stick this value in exprTgt as well
  if (Criteria.TwoWaySource)
  {
    exprTgt = this.LambdaAnd(exprTgt, t => t.TargetId == Criteria.SourceId
  }
}

// Do the same for all the other criteria

if (Critieria.TwoWaySource)
{
  // Source is two way, so now we need to OR the exprSrc and exprTgt expressions
  exprSrc = this.LambdaOr(exprSrc, exprTgt);
}

// Define a query that selects all objects from the underlying datasource
var query = from t in QueryObjDataContext
            select t;

// Filter the query based on the expression we just defined
query = query.Where(exprSrc);

// Our query is now fully defined so execute it against the DB
List<QueryObj> queriedObjects = query.ToList();

In the above code you'll notice that the actual magic is performed by the LambdaAnd and LambdaOr methods, which are as follows:

private Expression<Func<T, bool>> LambdaAnd<T>(Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
    var invokedExpr = Expression.Invoke(expr2, expr1.Parameters.Cast<Expression>());
    return Expression.Lambda<Func<T, bool>>(Expression.AndAlso(expr1.Body, invokedExpr), expr1.Parameters);
}

private Expression<Func<T, bool>> LambdaOr<T>(Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
    var invokedExpr = Expression.Invoke(expr2, expr1.Parameters.Cast<Expression>());
    return Expression.Lambda<Func<T, bool>>(Expression.OrElse(expr1.Body, invokedExpr), expr1.Parameters);
}

So, even though we need to get down and dirty with the class that underlies lambda expressions (that class is Expression<Func<T, bool>>) it's actually a very simple means of building up AND and OR relationships between existing queries. I think you were coming from a similar direction but you can make your code even neater if you look into the Expression<Func<T, bool>> class rather than working simply with Func<T, bool>. The Expression class contains a whole wealth of methods for creating pretty much any logic you want.

Thanks again, I look forward to seeing more from you in TDWTF comments.

boog · 2010-12-09 Reply Admin

Anonymous:
...welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas...

I suppose I can agree with that, as long as nobody claims that any of these ideas are intelligent, thought-out, or useful in any way.

2010-12-09 Reply Admin

boog:
Anonymous:
...welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas...
I suppose I can agree with that, as long as nobody claims that any of these ideas are intelligent, thought-out, or useful in any way.

Well obviously, people like Sutherlands prove that fact.

2010-12-09 Reply Admin

Anonymous:
Brian White:
If what you want is the most efficient sql, then you do not want a sql condition of where ( (SourceId = Critieria.SourceId [AND SourceName = Criteria.SourceName]) OR (TargetId = Criteria.SourceId [AND TargetName = Criteria.SourceName]) ) at all. You want to "select where the first condition ... union select where the second condition". Trust me, it will run faster. And if you're doing something that is really hard to do in Linq, why not just do dynamic sql in your database itself? It can be a reasonable solution in the few exceptional circumstances where it's actually needed. Most of the time it's used is a mis-use, but there are legitimate cases. At no point did your requirements mention that this sequential query building be done in .net at all, so why are you using the tool this way?
Yes, I know exactly how best to structure this query in SQL. But the point is I'm not writing the SQL directly, I'm using LINQ to SQL which is essentially an ORM layer that uses LINQ as the query language and generates the SQL for you. The only problem is structuring the LINQ so as to generate efficient SQL.
What I'm doing is not hard to express in LINQ, in fact it is very, very easy - the diffcult part is getting that LINQ to write decent SQL!! I admit it was my choice to use LINQ to SQL because I wanted to get my hands dirty with a new technology but I don't for one second think it was an inappropriate choice of technology for the job. This was a really trivial little database app that should have been a breeze to implement with LINQ to SQL. This is exactly what LINQ to SQL was designed for so why on Earth would I not expect it to be an adequate solution to the problam at hand?

If you know the right way to do this, why don't you use the right tool? LINQ to SQL is not the right tool for dynamically built sql with variable conditions.

boog · 2010-12-09 Reply Admin

Anonymous:
boog:
Anonymous:
...welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas...
I suppose I can agree with that, as long as nobody claims that any of these ideas are intelligent, thought-out, or useful in any way.
Well obviously, people like Sutherlands prove that fact.

And yet he wasn't the one who called this forum a "great place to flesh out ideas."

I'm just sayin'.

frits · 2010-12-09 Reply Admin

boog:
Anonymous:
boog:
Anonymous:
...welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas...
I suppose I can agree with that, as long as nobody claims that any of these ideas are intelligent, thought-out, or useful in any way.
Well obviously, people like Sutherlands prove that fact.
And yet he wasn't the one who called this forum a "great place to flesh out ideas."
I'm just sayin'.

I'm just here for the flesh.

2010-12-09 Reply Admin

Brian White:
Anonymous:
Brian White:
If what you want is the most efficient sql, then you do not want a sql condition of where ( (SourceId = Critieria.SourceId [AND SourceName = Criteria.SourceName]) OR (TargetId = Criteria.SourceId [AND TargetName = Criteria.SourceName]) ) at all. You want to "select where the first condition ... union select where the second condition". Trust me, it will run faster. And if you're doing something that is really hard to do in Linq, why not just do dynamic sql in your database itself? It can be a reasonable solution in the few exceptional circumstances where it's actually needed. Most of the time it's used is a mis-use, but there are legitimate cases. At no point did your requirements mention that this sequential query building be done in .net at all, so why are you using the tool this way?
Yes, I know exactly how best to structure this query in SQL. But the point is I'm not writing the SQL directly, I'm using LINQ to SQL which is essentially an ORM layer that uses LINQ as the query language and generates the SQL for you. The only problem is structuring the LINQ so as to generate efficient SQL.
What I'm doing is not hard to express in LINQ, in fact it is very, very easy - the diffcult part is getting that LINQ to write decent SQL!! I admit it was my choice to use LINQ to SQL because I wanted to get my hands dirty with a new technology but I don't for one second think it was an inappropriate choice of technology for the job. This was a really trivial little database app that should have been a breeze to implement with LINQ to SQL. This is exactly what LINQ to SQL was designed for so why on Earth would I not expect it to be an adequate solution to the problam at hand?

If you know the right way to do this, why don't you use the right tool? LINQ to SQL is not the right tool for dynamically built sql with variable conditions.

I have to disagree with this because I've done it!!! Not only have I done it, but I was able to do it efficiently in only a handful of lines of code (if you want the specifics please refer to my previous reply to linkgoron, it includes code samples). So no, obviously it is not the wrong tool for the job. All that was wrong for the job was the syntactic sugar that Microsoft have sprinkled over the top of LINQ, which is lambda statements. This was the very point of my original post, that syntactic sugar limits the developer whereas understanding and using the underlying classes gets the job done simply and effectively. My scenario demonstrates this with crystal clarity.

I really can't make it any clearer for you. Using an ORM over a SQL database is exactly the right tool for the job of creating a simple SQL database application. It is exactly what they're designed for!!! LINQ to SQL was the right tool for the job and it truly worked wonders, as soon as I dropped the syntactic sugar and got to grips with the actual guts of LINQ.

boog · 2010-12-09 Reply Admin

Brian White:
If you know the right way to do this, why don't you use the right tool? LINQ to SQL is not the right tool for dynamically built sql with variable conditions.

To be fair, he was just experimenting with LINQ to SQL for the experience.

Still, thank you. I was waiting for someone to point it out. Getting your hands dirty for experience is one thing, but in the end if you know how to write it clearly in a few lines of SQL, just do that. Developers that inherit your code will thank you for it.

2010-12-09 Reply Admin

boog:
Anonymous:
boog:
Anonymous:
...welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas...
I suppose I can agree with that, as long as nobody claims that any of these ideas are intelligent, thought-out, or useful in any way.
Well obviously, people like Sutherlands prove that fact.
And yet he wasn't the one who called this forum a "great place to flesh out ideas."

Of course he wasn't, he bailed out as soon as he realised he was hopelessly wrong.

But hey, I was only being sarcastic, I'm sure he's a pretty smart guy. Ha, there I go again! But seriously folks, we all know it's implied that nothing good ever comes out of these comments...

2010-12-09 Reply Admin

boog:
Brian White:
If you know the right way to do this, why don't you use the right tool? LINQ to SQL is not the right tool for dynamically built sql with variable conditions.
To be fair, he was just experimenting with LINQ to SQL for the experience.
Still, thank you. I was waiting for someone to point it out. Getting your hands dirty for experience is one thing, but in the end if you know how to write it clearly in a few lines of SQL, just do that. Developers that inherit your code will thank you for it.

If my experience with developers is anything to go by (I am one and I lead a team of them so that must count for something) they will NOT appreciate inheriting anything that requires knowledge of SQL or general DB admin skills. The only reason I got this assignment in the first place was because I was the only one willing to go anywhere near a SQL Server DB ("that's the DBA's job!"). This in mind, it seemed prudent to check out this flashy new ORM layer that MS have been touting. It seemed like valuable experience to gain and it was.

I also have to stress once again that LINQ to SQL worked a treat, at no time have I denigrated it because it did the job perfectly - as soon as I dropped the syntactic sugar and started working with the underlying LINQ classes. I know my team will be able to inherit this code and go straight to work on it, something they couldn't readily do if half of it was coded as SQL in the underlying database (yeah I know, I need a better team, but when life gives you lemons...).

boog · 2010-12-09 Reply Admin

Anonymous:
...they will NOT appreciate inheriting anything that requires knowledge of SQL...I was the only one willing to go anywhere near a SQL Server DB ("that's the DBA's job!").

The developers you've worked with certainly have a flawed understanding of a DBA's responsibilities. That is, compared to my own education and experience.

Anonymous:
I also have to stress once again that LINQ to SQL worked a treat...

Fair enough, stress away. Nowhere did I say that LINQ to SQL wouldn't work like a treat.

Still, I don't see your justification, other than purely personal taste on the part of your team (and for your own experience).

Anonymous:
I know my team will be able to inherit this code and go straight to work on it, something they couldn't readily do if half of it was coded as SQL in the underlying database (yeah I know, I need a better team, but when life gives you lemons...).

Ok, you've convinced me that my comment about developers being happy to inherit SQL was wrong. Also, I'm glad you can admit there's something wrong with a team of developers working on an application that queries a database refusing to write code that actually queries a database.

2010-12-09 Reply Admin

Ale:
A Date API that sucks. Well, every Date API known do man sucks. Please, prove me wrong.

No, you are correct. But Date APIs only suck because dates themselves (as implemented by mankind) suck.... plus the date systems tend to get changed every few years (to adjust for leap-seconds or the king's birthday or whatnot), instantly making all existing Date/time API code buggy.

The only proper way to implement date/time is femtoseconds-since-big-bang (uint128). Everything else is a fragile hack.

2010-12-09 Reply Admin

nB:
FWIW we have an intentional WTF in one of our libs. It's in there specifically to see if anyone ever actually looks at our code. So far, no one has commented/asked/posted it here. (it's along the lines of the above mentioned code, in that it works fine but is damn awkward.)

I have the same thing in some security code I maintain, it's a fairly obvious comment on a potential gaping security hole in the code, inserted to see if any of the "I only trust security code if it's OSS" crowd will notice. So far no-one has, but then it's only been twelve years...

2010-12-09 Reply Admin

stinkaaaaaa ffs yea.

2010-12-09 Reply Admin

BentFranklin:
Suckiness is measured in inches of mercury.

No, it's measured in milli-MUMPS.

2010-12-10 Reply Admin

frits:
boog:
Anonymous:
boog:
Anonymous:
...welcome to the heady world of TDWTF comments! It can be a bit tumultuous at times but it's a great place to flesh out ideas...
I suppose I can agree with that, as long as nobody claims that any of these ideas are intelligent, thought-out, or useful in any way.
Well obviously, people like Sutherlands prove that fact.
And yet he wasn't the one who called this forum a "great place to flesh out ideas."
I'm just sayin'.

I'm just here for the flesh.

I'm just here for the ambience. This is the venue for the wine-and-cheese isn't it?

2010-12-12 Reply Admin

on some older versions of PHP, it actually was that bad:

time php -r 'var_dump(strtotime("NOW + 40000000000 DAYS"));' bool(false)

real 0m14.337s user 0m14.337s sys 0m0.000s

2010-12-13 Reply Admin

Don't laugh. I can actually do that. My home PC is a ten year old Compaq.

2010-12-13 Reply Admin

Rex:
Don't laugh. I can actually do that. My home PC is a ten year old Compaq.

Dammit! This was supposed to be a response to the guy who was listening to his pc! WTF!

silverpie · 2010-12-16 Reply Admin

Well, for Persian, you'll just need to tweak the month lengths. Ethiopic will require a 5DayWheel, and depending on what variant of Saka you need, you might have to have a 32DayWheel.

</deadpan>

2010-12-24 Reply Admin

No, he bailed the second you continued to treat him like shit and thus demonstrated that your value was so low that to continue interacting with you would only serve to drag him down with you.

You say no hard feelings, but your entire comment was making fun of him. And now he has the upper hand, since he can easily claim that he didn't continue to help you not because of lack of knowledge but because you are a shitty person who doesn't deserve the help.

Merry Christmas, asshole.

2012-07-17 Reply Admin

Dom:
What's the fuss with that parseInt() replacement? The default implementation tries to guess the base - and ends up with treating numbers as octals occasionally.

This is TRWTF. Even in programming a number is MUCH more likely to be base 10 than base 8, and should be treated as such unless clearly specified (no, a leading 0 does not count as clearly specified).

NO I DON'T CARE IF THE LAST MESSAGE IN THIS THREAD IS FROM 2010 FUCK THE POLICE

Confessions: rachetingDateObject

Leave a comment on “Confessions: rachetingDateObject”