• AzureDiamond (unregistered)

    using await instead of just Task.Run in the main function would make it run one task and then the other. the right way to do this in parallel is something like Task.WaitAll(ProcessLogQueue(), ProcessFilesInFolder());

  • L (unregistered) in reply to AzureDiamond

    except neither of those functions are async. they used Task.Run instead of just spawning actual threads

  • (nodebb)

    We first try and load a config file, and if we fail, we quit.

    No. We try to load the file. (If we failed, we clearly didn't try and load, because we didn't load.)

    Then, if the word "Test" appears in any of the CLI arguments, we run a test of some kind, and then quit

    Or "test" or "tEsT" (etc.) because it's a case insensitive comparison (the "true" at the end ends up in a parameter called "ignoreCase"(1)...). And it's not "appears in", but "is exactly" (subject to the vagaries of what "exactly" means for a case-insensitive comparison).

    (1) See e.g. https://learn.microsoft.com/en-us/dotnet/api/system.string.compare?view=net-10.0#system-string-compare(system-string-system-string-system-boolean)

  • (nodebb)

    The customer refuses to run the service, because they ran it and it created duplicate entries. Now they don't trust it to work correctly.

    I wish all my customers were that smart.

  • (nodebb) in reply to Steve_The_Cynic

    No. We try to load the file.

    I see that you & I have matching pet peeves.

  • Jaloopa (unregistered) in reply to dpm

    I see that you & I have matching pet peeves.

    Missed a perfect chance to annoy other pendants with slightly different pet peeves by saying "me and you"

  • (nodebb) in reply to Jaloopa

    I thought of that but couldn't bring myself to do it. James Dean, I am not.

  • (nodebb)

    Mixing traditional threading (Thread.Sleep) with parallel executions (Task) which enqueue via a safe execution and synchronization context into a thread pool is a super bad idea. Especially with stuff like Sleep or Abort you can literally can deadlock or block other tasks, so this code is pretty much a time bomb if it hasn't happened multiple times already.

    Addendum 2026-01-26 09:01: Personally I don't understand what is so hard about this.

    • If it's traditional threading, do traditional threading.
    • If it's parallel processing, do parallel processing.
    • If it's asynchronous processing, do asynchronous processing.
    • If it's traditional asynchronous processing, throw it away and write asynchronous processed code from scratch.

    It's pretty easy, as long as you know the basics of what you are doing.

  • (author) in reply to L

    You can pass Task objects to the WaitAll.

  • (nodebb)

    It doesn't keep the filehandle open, which I'd make a comment about the performance overhead

    I notice that the file handle is not kept open and that before every write there is a check for whether the file exists. My psychic WTF detector says that there is a separate process which moves the log file from "logFile" to "logFile_yesterday" every day at midnight. Succeeds almost always.

  • xtal256 (unregistered) in reply to sibtrag

    No because they append the date to the current log file.

  • Foo AKA Fooo (unregistered) in reply to dpm

    Come on, I'm sure you could of.

  • (nodebb)

    The correct answer how to fix the logging is use Serilog.File BTW. There's so much wrong with how they approach logging, no point in reinventing the wheel. Fixing it would take weeks.

  • Gilbert (unregistered)

    FULLZ UPDATED 2026 USA UK CANADA SSN NIN SIN INFO with ADDRESS DL Photos front & back with Selfie Passport Photos IT|SP|RUS|AUS|BR|FR amny Countries DL photos available

    Children FUllz USA 2011-2023 Young & Old age FUllz 1930-2010 High CS Pros 700+ Comapny EIN Business Fullz LLC EIN Docs with DL Dead Fullz CC with CVV & Billin Address

    NIN Fullz with Address NIN Fullz with address Sort Code & Account number NIN UK Fullz with DL UK DL photos front back with Selfie UK Passport Photos UK CC fullz

    SIN Fullz with Address Canada DL Photos Front Back with Selfie CA Passpoprt Photos CA Email & Phone Number Active Leads Live CA Fullz

    Germany|Spain|Australia Fullz with Address & DOB Email Leads (Forex, Crypto, Casino, Investors, CEO's, Crypto, Crypto Exchanges) Sweep Stakes Active Combos & B2B Leads

    Tools & Tutorials available Carding Cash Out Scripting Spa--mming SMTP RDP C-panels Shells Web-Mailers SMS & Email Bulk Senders Look-Up Tutorials

    Telegram@killhacks - @ leadsupplier What's App - (+1) 727..788..6129 TG Channel - t.me/leadsproviderworldwide Discord - @ leads.seller VK Messenger - @ leadsupplier Signal - @ killhacks.90 Zangi - 17-7369-4210 Email - exploit.tools4u at gmail dot com https://about.me/gilberthong

    Many Other Stuff available in our shop Active & Live Fullz with guarantee Providing Replacements if anything found invalid Available 24/7

    #fulz #leads #emailleads #sweepstakes #cryptoleads #casinoleads #ssnleads #dlphoto #usaleads #canadaleads #fullzusa #fullzuk #whatsapp #facebook #activeleads #ccshop #cvvdumps #usadocs #highcreditscorefullz #eincompanydocs #kycstuff #infokyc #validfullz #validvendor

  • (nodebb) in reply to Steve_The_Cynic

    We first try and load a config file, and if we fail, we quit.

    No. We try to load the file. (If we failed, we clearly didn't try and load, because we didn't load.)

    It might be a short circuiting "and".

  • Craig (unregistered) in reply to L

    Task.Run returns a Task which can be awaited. It's inherently async if you want it to be.

    I agree with the initial suggestion; you can also add things like cancellation tokens to support halting long-running tasks. Also, for something like this, I would favor (and have favored) a BlockingCollection which will wait when empty until something lands, which is much better than doing your own waits.

  • Jessica (unregistered)

    Just to confirm, I immediately fixed the logging by using Serilog.File. Then threw all the code away and used the nice new modern way of writing a long running service from a console app, using proper Async processes and memory caching and actually checking the API for existing records.... It was a fun little project and I got to play with more modern code before going back to the main WinForms stuff.

    The main issues the client had was the fact that they couldn't run the process on multiple machines because of the file cache. But when we dug into why they wanted to run on multiple machines, it was because the app would just silently die once it was finished, or hang if the machine lock screen came on. Someone was running it and then monitoring their machine for the next 6 hours to make sure it didn't lock. They wanted to be able to split up the csv files and have multiple people running the app to process it. Then it would be 2 people tied up for 3 hours keeping their machines awake. It was a wonderful feeling being able to pass over an app that worked the way it was intended (I found the original Jira that described what they wanted - didn't base my rewrite on what it actually did!).

Leave a comment on “Threading the Needle in a Haystack of Files”

Log In or post as a guest

Replying to comment #690534:

Log Out

« Return to Article