• I'm not a robot (unregistered)
    It's certainly the worst imaginable way to copy
    I see someone lacks imagination....
  • matsch (unregistered)

    That idiom is actually a feature because shell redirection retains owner, group and permissions of the target file in case it already exists. And I'd wager that it's POSIX sh compatible that way and not bash-specific. I didn't look it up, though.

  • my name (unregistered)

    is this 8bit-safe? my hunch is, do this with a binary file and you have garbage

  • commenter (unregistered)

    More Free Software hate as usual from the committed MS shill (you've been at it for years and it's very obvious, I see you), and nice work approving the FUD comment above before any of the others.

  • commenter (unregistered)

    Just wanted to apologise for my previous comment - some people bat for MS because of EEE-induced brain damage, not out of malice or lack of principles, and it was unfair to attribute your long history of vague and unfounded doubt-sowing (and equally vapid MS boosterism) to the latter and not the former.

  • redshirt (unregistered)

    What happens if myDestFile already exists? If I'm not mistaken this code will append the data of mySourceFile at the end.

    In extremely narrow situations I can see that as the intended result (if both are TXT, for example) but if there is any other data in either this will just leave the user with a junk file that can't be used by anything.

  • Sauron (unregistered)

    If the output of cat isn't "Meow" , then it shouldn't be used.

  • (nodebb) in reply to my name

    No, it's fine (easy to test, compare a hash of /bin/cat against a version of the file you created using this cat-copy method).

  • (nodebb)

    And I'm sure that there are input files for which this will break.

    It's more accurate to say that there are objects in the file system for which it will break, almost exclusively things that aren't really files (listening UNIX sockets, device files, etc.). cp won't necessarily do something intelligent with them, but whatever it does will be less stupid than what cat-greater will do.

  • Kell S (unregistered)

    I can't wait to see the result when a link to /dev/zero is in the directory that is processed.

  • Kythyria (unregistered) in reply to Steve_The_Cynic

    It will also break if there is any metadata that cp would copy and which the default for new files in that location is not the same.

    Symbolic links, perhaps. xattrs. Permissions. The assumption that a filesystem is a strict tree whose interior nodes are a map of string to node, and leaves are byte arrays, is weirdly common considering how few OSes it's true for.

  • (author) in reply to commenter

    Is this some kind of new trolling routine? Am I out of the loop?

  • macwhiz (unregistered) in reply to redshirt

    No, the > operator will overwrite the destination file. If you want to concatenate the output of the cat command to the end of the file, you'd need to use the >> operator. The cat command itself only concatenates the list of files it's passed into a single output stream.

  • DigitalBits (unregistered) in reply to commenter

    As someone who loves linux, bash is awful. You can learn to use it, and you can even be competent in it, but it's a badly designed mess.

    Of course, microsofts old equivalent (cmd) is even worse, though at least powershell seems to be well designed, even if it's overly verbose and annoying to use. There might be other better linux shells (zsh for example), but AFAIK they are all "bash-ish" in style and no easier to write scripts for.

    I'd rather write a python script than go anywhere near a shell scripting language.

  • (author) in reply to DigitalBits

    That used to be true for Python, but the 2/3 conversion broke that for a number of years: you couldn't guarantee Python was installed in the first place, but now you couldn't guarantee which Python was installed. Bash has its warts, but I still prefer it for anything at a relatively low level of complexity. If I'm reaching for Python, it's because I have a high degree of control over the target environment and can control what is installed there and am doing a task that's fairly complicated (100+ line script).

    Powershell is nice in concept, but the ergonomics are awful, and I'm actually not entirely sold on passing objects between processes. Passing text streams might be a very 1970 way of sharing data, but it's a very flexible way of sharing data. If only we had a better tool that AWK for splitting those streams up. It's… awkward.

  • (nodebb) in reply to Kythyria

    Good points. I hadn't thought about symlinks (which will become normal files in the output), and of course the permissions, even on ordinary files, won't be copied (the destinations will universally have whatever permissions your UMASK puts on there, often stuff like 660/rw-rw----).

    Such a shame there's no POSIX equivalent of CopyFileExW().

  • Jan v/d Broek (unregistered)

    cp filename filename

    will complain, where

    cat filename >filename

    leaves you with an empty file.

  • (nodebb) in reply to matsch

    A very handy feature - cat /dev/null > /var/log/unruly_file.log is a great way of truncating large log files without messing with the original file permissions

  • (nodebb) in reply to DigitalBits

    Out of curiosity: Could you elaborate on why you think bash is a "a badly designed mess"?

    I'm with Remy here - due to the python 2/3 snafu and installed-python-version roulette (coupled with inconsistent system-package handling), I'd rather pick bash for simpler tasks rather than reaching for python, especially for scripts which need to run on other machines.

    If I need to do this in python - especially for scripts which need non-default libs - I would have to define a virtual environment (first hoping that the python version of choice is installed on the system) and then a requirements.txt to make sure that all deps are present.

    Easier to just have a bash script (or even sh) which would work on multiple systems

  • Sou Eu (unregistered) in reply to redshirt

    If the destination file already exists, a single greater-than symbol means to replace the contents; two greater-than symbols would mean to append.

  • Jonathan (unregistered) in reply to MobyDuck

    Believe it or not, just

    > /var/log/unruly_file.log

    will do the same thing. It's kind of cute, but it's too confusing and non-obvious for me to advocate using it. But I do grudgingly admire it when I've seen others use it.

  • Jonathan (unregistered) in reply to The Beast in Black

    Yes, bash has maximum portability, and I've written 250+ line bash scripts when that was required, but hated it.

    Bash is bad because

    • string testing and manipulation is clunky, and if statements are complicated. [ vs [[, "=" is regexp, not equality, handling of empty strings, which regexp conventions does command X use, etc.
    • filenames now have spaces in them, and writing bash scripts that can handle this is a PITA

    Bash is great at starting other processes and dealing with steams of date.

    My smart-ass line is "the most important thing to know about bash is when to stop using it". Because my fallback language is Perl, not Python, the 10+ year old version that is almost certainly installed will let do all the munging I want (though using external programs is harder, but sed, awk, grep, etc. have lovely equivalents within Perl.)

  • Duke of New York (unregistered) in reply to commenter

    There are some surprising variations among installed Bash versions too, such as whether particular commands create subshells. That said, it is decent for the tasks it was designed for, which are much smaller than the range of problems that exist in the presence of GUIs and web sites.

    Oh, and "commenter", take a chill pill. Everyone was not put on earth to kiss RMS's foot (yuck).

  • Randal L. Schwartz (github) in reply to DigitalBits

    I'd rather write a python script than go anywhere near a shell scripting language.

    Another reason Perl was an early easy-sell as well. No more worries about whitespace and multiple quotes-within-quotes-within-quotes. And no forking for common operations.

  • Abigail (unregistered)

    cat mySourceFile > myDestFile will give an error if myDestFile exists and is a directory, where cp mySourceFile myDestFile will copy the file to a file named mySourceFile inside myDestFile.

    I sometimes use cat mySourceFile > myDestFile if I don't want any file permissions to be copied and am too lazy to remember which option to give to cp to prevent that.

  • TheCPUWizard (unregistered)

    For thosae who post about permissions and the like (including "non-file" entries.... perhaps the behavior of "cat>file" is actually the desired one? Of course since there are no comments or documentation posted, one can not tell [oh yea, the documentation is on a coffee stained piece of paper in Sid's bottom filling cabinet, but it is locked, and Sid had the key when he left the company 10 yeas ago --- but it IS documented.....

  • (nodebb) in reply to Jonathan

    Yeah, bash's string handling does suck sweaty donkey balls - even with pipes to other utilities :D

    If there's a lot of string manipulation involved, I definitely reach for some tool other than bash.

  • (nodebb)

    Bonus points for copying a video file by re-encoding it into the same format and resolution. Ditto for audio files.

  • Duke of New York (unregistered)

    Today's article illustrates the main difficulty of doing anything with Bash or any other Unix shell: a shell doesn't "do anything"; it orchestrates other programs that do it. So your proficiency as a shell script author will depend on knowing the ins and outs (literally) of a couple dozen other programs. I acquired that knowledge by being in the right place at the right time. Recently I wanted to present an introduction to shell scripting, so I looked all over for a single list of the common scripting commands — not the common interactive commands! — and came up short. So one should not be surprised to find shell scripts doing things in various forced ways.

  • (nodebb) in reply to Mr. TA

    You unmitigated monster! I like you.

  • mihi (unregistered)

    For truncating a file , you can also use

    :> /var/log/somefile

    and I think that when using cp with an existing destination, will unlink the destination file and create a new inode for it while redirection will truncate the destination file and append the rest. Which is relevant if the file is large and still open by some process, as the old file will need disk space even though it is already unlinked.

  • xtal256 (unregistered) in reply to Remy Porter

    "Passing text streams might be a very 1970 way of sharing data, but it's a very flexible way of sharing data."

    But passing objects is even more flexible. With text, it's just text. With objects, you can have more data types (no more stringly typed data). With text, you have to know how to parse it. With objects, you get fields that you can inspect and get exactly what you want.

    PowerShell makes it easy to format data when it's printed to the console in a way that would be very hard with text. If you want to print a list with columns that is truncated at the console width (i.e. don't assume 80 characters) then to do so with text would require each program know how wide the console is and have it's own formatting code. Kinda breaks the "do one thing well" that Linux/Unix is known for.

  • Officer Johnny Holzkopf (unregistered) in reply to I'm not a robot

    What about the obvious way: "dd if=${INFILE} of=${OUTFILE} iflag=direct oflag=direct bs=1 speed=1" - because we want a byte-wise exact copy? There's plenty of potential for optimization hidden in the speed= parameter (use for promotion)!

  • (nodebb)

    This WTF post was amusing. However the comment section is amazing. Apparently that one post is pushing buttons for a lot of folks -- those who need to troll, those who know how a un*x system works, and those who clearly don't but still boast their incorrect assumptions without apparently even bothering to $ man bash first.

    I love to RTFM the bash man page. After 2 decades of using and abusing bash, I still need to read it from time to time to remember some obscure syntax, and learn yet another little detail I had missed all along in the process.

  • (nodebb) in reply to xtal256

    But passing objects is even more flexible. With text, it's just text.

    On the other hand, passing around streams of bytes is more universal. Any file can be seen as a sequence of bytes, even before an interpretation was associated with it. It is also a concept, that is unlikely to ever change, so in a sense it is infinitely forward-compatible. It is also more portable, because any system will usually be able to process streams of bytes, since they are the lowest-level representation of data.

    That said, if there were a defacto-default object-oriented shell for Linux, it would make me happy. Python is great for scripting, definitely much easier to debug especially, but for most output data, passing it around as "tables" would indeed be more useful.

    But then the program code would suddenly be specific to shells of that system, instead of being portable across different operating systems... So in order to really be better than "stream of bytes" output, the shell would also need to be available on all operating systems.

    In that sense I somehow feel, that "stream of bytes" fits open source utilities better than object oriented variants. Produce output that can be consumed by human interactors on any system, leave interpretation of the data to the people building automations around it, and find conventions for representing structured data in an both easily parsed and manner that is also well formatted for human consumption. That latter part is probably currently not quite there, though most programs where it actually is relevant already provide an easily processed tabular output format – what's missing is a clear standardization for the handling of values represented by strings with whitespace, and for accessing table columns by unique, LANG-independent names instead of column number.

  • WTFGuy (unregistered)

    @R3D3: I know, I know!!

    Let's all use JSON & Javascript. That's the universal text format for data and the universal shell scripting language.

    Yes, I'm kidding.

    Ultimately your argument, which Im not entirely unsympathetic to, is that the lowest common denominator is best because it's the most common. I prefer to focus more on the "lowest", and try to avoid using "lowest" [whatever] when a higher [whatever] is available.

    How many of us truly are writing code for an infrastructure with no configuration control, an unbounded list of OSes & versions and shells and ... to be compatible with, etc.? Certain public library authors sure are. At my shop we've got a rather more limited array of environments to keep compatibility with.

  • (nodebb) in reply to I'm not a robot

    I'll say! There's no use of Excel anywhere I can see.

  • (nodebb) in reply to commenter

    In the same comment that you supposedly apologize, you go on to say that anybody who likes working with Microsoft tech must be brainwashed? Sounds like "sorry, but not sorry" to me.

    You're either trolling, or you're a blind devotee to certain misguided (some of which are outdated) views and beliefs. You come across as someone who puts opinion above pragmatism and if I were you I would take a long hard think about how that might affect how other people view you in a professional capacity.

  • A Nonny Moose (unregistered) in reply to xtal256

    The difficulty with an OO shell is when command A outputs object type X and command B expects object type Y. The in-between manipulation can be just as hard if not harder than text stream munging.

    That said Bash and PowerShell both work well enough and I can muddle my way through tasks with either one.

  • richarson (unregistered) in reply to Athanasius

    So it's a copycat?

    Damn, it was right there all along...

  • richarson (unregistered) in reply to Jonathan

    string testing and manipulation is clunky, and if statements are complicated, [ vs [[,...

    yeah, but you get used to it, like with any other clunky feature of any other language

    "=" is regexp, not equality, what????? "=" is equality, not regexp, you might be confusing it with "=~"?

    handling of empty strings, which regexp conventions does command X use, etc. filenames now have spaces in them, and writing bash scripts that can handle this is a PITA

    I think a PITA is a bit extreme, I've been writing "spaces in filenames" compliant shell script for over 15 years now but I'm not a programmer/developer, I'm a sysadmin so maybe it's just getting used to the tools you have

    I'm with you though in that if you're gonna replace bash for your scripting needs, perl might be your best choice for many reasons (even if I don't particularly like perl myself)

  • Duke of New York (unregistered) in reply to Jonathan

    There’s also a time to stop using Perl, and it’s “years ago.”

    “Please give me $$_[$#]->{query}. I want to s:%(\x\x):chr(hex $1):ge it.” Statements dreamed up by the utterly Deranged

  • (nodebb) in reply to Duke of New York

    Firstly, that Perl code that you posted isn't even valid; for starters, \x\x is wrong and will not parse.

    Second, Perl code can be be written to be quite readable:

    s{
    %(\x65)
}{
    chr(
        hex $1  # Interpret $1 as a hex string; returns int.
    ); # Convert character ordinal to corresponding character.
}gex; # global matching, execute replacement as code, ignore whitespace/allow-multiline/allow-comments.

    Perl can be written very elegantly if one cares to do so, same as with any language. Anyone can criticize a lang that they do know or do not know well. Your post frankly borders on disinformation.

  • Duke of New York (unregistered)
    Comment held for moderation.
  • Ankit Chaudhary (unregistered)
    Comment held for moderation.
  • (nodebb) in reply to gordonfish

    Perl can be written very elegantly if one cares to do so, same as with any language. Anyone can criticize a lang that they do know or do not know well. Your post frankly borders on disinformation.

    The point was probably more about Perls somewhat hard-to-remember syntax. Same as I can never remember, which is the correct idiom for stripping the extension from a filename in Bash, that's definitely easier in Perl (mostly because regexps use at least similar syntax across various languages), before even accounting for appropriate utility functions, that do exist.

    Perl 5 has its fair share of annoyances, but it makes a powerful, probably-everywhere-available scripting language, with an easy mechanism for ensuring that a minimal version is installed, and at the same time ensuring that your code does not use features that require a higher version.

    Also, I just learned that Perl 6 is called "Raku" since a few years. I vaguely remember "Rakudo" as the main implementation, but I never realized, that the language itself had been renamed.

  • commenter (unregistered) in reply to Jonathan Lydall
    Comment held for moderation.
  • (nodebb) in reply to my name

    is this 8bit-safe? my hunch is, do this with a binary file and you have garbage

    Your hunch would be wrong. cat is 8-bit clean (on Unix systems). In fact it is the canonical way to reconstitute files that have previously been split into smaller parts.

  • T.E. (unregistered) in reply to R3D3

    Wrote: but for most output data, passing it around as "tables" would indeed be more useful

    Ah yes. Think of the IBM i.

    The pipe operator is terrific: no need to think a filename and a place to put it, no need to delete that file when you are done. But it is wonderful to have a system which speaks "tables" natively.

    Terry.

  • nasch (unregistered)
    Comment held for moderation.

Leave a comment on “Just a Copy of Copy”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article