LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-30-2024, 10:21 AM   #1
RandomTroll
Senior Member
 
Registered: Mar 2010
Distribution: Slackware
Posts: 2,141

Rep: Reputation: 275Reputation: 275Reputation: 275
Why don't file-aimed utilities skip directories unless told to recurse?


[Log in to get rid of this advertisement]
Some of my directories have, mostly, subdirectories in them. When I run a utility that I aim at all files, for instance:
Code:
grep banana *
, error reports on the directories swamp target output. I usually redirect error output to /dev/null, though using -d skip would be better, but I never want to be told that my file-aimed utility doesn't work on a directory, would rather have it do that silently. Most utilities lack grep's -d switch, for which the default action should be skip.
 
Old 12-30-2024, 03:35 PM   #2
Keith Hedger
Senior Member
 
Registered: Jun 2010
Location: Wiltshire, UK
Distribution: Void, Linux From Scratch, Slackware64
Posts: 3,205

Rep: Reputation: 868Reputation: 868Reputation: 868Reputation: 868Reputation: 868Reputation: 868Reputation: 868
"Why don't file-aimed utilities skip directories unless told to recurse?" 'cos thats the way the developer wrote it!
 
1 members found this post helpful.
Old 12-30-2024, 06:54 PM   #3
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,417

Rep: Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749
* is evaluated by the shell before even the grep command has been started.
At the end all the content of * (files, dirs, links, whatever) will be passed to the command.
grep can only state that those arguments are not "greppable" at all.
So if you don't want to use grep on a dir don't pass it to grep, don't use *.

Otherwise you can use set -x to see what's going on.
 
1 members found this post helpful.
Old 12-31-2024, 01:23 AM   #4
RandomTroll
Senior Member
 
Registered: Mar 2010
Distribution: Slackware
Posts: 2,141

Original Poster
Rep: Reputation: 275Reputation: 275Reputation: 275
Quote:
Originally Posted by Keith Hedger View Post
'cos thats the way the developer wrote it!
They've all changed over time.

Quote:
Originally Posted by pan64 View Post
* is evaluated by the shell before even the grep command has been started.
grep has a -d switch; read is the default; skip could be. There could be an environment variable GREP, analogous to LESS, to set default switches.

Quote:
Originally Posted by pan64 View Post
if you don't want to use grep on a dir don't pass it to grep, don't use *.
This is easier said than done. It can be hard to pass an argument that applies to all files and no directories.
 
Old 12-31-2024, 03:29 AM   #5
lvm_
Senior Member
 
Registered: Jul 2020
Posts: 1,290

Rep: Reputation: 425Reputation: 425Reputation: 425Reputation: 425Reputation: 425
Quote:
Originally Posted by RandomTroll View Post
There could be an environment variable GREP, analogous to LESS, to set default switches.
If you cared to read the man page, you'd find out that there is such a variable.
 
Old 12-31-2024, 03:42 AM   #6
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,417

Rep: Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749
Quote:
Originally Posted by RandomTroll View Post
grep has a -d switch; read is the default; skip could be.
again, it is made by the shell. grep will get those dirs, one by one, as an argument. What do you think what should grep do with them?
Dirs are skipped, exactly as you explained. The only thing is grep will report it. And may set the exit code.
Quote:
Originally Posted by RandomTroll View Post
There could be an environment variable GREP, analogous to LESS, to set default switches.
and there is a way to do that. You can define an alias to add your flags, or use GREP_OPTIONS, if available

Quote:
Originally Posted by RandomTroll View Post
This is easier said than done. It can be hard to pass an argument that applies to all files and no directories.
Well, if it is really needed....
 
Old 01-01-2025, 01:04 AM   #7
RandomTroll
Senior Member
 
Registered: Mar 2010
Distribution: Slackware
Posts: 2,141

Original Poster
Rep: Reputation: 275Reputation: 275Reputation: 275
Quote:
Originally Posted by lvm_ View Post
If you cared to read the man page, you'd find out that there is such a variable.
I read the man page before I started this thread. I saw that there's GREP_COLORS to specify colors used in the display, but no others. What's so special about colors?

Quote:
Originally Posted by pan64 View Post
again, it is made by the shell. grep will get those dirs, one by one, as an argument. What do you think what should grep do with them?
What I want.

Quote:
Originally Posted by pan64 View Post
Dirs are skipped, exactly as you explained.
They aren't treated as they would be if one used the -d skip switch, which seems to be the definition of skipping.

Quote:
Originally Posted by pan64 View Post
You can define an alias to add your flags
Then I have to remember to use the alias and it won't work on other computers.

Quote:
Originally Posted by pan64 View Post
use GREP_OPTIONS, if available
It doesn't work, and the page to which you refer is old and says it will be removed. It used to exist, now doesn't. The decision behind that is the answer to my question.

Quote:
Originally Posted by pan64 View Post
Well, if it is really needed....
That's not the standard. There's lots of stuff that's convenient but unnecessary. Lots of other utilities read environment variables; grep reads GREP_COLORS.
 
Old 01-01-2025, 01:57 AM   #8
___
Member
 
Registered: Apr 2023
Posts: 208
Blog Entries: 1

Rep: Reputation: Disabled
alias it to have -r

Quote:
-d ACTION, --directories=ACTION
If an input file is a directory, use ACTION to process it.
By default, ACTION is read, i.e., read directories just as
if they were ordinary files. If ACTION is skip, silently
skip directories. If ACTION is recurse, read all files
under each directory, recursively, following symbolic links
only if they are on the command line. This is equivalent
to the -r option.
What other utils annoy you by not recursing?


BonusTurd: ack with globstar: https://stackoverflow.com/questions/...formance-issue


Question for LQgurus: "By default, ACTION is read, i.e., read directories just as if they were ordinary files"
doesn't seem true! It refuses & says: ... Is a directory

Last edited by ___; 01-01-2025 at 02:43 AM.
 
Old 01-01-2025, 04:07 AM   #9
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Distribution: Mint/MATE
Posts: 3,006

Rep: Reputation: 1286Reputation: 1286Reputation: 1286Reputation: 1286Reputation: 1286Reputation: 1286Reputation: 1286Reputation: 1286Reputation: 1286
The shell glob examines the directory contents and does not further examine the file types.
You can use find
Code:
find . -name "*" -f -print0 | xargs -0 grep banana
Code:
find . -maxdepth 1 -name "*" -f -print0 | xargs -0 grep banana
Switch your shell to zsh? In zsh you can do
Code:
grep banana *(.)

Last edited by MadeInGermany; 01-01-2025 at 06:54 AM.
 
Old 01-01-2025, 05:16 AM   #10
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,761

Rep: Reputation: 925Reputation: 925Reputation: 925Reputation: 925Reputation: 925Reputation: 925Reputation: 925Reputation: 925
glob * is handy for global files copy, move and such
For doing stuff more specific, it could be better to use *.extension (if files have extensions)
You can even use multiple extensions matches like *.{pdf,PDF,jpg,JPG}
 
Old 01-05-2025, 09:51 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,417

Rep: Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749
Quote:
Originally Posted by RandomTroll View Post
What I want.
unfortunately it is not the way how programs work. They never know what you (or me) wanted, but they do what was implemented.
The program which will do exactly what I want does not exist (at this moment).

Quote:
Originally Posted by RandomTroll View Post

They aren't treated as they would be if one used the -d skip switch, which seems to be the definition of skipping.
You still don't understand, grep will not get the *, will not evaluate/expand it by itself and therefore cannot select only files.
It is the shell which will do the command line evaluation, before execution, and * will be replaced to the list of items in the current directory. Including everything (what is inside). And the shell does not care the arguments passed to grep (like -d).
So grep will get that list (of everything) and will try to use them one by one. In case of a file the search will be executed, in case of non-file arguments it will be reported: XXX is a directory (and the pattern won't be searched in a dir, it would have a different meaning).

___ explained (post #8), --directories=skip will tell grep to skip dirs silently do not print that message.
Additionally you may try --exclude-dir=* too (just need to use correct quotations).

Yes if you need this you need to create an alias containing your additional option(s), use a grep which can handle GREP_OPTIONS, or create a function to do that. And yes, you need to do that on all of your hosts.
 
Old 01-05-2025, 08:48 PM   #12
RandomTroll
Senior Member
 
Registered: Mar 2010
Distribution: Slackware
Posts: 2,141

Original Poster
Rep: Reputation: 275Reputation: 275Reputation: 275
Quote:
Originally Posted by pan64 View Post
unfortunately it is not the way how programs work.
grep used to work this way. I asked why it doesn't any longer.

Quote:
Originally Posted by pan64 View Post
You still don't understand, grep will not get the *
I always understood. grep used to read an environment variable that one could set. I asked why it doesn't any longer.

Last edited by RandomTroll; 01-05-2025 at 08:50 PM.
 
Old 01-06-2025, 03:12 AM   #13
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,417

Rep: Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749
Quote:
Originally Posted by RandomTroll View Post
grep used to work this way. I asked why it doesn't any longer.


I always understood. grep used to read an environment variable that one could set. I asked why it doesn't any longer.
I think it was never working that way. That's why it doesn't do it "any longer".

GREP_OPTIONS was intentionally removed, and that is also explained on the linked page.
 
Old 01-06-2025, 11:46 AM   #14
RandomTroll
Senior Member
 
Registered: Mar 2010
Distribution: Slackware
Posts: 2,141

Original Poster
Rep: Reputation: 275Reputation: 275Reputation: 275
Quote:
Originally Posted by pan64 View Post
I think it was never working that way. That's why it doesn't do it "any longer".

GREP_OPTIONS was intentionally removed, and that is also explained on the linked page.
How can it never have worked that way and also be intentionally removed?

I just rewrote grep's source to make it skip directories by default. It works. If I can do it, they can. Notice I don't say that they should. I asked why they don't even have an environment variable to read to make it work that way.
 
Old 01-06-2025, 11:51 AM   #15
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 23,417

Rep: Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749Reputation: 7749
Quote:
Originally Posted by RandomTroll View Post
How can it never have worked that way and also be intentionally removed?
those are two different and independent things.
1. grep reports dirs
2. the removal of that variable

Quote:
Originally Posted by RandomTroll View Post
I just rewrote grep's source to make it skip directories by default. It works. If I can do it, they can. Notice I don't say that they should. I asked why they don't even have an environment variable to read to make it work that way.
this may work now for you, but not for everybody and everywhere.

Last edited by pan64; 01-06-2025 at 11:54 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
scp recurse but NOT overwrite? Goog Linux - Software 3 12-04-2012 09:55 PM
copy command cp -R does not recurse directories ankuryu DamnSmallLinux 1 05-15-2007 07:35 AM
how to recurse a cat? paledread Linux - Software 9 05-20-2006 12:30 PM
Recurse a directory Peter Shepard Linux - General 1 12-23-2005 03:42 PM
recurse through subdirs? Axion Linux - Software 4 02-16-2004 09:12 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:26 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration