Weird behavior with find command when tarring files

[Hameed, Amir]

Hi,
I am using the find command to TAR up files that are older than 4 hours:

find . type f -mmin +239 | xargs tar -cvf /tmp/test.tar

The command tars up files and seems to finish fine. However, when I untar and count the number of files against the count of files that should have been captured (find . type f -mmin +239 | xargs ls -l | wc -l) there is a huge difference and the files captured by tar were way less than the file listed for the same time.

This is a strange behavior. What am I doing wrong (I am sure I am doing something wrong)?


Thanks,
Amir

[Andre Maasikas]

Hi,
see "man xargs"
xargs will execute the tar command with as much filenames as it thinks is
allowed on command line
(might be 128kb). If you have more files then it executes the command again
and the tar
will overwrite the first /tmp/test.tar etc...
See if tar has on option to to create/append to an existing tar file
or write the find output to a temp file and use tar option *--files-from*=
*FILE*

Andre

Post by Hameed, Amir
Hi,
find . type f -mmin +239 | xargs tar -cvf /tmp/test.tar
The command tars up files and seems to finish fine. However, when I untar
and count the number of files against the count of files that should have
been captured (find . type f –mmin +239 | xargs ls –l | wc -l) there is a
huge difference and the files captured by tar were way less than the file
listed for the same time.
This is a strange behavior. What am I doing wrong (I am sure I am doing something wrong)?
Thanks,
Amir

[Stefan Knecht]

Use tar -r instead of tar -c

It will create the archive if it doesn't exist and append to it if it does.
Your call to tar -c will probably overwrite the file each time it runs.

Post by Hameed, Amir
Hi,
find . type f -mmin +239 | xargs tar -cvf /tmp/test.tar
The command tars up files and seems to finish fine. However, when I untar
and count the number of files against the count of files that should have
been captured (find . type f –mmin +239 | xargs ls –l | wc -l) there is a
huge difference and the files captured by tar were way less than the file
listed for the same time.
This is a strange behavior. What am I doing wrong (I am sure I am doing something wrong)?
Thanks,
Amir

[Hameed, Amir]

I appreciate for all the feedback that I have received. It helped me resolved the issue.

Thanks
From: Stefan Knecht <***@gmail.com>
Sent: Friday, October 19, 2018 2:29 AM
To: Hameed, Amir <***@xerox.com>
Cc: oracle-l-freelists <oracle-***@freelists.org>
Subject: Re: Weird behavior with find command when tarring files

Use tar -r instead of tar -c

It will create the archive if it doesn't exist and append to it if it does. Your call to tar -c will probably overwrite the file each time it runs.

On Fri, Oct 19, 2018 at 3:58 AM Hameed, Amir <***@xerox.com<mailto:***@xerox.com>> wrote:
Hi,
I am using the find command to TAR up files that are older than 4 hours:

find . type f -mmin +239 | xargs tar -cvf /tmp/test.tar

The command tars up files and seems to finish fine. However, when I untar and count the number of files against the count of files that should have been captured (find . type f –mmin +239 | xargs ls –l | wc -l) there is a huge difference and the files captured by tar were way less than the file listed for the same time.

This is a strange behavior. What am I doing wrong (I am sure I am doing something wrong)?


Thanks,
Amir

[Dave Herring]

Amir, I'd go with:

% tar cvf /tmp/test.tar `find . type f -mmin +239 -print` | tee test.log

% wc -l test.log

% tar tvf /tmp/test.tar | wc -l

Lines 2 and 3 will help in verification, in case you don't trust the tar
command. "tee" is used to you can watch output and save it as well.

Dave

Post by Hameed, Amir
I appreciate for all the feedback that I have received. It helped me resolved the issue.
Thanks
*Sent:* Friday, October 19, 2018 2:29 AM
*Subject:* Re: Weird behavior with find command when tarring files
Use tar -r instead of tar -c
It will create the archive if it doesn't exist and append to it if it
does. Your call to tar -c will probably overwrite the file each time it
runs.
Hi,
find . type f -mmin +239 | xargs tar -cvf /tmp/test.tar
The command tars up files and seems to finish fine. However, when I untar
and count the number of files against the count of files that should have
been captured (find . type f –mmin +239 | xargs ls –l | wc -l) there is a
huge difference and the files captured by tar were way less than the file
listed for the same time.
This is a strange behavior. What am I doing wrong (I am sure I am doing something wrong)?
Thanks,
Amir
--
//
zztat - The Next-Gen Oracle Performance Monitoring and Reaction Framework!

[Jared Still]

In addition to the other fine comments, you should get in the habit of
dealing with filenames that have spaces.

It seems impossible to anymore to avoid this, as you don't always have
control over it.

Here's how: use the '-print0' option for find, and the '-0' option for
xargs.

find . type f -mmin +239 -print0 | xargs -0 tar -cvf /tmp/test.tar



Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist
Principal Consultant at Pythian
Pythian Blog http://www.pythian.com/blog/author/still/
Github: https://github.com/jkstill

Post by Hameed, Amir
Hi,
find . type f -mmin +239 | xargs tar -cvf /tmp/test.tar
The command tars up files and seems to finish fine. However, when I untar
and count the number of files against the count of files that should have
been captured (find . type f –mmin +239 | xargs ls –l | wc -l) there is a
huge difference and the files captured by tar were way less than the file
listed for the same time.
This is a strange behavior. What am I doing wrong (I am sure I am doing something wrong)?
Thanks,
Amir

[p***@gmail.com]

Thanks for the tip, I didn’t know about -print0. Is this option generally available on older Unix (hpux, aix, solaris)? I have been using double quotes in most of my code for a couple years around file variables in anticipation of being able to run on Windows environments. Have not dare tested that yet however.



One other thing to note here. Be really careful with xargs. I believe it executes on null input too. I don’t recall the exact issue but I have been burned at least twice and good enough to recall you need to be very careful with it if you are deleting or modifying things. Also stay away from “find .” and make sure you try to use a full path there. This is all obvious maybe and only used here for the sake of examples but throwing it out there for the newbies if any are reading.



Ethan Post

https://www.linkedin.com/in/ethanraypost/

https://arclogicsoftware.com

https://twitter.com/poststop





From: oracle-l-***@freelists.org <oracle-l-***@freelists.org> On Behalf Of Jared Still
Sent: Friday, October 26, 2018 1:49 AM
To: Hameed, Amir <***@xerox.com>
Cc: oracle-***@freelists.org
Subject: Re: Weird behavior with find command when tarring files



In addition to the other fine comments, you should get in the habit of dealing with filenames that have spaces.



It seems impossible to anymore to avoid this, as you don't always have control over it.



Here's how: use the '-print0' option for find, and the '-0' option for xargs.



find . type f -mmin +239 -print0 | xargs -0 tar -cvf /tmp/test.tar








Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist

Principal Consultant at Pythian

Pythian Blog http://www.pythian.com/blog/author/still/

Github: https://github.com/jkstill

[Jared Still]

yes, the empty input can be dangerous, use -E to deal with that

find tmp -name notexist | xargs ls

12c-connect.txt blog git-ignore-template
nohup.out snow.sh
DBI so github-config-template nwt
snow2.sh
Desktop build ht-detect.sh ogg
sounds
Documents c html old
sqlnet.log
Downloads cdrom imagejpeg_0.jpg
ora112304a.jks.com sqltext.txt
FlameGraph checking iostat
oracle ssh-find-agent
...

find tmp -name notexist | xargs -E ls -l
find tmp -name j | xargs -E ls -l

tmp/run_stats_load/j
tmp/performance/th/j
tmp/sqlstats/perl/j
tmp/sqlstats/ddl/j
tmp/j
tmp/cursor/j
***@poirot ~ $
now imagine the ls command was instead rm...

Thanks for the tip, I didn’t know about -print0. Is this option generally
available on older Unix (hpux, aix, solaris)? I have been using double
quotes in most of my code for a couple years around file variables in
anticipation of being able to run on Windows environments. Have not dare
tested that yet however.
One other thing to note here. Be really careful with xargs. I believe it
executes on null input too. I don’t recall the exact issue but I have been
burned at least twice and good enough to recall you need to be very careful
with it if you are deleting or modifying things. Also stay away from “find
.” and make sure you try to use a full path there. This is all obvious
maybe and only used here for the sake of examples but throwing it out there
for the newbies if any are reading.
Ethan Post
https://www.linkedin.com/in/ethanraypost/
https://arclogicsoftware.com
https://twitter.com/poststop
Behalf Of *Jared Still
*Sent:* Friday, October 26, 2018 1:49 AM
*Subject:* Re: Weird behavior with find command when tarring files
In addition to the other fine comments, you should get in the habit of
dealing with filenames that have spaces.
It seems impossible to anymore to avoid this, as you don't always have control over it.
Here's how: use the '-print0' option for find, and the '-0' option for xargs.
find . type f -mmin +239 -print0 | xargs -0 tar -cvf /tmp/test.tar
Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist
Principal Consultant at Pythian
Pythian Blog http://www.pythian.com/blog/author/still/
Github: https://github.com/jkstill
--

Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist
Principal Consultant at Pythian
Pythian Blog http://www.pythian.com/blog/author/still/
Github: https://github.com/jkstill