a progress bar for 'cp' and 'mv'

tags 185152 +patch

And progress bars in mv, rm and, most important, dd would also be nice!

Best regards,

Onno

# Automatically generated email from bts, devscripts version 2.8.10
merge 284514 185152

tags 68603 wontfix
tags 185152 wontfix
tags 17025 wontfix
retitle 120650 coreutils: chmod: Please add R, W, S and T options
retitle 210475 coreutils: stty: Please add option to force stty to act immediately, not to wait for output to be empty
retitle 21750 coreutils: install: Please add option to compress files
tags 21750 wontfix
tags 48038 upstream
retitle 55218 coreutils: ls: Please add option to sort by type (i.e., all directories before all files)
tags 56844 wontfix
tags 89335 wontfix
stop

Hello.

I wonder if there is an updated version of that patch which works for the
current version of the coreutils in Debian Sid.

I found a HowTo but it’s for Sarge’s version and won’t get applied successfully.

Regards, Mathias

--
debian/rules

merge 185152 389326

Hello.

For anyone interested I’ll attach the necessary patch, working at least for
version 5.97-5. I made sure it builds using the acl and selinux patches.

Regards, Mathias

--
debian/rules

There are a lot of cases when you're copying something locally that being able
to have a progress indicator like you get with scp or rsync would be nice. And
rather than writing Yet Another Utility, it seems like the right thing to do is
for cp to provide the functionality.

Hi Jeremy,
Many people have requested this, over the years, and there are probably a few
complete patch sets implementing it. Why do you think it's best to add this to
cp, when you can already use rsync as a cp-replacement?

(In reply to comment #1)
> Many people have requested this, over the years, and there are probably a few
> complete patch sets implementing it. Why do you think it's best to add this to
> cp, when you can already use rsync as a cp-replacement?

Because then you have a dependency on rsync instead of cp... cp is pretty much
guaranteed to exist, rsync not as much. Also, rsync is in /usr/bin whereas cp
is in /bin. And rsync is 5 times larger.

Given that a lot of the time you're going to care about text mode cp and
progress is in early boot, you want to avoid all of the above.

Well, consider that some folks want a GUI-oriented progress bar, and some want
text-only. Some are happy to incur the cost of a curses library, others want
the minimal footprint and are happy with less eye candy. We've hashed out all
of this, over the years, and came to the conclusion that if there is to be any
such thing associated with GNU cp (and mv, and dd, and maybe others, like
install), then the mechanism must be sufficiently generic to work with an
independent program that would be in charge of displaying the progress meter.
Then, the core copying code needn't be polluted with code that some will never
want or use. Here's a proposed implementation from 2004:
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/1663

(In reply to comment #3)
> Well, consider that some folks want a GUI-oriented progress bar, and some want
> text-only. Some are happy to incur the cost of a curses library, others want
> the minimal footprint and are happy with less eye candy. We've hashed out all
> of this, over the years, and came to the conclusion that if there is to be any
> such thing associated with GNU cp (and mv, and dd, and maybe others, like
> install), then the mechanism must be sufficiently generic to work with an
> independent program that would be in charge of displaying the progress meter.
> Then, the core copying code needn't be polluted with code that some will never
> want or use. Here's a proposed implementation from 2004:
> http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/1663

And at this point, you have entered well into the realm of over-engineering a
simple problem ;-)

If you're using curses, etc and wanting to show the output of cp, you're already
writing an app at which point the part of cp that are going to be cared about
aren't that hard to implement. The value here is in being able to have a shell
script give some form of progress feedback as it copies a large file.

From a distance, most problems look simpler than they are :-)

If you're interested in copying a single file, then maybe dd already provides
what you want? From dd --help:

Sending a USR1 signal to a running `dd' process makes it
print I/O statistics to standard error and then resume copying.

  $ dd if=/dev/zero of=/dev/null& pid=$!
  $ kill -USR1 $pid; sleep 1; kill $pid
  18335302+0 records in
  18335302+0 records out
  9387674624 bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s

With that basis, a little wrapper script could do what you want.

It seems your patch have an error with coreutils 5.97-5.3:

if
colorgcc -std=gnu99 -DHAVE_CONFIG_H -DLIBDIR=\"/usr/lib\" -I. -I. -I.. -I.. -I. -g -DSYSLOG_SUCCESS -DSYSLOG_FAILURE -DSYSLOG_NON_ROOT -O2 -MT
utimecmp.o -MD -MP -MF ".deps/utimecmp.Tpo" -c -o utimecmp.o utimecmp.c; \
        then mv -f ".deps/utimecmp.Tpo" ".deps/utimecmp.Po"; else
rm -f ".deps/utimecmp.Tpo"; exit 1; fi
In file included from utimecmp.c:41:
utimens.h:2: error: conflicting types for 'futimens'
/usr/include/sys/stat.h:370: error: previous declaration of 'futimens' was
here

I realize that this is just a wishlist level item but I wanted to
register my objection to the additional code to implement progress
bars to cp, mv, rm, etc. Those utilities are already too bloated.

Also the functionality of progress bars exists in rsync. If someone
wants a tool with everything including the kitchen sink then using
rsync works well and already exists.

  rsync --progress -av SOURCE DESTINATION

Bob

Hi Victor-Philipp.

Victor-Philipp Busch, 13.07.2007 11:31:
> It seems your patch have an error with coreutils 5.97-5.3:
>
> if
> colorgcc -std=gnu99 -DHAVE_CONFIG_H -DLIBDIR=\"/usr/lib\" -I. -I. -I.. -I.. -I. -g -DSYSLOG_SUCCESS -DSYSLOG_FAILURE -DSYSLOG_NON_ROOT -O2 -MT
> utimecmp.o -MD -MP -MF ".deps/utimecmp.Tpo" -c -o utimecmp.o utimecmp.c; \
> then mv -f ".deps/utimecmp.Tpo" ".deps/utimecmp.Po"; else
> rm -f ".deps/utimecmp.Tpo"; exit 1; fi
> In file included from utimecmp.c:41:
> utimens.h:2: error: conflicting types for 'futimens'
> /usr/include/sys/stat.h:370: error: previous declaration of 'futimens' was
> here

No, that’s a bug in coreutils. However, I don’t know why it doesn’t work for you
where it did for me. See #433394 for a fix. (Might still require backporting
since the patch over there is for version 6.0.)

I updated the patch to work with coreutils 6.0-1 from Experimental. As ususal,
use at your own risk; it works fine here, though.

Regards, Mathias

--
debian/rules

I agree that a progress bar in cp & mv would be very nice at times. rsync can
be very overkill (or unusable in certain situations (like writing to an FS
that doesn't support dot files - vfat, for example - and I would prefer not to
use rsync's -T option to write to yet another fs).

This bug exists on other forums as well:
https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/64067

# Automatically generated email from bts, devscripts version 2.10.11
tags 185152 - patch

Based on the date this bug was created, it appears to have been reported
against rawhide during the development of a Fedora release that is no
longer maintained. In order to refocus our efforts as a project we are
flagging all of the open bugs for releases which are no longer
maintained. If this bug remains in NEEDINFO thirty (30) days from now,
we will automatically close it.

If you can reproduce this bug in a maintained Fedora version (7, 8, or
rawhide), please change this bug to the respective version and change
the status to ASSIGNED. (If you're unable to change the bug's version
or status, add a comment to the bug and someone will change it for you.)

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we're following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

Adding words future feature and moving back to ASSIGNED. As Jim said, there are
already some complete patches implementing the progress bar feature to cp, mv
and install - usually via long option or via -g option. So far it seems that
upstream is not interested in accepting progress bar feature (even via long
option) and making fork will mean one more patch which will be not oneliner and
will need some maintainance as cp/mv/install source codes are changing quiet a
lot between releases.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi there,

Updated patch for coreutils at least version 7.1.

Regards, Mathias

- --
debian/rules
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAknG0O0ACgkQYfUFJ3ewsJiBlQCfQejSCG42ptuvrHhKL4vGByXU
6t8AmwS7e/h1XU+mWkXbsjJvfLXrapJa
=pKDe
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi.

Here’s the patch refreshed against the latest coreutils 8.1.

Regards, Mathias

- --
debian/rules
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAksL2vYACgkQYfUFJ3ewsJj0wACeLK/CzGP4pu7RXxuEdtib3q3d
epQAn3ruREoL6Zv/jJfrD39JundZHlNR
=5VqH
-----END PGP SIGNATURE-----

Time to close that bugzilla...
Discussed many times on upstream list (see archives),
in bugzillas of other linux distributions (e.g.
https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/64067
http://www.linux-archive.org/archlinux-development/167447-add-g-option-cp-mv.html
)

If you really want to have this behaviour, you could use some shell script alias to get this. One of such scripts is available at http://chris-lamb.co.uk/2008/01/24/can-you-get-cp-to-give-a-progress-bar-like-wget/ . Closing WONTFIX.

As of F19, the cp command has the -v (--verbose) option that shows what files are being copied:

$ man cp
...
       -v, --verbose
              explain what is being done
...

$ tar -xf python-bugzilla-0.9.0.tar.bz2
$ ls
python-bugzilla-0.9.0/ python-bugzilla-0.9.0.tar.bz2
$
$ cp -v -a python-bugzilla-0.9.0 foo-1
‘python-bugzilla-0.9.0’ -> ‘foo-1’
‘python-bugzilla-0.9.0/.mailmap’ -> ‘foo-1/.mailmap’
‘python-bugzilla-0.9.0/bz-api-notes.txt’ -> ‘foo-1/bz-api-notes.txt’
... [snipped] ...
‘python-bugzilla-0.9.0/bin/bugzilla’ -> ‘foo-1/bin/bugzilla’
‘python-bugzilla-0.9.0/bugzilla.1’ -> ‘foo-1/bugzilla.1’
‘python-bugzilla-0.9.0/setup.py’ -> ‘foo-1/setup.py’
$ ls
foo-1/ python-bugzilla-0.9.0/ python-bugzilla-0.9.0.tar.bz2
$

Tested with:
$ rpm -qf `which cp`
coreutils-8.21-11.fc19.x86_64

(In reply to Steve Tyler from comment #12)
> As of F19, the cp command has the -v (--verbose) option that shows what
> files are being copied:

The --verbose option has been there at least since fileutils-3.8.3b (released in 1993), I did not dig deeper in the history...

Thanks for your digging, Kamil. None of the earlier comments mentioned the --verbose option, so I assumed that it must have been added more recently than 2010 (Comment 11).