I'm experiencing the "stuck in in progress" problem that I think this refers to (" backup-5.8.2014_22-36-09_user.tar.gz (Thu May 8 22:36:09 2014) [in progress] "). What log file captures backup wizard errors? Thanks.
Hey there! I would expect any errors from cPanel tools to show up in the main log at /usr/local/cpanel/logs/error_log. Can you check that log while watching the interface and see if there is anything helpful there?
I am getting warnings about /tmp being at 86% full, might that have an impact on the backup wizard? Apparently CentOS 7 doesn't clean up /tmp on reboot, and based on Google results cleaning up /tmp can be tricky.
In any case, below is the error_log contents around a full backup attempt from the cPanel Backup Wizard on May 27 03:47:46:
In any case, below is the error_log contents around a full backup attempt from the cPanel Backup Wizard on May 27 03:47:46:
[2023-05-27 03:32:46 -0400] info [cpsrvd] version 11.110.0.6 online
==> cpsrvd 11.110.0.6 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2023-05-27 03:41:30 -0400] info [cpsrvd] version 11.110.0.6 online
[2023-05-27 03:42:10 -0400] info [whostmgrd] Successfully verified signature for cpanel (key types: release).
cpsrvd 29aa9dfdb653c: Cpanel::Exception::NetSSLeay/(XID 9h8yhg) Net::SSLeay::ssl_write_all(SCALAR(0x3066058)) produced an operating system error (EPIPE, Broken pipe) and 1 [asis,OpenSSL] [numerate,_6,error,errors]: ARRAY(0x30ff980)
at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64.
Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x3034680), SCALAR(0x3066058)) called at /usr/local/cpanel/Cpanel/Server/Response.pm line 218
Cpanel::Server::Response::__ANON__(SCALAR(0x3066058)) called at /usr/local/cpanel/Cpanel/Server/Responders/Stream.pm line 45
Cpanel::Server::Responders::Stream::write(Cpanel::Server::Responders::Stream=HASH(0x3066100), 6) called at /usr/local/cpanel/Cpanel/Server/Responder.pm line 176
Cpanel::Server::Responder::finish(Cpanel::Server::Responders::Stream=HASH(0x3066100), 2) called at /usr/local/cpanel/Cpanel/Server/Responder.pm line 91
Cpanel::Server::Responder::readonly_from_input_and_send_response(Cpanel::Server::Responders::Stream=HASH(0x3066100)) called at /usr/local/cpanel/Cpanel/Server/Response.pm line 142
Cpanel::Server::Response::send_response(Cpanel::Server::Response=HASH(0x3034920), Cpanel::Server::Response::Source::ReadOnlyString=HASH(0x3056498)) called at cpsrvd.pl line 3113
cpanel::cpsrvd::servcontent("document", "./frontend/jupiter/libraries/fontawesome/webfonts/fa-solid-90"..., "use_magic", 1, "static", 1, "content_type", "application/font-woff2") called at cpsrvd.pl line 2834
cpanel::cpsrvd::dodoc_cpaneld() called at cpsrvd.pl line 2113
cpanel::cpsrvd::dodoc(HASH(0x1c1e540)) called at cpsrvd.pl line 1802
cpanel::cpsrvd::handle_one_connection(5) called at cpsrvd.pl line 1149
cpanel::cpsrvd::script() called at cpsrvd.pl line 448
Cpanel::Exception::NetSSLeay/(XID jhh3dn) Net::SSLeay::ssl_write_all(HTTP/1.1 500 Internal Error\x{0d}\x{0a}Connection: close\x{0d}\x{0a}Content-type: text/html; charset="utf-8"\x{0d}\x{0a}X-Error-Message: Error ID 29aa9dfdb653c\x{0d}\x{0a}\x{0d}\x{0a}) produced an operating system error (EPIPE, Broken pipe) and 1 [asis,OpenSSL] [numerate,_6,error,errors]: ARRAY(0x30ffa88)
at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64.
Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x3034680), "HTTP/1.1 500 Internal Error\x{d}\x{a}Connection: close\x{d}\x{a}Content-type:"...) called at /usr/local/cpanel/Cpanel/Server.pm line 1693
Cpanel::Server::write_buffer(Cpanel::Server=HASH(0x2e489c0), "HTTP/1.1 500 Internal Error\x{d}\x{a}Connection: close\x{d}\x{a}Content-type:"...) called at /usr/local/cpanel/Cpanel/Server.pm line 693
Cpanel::Server::_send_internal_error_headers_if_needed(Cpanel::Server=HASH(0x2e489c0), 500, "Error ID 29aa9dfdb653c") called at /usr/local/cpanel/Cpanel/Server.pm line 679
Cpanel::Server::internal_error_with_http_status(Cpanel::Server=HASH(0x2e489c0), 500, "Error ID 29aa9dfdb653c") called at /usr/local/cpanel/Cpanel/Server.pm line 664
Cpanel::Server::internal_error(Cpanel::Server=HASH(0x2e489c0), "Error ID 29aa9dfdb653c") called at cpsrvd.pl line 1001
cpanel::cpsrvd::__ANON__(Cpanel::Exception::NetSSLeay=HASH(0x3066a00)) called at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64
Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x3034680), SCALAR(0x3066058)) called at /usr/local/cpanel/Cpanel/Server/Response.pm line 218
Cpanel::Server::Response::__ANON__(SCALAR(0x3066058)) called at /usr/local/cpanel/Cpanel/Server/Responders/Stream.pm line 45
Cpanel::Server::Responders::Stream::write(Cpanel::Server::Responders::Stream=HASH(0x3066100), 6) called at /usr/local/cpanel/Cpanel/Server/Responder.pm line 176
Cpanel::Server::Responder::finish(Cpanel::Server::Responders::Stream=HASH(0x3066100), 2) called at /usr/local/cpanel/Cpanel/Server/Responder.pm line 91
Cpanel::Server::Responder::readonly_from_input_and_send_response(Cpanel::Server::Responders::Stream=HASH(0x3066100)) called at /usr/local/cpanel/Cpanel/Server/Response.pm line 142
Cpanel::Server::Response::send_response(Cpanel::Server::Response=HASH(0x3034920), Cpanel::Server::Response::Source::ReadOnlyString=HASH(0x3056498)) called at cpsrvd.pl line 3113
cpanel::cpsrvd::servcontent("document", "./frontend/jupiter/libraries/fontawesome/webfonts/fa-solid-90"..., "use_magic", 1, "static", 1, "content_type", "application/font-woff2") called at cpsrvd.pl line 2834
cpanel::cpsrvd::dodoc_cpaneld() called at cpsrvd.pl line 2113
cpanel::cpsrvd::dodoc(HASH(0x1c1e540)) called at cpsrvd.pl line 1802
cpanel::cpsrvd::handle_one_connection(5) called at cpsrvd.pl line 1149
cpanel::cpsrvd::script() called at cpsrvd.pl line 448
[2023-05-27 03:50:10 -0400] info [tailwatchd] chkservd::DiskUsage Notification => [email protected]***** via EMAIL [eventimportance => High (1)]
[2023-05-27 03:50:10 -0400] info [tailwatchd] chkservd::DiskUsage Notification => [email protected]***** via PAGER [eventimportance => High (1)]
[2023-05-27 03:52:09 -0400] warn [Internal Warning while parsing [stdin] 11363] (XID p58fxt) “/usr/local/cpanel/bin/pkgacct” reported error code “1” when it ended: at /usr/local/cpanel/Cpanel/ForkAsync.pm line 110.
Cpanel::ForkAsync::_print(Cpanel::Exception::ProcessFailed::Error=HASH(0x2784698)) called at /usr/local/cpanel/Cpanel/ForkAsync.pm line 75
Cpanel::ForkAsync::do_in_child(CODE(0x247f728)) called at bin/admin/Cpanel/backup.pl line 265
STDOUT: [2023-05-27 03:47:46 -0400] pkgacct started.
[2023-05-27 03:47:46 -0400] pkgacct version 10 - user : civiprod - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - backup: 0 - archive version: 4 - running with uid 0
[2023-05-27 03:47:46 -0400] pkgacct using '/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 1 --blocksize 4096 --rsyncable' to compress archives
[2023-05-27 03:47:46 -0400] pkgacct working dir : /home/11363.BIN_ADMIN_CPANEL_BACKUP_PL__.2e28c4d1.tmp/backup-5.27.2023_03-47-46_civiprod
[2023-05-27 03:47:46 -0400] Copying Reseller Config...[2023-05-27 03:47:46 -0400] Done
[2023-05-27 03:47:46 -0400] Copying Suspension Info (if needed)...[2023-05-27 03:47:46 -0400] Done
[2023-05-27 03:47:46 -0400] Copying Team Info (if needed)...[2023-05-27 03:47:46 -0400] Done
[2023-05-27 03:47:46 -0400] Copying installed SSL certificates and keys...[2023-05-27 03:47:46 -0400] Performing “ApacheTLS” component....
[2023-05-27 03:47:46 -0400] Completed “ApacheTLS” component.
[2023-05-27 03:47:46 -0400] Done
[2023-05-27 03:47:46 -0400] Copying DKIM keys....[2023-05-27 03:47:46 -0400] Done
[2023-05-27 03:47:46 -0400] Copying Bandwidth Data....[2023-05-27 03:47:46 -0400] Performing “Bandwidth” component....
Summary databases … done!
[2023-05-27 03:47:47 -0400] Completed “Bandwidth” component.
[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Copying Dns Zones.......beyondcollegenow.org......cc.unidosnow.org......beyondcollegenow.com...[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Copying Mail files....[2023-05-27 03:47:47 -0400] Performing “MailConfig” component....
[2023-05-27 03:47:47 -0400] Completed “MailConfig” component.
[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Copying proftpd file....[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Performing “Logs” component....
...log file sizes [247033 byte(s)]......beyondcollegenow.com.cc.unidosnow.org-ssl_log......cc.unidosnow.org-bytes_log......beyondcollegenow.cc.unidosnow.org......beyondcollegenow.com.cc.unidosnow.org......cc.unidosnow.org-ssl_log......beyondcollegenow.cc.unidosnow.org-ssl_log......cc.unidosnow.org...[2023-05-27 03:47:47 -0400] Completed “Logs” component.
[2023-05-27 03:47:47 -0400] Copy userdata...[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Copy custom virtualhost templates...[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Copying mailman lists and archives....Done copying mailman lists and archives.
[2023-05-27 03:47:47 -0400] Copying homedir.............
[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Fixing up EA4 .htaccess blocks: /home/civiprod/public_html/.htaccess /home/civiprod/beyondcollegenow/.htaccess Done.
[2023-05-27 03:47:47 -0400] Performing “Postgresql” component....
[2023-05-27 03:47:47 -0400] Completed “Postgresql” component.
[2023-05-27 03:47:47 -0400] Performing “Mysql” component....
Load watching suspended due to SIGUSR1
[2023-05-27 03:47:47 -0400] Determining mysql dbs...[2023-05-27 03:47:47 -0400] ...mysqldump version: 5.7.42...[2023-05-27 03:47:47 -0400] ...mysql version: 5.7...[2023-05-27 03:47:47 -0400] Saving mysql privs...[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Saving mysql authentication information...[2023-05-27 03:47:47 -0400] Done
[2023-05-27 03:47:47 -0400] Storing MySQL databases...[2023-05-27 03:47:47 -0400] Storing database civiprod_civicrm
[2023-05-27 03:47:51 -0400] Storing database civiprod_drupal
[2023-05-27 03:47:52 -0400] Storing database civiprod_logging
[2023-05-27 03:47:56 -0400] Done
[2023-05-27 03:47:56 -0400] ...Done
[2023-05-27 03:47:56 -0400] ...Done
Load watching resumed due to SIGUSR2
[2023-05-27 03:47:56 -0400] Completed “Mysql” component.
[2023-05-27 03:47:56 -0400] Performing “MysqlRemoteNotes” component....
[2023-05-27 03:47:56 -0400] Completed “MysqlRemoteNotes” component.
[2023-05-27 03:47:56 -0400] Performing “CpUserFile” component....
[2023-05-27 03:47:56 -0400] Completed “CpUserFile” component.
[2023-05-27 03:47:56 -0400] Performing “Cron” component....
[2023-05-27 03:47:56 -0400] Readable crontab file (/var/spool/cron/civiprod) found; copying …
[2023-05-27 03:47:56 -0400] Completed “Cron” component.
[2023-05-27 03:47:56 -0400] Performing “Quota” component....
[2023-05-27 03:47:56 -0400] Completed “Quota” component.
[2023-05-27 03:47:56 -0400] Performing “Integration” component....
[2023-05-27 03:47:56 -0400] Completed “Integration” component.
[2023-05-27 03:47:56 -0400] Performing “AuthnLinks” component....
[2023-05-27 03:47:56 -0400] Completed “AuthnLinks” component.
[2023-05-27 03:47:56 -0400] Performing “APITokens” component....
[2023-05-27 03:47:56 -0400] Completed “APITokens” component.
[2023-05-27 03:47:56 -0400] Performing “DNSSEC” component....
[2023-05-27 03:47:56 -0400] Completed “DNSSEC” component.
[2023-05-27 03:47:56 -0400] Performing “Custom” component....
[2023-05-27 03:47:56 -0400] No custom components to perform.
[2023-05-27 03:47:56 -0400] Completed “Custom” component.
[2023-05-27 03:47:56 -0400] Performing “AutoSSL” component....
[2023-05-27 03:47:56 -0400] Completed “AutoSSL” component.
[2023-05-27 03:47:56 -0400] Storing Subdomains....
[2023-05-27 03:47:56 -0400] Done
[2023-05-27 03:47:56 -0400] Storing Parked Domains....
[2023-05-27 03:47:56 -0400] Done
[2023-05-27 03:47:56 -0400] Storing Addon Domains....
[2023-05-27 03:47:56 -0400] Performing “Password” component....
[2023-05-27 03:47:56 -0400] Completed “Password” component.
[2023-05-27 03:47:56 -0400] Performing “DigestShadow” component....
[2023-05-27 03:47:56 -0400] Completed “DigestShadow” component.
[2023-05-27 03:47:56 -0400] Copying shell.......[2023-05-27 03:47:56 -0400] Done
[2023-05-27 03:47:56 -0400] Performing “WebCalls” component....
[2023-05-27 03:47:56 -0400] Completed “WebCalls” component.
[2023-05-27 03:47:56 -0400] Performing “BrandCustomizations” component....
[2023-05-27 03:47:56 -0400] Completed “BrandCustomizations” component.
[2023-05-27 03:47:56 -0400] Performing “PublicContact” component....
[2023-05-27 03:47:56 -0400] Completed “PublicContact” component.
[2023-05-27 03:47:56 -0400] Performing “MailLimits” component....
[2023-05-27 03:47:56 -0400] Completed “MailLimits” component.
[2023-05-27 03:47:56 -0400] Performing “LinkedNodes” component....
[2023-05-27 03:47:56 -0400] Completed “LinkedNodes” component.
[2023-05-27 03:47:56 -0400] Creating Archive ....................................................................................
..........
[2023-05-27 03:52:09 -0400]
[2023-05-27 03:52:09 -0400] One or more files in the home directory were not readable and were not copied. Please review the home directory upon completion of transfer
[2023-05-27 03:52:09 -0400] WARN: Warning(s) encountered in tar during archiving:
[2023-05-27 03:52:09 -0400] /home/civiprod/public_html/sites/default/files/Parent_Cert.docx: Cannot open file: Permission denied at bin/pkgacct.pl line 1446.
[2023-05-27 03:52:09 -0400] /home/civiprod/public_html/un/drupal_upgrade_archive/phpinfo.php: Cannot open file: Permission denied at bin/pkgacct.pl line 1446.
[2023-05-27 03:52:09 -0400] /home/civiprod/.trash/backup-1.5.2023_20-05-37_civiprod.tar.gz: Cannot write file to archive: Broken pipe at bin/pkgacct.pl line 1446.
[2023-05-27 03:52:09 -0400] b_find(): /home/11363.BIN_ADMIN_CPANEL_BACKUP_PL__.2e28c4d1.tmp/backup-5.27.2023_03-47-46_civiprod/htaccess/_home_civiprod_public_html_.htaccess: Broken pipe
[2023-05-27 03:52:09 -0400] b_find(): /home/11363.BIN_ADMIN_CPANEL_BACKUP_PL__.2e28c4d1.tmp/backup-5.27.2023_03-47-46_civiprod/htaccess/_home_civiprod_public_html_.htaccess: Broken pipe[2023-05-27 03:52:09 -0400] WARN: Warning(s) encountered in gzip during archiving:
[2023-05-27 03:52:09 -0400] pigz: abort: write error on <stdout> (No space left on device)[2023-05-27 03:52:09 -0400] ERROR: The subprocess “/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 1 --blocksize 4096 --rsyncable” reported error number 28 when it ended.: pigz: abort: write error on <stdout> (No space left on device) at /usr/local/cpanel/Cpanel/IO/Tarball.pm line 325.
Cpanel::Exception::ProcessFailed::Error/(XID p58fxt) “/usr/local/cpanel/bin/pkgacct” reported error code “1” when it ended:
at /usr/local/cpanel/Cpanel/CpuWatch.pm line 113.
Cpanel::CpuWatch::_run("program", "/usr/local/cpanel/bin/pkgacct", "args", ARRAY(0x276ebf8), "stdout", GLOB(0x2772ed0), "stderr", GLOB(0x2772ed0), ...) called at /usr/local/cpanel/Cpanel/CpuWatch.pm line 42
Cpanel::CpuWatch::run("program", "/usr/local/cpanel/bin/pkgacct", "args", ARRAY(0x276ebf8), "stdout", GLOB(0x2772ed0), "stderr", GLOB(0x2772ed0)) called at bin/admin/Cpanel/backup.pl line 193
main::__ANON__() called at /usr/local/cpanel/3rdparty/perl/536/cpanel-lib/Try/Tiny.pm line 100
eval {...} called at /usr/local/cpanel/3rdparty/perl/536/cpanel-lib/Try/Tiny.pm line 91
Try::Tiny::try(CODE(0x2772dc8), Try::Tiny::Catch=REF(0xea38a0)) called at bin/admin/Cpanel/backup.pl line 207
main::__ANON__() called at /usr/local/cpanel/Cpanel/ForkAsync.pm line 72
eval {...} called at /usr/local/cpanel/Cpanel/ForkAsync.pm line 72
Cpanel::ForkAsync::do_in_child(CODE(0x247f728)) called at bin/admin/Cpanel/backup.pl line 265
Timed out while waiting to flush final write queue contents! at /usr/local/cpanel/Cpanel/Server/WebSocket/AppBase/Streamer.pm line 393.
==> cpsrvd 11.110.0.6 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2023-05-27 04:20:10 -0400] info [cpsrvd] version 11.110.0.6 online
==> cpsrvd 11.110.0.6 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2023-05-27 04:30:02 -0400] info [cpsrvd] version 11.110.0.6 online
From the below line in your logs, it clearly points out that there is insufficient space on your server hard disk for backup to complete.
Make sure you have sufficient free space for backup to complete.
Code:
[2023-05-27 03:52:09 -0400] pigz: abort: write error on <stdout> (No space left on device)[2023-05-27 03:52:09 -0400] ERROR: The subprocess “/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 1 --blocksize 4096 --rsyncable” reported error number 28 when it ended.: pigz: abort: write error on <stdout> (No space left on device) at /usr/local/cpanel/Cpanel/IO/Tarball.pm line 325.
Thanks, I was lost in the detail. The /home/.trash folder had grown out of control. The only guidance I can find from cPanel to manage disk usage is here. Very manual!
I still need to tackle /tmp and that guidance is no help there. Apparently deleting files in /tmp can break things. Lots of recommendations to clean out /tmp on shutdown. Seems extreme to require a reboot, is there a safe alternative? If it takes a reboot, would appreciate thoughts on how to run some local commands during server reboot. TIA.
I still need to tackle /tmp and that guidance is no help there. Apparently deleting files in /tmp can break things. Lots of recommendations to clean out /tmp on shutdown. Seems extreme to require a reboot, is there a safe alternative? If it takes a reboot, would appreciate thoughts on how to run some local commands during server reboot. TIA.
Last edited:
@SRQ_civicrm - I wouldn't be afraid of /tmp. You can see what's in there with the typical "du -h --max-depth 1" and "ls" commands like any other system, and then you'll be able to see what is using the space. The only thing you really can't remove is the MySQL sock file link - anything else should be temporary to running processes and not cause the server to fill up.
You can also increase the size of /tmp if necessary:
You can also increase the size of /tmp if necessary:
| Thread starter | Similar threads | Forum | Replies | Date |
|---|---|---|---|---|
| A | Incremental backup stuck at "Pruning backup directory" | Backups | 1 | |
| I | Old backups stuck in backup folder | Backups | 5 | |
| B | How to stop/terminate a backup restoration that is stuck on "Restoring Account" ? | Backups | 1 | |
|
Backup Restore Stuck | Backups | 3 | |
| P | cPanel Backup stuck? | Backups | 3 |