/etc/passwd getting corrupted

EWD

Well-Known Member
PartnerNOC
Aug 19, 2003
165
0
166
NY
Hi Guys,

I have a weird situation here that started last night.

On 2 servers now I've had the /etc/passwd file corrupted.
Each time the following is what is found in the file:

Code:
pst0^E^G^C^@^@^@^U^D^B^@^@^@^@^@^@^@^KDEADDOMAINS
        unlimited^@^@^@^FMAXSUB
        127.0.0.1^@^@^@^BIP
^@^@^@^@^FDOMAIN
        unlimited^@^@^@^FMAXSQL
        unlimited^@^@^@^FMAXFTP^H^À^@^@^@^FHASCGI^H^À^@^@^@^DDEMO
^\../../../../../../etc/passwd^@^@^@^DUSER
^Gdefault^@^@^@^KFEATURELIST^D^B^@^@^@^@^@^@^@^GDOMAINS
        unlimited^@^@^@^GBWLIMIT

0000000000^@^@^@        STARTDATE
^Droot^@^@^@^EOWNER^H^À^@^@^@^GMAXPARK
^Bx3^@^@^@^BRS
        undefined^@^@^@^DPLAN^H^À^@^@^@^HMAXADDON
^Genglish^@^@^@^DLANG
        unlimited^@^@^@^FMAXLST
        unlimited^@^@^@^FMAXPOP
All seems to be data from hosting packages.
Nothing in the logs other then errors about users not existing(because /etc/passwd is wiped)

Anyone seen anything like this happening?

Thanks :)
 

nazmy

Member
Oct 31, 2004
16
0
151
yeah we faced this too, hope cpanel got this fixed ASAP this is a serious problem
 

EWD

Well-Known Member
PartnerNOC
Aug 19, 2003
165
0
166
NY
Ok do you have a ticket in on this?

I am going to put one in now and reference this thread.

Glad it is not just me :D
 

LS_Drew

Well-Known Member
Feb 20, 2003
187
0
166
I've had this happen on two machines today, one of them twice.

The second time, /etc/passwd, /etc/group, and /etc/shadow were all empty.
 

LS_Drew

Well-Known Member
Feb 20, 2003
187
0
166
WHM 11.2.0 cPanel 11.8.0-C15689

Centos 4.5

That's one of them.

the other one that did it twice today, I downgraded to stable after it did it again. I should have noted the version, but I didn't.

When I did the downgrade, I got an email to update perl, which I did immediately.
 

Todd Mitchell

Well-Known Member
Staff member
Nov 13, 2006
301
1
243
Houston, TX
I recommend updating to the latest version of the build. The only thing I can see so far with this issue is that all reported cases of this are running an out of date build.

The latest builds are:

11.11.0-STABLE_16999
11.15.0-RELEASE_17665
11.15.0-CURRENT_17700
11.15.0-EDGE_17700

We are still looking into this issue to pinpoint the exact cause.
 

EWD

Well-Known Member
PartnerNOC
Aug 19, 2003
165
0
166
NY
Just had it happen to another one:

WHM 11.2.0 cPanel 11.11.0-R16709
CENTOS Enterprise 4.5 i686 - WHM X v3.1.0
 

cPanelNick

Administrator
Staff member
Mar 9, 2015
3,488
35
158
cPanel Access Level
DataCenter Provider
Hi Nick,

So far all older.
I would advise updating to 16800+ on all machines if you haven't already done so (as this is the safest course of action at this point). These builds have much safer locking code. However I'm not sure if its 16800+ or a later version that solves the problem at this time. Likewise the latest builds appear immune if this is related.

Update: It has now has something to do with changing passwords, however this is not yet confirmed.
 

cPanelNick

Administrator
Staff member
Mar 9, 2015
3,488
35
158
cPanel Access Level
DataCenter Provider
Can you check the rlog on /etc/passwd? That might be helpful in determining the cause.

You should be able to restore the latest editted version of /etc/passwd by running

co /etc/passwd
chmod 644 /etc/passwd
 

EWD

Well-Known Member
PartnerNOC
Aug 19, 2003
165
0
166
NY
Can you check the rlog on /etc/passwd?

Nothing really showing at the time it happened:

Code:
[email protected] [~]# rlog /etc/passwd

RCS file: /etc/passwd,v
Working file: /etc/passwd
head: 1.56
branch:
locks: strict
        root: 1.56
access list:
symbolic names:
keyword substitution: kv
total revisions: 56;    selected revisions: 56
description:
Init by /scripts/realadduser
----------------------------
revision 1.56   locked by: root;
date: 2007/10/10 12:56:25;  author: root;  state: Exp;  lines: +0 -1
Modified by /usr/local/cpanel/whostmgr/bin/whostmgr .
----------------------------
revision 1.55
date: 2007/08/31 23:30:49;  author: root;  state: Exp;  lines: +1 -0
Modified by /scripts/realadduser .
----------------------------
revision 1.54
date: 2007/07/01 14:23:56;  author: root;  state: Exp;  lines: +1 -1
Modified by /usr/local/cpanel/whostmgr/bin/whostmgr .
----------------------------
 

mweb

Member
Mar 11, 2003
23
0
151
One server:

WHM 11.2.0 cPanel 11.8.0-C15845
CENTOS Enterprise 3.9 i686 - WHM X v3.1.0

Coincidentally, it just recently started receiving regular nessus scanning from scanalert.com, and just resumed web log processing after a short hiatus. Not sure if that matters, but it does put ../../../../../etc/passwd in the error logs.
 

eperdeme

Well-Known Member
Oct 15, 2003
58
0
156
Manchester, UK
cPanel Access Level
DataCenter Provider
Hmm, same problem also.

WHM 11.2.0 cPanel 11.6.0-R15076
REDHAT Enterprise 5 i686 - WHM X v3.1.0

Also started on 20th after months of no problems. Appears to be a possible exploit in the wild I would presume causing this.
 
Last edited:

lbccserv

Active Member
Mar 23, 2004
38
0
156
I have the same issue:

WHM 11.2.0 cPanel 11.7.0-E14936
FEDORA 2 i686 - WHM X v3.1.0

moving passwd.OLD ---> passwd fixed the issue for me as well
 
Last edited:

handsonhosting

Well-Known Member
Feb 17, 2002
151
0
316
Omaha, NE
cPanel Access Level
Root Administrator
Hey Emerson (fancy meeting you here! :))

Just thought I'd document another incident for this issue.

cPanel Version 11.8.0-C16276

Running an upgrade on that server currently. What bothers me is that there was NO changes made to that server since that version installation MONTHS ago.

There shouldn't have been ANY password changes etc, as there is only ONE accounton the machine and it's used strictly as a storage space for myself.
 

VeZoZ

Well-Known Member
Dec 14, 2002
245
0
166
cPanel Access Level
DataCenter Provider
Had this happen at the beginning of September

At the time I was told this could have been caused by anything from a Race condition, corrupt library or to a Load issue at the time the /etc/passwd file was modified when this occurred. It was also unlikely to occur again. I was also the only one to experience this problem and was told it was not a cPanel bug.

Well thanks to this post I at least know it more than likely wasn't something I could have stopped.
 

EWD

Well-Known Member
PartnerNOC
Aug 19, 2003
165
0
166
NY
Hey Emerson (fancy meeting you here! )
-- Sup dude :)

Just thought I'd document another incident for this issue.

cPanel Version 11.8.0-C16276

Running an upgrade on that server currently. What bothers me is that there was NO changes made to that server since that version installation MONTHS ago.
-- Same here. The machines that have been affected did not have any changes done.

There shouldn't have been ANY password changes etc, as there is only ONE accounton the machine and it's used strictly as a storage space for myself.
-- What do you store in there, you fishing gear? :D