#387 Issue `closed`: REAR with BACKUP=NSR breaks YUM on RHELv6¶

Labels: support / question, won't fix / can't fix / obsolete

pteegarden opened issue at 2014-04-03 01:19:¶

"I have installed a standard system with RHEL6.

I installed "rear" (Relax and Recover) from the epel. No problem.
I installed the dependencies (genisoimage, syslinux, and mtools) No problem.

If I run "rear mkrescue" with the standard configuration (OUTPUT=ISO), all is well.
IF I run "rear mkrescue" with the additional configuration option (BACKUP=NSR), it breaks any further attempt to use YUM.

It does this, as far as I can tell, because it changes the standard library association from

/lib64/libexpat.so.1 -> libexpat.so.1.5.2

to

/lib64/libexpat.so.1 -> ./libexpat.so

I can get YUM working again by putting the original link back in place.

gdha commented at 2014-04-03 06:22:¶

Strange story - rear should not touch the live system. However, the subject and content saying something different - OUTPUT=NSR and BACKUP=NSR ?
Obviously, only BACKUP=NSR is valid.
Perhaps, show us the local.conf file and add (via gist) a full logging and debugging session to proof your case? I really cannot belief it so far.

pteegarden commented at 2014-04-03 17:29:¶

Yes, the subject should say BACKUP=NSR, not OUTPUT=NSR. Is there a way to change that?

local.conf simply contains the following two lines:

OUTPUT=ISO
BACKUP=NSR

For completeness, os.conf contains:

OS_VENDOR=RedHatEnterpriseServer
OS_VERSION=6

The rear logging doesn't indicate that anything is wrong:

[root@wedjat tmp]# rear -v mkrescue
Relax-and-Recover 1.15 / Git
Using log file: /var/log/rear/rear-wedjat.log
Creating disk layout
Creating root filesystem layout
EMC Networker will recover these filesystems: / /boot /home
TIP: To login as root via ssh you need to set up /root/.ssh/authorized_keys or SSH_ROOT_PASSWORD in your configuration file
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Making ISO image
Wrote ISO image: /var/lib/rear/output/rear-wedjat.iso (100M)
Saving result files with NSR (EMC NetWorker)
If the RETENTION_TIME="1 day" is too low please add RETENTION_TIME variable in /etc/rear/local.conf
 pool           retent  name
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= [...I inserted dashes in this line...]
ddservers       03/22/14 /var/lib/rear/output/rear-wedjat.iso

One might be lulled into thinking that everything is OK at this point, since the system is running without incident.
However, to illuminate that all is not well, simply type a YUM command. I use something simple like asking YUM to list it's repos:

[root@wedjat tmp]# yum repolist
Plugin "subscription-manager" can't be imported
Loaded plugins: product-id, rhnplugin, security
/usr/lib64/python2.6/xmlrpclib.py:612: DeprecationWarning: The xmllib module is obsolete.  Use xml.sax instead.
  import xmllib # lazy subclassing (!)
This system is receiving updates from RHN Classic or RHN Satellite.
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 285, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 136, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 438, in doCommands
    return self.yum_cli_commands[self.basecmd].doCommand(self, self.basecmd, self.extcmds)
  File "/usr/share/yum-cli/yumcommands.py", line 863, in doCommand
    base.repos.populateSack()
  File "/usr/lib/python2.6/site-packages/yum/repos.py", line 308, in populateSack
    sack.populate(repo, mdtype, callback, cacheonly)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 165, in populate
    if self._check_db_version(repo, mydbtype):
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 223, in _check_db_version
    return repo._check_db_version(mdtype)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1256, in _check_db_version
    repoXML = self.repoXML
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1455, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1447, in _getRepoXML
    self._loadRepoXML(text=self)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1437, in _loadRepoXML
    return self._groupLoadRepoXML(text, self._mdpolicy2mdtypes())
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1412, in _groupLoadRepoXML
    if self._commonLoadRepoXML(text):
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1240, in _commonLoadRepoXML
    self._repoXML = self._parseRepoXML(result)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1036, in _parseRepoXML
    return repoMDObject.RepoMD(self.id, local)
  File "/usr/lib/python2.6/site-packages/yum/repoMDObject.py", line 124, in __init__
    self.parse(srcfile)
  File "/usr/lib/python2.6/site-packages/yum/repoMDObject.py", line 140, in parse
    parser = iterparse(infile)
  File "/usr/lib/python2.6/site-packages/yum/misc.py", line 1141, in cElementTree_iterparse
    _cElementTree_import()
  File "/usr/lib/python2.6/site-packages/yum/misc.py", line 1136, in _cElementTree_import
    import cElementTree
ImportError: No module named cElementTree
[root@wedjat tmp]#

This happens because the /lib64/libexpat.so.1 has been changed.
Changing the link back (as explained in my first post) fixes the problem.

Would you prefer a "rear -dv mkrescue"?

pteegarden commented at 2014-04-03 17:48:¶

I changed the title. I am a newbie at using GIT, so my learning curve is a little slow.
Is there a way to upload and attach a file (like /var/log/rear/rear-wedjat.log)?

gdha commented at 2014-04-04 14:42:¶

@pteegarden perhaps you could install yum-plugin-verify (not sure is has the same name on CentOS) and then verify

[gdha@fedora20 ~]$ ll /lib64/libexpat.so.1
lrwxrwxrwx. 1 root root 17 Dec 18 09:05 /lib64/libexpat.so.1 -> libexpat.so.1.6.0
[gdha@fedora20 ~]$ rpm -qf /lib64/libexpat.so.1
expat-2.1.0-7.fc20.x86_64
[gdha@fedora20 ~]$ yum verify expat
Loaded plugins: langpacks, refresh-packagekit, verify
verify done

If it shows some non-compliant then re-install the package and verify it shows the correct information before running rear again. After running rear do the yum verify again. It could be that EMC NetWorker is responsible for changing the link. We need to eliminate the one or the other. Does this sound like a plan?

pteegarden commented at 2014-04-04 21:06:¶

Installed yum-plugin-verify and ran it:

[root@wedjat lib64]# ll libexpat*
-rwxr-xr-x  1 root root 139368 Aug  6  2012 libexpat.so
lrwxrwxrwx  1 root root     17 Apr  4 13:29 libexpat.so.1 -> libexpat.so.1.5.2
-rwxr-xr-x. 1 root root 167648 Apr 27  2012 libexpat.so.1.5.2

[root@wedjat lib64]# yum verify expat
Loaded plugins: product-id, rhnplugin, security, subscription-manager, verify
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
verify done

[root@wedjat lib64]# rpm -qf /lib64/libexpat.so.1
expat-2.0.1-11.el6_2.x86_64

I then ran rear -dvS mkrescue and looked at the libexpat.so.1 link after every step.
The original link was OK up to this question:

Press ENTER to include '/usr/share/rear/build/GNU/Linux/10_copy_as_is.sh' ...

After I pressed enter, the link changed to
lrwxrwxrwx 1 root root 13 Apr 4 13:45 /lib64/libexpat.so.1 -> ./libexpat.so

It surprised me that yum verify expat actually executed with this "bad" link in place:

[root@wedjat lib64]# yum verify expat
Plugin "subscription-manager" can't be imported
Loaded plugins: product-id, rhnplugin, security, verify
/usr/lib64/python2.6/xmlrpclib.py:612: DeprecationWarning: The xmllib module is obsolete.  Use xml.sax instead.
  import xmllib # lazy subclassing (!)
This system is receiving updates from RHN Classic or RHN Satellite.
==================== Installed Packages ====================
expat.x86_64 : An XML parser library
    File: /lib64/libexpat.so.1
        Problem:  symlink does not match
        Current:  ./libexpat.so
        Original: libexpat.so.1.5.2
verify done

[root@wedjat lib64]# ll libexpat*
-rwxr-xr-x  1 root root 139368 Aug  6  2012 libexpat.so
lrwxrwxrwx  1 root root     13 Apr  4 13:45 libexpat.so.1 -> ./libexpat.so
-rwxr-xr-x. 1 root root 167648 Apr 27  2012 libexpat.so.1.5.2

"yum repolist" or "yum check-update" fails at this point, as noted before.

pteegarden commented at 2014-04-04 21:12:¶

Putting the link back in place makes it happier:

[root@wedjat lib64]# ll libexpat*
-rwxr-xr-x  1 root root 139368 Aug  6  2012 libexpat.so
lrwxrwxrwx  1 root root     13 Apr  4 13:45 libexpat.so.1 -> ./libexpat.so
-rwxr-xr-x. 1 root root 167648 Apr 27  2012 libexpat.so.1.5.2

[root@wedjat lib64]# rm libexpat.so.1
rm: remove symbolic link `libexpat.so.1'? y

[root@wedjat lib64]# ln -s libexpat.so.1.5.2 libexpat.so.1

[root@wedjat lib64]# ll libexpat*
-rwxr-xr-x  1 root root 139368 Aug  6  2012 libexpat.so
lrwxrwxrwx  1 root root     17 Apr  4 14:01 libexpat.so.1 -> libexpat.so.1.5.2
-rwxr-xr-x. 1 root root 167648 Apr 27  2012 libexpat.so.1.5.2

[root@wedjat lib64]# yum verify expat
Loaded plugins: product-id, rhnplugin, security, subscription-manager, verify
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
verify done

[root@wedjat lib64]# yum repolist
Loaded plugins: product-id, rhnplugin, security, subscription-manager, verify
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
This system is receiving updates from RHN Classic or RHN Satellite.
repo id                                                  repo name                                                                                     status
epel                                                     Extra Packages for Enterprise Linux 6 - x86_64                                                10,662
rhel-x86_64-server-6                                     Red Hat Enterprise Linux Server (v. 6 for 64-bit x86_64)                                      12,405
repolist: 23,067
[root@wedjat lib64]#

gdha commented at 2014-04-06 18:32:¶

would like to see a full debugging log file from rear to watch the script mentioned above in full action. Please upload it to gist https://gist.github.com/ and link the url into this issue

pteegarden commented at 2014-04-07 17:25:¶

/var/log/rear/rear-wedjat.log uploaded to
https://gist.github.com/anonymous/bda06f447a57a3cb5a2c

gdha commented at 2014-04-08 07:23:¶

The only reference I found was /usr/lib/nsr/lib64/libexpat.so in the log.
If you could redo the exercise with options /usr/sbin/rear -DdvS mkrescue then we see the real debugging. The above libexpat must be dealt with wrongly somehow? But, now I cannot see it . Thank you for your investigation time.

pteegarden commented at 2014-04-09 00:18:¶

Did a “rear -DdvS mkrescue “

good up to the prompt
"Press ENTER to include '/usr/share/rear/build/GNU/Linux/10_copy_as_is.sh' ..."

After hitting the CR, the link changes as before.

[root@wedjat ~]# yum verify expat
Plugin "subscription-manager" can't be imported
Loaded plugins: product-id, rhnplugin, security, verify
/usr/lib64/python2.6/xmlrpclib.py:612: DeprecationWarning: The xmllib module is obsolete. Use xml.sax instead.
import xmllib # lazy subclassing (!)
This system is receiving updates from RHN Classic or RHN Satellite.
==================== Installed Packages ====================
expat.x86_64 : An XML parser library
File: /lib64/libexpat.so.1
Problem: symlink does not match
Current: ./libexpat.so
Original: libexpat.so.1.5.2
verify done

the /var/log/rear/ rear-wedjat.log file is 5.23 MB.
The URL is
https://gist.github.com/anonymous/914e0c3d85209b0b6709

gdha commented at 2014-04-11 13:20:¶

 ls -l /usr/lib/nsr/lib64/libexp*
-rwxr-xr-x 1 root root 137064 Aug  7  2012 /usr/lib/nsr/lib64/libexpat.so
lrwxrwxrwx 1 root root     13 Jun 20  2013 /usr/lib/nsr/lib64/libexpat.so.1 -> ./libexpat.so

The 10_copy_as_is.sh does a tar create and extract, but that does work fine (checked this manually with nsr). For the rest I don't see anything wrong, which means I probably don't understand it...

pteegarden commented at 2014-04-18 21:55:¶

Ok… I don’t understand it either.
I was hoping that some insight would spring forth, but so far…??

What is our next step?

From: gdha [mailto:notifications@github.com]
Sent: Friday, April 11, 2014 6:21 AM
To: rear/rear
Cc: Teegarden, Paul
Subject: Re: [rear] REAR with BACKUP=NSR breaks YUM on RHELv6 (#387)

ls -l /usr/lib/nsr/lib64/libexp*

-rwxr-xr-x 1 root root 137064 Aug 7 2012 /usr/lib/nsr/lib64/libexpat.so

lrwxrwxrwx 1 root root 13 Jun 20 2013 /usr/lib/nsr/lib64/libexpat.so.1 -> ./libexpat.so

The 10_copy_as_is.sh does a tar create and extract, but that does work fine (checked this manually with nsr). For the rest I don't see any wrong, which means I probably don't understand it...

—
Reply to this email directly or view it on GitHubhttps://github.com/rear/rear/issues/387#issuecomment-40202274.