#38 Issue closed: Implement automated testing (and define supported targets)

dagwieers opened issue at 2012-03-29 12:46:

As mentioned often on the mailinglist we need to have automated testing infrastructure and a defined set of supported targets in order to guarantee a certain level of stability and find distribution-specific regressions.

There are a few upcoming stuff we would like to improve which is going to be a major pain without such infrastructure, notably:

  • reorganizing the source structure (#5) partly solved
  • standardize bash syntax for the better (#6)
  • getting rid of the /usr/lib and /usr/bin symlink (#36) already solved
  • redesign OUTPUT_URL and BACKUP_URL logic (#37)
  • fix identation and cosmetic changes (#57)

So let's start discussing the list of supported targets, and what is required for the ideal automated testing infrastructure.

Let the games begin !

Automated testing infrastructure

Requirements

  • Orchestrate: build VM, run rear mkbackup, reboot VM using rescue image and recover VM (using Ansible)
  • Provide answers non-interactively when needed
  • Centralize all log-files (for future reference/analysis)
  • Verify the original system against the restored system (compare VMs)
  • Send out email when system fails (anywhere) -> requires modifications so that Rear can mail log on error

Variations

  • Do the same with smaller/larger target VMs
  • Use different system configurations
    • e.g. LVM, filesystems, SWRAID
  • Use different boot media
    • e.g. ISO, PXE, tape, USB
  • Use different backup strategies
    • e.g. TAPE, NFS, CIFS, rsync, USB, ...

Support targets

Supported distributions

Supported distributions are distributions that have a low probability to break (Enterprise distributions) and have a large userbase and number of contributors. Due to limited access to hardware, only x86 and x86_64 architectures are supported at this time.

  • Debian 6
  • RHEL4 - Still supported until 2015 - @dagwieers
  • RHEL5 - Large userbase, still supported until 2017 - @dagwieers
  • RHEL6 - Growing userbase, still supported until 2020 - @dagwieers
  • SLES 11 SP2
  • Ubuntu LTS 10.04
  • Ubuntu LTS 12.04

Tested distributions

Tested distributions are distributions that have some interest from contributors so we can guarantee that regressions are being fixed.

  • Arch - @jhoekx
  • Fedora - @gdha
  • Gentoo
  • Ubuntu
  • OpenSUSE

Please add your distribution(s) here, when applicable add version and contributor names.

gdha commented at 2015-11-20 20:32:

The BAREOS team has implemented automated testing for rear with bareos. It is at least better then nothing so far.

[Export of Github issue for rear/rear.]