Add configuration files to rollback