What's it take to get proper testing?

[JustinK]

Taking this out of the previous discussion because it probably wasn't pulling people in to check on it with that topic, what would it take to get a change in the way updates are put out? How many &votes&? I know a lot of hosts as well as resellers that are a little tired of the updates coming through and having their servers used as a beta environment. I realize that people can always switch control panels, but why can't we just improve the way this one is put out? It would only improve cpanel to have better beta testing.

We need a way to update to something where the features have all been tested (not only by darkorb as that hasn't been working out) and proved stable. Granted that bugs may pop up still, but at least it shouldn't be as bad as just releasing a full blown test on production servers. It would be great to be able to do an update where all the features there have been tested for a good couple of weeks, instead of waiting for one problem to be fixed, only to download it and find one problem's been fixed, but two more new features got put in with bugs.

Why not set certain features to be done and tested (letting us know these plans ahead of time via a mailing list or something) for a certain release and then letting people download and use that release for a week. Fix any further bugs (this is AFTER 2 or so weeks of testing beforehand) and then begin work on the next release. You don't have to be coming out with new features once a week. Give a good long gap inbetween features and get some breathes in.

Help us help you. There are some of us here more than willing to go through with such a beta server idea, it just needs to be implimented. This is not meant to be a flame or anything of that nature. Nick's been extremely helpful with everything, I and others just feel there needs to be a little change in the structure of releases.

 

[websnail]

[quote:3d02d26b7c][i:3d02d26b7c]Granted that bugs may pop up still, but at least it shouldn't be as bad as just releasing a full blown test on production servers. It would be great to be able to do an update where all the features there have been tested for a good couple of weeks, instead of waiting for one problem to be fixed, only to download it and find one problem's been fixed, but two more new features got put in with bugs.[/i:3d02d26b7c][/quote:3d02d26b7c]
This is definitely a point that is bothering a number of end-users and speaking as a developer myself the fact that there's $100 a month per server going into this there really shouldn't be any excuse for not testing things properly in a &true scenario& before releasing. If I did this with my end users I'd get lynched.

One possible solution might be to look for bodies willing to do soak testing in a semi-production environment where they are offering &free& services to the great unwashed who want everything and nothing and are quick to complain. They can't expect to get something for nothing and you get to test things in a realistic environment. Everyone wins.

Certainly that would improve feedback, albeit complete with expletives.

Ultimately thought it's worth considering that with other panel software in development and looking to address issues such as these... if new &features& are more thoroughly tested then there's less chance of a mass abandon ship when realistic/comparable alternatives becomes available.

Just my, albeit limited, 2penneth worth.

 

[Jacynthe]

I couldn't agree more with that. The host where I am updated CPanel to version 5 and now I find myself explaining to our clients why &Yes, there is a function change your domain password& in Cpanel and &No, it doesn't work.& &Yes, I know there's a bug and that there is no such word as spooglan34 in the dictionnary&...

As a theme/skin designer for CPanel (albeit a new one at it), I find myself always wondering if it's really worth it to include all the functions since some of them give us more headaches than functionnality at the time of their implementation.

Don't become the next Microsoft please: test your stuff before making it available. Don't just be happy with patching the same thing 6 times. Please?

 

[qualitum]

I agree.
Does anyone remember the &gorgo& theme in WHM? Even a blind tester would have seen that it didn't work. Ehm, maybe not, but you get the point, right?

 

[Keegan]

This may be an indication of a lack of a lab on the development side.

Here, and in another location there is a duplicate server lab. (Two different lab results)

If they aren't using the lab, and following strict protocal for development (Keeping accurate and stepped logs) this type of inaccurate development, and corruption of intended results will make itself known.

Which is what I think you are referring to.

So, there might be a need for better development protocal and the original thread starter may have a point that should not only be considered but heeded.

 

[shortland]

I agree with Justink. I like Cpanel, but the problems with bugs popping up following updates is getting to be a problem for my clients and me. I do not mind waiting for features to be added and updated provided what we get at the end is a more stable and bug free product.

 

[bear]

Exactly so, all points made so far.
Not only does this make it difficult to keep up with testing at our end, but also for documentation to provide our clients. With rapid changes to what's included, and not knowing what is going to work (or break) with these changes, I find myself spending loads of time trying everything out very frequently, just so I can be informed when the clients begin complaining something's broken.

Better testing at the provider's end, less 'extras' alongside bug fixes, and I'd be much happier using this, personally.
As websnail said, correcting this now, means less folks will be eager to jump ship when the competition releases an alternative. Preventative measures now will almost certainly pay off in the long run.

 

[Jacynthe]

Doesn't work when you don't have root access and use virtual web hosting.

 

[benoire]

I concur with everything here I think a beta/stable option would be excellent, whereby you can choose just to have updates you know are going to work.

 

[JustinK]

[quote:d733ddcc4c][i:d733ddcc4c]Originally posted by thaphantom[/i:d733ddcc4c]
Now if u look under Cpanel/WHM Update Preferences cant you change it to Manual Updates Only. This wont give u the betas and when u see they work fine, you can update it.
I could see darkorb putting in an option like Beta/Stable in there

and then have him mark each release either beta or stable and the users can then click through it that way...

I dunno my .02 on it[/quote:d733ddcc4c]

I do manual updates on all but one server and of course the security updates. The only problem with updating is, you can wait until one feature is stable and as soon as you do the update, you discover that something else has been added in that still needs more testing. I feel there needs to be some solid features for a version before releasing it as an update instead of having one thing fixed and another that still needs further testing.

I'm extremely interested in what the developers think of this. This is in no way meant to put them down, but the idea is to make the product more solid.

 

[SHSaeed]

Good point Justin. It would be great if they could release BETA and STABLE versions and give us the choice to choose if we only want to STABLE releases or BETA as well.

 

[Jacynthe]

[quote:9a888a75fc][i:9a888a75fc]Originally posted by thaphantom[/i:9a888a75fc]
if you dont like that u need to get a new webhost...[/quote:9a888a75fc]

Boy... I'd have to be one bitchy impatient puppy to change host ONLY because I don't like how they update CPanel!!!

 

[mccmikey]

Perhaps Cpanel developers could release cpanel in the same way that Debian releases their versions of their Operating System.

ie - they have three versions

- Stable (currently called Woody)
- Testing - usually stable, with newer features
- Unstable - cutting edge, but not guaranteed stable.

(See http://www.debian.org/releases/ for more info.)

Two versions would probably be sufficient for Cpanel - and give the clients a choice of which one they use; and a means of upgrading as testing eventually becomes stable, and stable becomes stale

I agree entirely with the need for better testing - I am a client of a host, not a host, but have still had fancy things happen from time to time; and blamed the host for them. Were I in the drivers seat I'd be ghosting my machine before each update based on what I've heard

Another suggestion: Perhaps you could have an add/remove components option; so Cpanel hosts could remove a feature that is not working yet - for example if Horde was broken, they could uninstall or 'hide' it Of course, not having ever been on the server-side of Cpanel, I have no experince there

Cheers, Mike.

 

[ZachICU]

Commenting on a post from above about how you can turn the updates to manual.

That was a much needed move, however many of these updates contain bug updates, but also create new headaches at the same time.

More documentation on what each release updates would be very helpful if better testing doesn't happen.

For example, release 100 fixes a bug with horde, release 101 updates the mail system to cpanel6 (cause i know we are all anticipating that haha). When I go to update I can pick just to update to release 100, verse being forced to updated to release 101.

Just my ideas.

Zach

 

[kblitz]

I agree wholehartedly with the idea of proper beta testing,

Kieran

 

[bdraco]

[quote:c1ff0020e2][i:c1ff0020e2]Originally posted by ZachICU[/i:c1ff0020e2]

Commenting on a post from above about how you can turn the updates to manual.

That was a much needed move, however many of these updates contain bug updates, but also create new headaches at the same time.

More documentation on what each release updates would be very helpful if better testing doesn't happen.

For example, release 100 fixes a bug with horde, release 101 updates the mail system to cpanel6 (cause i know we are all anticipating that haha). When I go to update I can pick just to update to release 100, verse being forced to updated to release 101.

Just my ideas.

Zach[/quote:c1ff0020e2]

Check /usr/local/cpanel/ChangeLog for details on each update.

---

All future releases should start being tagged &RELEASE& instead of &STABLE.& When one is deamed stable enough it will be marked as STABLE and the option to follow the STABLE or RELEASE trees will appear in WHM.

 

[ukhost]

Hi,

Now layer2 shows 116 as the latest build but on doing a manual update it says 114 is the latest build!!

Kind Regards,

Neil

 

[bdraco]

[quote:f5e1bea2fd][i:f5e1bea2fd]Originally posted by ukhost[/i:f5e1bea2fd]

Hi,

Now layer2 shows 116 as the latest build but on doing a manual update it says 114 is the latest build!!

Kind Regards,

Neil[/quote:f5e1bea2fd]

Yup.. you are now on the RELEASE tree.

 

[JustinK]

[quote:8d8ec0a5d7][i:8d8ec0a5d7]Originally posted by bdraco[/i:8d8ec0a5d7]
Check /usr/local/cpanel/ChangeLog for details on each update.

---

All future releases should start being tagged &RELEASE& instead of &STABLE.& When one is deamed stable enough it will be marked as STABLE and the option to follow the STABLE or RELEASE trees will appear in WHM.
[/quote:8d8ec0a5d7]

It's a start in the right direction. Thank you Nick. Would it be possible to have the ChangeLog visible through cpanel.net? It would be great to see what changes get made before updating.

 

[bdraco]

[quote:6068d237cd][i:6068d237cd]Originally posted by JustinK[/i:6068d237cd]

[quote:6068d237cd][i:6068d237cd]Originally posted by bdraco[/i:6068d237cd]
Check /usr/local/cpanel/ChangeLog for details on each update.

---

All future releases should start being tagged &RELEASE& instead of &STABLE.& When one is deamed stable enough it will be marked as STABLE and the option to follow the STABLE or RELEASE trees will appear in WHM.
[/quote:6068d237cd]

It's a start in the right direction. Thank you Nick. Would it be possible to have the ChangeLog visible through cpanel.net? It would be great to see what changes get made before updating. [/quote:6068d237cd]

You should be able to get it off http://layer1.cpanel.net/