Public Beta testing not optimal

[3phase]

Even when Ableton showed a massive effort to get everything done in time, and they really was hard working with an update every day, some bugs sneeked thru.

It´s´probably unavoidable and before the summer is over there will be probably a fix..but...

I had the impression that some major bugs turned up after the initial beta period... And clearing them can bring other evils up.. So it seems
that the open beta period was started too late.
I guess the problem was that the limited 500 first betas went to real ableton fans.. The ones that was waiting for the betas...
Not bad to have them..But they maybe dont do as critical tests as the not so 100% ableton happy people? They maybe dont check the critical points well enough and they maybe dont have waves licences and therefore cant report on bugs related to hacked plugs?

However looking at this forum from time to time showed that there was many bugs found and sorted during this initial beta period...
I just was a bit surprised that i stumbled within 5 minutes after loading the first public beta over a bug...i stumbled so heavy that i stopped testing after that . And the moves i have done would have been the same weeks ago..so this bug could have been found much earlier if the public beta wouldnt have been so limited.

[peeddrroo]

They maybe dont check the critical points well enough and they maybe dont have waves licences and therefore cant report on bugs related to hacked plugs?

well, i think i would rather trust a registered Live user to have a valid waves licence than someone that got his Live5 beta serial when it went 100% public.

I just was a bit surprised that i stumbled within 5 minutes after loading the first public beta over a bug...i stumbled so heavy that i stopped testing after that

and you dare making comments about the beta testing?
if everybody acted like you did, i guess there would be much more bugs in Live5.
so who's to blame?

[3phase]

They maybe dont check the critical points well enough and they maybe dont have waves licences and therefore cant report on bugs related to hacked plugs?

well, i think i would rather trust a registered Live user to have a valid waves licence than someone that got his Live5 beta serial when it went 100% public.

I just was a bit surprised that i stumbled within 5 minutes after loading the first public beta over a bug...i stumbled so heavy that i stopped testing after that

and you dare making comments about the beta testing?
if everybody acted like you did, i guess there would be much more bugs in Live5.
so who's to blame?

It´s not my job to do beta tests... I stopped testing because i realized that my attitude towards ableton since Live4 became to negativ to do objektiv
tests..
This dont prevented me from beeing found by a bug within 5 minutes... I allways try the things first that dont work easy... i ve a talent for that... I havent done timing measurements with live 5 yet...hope its ok..
I havent done the measurements because i dont like to find something.

I need Live 5... I either work with Live 3 via rewire in logic or i need a Live version that handles midi. Live 4 was not able to adress that issue and therefor a waste of money for me.

[peeddrroo]

1. nobody is forced to do some beta testing
2. how can you think that making public beta testing can be bad for the final product?
3. you found a bug while beta testing? wow, that's almost incredible. i thought beta testing was just to check that the program is a final release.

ok, i don't want to sound harsh. but can you think of a better way to test the product than doing it the way the abes do it?
making it 100% private and only for 1000 persons? or even 2000?

there will always be someone complaining (and someone else to tell him he shouldn't )

[3phase]

and you dare making comments about the beta testing?
if everybody acted like you did, i guess there would be much more bugs in Live5.
so who's to blame?

If everybody that has tested Live 5 would have found one bug there would have been 500 bugs + nobody knows how many people tested in the open period...
so i did my part...findin one bug must be enough

[3phase]

1. nobody is forced to do some beta testing
2. how can you think that making public beta testing can be bad for the final product?
3. you found a bug while beta testing? wow, that's almost incredible. i thought beta testing was just to check that the program is a final release.

ok, i don't want to sound harsh. but can you think of a better way to test the product than doing it the way the abes do it?
making it 100% private and only for 1000 persons? or even 2000?

there will always be someone complaining (and someone else to tell him he shouldn't )

public testing is good..but it has to be backed up with some professional testing. At least when you keep the testers circle small you should be a bit selectiv about the 500 participants.. When you do it by accident as the abletons did you have 50% trained users and 50% bloody beginners that dont care about rewire,timing,waves plugs...
So my critic was not again public betatesting just the way the testers came together was a bit like a shoot in the dark and i think therefore some easy to find bugs stayed up to the open period unfound.. Thats only my theorie, i might be wrong...but maybe worth to discuss...

just an add: criteria for selecting 500 testers ( and 500 should be enough to find any bug)

1)Time of beeing licenced user..it should be more than one year...

2)Setup of the user. Its useless whenn from 500 betatesterst 489 are on Pc..
Even when it needs some time its probably worth to spend one day to select a set of testers that represent a wide varity of possible setups.

[zion15]

3phase, it's pretty hard to pick testers by time of owning license or platform. for all you know, someone who's owned the license a month might be using live 10 hours daily in a complex studio setup whereas someone who's had the license for a year is a slacking graphic artist who is just banging techno tracks every now and then for therapy. i think the "first come first served"-idea wasn't perfect but it's one good alternative since the people who notice the announcement first are bound to be pretty active in the community at least...

by the way, was there a some kind of announcement from ableton that they were NOT using any professional fulltime testers or did someone tell you people about that? i'm just wondering since for all i know there might've been a few pro guys or a twenty man team or whatever doing testing in addition to the public beta.

[Alex]

Hi folks,

I moved this thread to the 'General' forum.

regards,
Alex

[Machinate]

1333 topics, 5986 posts in less than two months. 20 betas, tons and tons of bug-fixes, even though some users have only bothered to reported one bug each I'd say that is as good a process as any.

[raapie]

I think this beta period was the longest for Ableton right? I felt that even the first betas were more stable than I expected.

Ableton has improven their test-methods a lot. I think they are doing it perfectly. Beta testing for 5 was just great I guess. Even non-Live users could start testing it, which is great. No limitations even!

If there are issues to be solved I am sure the Abletoners will create an update as soon as possible. That's what I like about Live. It's been properly tested, but when a serious issue comes up there's a big change it will be solved quickly.

[statlerandwaldorf]

Simply.

You don't have to buy it if you don't want to. If you have an old version you still have exactly what you paid for. It is a choice.

I personally use live 5 as a professional DAW and I am yet to find any bugs in the official release that can be directly attributed to live as opposed to plugins.

[3phase]

I think this beta period was the longest for Ableton right? I felt that even the first betas were more stable than I expected.

Ableton has improven their test-methods a lot. I think they are doing it perfectly. Beta testing for 5 was just great I guess. Even non-Live users could start testing it, which is great. No limitations even!

If there are issues to be solved I am sure the Abletoners will create an update as soon as possible. That's what I like about Live. It's been properly tested, but when a serious issue comes up there's a big change it will be solved quickly.

Ok..you convinced me..i will order the update...

[anonymouse]

Even when Ableton showed a massive effort to get everything done in time, and they really was hard working with an update every day, some bugs sneeked thru.

It´s´probably unavoidable and before the summer is over there will be probably a fix..but...

I had the impression that some major bugs turned up after the initial beta period... And clearing them can bring other evils up.. So it seems
that the open beta period was started too late.
I guess the problem was that the limited 500 first betas went to real ableton fans.. The ones that was waiting for the betas...
Not bad to have them..But they maybe dont do as critical tests as the not so 100% ableton happy people? They maybe dont check the critical points well enough and they maybe dont have waves licences and therefore cant report on bugs related to hacked plugs?

However looking at this forum from time to time showed that there was many bugs found and sorted during this initial beta period...
I just was a bit surprised that i stumbled within 5 minutes after loading the first public beta over a bug...i stumbled so heavy that i stopped testing after that . And the moves i have done would have been the same weeks ago..so this bug could have been found much earlier if the public beta wouldnt have been so limited.

What a silly post.

There is no such thing as perfectly bug free software. Even air traffic control systems crash from time to time. Those with experience in software development will tell you; in complex applications bugs will ALWAYS slip through.

I think Ableton has the best approach of all the audio developer firms.

Given that you gave up after finding your first bug, I think probably your place in the beta group was a waste of time.

If you felt there were issues that weren't tested by other beta users, why didn't you test them yourself? or at least post a query to the forum?

No doubt there will be a 5.1 in several months-time to clear up some issues, but on the whole their participative beta period with the users is great.

Professional testers are an expensive resource for any software firm, both financially and organisationally. And not least because they are only required around major releases.

[3phase]

Even when Ableton showed a massive effort to get everything done in time, and they really was hard working with an update every day, some bugs sneeked thru.

It´s´probably unavoidable and before the summer is over there will be probably a fix..but...

I had the impression that some major bugs turned up after the initial beta period... And clearing them can bring other evils up.. So it seems
that the open beta period was started too late.
I guess the problem was that the limited 500 first betas went to real ableton fans.. The ones that was waiting for the betas...
Not bad to have them..But they maybe dont do as critical tests as the not so 100% ableton happy people? They maybe dont check the critical points well enough and they maybe dont have waves licences and therefore cant report on bugs related to hacked plugs?

However looking at this forum from time to time showed that there was many bugs found and sorted during this initial beta period...
I just was a bit surprised that i stumbled within 5 minutes after loading the first public beta over a bug...i stumbled so heavy that i stopped testing after that . And the moves i have done would have been the same weeks ago..so this bug could have been found much earlier if the public beta wouldnt have been so limited.

What a silly post.

There is no such thing as perfectly bug free software. Even air traffic control systems crash from time to time. Those with experience in software development will tell you; in complex applications bugs will ALWAYS slip through.

I think Ableton has the best approach of all the audio developer firms.

Given that you gave up after finding your first bug, I think probably your place in the beta group was a waste of time.

If you felt there were issues that weren't tested by other beta users, why didn't you test them yourself? or at least post a query to the forum?

No doubt there will be a 5.1 in several months-time to clear up some issues, but on the whole their participative beta period with the users is great.

Professional testers are an expensive resource for any software firm, both financially and organisationally. And not least because they are only required around major releases.

No no...i remember the Live 4 debacle..I paid 120 or 130 euros for the update and got something that was totally unusable..month later an update cleared the crashes but ruined the rewire connection with Logic..than another update cleared that...a few weeks ago...now there is Live 5..
I ve any right in the world to be skeptic..especially when minutes after the release the forum gets filled with crashreports... makes me currious..
I am however sure that Ableton dont likes to repeat the Live 4 desaster. Even when a big amount of users dont sees problems with Live4..but when you compare it with the competitors there was big problems..
They even stated that lot of Live 4 Bugs got eliminated during the Live 5 testing.. Now they use the mess to advertise Live 5 ..funny..

I dont expect Live 5 to work properly wright now but will update anyway to start doing the clipexports in the new format..Its weeks i can use the program without even trying the rewire connection...

[anti-banausic]

My, My......

I have never heard so many whiny people.

Oh, I forgot that the world revolves around you.

There are still probably many little, or for some, big bugs left in LIVE5. Just as there are still bugs in Cubase, Nuendo, etc.......but blaming the original testers, who took their time to try to find bugs, is rather a shit thing to do.

Everybody uses LIVE differently, and also, everybodies systems are slightly different. That means that a bug may be found by one individual and not another.

You know, I wanted to write so much more, but all the negativity from trolls on this board is making me realize that life is too short.

Cheers,
AB