Quality Guarantee

  • 2
  • Problem
  • Updated 6 years ago
  • Solved
Wondering if Yola can explain their QA program for development?

Generally a QA program is set in place to guaratee a high likelihood of a low incidence of errors by development procedures. It is generally defined in statistical terms with tolerances being set for the likelihood of errors. There are some errors which should never happen and these are bound by tolerances set much higher. For example an error in broadcasting sites should never happen. that should be picked up before release by the QC program.

Yola says that it has a QA program running. My feeling is that this should be reviewed urgently and real controls be put in to place. This is really poor and wouldn't even pass the primary inspection of any accreditation system I have come across. I'm not saying that they should be accredited but rather that there should be some guarantee of quality in their processes. Errors of a noticeable level are happening frequently and this really suggests that the QA program is ineffective.

Users rightly expect some reliability of quality in the hosts systems of management and even more so for paying customers. We implore you to give us confidence in your system and our livelihoods in some cases.
Photo of Gop

Gop

  • 7473 Posts
  • 2115 Reply Likes

Posted 6 years ago

  • 2
Photo of debbym

debbym

  • 125 Posts
  • 2 Reply Likes
Excellent point.
Photo of lisaretief

lisaretief

  • 114 Posts
  • 21 Reply Likes
Hi Gop,

I am the person in charge of all software and quality assurance (QA) engineers at Yola. I am also the person responsible for the smooth running of our software and systems once they are deployed as artifacts that you use daily and rely on.

I want to try to explain the difference between quality assurance (QA) and application monitoring. QA happens before new software is release. Monitoring tells us when something is going wrong in our production environment.

It is not the responsibility of QA to be able to foresee all situations that could occur after the software has been released. There are many things that can occur that we need to respond to daily which QA can not predict.

Some examples can be that a data center loses connectivity, or that a 3rd party partner we rely on is having issues. Then it becomes our job to minimise the impact of those unforeseen eventualities on you, our customers.

In most cases we do so successfully. In others, you feel the impact and we use forums such as these to let you know what is going on.

At all times, we prioritize you as the customer. We make sure your data is safe, and that your website remains available. If it is unavoidable, we make sure your website is unavailable for the least possible time. We have a track record of 99,99% uptime for your site.

If there are any specifics you would like to ask me about, please feel free to do so.

Lisa
Photo of Jacob

Jacob

  • 46 Posts
  • 0 Reply Likes
good to know, also note technogoy has it's problems that we cannot control.
jacob
Photo of Gop

Gop

  • 7473 Posts
  • 2115 Reply Likes
Hello Lisa,

Thank you for your response.

Thank you for differentiating between QA and Monitoring. I understand that. I don't understand though why you would think that monitoring isn't part of QA? Being able to detect an issue as early as possible is a Quality issue and I'm surprised that it's not part of a quality program. It must be better to have an early warning ability rather than frustrated customers? QA is not a predictor of issues I know but rather a measure of a parameter or parameters that are associated with a pattern of failure or non-compliance. QA is a package covering all aspects of production.

It's really not a question of where the problem occurs but rather is there a problem? I presume that you have some kind of monitoring at the four corners of the globe that alerts you to non-reception at such points? If that is alerted to then you would have to determine whether or not it was in the realm of Yola or completely outside. That would allow you the most efficient focus on a fix. It would be part of a comprehensive QA to alert your customers in those parts that their sites are not being broadcast to those areas.

You mentioned that the forum is used to alert customers of issues and as such is a very important part of your QA plan. This really is not working well, as it seems that most issues are being alerted to by users and this then is sent back to yourselves(?). Generally there is quite a delay and many notices being published before it's acknowledged by Yola. The customer base is a tertiary QC monitor but not a popular way to monitor this way. That's when the problems of dissatisfaction and discontent start. As soon as an announcement is made for some problem the mood of the forum changes noticeably. The doubts of the user are dispelled in part and there is acceptance by many that the issue is being worked on, without that it's a question mark and high frustration.

Using your definition of QA then I think there is a problem with this. There have been quite a few release issues where problems have been seen. One occurrence was the reversion back to older versions of the site and broadcast in this way. I'm surprised that a QA system didn't pick this overt error up. Issues with recommended browsers and problems across the board or nearly so such as FF and text alignment, inability to display images or only some images, or only some pages. They are all QA issues and very important to detect before they cause problems to the user base. Of course if it is acceptable to have a 10% error rate and your true rate is 9.9% then you have effective QC by definition. I would think that a 1 in five hundred would more likely be acceptable as a target.

Technology does bring it's own problems and may not be able to be fixed easily but I think that to monitor the outcome for quality is possible and with an alert is beneficial. I don't think that the 99.99% is the be all and end all because that uptime doesn't show the quality it only shows the broadcast, errors and all. I think Disraeli, Huxley, Twain and Shaw made comments on such kind of argument..
Photo of debbym

debbym

  • 125 Posts
  • 2 Reply Likes
That's quite a lecture,Lisa. Excellent response again, Gop.

I have experience of working with International Standards and QA in broadcast equipment and cannot disagree with Lisa when she states "It is not the responsibility of QA to be able to foresee all situations that could occur after the software has been released". However I would suggest for instance that QA should identify during software testing pre-release issues like that of the much vaunted coloured boxes which looked very smart in FF but were flat as a pancake in IE and difficulty in adding text. One of those things which caused real aggravation which could have been avoided if QA was on top of things.

Technology or third party suppliers aren't perfect - we all know that but when things go wrong which are out of your control, but I think what Gop and other Yola customers are asking is why is it like pulling teeth to get some sort of statement or even admission that something isn't working. The scripted answers come out about browsers and caches, read the tutorials, then "the wait for 48 hours whilst our engineers/DNS team look at it" etc. Quite often I've been left with a feeling that I was I was being stupid, or that I was going slightly mad even though I knew that from the evidence of my own eyes things were not working eg saving in sitebuilder and uploading files.

Surely someone is monitoring what is going on in the Forum? It doesn't take much wit to see a trend developing, like sitebuilder not saving changes, problems with file manager etc. That should be flagged up by the Monitor, a status update made and a watching brief and regular updates posted.

I bet if a word search was done through the Forum posts, the word(s) which would come up most often would be "Frustration" or "Frustrated". This could be so easily avoided. A Community Forum is a discussion group of interested people who come up with helpful suggestions and solutions. However it is not a subsitute for Customer Support, merely an adjunct to it.

Ahha I hear you says, there is help in the site builder. If my broadband or telephone service isn't working my first port of call would be the service status update. So,what's happened to the Yola status blog? I have yet to get it to load. It's a simple tool to update customers so why is it not in use. That should be visible on the Dashboard as I guess not every Yola client follows the Forum or are active members - if they did, there would be more than 21663 supporting it. Clearly if the Forum is being used to update people about what is going on with issues that are causing everyone problems then it patently isn't working.

It seems to me that if someone who is one of the most erudite members of the community and a staunch supporter of Yola is disgruntled, then something is going wrong.
Photo of lisaretief

lisaretief

  • 114 Posts
  • 21 Reply Likes
My statement about QA responsibility was in reference to deciding when software ready for release. And yes, that does include cross-browser testing.

Of course quality extends into everything we (Yola) do, from the day we start to define a product and throughout the lifespan of that product. That includes application monitoring and responding to issues that were not predictable.

Of course issues happen, and we appreciate that you as part of our customer base let us know if we have missed anything.

http://twitter.com/#!/yola is probably the best place to go to see broadcasts of serious issues. This has replaced the status blog.
Photo of debbym

debbym

  • 125 Posts
  • 2 Reply Likes
QA regarding software releases was also what I was referring to in some part and IMHO that you have fallen down on this recently. I was only drawing this to your attention as part of your (paying) customer base which I am within my rights to do. I also see very little evidence of monitoring.

As for the customer support v customer service issue I have just had to LOL.

When did Twitter replace the status blog? I must have missed that one!

a) I don't use Twitter (yes I know I'm bizarre but I am one of few who don't)

b) If the staus blog is no longer in use - why the ********************** is there a a link to it on the 'Help' page. UNBELIEVABLE.