From: Felipe Contreras < felipe contreras gmail com> To: desktop-devel-list < desktop-devel-list gnome org> Subject: GNOME user survey 2011 (v4) Date: Wed, 17 Aug 2011 18:23:05 +0300 Hi, Here's the fourth version of the survey, only tiny minor changes, it seems it's stabilized as there isn't many more comments. Shall we start planning the deployment? Who can get it into the site? Can I have access? How about an application that pops notifications similar to this one? Would such a thing be accepted? Cheers. === 01. Which of the following images best resemble your desktop? === (image selection) - GNOME 2 - GNOME 3 - Unity - KDE === 02. Overall, how happy are you with GNOME? === (single choice) * unhappy * not so happy * happy * very happy * completely ecstatic === 03. Where do you run GNOME? === (multiple choice, with other) + Desktop + Laptop + Netbook + Tablet === 04. Which GNOME version(s) are you using? === (multiple choice, with other) + 3.2 + 3.0 + 2.x + I don't know + I'm not using it currently + other, please specify === 05. How long have you been using GNOME? (years) === (numeric) === 06. How do you compare your current GNOME version with the version from one year ago? === (single choice) * better * no changes * worse * cannot say === 07. Does GNOME do what you want? === (single choice) * Everything * Mostly * Somewhat * Barely * Not at all === 08. How happy are you with GNOME in regards to == (matrix) Columns: unhappy / not so happy / happy / very happy / completely ecstatic + ease of use + documentation + language availability + accessibility + community === 09. Which other desktop environments have you used in recent years? == (multiple choice, with other) + KDE + Unity + XFCE + LXDE + Enlightenment + other (please specify) === 10. How many years of experience do you have using computers? === (numeric) === 11. How often do you use terminal/console? == (single choice) * What is that? * When I have no other option * I can't live without them * Is there anything else? === 12. Have you contributed to the GNOME project? === (single choice) * Yes * No === 13. Have you contacted the GNOME team? === (single choice) * Yes, successfully * Yes, unsuccessfully * No, I don't know how * No, never had the need === 14. If you could change three things in GNOME, what would they be? === (free form) === 15. Do you have any comments or suggestions for the GNOME team? === (free form) -- Felipe Contreras
From: Stormy Peters < stormy gnome org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Thu, 18 Aug 2011 11:53:47 -0600 Hi Felipe, Thanks for trying to get feedback from users. This is something that is really hard to do. It might also be worth contacting a company that does this professionally to see if they can help us. I do not think you will be able to do very much with the answers to the questions you ask below. It's going to be a lot of work for data that is not useful. Let me try to explain. On Wed, Aug 17, 2011 at 9:23 AM, Felipe Contreras < felipe contreras gmail com> wrote: > Hi, > Here's the fourth version of the survey, only tiny minor changes, it > seems it's stabilized as there isn't many more comments. > Shall we start planning the deployment? Who can get it into the site? > Can I have access? Where are we deploying it? How are we going to get people to take it? > How about an application that pops notifications similar to this one? > Would such a thing be accepted? Within GNOME? Would the distros agree to ship it? > === 01. Which of the following images best resemble your desktop? === > (image selection) > - GNOME 2 > - GNOME 3 > - Unity > - KDE I think if we want to get average users, most of them are not going to know what GNOME is. I love GNOME and I've been using GNOME for years and working with GNOME, and I still don't really know what all is GNOME on my desktop. I think the questions will have to be much more specific. > === 02. Overall, how happy are you with GNOME? === > (single choice) > * unhappy > * not so happy > * happy > * very happy > * completely ecstatic If people tell you they are happy or unhappy with GNOME, what are you going to do with that? If they are unhappy what are you going to fix? If they are happy, what did they like? The color, the menus, the windows, the apps, ...??? > === 03. Where do you run GNOME? === > (multiple choice, with other) > === 04. Which GNOME version(s) are you using? === > (multiple choice, with other) > === 05. How long have you been using GNOME? (years) === > (numeric) The previous 3 questions are only useful if it will somehow help you understand the other answers better. > === 06. How do you compare your current GNOME version with the version > from one year ago? === > (single choice) > * better > * no changes > * worse > * cannot say They aren't going to know what's GNOME versus what's the distro. And maybe they like the help better and the menus less. Or maybe it's missing their favorite feature. This is way too vague ... > === 07. Does GNOME do what you want? === > (single choice) > * Everything > * Mostly > * Somewhat > * Barely > * Not at all If they say "not at all", what are we missing? > === 08. How happy are you with GNOME in regards to == > (matrix) > Columns: unhappy / not so happy / happy / very happy / completely ecstatic > + ease of use > + documentation > + language availability > + accessibility > + community This gets closer but it still way too high level. > === 09. Which other desktop environments have you used in recent years? == > (multiple choice, with other) > + KDE > + Unity > + XFCE > + LXDE > + Enlightenment > + other (please specify) Are you planning on polling just the open source community? Nobody else is going to have any idea what these are. They would be more likely to know Windows and Mac. But I'm not sure that this question is useful for us to act on. > === 10. How many years of experience do you have using computers? === > (numeric) Does it matter? > === 11. How often do you use terminal/console? == > (single choice) > * What is that? > * When I have no other option > * I can't live without them > * Is there anything else? All the questions after this assume a knowledge of GNOME and how our community works. That's fine if you are polling developers. If you are polling average users, then I think it's not worth asking. Stormy > === 12. Have you contributed to the GNOME project? === > (single choice) > * Yes > * No > === 13. Have you contacted the GNOME team? === > (single choice) > * Yes, successfully > * Yes, unsuccessfully > * No, I don't know how > * No, never had the need > === 14. If you could change three things in GNOME, what would they be? === > (free form) > === 15. Do you have any comments or suggestions for the GNOME team? === > (free form) > -- > Felipe Contreras > _______________________________________________ > desktop-devel-list mailing list > desktop-devel-list gnome org > http://mail.gnome.org/mailman/listinfo/desktop-devel-list
From: Alan Cox < alan lxorguk ukuu org uk> To: Stormy Peters < stormy gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Thu, 18 Aug 2011 20:20:39 +0100 > I do not think you will be able to do very much with the answers to the > questions you ask below. It's going to be a lot of work for data that is not > useful. Let me try to explain. I thhink there is a better way to do this Felipe should do it without the Gnome oligarchy and then put the findings up on Linux Weekly News. That way we'll learn something, if not everything and it can no longer be stalled forever by bickering. And then well it's up to people if they listen, what they do with the data and how they follow it up. Sure the results will need reviewing with a little car - but thats true of any survey even one you paid through the nose for, in fact often more so because they more you pay the harder some of them will work to make sure you get the answers they think you want to hear 8) Plus the resulting debate may well answer even more questions than the survey ever did... Alan
From: Stormy Peters < stormy gnome org> To: Alan Cox < alan lxorguk ukuu org uk> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Thu, 18 Aug 2011 14:21:29 -0600 On Thu, Aug 18, 2011 at 1:20 PM, Alan Cox < alan lxorguk ukuu org uk> wrote: > And then well it's up to people if they listen, what they do with the > data and how they follow it up. Sure the results will need reviewing with > a little car The answers are so vague that you are not going to be able to follow up on them. So they are unhappy with GNOME. Then what? > Plus the resulting debate may well answer even more questions than the > survey ever did... If you care about debating and learning from our core group of dedicated supporters, yes. If you care about average users, well, I doubt you'll learn much from them this way. The questions and answers are just too vague and those people are likely not reading LWN so they won't be able to follow up with us that way. Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does know what GNOME is, knows how to contact the GNOME team and can tell you what version of GNOME they are using. And if they do, what is the survey going to tell you? That they do or don't like GNOME? And how long they have been using GNOME? What are we going to do with that information? Before any survey, you should know how you are going to use the information so that you can be sure to ask the right questions. Stormy
From: Alan Cox < alan lxorguk ukuu org uk> To: Stormy Peters < stormy gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Thu, 18 Aug 2011 22:37:33 +0100 > The answers are so vague that you are not going to be able to follow up on > them. So they are unhappy with GNOME. Then what? Then at the very least you've got some picture of what is going on and you can try and trigger discussion about why people are unhappy (or indeed happy). > Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does > know what GNOME is, knows how to contact the GNOME team and can tell you > what version of GNOME they are using. And if they do, what is the survey There are going to be a large number of users whose viewpoint is essentially "don't care", how you measure them is hard in pretty much any situation. A truely random sample of Gnome users will be hard to get by any approach. > going to tell you? That they do or don't like GNOME? And how long they have > been using GNOME? What are we going to do with that information? Use it to work out what questions it would be interesting to ask next year ? Look at what shows up in terms of additional comments. Look at the discussion around it, drop in the odd 'Why ?' question of your own. Use it to kickstart a secondary debate on the gnome site. (And btw while they won't read LWN people will link to it and discuss it in other places too) There is a second thing here too IMHO. The questions that could be asked and fixing them are currently buried in the debate. I can't see how progress will be made on picking questions usefully until someone moves from trying to achieve consensus to picking what they think is best based upon the resposes and just doing it regardless of whether each question is considered wrong by 5% of the people in the debate. > Before any survey, you should know how you are going to use the information > so that you can be sure to ask the right questions. So I could equally have said "Why release Gnome 3.0, we know it isn't perfect and there are wrong things". Releasing it was better than stasis, it provided a learning experience that will make 3.2 much better I am sure. Doing nothing achieves nothing, doing something achieves learning. You may well not learn what you intended but you will learn something including quite possibly how to do future surveys better. I'm not saying its necessarily a great approach but it's vastly superior to people sitting around picking holes in the idea until it never happens. Right now this seems to be in blocking mode, and blocking a volunteer off to do stuff and see what happens be it code or otherwise is usually the wrong thing to do. Sure -t here is a good case for not describing it in any way that suggests its GNOME foundation endorsed or driven. Gnome grew from a comically clueless 0.1 tarball, surveys can do likewise. Alan
From: Felipe Contreras < felipe contreras gmail com> To: Stormy Peters < stormy gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 00:21:21 +0300 Hi, On Thu, Aug 18, 2011 at 8:53 PM, Stormy Peters < stormy gnome org> wrote: > Thanks for trying to get feedback from users. This is something that is > really hard to do. It might also be worth contacting a company that does > this professionally to see if they can help us. > > I do not think you will be able to do very much with the answers to the > questions you ask below. It's going to be a lot of work for data that is not > useful. Let me try to explain. > > On Wed, Aug 17, 2011 at 9:23 AM, Felipe Contreras > < felipe contreras gmail com> wrote: >> Here's the fourth version of the survey, only tiny minor changes, it >> seems it's stabilized as there isn't many more comments. >> >> Shall we start planning the deployment? Who can get it into the site? >> Can I have access? > > Where are we deploying it? How are we going to get people to take it? Have you read the original thread? http://thread.gmane.org/gmane.comp.gnome.desktop/45432/focus=45456 I originally suggested questionpro.com, as it was the best option I could find. Later on Frederic Muller suggested to use limesurvey which supposedly it's installed in GNOME servers. As for getting people to take it, I was suggesting the usual info distribution channels; blogs, planets, online magazines, twitter, Google+, facebook, etc. In addition to that I suggested a new software component that would read information from the Internet, and pop up a notification to the user, the user would be able to disable these notifications easily, of course. Perhaps for GNOME 3.2. But I didn't hear a lot of encouragement for that idea. >> How about an application that pops notifications similar to this one? >> Would such a thing be accepted? > > Within GNOME? Would the distros agree to ship it? >> >> === 01. Which of the following images best resemble your desktop? === >> (image selection) >> >> - GNOME 2 >> - GNOME 3 >> - Unity >> - KDE > > I think if we want to get average users, most of them are not going to know > what GNOME is. That is the purpose of this question, they don't have to know, they just select the image that resembles what they are using. > I love GNOME and I've been using GNOME for years and working with GNOME, and > I still don't really know what all is GNOME on my desktop. > > I think the questions will have to be much more specific. How do you make this question more specific? They are images, you just have to select one. >> === 02. Overall, how happy are you with GNOME? === >> (single choice) >> >> * unhappy >> * not so happy >> * happy >> * very happy >> * completely ecstatic > > If people tell you they are happy or unhappy with GNOME, what are you going > to do with that? If they are unhappy what are you going to fix? If they are > happy, what did they like? The color, the menus, the windows, the apps, > ...??? Baby steps, first, let's get the results. If they are happy, great, not much to do, if they are unhappy, well, then some further actions after this survey might be needed. Having said that, we might find some clues in the rest of the survey, as this question is somewhat split into multiple groups later on, and in the worst case scenario, there's the free-form comments. >> === 03. Where do you run GNOME? === >> (multiple choice, with other) >> >> === 04. Which GNOME version(s) are you using? === >> (multiple choice, with other) >> >> === 05. How long have you been using GNOME? (years) === >> (numeric) > > The previous 3 questions are only useful if it will somehow help you > understand the other answers better. Also the previous one. Maybe people don't like GNOME 2 that much, but they do like GNOME 3... >> === 06. How do you compare your current GNOME version with the version >> from one year ago? === >> (single choice) >> >> * better >> * no changes >> * worse >> >> * cannot say > > They aren't going to know what's GNOME versus what's the distro. And maybe > they like the help better and the menus less. Or maybe it's missing their > favorite feature. This is way too vague ... The purpose is to get some sense of progress. Say, the respondents using GNOME 2 are answering "worse" a lot, it might be worth investigating what might have been those changes. How would you make this question less vague? >> === 07. Does GNOME do what you want? === >> (single choice) >> >> * Everything >> * Mostly >> * Somewhat >> * Barely >> * Not at all > > If they say "not at all", what are we missing? Again, then you might want to take further actions beyond this survey. Most likely though, you would be able to find some correlations between this question to other ones. Maybe it's only people with a lot of experience with computers that would answer in such way. We would know only after getting the results. >> === 08. How happy are you with GNOME in regards to == >> (matrix) >> >> Columns: unhappy / not so happy / happy / very happy / completely >> ecstatic >> + ease of use >> + documentation >> + language availability >> + accessibility >> + community > > This gets closer but it still way too high level. At least is something. If you have a better idea, why not share it? >> === 09. Which other desktop environments have you used in recent years? == >> (multiple choice, with other) >> >> + KDE >> + Unity >> + XFCE >> + LXDE >> + Enlightenment >> >> + other (please specify) > > Are you planning on polling just the open source community? Nobody else is > going to have any idea what these are. They would be more likely to know > Windows and Mac. But I'm not sure that this question is useful for us to act > on. Right, it's missing Windows and OS X (Mac). It might be useful to correlate the results; perhaps the people using OS X have a tendency to like GNOME 3, and Windows people a tendency to prefer GNOME 2. Also, if many people select these "open source" DE's, that would also give you an idea of what kind of people are answering the survey. >> === 10. How many years of experience do you have using computers? === >> (numeric) > > Does it matter? Yes, again, correlation; you might find that new-comers like GNOME 3, and old farts GNOME 2. >> === 11. How often do you use terminal/console? == >> (single choice) >> >> * What is that? >> * When I have no other option >> * I can't live without them >> * Is there anything else? > > All the questions after this assume a knowledge of GNOME and how our > community works. That's fine if you are polling developers. If you are > polling average users, then I think it's not worth asking. We are polling both, and it's important to know how many people are directly involved, so can figure out if there's a strong bias in those answers, and much it would affect the totals. Also users that try to give feedback are still users, not developers. Specially #14, anybody can chime in there. >> === 12. Have you contributed to the GNOME project? === >> (single choice) >> >> * Yes >> * No >> >> === 13. Have you contacted the GNOME team? === >> (single choice) >> >> * Yes, successfully >> * Yes, unsuccessfully >> * No, I don't know how >> * No, never had the need >> >> === 14. If you could change three things in GNOME, what would they be? === >> (free form) >> >> === 15. Do you have any comments or suggestions for the GNOME team? === >> (free form) -- Felipe Contreras
From: Felipe Contreras < felipe contreras gmail com> To: Stormy Peters < stormy gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org>, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 00:41:03 +0300 On Thu, Aug 18, 2011 at 11:21 PM, Stormy Peters < stormy gnome org> wrote: > On Thu, Aug 18, 2011 at 1:20 PM, Alan Cox < alan lxorguk ukuu org uk> wrote: >> >> And then well it's up to people if they listen, what they do with the >> data and how they follow it up. Sure the results will need reviewing with >> a little car > > The answers are so vague that you are not going to be able to follow up on > them. So they are unhappy with GNOME. Then what? This is a simple survey, not some kind of magical questionnaire. Just having the information that users are unhappy is valuable already. Plus, there's the free-form suggestion text. >> Plus the resulting debate may well answer even more questions than the >> survey ever did... >> > If you care about debating and learning from our core group of dedicated > supporters, yes. If you care about average users, well, I doubt you'll learn > much from them this way. The questions and answers are just too vague and > those people are likely not reading LWN so they won't be able to follow up > with us that way. Well, we don't know what the average GNOME user looks like, do we? What we do know is that is the target user, and we have tried to identify them with the question in the survey. Supposing we get a few thousands of respondents, and only 10% qualify as "normal users", even then, if you filter those answers, that should give you statistically significant information about the whole target user-base. Oh, and maybe a lot of people don't read LWN, but they don't have too, just like anything viral, the link to the survey would spread, and eventually at least few "normal users" are bound to have a geek friend that would show them the link. > Of course, maybe I'm wrong. Perhaps the average user of Linux/GNOME does > know what GNOME is, knows how to contact the GNOME team and can tell you > what version of GNOME they are using. And if they do, what is the survey > going to tell you? That they do or don't like GNOME? And how long they have > been using GNOME? What are we going to do with that information? Suppose you are right, and we do get that "useless" (not in my books) information, what is the damage? Suppose however that we do find something useful there. I think it's totally worth trying. > Before any survey, you should know how you are going to use the information > so that you can be sure to ask the right questions. Not necessarily. Again, asking a useless question doesn't hurt anybody. Of course, if you have better questions, those should be prioritized over the ones that have less chance of being fruitful. -- Felipe Contreras
From: Felipe Contreras < felipe contreras gmail com> To: Alan Cox < alan lxorguk ukuu org uk> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 00:45:25 +0300 On Fri, Aug 19, 2011 at 12:37 AM, Alan Cox < alan lxorguk ukuu org uk> wrote: > Use it to work out what questions it would be interesting to ask next > year ? Look at what shows up in terms of additional comments. Look at the > discussion around it, drop in the odd 'Why ?' question of your own. Use > it to kickstart a secondary debate on the gnome site. Exactly. I forgot this crucial point. This has been the case in the Git survey; after years of doing it, we have found that some questions were missing (by looking at the comments box), and that some questions were not really providing much. Nothing is ever perfect, but having at least some results is better than nothing. -- Felipe Contreras
From: "Zeeshan Ali (Khattak)" < zeeshanak gnome org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list < desktop-devel-list gnome org>, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 01:20:53 +0300 On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras < felipe contreras gmail com> wrote: > > Nothing is ever perfect, but having at least some results is better > than nothing. Since you have repeated this assertion a few times, I must ask: What if the results are all wrong and we don't have any way of knowing that? Would those results still be better than nothing in your opinion? -- Regards, Zeeshan Ali (Khattak) FSF member#5124
From: Alan Cox < alan lxorguk ukuu org uk> To: "Zeeshan Ali (Khattak)" < zeeshanak gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 00:02:02 +0100 On Fri, 19 Aug 2011 01:20:53 +0300 "Zeeshan Ali (Khattak)" < zeeshanak gnome org> wrote: > On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras > < felipe contreras gmail com> wrote: > > > > Nothing is ever perfect, but having at least some results is better > > than nothing. > > Since you have repeated this assertion a few times, I must ask: What > if the results are all wrong and we don't have any way of knowing > that? Would those results still be better than nothing in your > opinion? In that hypothetical case possibly not. But that isn't really likely to be the case even with a bad survey, especially if you start looking at how people used the open comments and asking why questions, or looking at the debate it triggers. You can learn things even by asserting a position and seeing the responses you get. Alan
From: Felipe Contreras < felipe contreras gmail com> To: "Zeeshan Ali (Khattak)" < zeeshanak gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org>, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 02:09:46 +0300 On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) < zeeshanak gnome org> wrote: > On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras > < felipe contreras gmail com> wrote: >> >> Nothing is ever perfect, but having at least some results is better >> than nothing. > > Since you have repeated this assertion a few times, I must ask: What > if the results are all wrong and we don't have any way of knowing > that? Would those results still be better than nothing in your > opinion? What do you mean by all wrong? Let's assume that the results show that 1000 people are not happy with GNOME. How can that be wrong? 1000 people responded that, the results were not somehow altered, or boycotted, the results are the results, and that's that. The *conclusions* based on the analysis of the results might be wrong, but that wouldn't be a problem of the survey, and if you are so afraid of that, you can ignore the results of the survey completely. I for one think the survey already has enough mechanisms to determine biases, and therefore come up with conclusions with a reasonable degree of certainty. But I wonder, can you come up with some example of "bad results" to the answers proposed here, and why exactly we wouldn't know they are "bad"? -- Felipe Contreras
From: "Zeeshan Ali (Khattak)" < zeeshanak gnome org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list < desktop-devel-list gnome org>, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 03:34:58 +0300 On Fri, Aug 19, 2011 at 2:09 AM, Felipe Contreras < felipe contreras gmail com> wrote: > On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) > < zeeshanak gnome org> wrote: >> On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras >> < felipe contreras gmail com> wrote: >>> >>> Nothing is ever perfect, but having at least some results is better >>> than nothing. >> >> Since you have repeated this assertion a few times, I must ask: What >> if the results are all wrong and we don't have any way of knowing >> that? Would those results still be better than nothing in your >> opinion? > > What do you mean by all wrong? Let's assume that the results show that > 1000 people are not happy with GNOME. How can that be wrong? Maybe they all lied? Maybe people who are satisfied do not want to or have time to take part in surveys and you only get people who are not happy into the survey? In which case, the results may show results that are not correct. i-e a significantly large number of participant say that they are very unhappy with GNOME but what if that number is nothing compared to the number of people who are very much satisfied with GNOME? I didn't say this so far because it might sound like I am trying to make a joke but since you still insist on your assertions about the survey, I feel I must say this: How do you know people in general like to participate in surveys? It is my observation that most people do not like to do that, unless they have something to complain about. Now this observation of mine could very well be wrong but how do we know that? Do we do a survey to find out if people like to participate in surveys? -- Regards, Zeeshan Ali (Khattak) FSF member#5124
From: Felipe Contreras < felipe contreras gmail com> To: "Zeeshan Ali (Khattak)" < zeeshanak gnome org> Cc: desktop-devel-list < desktop-devel-list gnome org>, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 14:17:02 +0300 On Fri, Aug 19, 2011 at 3:34 AM, Zeeshan Ali (Khattak) < zeeshanak gnome org> wrote: > On Fri, Aug 19, 2011 at 2:09 AM, Felipe Contreras > < felipe contreras gmail com> wrote: >> On Fri, Aug 19, 2011 at 1:20 AM, Zeeshan Ali (Khattak) >> < zeeshanak gnome org> wrote: >>> On Fri, Aug 19, 2011 at 12:45 AM, Felipe Contreras >>> < felipe contreras gmail com> wrote: >>>> >>>> Nothing is ever perfect, but having at least some results is better >>>> than nothing. >>> >>> Since you have repeated this assertion a few times, I must ask: What >>> if the results are all wrong and we don't have any way of knowing >>> that? Would those results still be better than nothing in your >>> opinion? >> >> What do you mean by all wrong? Let's assume that the results show that >> 1000 people are not happy with GNOME. How can that be wrong? > > Maybe they all lied? Maybe people who are satisfied do not want to > or have time to take part in surveys and you only get people who are > not happy into the survey? In which case, the results may show results > that are not correct. i-e a significantly large number of participant > say that they are very unhappy with GNOME but what if that number is > nothing compared to the number of people who are very much satisfied > with GNOME? > > I didn't say this so far because it might sound like I am trying to > make a joke but since you still insist on your assertions about the > survey, I feel I must say this: How do you know people in general like > to participate in surveys? It is my observation that most people do > not like to do that, unless they have something to complain about. Now > this observation of mine could very well be wrong but how do we know > that? Do we do a survey to find out if people like to participate in > surveys? Are you serious? That totally and completely speculative and unrealistic. Have you ever participated in making a survey? I have, as I have explained, for the Git survey. In my experience, only the people that want to help in some way do spend the amount of time required to fill the survey. But again, as I said, if there's no survey on Earth you could trust, just ignore the results. Results by themselves cannot hurt you. -- Felipe Contreras
From: Patryk Zawadzki < patrys pld-linux org> To: Felipe Contreras < felipe contreras gmail com> Cc: Alan Cox < alan lxorguk ukuu org uk>, desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 13:25:36 +0200 On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras < felipe contreras gmail com> wrote: > Are you serious? That totally and completely speculative and > unrealistic. Have you ever participated in making a survey? I have, as > I have explained, for the Git survey. In my experience, only the > people that want to help in some way do spend the amount of time > required to fill the survey. Could you at least make the answer options less emotional? Like exchange "happy" for "satisfied" etc. I don't remember answering "ecstatic" in the Git survey but that could be my bad memory. -- Patryk Zawadzki I solve problems.
From: Felipe Contreras < felipe contreras gmail com> To: Patryk Zawadzki < patrys pld-linux org> Cc: Alan Cox < alan lxorguk ukuu org uk>, desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 14:33:18 +0300 On Fri, Aug 19, 2011 at 2:25 PM, Patryk Zawadzki < patrys pld-linux org> wrote: > On Fri, Aug 19, 2011 at 1:17 PM, Felipe Contreras > < felipe contreras gmail com> wrote: >> Are you serious? That totally and completely speculative and >> unrealistic. Have you ever participated in making a survey? I have, as >> I have explained, for the Git survey. In my experience, only the >> people that want to help in some way do spend the amount of time >> required to fill the survey. > > Could you at least make the answer options less emotional? Like > exchange "happy" for "satisfied" etc. I don't remember answering > "ecstatic" in the Git survey but that could be my bad memory. That's a reasonable alternative. How about "pleased"? Any other people have an opinion? -- Felipe Contreras
From: Andy Wingo < wingo pobox com> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 15:08:02 +0200 On Fri 19 Aug 2011 13:33, Felipe Contreras < felipe contreras gmail com> writes: > That's a reasonable alternative. How about "pleased"? Any other people > have an opinion? You present yourself as reasonable by adjusting on the small points, but you ignore the feedback of greater importance. My opinion is that you are not the right person to lead an effort to gather feedback on GNOME. The Git survey, AFAIU, was done _with_ the git developers. This one, if you manage to bully it through, will be _in spite of_ the GNOME developers. It will not have the effect you desire. Andy -- http://wingolog.org/
From: Felipe Contreras < felipe contreras gmail com> To: Andy Wingo < wingo pobox com> Cc: desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 16:13:15 +0300 On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo < wingo pobox com> wrote: > On Fri 19 Aug 2011 13:33, Felipe Contreras < felipe contreras gmail com> writes: > >> That's a reasonable alternative. How about "pleased"? Any other people >> have an opinion? > > You present yourself as reasonable by adjusting on the small points, > but you ignore the feedback of greater importance. > > My opinion is that you are not the right person to lead an effort to > gather feedback on GNOME. Is there anyone in the universe able to create a user survey worthy of GNOME? Can you convince him of doing so? In the meantime, this is the best that we have. I will continue listening for constructive feedback, but comments such as "this is not good", "you are doing it wrong", "it's impossible", lead to nowhere. Besides, as Alan Cox said, it doesn't have to be perfect, like software, we can learn from the mistakes of the 2011 survey, and make a better one for 2012. Can we not? -- Felipe Contreras
From: Sam Thursfield < ssssam gmail com> To: Felipe Contreras < felipe contreras gmail com>, desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 14:55:25 +0100 On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras < felipe contreras gmail com> wrote: > On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo < wingo pobox com> wrote: >> On Fri 19 Aug 2011 13:33, Felipe Contreras < felipe contreras gmail com> writes: >> >>> That's a reasonable alternative. How about "pleased"? Any other people >>> have an opinion? >> >> You present yourself as reasonable by adjusting on the small points, >> but you ignore the feedback of greater importance. >> >> My opinion is that you are not the right person to lead an effort to >> gather feedback on GNOME. > > Is there anyone in the universe able to create a user survey worthy of > GNOME? Can you convince him of doing so? Gathering feedback does not necessarily require an online user survey. As stated, for a project which currently targets, among others, users who do not care what parts of their operating system can be labelled "GNOME" a survey is not a very reliable way of gathering feedback. Have you ever tried to explain, to a person who doesn't have an interest in software, what GNOME actually is? > In the meantime, this is the best that we have. I will continue > listening for constructive feedback, but comments such as "this is not > good", "you are doing it wrong", "it's impossible", lead to nowhere. > > Besides, as Alan Cox said, it doesn't have to be perfect, like > software, we can learn from the mistakes of the 2011 survey, and make > a better one for 2012. Can we not? I urge you to consider the fact that if the majority of people subscribed to desktop-devel-list don't have faith in idea of an online user survey, an online user survey is probably not going to much have effect on the views of the people who contribute to the discussions on desktop-devel-list, and since most of the GNOME community read desktop-devel-list you can probably extend this to all of the other GNOME mailing lists and IRC channels as well. Sam
From: Alan Cox < alan lxorguk ukuu org uk> To: Sam Thursfield < ssssam gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 16:12:19 +0100 > Gathering feedback does not necessarily require an online user survey. > As stated, for a project which currently targets, among others, users > who do not care what parts of their operating system can be labelled > "GNOME" a survey is not a very reliable way of gathering feedback. > > Have you ever tried to explain, to a person who doesn't have an > interest in software, what GNOME actually is? Yes - my MBA research was into Linux desktops some years ago and did involve looking at end users attitudes. The quick summary from then would be: Most users used the desktop they got by default (whether because they didn't know to to switch or were never annoyed enough to bother I didn't have time to find out) The managers wanted a system that was a free exact clone of windows look/feel because change was expensive (training, lost time etc) The technies in the organisation often inflicted their personal desktop preference on the entire company. If I wanted to look at the "Gnome 3 is crap" assertion I think I would tackle it a bit differently as so much online updating is going on nowdays. Collect statistics from a few Fedora and other mirror sites, correlate downloads together by IP/time and other evidence, and look at how many of them download which desktops or combination of desktops. Repeat this over time and plot graphs. Distro popularity shifts may also provide some evidence for this. The trouble is while that will tell you about movement and popularity it will not tell you why. So it's a way to evaluate the claim "Gnome 3 is crap loads of people are changing or holding back on updating their desktop" but it's not going to answer useful things. There is a bit of value in knowing if lots of people hate or love Gnome 3, but the real value is knowing how it could be better for users, and counting downloads won't do that. And if real non-technical end users are like the ones I dealt with then asking them probably won't help either. Particularly in the business world to many of them at the time Gnone was 'click on this splodge in the morning to write letters' 'click on that thing in the corner to turn it off'. They are not decision makers either - impress their boss 8) The more interested and technically motivated people on the other hand can tell you stuff, "power users" particularly. They tell you stuff that reflects a particular use and understanding case though. Similarly you can learn an enormous amount by seeing what people are struggling with and what they do to the desktop - eg the various 'how to fix Gnome 3' pages tell you a lot about what people wanted and which is non-obvious for configuration. They are also from people who liked it enough to persevere so made an effort. > I urge you to consider the fact that if the majority of people > subscribed to desktop-devel-list don't have faith in idea of an online > user survey, an online user survey is probably not going to much have > effect on the views of the people who contribute to the discussions on > desktop-devel-list, and since most of the GNOME community read > desktop-devel-list you can probably extend this to all of the other > GNOME mailing lists and IRC channels as well. Some days I think Miguel got the Ximian monkey dead right, except that there should have been three of them. Alan
From: Felipe Contreras < felipe contreras gmail com> To: Sam Thursfield < ssssam gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 19:40:49 +0300 On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield < ssssam gmail com> wrote: > On Fri, Aug 19, 2011 at 2:13 PM, Felipe Contreras > < felipe contreras gmail com> wrote: >> On Fri, Aug 19, 2011 at 4:08 PM, Andy Wingo < wingo pobox com> wrote: >>> On Fri 19 Aug 2011 13:33, Felipe Contreras < felipe contreras gmail com> writes: >>> >>>> That's a reasonable alternative. How about "pleased"? Any other people >>>> have an opinion? >>> >>> You present yourself as reasonable by adjusting on the small points, >>> but you ignore the feedback of greater importance. >>> >>> My opinion is that you are not the right person to lead an effort to >>> gather feedback on GNOME. >> >> Is there anyone in the universe able to create a user survey worthy of >> GNOME? Can you convince him of doing so? > > Gathering feedback does not necessarily require an online user survey. Indeed, do you have a better suggestion? > As stated, for a project which currently targets, among others, users > who do not care what parts of their operating system can be labelled > "GNOME" a survey is not a very reliable way of gathering feedback. > > Have you ever tried to explain, to a person who doesn't have an > interest in software, what GNOME actually is? Again, do you have a suggestion to get feedback in a more useful way? >> In the meantime, this is the best that we have. I will continue >> listening for constructive feedback, but comments such as "this is not >> good", "you are doing it wrong", "it's impossible", lead to nowhere. >> >> Besides, as Alan Cox said, it doesn't have to be perfect, like >> software, we can learn from the mistakes of the 2011 survey, and make >> a better one for 2012. Can we not? > > I urge you to consider the fact that if the majority of people > subscribed to desktop-devel-list don't have faith in idea of an online > user survey, an online user survey is probably not going to much have > effect on the views of the people who contribute to the discussions on > desktop-devel-list, and since most of the GNOME community read > desktop-devel-list you can probably extend this to all of the other > GNOME mailing lists and IRC channels as well. So the status quo, where there are absolutely no numbers whatsoever is preferred. Any attempt to gather quantifiable feedback is discouraged. IOW; the GNOME community does not care about what users have to say at all. -- Felipe Contreras
From: Stormy Peters < stormy gnome org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 11:25:58 -0600 On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras < felipe contreras gmail com> wrote: > On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield < ssssam gmail com> wrote: >> >> Gathering feedback does not necessarily require an online user survey. > Indeed, do you have a better suggestion? There are several other ways to get feedback. For example, user testing. I'm sure all the major distributions have done some user testing. Most large companies have a whole user testing team/group. I'm not a user testing expert but it involves giving people (both new and experienced) tasks to do, watching how they do it (without interfering) and then asking them about their experience. I know people who have successfully used http://www.usertesting.com/ for web sites. I don't know if a similar, inexpensive option exists for desktop software or not. Stormy
From: Felipe Contreras < felipe contreras gmail com> To: Stormy Peters < stormy gnome org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 20:30:52 +0300 On Fri, Aug 19, 2011 at 8:25 PM, Stormy Peters < stormy gnome org> wrote: > On Fri, Aug 19, 2011 at 10:40 AM, Felipe Contreras > < felipe contreras gmail com> wrote: >> >> On Fri, Aug 19, 2011 at 4:55 PM, Sam Thursfield < ssssam gmail com> wrote: >> > >> > Gathering feedback does not necessarily require an online user survey. >> >> Indeed, do you have a better suggestion? > > There are several other ways to get feedback. > > For example, user testing. I'm sure all the major distributions have done > some user testing. Most large companies have a whole user testing > team/group. And where are the results? Without evidence it's only wishful thinking. > I'm not a user testing expert but it involves giving people (both new and > experienced) tasks to do, watching how they do it (without interfering) and > then asking them about their experience. > > I know people who have successfully used http://www.usertesting.com/ for web > sites. I don't know if a similar, inexpensive option exists for desktop > software or not. Right, so nobody is going to do this. Is there any better suggestion that would actually be implemented? -- Felipe Contreras
From: Richard Hughes < hughsient gmail com> To: Felipe Contreras < felipe contreras gmail com> Cc: Andy Wingo < wingo pobox com>, desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 14:56:54 +0100 On 19 August 2011 14:13, Felipe Contreras < felipe contreras gmail com> wrote: > Is there anyone in the universe able to create a user survey worthy of > GNOME? Can you convince him of doing so? Do your survey with the questions you want, and come to your own conclusions. Blog about them if you want. You could even convince a distribution to include a popup with a link, although I think that's insane. Just don't tell people that it's from the GNOME project, in any way authorized or blessed by the ruling cabal[1] or developers. I'm pretty sure the majority of the people actually working on GNOME 3.2 don't want a survey at all. Sorry to be blunt. Richard. [1] http://ftp.gnome.org/conspiracy/
From: Felipe Contreras < felipe contreras gmail com> To: Richard Hughes < hughsient gmail com> Cc: Andy Wingo < wingo pobox com>, desktop-devel-list < desktop-devel-list gnome org> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 19:42:37 +0300 On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes < hughsient gmail com> wrote: > On 19 August 2011 14:13, Felipe Contreras < felipe contreras gmail com> wrote: >> Is there anyone in the universe able to create a user survey worthy of >> GNOME? Can you convince him of doing so? > > Do your survey with the questions you want, and come to your own > conclusions. Blog about them if you want. You could even convince a > distribution to include a popup with a link, although I think that's > insane. > > Just don't tell people that it's from the GNOME project, in any way > authorized or blessed by the ruling cabal[1] or developers. I'm pretty > sure the majority of the people actually working on GNOME 3.2 don't > want a survey at all. > > Sorry to be blunt. No, thanks for the direct feedback. So basically you are saying there's no way any survey of any quality would be blessed by the GNOME community. That certainly clarifies things. -- Felipe Contreras
From: Jonathon Jongsma < jonathon quotidian org> To: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 12:15:27 -0500 On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes < hughsient gmail com> wrote: > > On 19 August 2011 14:13, Felipe Contreras < felipe contreras gmail com> wrote: > >> Is there anyone in the universe able to create a user survey worthy of > >> GNOME? Can you convince him of doing so? > > > > Do your survey with the questions you want, and come to your own > > conclusions. Blog about them if you want. You could even convince a > > distribution to include a popup with a link, although I think that's > > insane. > > > > Just don't tell people that it's from the GNOME project, in any way > > authorized or blessed by the ruling cabal[1] or developers. I'm pretty > > sure the majority of the people actually working on GNOME 3.2 don't > > want a survey at all. > > > > Sorry to be blunt. > > No, thanks for the direct feedback. So basically you are saying > there's no way any survey of any quality would be blessed by the GNOME > community. That certainly clarifies things. > It seems obvious from most responses here that there are not very many people within the GNOME community that think that this sort of a survey would be beneficial, and worry that it may even be counter-productive. In response to this realization, you have apparently shifted into outrage mode. You pretend that it is impossible to simultaneously care about what users while also opposing a user survey that has no hope of being a representative sample of users. It is possible for well-meaning people to come to different conclusions on the best methods for achieving a certain goal. It seems that most people here don't agree with your methods. Please accept the fact that this does not mean that they hate users, despite your attempts to conflate the two things. You are free to proceed with your survey on your own. Others are free to not wish to join you. It's that simple. Can you please stop the faux outrage? jonner
From: Felipe Contreras < felipe contreras gmail com> To: Jonathon Jongsma < jonathon quotidian org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 20:42:53 +0300 On Fri, Aug 19, 2011 at 8:15 PM, Jonathon Jongsma < jonathon quotidian org> wrote: > On Fri, 2011-08-19 at 19:42 +0300, Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 4:56 PM, Richard Hughes < hughsient gmail com> wrote: >> > On 19 August 2011 14:13, Felipe Contreras < felipe contreras gmail com> wrote: >> >> Is there anyone in the universe able to create a user survey worthy of >> >> GNOME? Can you convince him of doing so? >> > >> > Do your survey with the questions you want, and come to your own >> > conclusions. Blog about them if you want. You could even convince a >> > distribution to include a popup with a link, although I think that's >> > insane. >> > >> > Just don't tell people that it's from the GNOME project, in any way >> > authorized or blessed by the ruling cabal[1] or developers. I'm pretty >> > sure the majority of the people actually working on GNOME 3.2 don't >> > want a survey at all. >> > >> > Sorry to be blunt. >> >> No, thanks for the direct feedback. So basically you are saying >> there's no way any survey of any quality would be blessed by the GNOME >> community. That certainly clarifies things. > > It seems obvious from most responses here that there are not very many > people within the GNOME community that think that this sort of a survey > would be beneficial, and worry that it may even be counter-productive. > In response to this realization, you have apparently shifted into > outrage mode. You pretend that it is impossible to simultaneously care > about what users while also opposing a user survey that has no hope of > being a representative sample of users. You might say you do, and you might even believe so, but if your actions demonstrate otherwise, perhaps you do not. If the GNOME community really cared about what users have to say, and this survey indeed does not have any hope of having a representative sample of users (I disagree), then wouldn't they take the reins and do it properly? > It is possible for well-meaning people to come to different conclusions > on the best methods for achieving a certain goal. Yes, whenever I have a disagreement on a method to develop some software, I just go ahead and do it that way, and then say; see? this is how it should be done. Saying "you are wrong" is easy, anybody can do that. > It seems that most > people here don't agree with your methods. Please accept the fact that > this does not mean that they hate users, despite your attempts to > conflate the two things. I would, if they went ahead with the "right methods" and got some user feedback, if not in the form of a survey, in any method. > You are free to proceed with your survey on your own. Others are free > to not wish to join you. It's that simple. Can you please stop the > faux outrage? Sure, I just wanted to make things clear. In fact, if they cared about user feedback, there would be some numbers available somewhere, and I wouldn't have to do this. -- Felipe Contreras
From: Richard Hughes < hughsient gmail com> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 18:50:57 +0100 On 19 August 2011 18:42, Felipe Contreras < felipe contreras gmail com> wrote: > Sure, I just wanted to make things clear. In fact, if they cared about > user feedback, there would be some numbers available somewhere, and I > wouldn't have to do this. We're not asking you to do anything. Please just run the poll on your personal blog and stop getting aggressive with developers on this mailing list. Thanks, Richard.
From: Felipe Contreras < felipe contreras gmail com> To: Richard Hughes < hughsient gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 21:12:44 +0300 On Fri, Aug 19, 2011 at 8:50 PM, Richard Hughes < hughsient gmail com> wrote: > On 19 August 2011 18:42, Felipe Contreras < felipe contreras gmail com> wrote: >> Sure, I just wanted to make things clear. In fact, if they cared about >> user feedback, there would be some numbers available somewhere, and I >> wouldn't have to do this. > > We're not asking you to do anything. I am not suggesting you are. > Please just run the poll on your > personal blog and stop getting aggressive with developers on this > mailing list. I am not being aggressive. All I am asking is for clarification; is there *anything* I could do to make the survey more acceptable to you guys, or are you opposed to the very idea of having a user survey blessed by GNOME? -- Felipe Contreras
From: Stormy Peters < stormy gnome org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 12:34:34 -0600 On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras < felipe contreras gmail com> wrote: > I am not being aggressive. All I am asking is for clarification; is > there *anything* I could do to make the survey more acceptable to you > guys, or are you opposed to the very idea of having a user survey > blessed by GNOME? Your answers sound aggressive to me but I think that's totally understandable given all the negative feedback. I gave my feedback. I'd want the survey to be much more detailed. "What do you think about this menu option on Cheese" seems like it would give more feedback than "do you like GNOME?" But I do not have time to help come up with the questions, so I agree with many folks that say you'll have to take the feedback you've gotten and move forward. Giving feedback does not mean providing alternatives or working on the project. It's easy to give feedback. It's much harder and more time consuming to incorporate that feedback. You asked for feedback, you got some. If you want those people's approval, then you'll probably have to incorporate that feedback. If you aren't planning on incorporating it, then it's probably best to stop insisting that people need to provide alternatives if they give negative feedback. Obviously, you don't need everyone's approval to move forward. Rarely does any project get 100% approval. How you move forward, how much feedback you want and how you use that feedback is up to you. Good luck! Stormy
From: Felipe Contreras < felipe contreras gmail com> To: Stormy Peters < stormy gnome org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 23:08:51 +0300 On Fri, Aug 19, 2011 at 9:34 PM, Stormy Peters < stormy gnome org> wrote: > On Fri, Aug 19, 2011 at 12:12 PM, Felipe Contreras > < felipe contreras gmail com> wrote: >> >> I am not being aggressive. All I am asking is for clarification; is >> there *anything* I could do to make the survey more acceptable to you >> guys, or are you opposed to the very idea of having a user survey >> blessed by GNOME? > > Your answers sound aggressive to me but I think that's totally > understandable given all the negative feedback. Perhaps you are not used to straight-forward communication. I'm not trying to aggravate anyone. > I gave my feedback. I'd want the survey to be much more detailed. "What do > you think about this menu option on Cheese" seems like it would give more > feedback than "do you like GNOME?" But I do not have time to help come up > with the questions, so I agree with many folks that say you'll have to take > the feedback you've gotten and move forward. Trying to do that would create a huge survey that most likely most people will not even try to answer. If somebody really detests certain menu option on Cheese, I'm sure they'll let you know in the comments box. Who knows, maybe it turns out the part that most people are not satisfied with is the documentation, those kinds of results might trigger some interesting debate. Or maybe you are right, and we wouldn't not get anything useful, but at least we would have some ideas for the next survey. > Giving feedback does not mean providing alternatives or working on the > project. It's easy to give feedback. It's much harder and more time > consuming to incorporate that feedback. You asked for feedback, you got > some. If you want those people's approval, then you'll probably have to > incorporate that feedback. I have incorporated all the feedback that can be incorporated. The rest is too vague, or not actionable. What do *you* think must absolutely be changed in the survey? > If you aren't planning on incorporating it, then > it's probably best to stop insisting that people need to provide > alternatives if they give negative feedback. Huh? That's a very broad statement. Let's be clear, I have not turned away any feedback. Let's analyze for example the claims by Allan Day: --- > When you do survey research, you have to be certain that one person > understands the questions in the same way that another person does. Generally yes. Is that achievable in all the questions in this survey? Probably not (would love to hear some suggestions otherwise). Which why some other questions are asked to determine the people that might be thinking in other terms. (I already explained that) > You've also got the representativeness problem. Your sample will inevitably > be unrepresentative, probably highly so. Says who? What if we get 10 million answers? That would be such a big chunk of the total population that this problem is not a big deal. Or what if there's a notification app embedded in GNOME 3.2. That would not only maximize the reponders, but also maximize the randomness. Wouldn't it? (I already explained that) > your survey results will be misleading That's very useful. Now, how about some ideas to make the results less misleading? --- What exactly do you want me to do with that feedback? (aside from what I have already done) I am all ears. > Obviously, you don't need everyone's approval to move forward. Rarely does > any project get 100% approval. > > How you move forward, how much feedback you want and how you use that > feedback is up to you. I want the approval of the GNOME community, and I am willing to accept all suggestions for improvement in order to get that. So, what should I do? -- Felipe Contreras
From: Matthew Garrett < mjg59 srcf ucam org> To: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 17:24:18 +0100 On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: > Doing nothing achieves nothing, doing something achieves learning. You > may well not learn what you intended but you will learn something > including quite possibly how to do future surveys better. Any survey that isn't a carefully controlled randomly selected sample of users doesn't result in learning. It results in data that forms some sort of rorschach blot. Everyone will see what they want to see. Those who believe that Gnome 3 is a step back will point out that the majority of responses are negative. Those who believe it's a step forward will point out that happy users are going to be far less inclined to respond. There's no way whatsoever to determine how representative the responses are, and so there's no way whatsoever to learn anything about the population. All we'd learn is that some users like Gnome 3 and some users don't, and that's something we *already know*. So we'd gain nothing, but we'd guarantee another huge set of arguments which would themselves also tell us nothing. > I'm not saying its necessarily a great approach but it's vastly superior > to people sitting around picking holes in the idea until it never happens. I disagree. Doing something that sucks more time and energy away from development without actually telling us anything in return is worse than that not happening. Felipe is obviously free to do whatever he wants, but there's no benefit in Gnome itself participating in any way. If we want to find out what our users think then the only way to do that is to have professional involvement and a random sample set. -- Matthew Garrett | mjg59 srcf ucam org
From: Felipe Contreras < felipe contreras gmail com> To: Matthew Garrett < mjg59 srcf ucam org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 20:14:25 +0300 On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett < mjg59 srcf ucam org> wrote: > On Thu, Aug 18, 2011 at 10:37:33PM +0100, Alan Cox wrote: > >> Doing nothing achieves nothing, doing something achieves learning. You >> may well not learn what you intended but you will learn something >> including quite possibly how to do future surveys better. > > Any survey that isn't a carefully controlled randomly selected sample of > users doesn't result in learning. Unless the biases are identified, which we are trying to do. Moreover, I have tried to push the idea to have an automatic notification, which would maximize the number of responders, and thus increase the randomization. But apparent this idea is not welcome. So, ideas to improve the randomization are dismissed, and then you say without randomization, the survey is not useful. IOW; you are intentionally deadlocking the proposal. > It results in data that forms some > sort of rorschach blot. It might if you look at it as a whole, but you can try to dissect it. > Everyone will see what they want to see. Those > who believe that Gnome 3 is a step back will point out that the majority > of responses are negative. Those who believe it's a step forward will > point out that happy users are going to be far less inclined to respond. > > There's no way whatsoever to determine how representative the responses > are, and so there's no way whatsoever to learn anything about the > population. All we'd learn is that some users like Gnome 3 and some > users don't, and that's something we *already know*. So we'd gain > nothing, but we'd guarantee another huge set of arguments which would > themselves also tell us nothing. That's an assumption. What if we get 10 million responses? Would you still claim that the results are not representative? I think only *after* getting the results you would be able to say anything about it's representativeness. Something more realistic, say you get at least 300 responses that don't have any "geek" bias, that would be more than enough to make some statistically significant conclusions. >> I'm not saying its necessarily a great approach but it's vastly superior >> to people sitting around picking holes in the idea until it never happens. > > I disagree. Doing something that sucks more time and energy away from > development without actually telling us anything in return is worse than > that not happening. Felipe is obviously free to do whatever he wants, > but there's no benefit in Gnome itself participating in any way. If we > want to find out what our users think then the only way to do that is to > have professional involvement and a random sample set. This is not sucking any time and energy from anybody, I just need access to the server that has limesurvey installed, or somebody else can do that (can't take that much time), I would contact all the relevant news sites and make the relevant posts in social media. All that that is needed from GNOME people is a blessing. -- Felipe Contreras
From: Matthew Garrett < mjg59 srcf ucam org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 19:20:46 +0100 On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: > On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett < mjg59 srcf ucam org> wrote: > > Any survey that isn't a carefully controlled randomly selected sample of > > users doesn't result in learning. > > Unless the biases are identified, which we are trying to do. You can only identify the biases if you already know the population, and you can only know the population if you've got a random sample set to begin with. > Moreover, I have tried to push the idea to have an automatic > notification, which would maximize the number of responders, and thus > increase the randomization. But apparent this idea is not welcome. It doesn't help. The people most likely to respond to an irritating popup that disrupts their work are people who already feel that gnome 3 is an irritating piece of software that disrupts their work. You can't get a random sample in-band. > So, ideas to improve the randomization are dismissed, and then you say > without randomization, the survey is not useful. IOW; you are > intentionally deadlocking the proposal. I am saying that your results aren't useful unless your sample is random. I don't know of a good way to obtain a representative sample. > > Everyone will see what they want to see. Those > > who believe that Gnome 3 is a step back will point out that the majority > > of responses are negative. Those who believe it's a step forward will > > point out that happy users are going to be far less inclined to respond. > > > > There's no way whatsoever to determine how representative the responses > > are, and so there's no way whatsoever to learn anything about the > > population. All we'd learn is that some users like Gnome 3 and some > > users don't, and that's something we *already know*. So we'd gain > > nothing, but we'd guarantee another huge set of arguments which would > > themselves also tell us nothing. > > That's an assumption. What if we get 10 million responses? Would you > still claim that the results are not representative? Yes, because you have no idea how big the population is. Maybe 10 million is the total population and it's representative. Maybe it's 50% of the population, disproportionately biased towards those of a given prior opinion. You can't know. > I think only *after* getting the results you would be able to say > anything about it's representativeness. > > Something more realistic, say you get at least 300 responses that > don't have any "geek" bias, that would be more than enough to make > some statistically significant conclusions. It really wouldn't. > > I disagree. Doing something that sucks more time and energy away from > > development without actually telling us anything in return is worse than > > that not happening. Felipe is obviously free to do whatever he wants, > > but there's no benefit in Gnome itself participating in any way. If we > > want to find out what our users think then the only way to do that is to > > have professional involvement and a random sample set. > > This is not sucking any time and energy from anybody, I just need > access to the server that has limesurvey installed, or somebody else > can do that (can't take that much time), I would contact all the > relevant news sites and make the relevant posts in social media. All > that that is needed from GNOME people is a blessing. The sucking of time and energy would come from the argument over the results afterwards. -- Matthew Garrett | mjg59 srcf ucam org
From: Felipe Contreras < felipe contreras gmail com> To: Matthew Garrett < mjg59 srcf ucam org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 22:26:08 +0300 On Fri, Aug 19, 2011 at 9:20 PM, Matthew Garrett < mjg59 srcf ucam org> wrote: > On Fri, Aug 19, 2011 at 08:14:25PM +0300, Felipe Contreras wrote: >> On Fri, Aug 19, 2011 at 7:24 PM, Matthew Garrett < mjg59 srcf ucam org> wrote: >> > Any survey that isn't a carefully controlled randomly selected sample of >> > users doesn't result in learning. >> >> Unless the biases are identified, which we are trying to do. > > You can only identify the biases if you already know the population, and > you can only know the population if you've got a random sample set to > begin with. That's not true. You might need that if you want to account for *all* the biases, which nobody can do anyway. What most people do is try to figure them out, chances are you might be missing some of the biases, but hopefully the unidentified misrepresented group won't be that big anyway, and thus wouldn't affect so much the analysis. If it turns out that a significant bias is not identified beforehand, that can be tackled in the next survey in 2012. >> Moreover, I have tried to push the idea to have an automatic >> notification, which would maximize the number of responders, and thus >> increase the randomization. But apparent this idea is not welcome. > > It doesn't help. The people most likely to respond to an irritating > popup that disrupts their work are people who already feel that gnome 3 > is an irritating piece of software that disrupts their work. You can't > get a random sample in-band. It doesn't help? It does randomize the sample more, doesn't it? Maybe it's not perfectly randomized, but nothing can ever be perfect. >> So, ideas to improve the randomization are dismissed, and then you say >> without randomization, the survey is not useful. IOW; you are >> intentionally deadlocking the proposal. > > I am saying that your results aren't useful unless your sample is > random. I don't know of a good way to obtain a representative sample. There's no such thing as 0% random, or 100% random, all we can thrive for is to increase the randomness. And I already explained that non-random samples are already useful if you can identify the biases. >> > Everyone will see what they want to see. Those >> > who believe that Gnome 3 is a step back will point out that the majority >> > of responses are negative. Those who believe it's a step forward will >> > point out that happy users are going to be far less inclined to respond. >> > >> > There's no way whatsoever to determine how representative the responses >> > are, and so there's no way whatsoever to learn anything about the >> > population. All we'd learn is that some users like Gnome 3 and some >> > users don't, and that's something we *already know*. So we'd gain >> > nothing, but we'd guarantee another huge set of arguments which would >> > themselves also tell us nothing. >> >> That's an assumption. What if we get 10 million responses? Would you >> still claim that the results are not representative? > > Yes, because you have no idea how big the population is. Maybe 10 > million is the total population and it's representative. Maybe it's 50% > of the population, disproportionately biased towards those of a given > prior opinion. You can't know. Do you have any idea what is the likelihood of that happening? Try throwing a dice 10 times and always getting 1-3. Even if the dice is rigged, it's very unlikely. It gets exponentially less likely 1 million times. >> I think only *after* getting the results you would be able to say >> anything about it's representativeness. >> >> Something more realistic, say you get at least 300 responses that >> don't have any "geek" bias, that would be more than enough to make >> some statistically significant conclusions. > > It really wouldn't. Yes it would. Check Cochran's formulas. 300 unbiased responses gives you already good statistical power, after a certain point it doesn't matter much what is the total population; 10m, 30m, 1m. The likelihood that would would get 300 unbiased responses all pointing to the wrong direction is almost nothing, in fact a few dozens would do (if they are truly random). There are some simple calculators online: http://www.surveysystem.com/sscalc.htm But yeah, since there's going to be bias, you need more. >> > I disagree. Doing something that sucks more time and energy away from >> > development without actually telling us anything in return is worse than >> > that not happening. Felipe is obviously free to do whatever he wants, >> > but there's no benefit in Gnome itself participating in any way. If we >> > want to find out what our users think then the only way to do that is to >> > have professional involvement and a random sample set. >> >> This is not sucking any time and energy from anybody, I just need >> access to the server that has limesurvey installed, or somebody else >> can do that (can't take that much time), I would contact all the >> relevant news sites and make the relevant posts in social media. All >> that that is needed from GNOME people is a blessing. > > The sucking of time and energy would come from the argument over the > results afterwards. Why? If there big unidentified biases, or the analysis is unsound, you can just ignore them. But what if they are not and there's really something useful there? Then the energy will be well spent. Would you rather keep walking carelessly in a dark room, or would you try to use your cellphone's screen to guide you? Sure, the light of the cellphone is very poor, and you might see things that are not really there, but at least it's something. To me GNOME is hitting everything in the room as it's going forward, and saying; I'm fine, I know where I'm going... -- Felipe Contreras
From: Matthew Garrett < mjg59 srcf ucam org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 21:14:11 +0100 On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: > > Yes, because you have no idea how big the population is. Maybe 10 > > million is the total population and it's representative. Maybe it's 50% > > of the population, disproportionately biased towards those of a given > > prior opinion. You can't know. > > Do you have any idea what is the likelihood of that happening? Try > throwing a dice 10 times and always getting 1-3. Even if the dice is > rigged, it's very unlikely. It gets exponentially less likely 1 > million times. That's clearly wrong. If you have a bucket of red balls and blue balls and you draw 10 million balls, and you find that you drew 6 million red balls and 4 million blue balls, what does that tell you? If you're sampling randomly it tells you that there are more red balls than blue balls. If you're subconsciously preferring to pick up red balls then it tells you nothing. So we need to avoid subconsciously picking red balls, which means we need to pick users randomly which is something we can't do with a voluntary survey. Cochran's formulas don't apply here because you're not picking your sample set at random. -- Matthew Garrett | mjg59 srcf ucam org
From: Felipe Contreras < felipe contreras gmail com> To: Matthew Garrett < mjg59 srcf ucam org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 23:43:01 +0300 On Fri, Aug 19, 2011 at 11:14 PM, Matthew Garrett < mjg59 srcf ucam org> wrote: > On Fri, Aug 19, 2011 at 10:26:08PM +0300, Felipe Contreras wrote: >> > Yes, because you have no idea how big the population is. Maybe 10 >> > million is the total population and it's representative. Maybe it's 50% >> > of the population, disproportionately biased towards those of a given >> > prior opinion. You can't know. >> >> Do you have any idea what is the likelihood of that happening? Try >> throwing a dice 10 times and always getting 1-3. Even if the dice is >> rigged, it's very unlikely. It gets exponentially less likely 1 >> million times. > > That's clearly wrong. If you have a bucket of red balls and blue balls > and you draw 10 million balls, and you find that you drew 6 million red > balls and 4 million blue balls, what does that tell you? If you're > sampling randomly it tells you that there are more red balls than blue > balls. If you're subconsciously preferring to pick up red balls then it > tells you nothing. So we need to avoid subconsciously picking red balls, > which means we need to pick users randomly which is something we can't > do with a voluntary survey. Cochran's formulas don't apply here because > you're not picking your sample set at random. That's a very bad example. An example closer to reality would be that color is indeed the bias, but we are not interested in the color, but the size of the balls. After the survey, we find out that overall, red balls are bigger than blue balls. Fortunately we don't care about the proportion of blue vs red balls in the total population, we only care about blue balls, so, we only consider the size of those. In the GNOME case, the color of the balls corresponds to the bias we want to identify; like geekness, and the size is the actual thing we are interested on, which is their happiness. We only care about non geeks (blue balls), as many GNOME people have stated, the real target users are the ones that don't even know what is GNOME. Now, if what you are worried about is the self-selection bias, we can add a new question "Why are you taking this survey?" with the option "Somebody is pushing me", and encourage people to push their relatives/colleagues/friends to fill the survey (just like a "professional" firm would, except "crowd-sourced"). Then, for external validity, you only consider the results of the people that answered "Somebody is pushing me" (they don't have self-selection bias). -- Felipe Contreras
From: Alan Cox < alan lxorguk ukuu org uk> To: Matthew Garrett < mjg59 srcf ucam org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 18:15:03 +0100 > Any survey that isn't a carefully controlled randomly selected sample of > users doesn't result in learning. It results in data that forms some You need truely or reasonably random samples for certain kinds of activities and analysis in particularly quantitative analysis when you want to perform p tests and the like. You don't need it in order to learn merely to generate statistical proofs and those are often quite useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have much value. You do not need it for explorative learning. Small children do not need to open a statistically valid sample of randomly chosen doors to learn about doors ! I for one would not be surprised if a lot of responses were not more positive than some seem to think. There has been time for people to use it and adjust and apply the fixes. Even odder there is no Gnome fork. If as I hear 'Gnome 3 is hated by technical people' and there are enough who care there ought to be a Gnome fork by now. But what do we have - exde, dead, turned into a one page rant and no code Mate - described by phoronix as "The Mate Desktop Environment fork of GNOME2 was started by an Arch Linux user back in June, but it hasn't yet gained too much traction and is mostly just talked about on various forums around the web. " which about sums it up. > sort of rorschach blot. Everyone will see what they want to see. Those > who believe that Gnome 3 is a step back will point out that the majority > of responses are negative. Those who believe it's a step forward will > point out that happy users are going to be far less inclined to respond. You seem to be assuming the results and that the only question of interest is "does gnome 3 suck". > nothing, but we'd guarantee another huge set of arguments which would > themselves also tell us nothing. Those will tell you a lot if someone analyses them. Again you may not be able to do formal mathematical tests on them but so what. > If we want to find out what our users think then the only way to do that is to > have professional involvement and a random sample set. Of course, and the only way to produce a kernel or desktop is to hire professionals to do it for you no doubt. Alan
From: Matthew Garrett < mjg59 srcf ucam org> To: Alan Cox < alan lxorguk ukuu org uk> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 19:28:24 +0100 (Resend: Managed to leave d-d-l off Cc: by accident) On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: > > Any survey that isn't a carefully controlled randomly selected sample of > > users doesn't result in learning. It results in data that forms some > > You need truely or reasonably random samples for certain kinds of > activities and analysis in particularly quantitative analysis when you > want to perform p tests and the like. You don't need it in order to > learn merely to generate statistical proofs and those are often quite > useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have > much value. You do not need it for explorative learning. Small children > do not need to open a statistically valid sample of randomly chosen doors > to learn about doors ! I am all for making it easier for people to give feedback about Gnome, but presenting it as a survey gives a strong implication that the results are meaningful as an aggregate rather than as a collection of anecdotes. If we want to hear form users, let's make it easy for users to talk to us. A survey isn't the way to achieve that. > > sort of rorschach blot. Everyone will see what they want to see. Those > > who believe that Gnome 3 is a step back will point out that the majority > > of responses are negative. Those who believe it's a step forward will > > point out that happy users are going to be far less inclined to respond. > > You seem to be assuming the results and that the only question of interest > is "does gnome 3 suck". I'm assuming that the sort of people who are going to go to the effort of filling out a survey are likely to be closer to the population discussing things on lwn than the population of usres in general. That may be entirely untrue! But if we get the opposite results then it still doesn't tell us anything that's actually true, and it's still an opportunity to argue the issue rather than focus on making software better. > > If we want to find out what our users think then the only way to do that is to > > have professional involvement and a random sample set. > > Of course, and the only way to produce a kernel or desktop is to hire > professionals to do it for you no doubt. If you went back to 1991 and wanted a production-quality kernel within a year, Linux probably wouldn't be your starting point. There'd be a learning process involved with setting up a professional-quality survey team, and the first few attempts would be pretty buggy. We'd get there in time, but until then... -- Matthew Garrett | mjg59 srcf ucam org
From: Felipe Contreras < felipe contreras gmail com> To: Matthew Garrett < mjg59 srcf ucam org> Cc: desktop-devel-list gnome org, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 22:53:46 +0300 On Fri, Aug 19, 2011 at 9:28 PM, Matthew Garrett < mjg59 srcf ucam org> wrote: > (Resend: Managed to leave d-d-l off Cc: by accident) > On Fri, Aug 19, 2011 at 06:15:03PM +0100, Alan Cox wrote: >> > Any survey that isn't a carefully controlled randomly selected sample of >> > users doesn't result in learning. It results in data that forms some >> >> You need truely or reasonably random samples for certain kinds of >> activities and analysis in particularly quantitative analysis when you >> want to perform p tests and the like. You don't need it in order to >> learn merely to generate statistical proofs and those are often quite >> useless anyway. Proviing gnome 3 is great/indifferent/sucks doesn't have >> much value. You do not need it for explorative learning. Small children >> do not need to open a statistically valid sample of randomly chosen doors >> to learn about doors ! > > I am all for making it easier for people to give feedback about Gnome, > but presenting it as a survey gives a strong implication that the > results are meaningful as an aggregate rather than as a collection of > anecdotes. If we want to hear form users, let's make it easy for users > to talk to us. A survey isn't the way to achieve that. Again, any better suggestions? I tried many of them back in 2007, and got nowhere, I think a user survey is the best one we've got. >> > sort of rorschach blot. Everyone will see what they want to see. Those >> > who believe that Gnome 3 is a step back will point out that the majority >> > of responses are negative. Those who believe it's a step forward will >> > point out that happy users are going to be far less inclined to respond. >> >> You seem to be assuming the results and that the only question of interest >> is "does gnome 3 suck". > > I'm assuming that the sort of people who are going to go to the effort > of filling out a survey are likely to be closer to the population > discussing things on lwn than the population of usres in general. That > may be entirely untrue! But if we get the opposite results then it still > doesn't tell us anything that's actually true, and it's still an > opportunity to argue the issue rather than focus on making software > better. We most likely are going to be able to identify that bias. Let's make some wild guesses; 50% of the people that use GNOME 3 like it, and 50% don't. Of that amount, 90% seem to be geeks. In the remaining 10%, the people that use GNOME 2 show 80% happiness, and of GNOME 3 it's only 60%. But you still don't think there's any value in there, fair enough. Then we dig through that 40% subset that didn't like GNOME 3 and take a look at their comments, and we find "Very strange", "Can't get used to it", and things like that. At that point we might want to see if they left an email to contact them, and then try to gather more detailed feedback. I think there's a chance that this survey could tell us something that's "actually true". >> > If we want to find out what our users think then the only way to do that is to >> > have professional involvement and a random sample set. >> >> Of course, and the only way to produce a kernel or desktop is to hire >> professionals to do it for you no doubt. > > If you went back to 1991 and wanted a production-quality kernel within a > year, Linux probably wouldn't be your starting point. There'd be a > learning process involved with setting up a professional-quality survey > team, and the first few attempts would be pretty buggy. We'd get there > in time, but until then... Until then it's better to have nothing? -- Felipe Contreras
From: Matthew Garrett < mjg59 srcf ucam org> To: Felipe Contreras < felipe contreras gmail com> Cc: desktop-devel-list gnome org, Alan Cox < alan lxorguk ukuu org uk> Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 21:05:26 +0100 On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: > > If you went back to 1991 and wanted a production-quality kernel within a > > year, Linux probably wouldn't be your starting point. There'd be a > > learning process involved with setting up a professional-quality survey > > team, and the first few attempts would be pretty buggy. We'd get there > > in time, but until then... > > Until then it's better to have nothing? It's better to have no data than to have misleading data. -- Matthew Garrett | mjg59 srcf ucam org
From: Alan Cox < alan lxorguk ukuu org uk> To: Matthew Garrett < mjg59 srcf ucam org> Cc: desktop-devel-list gnome org Subject: Re: GNOME user survey 2011 (v4) Date: Fri, 19 Aug 2011 21:20:59 +0100 On Fri, 19 Aug 2011 21:05:26 +0100 Matthew Garrett < mjg59 srcf ucam org> wrote: > On Fri, Aug 19, 2011 at 10:53:46PM +0300, Felipe Contreras wrote: > > > If you went back to 1991 and wanted a production-quality kernel within a > > > year, Linux probably wouldn't be your starting point. There'd be a > > > learning process involved with setting up a professional-quality survey > > > team, and the first few attempts would be pretty buggy. We'd get there > > > in time, but until then... > > > > Until then it's better to have nothing? > > It's better to have no data than to have misleading data. It's better to have no desktop than one that might not be production quality ? Same argument, same problem. PS data is never misleading. It's presentation maybe misleading but the data is just bits. I do think the comments on more open and why fill in the box type questions are on the button for the reasons expressed about sample size, randomness and what it would be useful to learn. Or perhaps rerun Federico's survey ? Alan