Openmoko Bugtracker

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Openmoko Bugtracker

Holger Freyther-2
Hey,

the signal noise ratio of the bugtracker is in a state that I'm close to
ignoring it completely.

It can not be that:
        - People say +1 and me too. It is not a popularity contest
        - People playing with components, milestones, severity...
        - Hijacking bug reports. How and when should such a hijacked bug be closed?
        - Not checking for duplicates before posting their bug.

If things continue that way I'm _forced_ to ask our engineers to not waste
their time with the bugtracker.

Rule of thumb: The more precise your bug is formulated and you act on requests
to help triaging the bug the more interested the developer is to fix that.

z.

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Steven **-3
Is it better to leave all those fields blank then?  Apparently, some
of them have special meaning.  Or is there some sort of guide/readme
to instruct users what fields mean?

-Steven

On Mon, Aug 11, 2008 at 11:16 AM, Holger Freyther <[hidden email]> wrote:

> Hey,
>
> the signal noise ratio of the bugtracker is in a state that I'm close to
> ignoring it completely.
>
> It can not be that:
>        - People say +1 and me too. It is not a popularity contest
>        - People playing with components, milestones, severity...
>        - Hijacking bug reports. How and when should such a hijacked bug be closed?
>        - Not checking for duplicates before posting their bug.
>
> If things continue that way I'm _forced_ to ask our engineers to not waste
> their time with the bugtracker.
>
> Rule of thumb: The more precise your bug is formulated and you act on requests
> to help triaging the bug the more interested the developer is to fix that.
>
> z.
>
> _______________________________________________
> devel mailing list
> [hidden email]
> https://lists.openmoko.org/mailman/listinfo/devel
>

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Mike Montour
In reply to this post by Holger Freyther-2

On 11-Aug-08, at 9:16 AM, Holger Freyther wrote:

> Hey,
>
> the signal noise ratio of the bugtracker is in a state that I'm  
> close to
> ignoring it completely.
>
> It can not be that:
> - People say +1 and me too. It is not a popularity contest

The first "me too" might be useful, confirming that the issue is real  
and not just something that the original reporter was doing wrong.

As for the "popularity contest", Bugzilla has a "voting" capability  
that allows users to indicate the bugs that are most important from  
their perspective. I don't know if Trac has anything similar.

> - People playing with components, milestones, severity...

It would help to have reporting guidelines on the front page of http://docs.openmoko.org/trac/ 
   to tell users what the policy is. For example I recently re-opened  
#676 and I changed its severity from High to Normal. I don't know if  
that was the right thing to do, or if I should have left the severity  
field as-is.

Is it possible to configure Trac so that only certain people (OM  
employees) can edit the components/milestones/severity fields?

> - Hijacking bug reports. How and when should such a hijacked bug be  
> closed?

I would say to un-"hijack" it by creating a second report, linking the  
bugs to each other, and updating the description to make it clear that  
the original bug was only for a specific aspect of the problem. Then  
leave the original bug open until its specific problem has been  
resolved.

> - Not checking for duplicates before posting their bug.

Another item to add to the front-page reporting guidelines

How about adding a "recently closed as duplicate" filter to the list  
of available reports at http://docs.openmoko.org/trac/report/ , and  
instructing users to look at that before reporting any new issues?  
That would at least catch the case of users submitting multiple  
reports for recent bugs, although it doesn't help when someone re-
discovers a months-old bug.

> If things continue that way I'm _forced_ to ask our engineers to not  
> waste
> their time with the bugtracker.

If you do that, please at least hire somebody (or recruit a community  
volunteer) to go through the bugtracker on a regular basis and add  
some special tag to the items that should be looked at by the engineers.


_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Olivier Berger-3
In reply to this post by Holger Freyther-2
Holger Freyther <[hidden email]> writes:

> Hey,
>
> the signal noise ratio of the bugtracker is in a state that I'm close to
> ignoring it completely.
>
> It can not be that:
> - People say +1 and me too. It is not a popularity contest
> - People playing with components, milestones, severity...
> - Hijacking bug reports. How and when should such a hijacked bug be closed?
> - Not checking for duplicates before posting their bug.
>

Nor can it be that some @openmoko.org person closes tickets which are
fully legitimate, and pretend that people use unsupported software
whereas they do not (the gsmd case).


I can't believe you seriously mean what you wrote !!


Either use a bugtracker open to your customers (which we are, hence,
the right to complain for what goes wrong IMHO), or not at all...

...but if you have thousands of customers, you must be ready to cope with
flooding by the users when bugs are easily met, and when these are
quite annoying (like the pin and keyboard ones in OM 2008.8).

If you hadn't shipped buggy software (and had done more testing on
2008.8, instead of try and meeting the absurd magic 2008/08/08 date...
or you're so superstitious ? : that's no good way to do releases
IMHO), you wouldn't have gotten such amount of complaints.

Sorry, but you're not behaving like someone in charge of customer
support for a 400$ device, IMHO.


/me quite pissed !

You should be glad that users try and report in the BT instead of
trolling on the lists.

Please DO respect your CUSTOMERS, again.

--
Olivier BERGER
(OpenPGP: 1024D/B4C5F37F)
http://www.olivierberger.com/weblog/

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

andy green-3
In reply to this post by Mike Montour
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Somebody in the thread at some point said:

|> If things continue that way I'm _forced_ to ask our engineers to not
|> waste
|> their time with the bugtracker.
|
| If you do that, please at least hire somebody (or recruit a community
| volunteer) to go through the bugtracker on a regular basis and add
| some special tag to the items that should be looked at by the engineers.

FWIW I seem to get a copy of all bug traffic in my inbox and at least
glance at everything.  Sometimes you get tantalizing clue from what is
reported that is very hard to come at yourself, like SD Card affecting
GPS).  It's hard to say we "waste our time" looking at what prompted
people to complain from the field, it all means something.

If you think about it, our current bug traffic is pretty much nothing
compared to what we would have to service if and when we start shipping
"real numbers" of phones.  The best answer in that case isn't going to
be "ignore it harder", it would need more infrastructure along Mike's
ideas, sort for first level support staff person who asks the questions
that should have been answered in the first place, etc.

- -Andy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkigpJUACgkQOjLpvpq7dMp4pgCcDfGMdiFJ7KyROj8JbxlDuan1
6PgAn2PqoZVj1d7fkD9FWn0o8Vzf9tTn
=/N/J
-----END PGP SIGNATURE-----

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Holger Freyther-2
On Monday 11 August 2008 22:44:05 Andy Green wrote:

> Somebody in the thread at some point said:
> |> If things continue that way I'm _forced_ to ask our engineers to not
> |> waste
> |> their time with the bugtracker.
> |
> | If you do that, please at least hire somebody (or recruit a community
> | volunteer) to go through the bugtracker on a regular basis and add
> | some special tag to the items that should be looked at by the engineers.
>
> FWIW I seem to get a copy of all bug traffic in my inbox and at least
> glance at everything.  Sometimes you get tantalizing clue from what is
> reported that is very hard to come at yourself, like SD Card affecting
> GPS).  It's hard to say we "waste our time" looking at what prompted
> people to complain from the field, it all means something.
>
> If you think about it, our current bug traffic is pretty much nothing
> compared to what we would have to service if and when we start shipping
> "real numbers" of phones.  The best answer in that case isn't going to
> be "ignore it harder", it would need more infrastructure along Mike's
> ideas, sort for first level support staff person who asks the questions
> that should have been answered in the first place, etc.

Sure, but with real numbers you will never ever reach the engineer responsible
for the issue. So no matter what kind of stuff you write, the engineer will
not see that but something that has been prepared, filtered, triaged.

A small example. Imagine you would be tick and responsible for assassin. For
some reason (probably people are unsure which component to use) people
select "Assassin" as component and tick gets the bug mail. So in his case
19/20 bug mails he gets are actually not for him so it is quite easy that
this one real bug mail that needs attention is lost.

So yes as mike said we might need to educate people to not use certain fields
(severity, milestone, component) and to stop the too many me too's. Actually
I think in most cases the confirmation is not really needed as the engineers
needs to be able to reproduce the issue anyway. We can add voting or just
count the number of CC's if we are unsure about severity. If educating fails,
what should we do? Allow only certain people (inside and outside of openmoko)
to change these fields? What if hijacking of issues do not stop? Should one
discuss bugs on the mailinglist and then move that information into the
bugtracker?

E.g. with the SIM PIN Dialog it is pretty obvious that the severity is high, a
software problem and that it impacts many people and the first output of
logread was already really helpful.


regards
        z.

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Holger Freyther-2
In reply to this post by Steven **-3
On Monday 11 August 2008 19:23:42 Steven ** wrote:
> Is it better to leave all those fields blank then?  Apparently, some
> of them have special meaning.  Or is there some sort of guide/readme
> to instruct users what fields mean?

I really like mike's answer. Let us see how well that would work and if
un-hijacking is really possible.... time for more documentation.


z.

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Werner Almesberger
In reply to this post by Holger Freyther-2
Holger Freyther wrote:
> So yes as mike said we might need to educate people to not use certain fields
> (severity, milestone, component)

Remove "project management" fields like priority, severity, and milestone ?

> and to stop the too many me too's. Actually
> I think in most cases the confirmation is not really needed as the engineers
> needs to be able to reproduce the issue anyway.

It can add credibility to a confusing bug report, though.

> We can add voting or just count the number of CC's

I'm note sure Ccs are really a good indicator. Since bugs get copied
to the lists anyway, people interested in their progress may not
have much of an incentive to subscribe to the bug.

A "silent" (i.e., doesn't trigger a mail to be sent) counter sounds
like a good thing to try.

Speaking of "silent", I wouldn't mind if only changes that add text
trigger mails. Cc list changes tend to be irrelevant. Even changes
of assignment, etc., are usually not interesting if you're subscribed
to the list where the bug originally appeared.

> Should one
> discuss bugs on the mailinglist and then move that information into the
> bugtracker?

It would be great if the bug tracker would just record any discussion
thread(s) that the bug report and updates spawn. That way, one could
process bugs by mail, which should be a lot more efficient than the
Web interface.

Regarding hijacking, how about just ignoring the noise ? (And let
others take care of the flaming ;-)

- Werner

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Holger Freyther-2
In reply to this post by Olivier Berger-3
On Monday 11 August 2008 22:00:20 Olivier Berger wrote:

Hey Olivier,

> quite annoying (like the pin and keyboard ones in OM 2008.8).

Please reread the bugs. I did close both of them as WORKSFORSOME as there
never can be a real fix for all the things that were brought up. In my
WORKSFORSOME close you can see the issues I identified and it is up to the
guy that mentions the issue to file a ticket for that or file tickets for
issues I have not identified in the message. One issue one ticket, anything
else is not manageable. How should QA ever check if all the different
mentioned issues are fixed? This will ask for duplicates but luckily there is
an easy way to deal with duplicates.



>
> If you hadn't shipped buggy software (and had done more testing on
> 2008.8, instead of try and meeting the absurd magic 2008/08/08 date...
> or you're so superstitious ? : that's no good way to do releases
> IMHO), you wouldn't have gotten such amount of complaints.

Our test plans should be available to everyone. One is free to take a look and
provide feedback. I'm sure QA is interested in improving them.

I'm actually surprised we don't get more bugs, what I'm surprised is the
emotions and culture. I have not seen this at bugs.kde.org,
bugzilla.webkit.org and Opie's mantis, so I wonder what they do right and
what is wrong with docs.openmoko.org.


> Sorry, but you're not behaving like someone in charge of customer
> support for a 400$ device, IMHO.
>
>
> /me quite pissed !

Well, I'm not in charge of customer support. I'm just contracted to work
on "Qtopia on X11" but I happen to be the one most active in it (both in my
paid and _unpaid_ time) because I do care that the technical issues get
resolved and I'm on of the fews that use the neo as primary phone. So instead
of being pissed off you are free to help to make best use of my resource by
having one bug for one issue and improving the signal to noise ratio.



> You should be glad that users try and report in the BT instead of
> trolling on the lists.
>
> Please DO respect your CUSTOMERS, again.

I care so much that I read every single bug report and most of the comments.
It is at a stage where I don't fix any bug at all but try to help users to
make good use of the bugtracker, fix components, ask for log messages, try to
figure what image they have and did to it...because no one else is doing that
and I think that is very important...

The summary of today would be: No progress in putting things into the testing
feed, but dealt with 1st level support.

So think about it. If people would help to make the bugtracker more usable I
could roll out a testing package for Qtopia that to my believe fixes the SIM
PIN Dialog and No Network issue. And I think with little work everyone can
benefit.


have a good night and please consider helping
        z.

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Sjors
In reply to this post by Holger Freyther-2
On Tue, Aug 12, 2008 at 2:16 AM, Holger Freyther <[hidden email]> wrote:

> Hey,
>
> the signal noise ratio of the bugtracker is in a state that I'm close to
> ignoring it completely.
>
> It can not be that:
>        - People say +1 and me too. It is not a popularity contest
>        - People playing with components, milestones, severity...
>        - Hijacking bug reports. How and when should such a hijacked bug be closed?
>        - Not checking for duplicates before posting their bug.

This seems to me, as has been suggested by others, a simple design
issue. The bug tracking system is way too complicated for anyone but
the most experienced developers. It is great that you make the bug
tracking available for everyone, but each of the issues you are
mentioning are completely inevitable.

And they will get worse with more users too; even if 90% of the users
was completely educated, the remaining 10% would produce an unbarable
noise.

I already filed a bug report on the admin trac about this issue two
days ago, speeking of not checking for duplicates :-)

https://admin-trac.openmoko.org/trac/ticket/1499

I have a number of suggestions:
* normal users can only fill in summary and description and can not
even see the other options
* a dedicated mailing list for new issues, e.g. [hidden email]
* a clear "tutorial" on how to properly process these issues (i.e.
look for duplicates, assign to the right department)
* anyone who is willing to follow the tutorial can join the ticketes
mailinglist and ask for increased privileges

Furthermore:
* different mailing lists for different components
* only mail tickets of the right component to the right mailinglist
* do not mail the entire list upon changes of the tickets (although
people can also just mute the topic in their mail client (e.g. gmail:
press m)

This should free up this devel list a lot; current traffic is too high
for anyone but full time employers.

Personally, I would really like to see *more* people file bug reports,
rather than less.

Sjors

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
roh
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

roh
Administrator
In reply to this post by Mike Montour
Mike Montour wrote:

> On 11-Aug-08, at 9:16 AM, Holger Freyther wrote:
>
>> Hey,
>>
>> the signal noise ratio of the bugtracker is in a state that I'm  
>> close to
>> ignoring it completely.
>>
>> It can not be that:
>> - People say +1 and me too. It is not a popularity contest
>
> The first "me too" might be useful, confirming that the issue is real  
> and not just something that the original reporter was doing wrong.
>
> As for the "popularity contest", Bugzilla has a "voting" capability  
> that allows users to indicate the bugs that are most important from  
> their perspective. I don't know if Trac has anything similar.
>
>> - People playing with components, milestones, severity...
>
> It would help to have reporting guidelines on the front page of http://docs.openmoko.org/trac/ 
>    to tell users what the policy is. For example I recently re-opened  
> #676 and I changed its severity from High to Normal. I don't know if  
> that was the right thing to do, or if I should have left the severity  
> field as-is.

yes, please. when someone gets to write these guidelines i will happily
hack them into our skin, see

http://admin-trac.openmoko.org/trac/ticket/1499

> Is it possible to configure Trac so that only certain people (OM  
> employees) can edit the components/milestones/severity fields?

in theory yes, but then i would be forced to work on loads of valid
requests from people to be added to that group.
currently all authenticated users have TICKET_MODIFY and TICKET_CREATE
as well as TICKET_EDIT_CC.

also see
https://docs.openmoko.org/trac/wiki/TracPermissions

> How about adding a "recently closed as duplicate" filter to the list  
> of available reports at http://docs.openmoko.org/trac/report/ , and  
> instructing users to look at that before reporting any new issues?  
> That would at least catch the case of users submitting multiple  
> reports for recent bugs, although it doesn't help when someone re-
> discovers a months-old bug.

something like this? https://docs.openmoko.org/trac/report/12

--

Joachim Steiger
Openmoko Central Services

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

andy green-3
In reply to this post by Holger Freyther-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Somebody in the thread at some point said:

| A small example. Imagine you would be tick and responsible for
assassin. For
| some reason (probably people are unsure which component to use) people
| select "Assassin" as component and tick gets the bug mail. So in his case
| 19/20 bug mails he gets are actually not for him so it is quite easy that
| this one real bug mail that needs attention is lost.

I can already imagine it about the ambiguously but inticingly named
"System Software..." category.

| So yes as mike said we might need to educate people to not use certain
fields
| (severity, milestone, component) and to stop the too many me too's.
Actually
| I think in most cases the confirmation is not really needed as the
engineers
| needs to be able to reproduce the issue anyway. We can add voting or just
| count the number of CC's if we are unsure about severity. If educating
fails,
| what should we do? Allow only certain people (inside and outside of
openmoko)
| to change these fields? What if hijacking of issues do not stop?
Should one
| discuss bugs on the mailinglist and then move that information into the
| bugtracker?

I wouldn't call it hijacking if we give them the field to edit in the
first place they're blameless if the meddle with it.  It would be better
if the bug owner alone marked it with his opinion of where it fitted in
his other priorities, because typically every bug will be "blocker" for
a customer.

- -Andy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAkihPM4ACgkQOjLpvpq7dMpekgCfSBlA1/9R9xrF7IyiPcRl/xGW
1vcAnjqXG9v7YStZl+QyumLvD8kTV41Z
=5N7Z
-----END PGP SIGNATURE-----

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Steven **-3
In reply to this post by roh
Are you sure?  Because I can no longer edit the CC field.  Which is an
issue, since it seems the tickets I create don't get sent to the devel
list by default. (I assume there's something special that needs added
to make that happen).

-Steven

On Mon, Aug 11, 2008 at 10:33 PM, Joachim Steiger <[hidden email]> wrote:
>> Is it possible to configure Trac so that only certain people (OM
>> employees) can edit the components/milestones/severity fields?
>
> in theory yes, but then i would be forced to work on loads of valid
> requests from people to be added to that group.
> currently all authenticated users have TICKET_MODIFY and TICKET_CREATE
> as well as TICKET_EDIT_CC.

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Hans L
On Mon, Aug 11, 2008 at 11:16 AM, Holger Freyther <[hidden email]> wrote:
> ...
>        - People playing with components, milestones, severity...

One problem I noticed that might be partially a cause of your woes
seems to be a bug in Trac.  When creating a new Ticket, the default
priority and severity are "normal".  This seems fine and reasonable,
however, I noticed that if you drop down the select box to look at the
other options, but DON'T even select a new option, it automatically
sets to the top option(highest priority, blocker severity, component
Assassin, etc.)  This breaks the usual expected functionality of html
select boxes.  Dropping the list down should not change the value
unless the user explicitly chooses to do so.

I think this has a potential to cause accidental bad input when
creating bug reports.

-Hans

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
roh
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

roh
Administrator
In reply to this post by Steven **-3



> On Mon, Aug 11, 2008 at 10:33 PM, Joachim Steiger <[hidden email]> wrote:
>>> Is it possible to configure Trac so that only certain people (OM
>>> employees) can edit the components/milestones/severity fields?
>> in theory yes, but then i would be forced to work on loads of valid
>> requests from people to be added to that group.
>> currently all authenticated users have TICKET_MODIFY and TICKET_CREATE
>> as well as TICKET_EDIT_CC.

Steven ** wrote:
> Are you sure?  Because I can no longer edit the CC field.


yes sorry i didnt write a mail about this yet:

i got asked to change this and will propably also remove setting some
fields from the NewTicket form soon.

for now its configured that the internal developers get TICKET_MODIFY
while regular 'i clicked an account' users get TICKET_APPEND.

adding more users to that group is not a problem at all.
just contact me with a good excuse ;)
as long as i am sure you know how to use a bugtracker and what fields to
touch and which to leave there shouln't be a problem.

we just need to make sure that somebody who reports a bug does not mess
with fields we need to get some workflow into actually working down
tickets and fixing bugs, instead of wasting time sorting stuff when
somebody will just meddle with it minutes later.

> Which is an
> issue, since it seems the tickets I create don't get sent to the devel
> list by default. (I assume there's something special that needs added
> to make that happen).

not really

just make sure the bug has component 'unknown' (the default) or 'host
utilities' on adding the ticket.

we probably will make the NewTicket form quite basic and let the
developers set these fields when 'reviewing' the bug. thus we know
somebody who understands the real problem sets things like priority,
severity and the component.

the problem is: for a 'user' a bug is always 'high' priority and
'severe', regardless what it is.

we need to focus on solving bugs, not to file them, sort them again, and
again, and again



--

Joachim Steiger
Openmoko Central Services

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel
Reply | Threaded
Open this post in threaded view
|

Re: Openmoko Bugtracker

Steven **-3
Not always.  I did submit a bug that was trivial (had to do with the
GTK theme).  :-)

-Steven

On Tue, Aug 12, 2008 at 11:29 AM, Joachim Steiger <[hidden email]> wrote:
> the problem is: for a 'user' a bug is always 'high' priority and
> 'severe', regardless what it is.

_______________________________________________
devel mailing list
[hidden email]
https://lists.openmoko.org/mailman/listinfo/devel