docs: reporting-issues.rst: improved process esp. for stable regressions
Provide a shorter and easier process for users that deal with regressions in stable and longterm kernels, as those should be reported quickly. To realize this in the least-confusing way and without having steps multiple times in different places, split the 'search for existing reports' into two. That has the additinal benefit that users will search for them quickly when going through the step by step guide and thus will save them trouble if the find reports. Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info> CC: Randy Dunlap <rdunlap@infradead.org> Link: https://lore.kernel.org/r/d934c15e536bceeff5c40a126930ddf803548e08.1616181657.git.linux@leemhuis.info Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This commit is contained in:
parent
9bc4430db5
commit
4b9d49d1ec
|
@ -100,6 +100,11 @@ process won't feel wasted in the end:
|
||||||
willing to install the latest Linux version yourself. Be aware the latter
|
willing to install the latest Linux version yourself. Be aware the latter
|
||||||
will often be needed anyway to hunt down and fix issues.
|
will often be needed anyway to hunt down and fix issues.
|
||||||
|
|
||||||
|
* Perform a rough search for existing reports with your favorite internet
|
||||||
|
search engine; additionally, check the archives of the Linux Kernel Mailing
|
||||||
|
List (LKML). If you find matching reports, join the discussion instead of
|
||||||
|
sending a new one.
|
||||||
|
|
||||||
* See if the issue you are dealing with qualifies as regression, security
|
* See if the issue you are dealing with qualifies as regression, security
|
||||||
issue, or a really severe problem: those are 'issues of high priority' that
|
issue, or a really severe problem: those are 'issues of high priority' that
|
||||||
need special handling in some steps that are about to follow.
|
need special handling in some steps that are about to follow.
|
||||||
|
@ -122,16 +127,18 @@ process won't feel wasted in the end:
|
||||||
needs to get reported to the kernel developers separately, unless they are
|
needs to get reported to the kernel developers separately, unless they are
|
||||||
strongly entangled.
|
strongly entangled.
|
||||||
|
|
||||||
|
* If you are facing a regression within a stable or longterm version line
|
||||||
|
(say something broke when updating from 5.10.4 to 5.10.5), scroll down to
|
||||||
|
'Dealing with regressions within a stable and longterm kernel line'.
|
||||||
|
|
||||||
* Locate the driver or kernel subsystem that seems to be causing the issue.
|
* Locate the driver or kernel subsystem that seems to be causing the issue.
|
||||||
Find out how and where its developers expect reports. Note: most of the
|
Find out how and where its developers expect reports. Note: most of the
|
||||||
time this won't be bugzilla.kernel.org, as issues typically need to be sent
|
time this won't be bugzilla.kernel.org, as issues typically need to be sent
|
||||||
by mail to a maintainer and a public mailing list.
|
by mail to a maintainer and a public mailing list.
|
||||||
|
|
||||||
* Search the archives of the bug tracker or mailing list in question
|
* Search the archives of the bug tracker or mailing list in question
|
||||||
thoroughly for reports that might match your issue. Also check if you find
|
thoroughly for reports that might match your issue. If you find anything,
|
||||||
something with your favorite internet search engine or in the Linux Kernel
|
join the discussion instead of sending a new report.
|
||||||
Mailing List (LKML) archives. If you find anything, join the discussion
|
|
||||||
instead of sending a new report.
|
|
||||||
|
|
||||||
After these preparations you'll now enter the main part:
|
After these preparations you'll now enter the main part:
|
||||||
|
|
||||||
|
@ -187,17 +194,15 @@ After these preparations you'll now enter the main part:
|
||||||
help yourself, if you don't get any help or if it's unsatisfying.
|
help yourself, if you don't get any help or if it's unsatisfying.
|
||||||
|
|
||||||
|
|
||||||
Reporting issues only occurring in older kernel version lines
|
Reporting regressions within a stable and longterm kernel line
|
||||||
-------------------------------------------------------------
|
--------------------------------------------------------------
|
||||||
|
|
||||||
This section is for you, if you tried the latest mainline kernel as outlined
|
This subsection is for you, if you followed above process and got sent here at
|
||||||
above, but failed to reproduce your issue there; at the same time you want to
|
the point about regression within a stable or longterm kernel version line. You
|
||||||
see the issue fixed in older version lines or a vendor kernel that's regularly
|
face one of those if something breaks when updating from 5.10.4 to 5.10.5 (a
|
||||||
rebased on new stable or longterm releases. If that case follow these steps:
|
switch from 5.9.15 to 5.10.5 does not qualify). The developers want to fix such
|
||||||
|
regressions as quickly as possible, hence there is a streamlined process to
|
||||||
* Prepare yourself for the possibility that going through the next few steps
|
report them:
|
||||||
might not get the issue solved in older releases: the fix might be too big
|
|
||||||
or risky to get backported there.
|
|
||||||
|
|
||||||
* Check if the kernel developers still maintain the Linux kernel version
|
* Check if the kernel developers still maintain the Linux kernel version
|
||||||
line you care about: go to the front page of kernel.org and make sure it
|
line you care about: go to the front page of kernel.org and make sure it
|
||||||
|
@ -210,6 +215,30 @@ rebased on new stable or longterm releases. If that case follow these steps:
|
||||||
kernel. Ensure this kernel is not tainted and still shows the problem, as
|
kernel. Ensure this kernel is not tainted and still shows the problem, as
|
||||||
the issue might have already been fixed there.
|
the issue might have already been fixed there.
|
||||||
|
|
||||||
|
* Send a short problem report by mail to the people and mailing lists the
|
||||||
|
:ref:`MAINTAINERS <maintainers>` file specifies in the section 'STABLE
|
||||||
|
BRANCH'. Roughly describe the issue and ideally explain how to reproduce
|
||||||
|
it. Mention the first version that shows the problem and the last version
|
||||||
|
that's working fine. Then wait for further instructions.
|
||||||
|
|
||||||
|
The reference section below explains each of these steps in more detail.
|
||||||
|
|
||||||
|
|
||||||
|
Reporting issues only occurring in older kernel version lines
|
||||||
|
-------------------------------------------------------------
|
||||||
|
|
||||||
|
This subsection is for you, if you tried the latest mainline kernel as outlined
|
||||||
|
above, but failed to reproduce your issue there; at the same time you want to
|
||||||
|
see the issue fixed in older version lines or a vendor kernel that's regularly
|
||||||
|
rebased on new stable or longterm releases. If that case follow these steps:
|
||||||
|
|
||||||
|
* Prepare yourself for the possibility that going through the next few steps
|
||||||
|
might not get the issue solved in older releases: the fix might be too big
|
||||||
|
or risky to get backported there.
|
||||||
|
|
||||||
|
* Perform the first three steps in the section "Dealing with regressions
|
||||||
|
within a stable and longterm kernel line" above.
|
||||||
|
|
||||||
* Search the Linux kernel version control system for the change that fixed
|
* Search the Linux kernel version control system for the change that fixed
|
||||||
the issue in mainline, as its commit message might tell you if the fix is
|
the issue in mainline, as its commit message might tell you if the fix is
|
||||||
scheduled for backporting already. If you don't find anything that way,
|
scheduled for backporting already. If you don't find anything that way,
|
||||||
|
@ -218,22 +247,13 @@ rebased on new stable or longterm releases. If that case follow these steps:
|
||||||
deemed unsuitable for backporting. If backporting was not considered at
|
deemed unsuitable for backporting. If backporting was not considered at
|
||||||
all, join the newest discussion, asking if it's in the cards.
|
all, join the newest discussion, asking if it's in the cards.
|
||||||
|
|
||||||
* Check if you're dealing with a regression that was never present in
|
|
||||||
mainline by installing the first release of the version line you care
|
|
||||||
about. If the issue doesn't show up with it, you basically need to report
|
|
||||||
the issue with this version like you would report a problem with mainline
|
|
||||||
(see above). This ideally includes a bisection followed by a search for
|
|
||||||
existing reports on the net; with the help of the subject and the two
|
|
||||||
relevant commit-ids. If that doesn't turn up anything, write the report; CC
|
|
||||||
or forward the report to the stable maintainers, the stable mailing list,
|
|
||||||
and those who authored the change. Include the shortened commit-id if you
|
|
||||||
found the change that causes it.
|
|
||||||
|
|
||||||
* One of the former steps should lead to a solution. If that doesn't work
|
* One of the former steps should lead to a solution. If that doesn't work
|
||||||
out, ask the maintainers for the subsystem that seems to be causing the
|
out, ask the maintainers for the subsystem that seems to be causing the
|
||||||
issue for advice; CC the mailing list for the particular subsystem as well
|
issue for advice; CC the mailing list for the particular subsystem as well
|
||||||
as the stable mailing list.
|
as the stable mailing list.
|
||||||
|
|
||||||
|
The reference section below explains each of these steps in more detail.
|
||||||
|
|
||||||
|
|
||||||
Reference section: Reporting issues to the kernel maintainers
|
Reference section: Reporting issues to the kernel maintainers
|
||||||
=============================================================
|
=============================================================
|
||||||
|
@ -329,44 +349,25 @@ better than not reporting the issue at all: sometimes such reports directly or
|
||||||
indirectly will help to get the issue fixed over time.
|
indirectly will help to get the issue fixed over time.
|
||||||
|
|
||||||
|
|
||||||
Search for existing reports
|
Search for existing reports, first run
|
||||||
---------------------------
|
--------------------------------------
|
||||||
|
|
||||||
*Search the archives of the bug tracker or mailing list in question
|
*Perform a rough search for existing reports with your favorite internet
|
||||||
thoroughly for reports that might match your issue. Also check if you find
|
search engine; additionally, check the archives of the Linux Kernel Mailing
|
||||||
something with your favorite internet search engine or in the Linux Kernel
|
List (LKML). If you find matching reports, join the discussion instead of
|
||||||
Mailing List (LKML) archives. If you find anything, join the discussion
|
sending a new one.*
|
||||||
instead of sending a new report.*
|
|
||||||
|
|
||||||
Reporting an issue that someone else already brought forward is often a waste
|
Reporting an issue that someone else already brought forward is often a waste of
|
||||||
of time for everyone involved, especially you as the reporter. So it's in your
|
time for everyone involved, especially you as the reporter. So it's in your own
|
||||||
own interest to thoroughly check if somebody reported the issue already. Thus
|
interest to thoroughly check if somebody reported the issue already. At this
|
||||||
do not hurry with this step of the reporting process. Spending 30 to 60 minutes
|
step of the process it's okay to just perform a rough search: a later step will
|
||||||
or even more time can save you and others quite a lot of time and trouble.
|
tell you to perform a more detailed search once you know where your issue needs
|
||||||
|
to be reported to. Nevertheless, do not hurry with this step of the reporting
|
||||||
|
process, it can save you time and trouble.
|
||||||
|
|
||||||
The best place to search is the bug tracker or the mailing list where your
|
Simply search the internet with your favorite search engine first. Afterwards,
|
||||||
report needs to be filed. You'll find quite a few of those lists on
|
search the `Linux Kernel Mailing List (LKML) archives
|
||||||
`lore.kernel.org <https://lore.kernel.org/>`_, but some are hosted in
|
<https://lore.kernel.org/lkml/>`_.
|
||||||
different places. That for example is the case for the ath10k WiFi driver used
|
|
||||||
as example in the previous step. But you'll often find the archives for these
|
|
||||||
lists easily on the net. Searching for 'archive ath10k@lists.infradead.org' for
|
|
||||||
example will quickly lead you to the `Info page for the ath10k mailing list
|
|
||||||
<https://lists.infradead.org/mailman/listinfo/ath10k>`_, which at the top links
|
|
||||||
to its `list archives <https://lists.infradead.org/pipermail/ath10k/>`_.
|
|
||||||
|
|
||||||
Sadly this and quite a few other lists miss a way to search the archives. In
|
|
||||||
those cases use a regular internet search engine and add something like
|
|
||||||
'site:lists.infradead.org/pipermail/ath10k/' to your search terms, which limits
|
|
||||||
the results to the archives at that URL.
|
|
||||||
|
|
||||||
Additionally, search the internet and the `Linux Kernel Mailing List (LKML)
|
|
||||||
archives <https://lore.kernel.org/lkml/>`_, as maybe the real culprit might be
|
|
||||||
in some other subsystem. Searching in `bugzilla.kernel.org
|
|
||||||
<https://bugzilla.kernel.org/>`_ might also be a good idea, but if you find
|
|
||||||
anything there keep in mind: most subsystems expect reports in different
|
|
||||||
places, hence those you find there might have not even reached the people
|
|
||||||
responsible for the subsystem in question. Nevertheless, the data there might
|
|
||||||
provide valuable insights.
|
|
||||||
|
|
||||||
If you get flooded with results consider telling your search engine to limit
|
If you get flooded with results consider telling your search engine to limit
|
||||||
search timeframe to the past month or year. And wherever you search, make sure
|
search timeframe to the past month or year. And wherever you search, make sure
|
||||||
|
@ -387,6 +388,15 @@ developers might look for people that can provide additional information or
|
||||||
test a proposed fix. Jump to the section 'Duties after the report went out' for
|
test a proposed fix. Jump to the section 'Duties after the report went out' for
|
||||||
details on how to get properly involved.
|
details on how to get properly involved.
|
||||||
|
|
||||||
|
Note, searching `bugzilla.kernel.org <https://bugzilla.kernel.org/>`_ might also
|
||||||
|
be a good idea, as that might provide valuable insights or turn up matching
|
||||||
|
reports. If you find the latter, just keep in mind: most subsystems expect
|
||||||
|
reports in different places, as described below in the section "Check where you
|
||||||
|
need to report your issue". The developers that should take care of the issue
|
||||||
|
thus might not even be aware of the bugzilla ticket. Hence, check the ticket if
|
||||||
|
the issue already got reported as outlined in this document and if not consider
|
||||||
|
doing so.
|
||||||
|
|
||||||
|
|
||||||
Issue of high priority?
|
Issue of high priority?
|
||||||
-----------------------
|
-----------------------
|
||||||
|
@ -591,9 +601,25 @@ due to faulty hardware apart from a kernel issue that rarely happens and thus
|
||||||
is hard to reproduce.
|
is hard to reproduce.
|
||||||
|
|
||||||
|
|
||||||
Locate kernel area that causes the issue
|
Regression in stable or longterm kernel?
|
||||||
----------------------------------------
|
----------------------------------------
|
||||||
|
|
||||||
|
*If you are facing a regression within a stable or longterm version line
|
||||||
|
(say something broke when updating from 5.10.4 to 5.10.5), scroll down to
|
||||||
|
'Dealing with regressions within a stable and longterm kernel line'.*
|
||||||
|
|
||||||
|
Regression within a stable and longterm kernel version line are something the
|
||||||
|
Linux developers want to fix badly, as such issues are even more unwanted than
|
||||||
|
regression in the main development branch, as they can quickly affect a lot of
|
||||||
|
people. The developers thus want to learn about such issues as quickly as
|
||||||
|
possible, hence there is a streamlined process to report them. Note,
|
||||||
|
regressions with newer kernel version line (say something broke when switching
|
||||||
|
from 5.9.15 to 5.10.5) do not qualify.
|
||||||
|
|
||||||
|
|
||||||
|
Check where you need to report your issue
|
||||||
|
-----------------------------------------
|
||||||
|
|
||||||
*Locate the driver or kernel subsystem that seems to be causing the issue.
|
*Locate the driver or kernel subsystem that seems to be causing the issue.
|
||||||
Find out how and where its developers expect reports. Note: most of the
|
Find out how and where its developers expect reports. Note: most of the
|
||||||
time this won't be bugzilla.kernel.org, as issues typically need to be sent
|
time this won't be bugzilla.kernel.org, as issues typically need to be sent
|
||||||
|
@ -769,63 +795,39 @@ modified during tree-wide cleanups by developers that do not care about the
|
||||||
particular driver at all.
|
particular driver at all.
|
||||||
|
|
||||||
|
|
||||||
Search for existing reports
|
Search for existing reports, second run
|
||||||
---------------------------
|
---------------------------------------
|
||||||
|
|
||||||
*Search the archives of the bug tracker or mailing list in question
|
*Search the archives of the bug tracker or mailing list in question
|
||||||
thoroughly for reports that might match your issue. Also check if you find
|
thoroughly for reports that might match your issue. If you find anything,
|
||||||
something with your favorite internet search engine or in the Linux Kernel
|
join the discussion instead of sending a new report.*
|
||||||
Mailing List (LKML) archives. If you find anything, join the discussion
|
|
||||||
instead of sending a new report.*
|
|
||||||
|
|
||||||
Reporting an issue that someone else already brought forward is often a waste
|
As mentioned earlier already: reporting an issue that someone else already
|
||||||
of time for everyone involved, especially you as the reporter. So it's in your
|
brought forward is often a waste of time for everyone involved, especially you
|
||||||
own interest to thoroughly check if somebody reported the issue already. Thus
|
as the reporter. That's why you should search for existing report again, now
|
||||||
do not hurry with this step of the reporting process. Spending 30 to 60 minutes
|
that you know where they need to be reported to. If it's mailing list, you will
|
||||||
or even more time can save you and others quite a lot of time and trouble.
|
often find its archives on `lore.kernel.org <https://lore.kernel.org/>`_.
|
||||||
|
|
||||||
The best place to search is the bug tracker or the mailing list where your
|
But some list are hosted in different places. That for example is the case for
|
||||||
report needs to be filed. You'll find quite a few of those lists on
|
the ath10k WiFi driver used as example in the previous step. But you'll often
|
||||||
`lore.kernel.org <https://lore.kernel.org/>`_, but some are hosted in
|
find the archives for these lists easily on the net. Searching for 'archive
|
||||||
different places. That for example is the case for the ath10k WiFi driver used
|
ath10k@lists.infradead.org' for example will lead you to the `Info page for the
|
||||||
as example in the previous step. But you'll often find the archives for these
|
ath10k mailing list <https://lists.infradead.org/mailman/listinfo/ath10k>`_,
|
||||||
lists easily on the net. Searching for 'archive ath10k@lists.infradead.org' for
|
which at the top links to its
|
||||||
example will quickly lead you to the `Info page for the ath10k mailing list
|
`list archives <https://lists.infradead.org/pipermail/ath10k/>`_. Sadly this and
|
||||||
<https://lists.infradead.org/mailman/listinfo/ath10k>`_, which at the top links
|
quite a few other lists miss a way to search the archives. In those cases use a
|
||||||
to its `list archives <https://lists.infradead.org/pipermail/ath10k/>`_.
|
regular internet search engine and add something like
|
||||||
|
|
||||||
Sadly this and quite a few other lists miss a way to search the archives. In
|
|
||||||
those cases use a regular internet search engine and add something like
|
|
||||||
'site:lists.infradead.org/pipermail/ath10k/' to your search terms, which limits
|
'site:lists.infradead.org/pipermail/ath10k/' to your search terms, which limits
|
||||||
the results to the archives at that URL.
|
the results to the archives at that URL.
|
||||||
|
|
||||||
Additionally, search the internet and the `Linux Kernel Mailing List (LKML)
|
It's also wise to check the internet, LKML and maybe bugzilla.kernel.org again
|
||||||
archives <https://lore.kernel.org/lkml/>`_, as maybe the real culprit might be
|
at this point.
|
||||||
in some other subsystem. Searching in `bugzilla.kernel.org
|
|
||||||
<https://bugzilla.kernel.org/>`_ might also be a good idea, but if you find
|
|
||||||
anything there keep in mind: most subsystems expect reports in different
|
|
||||||
places, hence those you find there might have not even reached the people
|
|
||||||
responsible for the subsystem in question. Nevertheless, the data there might
|
|
||||||
provide valuable insights.
|
|
||||||
|
|
||||||
If you get flooded with results consider telling your search engine to limit
|
For details how to search and what to do if you find matching reports see
|
||||||
search timeframe to the past month or year. And wherever you search, make sure
|
"Search for existing reports, first run" above.
|
||||||
to use good search terms; vary them a few times, too. While doing so try to
|
|
||||||
look at the issue from the perspective of someone else: that will help you to
|
|
||||||
come up with other words to use as search terms. Also make sure not to use too
|
|
||||||
many search terms at once. Remember to search with and without information like
|
|
||||||
the name of the kernel driver or the name of the affected hardware component.
|
|
||||||
But its exact brand name (say 'ASUS Red Devil Radeon RX 5700 XT Gaming OC')
|
|
||||||
often is not much helpful, as it is too specific. Instead try search terms like
|
|
||||||
the model line (Radeon 5700 or Radeon 5000) and the code name of the main chip
|
|
||||||
('Navi' or 'Navi10') with and without its manufacturer ('AMD').
|
|
||||||
|
|
||||||
In case you find an existing report about your issue, join the discussion, as
|
Do not hurry with this step of the reporting process: spending 30 to 60 minutes
|
||||||
you might be able to provide valuable additional information. That can be
|
or even more time can save you and others quite a lot of time and trouble.
|
||||||
important even when a fix is prepared or in its final stages already, as
|
|
||||||
developers might look for people that can provide additional information or
|
|
||||||
test a proposed fix. Jump to the section 'Duties after the report went out' for
|
|
||||||
details on how to get properly involved.
|
|
||||||
|
|
||||||
|
|
||||||
Install a fresh kernel for testing
|
Install a fresh kernel for testing
|
||||||
|
@ -1512,32 +1514,11 @@ easier. And with a bit of luck there might be someone in the team that knows a
|
||||||
bit about programming and might be able to write a fix.
|
bit about programming and might be able to write a fix.
|
||||||
|
|
||||||
|
|
||||||
Details about reporting issues only occurring in older kernel version lines
|
Reference for "Reporting issues only occurring in older kernel version lines"
|
||||||
---------------------------------------------------------------------------
|
-----------------------------------------------------------------------------
|
||||||
|
|
||||||
This subsection provides details for steps you need to take if you could not
|
This subsection provides details for step you need to perform if you face a
|
||||||
reproduce your issue with a mainline kernel, but want to see it fixed in older
|
regression within a stable and longterm kernel line.
|
||||||
version lines (aka stable and longterm kernels).
|
|
||||||
|
|
||||||
Some fixes are too complex
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Prepare yourself for the possibility that going through the next few steps
|
|
||||||
might not get the issue solved in older releases: the fix might be too big
|
|
||||||
or risky to get backported there.*
|
|
||||||
|
|
||||||
Even small and seemingly obvious code-changes sometimes introduce new and
|
|
||||||
totally unexpected problems. The maintainers of the stable and longterm kernels
|
|
||||||
are very aware of that and thus only apply changes to these kernels that are
|
|
||||||
within rules outlined in 'Documentation/process/stable-kernel-rules.rst'.
|
|
||||||
|
|
||||||
Complex or risky changes for example do not qualify and thus only get applied
|
|
||||||
to mainline. Other fixes are easy to get backported to the newest stable and
|
|
||||||
longterm kernels, but too risky to integrate into older ones. So be aware the
|
|
||||||
fix you are hoping for might be one of those that won't be backported to the
|
|
||||||
version line your care about. In that case you'll have no other choice then to
|
|
||||||
live with the issue or switch to a newer Linux version, unless you want to
|
|
||||||
patch the fix into your kernels yourself.
|
|
||||||
|
|
||||||
Make sure the particular version line still gets support
|
Make sure the particular version line still gets support
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
@ -1581,7 +1562,85 @@ Reproduce issue with the newest release
|
||||||
Before investing any more time in this process you want to check if the issue
|
Before investing any more time in this process you want to check if the issue
|
||||||
was already fixed in the latest release of version line you're interested in.
|
was already fixed in the latest release of version line you're interested in.
|
||||||
This kernel needs to be vanilla and shouldn't be tainted before the issue
|
This kernel needs to be vanilla and shouldn't be tainted before the issue
|
||||||
happens, as detailed outlined already above in the process of testing mainline.
|
happens, as detailed outlined already above in the section "Install a fresh
|
||||||
|
kernel for testing".
|
||||||
|
|
||||||
|
Report the regression
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
*Send a short problem report by mail to the people and mailing lists the
|
||||||
|
:ref:`MAINTAINERS <maintainers>` file specifies in the section 'STABLE
|
||||||
|
BRANCH'. Roughly describe the issue and ideally explain how to reproduce
|
||||||
|
it. Mention the first version that shows the problem and the last version
|
||||||
|
that's working fine. Then wait for further instructions.*
|
||||||
|
|
||||||
|
When reporting a regression that happens within a stable or longterm kernel
|
||||||
|
line (say when updating from 5.10.4 to 5.10.5) a brief report is enough for
|
||||||
|
the start to get the issue reported quickly. Hence a rough description is all
|
||||||
|
it takes.
|
||||||
|
|
||||||
|
But note, it helps developers a great deal if you can specify the exact version
|
||||||
|
that introduced the problem. Hence if possible within a reasonable time frame,
|
||||||
|
try to find that version using vanilla kernels. Lets assume something broke when
|
||||||
|
your distributor released a update from Linux kernel 5.10.5 to 5.10.8. Then as
|
||||||
|
instructed above go and check the latest kernel from that version line, say
|
||||||
|
5.10.9. If it shows the problem, try a vanilla 5.10.5 to ensure that no patches
|
||||||
|
the distributor applied interfere. If the issue doesn't manifest itself there,
|
||||||
|
try 5.10.7 and then (depending on the outcome) 5.10.8 or 5.10.6 to find the
|
||||||
|
first version where things broke. Mention it in the report and state that 5.10.9
|
||||||
|
is still broken.
|
||||||
|
|
||||||
|
What the previous paragraph outlines is basically a rough manual 'bisection'.
|
||||||
|
Once your report is out your might get asked to do a proper one, as it allows to
|
||||||
|
pinpoint the exact change that causes the issue (which then can easily get
|
||||||
|
reverted to fix the issue quickly). Hence consider to do a proper bisection
|
||||||
|
right away if time permits. See the section 'Special care for regressions' and
|
||||||
|
the document 'Documentation/admin-guide/bug-bisect.rst' for details how to
|
||||||
|
perform one.
|
||||||
|
|
||||||
|
|
||||||
|
Reference for "Reporting regressions within a stable and longterm kernel line"
|
||||||
|
------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
This section provides details for steps you need to take if you could not
|
||||||
|
reproduce your issue with a mainline kernel, but want to see it fixed in older
|
||||||
|
version lines (aka stable and longterm kernels).
|
||||||
|
|
||||||
|
Some fixes are too complex
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
*Prepare yourself for the possibility that going through the next few steps
|
||||||
|
might not get the issue solved in older releases: the fix might be too big
|
||||||
|
or risky to get backported there.*
|
||||||
|
|
||||||
|
Even small and seemingly obvious code-changes sometimes introduce new and
|
||||||
|
totally unexpected problems. The maintainers of the stable and longterm kernels
|
||||||
|
are very aware of that and thus only apply changes to these kernels that are
|
||||||
|
within rules outlined in 'Documentation/process/stable-kernel-rules.rst'.
|
||||||
|
|
||||||
|
Complex or risky changes for example do not qualify and thus only get applied
|
||||||
|
to mainline. Other fixes are easy to get backported to the newest stable and
|
||||||
|
longterm kernels, but too risky to integrate into older ones. So be aware the
|
||||||
|
fix you are hoping for might be one of those that won't be backported to the
|
||||||
|
version line your care about. In that case you'll have no other choice then to
|
||||||
|
live with the issue or switch to a newer Linux version, unless you want to
|
||||||
|
patch the fix into your kernels yourself.
|
||||||
|
|
||||||
|
Common preparations
|
||||||
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
*Perform the first three steps in the section "Reporting issues only
|
||||||
|
occurring in older kernel version lines" above.*
|
||||||
|
|
||||||
|
You need to carry out a few steps already described in another section of this
|
||||||
|
guide. Those steps will let you:
|
||||||
|
|
||||||
|
* Check if the kernel developers still maintain the Linux kernel version line
|
||||||
|
you care about.
|
||||||
|
|
||||||
|
* Search the Linux stable mailing list for exiting reports.
|
||||||
|
|
||||||
|
* Check with the latest release.
|
||||||
|
|
||||||
|
|
||||||
Check code history and search for existing discussions
|
Check code history and search for existing discussions
|
||||||
|
@ -1636,164 +1695,6 @@ discussions abound it.
|
||||||
join the discussion: mention the version where you face the issue and that
|
join the discussion: mention the version where you face the issue and that
|
||||||
you would like to see it fixed, if suitable.
|
you would like to see it fixed, if suitable.
|
||||||
|
|
||||||
Details about reporting issues only occurring in older kernel version lines
|
|
||||||
---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
This subsection provides details for steps you need to take if you could not
|
|
||||||
reproduce your issue with a mainline kernel, but want to see it fixed in older
|
|
||||||
version lines (aka stable and longterm kernels).
|
|
||||||
|
|
||||||
Some fixes are too complex
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Prepare yourself for the possibility that going through the next few steps
|
|
||||||
might not get the issue solved in older releases: the fix might be too big
|
|
||||||
or risky to get backported there.*
|
|
||||||
|
|
||||||
Even small and seemingly obvious code-changes sometimes introduce new and
|
|
||||||
totally unexpected problems. The maintainers of the stable and longterm kernels
|
|
||||||
are very aware of that and thus only apply changes to these kernels that are
|
|
||||||
within rules outlined in 'Documentation/process/stable-kernel-rules.rst'.
|
|
||||||
|
|
||||||
Complex or risky changes for example do not qualify and thus only get applied
|
|
||||||
to mainline. Other fixes are easy to get backported to the newest stable and
|
|
||||||
longterm kernels, but too risky to integrate into older ones. So be aware the
|
|
||||||
fix you are hoping for might be one of those that won't be backported to the
|
|
||||||
version line your care about. In that case you'll have no other choice then to
|
|
||||||
live with the issue or switch to a newer Linux version, unless you want to
|
|
||||||
patch the fix into your kernels yourself.
|
|
||||||
|
|
||||||
Make sure the particular version line still gets support
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Check if the kernel developers still maintain the Linux kernel version
|
|
||||||
line you care about: go to the front page of kernel.org and make sure it
|
|
||||||
mentions the latest release of the particular version line without an
|
|
||||||
'[EOL]' tag.*
|
|
||||||
|
|
||||||
Most kernel version lines only get supported for about three months, as
|
|
||||||
maintaining them longer is quite a lot of work. Hence, only one per year is
|
|
||||||
chosen and gets supported for at least two years (often six). That's why you
|
|
||||||
need to check if the kernel developers still support the version line you care
|
|
||||||
for.
|
|
||||||
|
|
||||||
Note, if kernel.org lists two 'stable' version lines on the front page, you
|
|
||||||
should consider switching to the newer one and forget about the older one:
|
|
||||||
support for it is likely to be abandoned soon. Then it will get a "end-of-life"
|
|
||||||
(EOL) stamp. Version lines that reached that point still get mentioned on the
|
|
||||||
kernel.org front page for a week or two, but are unsuitable for testing and
|
|
||||||
reporting.
|
|
||||||
|
|
||||||
Search stable mailing list
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Check the archives of the Linux stable mailing list for existing reports.*
|
|
||||||
|
|
||||||
Maybe the issue you face is already known and was fixed or is about to. Hence,
|
|
||||||
`search the archives of the Linux stable mailing list
|
|
||||||
<https://lore.kernel.org/stable/>`_ for reports about an issue like yours. If
|
|
||||||
you find any matches, consider joining the discussion, unless the fix is
|
|
||||||
already finished and scheduled to get applied soon.
|
|
||||||
|
|
||||||
Reproduce issue with the newest release
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Install the latest release from the particular version line as a vanilla
|
|
||||||
kernel. Ensure this kernel is not tainted and still shows the problem, as
|
|
||||||
the issue might have already been fixed there.*
|
|
||||||
|
|
||||||
Before investing any more time in this process you want to check if the issue
|
|
||||||
was already fixed in the latest release of version line you're interested in.
|
|
||||||
This kernel needs to be vanilla and shouldn't be tainted before the issue
|
|
||||||
happens, as detailed outlined already above in the process of testing mainline.
|
|
||||||
|
|
||||||
Check code history and search for existing discussions
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Search the Linux kernel version control system for the change that fixed
|
|
||||||
the issue in mainline, as its commit message might tell you if the fix is
|
|
||||||
scheduled for backporting already. If you don't find anything that way,
|
|
||||||
search the appropriate mailing lists for posts that discuss such an issue
|
|
||||||
or peer-review possible fixes; then check the discussions if the fix was
|
|
||||||
deemed unsuitable for backporting. If backporting was not considered at
|
|
||||||
all, join the newest discussion, asking if it's in the cards.*
|
|
||||||
|
|
||||||
In a lot of cases the issue you deal with will have happened with mainline, but
|
|
||||||
got fixed there. The commit that fixed it would need to get backported as well
|
|
||||||
to get the issue solved. That's why you want to search for it or any
|
|
||||||
discussions abound it.
|
|
||||||
|
|
||||||
* First try to find the fix in the Git repository that holds the Linux kernel
|
|
||||||
sources. You can do this with the web interfaces `on kernel.org
|
|
||||||
<https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/>`_
|
|
||||||
or its mirror `on GitHub <https://github.com/torvalds/linux>`_; if you have
|
|
||||||
a local clone you alternatively can search on the command line with ``git
|
|
||||||
log --grep=<pattern>``.
|
|
||||||
|
|
||||||
If you find the fix, look if the commit message near the end contains a
|
|
||||||
'stable tag' that looks like this:
|
|
||||||
|
|
||||||
Cc: <stable@vger.kernel.org> # 5.4+
|
|
||||||
|
|
||||||
If that's case the developer marked the fix safe for backporting to version
|
|
||||||
line 5.4 and later. Most of the time it's getting applied there within two
|
|
||||||
weeks, but sometimes it takes a bit longer.
|
|
||||||
|
|
||||||
* If the commit doesn't tell you anything or if you can't find the fix, look
|
|
||||||
again for discussions about the issue. Search the net with your favorite
|
|
||||||
internet search engine as well as the archives for the `Linux kernel
|
|
||||||
developers mailing list <https://lore.kernel.org/lkml/>`_. Also read the
|
|
||||||
section `Locate kernel area that causes the issue` above and follow the
|
|
||||||
instructions to find the subsystem in question: its bug tracker or mailing
|
|
||||||
list archive might have the answer you are looking for.
|
|
||||||
|
|
||||||
* If you see a proposed fix, search for it in the version control system as
|
|
||||||
outlined above, as the commit might tell you if a backport can be expected.
|
|
||||||
|
|
||||||
* Check the discussions for any indicators the fix might be too risky to get
|
|
||||||
backported to the version line you care about. If that's the case you have
|
|
||||||
to live with the issue or switch to the kernel version line where the fix
|
|
||||||
got applied.
|
|
||||||
|
|
||||||
* If the fix doesn't contain a stable tag and backporting was not discussed,
|
|
||||||
join the discussion: mention the version where you face the issue and that
|
|
||||||
you would like to see it fixed, if suitable.
|
|
||||||
|
|
||||||
Check if it's a regression specific to stable or longterm kernels
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
*Check if you're dealing with a regression that was never present in
|
|
||||||
mainline by installing the first release of the version line you care
|
|
||||||
about. If the issue doesn't show up with it, you basically need to report
|
|
||||||
the issue with this version like you would report a problem with mainline
|
|
||||||
(see above). This ideally includes a bisection followed by a search for
|
|
||||||
existing reports on the net; with the help of the subject and the two
|
|
||||||
relevant commit-ids. If that doesn't turn up anything, write the report; CC
|
|
||||||
or forward the report to the stable maintainers, the stable mailing list,
|
|
||||||
and those who authored the change. Include the shortened commit-id if you
|
|
||||||
found the change that causes it.*
|
|
||||||
|
|
||||||
Sometimes you won't find anything in the previous step: the issue you face
|
|
||||||
might have never occurred in mainline, as it is caused by some change that is
|
|
||||||
incomplete or not correctly applied. To check this, install the first release
|
|
||||||
from version line you care about, e.g., if you care about 5.4.x, install 5.4.
|
|
||||||
|
|
||||||
If the issue doesn't show itself there, it's a regression specific to the
|
|
||||||
particular version line. In that case you need to report it like an issue
|
|
||||||
happening in mainline, like the last few steps in the main section in the above
|
|
||||||
outline.
|
|
||||||
|
|
||||||
One of them suggests doing a bisection, which you are strongly advised to do in
|
|
||||||
this case. After finding the culprit, search the net for existing reports
|
|
||||||
again: not only search for the exact subject and the commit-id (proper and
|
|
||||||
shortened to twelve characters) of the change, but also for the commit-id
|
|
||||||
(proper and shortened) mentioned as 'Upstream commit' in the commit message.
|
|
||||||
|
|
||||||
Write the report; just keep a few specialties in mind: CC or forward the report
|
|
||||||
to the stable maintainers, the stable mailing list, which the :ref:`MAINTAINERS
|
|
||||||
<maintainers>` file mentions in the section "STABLE BRANCH". If you performed a
|
|
||||||
successful bisection, CC the author of the change and include its subject and
|
|
||||||
the shortened commit-id.
|
|
||||||
|
|
||||||
Ask for advice
|
Ask for advice
|
||||||
~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~
|
||||||
|
|
Loading…
Reference in New Issue