linux-sg2042/scripts/checkkconfigsymbols.py

140 lines
4.6 KiB
Python
Raw Normal View History

checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
#!/usr/bin/env python
"""Find Kconfig identifiers that are referenced but not defined."""
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
# (c) 2014 Valentin Rothberg <valentinrothberg@gmail.com>
# (c) 2014 Stefan Hengelein <stefan.hengelein@fau.de>
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
#
# Licensed under the terms of the GNU GPL License version 2
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
import os
import re
from subprocess import Popen, PIPE, STDOUT
# regex expressions
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
OPERATORS = r"&|\(|\)|\||\!"
FEATURE = r"(?:\w*[A-Z0-9]\w*){2,}"
DEF = r"^\s*(?:menu){,1}config\s+(" + FEATURE + r")\s*"
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
EXPR = r"(?:" + OPERATORS + r"|\s|" + FEATURE + r")+"
STMT = r"^\s*(?:if|select|depends\s+on)\s+" + EXPR
SOURCE_FEATURE = r"(?:\W|\b)+[D]{,1}CONFIG_(" + FEATURE + r")"
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
# regex objects
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
REGEX_FILE_KCONFIG = re.compile(r".*Kconfig[\.\w+\-]*$")
REGEX_FEATURE = re.compile(r"(" + FEATURE + r")")
REGEX_SOURCE_FEATURE = re.compile(SOURCE_FEATURE)
REGEX_KCONFIG_DEF = re.compile(DEF)
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
REGEX_KCONFIG_EXPR = re.compile(EXPR)
REGEX_KCONFIG_STMT = re.compile(STMT)
REGEX_KCONFIG_HELP = re.compile(r"^\s+(help|---help---)\s*$")
REGEX_FILTER_FEATURES = re.compile(r"[A-Za-z0-9]$")
def main():
"""Main function of this module."""
source_files = []
kconfig_files = []
defined_features = set()
referenced_features = dict() # {feature: [files]}
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
# use 'git ls-files' to get the worklist
pop = Popen("git ls-files", stdout=PIPE, stderr=STDOUT, shell=True)
(stdout, _) = pop.communicate() # wait until finished
if len(stdout) > 0 and stdout[-1] == "\n":
stdout = stdout[:-1]
for gitfile in stdout.rsplit("\n"):
if ".git" in gitfile or "ChangeLog" in gitfile or \
".log" in gitfile or os.path.isdir(gitfile):
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
continue
if REGEX_FILE_KCONFIG.match(gitfile):
kconfig_files.append(gitfile)
else:
# all non-Kconfig files are checked for consistency
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
source_files.append(gitfile)
for sfile in source_files:
parse_source_file(sfile, referenced_features)
for kfile in kconfig_files:
parse_kconfig_file(kfile, defined_features, referenced_features)
print "Undefined symbol used\tFile list"
for feature in sorted(referenced_features):
# filter some false positives
if feature == "FOO" or feature == "BAR" or \
feature == "FOO_BAR" or feature == "XXX":
continue
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
if feature not in defined_features:
if feature.endswith("_MODULE"):
# avoid false positives for kernel modules
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
if feature[:-len("_MODULE")] in defined_features:
continue
files = referenced_features.get(feature)
print "%s\t%s" % (feature, ", ".join(files))
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
def parse_source_file(sfile, referenced_features):
"""Parse @sfile for referenced Kconfig features."""
lines = []
with open(sfile, "r") as stream:
lines = stream.readlines()
for line in lines:
if not "CONFIG_" in line:
continue
features = REGEX_SOURCE_FEATURE.findall(line)
for feature in features:
if not REGEX_FILTER_FEATURES.search(feature):
continue
sfiles = referenced_features.get(feature, set())
sfiles.add(sfile)
referenced_features[feature] = sfiles
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
def get_features_in_line(line):
"""Return mentioned Kconfig features in @line."""
return REGEX_FEATURE.findall(line)
def parse_kconfig_file(kfile, defined_features, referenced_features):
"""Parse @kfile and update feature definitions and references."""
lines = []
skip = False
with open(kfile, "r") as stream:
lines = stream.readlines()
for i in range(len(lines)):
line = lines[i]
line = line.strip('\n')
line = line.split("#")[0] # ignore comments
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
if REGEX_KCONFIG_DEF.match(line):
feature_def = REGEX_KCONFIG_DEF.findall(line)
defined_features.add(feature_def[0])
skip = False
elif REGEX_KCONFIG_HELP.match(line):
skip = True
elif skip:
# ignore content of help messages
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
pass
elif REGEX_KCONFIG_STMT.match(line):
features = get_features_in_line(line)
# multi-line statements
checkkconfigsymbols.sh: reimplementation in python The scripts/checkkconfigsymbols.sh script searches Kconfig features in the source code that are not defined in Kconfig. Such identifiers always evaluate to false and are the source of various kinds of bugs. However, the shell script is slow and it does not detect such broken references in Kbuild and Kconfig files (e.g., ``depends on UNDEFINED´´). Furthermore, it generates false positives. The script is also hard to read and understand, and is thereby difficult to maintain. This patch replaces the shell script with an implementation in Python, which: (a) detects the same bugs, but does not report previous false positives (b) additionally detects broken references in Kconfig and all non-Kconfig files, such as Kbuild, .[cSh], .txt, .sh, defconfig, etc. (c) is up to 75 times faster than the shell script (d) only checks files under version control The new script reduces the runtime on my machine (i7-2620M, 8GB RAM, SSD) from 3m47s to 0m3s, and reports 938 broken references in Linux v3.17-rc1; 419 additional reports of which 16 are located in Kconfig files, 287 in defconfigs, 63 in ./Documentation, 1 in Kbuild. Moreover, we intentionally include references in comments, which have been ignored until now. Such comments may be leftovers of features that have been removed or renamed in Kconfig (e.g., ``#endif /* CONFIG_MPC52xx */´´). These references can be misleading and should be removed or replaced. Note that the output format changed from (file list <tab> feature) to (feature <tab> file list) as it simplifies the detection of the Kconfig feature for long file lists. Signed-off-by: Valentin Rothberg <valentinrothberg@gmail.com> Signed-off-by: Stefan Hengelein <stefan.hengelein@fau.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2014-09-27 22:30:45 +08:00
while line.endswith("\\"):
i += 1
line = lines[i]
line = line.strip('\n')
features.extend(get_features_in_line(line))
for feature in set(features):
paths = referenced_features.get(feature, set())
paths.add(kfile)
referenced_features[feature] = paths
if __name__ == "__main__":
main()