180 lines
8.2 KiB
TeX
180 lines
8.2 KiB
TeX
\section{Research Design}
|
|
\label{sec:approach}
|
|
|
|
|
|
In this section,
|
|
we present our research questions and dataset.
|
|
|
|
\subsection{Research questions}
|
|
|
|
In our investigation of community profile documentation,
|
|
we focus on answering the following two research questions.
|
|
|
|
\vspace{1em}
|
|
\textbf{RQ1. Documentation prevalence.}
|
|
What is the prevalence of community profile documentation in GitHub? Moreover, how does the prevalence vary per programming language, repository owner type, repository age, and community size?
|
|
|
|
|
|
To answer this question,
|
|
we randomly sampled OSS project from GitHub, obtained their basic information, and checked out their latest snapshots.
|
|
For each kind of documentation
|
|
we looked for its existence in each project and computed its prevalence in all the projects.
|
|
Then, we grouped all the projects by programming language and repository owner, respectively,
|
|
and computed documentation prevalence in different groups.
|
|
Finally,
|
|
for each kind of documentation
|
|
we classified projects into two groups according to whether they include the documentation,
|
|
and compared the distribution of repository age and community size between two groups, respectively.
|
|
|
|
|
|
\vspace{1em}
|
|
\textbf{RQ2. Documentation maintenance.}
|
|
How is the community profile documentation maintained?
|
|
Specifically,
|
|
what are the location, creation, maintainers, update frequency and change-triggering events of community profile documentation?
|
|
|
|
To answer this question,
|
|
we first extracted the parent directory of community profile documentation, and computed the occurrence frequency of different locations for each kind of documentation.
|
|
Then,
|
|
we obtained all the historical changes of each document, and
|
|
extracted the author, time, and summary of each change,
|
|
which were used to determine the creation (the first change time), maintainers (all the change authors), update frequency (the averaged interval between each two adjacent changes) and change-triggering events (the keywords frequently mentioned in change summaries) of documentation.
|
|
|
|
|
|
\subsection{Dataset}
|
|
\label{dataset}
|
|
|
|
The dataset used in this paper includes 2,000 OSS projects hosted on GitHub.
|
|
The following sections present the details about how we obtained these projects and how we identified the community profile documentation and their change history from these projects.
|
|
|
|
\subsubsection{Studied projects}
|
|
We randomly selected 2,000 GitHub non-fork public repository using GitHub API (\url{https://api.github.com/search/repositories?q=[search\_criteria]}) which returns repositories based on the search criteria.
|
|
In our API calls,
|
|
the criteria comprises two parts:
|
|
|
|
\begin{itemize}
|
|
\item \textit{Project filter} (\texttt{forks>2}):
|
|
We studied on projects with more than 2 forks on GitHub.
|
|
In general, developers contribute to a GitHub project via the fork-pull model
|
|
(\ie developers first \textit{fork} the repository of a project, then make changes based on the forked repository, and finally send a pull-request to notify the project maintainers to \textit{pull} the submitted changes).
|
|
Therefore, the number of forks can be used as a measure of the community size of an OSS project,
|
|
\ie the more forks a project has,
|
|
the larger its community is.
|
|
We consider a project with more than 2 forks may have formed its own community.
|
|
|
|
|
|
\item \textit{Random effect} (\texttt{created:\%s..\%s}):
|
|
The API can only return 1,000 search results at most in total.
|
|
Consequently,
|
|
we cannot first get all the repositories with more than 2 forks and then randomly get a sample from them.
|
|
To tackle this issue,
|
|
we achieved the random effect on each individual API call.
|
|
Specifically,
|
|
each time we called the API,
|
|
we randomly generated a time window (an hour) using the \texttt{created} parameter and retrieved a repository created within the window.
|
|
|
|
\end{itemize}
|
|
|
|
|
|
From the results returned by the API,
|
|
we can parse the basic information of the project,
|
|
including \texttt{user}, \texttt{full\_name}, \texttt{language}, \texttt{description}, and \texttt{pushed\_at}.
|
|
With this information,
|
|
we further discarded repositories that are not for software development (identified by: \texttt{language} is empty) and repositories that only hosts a mirror on GitHub (identified by: \texttt{description} contains the keyword \textit{``mirror''}) and repositories that are already ``dead'' (there was no code update within the past three months; identified by examining \texttt{pushed\_at}).
|
|
|
|
At the examination time,
|
|
% (2020-07-18),
|
|
there are 1,801,066 repositories with more than 2 forks on GitHub.
|
|
Therefore,
|
|
our sample of 2,000 repositories results in
|
|
a margin error of 2.19\% at a confidence level of 95\%,
|
|
which is statistically representative.
|
|
Table~\ref{tab:sample_repos} lists the quantitative overview of our studied repositories.
|
|
|
|
% 1500-2.52%
|
|
% 1600-2.439%
|
|
% 1867-2.256% (forks>=10)
|
|
% 1000-3.091% (forks>=100)
|
|
|
|
|
|
|
|
\begin{table}[ht!]
|
|
\centering
|
|
\caption{Overview of the studied projects}
|
|
\begin{tabular}{r c c c ccc}
|
|
\bottomrule
|
|
\textbf{Metric} &\textbf{Min} &\textbf{25\%} &\textbf{Median} & \textbf{75\%} & \textbf{Max} & \textbf{Mean}\\
|
|
\midrule
|
|
\textbf{\#Forks} &3 &34 &101 &287 &69447 &385.38\\
|
|
\textbf{Age (month)} &0.24 &21.36 &49.05 &81.92 &151.33 &53.94\\
|
|
|
|
\toprule
|
|
\end{tabular}
|
|
\label{tab:sample_repos}
|
|
\end{table}
|
|
|
|
|
|
\subsubsection{Identification of profile documentation}
|
|
\label{iden}
|
|
|
|
We identify community profile documentation using heuristics based on
|
|
recommended practices by GitHub~\footnote{\url{https://docs.github.com/en/github/building-a-strong-community}}.
|
|
According to GitHub's recommendation,
|
|
developers can place the community profile documentation at the
|
|
\textit{root} directory, \textit{docs} directory, or the hidden \textit{.github} directory,
|
|
so that GitHub can automatically find and display the documentation.
|
|
Moreover,
|
|
each kind of documentation has a recommended specific name.
|
|
For example, a \RDM file should be named \textit{``readme''} (not case sensitive) usually with an extension of \textit{.md} or \textit{.txt}.
|
|
Therefore,
|
|
we identified community profile documentation
|
|
by examining names of the files located at the three directories.
|
|
|
|
|
|
First,
|
|
we obtained the most recent commit from each project via a GitHub API (\url{https://api.github.com/repos/[repository\_slug]/commits?page=1&per_page=1&order=desc}).
|
|
With the returned commit hash value (\ie \texttt{sha}),
|
|
we retrieved the latest snapshot of source code repository via another API (\url{https://api.github.com/repos/[repository\_slug]/git/trees/[sha]}).
|
|
From the returned results,
|
|
we can obtain all the files in the project.
|
|
Then,
|
|
we looked for the existence of each kind of documentation
|
|
by going through the three directories (\ie \textit{root}, \textit{docs}, and \textit{.github}) and checking the name of each file.
|
|
|
|
% checking whether the name a file contains the keyword correspond to a specific documentation,
|
|
% as shown in Table~\ref{tab:doc_name}.
|
|
|
|
|
|
% \begin{table}[ht!]
|
|
% \small
|
|
% \centering
|
|
% \caption{Overall percentage of repositories including each type of documentation}
|
|
% \begin{tabular}{@{}r c c ccc@{}}
|
|
% \bottomrule
|
|
% \textbf{Documentation} &\RDM & \LIC &\CON & \COD &\TMP\\
|
|
% \midrule
|
|
|
|
% \tabincell{c}{\textbf{Name keyword}} & \textit{readme} & \textit{license} & \textit{contributing} & \textit{code\_of\_conduct} & \tabincell{c}{\textit{issue\_template},\\ \textit{pull\_request\_template}} \\
|
|
|
|
% \toprule
|
|
% \end{tabular}
|
|
% \label{tab:doc_name}
|
|
% \end{table}
|
|
|
|
|
|
|
|
\subsubsection{Change history of profile documentation}
|
|
To obtain the change history of a specific file in a project,
|
|
a common way is to clone the remote repository into a local machine~\cite{2018Adding,Vendome2017License},
|
|
and then use \texttt{git} commands (\eg \texttt{git log [file\_name]} and \texttt{git show [commit\_sha]}) to obtain the list of historical changes (\ie code commits) on the file.
|
|
However,
|
|
it is a time-consuming and storage-intensive method.
|
|
Instead,
|
|
we developed a tool to extract the GitHub html pages displaying a file's change history.
|
|
The input of the tool is a URL (https://github.com/[repository\_slug]/co\-mmits/[branch]/[file\_name])
|
|
which varies by changing the repository name, default branch, and file name.
|
|
From the pages,
|
|
the tool identified a list of changes,
|
|
from each of which,
|
|
it extracted the change author, the change time,
|
|
and the change summary(\ie commit message). |