The popgenInfo project was designed to be an educational tool that can move at the same pace as method development by allowing people skilled in population genetics analysis to submit workflows that demonstrate the strengths of any given analysis in R. We use git and GitHub to accept contributions via pull requests using GitHub flow.
What is git?
Git is a version control program. It allows you to keep track of all changes that happen when you are writing, analyzing data, or coding. You can think of it as a sort of electronic lab notebook. Just like you would record any weights or measures carefully in your bench lab notebook, so should you record any analyses you do on the computer. With git, you “add” and “commit” changes that you make. These are analogous to writing down the values of your measurements and noting how you took them.
The basic steps of the process are:
* These should only be done once.
^ Start here if you already have cloned your fork.
This particular tutorial will show you the basics of contributing to the popgenInfo website using the R package git2r, which was developed by the rOpenSci project. This package allows for a nearly seamless and uniform interface for all contributors regardless of operating system. A companion set of videos to go along with this tutorial can be found at https://www.youtube.com/playlist?list=PLSFzyC3wp8-doiOcjDLlryIojK29qX2NL.
We assume that you have already done a few things:
If you have also installed git, you can set up Rstudio to work with git to help you commit changes, pull, and push. For the purposes of this tutorial, these things are not necessary.
If you have already forked and cloned the repository, go to Step 1.
Go to https://github.com/NESCent/popgenInfo and click on the “Fork” button. This will create a copy of the NESCent popgenInfo repository to your account. After you’ve forked the repository, you never have to fork it again!
Now that you have your repository forked to your account, you will need to clone it to your computer. Just like forking, you only have to clone your fork to your computer once. It will live there until you decide to remove it. We will need two things to clone our repository:
forks in your Documents)First, we need to make sure that we are working in the folder where we want to set up our fork. In this tutorial, we will use setwd() to do this via the R console, but it is possible to do this via Rstudio.
If you are using Windows:
If you are using OSX:
Now, we create the directory and then move there:
We can use the function list.files() to see what’s inside the directory:
## character(0)Currently, there’s nothing here. We will use the clone() function from git2r to make a copy of our repository. If you type help("clone", "git2r") in your R console, you can see documentation about the function. We need two things from this function:
The URL for the git repository is simply just the URL for your fork with .git at the end of it. For mine, it’s https://github.com/zkamvar/popgenInfo.git. To keep things simple, we will name the folder we want to put the repository in “popgenInfo”.
library("git2r")
clone(url = "https://github.com/zkamvar/popgenInfo.git",
local_path = "popgenInfo")## cloning into '/tmp/RtmpdYbN3L/popgenInfo'...
## Receiving objects: 1% (25/2466), 9 kb
## Receiving objects: 11% (272/2466), 64 kb
## Receiving objects: 21% (518/2466), 104 kb
## Receiving objects: 31% (765/2466), 1984 kb
## Receiving objects: 41% (1012/2466), 7470 kb
## Receiving objects: 51% (1258/2466), 11709 kb
## Receiving objects: 61% (1505/2466), 11757 kb
## Receiving objects: 71% (1751/2466), 11789 kb
## Receiving objects: 81% (1998/2466), 11837 kb
## Receiving objects: 91% (2245/2466), 12039 kb
## Receiving objects: 100% (2466/2466), 12039 kb, done.## Local: master /tmp/RtmpdYbN3L/popgenInfo
## Remote: master @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [2cb0623] 2017-01-04: update DESCRIPTIONNow when we look at our directory, we can see that we have one folder called “popgenInfo”:
## [1] "popgenInfo"We can move into the folder with setwd() and look at all the files in there.
## [1] "build" "circle.yml"
## [3] "CONDUCT.md" "CONTRIBUTING_WITH_GIT2R.Rmd"
## [5] "CONTRIBUTING.md" "data"
## [7] "DATAFORMATS.md" "DESCRIPTION"
## [9] "develop" "developer_example.Rmd"
## [11] "GETSTARTED.md" "images"
## [13] "index.md" "install"
## [15] "LEARN.md" "LICENSE"
## [17] "Makefile" "PACKAGES.md"
## [19] "popgenInfo.Rproj" "R_MARKDOWN.md"
## [21] "README.md" "SNP.md"
## [23] "SSR.md" "TEMPLATE.Rmd"
## [25] "use" "USINGR.md"
## [27] "WORKFLOWS.md"Now we have successfully cloned our repository into our computer using git2r. Next, we will set up our credentials and keep our master branch up to date.
To access the repository, all you have to do is open it by double-clicking on the “popgenInfo.Rproj” file. This will open Rstudio to this folder and set it as the working directory. After that, in your R console, type:
## Local: master /tmp/RtmpdYbN3L/popgenInfo
## Remote: master @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [2cb0623] 2017-01-04: update DESCRIPTIONThis tells R that you have a repository in this folder and that you are referring to it as repo. The output of repo is a summary of your repository:
In order to be able to synch your clone with your fork, GitHub needs to know that you are really who you say you are. This means that you need to associate your name, email, and a secret token with this repository. The first two items are easily done within R. You can use the function config() to set and confirm your name and email:
config(repo,
user.name = "Zhian Kamvar",
user.email = "kamvarz@science.oregonstate.edu")
config(repo)## system:
## credential.helper=cache --timeout=3600
## push.default=simple
## local:
## branch.master.merge=refs/heads/master
## branch.master.remote=origin
## core.bare=false
## core.filemode=true
## core.logallrefupdates=true
## core.repositoryformatversion=0
## remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
## remote.origin.url=https://github.com/zkamvar/popgenInfo.git
## user.email=kamvarz@science.oregonstate.edu
## user.name=Zhian KamvarNext, we need to set our secret token. Known as a Personal Access Token (PAT), this is a 40 character cryptographic token that is like a long and complicated password. They both allow access to your GitHub repository, but the difference between a PAT and a password is that a PAT can be stored as text on your computer (which you should NEVER do with passwords) because it is easy to generate and (most importantly) easy to remove. We can create a PAT from GitHub and then we will store this in a file called .Renviron that is read every time you restart your R session, allowing you to have this token whenever you need to push changes to your fork on GitHub. The instructions to generate your PAT are here.
Here’s an example of what a .Renviron file with a PAT will look like on the inside:
GITHUB_PAT="97cc6bf86c31a42fca2de32884cd1f1c4b1102ba"Once you have created your PAT, make a text file called .Renviron inside of the repository and paste your PAT in that file as shown above. This will not be tracked by git, and will only exist on your computer.
The next time you open R from popgenInfo.Rproj, this token will be available for GitHub to ensure that you are who you say you are. It is important to note that, if you change or delete that PAT, you must replace it here.
In the next step, we will ensure that our fork is up to date with NESCent. In order to do that, we need to tell our local copy that it can take updates from NESCent. We do this by adding NESCent as a remote called ‘upstream’ using the function remote_add() and checking to make sure it worked with the function remotes().
remote_add(repo,
name = "upstream",
url = "https://github.com/NESCent/popgenInfo.git")
# Check our remotes
remotes(repo)## [1] "origin" "upstream"With all of this information set, we can begin working on our repository.
The first thing we need to do is make sure that our master branch is up to date. Since other people are also contributing to this project, it is possible for changes to be made even within seconds of you forking the repository. In this section we will show you how you can keep your master branch up to date in your local repository and your fork on github.
Let’s assume that you’ve just opened a fresh Rstudio session with the popgenInfo.Rproj file. First, let’s load git2r and access our repository.
Now that we have our repository loaded, we should make sure that we are on the master branch. We can do this using the checkout() function from git2r.
## Local: master /tmp/RtmpdYbN3L/popgenInfo
## Remote: master @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [2cb0623] 2017-01-04: update DESCRIPTIONNow, whether or not we suspect changes from NESCent, it’s always good form to update your local repository before creating any new branches. We can do it this by first fetching ALL changes that have happened on the NESCent popgenInfo repository, and then merging the NESCent master branch into our master branch. We will use the git2r functions fetch() and merge() to accomplish this.
First, we fetch all of the changes that have happened. If there are no changes, nothing will appear, but if there is new content on the master branch or new branches on the repository, they will be downloaded and each branch will be printed to your screen. For our purposes, we are only concerned with the changes on the master branch, which will look something like this: ## [new] b62a78c1fcd384a3ea23 refs/remotes/upstream/master.
## [new] 2bf7239c6367e2f80cb0 refs/remotes/upstream/add-weekly-build
## [new] e0b711a1189af7cc2b58 refs/remotes/upstream/gh-pages
## [new] b3aad3d976eac71bc4ad refs/remotes/upstream/master
## [new] baa1358367cedf7a7fda refs/remotes/upstream/site-images
## [new] 16b52608de7f5ed41d04 refs/remotes/upstream/summaryseqstats
## [new] 8840ff6e3782db926b03 refs/remotes/upstream/update-makefile-doc
## [new] c7e4171b3786e398e39b refs/remotes/upstream/web-devel
## [new] 04853f9353448f1b002c refs/remotes/upstream/zkamvar-patch-1
## [new] 021b4de353dff06f0c44 refs/tags/v1.0.0It’s important to note that our local repository is still unchanged at this point as we have just downloaded the updates from NESCent. To update our local repository, we should merge the changes from NESCent’s master branch into our master branch:
## Merge: Fast-forward## Local: master /tmp/RtmpdYbN3L/popgenInfo
## Remote: master @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [b3aad3d] 2019-09-16: Merge pull request #218 from NESCent/zkamvar-patch-1If there were any changes, the Head field will have a more current commit than when we intialized it above.
Since we will never work directly on the master branch, there should be no conflicts from the NESCent branch and we now know that our branch is up to date. Assuming everything worked, we can push this to our fork on GitHub. We will use git2r’s push() function to do this. We also need to supply our PAT in order for this to work. To do so, we will use the git2r function cred_token():
Now, when you check your fork, it should be updated with the most recent changes. From here, we can checkout a new branch and then proceed with adding our changes or new workflow.
When you add contributions, the best practice is to create a new branch off of the master. A branch is a term that indicates a sort of sandbox for a repository that can become permanent. By default, git repositories all have a “master” branch. When you want to experiment with something else, add new content, or simply just fix a typo, but you don’t want to disturb the original copy, you create a new branch. Good practice is to name the branch in a way that succinctly describes what you are doing with the branch. In this step, we will create a branch, push it to our fork, and track it.
Most importantly we need to make sure that we are on the master branch and it is up to date. To double plus make sure that you are on the master branch, check it out:
## Local: master /tmp/RtmpdYbN3L/popgenInfo
## Remote: master @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [b3aad3d] 2019-09-16: Merge pull request #218 from NESCent/zkamvar-patch-1For reasons that will reveal themselves later, please run these function definitions:
And, of course, don’t forget your credentials:
Assuming you’ve updated your master branch as shown above, we can create a branch from the master using the function checkout(). Let’s assume that we want to create a vignette for analyzing spatial statistics. I’m going to lay out a few steps here all at once because these are the steps you want to take when you create a brand new branch and make sure it exists on your computer and on your fork:
In the first step, we want to name the branch. Since we are creating a new vignette, it’s ideal for the branch name to be the same as the vignette. As shown in the Best Practices guidelines, you should name this vignette with the date and the subject. Since we are contributing a vignette on spatial stats and committing for the first time on 2015-12-16, the new branch and vignette will be called “2015-12-16-spatial- stats”.
In code, the steps above would look like this:
BRANCH <- "2015-12-16-spatial-stats" # 1
checkout(repo, BRANCH, create = TRUE) # 2
push(repo, name = "origin", refspec = my_branch(BRANCH), credentials = cred) # 3
branch_set_upstream(head(repo), my_origin(BRANCH)) # 4Let’s examine these one by one. First, you create a variable that will hold the name of the branch you want to create. We are using this convention to allow us to use this name multiple times. After that, we create that branch from our master branch. We can take a look at what our branch looks like at that point.
## Local: 2015-12-16-spatial-stats /tmp/RtmpdYbN3L/popgenInfo
## Head: [b3aad3d] 2019-09-16: Merge pull request #218 from NESCent/zkamvar-patch-1## Warning: 'head.git_repository' is deprecated.
## Use 'repository_head' instead.
## See help("Deprecated")Notice here that you now only have two lines in the output, Local and Head. This is because the branch doesn’t exist on your GitHub fork. This is where the next two steps come in. We will push that branch and then make sure that our branch is tracking the copy on the fork.
push(repo, name = "origin", refspec = my_branch(BRANCH), credentials = cred) # 3
branch_set_upstream(head(repo), my_origin(BRANCH)) # 4
repo## Local: 2015-12-16-spatial-stats /tmp/RtmpdYbN3L/popgenInfo
## Remote: 2015-12-16-spatial-stats @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [b3aad3d] 2019-09-16: Merge pull request #218 from NESCent/zkamvar-patch-1We can now see from the output that we have a Remote set up. Once we have this, we can start making changes! If you are adding a new vignette, please copy TEMPLATE.Rmd to the use/ directory and give it a new name that is meaningful to your contribution (good practice is to name it the same as your branch). If you are including any data, place it in the data/ directory. Once you add these files, you need to to add these files and then you should commit, which we will show you how to do below.
If you are using Rstudio, you can use it to integrate with git. This allows you to do things like commit, push, and pull. Hadley Wickham, chief scientist at Rstudio has put up this helpful tutorial on using git through Rstudio. If you aren’t using Rstudio, but you don’t want to access the terminal (command line), this section will show you basic git commands with git2r that will allow you to work on your vignette and keep it up to date.
Recall that using git is analogous to keeping a good, detailed lab notebook. When you make any changes, you record those changes (git add) and then say why you made the changes (git commit).
Let’s say you’ve already copied the TEMPLATE.Rmd file to the use/ directory and renamed it to 2015-12-16-spatial-stats.Rmd and added some data called spatial_data.csv. When you place a file in your repository, git will not pay attention to any changes you do until you specifically add it. Until then it is “Untracked”. You can see this by running status():
## Untracked files:
## Untracked: data/spatial_data.csv
## Untracked: use/2015-12-16-spatial-stats.RmdSince we’ve just added the files, we can use add() to tell git we want to stage them for committing.
## Staged changes:
## New: data/spatial_data.csv
## New: use/2015-12-16-spatial-stats.RmdWhen we have done this, we can commit to our changes by using commit(). A commit is basically placing a record of what you did and, importantly, why you did it. Your commit message should record this. Commit messages should be able to fit on a single line. Optionally, if you want to be able to add more detail, you can enter more lines below the message:
msg <- " started spatial stats vignette and added data
I copied the template to the use folder and placed
spatial data in the data folder.
"
commit(repo, msg, session = TRUE)## [14881ea] 2019-09-16: started spatial stats vignette and added data## working directory clean## Local: 2015-12-16-spatial-stats /tmp/RtmpdYbN3L/popgenInfo
## Remote: 2015-12-16-spatial-stats @ origin (https://github.com/zkamvar/popgenInfo.git)
## Head: [14881ea] 2019-09-16: started spatial stats vignette and added dataNote that we put
session = TRUEin the commit function. This automatically puts your R session information in the commit, which gives more information about package versions you were working with when you were editing the vignette.
You can see a summary of what you just did by looking at the summary of your commit:
## Commit: 14881ea668e07d9483287feb9ad2de4851136e20
## Author: Zhian Kamvar <kamvarz@science.oregonstate.edu>
## When: 2019-09-16 00:36:03
##
## started spatial stats vignette and added data
##
## I copied the template to the use folder and placed
## spatial data in the data folder.
##
##
## sessionInfo:
## R version 3.6.1 (2019-07-05)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Debian GNU/Linux 9 (stretch)
##
## Matrix products: default
## BLAS/LAPACK: /usr/lib/libopenblasp-r0.2.19.so
##
## locale:
## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
## [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
## [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] git2r_0.26.1
##
## loaded via a namespace (and not attached):
## [1] compiler_3.6.1 magrittr_1.5 tools_3.6.1 htmltools_0.3.6
## [5] yaml_2.2.0 Rcpp_1.0.2 stringi_1.4.3 rmarkdown_1.15
## [9] knitr_1.24 stringr_1.4.0 xfun_0.9 digest_0.6.20
## [13] evaluate_0.14
## 2 files changed, 127 insertions, 0 deletions
## data/spatial_data.csv | -0 + 6 in 1 hunk
## use/2015-12-16-spatial-stats.Rmd | -0 +121 in 1 hunkNow you can see that your commit message was recorded in the repository. Since we see that there is a Remote associated with this fork, we can simply use push() to push our changes up to the fork.
Now that you’ve successfully set up git, created a new branch, and created a new vignette, it’s time to fill it with content. You can work in your vignette like you would write in any other Rmarkdown document and keep track of your changes like we showed above by adding and committing your changes followed by pushing these up to your fork. Once you are finished with all of your changes, you can create a pull request from your GitHub fork to NESCent.
You can tell R to open a browser for you to create your pull request. The URL for this is in the form of https://github.com/USER/popgenInfo/pull/new/BRANCH. We can create this URL by using the function paste() to glue the pieces together and then use the utils::browseURL() function to open up a window. Note we are using the variable called BRANCH that we named above.
## [1] "https://github.com/zkamvar"## [1] "https://github.com/zkamvar/popgenInfo/pull/new/2015-12-16-spatial-stats"Now that you have the URL, you can browse to it.
If you don’t want to do this from R, you can do it through your web browser.
Your pull request begins a process of open peer review where content, accuracy, and style are considered. If changes are suggested, you should revise them by making changes on your fork and repeating the process in Step 3. Your pull request will be automatically updated once you push the changes to your fork.
Finally, when all questions and concerns have been addressed, the pull request may be merged into the NESCent repository as long as two of the maintainers have signaled their approval.
From this tutorial, you should have learned how to:
The skills demonstrated in this tutorial are not exclusive for popgenInfo, but they can be used when writing up your own reports or analyses. Using git may not feel comfortable at this moment, but just like it takes practice to learn how to keep a lab notebook, with practice, you will become comfortable using git to keep track of your workflows, making your science more reproducible.
Once your pull request passes peer review and is published, you can update your fork by starting from Step 1: checkout your master branch, fetch the changes from NESCent, merge those changes, and push them up to the master branch on your fork to make it even with NESCent.
This shows us useful information for reproducibility. Of particular importance are the versions of R and the packages used to create this workflow. It is considered good practice to record this information with every analysis.
## ─ Session info ───────────────────────────────────────────────────────────────────────────────────
## setting value
## version R version 3.6.1 (2019-07-05)
## os Debian GNU/Linux 9 (stretch)
## system x86_64, linux-gnu
## ui X11
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz Etc/UTC
## date 2019-09-16
##
## ─ Packages ───────────────────────────────────────────────────────────────────────────────────────
## package * version date lib source
## assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.1)
## backports 1.1.4 2019-04-10 [1] CRAN (R 3.6.1)
## callr 3.3.1 2019-07-18 [1] CRAN (R 3.6.1)
## cli 1.1.0 2019-03-19 [1] CRAN (R 3.6.1)
## crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.1)
## desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.1)
## devtools 2.2.0 2019-09-07 [1] CRAN (R 3.6.1)
## digest 0.6.20 2019-07-04 [1] CRAN (R 3.6.1)
## DT 0.8 2019-08-07 [1] CRAN (R 3.6.1)
## ellipsis 0.2.0.1 2019-07-02 [1] CRAN (R 3.6.1)
## evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.1)
## fs 1.3.1 2019-05-06 [1] CRAN (R 3.6.1)
## git2r * 0.26.1 2019-06-29 [1] CRAN (R 3.6.1)
## glue 1.3.1 2019-03-12 [1] CRAN (R 3.6.1)
## htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.6.1)
## htmlwidgets 1.3 2018-09-30 [1] CRAN (R 3.6.1)
## knitr 1.24 2019-08-08 [1] CRAN (R 3.6.1)
## magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.1)
## memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.1)
## pkgbuild 1.0.5 2019-08-26 [1] CRAN (R 3.6.1)
## pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.1)
## prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.6.1)
## processx 3.4.1 2019-07-18 [1] CRAN (R 3.6.1)
## ps 1.3.0 2018-12-21 [1] CRAN (R 3.6.1)
## R6 2.4.0 2019-02-14 [1] CRAN (R 3.6.1)
## Rcpp 1.0.2 2019-07-25 [1] CRAN (R 3.6.1)
## remotes 2.1.0 2019-06-24 [1] CRAN (R 3.6.1)
## rlang 0.4.0 2019-06-25 [1] CRAN (R 3.6.1)
## rmarkdown 1.15 2019-08-21 [1] CRAN (R 3.6.1)
## rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.1)
## sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.1)
## stringi 1.4.3 2019-03-12 [1] CRAN (R 3.6.1)
## stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.1)
## testthat 2.2.1 2019-07-25 [1] CRAN (R 3.6.1)
## usethis 1.5.1 2019-07-04 [1] CRAN (R 3.6.1)
## withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.1)
## xfun 0.9 2019-08-21 [1] CRAN (R 3.6.1)
## yaml 2.2.0 2018-07-25 [1] CRAN (R 3.6.1)
##
## [1] /usr/local/lib/R/site-library
## [2] /usr/local/lib/R/library