Skip to main content

Creating a patch for GNU GCC using Git

Overview and Target Audience

The GNU GCC project followed a blend of a traditional method with contemporary git tools when it comes to contributing code, making the experience unique from some other, git-based projects. This blog post will explore different aspects of the process, helpful commands, and various scripts that would make the experience more pleasent for new contributors. While this blog aims to help new contributors get acustomed to the GNU GCC code culture and make contributing easier, it must be stressed that this is not in any way in-depth exploration of the process. This should help you put your first foot forward; the community will help you take the rest of the steps from that point on. This post also assumes the user is in a POSIX environment (e.g. Linux, FreeBSD).

Git and GNU

As stated in this phoronix post, GNU GCC made a full transition to git early 2020. As of this writing, the community seems to be adjusting to the new tools. GNU GCC hosts its own git server as well as a Github mirror. As far as contributing code goes, making a PR is not the way to go; rather, the contributor is expected to make a patch file and submit it to their gcc-patches@gcc.gnu.org mailing list, where the patch will go through the process of being reviewed, fixed, and, hopefully, committed. Fortunately, git provides some useful tools for these purposes.

Useful Git commands

Some of the more famous git commands include: checkout, branch, commit, pull, push, clone, and merge. This post will not cover these commands as their detail instructions and examples are fairly easy to find. Instead, this post will cover some of the less-famous commands that are nonetheless very useful for projects such as GNU GCC.

format-patch

This git command can create an email-ready text file, complete with a from-address, subject line, a message, and a patch content. By default, it would go through each git commit and create a patch for each commit. Two of its noteable flags are:

  • -o: defines the destination of the output,
  • -M: defines the branch the current branch should compare to.
The command uses the commit title as the subject line, preceding it with [PATCH]. Different string may be used to preceed it if needed. The commit message body will be a part of the email body, along with the diff for patch.

send-email

This git command is useful for developers witihout a proper mailing client that would process the patch file. In particular, GMail is not very suitable for the process as it may handle its own formatting differently when content is pasted through the browser client. This command also allows easy way to script the process so that the patch can be sent with minimal user interaction. Some noteable flags are --to and --cc. They can be used multiple times to designate the receipient of the email.

NOTE: The --to and --cc flags also exits for the format-patch command and maybe used at the patch creation process if desired.

To use this feature with GMail, the account holder must supply additional information in the ~/.gitconfig file.

[user]
	email = your-email-id-here@gmail.com
	name = your-github-id-here

[sendemail]
	smtpEncryption = tls
	smtpServer = smtp.gmail.com
	smtpUser = your-email-id-here@gmail.com
	smtpServerPort = 587
	smtpPass = your-gmail-App-password-here

Due to GMail security policy, putting your regular GMail password would not work for smtpPass: this password must be explicitely generated. Please refer to the related document for more information.

Python

The GNU GCC project makes use of python in various tasks. In particular, the contrib/ folder contains a number of useful python scripts that helps with the patch submission process. The Python community suggests using a virtual environment for each Python project by using commands like pipenv. This will keep each Python project dependencies separate from each other while also keep a document of Python dependencies, called Pipfile, for consistant project behavior across different development environment.

Setting up Pipfile

The GNU GCC unfortunately does not come with its own Pipfile. It is also not in its .gitignore file, too. For now, to keep a copy for use with the local repository, add the following line to the .git/info/exclude file: 

Pipfile*
. This will ignore both the Pipfile and Pipfile.lock. Please also install pipenv using your distro's package management program. Then, go to the root of the project directory and run: 
$ pipenv install requests unidef termcolor

This should install three packages necessary to run the two very useful Python scripts in the contrib/ directory.

Now, from the root of the directory, you can run scripts like mklog.py like this: 

 $ pipenv run contrib/mklog.py path-to-patch-file-here

Some useful scripts

contrib/mklog.py: ChangeLog generator

Each patch must contain a passage called ChangeLog as a part of the body of the message, preceeding the patch information. While the exact formatting of the ChangeLog is beyond the scope of the blog, thankfully, the contrib/ provides a useful script, called mklog.py, that assists with the process. This script accepts an existing patch and outputs a formatted, skeletal body approprpriate for the patch. It usually require more input from the user to fill out the specific changes the patch introduces to the code base. Note that this introduces a catch-22 situation: the output of the script should go into the body of the patch, ideally when the patch is being made, but the script cannot run without an existing patch as an input. This issue will later be addressed with a script to streamline the process.

contrib/check_GNU_style.py: Styling checker

Like all well-maintained project, GNU GCC also has styling notes that the code should follow. Unfortunately, the project doesn't seem to come with tools like prettier that can be used to automate the proper formatting. They do have the next best thing: a format checker for an existing patch: contrib/check_GNU_style.py. It takes a patch as a required argument and outputs a report of styling suggestions to the standard out. It may optionally create a file of a condensed version of styling suggestion as needed. By using this script, the contributor may then manually make styling changes to their code.

Helpful Scripts

As programs and scripts that adheres to the Unix philosophy, many of the above can do one thing and one thing really well. Each of them are well suited as a single step among multiple steps necessary to achieve the goal. Fortunately, this makes them perfect component for scripts to streamline the process. Following scripts are proof-of-concept to handle a given situation. It works well for me; I hope it serves you well. Don't forget to customize it to fit your development environment.

For reference, the directory structure is as follows: 

gcc/          # GCC local project directory that holds all GCC-related folders.
  - gcc/      # GCC local git repo directory, i.e. GCCTOP.
  - patch/    # patch destination.
  - gcc-build # GCC build directory where that runs configuration and make.

gcccheckpatch.sh

This script is more for quality-of-life improvements and contains no real additional logic. Please note the environmental variable GCCTOP refers to the root of the gcc local repo.

gcccheckpatch.sh: 
#!/usr/bin/env sh

pipenv run $GCCTOP/contrib/check_GNU_style.py $@

gccmklog.sh

This script accepts one argument: the path to the patch. It will then generate the ChangeLog formatted message and insert it to the body of the patch. It uses /tmp to store intermediate files that should automatically deleted by the script after the execution. 

#!/usr/bin/env bash


TARGETPATCH=$1
TEMPLOGFILE=/tmp/changelog
TEMPPATCHFILE=/tmp/temp.patch
PATCHCUTOFFLINE=$(sed -n '/^---$/=' $TARGETPATCH)
CUTOFFTOPLINE=$(( PATCHCUTOFFLINE - 1))
DATE=$(date "+%Y-%m-%d")
NAME="YOUR-NAME HERE"
EMAIL="your@email.here"

ChangeLogSetup(){
printf "" > $TEMPLOGFILE &&\
printf "" > $TEMPPATCHFILE &&\
return 0 || return 1
}

ChangeLogGen(){
  pipenv run $GCCTOP/contrib/mklog.py $TARGETPATCH > $TEMPLOGFILE &&\
return 0||return 2
}

ChangeLogPatchUpdate(){
sed -n "1,${CUTOFFTOPLINE} p" $TARGETPATCH >> $TEMPPATCHFILE &&\
echo "" >> $TEMPPATCHFILE &&\
echo "${DATE}  ${NAME}  <${EMAIL}>" >> $TEMPPATCHFILE &&\
echo "" >> $TEMPPATCHFILE &&\
cat $TEMPLOGFILE >> $TEMPPATCHFILE &&\
sed -n "${PATCHCUTOFFLINE},$ p" $TARGETPATCH >> $TEMPPATCHFILE &&\
return 0||return 3
}

ChangeLogPatchReplace(){
  mv $TARGETPATCH ${TARGETPATCH}.bk && cp $TEMPPATCHFILE $TARGETPATCH && return 0 || return 4
}

ChangeLogCleanup(){
/bin/rm $TEMPLOGFILE $TEMPPATCHFILE && return 0 || return 5
}

ChangeLog(){
ChangeLogSetup \
&& ChangeLogGen \
&& ChangeLogPatchUpdate \
&& ChangeLogPatchReplace \
&& ChangeLogCleanup \
&& $EDITOR $TARGETPATCH \
&& echo "Success: $TARGETPATCH" || echo "Something went wrong with $TARGETPATCH: exit code $?" >&2
}

main(){

ChangeLog

}

main

gccmksquashpatch.sh

This patch is a proof-of-concept script that aims to first create single patch for all changes made in the branch. This is akin to making a squash-merge on a PR. As such, it is suitable when the branch introduces small changes to the code base. It also assumes that gcc local repo is inside of a dedicated gcc folder that contains another folder, called "patch". As a note, as the GNU GCC project suggests making build directory outside of the local repo, having patch on the parent directory of local repo sounds reasonable. It outputs each patch into its own folder whose name is defined by the

date
command.

This branch accepts one argument for $TARGET branch. The patch will first generate a squash-merged branch called 

patch/feature-branch-name
by comparing the $BASE branch against the $TARGET branch. and outputs a patch to the patch folder. It can be modified to then send the patch over email using the other script. 

#!/usr/bin/env sh

BASE=base-branch-name
TARGET=$1
PATCHBRANCH=patch/feature-branch-name
DATE=$(date "+%Y%m%d-%H%M%S")
OUTPUT=$GCCTOP/../patch/$DATE

PatchGen(){
git checkout $BASE &&\
git branch -D $PATCHBRANCH &&\
git checkout -b $PATCHBRANCH &&\
git merge --squash $TARGET &&\
git commit &&\
git format-patch -o $HOME/project/gcc/patch/$DATE -M $BASE &&\
return 0 || echo "Something went wrong in PatchGen"
return 1
}

main(){

PatchGen \
&& $HOME/bin/gccmklog.sh $OUTPUT/*patch \ #since there's just one squashed, let's run mklog
#&& $HOME/bin/gsocsendemail.sh $OUTPUT/*patch #uncomment this line if this command would send the patch email

git checkout $TARGET
}

main

gccsendpatch.sh

Once the patch is created, this simple script will send it to the recipients of the patch. As the blog is about sending the patch to gcc-patches mailing list, the appropriate address is added to --cc flag. It also provides one last chance to look over the messages if needed.

This script accepts one argument, which is the path name to the patch.

Provided the ~/.gitconfig is set up properly, the patch should be received by the community in short order. 

#!/usr/bin/env sh

git send-email --to=first-recipient@email.address --to=second-recipient@email.address --cc=gcc-patches@gcc.gnu.org --confirm=always $@

Some useful information

Labels:

Comments

Post a Comment

Popular posts from this blog

Debugging with GCC: GIMPLE

GCC and GIMPLE One of the very first thing GCC asks the GSoC applicants to do, even before writing the application, is to try various different debugging techniques using GCC. I was personally familiar with the basic, compile-with-g-flag-and-use-gdb method. Turns out, there's more: GIMPLE. A Simple but Non-trivial Program Problem Description The instruction asks to compile a simple but non-trivial program with some flags that generates debugging information: -O3 -S -fdump-tree-all -fdump-ipa-all -fdump-rtl-all . Because I was keep reading on ways to debug GCC just prior to the statement, I immediately thought "GCC" and tried make -j8 CXXFLAGS="-O3 -S -fdump-tree-all -fdump-ipa-all -fdump-rtl-all" . This was a mistake: turns out, GCC can't be compiled with those flags. Thankfully, GCC developers have a very active IRC channel for me to signal SOS. Resolution jakub and segher were quick to respond to my call for help. jakub: it isn't meant that y...
1 comment
Read more

Researching Awl Pike

Last time... In my previous commit, I had the pleasure of studying the guns and their ammo and how they are reflected in the world of Cataclysm-DDA. This time, the some members of the community were buzzing about a 3-meter-long pointy stick called awl pike. Awl Pike The awl pike is basically an extra-long spear designed specifically for use in a formation attack. A formation of 3+ ranks hold the longest possible stick at their disposal and poke the enemy that stands in front of them or charging at them. It is very powerful weapon against things that stands in front of them, but due to the need for formation and the shear unwieldiness of the weapon, it is not very versatile and adaptable to changing circumstances. In a one-on-one combat, the weapon would be very difficult to stay effective. How do I know all this? Well, there was the entire discussion about it in the issue page. Like I said, this kind of games brings out the geeks from their closets and provide the floor to sp...
Post a Comment
Read more