Trac and Git: The Right Way
This post is about configuring Trac and Git to work together properly. As you might be knowing, Git is a DVCS – Distributed Version Control System and Trac is a project management system. By default, Trac supports SVN, but there are plugins for Git and Mercurial. I don’t know if there are plugins for other repositories like Bazaar, etc.
In the default configuration in Trac’s Git plugin, the repository is not cached and setting repository_sync_per_request to blank doesn’t stop Trac from syncing repositories on each request. This is a big performance disadvantage and can big a real big trouble if you have lots of repositories with quite a lot of commits.
The solution is simple, you have to enable caching in Trac’s configuration. But that creates another problem: updating Trac’s cache when new commits are pushed to the repository. While the official plugin page has scripts, but some comments (at the time of writing this) said that they don’t work, so I devised my own way.
To enable repository caching in Trac:
cached_repository = true
Now, you have to setup a post-recieve hook on the remote repository. I was using gitolite, so the repositories are stored as /var/lib/gitolite/repositories/{NAME}.git and hooks for the repository are stored in /var/lib/gitolite/repositories/{NAME}.git/hooks.
In the hooks directory, create a new file (as the git user or whatever user gitolite is configured with) with following contents (see below for explanation):
#!/bin/bash
# Script written by NileshGR; http://nileshgr.com
# This script is under BSD License
tracenv=/var/lib/trac/foo # Full path to trac environment
tracadmin="trac-admin" # Path to trac-admin
reponame="(default)" # Repository name
while read -s line
do
[ -z "$line" ] && break
commithash=$(echo $line | cut -d' ' -f2)
$tracadmin "$tracenv" changeset added "$reponame" $commithash || $tracadmin "$tracenv" repository resync "$reponame"
done
What the script does:
Git provides information about new commits to the post-receive hook via stdin in the following format (per line, there can be multiple lines):
<old value> <new value> <ref name>
We read stdin till the line is blank and extract the new commit hash using cut at each iteration then call trac-admin for every changeset.
Now, you might wonder why the other part for repository resync is there. When a commit is deleted, trac fails with a no such changeset error. The or condition handles that case, it calls for a full resync when such an error occurs.
There might be some way to inform trac about deleted commits, but I don’t know. I could come up only with this. If you have something else, do comment.
NileshGR 
Nilesh, this is great stuff!
You’re saying that you are using gitolite. What about multiple repositories for one trac environment? What about multiple trac instances with one gitolite?
How would it be possible to track commits from all branches in a trac environment?
e.g. like this script: https://github.com/arruda/TRAC-GIT-HOOKS/blob/master/trac-post-receive-hook-0.12-new-commits-from-all-branches-with-logfile.py
BTW, I updated your hook to serve my needs, but I can’t figure out how to skip over gitolite-admin commits… See: http://pastie.org/4756702
I also found this script here, which allows ticket updates: https://github.com/JensTimmerman/TRAC-SVN-to-GIT-migration/blob/master/hooks/trac-post-receive-hook.py
LikeLike
Glad that you liked it 🙂
In case of multiple trac instances or multiple git repositories for one trac, I just add the hook to every repository I want and modify the variables $tracenv & $reponame.
It might be even possible to map $tracenv dynamically using current directory of the script.
I saw your modified script, you are calling changeset added and resync both — overkill. Don’t do that.
As I mentioned in my post, the resync is required only when you delete commits and do a git push -f, otherwise changeset added works fine.
When there’s a deleted commit, changeset added exits with an error, and I’ve used the same property in my hook by executing resync when there’s an error.
About tracking all branches in one trac environment, I think the existing hook suffices. post-receive hook is called for every commit add to the repository as a whole, and not for a particular branch. Trac is smart enough to map the commits into their respective branches.
LikeLike
Hey Nilesh,
Thanks for your reply! : )
I modified the script to support multiple environments with multiple repositories each. (Unfortunately not cross-referencing to different environments from one, or more git-repos.) http://pastie.org/4782090
It takes the name (i.e. GL_REPO) of the git-repo you’re pushing to, splits it according to a delimeter (e.g. a dash), then uses the first part to fill up the last bit of the tracenv path.
It then uses the rest of the string as the repo-name (the name that you put in trac, when adding a repo). Voila! : )
It’s also smart enough to put “(default)” as the reponame, if there’s nothing after the delimeter for the standard case of one repo with the name “(default)”.
It would be great to integrate ticket updates and other operations, like here: https://github.com/JensTimmerman/TRAC-SVN-to-GIT-migration/blob/master/hooks/trac-post-receive-hook.py
Maybe we can just replace trac-admin with that script and some small adjustments? Would you like to have a look at this?
The svn-hook of trac also got an update recently. It auto-detects trac-environments without resorting to naming conventions.
http://trac.edgewall.org/attachment/ticket/10847/trac-svn-hook.4.patch
On a side-note I’d like to suggest a license-change to BSD, as at the moment it would be illegal to distribute this hook with trac, AFAIK.
LikeLike
Great! Nice idea to use GL_REPO.
Regarding ticket integration, replacing trac-admin with our own script would be probably a very tedious task because trac-admin does many more things than just syncing repos 😛
I have no idea about Trac plugin development, but since Trac provides plugin hooks it would be more sensible to write a plugin instead of replacing the main script.
I would like to check that integration, but when I wrote this I was just trying out Trac and due lack of time to debug & study APIs, I switched Redmine. There’s a lot of flame-war associated with Trac/Redmine, but I like both: Trac because it’s Python and Redmine because, well, it’s simple for the “dumb user”.
If you start with the plugin development, do send me a ping sometime. I’m there on GH and BitBucket both (nileshgr).
Oh btw, license changed to BSD; thanks for the tip 🙂
For easier tracking, I’ve created a gist out of your script: https://gist.github.com/3768589 🙂
Post updates there now.
LikeLike
Hey again! : )
Thanks for changing the license and making the gist. I’ll post future code-updates there. (Could you maybe update the gist to reflect the license change?)
I too, tried redmine (actually, chiliproject, a better fork, but there’s a lot of tension between those two projects as well), but due to inferior Eclipse Mylyn integration (i.e. it just opens a browser inside eclipse instead of using the actual Mylyn rich client), I switched to trac.
Actually, at this point, I believe trac is architecturally superior to redmine & chili (in the sense of cleaner to integrate), albeit slightly more difficult to set up. Once you figure out what to put in your master config, “it just works”.
The big plus for me is the actually distinct environments, with the possibility of including a separate set of plugins for each one.
Before, I really fancied redmine/chili’s slick(er) interface, but
in effect, trac is just the database and the UI for the non-technical managers. The dev’s should be able to handle everything from inside eclipse or through git commits (that’s why I’d also like update, create, etc., but close is a good start ; ) ). Thus, the priority is on integration with the rest of the toolchain, and this is where trac wins, IMHO. Also, trac is BSD licensed, and I prefer a Free brewery vs. a free beer. : )
It seems like you did misunderstand me a little, though. I wouldn’t be as crazy as to suggest replacing trac-admin in this context, just to replace the call to it from our hook with a call to e.g. Jens Timmerman’s hook (https://github.com/JensTimmerman/TRAC-SVN-to-GIT-migration/blob/master/hooks/trac-post-receive-hook.py).
His hook supports commit messages like:
“Changed blah and foo to do this or that. Fixes #10 and #12, and refs #12. worked 10.2h” –
closing tickets 10 & 12, referencing #12 (that should work in our hook as well), but also including the time that one has worked on the tickets mentioned in the commit message.
It would also be reasonably easy to add additional commands to that boilerplate.
So, we’re not talking plugin development here, I hope, as I’d also have to pass in that case, due to lack of free (as in beer, in this case) time : )
LikeLike
Updated the gist to reflect the license change. Instead of using the gist, can you fork this repository of mine: https://github.com/nileshgr/utilities
Add the script there, and then send me a pull request when you’re done. Would be much better, no?
Right so, it’s not plugin development.
I’m pretty sure if I go through the svn-git hook and trac’s db structure, I’ll be able to figure out what it is exactly does and how to write one on my own to implement our needs, but as I said earlier, time constraints.
LikeLike
Right, let’s hope someone with more time joins hands : )
Meanwhile, I took the liberty to shamelessly plug the hook here: http://trac.edgewall.org/ticket/10730#comment:5, but I’ll be sure to keep you updated, if I get around to doing something myself.
Not sure about forking all of your utils, though ; )
LikeLike
LOL, ok. I asked you to fork my repository because it would be easier to track. If you don’t want to, no problem 🙂
LikeLike
> “Changed blah and foo to do this or that. Fixes #10 and #12, and refs #12. worked 10.2h” –
closing tickets 10 & 12, referencing #12 (that should work in our hook as well), but also including the time that one has worked on the tickets mentioned in the commit message.
FYI, there are at least two popular Trac plugins that provide that “worked 10.2h” feature in commit messages:
http://trac-hacks.org/wiki/TracHoursHowto#AddingHoursviaComments
http://trac-hacks.org/wiki/TimingAndEstimationPlugin#PostCommitHooks
In general, the best way to implement additional “command” features in commit messages is with a Trac plugin that implements an IRepositoryChangeListener interface — that way it can be agnostic of the version control backend that’s being used. That script you linked to seems to be for an older version of Trac, from before the IRepositoryChangeListener interface was introduced (in v0.12)
LikeLike
Hey, Ethan!
Thanks for the valuable feedback!
I also found this here: https://sites.google.com/site/icucodetools/, but ideal would be a generic commit-message parser, with a set of modifiable triggers/actions/workflows. The IRepositoryChangeListener seems to be like a good place to hook such a parser up to.
I’ll have a look at your plugin you mentioned further down. Looks helpful : )
LikeLike
Hey again! : )
Thanks for changing the license and making the gist. I’ll post future code-updates there. (Could you maybe update the gist to reflect the license change?)
I too, tried redmine (actually, chiliproject, a better fork, but there’s a lot of tension between those two projects as well), but due to inferior Eclipse Mylyn integration (i.e. it just opens a browser inside eclipse instead of using the actual Mylyn rich client), I switched to trac.
Actually, at this point, I believe trac is architecturally superior to redmine & chili (in the sense of cleaner to integrate), albeit slightly more difficult to set up. Once you figure out what to put in your master config, “it just works”.
The big plus for me is the actually distinct environments, with the possibility of including a separate set of plugins for each one.
Before, I really fancied redmine/chili’s slick(er) interface, but
in effect, trac is just the database and the UI for the non-technical managers. The dev’s should be able to handle everything from inside eclipse or through git commits (that’s why I’d also like update, create, etc., but close is a good start ; ) ). Thus, the priority is on integration with the rest of the toolchain, and this is where trac wins, IMHO. Also, trac is BSD licensed, and I prefer a Free brewery vs. a free beer. : )
It seems like you did misunderstand me a little, though. I wouldn’t be as crazy as to suggest replacing trac-admin in this context, just to replace the call to it from our hook with a call to e.g. Jens Timmerman’s hook (https://github.com/JensTimmerman/TRAC-SVN-to-GIT-migration/blob/master/hooks/trac-post-receive-hook.py).
His hook supports commit messages like:
“Changed blah and foo to do this or that. Fixes #10 and #12, and refs #12. worked 10.2h” –
closing tickets 10 & 12, referencing #12 (that should work in our hook as well), but also including the time that one has worked on the tickets mentioned in the commit message.
It would also be reasonably easy to add additional commands to that boilerplate.
So, we’re not talking plugin development here, I hope, as I’d also have to pass in that case, due to lack of free (as in beer, in this case) time : )
LikeLike
Really useful information, thanks. I’ve been using Trac+Gitolite together for a while now (one Trac, many Git repos) and have been afraid to even try enabling the cached_repository setting, so I’ve just been dealing with the slow performance instead 🙂 I’ll definitely revisit that now though.
You might be interested in a Trac plugin I recently released: https://github.com/boldprogressives/trac-GitolitePlugin
It provides some additional integration points between Trac and Gitolite. Specifically, it integrates Trac’s Repository Browser with Gitolite “R” permissions, so a Gitolite repository will be hidden in Trac unless the logged-in user has the Gitolite “R” permission. It also provides Trac Admin Panels for creating new Gitolite repositories, managing Gitolite repository permissions, and managing Gitolite user SSH keys.
LikeLike
Wow, this is great. No more messing with chmod to control access.
LikeLike