At Netresearch, we use JIRA for bug and issue tracking, and Gitorious for Git repository hosting and management.
Connecting both tools was a long-standing dream we had: Seeing commits directly in JIRA speeds up code reviews, and makes code activity more visible to other people.
Ways to connect JIRA to Git
Our scenery was as following:
- We had a self-hosted Gitorious instance running with over 500 git repositories on it, fine-tuned with a custom hook system to e.g. deploy applications automatically when a certain branch gets pushed. That’s not something you throw away easily.
- JIRA was used as issue tracker for all projects and would also stay.
We had to connect them both somehow. Here is a list of methods to show Git commits in JIRA:
Old JIRA Git Plugin
The old, open source JIRA Git Plugin gives you a “Git commits” tab in JIRA’s issue detail view.
Setup is very hard, because it requires the git repositories to be locally
available on the JIRA server. This would mean to clone all those 500+ repositories on the JIRA server, taking up some 12 gigabytes of data – and they must be kept in sync with the Gitorious repositories. Creating new repositories on Gitorious would mean administrative effort on the JIRA server, too.
Apart from that, you have to configure every single repository in JIRA itself. Do that 500 times.
So this was no solution for us. If you want to try it, the
source is still available on github.
Git Version Control Viewer
Someone took source of the old Old JIRA Git Plugin, updated it for JIRA 5 and now sells it as Git Version Control Viewer for the price tag of 700$ for 50 users.
The issues are the same with the old, free plugin – no solution for us.
FishEye is Atlassian’s premier source code hosting application, and is nicely integrated with JIRA via the FishEye plugin.
The problem is that it would completely replace our Gitorious server, something we’re not willing to do.
It also costs 2200 US$ initially for 50 users, and the same amount every following year if you want updates.
So again not a solution for us, since we don’t want to replace our source code server just to get JIRA integration.
Stash is also from Atlassian, offering Git hosting only. The connection to JIRA is made via the JIRA FishEye plugin.
We would also have to throw away Gitorious to make use of it, and had to pay 3300 US$ for it. It’s also in its infancy and not very feature-rich yet.
Again something we didn’t want to use.
Implementing the JIRA connection ourselves
Since there were no usable solutions for us to get JIRA and Gitorious connected, we decided to build it on our own.
We had the following choices:
- Use the JIRA FishEye plugin, and implement the FishEye API for Gitorious.
This would give us all the features of the FishEye plugin and ensure compatibility with future JIRA versions since Atlassian develops it further.The big downside is that there is no public documentation about the API, so we would have to reverse-engineer it all ourselves.
We know the FishEye Remote API page, but this doesn’t tell you even 30% of the API that’s needed for a connection with JIRA.
- Use the JIRA CVS plugin: It expects a CVS log file somewhere on the
JIRA server and parses that, and keeps an index of it in RAM.We’d need to create a CVS commit log file from our Git repositories and put that on the JIRA server.
- Sounds possible, but will likely cause issues with the available memory on the server. Links to Gitorious would probably be hard to implement.
- The SVN plugin cannot be used since it wants to run svn log on a subversion server, without a way to intercept it.
- Hack the Old JIRA Git pluginto not create its commit indexes from local git repositories but remote ones.This would require writing Java, something that we’re not proficient in, and adding a commit search api to Gitorious, since it doesn’t have that yet.
Option 1 was the most promising one, and so we chose to take the reverse engineering route.
Reverse engineering the JIRA-Fisheye connection
At first I used Wireshark to capture the HTTP requests between FishEye and JIRA, but that was very very very very tedious – you have to find
the first packet of the HTTP request, “follow TCP stream” and can then save the whole request and answer into a file. Then you have to reset the filter and find the next packet…
After doing that for some hours, I remembered Kore giving a talk about BSOAD, the Browser-SOA-Debugger, at the Berlin PHP Usergroup.
It was a real pain to setup, but after one or two hours it worked and showed me the jabber between JIRA and FishEye:
Unfortunately, it had some bugs (including a really serious one that broke all XML display, making BSOAD unusable for me). Since it was still the best software for capturing those HTTP requests, I fixed the bugs and sent pull requests #1 and #4 for BSOAD, as well as patches #5 and #6 for bdog.
After 2 days, the basic implementation was working, showing commits in JIRA for one repository. JIRA still crashed in the project view.
After 4 days, everything was smooth: No JIRA crashes anymore, and new repositories on Gitorious automatically got their hook registered
via a cron job. Links to Gitorious were working, too.
The result of our work is Klonfisch, running here at Netresearch for weeks without problems.
You can get it from the netresearch/klonfisch repository on github, along with detailled instructions how to get it up and running.
How it works
Klonfisch is a PHP application that sits between JIRA and Gitorious:
Gitorious sends commit information to Klonfisch which stores them in a MySQL database. This is done via “web hooks”.
JIRA on the other hand talks to Klonfisch, asking for commit information
for projects or single issues. Klonfisch searches in the commit database and returns them to JIRA. JIRA then displays the commits in the “Source” tab of issues and projects.
You just have to install the FishEye plugin for JIRA, which takes care of talking to Klonfisch and the commit visualization.
- Simulates a part of the FishEye API
- Works with the JIRA FishEye plugin 5.0.7
- Shows commits on issues (Tab “Source”)
- Shows commits for projects (Tab “Source”)
Currently missing features
- Showing files that got changed in a commit. Waiting for issue #128
- Project statistics
- Authentication – everyone can see all commits
- Activity streams