I took me a little bit but, with the help of Mikko and Martin, I've got a couple of add-ons running tests with Travis CI.
Before setting up Travis CI, you have to make some changes to the Setup Script of your package.
In my case, my add-on package only works for Plone versions 4.1 and later, so I have added Products.CMFPlone as a dependency:
… install_requires=[ 'setuptools', 'Products.CMFPlone>=4.1', ], extras_require={ 'test': ['plone.app.testing'], }, …
Products.CMFPlone contains a cut down feature set: just the things I need to run my tests.
Setting up Travis CI is pretty easy: just sign in and activate your GitHub Service Hook
Now, you need to configure your Travis CI build with a .travis.yml file in the root of your repo:
In my case I'm running tests for Plone's latest stable release (4.2 at I write this post) on top of Python 2.7.
Let's take a look at the travis.cfg buildout configuration file:
The main issue I experimented on my first tests was timeouts, so I have a couple of tricks here for you: first, we are extending the standard Plone testing buildout configuration that includes most declarations for us and takes care of always running the latest stable version; we are only going to use the test part. You need to add the package-extras just if your add-on is using plone.app.testing on the test option in extras_require of your package declaration as mentioned above.
We also need to add a socket-timeout of 3 seconds (only available on zc.buildout >= 1.5.0) and a list of allow-hosts to download the dependencies. This is pretty important and will avoid timeouts as Travis CI has hard time limits and timeouts are between 10 and 15 minutes for test suite runs (1).
Last, we have to replace the eggs option on the test part; we need to do this because we don't want to include neither Plone or plone.app.upgrade on the tests.
Finally, you can add a Status Image with a link back to the result of your last build on your README.txt file:
.. image:: https://secure.travis-ci.org/collective/your.package.png :target: http://travis-ci.org/collective/your.package
To run the tests you only need to make a push to your GitHub repo. Easy, isn't it?
For a live example of all I mentioned above, take a look at the collective.prettydate package.
Travis CI is really easy to set up and fun to use; I strongly recommend it and, if you like it, please show your love donating.