EzDevInfo.com

Have buildbot poll a git repository for new commits?

Is there a buildbot plugin that will poll a git repository for new commits, like the currently included changes.SVNPoller?

The closest I have found is git_buildbot.py, but it works as a post-commit hook, so will not work with my setup (using Github, and buildbot on a machine that github's post-commit cannot reach) - simply polling the git repository would work perfectly.

I currently have a build run once an hour, but there's not really any point in running the tests unless something has changed..

"Pretty" Continuous Integration for Python

This is a slightly.. vain question, but BuildBot's output isn't particularly nice to look at..

For example, compared to..

• phpUnderControl
• Jenkins
  • Hudson
• CruiseControl.rb

..and others, BuildBot looks rather.. archaic

I'm currently playing with Hudson, but it is very Java-centric (although with this guide, I found it easier to setup than BuildBot, and produced more info)

Basically: is there any Continuous Integration systems aimed at python, that produce lots of shiny graphs and the likes?

Update: Since this time the Jenkins project has replaced Hudson as the community version of the package. The original authors have moved to this project as well. Jenkins is now a standard package on Ubuntu/Debian, RedHat/Fedora/CentOS, and others. The following update is still essentially correct. The starting point to do this with Jenkins is different.

Update: After trying a few alternatives, I think I'll stick with Hudson. Integrity was nice and simple, but quite limited. I think Buildbot is better suited to having numerous build-slaves, rather than everything running on a single machine like I was using it.

Setting Hudson up for a Python project was pretty simple:

• Download Hudson from http://hudson-ci.org/
• Run it with java -jar hudson.war
• Open the web interface on the default address of http://localhost:8080
• Go to Manage Hudson, Plugins, click "Update" or similar
• Install the Git plugin (I had to set the git path in the Hudson global preferences)
• Create a new project, enter the repository, SCM polling intervals and so on
• Install nosetests via easy_install if it's not already
• In the a build step, add nosetests --with-xunit --verbose
• Check "Publish JUnit test result report" and set "Test report XMLs" to **/nosetests.xml

That's all that's required. You can setup email notifications, and the plugins are worth a look. A few I'm currently using for Python projects:

• SLOCCount plugin to count lines of code (and graph it!) - you need to install sloccount separately
• Violations to parse the PyLint output (you can setup warning thresholds, graph the number of violations over each build)
• Cobertura can parse the coverage.py output. Nosetest can gather coverage while running your tests, using nosetests --with-coverage (this writes the output to **/coverage.xml)

buildbot vs hudson/jenkins for C++ continuous integration

I'm currently using jenkins/hudson for continuous integration a large mostly C++ project. We have separate projects for trunk and every branch. Also, there are some related projects for the Java code, but the setup for those are fairly basic right now (we may do more later though). The C++ projects do the following:

• Builds everything with options for whether to reconfigure, do a clean build, or use a fresh checkout
• Optionally builds and runs all tests
• Optionally runs all tests using Valgrind's memcheck
• Runs cppcheck
• Generates doxygen documentation
• Publishes reports: unit tests, valgrind, cppcheck, compiler warnings, SLOC, open tasks, and code coverage (using gcov, gcovr, and the cobertura plugin)
• Deploys code nightly or on demand to a test environment and a package repository

Everything is configurable for automatic builds and optional for on demand builds. Underneath, there's a bash script that controls much of this, which farther depends on our build system, which uses automake and autoconf along with custom bash scripts.

We started using Hudson (at the time) because that's what the Java guys were using and we just wanted nightly builds. Since then, we've added a lot more and continue to add more. In some ways Hudson is great, but certainly isn't ideal.

I've looked at other solutions and the only one that looks like it could be a replacement is buildbot. Would buildbot be better for this situation? Is the investment worth it since we're already using Hudson? Why?

EDIT: Someone asked why I haven't found Hudson/Jenkins to be ideal. The short answer is that everything can be improved. I'm simply wondering if Jenkins is the best current solution for my use case or whether there is something better (buildbot?) that would be easier to maintain in the long run even as new requirements come up.

Buildbot parsing Python Unit test results

I have a test suite that outputs test results in the Python Unit Test format: http://docs.python.org/library/unittest.html

Is there an existing Buildbot module/plugin that can parse this form?

Example:

DigitalReadWrite_02                                         ... ok
DigitalReadWrite_03                                         ... ok
DigitalReadWrite_04                                         ... ok
PWMoutput_02  (PWM=128 50% LOW 49% HIGH)                    ... ok
PWMoutput_03  (PWM=128 50% LOW 49% HIGH)                    ... ok
PWMoutput_04  (PWM=128 50% LOW 49% HIGH)                    ... ok
--------------------------
Ran 6 tests in 1.652s

OK

I've written a custom parser, but it's only got the basic cases. Is it worth the effort to make it comprehensive for all flavors of Python Unit test format.

Support for multiple repositories using Buildbot

Currently Buildbot does not support multiple repositories. If one desires to have this then separate instances of Buildbot need to be run.

Still I'm curious if anyone has come up with a creative workaround to get this feature working anyway.

"Selling" trac/buildbot/etc to upper management

My team works mostly w/ Flex-based applications. That being said, there are nearly no conventions at all (even getting them to refactor is a miracle in itself) and the like.

Coming from a .NET + CruiseControl.NET background, I've been aching to getting everyone to use some decent tracking software (we're using a todo list coded in PHP now) and CI; I figured trac+BuildBot would be a nice option.

How would you convince upper management that this is the way to go, as well as some of the rules mentioned in this post? One of my main issues is that everyone codes without thinking (You'd be amazed at the type of "logic" this spawns...)

Thanks

What algorithm does buildbot use to assign builders to slaves?

I have a buildbot with some builders and two slave machines.

Some of the builders can run on one slave, and some of them can run on both machines.

What algorithm will buildbot use to schedule the builds? Will it notice that some builders can run on just one slave and that it should assign those that can run on both slaves to the less demanded one?

(I know buildbot can be used to run the same build on multiple architectures, say Windows, Linux, etc. We are using it to distribute builds for performance, because a single build is enough for us).

How to distribute and execute platform-specific unit tests?

We have a python project that we want to start testing using buildbot. Its unit tests include tests that should only work on some platforms. So, we've got tests that should pass on all platforms, tests that should only run on 1 specific platform, tests that should pass on platforms A, B, C and tests that pass on B and D.

What is the best way of doing this? Simple suites would be a hassle, since, as described, each test can have a different list of target platforms. I thought about adding "@run_on" and "@ignore_on" decorators that would match platforms to test methods. Is there anything better?

How can you "source" an environment inside a Buildbot step?

Within Buildbot I need to be able to "source" an environment before doing a compilation step.

If I was building the application from command line using bash I would have to do:

. envrionment-set-up-script
build_command

Within the build bot master.cfg file I have tried the following:

factory.addStep(ShellCommand(command=["source","environment-set-up-script"])
factory.addStep(ShellCommand(command=[".","environment-set-up-script"]))
factory.addStep(Configure(command=["source","environment-set-up-script"]))
factory.addStep(Configure(command=[".","environment-set-up-script"]))

All of which fail, this is because the command cannot be found, which makes sense as it is a bash builtin.

Also I do not think that this is the correct approach as the environment would not necessarily be used when the next step of the factory is called.

How to customize buildbot web pages

I am trying to make some extra web pages for a test buildbot, since I am planning to have one running my project.

Practically I would like to have a waterfall page that show the button to build a specific builder, close to the build name, instead than in the builder page only. I would also like to have some reference documents loaded from inside the builder work folder, and from other locations on the slave machine; using buttons to display or hide them.

I've looked at the manual and I do not see any info about how do you customize or create new html pages, that can leverage on the Buildbot features (like the templates already included with Buildbot do).

I have opened some pages, and see that there are some html files that actually has non-html code statements like

% macro
% for

And so on. I am not a web programmer so I am quite clueless about what should I look for. Tried to google the word macro for HTML and I just got a bunch of results related to Wiki customization; it does not look like it is Python language so I am quite lost.

Is there anyone that was successfully able to make custom pages for the buildbot, and could give me some pointers about what to learn?

Is there a way to test if anyone pushed to the remote repo using JGit from local repo?

I am designing a User interface in Java and I am trying to make my own buildbot like thing. So for that I want to create a function like buildbot's gitpoller in JGit. Is it possible?

Windows cloud instances for testing open-source project? [closed]

I work with SCons (www.scons.org), the open-source build tool. We do test-driven development, and we have a buildbot, but we're always short of Windows buildslaves. We'd like (in a perfect world) a large collection with different python versions, different compilers, etc. Is there any free or near-free cloud-based Windows instance we can use with buildbot? I've looked at Azure and Amazon EC2 -- EC2 micro instances seem like our best bet at the moment, but I'm wondering if there's anything truly free as in beer for open-source project testing. Anyone else doing anything like this with experience to share?

Which continuous integration server is able to queue jobs?

Use case:

CI server polls some VSC repository and runs test suite for each revision. And if two or more revisions were commited, even in a relatively small time interval, I want the CI server to put each of them in queue, run tests for each, store the results, and never run tests again for those commits. And I don't want the CI server to launch jobs in parallel, to avoid performance issues and crashes in case of many simultaneous jobs.

Which CI server is able to handle this?

My additional, less important requirement is that I use Python and it is desirable to use software written in Python, so I looked at the Buildbot project, and I especially want to see reviews for this tool in the matter of is it usable in general and is it capable of replacing most popular solutions like Travis or Jenkins.

How to have a buildbot GitPoller change source watch all branches?

I'm looking for a way to have a GitPoller changesource watch all branches instead of just one.

For now, either I specify branch='some branch' in the GitPoller constructor, or it defaults to master.

Even better would be to be able to specify some ref pattern to watch.

Is that something one does already? Or does it need to code another kind of GitPoller ?

Thanks.

Buildbot having issues with GitPoller

I've setup Buildbot and I'm trying to run a Gitpoller to check for changes to code. I'm getting the following error inside the twistd.log on the master machine.

2012-08-22 17:10:50+1000 [-] http client disconnected before results were sent
2012-08-22 17:11:11+1000 [-] http client disconnected before results were sent
2012-08-22 17:11:21+1000 [-] gitpoller: polling git repo at <git repo censored>
2012-08-22 17:11:31+1000 [-] gitpoller: repo poll failed
2012-08-22 17:11:31+1000 [-] Unhandled Error
    Traceback (most recent call last):
    Failure: twisted.internet.utils._UnexpectedErrorOutput: got stderr: "fatal: ambiguous argument 'master..origin/master': unknown revision or path not in the working tree.\nUse '--' to separate paths from revisions\n"

2012-08-22 17:11:31+1000 [-] gitpoller: no changes, no catch_up

I have SSH keys setup on the master and on the slave and both copied into Github. Anyone have any ideas what this could be?

I'm running the latest version of Buildbot .8.6

Thanks guys