Previously I used freshen because I know more python than ruby but there does seem to be a lot more support around cucumber than freshen. I did have a look at the python support in cucumber but everything I read about it seemed to point to it not being great.
Moving on to the point of this post. I was looking at using cucumber-nagios to run some of these tests more regularly on our nagios server. My theory was that I could run all the test regularly and report if any of the parts of the server fail. So testing further into the stack.
The cucumber-nagios website has some info and tutorials to get you started but I soon found that it lacks some of the test that I wanted for things like checking a 404 page etc.
My solution has been to use my existing tests, install cucumber-nagios and just use the output formatting to output to nagios.
cucumber --format Cucumber::Formatter::Nagios features/newserver/ CUCUMBER CRITICAL - Critical: 1, Warning: 2, 16 okay | passed=16; failed=1; nosteps=2; total=19; time=2
The last line above is the output of the command and is a format that Nagios can parse and act on. As you can see I still have one test failing. You can then either add the whole test suite into Nagios or pick and choose specific features. You may want to run all the tests occasionally but check some of the status and security features more regularly.