Saturday, March 29, 2014

Nice Python pack for Vim

I usually use Vim for my coding and all other stuff requiring text editing. There are a lot of plugins, scripts, configs, packs for Vim and a lot of them help with Python. But  so far I could not find the pack I'd love and I'm too lazy to make it on my own, so I usually throw out and re-create my whole Vim config dir once a year or so.

This time I used Klen's Python Mode and it's good judging by number of tools involved: almost all code style/syntax helpers I know (pep8, pep257, pyflakes, pylint, mccabe, and pylama to integrate them), autopep8 to help with formatting and stuff, rope for refactoring (sorry, I won't overwhelm this post with links – look for them in readme in github repo). It can more or less integrate with virtualenv and, of course, use Python syntax and identation plugins for Vim.

It provides commands for almost everything but doesn't provide a lot of convenient hot-keys. Also there are no hotkeys for Django stuff: I'd like to have hotkeys or some menu at least for launching Django's dev server and running tests (with quick-jumps to error positions, as usual). Those things are usually needed not only with Django, but there are usualy no standard ways to do it in Flask or whatnot (maybe Flask-Script will do, but I didn't check it out yet).

It can run syntax checkers on every write or on every write if file's changed, but it doesn't work very good when you write a lot of files in tabs at once with :wa (you'll see several windows with errors at once, one per each file). Maybe I'd like to have my test run on each write (preferably with errors in the same window as syntax checkers), or maybe not – tests can be pretty slow.

Also this and almost every other pack lack possibility to set some settings on per-project basis. Sometimes it can help if you need to set paths, scripts, syntax check options.

But if I start building my own vim-python pack someday, I'll probably fork Klen's one. I didn't see anything better with all those external tools so far.

Wednesday, March 12, 2014

Shinken – a Short Review of Nagios: The Next Generation

What's it all about

Whatever services you are running, there always a need for monitoring. You want to know that you service is down before you customers or bosses fire you :) There are a lot of open source software and hosted services for that, but services are usually featureless or costly (or both) and open source solutions are often featureless and outdated as well. But some of them are different.

For some reason Shinken doesn't get much of attention these days and I intend to fix it somehow. For those of you who don't know it yet, Shinken is monitoring software not very unlike Nagios (and backward-compatible with it in most cases). But it's totally re-written with Python and have a lot of ready to use 'modules' and 'configuration packs'.


Shinken's core was divided into several separated services: arbiter, scheduler, broker, poller, etc. connected over network. It gives both better scalability and better availability. You can host different services on different hosts, double them when needed, and not worry about monitoring while you restart web UI or something for update. Also several things (like web UI, for example, or data retention backend) are, in fact, modules which could be chosen out of several options.

Default web UI is nice, it looks like contemporary web UI should look like, it doesn't use cgi or something that outdated (for technology's sake, cgi, c'mon? it's 2014 already!).

If you were using Nagios before, migration should not be very hard. Quickstart documentation says you can use you Nagios configuration as is, but I would not recommend you to. I'd suggest trying to re-use big amounts of data like hosts descriptions while trying to incorporate other pieces into Shinken's own configuration structure. It would take some time, but it makes sense. Almost all third-party tools (plugins and stuff) will work as well.

Configuration can be loaded from various sources, you can monitor all your AWS instances automatically, or keep configuration in database with some configuration tools, or whatever.

It all may help someone build commercial monitoring service using Shinken. I'd love to see a lot of them around, cheap, feature-rich, and helping to develop great open source tool.
It's under active development so we can expect it to become nicer with every new release.


Well, it's not all good. There some minor disadvantages as well. For example, it heavily relies upon Nagios plugins (they are shell commands, in fact, but it does not matter) and several other tools built for Nagios. It ok, they good, but I'd prefer if they maintain their own fork (with repository similar to modules).

Also several things I found annoying in Nagios are still here. Why, for example, no one wrote notification module using Python library and templates instead of shell pipes and /usr/bin/mail? Apart from other reasons, it might help sending emails with CC/BCC for several receivers and with proper headers to organize emails in threads.

Documentation is far from being up-to-date and full. Especially quick start documentation. I've spent half an hour trying to get web UI working. Several new things (not available in Nagios) may require you to change you configuration structure a bit in order to get all the advantages. Notification filters rely on contact groups, default network service monitoring packs (like http or ssh) rely on templates instead of hostgroups, so you'd need to change all your hosts configurations unless you want to stick with you own custom service configurations, and so on. It's understandable that there are differences, and Shinken's way is usually better one, but when I read quickstart documentation' I'd prefer to see what those differences are.


Shinken's cool. If you are still using Nagios, consider at least trying it (it won't hurt to have both for a while, but you'll quickly realize that you don't need Nagios anymore). If you don't use any monitoring yet, but want to try something, try Shinken. If you arethinking about IT start-up, think about starting monitoring service, there are not much of cheap and feature-rich services out there. If you are Shinken user already, think about sharing some of your experience, tips and tricks with community. Maybe your module or pack is what everyone need? And I will continue to use it whereever I'd used Nagios before. There is nothing about monitoring that Shinken is not able of.

And I'm very eager to see any hosted service using Shinken, that was the one thing Nagios was not capable of at all.

Saturday, March 8, 2014

Pricing of good continuous integration service – reply to codeship

I've recently discovered nice continuous integration and continuous deployment service – It's not just another hosted Jenkins and it has some very interesting features (like integration with major code hosting services and development tools). But I won't probably use it except maybe for some pet projects of mine with no active development. Here are my reasons why:

  • Free plan is no good for even one project. 50 builds per month is not enough for anything: any project in active development usually needs several daily builds, even if not counting merges and feature branches. Commit early, commit often (and with CI it also means test often) is the usual approach of our days.
  • Cheapest paid plan costs $50 per month. Yes, it has unlimited almost everything, but fifty bucks is small, but significant amount of money and small business (or non-profit project) can usually utilize them better.
  • Setting up Jenkins, on the other hand, would cost you about $5/mo for hosting and couple of hours of sysadmin's work.
They also have larger plans and all paid plans are structured by number of concurrent builds, the only limited thing in the system. And since I was asked in twitter how would I structured pricing of such project, here follows my ideas about it:
  • "Pay as you go" pricing is good. AWS is very popular partially because of that. Plans with some amount of batteries included are good too, but it should not be mandatory. So if you just starting development and don't need much of testing or just ended active development and build/deploy only occasionally, you can pay a few bucks, but if you're active user, you pay more.
  • Free plans are not required if service is cheap. It's usually better to have ten customers paying $1 than to have one customer paying $10 and twenty free riders. I'm not saying that some free plans or time-limited free trials don't help to review and test, they do help, but good cheap service is better than bad free one.
  • Most costly resource for continuous integration service (apart from development cost) must be the build time (time of using build instances, if we're talking cloud, or or cpu time if we use some other virtual machines, like, on our own hardware). Maybe also storage, not quite sure. So it would be better to charge less for less time-consuming projects and more for more time-consuming. Number of concurrent builds, number of total builds are not that important: some projects may be compiled for hours, some are built and tested in a few seconds.
  • Maybe optional delayed builds could be a good way for saving for some users. Like AWS has "spot instances" to utilize unused resources. Delayed builds could either use spot instances or use reserved instances when no one else uses them. Because no everyone needs instant builds, I can imagine someone want, like, 30% discount for builds delayed up to one hour, for example.
  • No build time used – no money charged. Except maybe for something little for storage and other costs.
It's not like I'm going to start competition tomorrow, so my observations are mostly from user's viewpoint, but if I were starting it, I'd do it like I said and would probably take a significant market share (given the competitive technical qualities).

P.S. Also, small technical side note, I don't like their lack of periodic builds and ability to use custom git repositories (not on github or bitbucket) – some people just don't use github for various reasons (they claim to fix this soon though) and periodic builds might sometimes be helpful for "nightly" builds (there are projects so big that it's not worth it to build them more often anyway).