node.js ko
voyeur.io
voyeur.io
by Sidelab
Distributed node.js server monitoring. Like nagios but cooler and well, not as feature complete…
Judging Instructions
For the competition we focused very much on the non-CRUD elements of the solution. As such, the site is pre-configured to monitor a number of node VPS instances (including the competition Linode hosting).
After the judging period is over we will go back and add the ability to monitor additional servers, user registration, etc, but in the meantime you will have to be happy looking at and giving grief to the preconfigured servers.
While you can’t register a new server to monitor, you can interact with the front-end (click on a monitored server to expand the different sensors, etc).
Who
DamonOehlman — Brisbane, Australia
liamoehlman
liammclennan — Brisbane
stoolrossa — Brisbane, Australia
How
The project was written to assist with distributed server monitoring with a bent towards detailed monitoring of Node.js applications.
Monitoring Site
The hosted monitoring site (voyeur.io) is constructed from the following technologies:
Node.js
Frontend
The frontend is powered primarily by the excellent d3 visualization library. Something that was well worth playing around with – just wished we’d got time to implement the stacked CPU cores graph…
Monitored Site
An example site was put together that demonstrates the collection of system information, through a number of agents. Each of the agents is regularly executed based on a node-cron pattern.
Each agent collects some data which is stored into a local leveldb instance. LevelDB is extremely impressive and there are definitely some nice potential applications with Node. We used a personal fork (which implements leveldb iterators) of the node-leveldb module to handle storage and retrieval of the data.
Votes
Your Vote
Voting is now closed.
Other Votes
(3)
therazorbladejudge- contestant
it is definitely working but it has a bug that causes the graphs to stop if network round-trip time is ~ 2s. Try F5.
- therazorbladejudge
I tried on FF and Chrome. Still nothing.
- contestant
Interesting – I just tried it on Chrome, FF and Safari on my machine here and see graphs on all four monitored servers. If you have the time I’d certainly appreciate it if you could check to see if any any JavaScript exceptions are being reported.
- therazorbladejudge
Finally got it to work. Updated review.
- contestant
Thanks appreciate your persistence :)
hongymagic
contestantI think the idea is great. Some what similar to Joyent’s (no.de) server monitoring tools, but I think Joyent only supports logs on one SmartMachine.
Definite potential, I hope you follow it up and let it grow.
whoishozefajudgeseems like it could be useful, but unclear how it is better than existing system monitoring tools (including those you don’t have to install)
- contestant
Thanks for the feedback. The general idea (in the long term) is that you can work with the system in one of two ways:
Through a simple external HTTP test, and this is what most hosted monitoring solutions offer now.
Through more in-depth reporting via the JSON interface we expose on client machines.
Like you say there are a lot of options for getting something like option 1 going, but when it comes to option 2 most of the systems feel heavy and clunky. The goal for us was to get started on something that provided facilities for supporting both use cases 1 and 2, and it was easy to get up and running.
Still a long way to go, but I think we are happy with the start we made during the competition and intend to continue our efforts here.
Cheers, Damon.
martinciu
contestantCould use it some day.
mikealjudge- contestant
Thanks for the vote Mikeal, and thanks for your work on request. It’s a brilliant library I’ve used more than once…
mbleigh
contestantI feel like I’ve seen some services like this before, but it’s still neat to see in real time across multiple servers. I like the idea of hurting a server with a click. Maybe extend the app to have a Netflix-style “Chaos Monkey” that shoots servers in the head. ;)
- contestant
Michael – absolutely, there are definitely services like this kicking around. We believe ours is slightly different in the it is architected. It will support (eventually) both hosted monitoring and self-hosted installations, and the ability to do both.
Watch this space.
pfinettejudgeIt’s an interesting feature – but lacks a lot of polish to make it awesome. On the base level the graphs need work – they are hard to decipher and thus somewhat useless.
- contestant
Thanks for the feedback Pascal. I agree with your comment on polish – but I’m pleased with what we achieved given the time constraints.
The competition provided an excellent chance to have a play around with Mike Bostock’s D3 library, which is awesome, but does take a little getting used to.
eirikurn
contestantA smart idea, would love to have something like it for my own servers. Although I couldn’t understand how you planned to implement that with regards to security. Is it a web service or self-hosted? I had a hard time getting enjoyment from pushing the “HURT ME” button, I didn’t see any effect :D
- contestant
We really wanted to cater for both the hosted and the self-hosted situations, and I think what will be most achievable in the short term is the self-hosted solution.
We’re going to be separating the component pieces of the solution into separate repos over on Github, so it might be worth checking out https://github.com/voyeur-io in a week or so.
- contestant
The process that gathers the statistics has to run on the server being monitored. Currently the whole thing is deployable as an npm package so self-hosting should be simple.
rcrowleyjudgeThis is an ambitious tool. I’m remembering hearing second-hand that Paul Graham recommends people find a Unix tool that hasn’t been put in the browser and do that. You’re putting top(1) in the browser.
Technically, it falls down not just in feature incompleteness but in usability, too. Nothing but Flash has ever made my CPU fans spin like this site. Despite seeming to work constantly, the graphs don’t seem to update nearly often enough to keep up with the scrolling.
I desperately want this to be a real thing, so take my comments as encouragement.
- contestant
Thanks for the feedback Richard. Intention is to make this a real thing so we are definitely taking your comments on board :)
- contestant
You should have seen it when we had smooth graphs ;–)
In the current release I think we were too optimistic about the network performance between the browser and the web server. If the request doesn’t return in < 2s it causes problems with the graphs. I think for robustness we will have to make the visualization less real time.
aaronheckmannjudgeI like the concept. Couldn’t tell if the buttons actually did anything though. Could be a pretty useful way to monitor.3
- contestant
Thanks Aaron – the buttons do something, but you have to be a little aggressive with them to see the impact. We probably should have called the buttons “Tickle me” :)
- contestant
The servers with 4 red lines are representing quad cores. The ‘hurt me’ button will generate load on just one core. The graph will jump from close to zero up to 100%, but in the ui that is probably only 8 or ten pixels so its easy to miss, especially if other people are generating load.
mustardamus
contestantcarsonmcdonald
contestantmasylum
contestant
This can clearly be a very useful tool. Took me a while to get the graphs working but after three tries I finally got to see it. Missing a lot of features to make it useful and the graphs are not interactive enough to extract data. Hope you keep working on it.