node.js ko
fremont troll
fremont troll
by Cave Trolls
freemont troll attempts to take a GTFS(General Transit Feed Specification) zipfile via a link from the user and download it to the server. The server then extracts the zipfile via a childProcess called in node. Once extracted, a process is created to parse the GTFS feed and validate(producing a report) it using mongoosejs. After validation the GTFS feed is loaded into a database and then shown visually on a google map (stops).
tl;dr: Validates a GTFS feed, then shows the feed data on a google map.
Judging Instructions
On the main page, enter a city like Seattle or Portland or a GTFS feed link from http://www.gtfs-data-exchange.com/ and then click the “Upload” button. Then sit back and watch the magic!
Who
blakmatrix — Seattle, WA
eugeniy — Seattle, WA
How
cave trolls would like to thank:
node, express, less, jade, nko, mongoose, http-get, coffee-script, underscore, csv, socket.io, dnode, vows VIM, Notepad++ git, n, npm jquery, html5, ssh, winSCP, cygwin UNIX, linux, Win7 chrome, chromium, IE, firefox, safari
… and the almighty bit… its all just 1s and 0s anyways
Votes
Your Vote
Voting is now closed.
Other Votes
dshaw
judge
therazorbladejudgeUploaded the provided Seattle map but nothing happens after ‘Finished Validating shapes.txt’ – was there suppose to be more?
- contestant
Well technically it is using socket.io, we just decided to call it polling because that made the most sense, that or status. There was supposed to be more but we found our method to place markers for stops out of the mongoDB on the map didn’t work and then we ran out of time. But the app does validate GTFS feeds, and that was part of our goal. We’d hoped to make a nicer report, but again time was our enemy.
- therazorbladejudge
I don’t know much about GTFS feeds and how hard it is to validate them. Is this just a matter of validating a format or involves more?
- contestant
There is quite a bit involved actually, for a quick over view you can take a look at http://code.google.com/transit/spec/transit_feed_specification.html . A lot of it is ranges and formats, but then there is also a bunch of optional fields and files that also add in some extra complication. The largest validation starts when attempting to add rows into a database and validating against the individual tables making sure the data is consistant. Google already has an open source tool that does this here: http://code.google.com/p/googletransitdatafeed/wiki/FeedValidator . We felt like perhaps there could be a better way to do this validation and faster(!), however, this was a bigger project than 48 hours (without much planning—how we read the rules). We’re still working on this and plan to release it as a tool for our main app in the future.
- contestant
Sorry, here are the links (if markdown works): GTFS Specification
- therazorbladejudge
Thanks for the detailed explanation. I added another utility star based on the idea that having a JS validator for this format might be useful. Beyond that I’m really unable to judge this.
lachlanhardy
contestantIf I understand correctly what this does, I think it’s fantastic. Unfortunately, the incompleteness means I was a bit confused.
I’m guessing it’s targeted at those in the game rather than general public.
- contestant
Thanks! We plan to finish it after the judging is done.
martinciu
contestant- contestant
Thanks!
mdejudgeSite design is nice. I’m sure this would be more interesting if I knew what GTFS is. At the end it didn’t show the location on the map.
- contestant
GTFS is a common format for public transportation schedules and associated geographic information.
whereisciao
contestantInitial thoughts on design was ‘awesome’. Great colors and typography. Using the application was less than idea. I tried typing in “Seattle” into the search only to realize its expecting a url. Overall I applaude your use of public data.
- contestant
Thanks, I myself thought the same thing but we had more pressing matters when I thought of that.
wjgilmorejudgeLooks pretty cool, it’s stopping at “Finished Validating trips.txt” though? The routes are never actually plotted. Tried in both very recent versions of Firefox and Chrome using the suggested Seattle GTFS.
- contestant
Its actually still working, if you use a smaller feed like this one you can see the normal operations. Because the Linode has only 512MB RAM the program can do only so much at once. When we were testing we had 2-4GB and didn’t realize we would be limited in deployment.
chrismatthieujudgeTried Seattle and Portland several times with same result – Sorry, buddy! Something went wrong!
- contestant
That’s odd, usually that error only pops up when it cant access the database or JSON is limited in the browser. What browser/OS were you using and does it do the same thing now. We have a watchdog,monit, running in the background so if the database did go down it’ll be back up and running withing 1-2 minutes.
jolieodelljudgeTeam Cave Trolls: EXCELLENT use of public data + technology. I give you five gold stars for making something with actual utility.
However, I’m going to call the app incomplete for one reason: It doesn’t perform its intended tasks as quickly as a web app should. The average dude who takes public transpo isn’t going to wait around for 10 minutes or longer to read the log of validated text files and hope he gets the info he needs. Come to think of it, the average public transpo dude probably isn’t going to be looking up GTFS feeds, either… If there’s a way for y'all to finish this up, make it faster, perhaps store some of the necessary data on your end, etc., I’d love to chat with you in a month or two.
Overall, great idea, and please keep workin' on it!
- contestant
Thanks Jolie! This was only a tool for the bigger app we are now working on, we in no way expected (the thought was if there was an intended audience it would probably be the same folks who use the feed validator from here: http://code.google.com/p/googletransitdatafeed/wiki/FeedValidator Its something you have to download and then it TAKES forever to validate and produce a report) public transport users to upload a GTFS feed every-time they wanted to see where their bus may be at and the like. We have big plans but this was only a small attempt at a means to an end—a steppingstone if you may. We’ll chat with you in a few months when we’re ready to dazzle and provide a much needed service to the average user.
glenngillenjudgeDon’t think it’s as widely usable in terms of utility as some, but an interesting solution to a problem. Tried a few files, got through validating shapes.txt in all of them but nothing was ever updated on the map. Think the UI could be improved to give less emphasis to the log (is it important to see every step? Or would just the current step/state suffice?) and to make it more obvious on the map what had happened.
- contestant
Yes, the plan was to use less of the log and produce a validation report at the end, but that, and producing stops on the map required more time.
dylang
contestantIt says it validated the Seattle data but nothing really happened.
- contestant
The point was to validate the data, the second part we ran out of time and that was to take the validated data and produce stop markers on the map.
ek
contestantGreat use of design and tech. However it doesn’t feel complete and I couldn’t get it to work. I am looking forward to see where this is going. Bravo!
- contestant
Unfortunately we ran out of time, to produce anything on the map, but if you saw the text at the bottom you got it to work!
As far as I could tell, the final step of showing stops on the map is not working. I tried both of the preloaded files for Seattle and Portland, as well as linking to the file for LA.
One product design tip, if you continue development on this project, pull the entire feed from http://www.gtfs-data-exchange.com/feed for sample data. (Use YQL to get a json representation.)