Providing JSON data views onto database data in search
As part of working on Warwick's search engine, in the revamp we've been trying to leverage a lot of data that extends above and beyond just web pages and blog entries. This means we've been indexing a lot of data that is then stored in a database and refreshed periodically (typically, every 5 minutes). This is then included from a number of sources, such as our interactive map of campus and the XMPP/Jabber bot that we run at ask@warwick.ac.uk (try adding it to your Warwick IM account!) - but it all comes down to exposing the data in the end.
But hey, it's all about sharing, so knock yourself out with the data...
Search Results
http://search.warwick.ac.uk/search/query.json?q=accommodation&indexSection=sitebuilder
You can get search results for sitebuilder or blogs (or both) from the endpoint, for example at http://search.warwick.ac.uk/search/query.json?q=accommodation&indexSection=sitebuilder (replace the sitebuilder with blogs, or remove the indexSection altogether to get amalgamated results).
Example usage: A live search, or search results from the XMPP bot.
Free seats in workareas
http://search.warwick.ac.uk/search/workareas.json
ITS provides a service at seats.warwick.ac.uk that shows where free seats are in workareas, and where the workareas are.
Example usage: Showing where there are free seats in work areas near to your department; results in interactive map; embedded results in search results; querying the Ask bot:
Live Bus Tracking
http://search.warwick.ac.uk/search/businfo.json
Travel West Midlands have started to track the positions of buses in relation to some bus stops in real time (presumably by some radio or satellite system) and are publishing the information to customers at their mobile website. For the stops in and around campus, this is useful information to some, so we've indexed the information (only including those buses that are being tracked in real time).
Example usage: A live tracking page; embedded in search results; results in interactive map; querying the Ask bot:
What's on?
http://search.warwick.ac.uk/search/whatson.json
By amalgamating results from a number of sources (the Students' Union, Warwick Arts Centre, and the internal and external News and Events pages) into a single database, it's easy to query information from it and return a fairly comprehensive set of information about what's on on a particular day.
Example usage: Providing a feed of what's on today on campus; showing what's on in relevant searches; Ask bot results:
Where is this room?
http://search.warwick.ac.uk/search/rooms.json
Thanks to Emily in e-lab's tireless work collecting and collating the data, we also now have a database of information on what rooms there are on campus and where they are - 3018 rooms in total! As a former student myself, I've experienced the frustrating of receiving a timetable with rooms on and not knowing where they are. Hopefully this will be a help!
The JSON feed requires a query parameter, for example:
- http://search.warwick.ac.uk/search/rooms.json?name=r0.21&exactMatch=true
- http://search.warwick.ac.uk/search/rooms.json?buildingNumber=47
- http://search.warwick.ac.uk/search/rooms.json?name=cs1
Example usage: Linking room data to a map; results on interactive map; injection into search results:
What this exercise has taught me, if nothing else, is that everything would be a whole lot easier if every bit of data provided me an API to access it!
Mathew Mannion
15 Sep 2008 15:32
| 
Tags: Bus Timetables Free Seats Json Room Information Search
| 
Comments (9)
| 
Report a problem
Loading…
9 comments by 2 or more people
[Skip to the latest comment]Let me guess – current students only?
16 Sep 2008, 20:01
Mathew Mannion
To what, exactly? :) Search is completely open – as is all the information you see here. The Warwick IM service is only available for current members of the University, but you should be able to add ask@warwick.ac.uk if you use any Jabber/XMPP client (including Google Talk, although the Google Talk client doesn’t allow you to add non-GTalk users so you’d have to use something like Pidgin)
16 Sep 2008, 20:11
Phil Wilson
Excellent stuff. We have some jabber bots internally at Bath (http://wiki.bath.ac.uk/display/bucswebdev/Jabber+Bots) but nothing exposing our public information. I’ll see if I can put something similar together tomorrow!
Also, just to confirm that I can use the askbot from my own Jabber account!
16 Sep 2008, 21:19
Mathew Mannion
Cheers Phil, we also have a Bamboo reporting bot on server (as well as a bot that reports problems with our Nagios sensors) but HungryBot looks like it might be something we could consider adding. We’ve been looking at the possibility of adding useful PUSH data (you have a lecture in 10 minutes, your 9am has been cancelled because your lecturer is in the Maldives, there is a gunman on campus; that kind of thing) personalised for students/staff. However, there are political ramifications with actually getting to the data before we can move on to that kind of thing :)
16 Sep 2008, 21:38
Phil Wilson
If I do have a small request, it’d be that outgoing messages from ask@ are returned in the same type as the incoming.
i.e. if I open a chat window and send “help”, I don’t want to get an event window (i.e. a normal XMPP message stanza) with the response, I want it to be in my chat window. Assuming you’re using the Smack client libraries, I think it’s just a matter of setting type=”chat” on the Message object (from my memory at least!).
With regards to pushful data, do eLabs or other warwick groups have user-centred (rather than emergency) SMS tools? Feel free to email me back if you want to take this out of the public eye ;)
21 Sep 2008, 18:28
Mathew Mannion
Interestingly, when using Pidgin/Adium, ask messages do respond directly in the message window – bizarre! I’ve added message.setType(Type.chat) to the code and have redeployed the bot – can you let me know whether you still get the same behaviour now? Is this the behaviour with the Smack client?
We have investigated (and partially developed) an SMS system for pushing useful messages out, but it was sidetracked due to changing priorities, if I recall correctly. You’d have to talk to someone higher up to get the full story I think, since I didn’t work on it myself :) For the meantime, we are using JanetTxt for our emergency SMSing.
21 Sep 2008, 21:28
Mathew Mannion
Of course, I mean Spark, not Smack :)
21 Sep 2008, 21:38
Phil Wilson
I was using Psi at the time. Seems to work now, thanks!
22 Sep 2008, 10:38
Mathew Mannion
You can always add me (M.Mannion@warwick.ac.uk) if you want to discuss further :)
22 Sep 2008, 10:42
Add a comment
You are not allowed to comment on this entry as it has restricted commenting permissions.