Bringing Azure+OData+Silverlight Goodness to Open Data Apps in Ottawa

On Saturday April 24th 2010, at Open Data Ottawa Hackfest, a team built the ‘OttGuide’ application (here showing 6000+ bus stops on the Ottawa public bus transit network), built on the Open Data Application Framework using Silverlight and Bing Maps, accessing an OData service built with .NET 3.5 and hosted on Windows Azure and SQL Azure.  For more on open data and cross-platform togetherness, consider for the Make Web Not War conference on May 27th 2010 in Montreal.

odata1

This passed Saturday Apr 24th 2010, the very successful Open Data Ottawa Hackfest took place for the first time in Ottawa City Hall organised by Edward Ocampo-Gooding and friends.  A large number of people (enough to exhaust the free t-shirt supply) attended for 4+ hours from different fields (developers, designers, librarians, statisticians. media and city officials).  The event included opening presentations, a hack fest where attendees worked on applications showing use of Ottawa data and closed with a series of presentations showing the fruits of the attendees’ labour.

Prior to the event, the City of Ottawa (unlike cities like Vancouver and Edmonton) hadn’t officially published municipal data or a data publication policy.  This meant that attendees to the hackfest were using various interesting methods to get data for their application from screen scraping to manual data re-enty.

I had the opportunity to talk at length with the city’s CIO, Guy Michaud, and discovered that the city is in fact on the verge of agreeing a policy and then immediately publishing a few initial sets of satic data, most likely sometime in May.  It’s quite possible that the hackfest event helped push this action ahead, but Guy is clearly a guy with forward-looking ideas on the issue of data publishing.

I was told about the event a few weeks ago by friend and Microsoft Canada IT Pro Advisor Rick Claus who was planning on attending.  So, in those weeks, I took a look at some of the work that has been done to obtain Ottawa municipal data and at open data work in general.  In particular I looked in 3 things:

odata2

  • Craig Davey had done extensive work on taking Google-compatible data published on the City’s OCTranspo site, augmenting it, converting it to GTFS format and publishing it with updates in a feed.
  • I had been familiar for some time with Microsoft’s open protocol for providing access to queryable and updateable data sets in a RESTful way – OData.  I did further research into this.  OData builds on ADO.NET Data Services and is fully incorporated into WCF Data Services within the recently released .NET 4 framework.  OData is designed to be consumed (or produced) by many different clients on many platforms. There are client libraries available or in the works for .NET (3.5 with an update & 4.0), Silverlight 4.0, iPhone, Javascript, Windows Phone 7, PHP & Java.  In fact, any client that can make web requests and handle XML can access OData-based services.

The night before the event, on a total whim, I decided to knock up an OData service exposing OCTranpo data, so I: 

  • started with Craig’s GTFS-format data provided as CSV files in a ZIP file. 
  • created an ADO.NET Entity Framework Model in a new .NET 3.5 Class Library project in Visual Studio 2010 to represent the GTFS data with entities and relationships
  • used the designer tools to generate DDL script to create the database schema
  • generated the schema using the script
  • created a database in my local SQL Server 2008 server
  • wrote code to import the CSVs and use the ADO.NET Entity Framework classes to import the data which was taking 3+ hours
  • wrote new code to import the CSVs with the SQL using bulk copy class bypassing the model which did the import of over 2 million bus stop time rows in a few minutes
  • added a WCF Service project and removed the default service from the project template
  • added a WCF Data Service item, setting it up to use the class of the entity model I created and providing read access to all entities
  • copied the connection string from my entity model class library app.config into the WCF Service web.config
  • tested my OData service using the OData URL query syntax, here showing a list of all stops… but this could be queried to find things like a nearby stop (from GPS coordinates), the bus times for a stop, the trips (e.g. ‘saturday service to ABC on route x’) and routes (i.e. bus number) going through a stop, find the stops on a trip, when a specific service gets somewhere, or even for full journey planning, etc.

odata3

  • re-opened by solution as administrator so I could use the Azure Cloud tools
  • added a Windows Azure cloud project with no roles and added my existing WCF Service project to the roles
  • tested OData access running in the local Azure Development Fabric connecting to my local SQL Server database
  • created a SQL Azure server and database in the cloud using the SQL Azure portal
  • connected to SQL Azure using SQL Server Management Studio and created my database schema using a modified version of my existing DDL script (removing USE statements)
  • used the bcp utility to export tables from my local database to local files and then again to import those up to SQL Azure
  • updated the connection string in my web.config to use the SQL Azure database
  • tested OData access running in the local Azure Development Fabric connecting to the SQL Azure database
  • published by cloud project in Visual Studio, created a Hosted Service on the Windows Azure portal and deployed the application to Azure using the portal
  • tested OData access running in on Windows Azure connecting to the SQL Azure database

So I had an OData service exposing the OCTranspo bus routes, trips, stops and trip stop times – sweet!

When I got to the event at 3:15pm, I met up with Christian, Rick & John Weigelt (Microsoft Canada’s National Technology Officer).  They were looking at the VanGuide application and potential Ottawa data sources to hook up to it as a demonstration.  Demonstrations were due at 4pm.  Of course, as I’ve stated, good data sources were hard to find because the City of Ottawa hadn’t published any official open data sets.

OData service to the rescue…?

We set to work on integrating my sample OCTranspo OData service (which was already running the cloud), into the the Open Data Application Framework, transforming the VanGuide application into an ‘OttGuide’ application.  The framework includes a Silverlight application project that includes dynamic handling of web-based data sources (the dynamic list of landmark sources on the left of the UI) and asynchronous data retrieval and handling code (that takes KML and other format data and adds data points as map points on a Bing Maps Silverlight control).

We wanted to make this Ottawa specific, wo we went ahead and:

  • modified the data source list to show an entry for OCTranpo and removed the other ones
  • removed other Vancouver-specific map region options
  • added custom code to handle OCTranpo GTFS entities in a feed (our OData feed) using Linq to XML to create Landmark instances that the framework already understands
  • made the application zoom in on Ottawa
  • made some other cosmetic changes for Ottawa

The result is the application you see at the start of this blog entry.  For full disclosure, we got the full bus stop retrieval and map point creation working literally 1 minute after the public presentation section finished, but Christian managed to demonstrate the VanGuide version and the OData feed.

The ODAF framework means that an OttGuide application user can select bus stops, rate them, add comments, etc.  It would be relatively easy to add datasets for other ‘landmarks’ (e.g. water foundations, tourist spots, parks and even pot holes).

The same OData service could be used on clients like the new Windows Phone 7 platform to give bus services users valuable travel information :).  Perhaps more on that later…

Open data and open web technologies are current hot topics and OData is a great protocol to bring things together on many platforms.  Silverlight is a great technology to deliver rich user experiences on many platforms using data sources from a variety of platforms. 

Microsoft and Open Source Communities are sponsoring the Make Web Not War conference on May 27th 2010, showcasing the latest cross-platform techniques and technologies, including presentations, panels, workshops, a codefest, the FTW coding competition and a party!  Open Data, HTML 5, PHP, JQuery, Mobile, SEO and the Cloud, are amongst the topics being covered.  Speakers and panelists attending include myself, Microsoft Canada team members and many open source experts.

Make Web Not War

Advertisements

Google could win at least the Mobile Consumer Space

If you look at Microsoft’s Project Natal, you know that Microsoft is trying to go after the rest of the family in the gaming space.  Once those people become comfortable playing games like raggedy dolls 😉 they’ll be comfortable using whatever entertainment or service Microsoft provides on the box.

It goes without saying that Microsoft is big in business and will likely continue to be but that focus may continue to be their undoing in other market segments – just look at Windows Mobile.  Take the consumer who is buying their first or next mobile device and just moving into social networking or electronic communications or those that currently have no brand loyalty.  Here, Google could gradually and quietly take over from Symbian, Apple and Microsoft.

Google Wave + Google Search + other Google applications on an Android-based phone, a mini running Android or even on any other low-cost device with a browser, could be a winning formula and all that any social networking consumer needs.

While Microsoft will dominate business, gaming and home entertainment, Google may well end up dominating most of the mobile consumer space (with a little work on the UI – and imagine if Google and Adobe got together…).

Microsoft needs to come out with a Windows Mobile device and fast – like this year.  It needs to be a .NET-based OS and have a flourishing and up-front application/music market place.  That means dismissing their hardware partners and bring out a cheap Zune phone (while extending Game Studio Express to be App Studio Express) – it’s painful to other but it’s the only real way for Microsoft not to lose this space altogether, and not to Apple, but to Google.

Windows Live Mesh Gives Legs or Wheels to Microsoft Sync and Auto PC

Microsoft Sync has appeared in some Ford vehicles and is apparently coming to at least two other manufacturers soon.

It amazes me how utterly appalling the uptake is of Windows in the car industry.

Microsoft Sync with a Microsoft’s Live Mesh client opens up the ability to take contacts, music, continuous user experiences (such as phone calls, paused music and podcast bookmarks), to your car.

Combine that with a ‘Windows Live PC’ running on an ‘Xbox portable’ or Zune in the mesh as I’ve mentioned in this series of posts on Live Mesh, and you can really see the magic of software plus services coming together for a seamless user experience.

A ‘Windows Live PC’ gives the UMPC, ‘Microsoft PC’ or Xbox Portable a Future

In this series of posts I’ve talked about my concept of the ‘Windows Live PC’ as the trojan strategy in Microsoft’s Live Mesh.

I’ve talked about how such a virtual PC could be available on an Xbox 360, a Mac or other platforms.

One of problems with the Microsoft UMPC initiate has been that cost of PC capabilities in a small form-factor, and the need to up the component cost to provide Vista in that form factor.  This has made many UMPCs (so far built not by Microsoft, but by IHVs) more expensive than many notebook computers and with less power at the same price.

With my concept of the ‘Windows Live PC’ and minimal SSD storage, the UMPC could stop growing in power (and energy consumption, resulting in longer battery life) and just turn into a ‘Windows Live PC’ client.

In previous posts I suggested that such a client doesn’t have to be very powerful.  I also said that the xbox 360 is good enough.  In fact the original xbox is likely good enough too in many ways – even perhaps a PS2 or PS3!!

How about a PC the size of a Mac Mini or the size of a Zune?

What if Microsoft sold its own UMPC with SSD storage, the form-factor of something like a Samsung Q1 Ultra but not much processing power – how about an Xbox portable?

An Xbox portable would be the ultimate convergent future of Live Mesh, Xbox, Xbox Live, ‘Windows Live PC’, Xbox portable, WPF, Remote App, Windows Server 2008, Windows licensing, ISV solution channel, etc. 

Robbie Bach, J Allard, Ray Ozzie, Bob Muglia, Steve Ballmer & Bill Gates – take a look at this series of posts on Live Mesh – I know what you’re up to 🙂 and if you’re not then you should be – it’s a vision I want to be involved in one way or another from the outside or the inside…

Add a ‘Windows Live PC for Mac’ to your Live Mesh with Silverlight

In this series of posts I’ve introduced the idea of a virtual ‘Windows Live PC‘.  I’ve talked about how your Xbox could be the ubiquitous PC in your household without any software application installations, thanks to a potential expansion of the currently disclosed Web Desktop (storage service) in Microsoft Live Mesh, with the addition of RemoteApp from Windows Server 2008.

Silverlight 2.0+ is the SUPER TROJAN HORSE onto the Mac, Linux and I believe there could be more platforms to come (see next post…)

Some developers may already be wondering why they should bother with HTML, AJAX, DOMs, DHTML, Javascript, etc. now that they can provide a hugely rich WPF Windows application in a browser using their existing .NET skills (plus WPF), and when that browser can be IE and Firebox on Windows, Safari on Mac, and whatever it is on Linux, without any of the nightmare that cross-browser standards-compatibility creates.

In previous posts I said that my notional ‘Windows Live PC’ will run (via RemoteApp) on anything that can handle the necessary technology stack with the xbox 360 being more than enough.  It seems to me that Silverlight 2.0 (or perhaps a later interation) could easily talk the Remote Desktop protocol.  Once that happens Microsoft can be selling you a ‘Windows Live PC’ subscription on your Mac and all those Windows-targeting ISVs can now license their product onto a Mac or Linux!!!

Note that there is already a Remote Desktop Client for Mac, but with the potential for Microsoft to offer a virtual ‘Windows Live PC’ running full screen, the Mac could fade into just a remoteapp client to a ‘Windows Live PC’ albeit a great new channel for the Windows and ISV software licensing – now that’s what I call leveraging!