Saturday, 11 June 2011

Introducing the MapGuide Instant Setup utility

As someone who is getting more invovled with core mapguide development, having to build and run the installer (which may not even work properly and may pollute my registry with junk settings) just to be able to test my freshly built trunk working copy is a time-consuming process.

What would be really nice is to have some kind of tool/utility that allows me to instantly configure the freshly built source (that is prepared before building the installer) and have a functional server/webtier setup, completely bypassing the installer step.

Enter the Instant Setup utility

This utility requires building the source and preparing it. For a recap, this is:
  1. MgDev\build.bat -a=build -w=all
  2. MgDev\build.bat -a=install -w=all -o=C:\Staging\
  3. Installer\build.bat -a=prepare -source=C:\Staging\
These 3 steps will set up the built binaries (and Apache/Tomcat/PHP with templated config files) under C:\Staging\. This is where the Instant Setup utility comes in.

For the Source Directory enter C:\Staging\ (or whatever directory you chose instead)

For the Batch File Output Directory enter the directory where this utility will generate batch files to start the server and web tier in interactive mode.

The MapGuide Service Name is the service name that will be used when the configured MapGuide Server is installed as a service (not used at the moment).

Then comes the detailed bits:
  • The port numbers where the the configured server will use
  • The virtual directory the configured web tier will exist under
  • The port number where the configured apache will use
  • The service name to use if apache is installed as a service (not used at the moment)
  • The default viewer to use for the mapviewerajax virtual directory
  • Whether to enable Java, and what port to use for Tomcat if this is the case.
Once you have specified all this information, hitting the Configure button will do the following.
  1. Write your specified values into the templated Apache/Tomcat/PHP configuration files
  2. Write 2 batch files (mgserver.bat and mgwebtier.bat) to start the MapGuide Server and Apache both in interactive mode.
Once configured, your C:\Staging\ directory is now a functional MapGuide Server/Web Tier installation, albeit only in interactive mode at the moment.

Some may have noticed that I haven't mentioned IIS. That's because this support is not implemented yet. Eventually support for that will be added. For now, this will give you a functional Apache/PHP/Tomcat/Java configuration.

The source for the instant setup utility can be found here. I hope this tool simplifies your MapGuide hacking experience, as it did for me.

Monday, 6 June 2011

Introducing the (re-vamped) Maestro API: Part 2

For part 2 of this series I am going to talk about the overall design of the Maestro API.

Having used FDO for quite some time now, a lot of its API design has influenced my design of the Maestro API. Those who have used FDO may notice many parallels in the design when working with the Maestro API.

The first similarity is the heavy use of interfaces. FDO models its connections and commands via interfaces. In the Maestro API, we model the following in the same manner using interfaces:
  • MapGuide Server connections
  • MapGuide Service APIs
  • MapGuide Resource classes and message types
By having these types as interfaces, it allows the implementation of these interfaces to be totally different, and I will show later on in this post where having this quality is extremely beneficial.

The MapGuide Server connection is represented by the IServerConnection interface. This is the main point of entry in the Maestro API.

This contains the ResourceService and FeatureService interfaces to provide resource manipulation and feature querying capabilities. Additional MapGuide services are available via GetService method and you can interrogate the Capabilities property to determine what you can and can't do with this particular connection.

Now the reason this is an interface is because in the Maestro API, there are currently 2 implementations of this connection:
  • One that communicates with the mapagent via http. (Maestro.Http)
  • One that wraps the official MapGuide API. (Maestro.LocalNative)
Because this is an interface, the client application can use the same code regardless of implementation. As there is an implementation that wraps the official MapGuide API, you can even use MapGuide Maestro without even needing a web tier because MapGuide Maestro works against the interfaces defined in the Maestro API.

Now given that IServerConnection is an interface, how does one go about creating instances of it? Enter the ConnectionProviderRegistry

Simply call CreateConnection passing in the desired provider and the parameters needed to initialize it. The ConnectionProviderRegistry reads from a ConnectionProviders.xml file which contains all the registered connection providers. Each connection provider is an implementation of the IServerConnection interface.

So having acquired a connection, we can now interact with the services provided by MapGuide. Once again, these services are modeled as interfaces

Each service is defined in a corresponding interface. For reference, they are:
  • IFeatureService
  • IResourceService
  • IDrawingService
  • IMappingService (also contains rendering service APIs)
  • IFusionService
  • ITileService
  • ISiteService
Each service interface contains methods which correspond to what you see in the mapagent test page. These services are obtained via the GetService method on IServerConnection. For IResourceService and IFeatureService, these already exist as properties so you can access them directly. Once again because these are interfaces, the implementations can be wildly different:
  • For the http connection provider, these interfaces wrap http requests to the mapagent and automatically processes the XML responses from them.
  • For the local connection provider, these interfaces wrap the offical MapGuide service classes (eg. The implementation of IFeatureService wraps the methods of the MgFeatureService class, and does automatic conversion of MgByteReader values)
Finally, there is MapGuide's resource classes and message types. The MapGuide Server returns many different types of XML data. But with the Maestro API, instead of working with raw XML content and having to write a lot of boilerplate code, you get to work with strongly typed classes and interfaces, with XML serialization automatically handled for you

Now there's a lot of XML Schemas for all the various forms of XML data that MapGuide returns. Actually building the classes that represent this data by hand would've taken an eternity, so I've taken a shortcut for this particular case and used the xsd2code code generator to generate all the required classes from their XML Schemas. I used xsd2code for generating the classes instead of using the default xsd.exe tool for reasons I have already explained. The set of generated classes produced by xsd2code a much cleaner (relative to xsd.exe anyway) and gave me a very useful foundation to work with.

In the case of resource classes, using the generated classes as-is was still not good enough. Additional tweaking of this generated code was required. The reason is that each particular version of a resource was its own separate class when generated. Each particular version of a resource in its generated class form had nothing in common. Once again, interfaces come to save the day.

Now you may notice that there is only one interface for Layer Definitions, but there's been 3 additional revisions to date in the Layer Definition XML schema (ref 1, 2, 3, 4 and 5). So how were 4 separeately generated classes able to be reconciled into a single interface?

One of the fortunate things thus far, that Autodesk did when they introduced a new version of a given resource was that the schema revisions have been incremental and additive. Nothing is actually removed or altered in every schema revision thus far.

Because of this, the design of these interfaces is such that they represent the 1.0.0 version of the respective resource type each interface is modeling. Elements introduced in newer versions of the schema are exposed through a new version of the interface that extends the original.

So with this design, we have a consistent baseline interface for all resource types and we can interrogate the resource version to determine if we can cast these interfaces (or its child properties, which are also interfaces) to the newer versions to tap into the extra functionality exposed in these newer interfaces.

As is the case with the Web Layout in the above diagram. IWebLayout2 extends IWebLayout and maps to the 1.1.0 version of the Web Layout schema that introduced a new "ping server" property. However both can be accessed via the IWebLayout interface. We can check if the resource version is 1.1.0 to determine if it can be cast into the IWebLayout2 interface.

It is through this design that the Resource Editors in MapGuide Maestro are able to edit most, if not all known versions of a given resource type, and have the necessary intelligence to enable/disable certain UI elements for elements not supported by the edited resource version. This design allows Maestro to keep up to date as newer resource types or schema versions are made available, providing that these schema revisions stay true to their additive form.

This incremental quality of the XML schema revisions is also reflected in the generated code. Thus with the magic of partial classes and careful use of the C# pre-processor, getting different versioned classes to implement the same baseline interface was actually an easy task. Then it was a case of extracting the newer properties and methods introduced in newer schema versions into their own separate interface that extends the original.

The end result, is a consistent object-oriented API that is (mostly :) ) free of boilerplate code that you would normally write to deal with raw XML content, MgByteReaders and HTTP requests/responses. Any developer who has worked with the official MapGuide API would appreciate

So that's the high-level overview of the design of the Maestro API. Stay tuned for part 3, where I actually show some concrete examples of how to use this library.

MapGuide Application Development tips

Sometimes on the mapguide-users mailing list, we get questions that make me scratch my head sometimes. These are questions that can answer themselves with the use and knowlege of the right tools and configuration settings.

So giving the benefit of the doubt regarding programming ability, and simply assuming the case of "not knowing what's out there", here's the tools and configuration settings that I always use when developing MapGuide applications, which should greatly assist and simplify the process of developing and debugging MapGuide applications. Experienced web developers can probably tune out because you should already know these tools :)

1. Firebug

If you haven't heard of firebug or haven't used it before, stop reading this post and download it now. DO IT!

If firebug didn't exist, I probably wouldn't be taking Web Development as seriously (hell, I probably wouldn't do web development full stop!). If you do any development on the MapGuide Platform (or web development in general), you will have to write some javascript and firebug provides top-class debugging facilities.

But it doesn't just do javascript, it also does:
  • CSS/DOM debugging
  • HTTP request tracing / performance monitoring. (you can see all the requests that Fusion/AJAX viewer is making to your MapGuide Web Tier)
  • Lots more!
It is the complete web development package! Of course, you should already know that (and I shouldn't even have to be writing this post!)

2. Mozilla Firefox

Besides the fact that this is needed for firebug, Firefox is the epitomy of the modern web browser, and has lots of healthy competition from Google Chrome and other WebKit-based bretheren to help keep moving the web forward ... Unlike that browser from Redmond^!

But even still, the Firefox + Firebug combination provides a solid foundation for your design ensuring that the time spent in Internet Explorer is kept to a minimum*

My web development workflow is simply:
  1. Design and develop and test in Firefox / Firebug. This design will 99.9% work in all modern browsers.
  2. Contort and bastardise this design (hopefully not too much) to make it work in Internet Explorer
  3. Re-test against Firefox / Firebug to ensure no breakages.
Simply put, the Firefox + Firebug combination significantly reduces the time spent in the 2nd step of this workflow. The amount of pain and suffering (and the amount of swearing directed at Redmond) would be kept to a minimum.

3. Baretail + MapGuide log files

Believe it or not, MapGuide actually logs lots of information when its running (under default server configuration settings). If a viewer has made a request to the MapGuide Server, it is logged. If MapGuide throws an error, it is logged. If something has made the MapGuide Server fall over, chances are it is logged. Simply put, the logging facilities built into MapGuide provides you with a lot of information to work with. When things go south in your MapGuide application, the log files should almost always be the first port of call.

Baretail is a windows utility that is effectively the windows equivalent to the unix tail utility, showing you the latest entries in error.log and access.log (and any other log file) as they are written by the MapGuide Server. It is an invaluable resource to assist in debugging and problem diagnosis. Does your MapGuide application code not work? Try running it again while observing the mapguide logs under baretail. They might tell you something.

4. Set display_errors = On in php.ini

The PHP that is bundled with MapGuide uses production-level settings. This means any warnings or errors (like unhandled MapGuide exceptions) in any PHP code you may be running are silently swept under the carpet most of the time. Setting the display_errors setting to On in php.ini will reveal and display these errors. If PHP is your development environment, this should be the first thing you do on a fresh development install of MapGuide.

5. JavaScript lint

Sometimes I wish JavaScript was a compiled language. That way the theoretical javascript compiler can report to me all the syntax errors it finds in my javascript source code immediately instead of me having to wait until runtime to get a nasty surprise!

Well JavaScript will never become a compiled language, but we do have JavaScript Lint which is the closest thing we will ever get to a "compiler". Usage is simple, run the jsl.exe executable on your javascript source and watch it report any javascript syntax errors with matching line numbers!

Knowing these tools and configuration settings have made my web development life simpler, I hope this post makes your's simpler too!

^ Generally speaking, where IE gets it completely wrong is in two areas: 1) Certain parts of the DOM API and 2) Flagrant disregard or misinterpretation of certain CSS rules (most infamously, the box model bug). But it's 2011, these are known problems solved by almost every web framework out there, which you should be using already! Which is why I said that the Firefox + Firebug combination takes you 90% there.

* I think if all of us web developers charged a mandatory "IE tax" on our web development work, it would make some of these regressive organisations think twice about holding us (and the modern web) back by still mandating Internet Explorer 6 as the "standard web browser". Microsoft got the message, why won't you!? To be fair, none of the clients I have worked with in my career thus far fall under this category, but the fact that there are still such organisations out there makes baby Jesus cry! At least IE9 is a half-modern web browser, and if Microsoft is serious about its vision of application development for Windows 8, they have no choice but to make IE10 an actual functional modern web browser!