Saturday, 9 September 2017

Not all quiet on the MapGuide Maestro front

Just because MapGuide Maestro is a windows-only .net application does not mean the Maestro API driving it (and your own custom MapGuide applications using the Maestro API) need to bear the same platform constraint.

We have .net Core to run .net code cross-platform. We have .net standard as a specification for building truly cross-platform .net class libraries. The API surface used by the MaestroAPI highly intersects with what is provided by .net standard. All the signs say that we have the means to make MaestroAPI a truly cross-platform .net standard compatible class library, and that we should probably do it.

So since the last release, my main focus has been to port the Maestro API to target .net standard 2.0, thus expanding its potential reach beyond full .net Framework on windows.

It is most encouraging to see this on a Linux terminal after all this effort.



The tale of getting to this point is worthy of a blog post itself. The internet could do with more .net standard porting stories.

Friday, 8 September 2017

Announcing: mapguide-react-layout 0.10

Here's a new release of mapguide-react-layout with a metric bucketload of new features.

QuickPlot now feature complete

As blogged previously, the final missing piece of the QuickPlot (capture box rotation) has been implemented, bringing this component to feature parity with the Fusion original. The rotation being controlled through a new numerical slider in the QuickPlot UI


Selection Panel sub-selection

Through creative use of QUERYMAPFEATURES, the Selection Panel can now support highlighting selected features within a selection set. Useful for visually sifting through big selections.



Persistent Load Indicator

A user raised a very valid point that if you don't have the navigator (aka. Zoom slider) active, there is no actual load indicator visible as the only place we have a load indicator is on the navigator component itself.

This release adds a persistent load indicator that is represented by a thin blue animated bar at the top.

Blink and you might miss it in the GIF below.


Coordinate Tracker

This release includes the missing Coordinate Tracker widget.


Init Warnings

Any warnings that we encounter during viewer initialization are now collected and displayed at the end. Here's some warnings that could be shown.



API enhancements

  • NPM module: Now supports custom redux application state and reducers. Check out the updated example to see how this is done.
  • NPM module: Selection Panel supports custom rendering of the body when a selected feature is to be displayed (if you don't like the default table-based attribute display)
  • NPM module: There is now a flat "mapguide-react-layout" module where you can import everything from, instead of having to import everything piecemeal.
  • Browser Globals: You can now specify locale as a mount option
  • Browser Globals: You can now specify a post-init hook function as a mount option
  • Browser Globals: Redux getState and dispatch functions exposed through MapGuide.Application class
  • Browser Globals: All dispatchable redux actions available under MapGuide.Actions namespace
  • Browser Globals: You can now retrieve and invoke registered commands through getCommand()
Other Changes
  • OpenLayers updated to 4.3.2
  • Blueprint updated to 1.27
  • Now built with TypeScript 2.5
  • Removed es6-promise from the viewer bundle. All template HTML files script include the es6-promise polyfill separately so that the viewer will continue to work in Internet Explorer
  • Ported across the view size status bar component
  • Full extension property support for the CursorPosition widget


Project Home Page
Download
mapguide-react-layout on npm

Monday, 28 August 2017

React-ing to the need for a modern MapGuide viewer (Part 19): Highlighting selected features

When I was implementing the Selection Panel for mapguide-react-layout, this super-ancient Fusion ticket was on the back of my mind as being able to highlight selected features is a very useful feature to have.

Whereas I didn't really have an idea how this would be done in Fusion, for mapguide-react-layout I had an idea how this would be done.

The trick is to use the v2.6 QUERYMAPFEATURES operation with the following parameters:

  • REQUESTDATA=2 (asking for inline selection image only)
  • LAYERATTRIBUTEFILTER=0 (so that selection is not constrained by layer selectability and our current view)
  • FEATUREFILTER=(the selection XML sub-fragment that represents the selected feature)
  • PERSIST=0 (the most important parameter. We want this operation to not alter the selection set)
Once the QUERYMAPFEATURES operation is sent and we get a response, the key is to be able to line up the inline selection image (data URI) with the current map. To handle this part, we use an OL static image source to store the inline selection image.

When all the pieces are brought together, we finally have the ability to highlight selected features.


This will be in the next release of mapguide-react-layout.

Thursday, 10 August 2017

React-ing to the need for a modern MapGuide viewer (Part 18): Restoring the Quick Plot capture box

If you've been playing with mapguide-react-layout, you'll no doubt have noticed a glaring omission in the ported Quick Plot component.

The interactive map capture box.


Porting across the capture box sounded mostly simple:

  • Manage the temporary OL vector layer containing the capture box.
  • Attach an OL translate interaction so the capture box feature can be easily moved around by dragging the box around.
  • Auto-update box geometry based on change in paper size / orientation / scale
There was just one technical hurdle: Rotating the box.

For the longest time, I was wracking my brain figuring out how to replace the rotation "grip handle" using the new OpenLayers APIs and I couldn't figure it out. The APIs were there to rotate feature geometry. I heard about ol-rotate-feature and gave it a try, but I couldn't grasp how that interaction actually rotates features.

So I went back to the mental drawing board. OpenLayers has APIs to rotate features, so what we really just need was an intuitive UI input to enter the box rotation. If I couldn't figure out how to replicate the rotation "grip handle", then the next best thing was ... a numerical slider.

And with that, the missing piece was found and I was finally able to port over the final piece of functionality of the original Fusion QuickPlot widget.


In the above GIF, you might have noticed a message flash by.


This is merely a simple countermeasure against the fact that with current OpenLayers, you can rotate the map as well. So rather than deal with number crunching 2 sets of rotations, it is easier to just reset the rotation on the map and prevent the ability to rotate the map while the map capture box is active.

And with that, our Quick Plot component has reached feature parity with the original Fusion widget.

This (newly feature parity) Quick Plot will be available in the next release of mapguide-react-layout, whenever I decide that will be.

Wednesday, 2 August 2017

Announcing: mapguide-react-layout 0.9.6

This is a quick release to make the awesome new layer transparency sliders in the previous release not break down when used on a map that doesn't actually have external base layers (OSM, Bing, etc)

Project Home Page
Download
mapguide-react-layout on npm

Thursday, 27 July 2017

Announcing: mapguide-react-layout 0.9.5

This was originally going to be versioned 0.9.2, but the volume of changes was too big to be a bugfix-level single patch version bump, but at the same time, it was also not enough to warrant a minor version bump, so I decided to go half way with 0.9.5.

Here's what's new in this release.

Toggle-able Layer Transparency

The viewer options UI is now fleshed out to allow you to toggle transparency of:

  • The MapGuide map (and any tiled layer groups)
  • The MapGuide selection overlay
  • External Base Layers
To illustrate this, here's a self-explanatory GIF



This allows one to easily compare the MapGuide map against its base layer backdrops without requiring actual visibility toggling.

And yes, this works even on IE (11, the only version I care to support)

Sprite Icon Support

This release now supports the standard Fusion icon sprite. This will no longer load the individual icons for commands and widgets if it is clear they are referencing the standard icon sprite.


Targeted Command Support

If a command or widget requires execution in a New Window or a specific frame, the viewer will now support it. Note that if a command or widget is set to execute in a New Window, we won't actually spawn a new physical browser window, we'll run it in an iframe inside a BlueprintJS dialog component instead.

Other Changes/Fixes
  • Added support for extension properties for Buffer, FeatureInfo, Query, Search, SelectWithin, Theme.
  • Fixed Fusion MapMessage bar emulation
  • Fixed tooltip queries not being sent with pixel-buffered polygon geometries
  • Fixed zoom requests not snapping the scale to the closest finite list if viewing a tiled map
  • Legend now properly renders layers with multiple geometry styles
  • Fix excessive BlueprintJS toaster components being created and not cleaned up
  • Fix flyout menus requiring double-click to re-open (after clicking a menu item inside the first time)

Friday, 21 July 2017

So ... where's MapGuide Open Source 3.2?

Here's the story, since I gather not everyone reads the mapguide-users mailing list where I mentioned this subject many months ago.

I've decided (many months ago) to skip on making this release.

The differences between 3.1 and (a 3.2 release if I had decided to make one) is so small that it isn't worth investing the build resources on a 3.2 release cycle.

Since I'm skipping on a 3.2 release, it means that we have a good year-long window of solid development time to get some compelling features into the release after it (currently set as 3.3). Some of this development work is already starting to bear fruit.

Now that's not to say there isn't going to be a MapGuide release sometime between now and when 3.3 is out. I still do hope to put out the (hinted previously) patch releases for MGOS 2.6, 3.0 and 3.1 in between, but that requires me rebuilding my build infrastructure first and that is currently taking a back seat to landing some solid features into 3.3 first, so that's where things are at.

And as always. As these features land, you can expect this blog to talk about it.

Sunday, 9 July 2017

React-ing to the need for a modern MapGuide viewer (Part 17): Reason number 5537485 why react was the right choice

An issue cropped up where the legend was not properly rendering a given layer that has multiple geometry styles. This issue was easily reproducible with the Redline widget.

We were expecting to see this after creating a redline layer and drawing some objects.


But we got this instead


Because this legend is a react component, we can inspect it (and the problem layer node) with the React developer tools


Remember the important React motto: The UI is a function of props and state. The HTML content of the LayerNode should be reflective of the props and state given to it. We should've seen something that resembled 3 style icons. But nothing's there.

So let's just check that the layer model for this LayerNode component is indeed a layer with multiple geometry styles


Indeed it is, so that means that the LayerNode component is the culprit. It is not handling the case of multiple geometry styles properly.

As we've already set up our test infrastructure to make it easy to write and run tests, it should be easy to write up an enzyme unit test that shows what we were actually expecting to see when a LayerNode renders a layer that has multiple geometry styles

component.legend.spec.tsx


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
import * as React from "react";
import { shallow, mount, render } from "enzyme";
import { MapLayer } from "../src/api/contracts/runtime-map";
import { LayerNode } from "../src/components/legend";
import { ILegendContext } from "../src/components/context";

// Mocks the ILegendContext needed by LayerNode and other legend sub-components
function mockContext(): ILegendContext {
    return {
        getIconMimeType: () => "image/png",
        getStdIcon: (path: string) => path,
        getChildren: (id) => [],
        getCurrentScale: () => this.props.currentScale,
        getTree: () => {},
        getGroupVisibility: (group) => group.ActuallyVisible,
        getLayerVisibility: (layer) => layer.ActuallyVisible,
        setGroupVisibility: () => {},
        setLayerVisibility: () => {},
        getLayerSelectability: (layer) => true,
        setLayerSelectability: () => {},
        getGroupExpanded: (group) => true,
        setGroupExpanded: () => {},
        getLayerExpanded: (layer) => true,
        setLayerExpanded: () => {}
    };
}

describe("components/legend", () => {
    it("renders a multi-geom-style layer with a rule for each geom style", () => {
        const layer: MapLayer = {
            Type: 1,
            Selectable: true,
            LayerDefinition: "Session:841258e8-63f9-11e7-8000-0a002700000f_en_MTI3LjAuMC4x0AFC0AFB0AFA//testing.LayerDefinition",
            Name: "_testing",
            LegendLabel: "testing",
            ObjectId: "abcd12345",
            DisplayInLegend: true,
            ExpandInLegend: true,
            Visible: true,
            ActuallyVisible: true,
            ScaleRange: [
                {
                    MinScale: 0,
                    MaxScale: 10000,
                    FeatureStyle: [
                        {
                            Type: 4,
                            Rule: [
                                {
                                    Icon: "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABHNCSVQICAgIfAhkiAAAAB1JREFUOI1j/M/A8J+BAsBEieZRA0YNGDVgMBkAAFhtAh6Zl924AAAAAElFTkSuQmCC"
                                }
                            ]
                        },
                        {
                            Type: 4,
                            Rule: [
                                {
                                    Icon: "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABHNCSVQICAgIfAhkiAAAACVJREFUOI1jYBgFwwAwMjD8bcAjL8rAwKCNR56LibruGQVDFAAACkEBy4yPOpAAAAAASUVORK5CYII="
                                }
                            ]
                        },
                        {
                            Type: 4,
                            Rule: [
                                {
                                    Icon: "iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAABHNCSVQICAgIfAhkiAAAAFhJREFUOI3t0D0OQFAUROHPIwoNS7ZJe5BoKJR+OpVH4jUkTjv3TDI3w6Z2zooNeSSfKNQYIwd3NISH6sFfQLCkFmQJdkVIGlG+44mfLyjMaCPpgO7C7tkBAXgKXzBhmUQAAAAASUVORK5CYII="
                                }
                            ]
                        }
                    ]
                }
            ]
        };
        const wrapper = shallow(<LayerNode layer={layer} />, {
            context: mockContext()
        });
        const rules = wrapper.find("RuleNode");
        expect(rules.length).toBe(3); //One for each geom style
    });
});

Running this in jest confirms our expectations were not met:

Summary of all failing tests
 FAIL  test\component.legend.spec.tsx (6.75s)
  ● components/legend › renders a multi-geom-style layer with a rule for each geom style

    expect(received).toBe(expected)

    Expected value to be (using ===):
      3
    Received:
      0

      at Object. (test/component.legend.spec.tsx:149:30)
      at Promise.resolve.then.el (node_modules/p-map/index.js:42:16)
      at process._tickCallback (internal/process/next_tick.js:103:7)


Test Suites: 1 failed, 22 passed, 23 total
Tests:       1 failed, 94 passed, 95 total
Snapshots:   0 total
Time:        14.532s
Ran all test suites.

We expected 3 RuleNode components (one for each geometry style) to have been rendered, but we only got nothing.

A look at the LayerNode rendering shows why. It only considered the first feature style of any layer's scale range.

So once that was fixed, not only does our test pass, but we have visual confirmation that multi-geometry-style layers now render like they did in the Fusion and AJAX viewers.


So the reason for writing this post was just a re-affirmation of my choice for using React to build this viewer.

  • The top-quality developer/debugging experience.
  • The react way of thinking about UIs that allowed me to easily identify the culprit (the LayerNode component)
  • The top-quality testing ecosystem around React (Jest, enzyme) that allowed me to easily write a unit test on this component to confirm and verify my expectations

Friday, 30 June 2017

Announcing: MapGuide Maestro 6.0m7

So you've downloaded the new build of Fusion for your respective version of MapGuide Open Source so you can use the current iteration of Bing Maps as the v6/v7 versions of Bing Maps that Fusion was using will be shut down hours after this post is published.

But this iteration of Bing Maps now requires an API key to use it. So you sign up for an API key in the bing maps portal. Now where you do put this thing?

Well, here's a new release of MapGuide Maestro to help you with this very task among some new features and fixes.

Updated Fusion editor for Bing Maps

The fusion editor has been updated to work against the current iteration of Bing Maps. It has a new field for you to put your Bing Maps API key


When you put in the key and clicked Set API key it will write the API key into the top-level BingMapKey extension element of the Application Definition XML.


Also the available bing layer list has been updated to match what the current iteration provides.


The only difference is that the base layer formerly known as the "Hybrid" base layer has been renamed to "AerialsWithLabels". So if you use the bing hybrid layer you need to replace it with AerialsWithLabels.

This release of Maestro also includes new validation rules for Application Definitions to catch these new Bing Maps issues.


Auto-Configured Spatial Contexts in WMS Configuration Documents

In my adventures of consuming WMS services in MapGuide and building the initial WMS configuration document, I've been consistently observing a pattern like this:


The pattern being, that the names of the discovered spatial contexts usually follow the pattern of "EPSG:XXXX", yet we had to then go and manually fill in the correct coordinate systems one by one, even though the EPSG code is right there in the name!

With this release, we now automatically infer and fill in such coordinate systems for you



Other Changes

  • Fixed inability to preview maps using the local viewer if the map uses a transparent background color
  • Refine the error message when attempting to calculate meters-per-unit in MgCooker and you have no map or tile set selected. Selected is a misnomer, you have to actually check the respective tree node, and the error message has been updated to reflect that.
  • Package updates to DockPanelSuite and Newtonsoft.Json


Download

Using Bing Maps? Here's some new builds of Fusion.

I gave out the warning last month, and just in time before Microsoft pulls the plug on the Bing Maps legacy controls and services on June 30th 2017, here's some new builds of Fusion with support for the current iteration of Bing Maps.

The current iteration of Bing Maps now requires an API key, so if you don't have one you can signup for one at the bing maps portal.

I don't normally put out releases of Fusion like this. I'd normally would just roll it into a new point release of MapGuide Open Source, but I'm currently not in a suitable position at the moment to put out new builds of MapGuide, so this will have to suffice.

Download Fusion for your particular version of MapGuide here:

Fusion for MGOS 2.6

Fusion for MGOS 3.0

Fusion for MGOS 3.1

To install, just extract the zip file contents to the www directory of your MapGuide installation. Be sure to back up or remove the existing fusion directory before doing this.

Now, having not put out a new release of MapGuide for a while, there are also plenty of fixes and minor enhancements made to these respective branches of Fusion since the version bundled with their respective released version of MapGuide. You can check out the changelogs for these respective branches here:

Fusion for MGOS 2.6 Changelog

Fusion for MGOS 3.0 Changelog

Fusion for MGOS 3.1 Changelog

So you've got your Bing Maps API key, where in the Application Definition do you put this key in?

Maybe you want to check out the next post.

Monday, 26 June 2017

Announcing: mapguide-react-layout 0.9.1

This is a small bug fix release that brings updates to key libraries we use:

And includes the following fixes:
This release also has one small enhancement, the viewer now supports a mount option for specifying initial visibility of template elements (Legend, Task Pane, Selection Panel). Currently this will have the most impact when used with the Aqua and Sidebar viewer templates.

For other viewer templates, this new feature won't have much effect yet. For example, take the Slate template.


The very design of this template means that if you wanted all Legend/TaskPane/SelectionPanel to be hidden initially, you just cannot do it. Because they all reside in an accordion, one of these elements has to be visible. The original Fusion templates (at least slate from my memory) could conceivably have Legend/TaskPane/SelectionPanel all hidden because its sidebar was collapsible. That is not the case in this iteration of Slate (and the other ported templates with a sidebar). Having collapsible sidebars in these viewer templates will be a problem a future release will be fixing.

Finally, one thing that was actually possible in the 0.9 release, but I didn't really mention because I wanted to make sure this feature was working before actually announcing it is that Script Commands which is the InvokeScript replacement in mapguide-react-layout can be used with the standard viewer bundle. You no longer need to exclusively use the npm module to have support for Script Commands.

You can see an example of script command registration using the standard viewer bundle here.

Saturday, 3 June 2017

Look ma, no tooltip requests!

So what does tiles of ascii-like content give us?

UTFGrid tiles allows us to pre-render tooltip interaction data. Because we're pre-rendering such data, no mapagent requests are ever made, we can fetch for such tiles just like other map image tiles.

If you remember my previous notes about MapGuide scalability, session repositories add memory and administrative burden/bookkeeping to the running MapGuide Server. If you can get away with not having to create a session repository and keeping it alive, you should. Currently, tooltip requests sadly, require a runtime map (thus, requiring a session be created first and the initial runtime map state to be saved to its session repository).

Because this is pre-rendered tiles of interaction data, paired with map image tiles, it's a very scalable map viewing solution. No CREATERUNTIMEMAP operation was needed to demonstrate this example, nor did we have to create or hit any session-based resources.

Also, because UTFGrids are accessed using the same XYZ tile access scheme, we can re-use the XYZ tile provider in the Tile Set Definition and just specify the new TileFormat of UTFGRID


Accessing UTFGrid tiles is just like accessing XYZ tiles, via the GETTILEIMAGE mapagent operation, but using the UTFGrid Tile Set Definition as the resource id.

The usual mapping suspects of OpenLayers and Leaflet already support UTFGrids, so you can easily take advantage of UTFGrids when it finally lands in MapGuide proper.

Now to make sure UTFGrid support also works on our Linux builds before I draft up that RFC.

Thursday, 25 May 2017

Announcing: mapguide-react-layout 0.9

After a small hiatus (due to hacking on MapGuide proper and my day job), here's a new release of mapguide-react-layout.

Here's the highlights of this release.

Now available as an NPM module

This is the first release available on npm. The npm module allows you to customize the viewer in the following ways:
  • Creating your own custom viewer templates
  • Creating your own script commands
  • Customizing your viewer bundle by omitting features you will never use
Want to see how that's done? Check out the new example that demonstrates all of the above.

Short of cloning my github repo and hacking/building the source yourself, the npm module is the best way to fully customize nearly all aspects of the viewer.

Now re-uses the Fusion PHP backend

In previous releases, we bundled a separate copy of the PHP backend tailored to service the following commands:
  • Buffer
  • Query
  • Theme
  • FeatureInfo
  • Redline
  • QuickPlot
  • Search Commands
Once the need to make this a npm module arose, this idea proved to be un-workable.
  • We'd have to ask the user to make some intricate post-build set up in their build configurations to make sure the PHP backend content to a location relative to the viewer bundle so that the above commands will work.
  • Putting PHP code into npm, a JS package registry? Uh ... okay back to the drawing board!
So in the process of revising this idea, the solution turned out to be quite simple:

Just assume MapGuide is installed with Fusion and re-use its PHP backend services

And it turns out that this assumption was a very safe one. Like PHP, Fusion will always be installed with MapGuide on Windows or Linux.

And with this assumption, we just demand one requirement for anyone using this viewer or building a bundle with the npm module: You need to install it into a subdirectory of MapGuide's www directory, which I would assume you are already doing because that's what our current install instructions say!

With this change, my mission statement with this viewer needs a small refinement This is no longer a modern map viewer that is a replacement for Fusion. This is now a modern map viewer that happens to re-use some Fusion backend services.

Which is fine by me because another (useful) side-effect of this exercise is that ...

Partial support for Fusion viewer APIs

... I had to polyfill various client-side Fusion viewer APIs so that the existing front-end content for the above commands can work against the mapguide-react-layout viewer without any modifications required. A subset of Fusion events are also supported in this release.

However, do not expect full 100% replication of the Fusion viewer API here. If you're going to migrate to this viewer, I recommend you go the whole nine yards and take advantage of the simpler and cleaner viewer APIs offered by mapguide-react-layout

Updated and "smarter" Templates

This release includes a new redux state branch for viewer template element visibility.



With this branch we also have new redux actions to push changes to it. Applicable templates now subscribe to the state branch and be able to automatically toggle visibility of template elements in response to dispatched actions.

What this enables us to do is to apply some extra smarts to the Fusion templates. For example, running an InvokeURL command (targeted at the Task Pane) will automatically show/open the Task Pane because the template reducer function also listens on this particular InvokeURL action to push a [Task Pane is now visible] state, as demonstrated in the gif below


Notice how I didn't have to manually expand the Task Pane first? The template reducer function already took care of it because I ran an InvokeURL command to load the Query widget.

The LimeGold and TurquoiseYellow templates have been updated to use the new Tabs2 blueprint component as our existing Tabs component is deprecated and not to mention that Tabs2 actually plays nicer with our new redux state branch.

The sidebar and ajax-viewer templates can now work with Application Definitions under the following conditions:
  • Your appdef has a widget container named Toolbar that will house the primary toolbar
  • Your appdef has a widget container named MapContextMenu that will house the context menu
  • Your appdef has a widget container named TaskMenu that will house the menu in the Task Pane bar
If you create a new Application Definition document in Maestro, all of the above conditions will already be satisfied.



Also if you take a look at the Aqua template, the 3 element togglers on the top-right are gone now.



This is because we were able to find a suitable replacement for InvokeScript commands as described below.

Script Command Support (ie. The InvokeScript replacement)

This feature is the replacement for InvokeScript commands/widgets and currently requires using the npm module. The viewer provides a registry API that allows for custom commands to be registered with the viewer.

How can you link these commands to toolbars and menu items in your WebLayout/ApplicationDefinition? With the existing InvokeScript definition.

The key difference here is that this viewer completely ignores the inline script content part and just invokes the equivalent script command registered in the viewer by the same name as defined in the InvokeScript definition in the WebLayout/ApplicationDefinition.

As a result, the new methodology for using these types of commands is to "bake them in" to your own viewer bundle, while with this approach we still retain the existing authoring experience by re-using InvokeScript definitions. You just have to give it the same name as what they're registered under in the viewer bundle.

To illustrate this, the viewer by default has registered script commands for toggling the:
  • Task Pane
  • Legend
  • Selection Panel
These script commands leverage the new redux actions to push element visibility state to the layout template components, replicating the behaviour of their old InvokeScript counterparts.

Because they are also registered under the same names as their old InvokeScript counterparts, a standard new Application Definition (created by Maestro) will no longer show [X] ERROR placeholders for these commands when fed to any of the 5 ported Fusion templates.


As we how have replicated the old behaviour of the Aqua template, the hard-coded element togglers on the top-right are no longer needed.

Bing Maps Support

The viewer now supports Bing Maps as external base layers.

As mentioned previously, you will now need to acquire an API key to use Bing Maps as the API-key-less version will shut down on June 30, 2017. Fusion proper has the same problem and will also require an API key to use Bing Maps. The next release of MapGuide Maestro will have an updated Fusion editor to configure this.

Fusion Application Definitions passed to this viewer with Bing Maps support are assumed to be ones configured up with a Bing Maps API key and Bing Maps (not VirtualEarth) base layers.

Command Parameterization

This release implements initial support for parameterization of commands. In practical terms this means that the viewer will now start recognizing and support the various Fusion widget extension properties that you may have defined in the widgets of your Application Definition.

Currently, only the (recently ported over) geolocation command is covered, but the next release will begin to see greater support for other Fusion widgets.

API Documentation

With TypeDoc, we now have API documentation for the viewer.

This only currently covers the npm module, which won't really make much sense if you're trying to use the viewer from a "browser globals" context, which would be the case if you're trying to interact with the viewer from Task Pane content.

As a workaround, I've covered the applicable parts of the "browser globals" API in the ...

New Project Home Page

And this API documentation is linked from a brand-spanking new project home page!



Surprisingly, this was much harder to set up than originally thought. Turning on GitHub pages and getting it to link to the TypeDoc-generated API documentation was made difficult due to:
  • TypeDoc-generated documentation being prefixed with underscores
  • HTML files with underscores having a special meaning in Jekyll, the default static site generator for GitHub Pages that prevents us from linking to them.
Jekyll being written in Ruby, made dev/testing my project home pages on Windows a royal pain, so I had to look at other static site generators.

After a few days of evaluating different static site generators, the closest one that matched by basic needs of: 
  • Pumping out some static HTML pages from markdown files
  • Link to TypeDoc-generated HTML files from one of them
  • Have one or more theme that exudes some polish (hey, I build web applications, not web sites and definitely not designing web sites!)
was a PHP tool called couscous. It wasn't ideal. For example, deploy totally did not work for me, but there was fortunately a workaround. Compared to the other available options, this was the best of the lot for what I needed.

Hopefully the project site is informative enough to take you to where you need to go. If not, send me some feedback.

Other Changes
  • Viewer has been updated to use the latest and greatest React, Blueprint, TypeScript and OpenLayers.
  • Viewer uses the new OpenLayers ES2015 modules which means we now only use the parts of OL that we actually use and not inflate our viewer bundle with things we aren't using.
  • Viewer now 100% uses typings provided by npm @types. This means that we no longer need to use the external typings tool to acquire d.ts files, which is relevant to the npm module as you will get automatic TypeScript type definitions for mapguide-react-layout and its dependent libraries out of the box when you install the npm module.
  • Many other updated dependencies thanks to greenkeeper.io integration.
  • The viewer now supports frame targeting (ie. Target = SpecifiedFrame and you specified a frame name) for InvokeURL and Search commands
  • The viewer now supports running InvokeURL and Search commands in a "New Window". What we will actually do is run the command in a floating modal.

Friday, 19 May 2017

MapGuide tidbits: Fusion and Bing Maps

If you use Fusion with Bing Maps, you should start applying for an API key because on June 30th, 2017, the legacy Bing Maps controls (ie. The ones that don't require an API key) will shut down.

I gather Autodesk have already taken care of this problem for their next release of AIMS, but for the rest, I'll make sure that this is backported to older Fusion branches before the June 30th deadline.

And also to make sure that Maestro has the updated Fusion Editor UI to match.

Wednesday, 10 May 2017

vcpkg is very interesting too

Part of what spurred my interest in gRPC was its ease of consumption on Windows through vcpkg allowing me to easily build and prototype some gRPC services and be able to easily interop from clients in different programming languages.

For those not in the know, vcpkg is Microsoft's attempt at a package manager for C/C++ libraries on Windows. This tool is pertinent to my interests because, look at what we currently keeping in our MapGuide/FDO source repos!



That's several GBs of source code for external thirdparty libraries we need to build MapGuide and FDO. It's a situation we can't avoid at the moment (especially on Windows) because until vcpkg, we had no choice. There was nothing akin to "nuget for C++ libraries" for Windows. On Linux we at least have the option of offloading to system installed libraries provided by the default distro package manager.

With vcpkg, this would greatly simplify the MapGuide/FDO build system on Windows. Most of the libraries listed here already exist as ports in vcpkg and most are more up-to-date as well. Imagine just doing a svn sparse checkout, skipping having to download whole parts of Oem/Thirdparty, point the MapGuide/FDO sources to our existing pre-built vcpkg packages and enjoy much faster build times, because we're not wasting a majority of it building external libraries. It may be even fast enough to continuously integrate!

It would also almost certainly spell the end of the difficult story around custom GDAL binaries as well, because I've been doing my part to add/enhance (geospatially-)relevant vcpkg ports and to light their current GDAL vcpkg port up with as many features as technically and legally possible. If we ever do get to building FDO using vcpkg-sourced external libraries, you're gonna get a GDAL/OGR provider with maximum vector/raster format support (barring the obvious omissions like Oracle, DWG, ECW, etc due to legal hurdles in obtaining and using their respective libraries and SDKs)

A vcpkg-based MapGuide/FDO build system for Windows is definitely something on my radar.

React-ing to the need for a modern MapGuide viewer (Part 16): It didn't have to be this complicated

The major theme of the next release of mapguide-react-layout is to open up the floodgates to all sorts of viewer customizations.

In particular the next release will be my first foray into npm modules as I intend to publish this next release as one. The npm module allows for the following types of customizations:

  • Custom viewer layout templates
  • Custom viewer script commands
  • Being able to selectively include/exclude viewer features to reduce bundle size
These customization options will only be available through the npm module

One major roadblock that has appeared is that we currently include a verbatim copy of our Fusion PHP backend so that certain viewer commands will work:
  • Buffer
  • Query
  • Redline
  • Theme
  • Feature Information
  • QuickPlot
  • Search
How does this roadblock us? Well with this PHP backend bundled, the npm module story is more complicated:
  • It would involve some serious webpack (or some other post-build) shenanigans to get the backend PHP content deployed in the right place, relative to your custom viewer bundle.
  • It bloats the size of the npm module (based on my simulations using npm pack).
  • I feel extremely icky publishing a JavaScript library that bundles a PHP backend as a required dependency.
In hindsight, bundling a copy of this Fusion backend is totally not necessary. It is simpler to just make our viewer commands point and talk to the existing Fusion backend. When MapGuide is installed, Fusion and PHP will always be there, so this is a safe assumption to make. It's the same environmental assumption that allows for mapguide-rest to be an easy "drop into www" and it works.

By re-using the existing Fusion backend for these tools, there also another positive side effect. It means we have to also polyfill whatever Fusion viewer APIs are needed for the frontend HTML/JS content of these Fusion widgets to work within mapguide-react-layout without modifications. What this means is that migration should be even easier as we are now polyfilling various common Fusion viewer APIs in addition to the AJAX viewer APIs.

So with this next release, the viewer will now reuse the Fusion backend of your MapGuide installation. As a result, it also means the zip packages for this release will be much smaller as well.

Tuesday, 2 May 2017

It's like the early days of the internet!

Been going back on the MapGuide dev train.

One of the features I'm experimenting with is UTFGrid rendering.

I've mentioned in previous posts, how in the process of fleshing out an idea that you don't know whether it will work or not, that there comes a flash point. A moment where your idea tips from uncertainty to "yes it can be done, so let's get it done!"

I have just reached that moment again.


It's like the early days of dial-up internet. ASCII art galore!

So off the bat, 2 things I can see:

  • The Y-axis is flipped, oops.
  • I misread the UTFGrid spec. I'm not meant to do a pixel-by-pixel replacement here.
Once these 2 things are sorted, it's just a case of mapping each character to the feature attributes (that the renderer is also tracking) and we have enough data to assemble the final UTFGrid tile

Thursday, 23 March 2017

gRPC is very interesting

MapGuide in its current form is a whole bucket of assorted libraries and technologies:
  • We use FDO for spatial data access
  • We use ACE (Adaptive Communication Environment) for:
    • Basic multi-threading primitives like mutexes, threads, etc
    • TCP/IP communication between the Web Tier and the Server Tier
    • Implementing a custom RPC layer on top of TCP/IP sockets. All of the service layer methods you use in the MapGuide API? They're all basically RPC calls sent over TCP/IP for the MapGuide Server to invoke its server-side eqvivalent. Most of the other classes that you pass into these service methods are essentially messages that are serialized/deserialized through the TCP/IP sockets. When you think about it, the MapGuide Web API is merely an RPC client for the MapGuide Server, which itself is an RPC server that does the actual work
  • We use Berkeley DBXML for the storage of all our XML-based resources
    • We have an object-oriented subset of these resource types (Feature Sources, Layer Definitions, Map Definitions, Symbol Definitions) in the MdfModel library with XML serialization/parsing code in the MdfParser library
    • Our Rendering and Stylization Engine work off of these MdfModel classes to render the maps that you see on your viewer
  • We use xerces for XML reading/writing XML in and out of DBXML
  • We use a custom modified (and somewhat ancient) version of SWIG to generate wrappers for our RPC client so that you can talk to the MapGuide Server in:
    • .net
    • Java
    • PHP
So why do I mention all of this?

I mention this, because I've recently been checking out gRPC, a cross-platform, cross-language RPC framework from Google.

And from what I've seen so far, gRPC could easily replace and simplify most of the technology stack we're currently using for MapGuide:
  • ACE? gRPC is the RPC framework! The only reason we'd keep ACE around would be for multi-threading facilities, but the C++ standard library at this point would be adequate enough to replace that as well
  • DBXML/MdfModel/xerces? gRPC is driven by Google Protocol Buffers.
    • Protobuf messages are strongly typed classes that serialize/deserialize into compact binary streams and is more efficient and faster than slinging around XML. Ever bemoan the fact you have to currently work with XML to manipulate maps/layers/etc? In .net you are reprieved if you use the Maestro API (where we provide strongly-typed classes for all the resource XML types), but for the other languages you have to figure out how to use the XML APIs/services provided by Java/PHP to work with the XML blobs that the MapGuide API gives and expects. With protobuf, you have none of these problems.
    • Protobuf messages can evolve in a backward-compatible manner
    • Because protobuf messages are already strongly-typed classes, it makes MdfModel/MdfParser redundant if you get the Rendering/Stylization engine to work against protobuf messages for maps/layers/symbols/styles/etc
    • If we ever wanted to add support for Mapbox Vector Tiles (which seems to be the de-facto vector tile format), well the spec is protobuf-based so ...
    • Protobuf would mean we no longer deal in XML, so we don't need Xerces for reading/writing XML and DBXML as the storage database (and all its cryptic error messages that can bubble up from the Resource Service APIs) can be replaced with something simpler. We may not even need a database at this point. Dumping protobuf messages to a structured file system could probably be a simpler solution
  • SWIG? gRPC and protobuf can already generate service stubs and protobuf message classes in the languages we currently target:
    • .net
    • Java
    • PHP
    • And if we wanted, we can also instantly generate a gRPC-based MapGuide API for:
      • node.js
      • Ruby
      • Python
      • C++
      • Android Java
      • Objective-C
      • Go
    • The best thing about this? All of this generated code is portable in their respective platforms and doesn't involve native code interop through "flattened" interfaces of C code wrapping the original C++ code, which is what SWIG ultimately does for any language we want to generate wrapper bindings out of. If it does involve native code interop, it's a concern that is taken care of by the respective gRPC/protobuf implementation for that language.
  • Combine a gRPC-based MapGuide Server with grpc-gateway and we'd have an instant REST API to easily build a client-side map viewer out of
  • gRPC works at a scale that is way beyond what we can currently achieve with MapGuide currently. After all, this is what Google uses themselves for building their various services
If what I said above doesn't make much sense, consider a practical example.

Say we had our Feature Service (which as a user of the MapGuide API, you should be familiar with) as a gRPC service Definition

// Message definitions for the request/response types below are omitted for brevity but basically every request and
// response type mentioned below will have eqvivalent protobuf message classes automatically generated along with
// the service

// Provides an abstraction layer for the storage and retrieval of feature data in a technology-independent way.
// The API lets you determine what storage technologies are available and what capabilities they have. Access
// to the storage technology is modeled as a connection. For example, you can connect to a file and do simple
// insertions or connect to a relational database and do transaction-based operations.
service FeatureService {
    // Creates or updates a feature schema within the specified feature source.
    // For this method to actually delete any schema elements, the matching elements
    // in the input schema must be marked for deletion
    rpc ApplySchema (ApplySchemaRequest) returns (BasicResponse);
    rpc BeginTransaction (BeginTransactionRequest) returns (BeginTransactionResponse);
    // Creates a feature source in the repository identified by the specified resource
    // identifier, using the given feature source parameters.
    rpc CreateFeatureSource (CreateFeatureSourceRequest) returns (BasicResponse);
    rpc DeleteFeatures (DeleteFeaturesRequest) returns (DeleteFeaturesResponse);
    // Gets the definitions of one or more schemas contained in the feature source for particular classes.
    // If the specified schema name or a class name does not exist, this method will throw an exception.
    rpc DescribeSchema (DescribeSchemaRequest) returns (DescribeSchemaResponse);
    // This method enumerates all the providers and if they are FDO enabled for the specified provider and partial connection string.
    rpc EnumerateDataStores (EnumerateDataStoresRequest) returns (EnumerateDataStoresResponse);
    // Executes SQL statements NOT including SELECT statements.
    rpc ExecuteSqlNonQuery (ExecuteSqlNonQueryRequest) returns (ExecuteSqlNonQueryResponse);
    // Executes the SQL SELECT statement on the specified feature source.
    rpc ExecuteSqlQuery (ExecuteSqlQueryRequest) returns (stream DataRecord);
    // Gets the capabilities of an FDO Provider
    rpc GetCapabilities (GetCapabilitiesRequest) returns (GetCapabilitiesResponse);
    // Gets the class definition for the specified class
    rpc GetClassDefinition (GetClassDefinitionRequest) returns (GetClassDefinitionResponse);
    // Gets a list of the names of all classes available within a specified schema
    rpc GetClasses (GetClassesRequest) returns (GetClassesResponse);
    // Gets a set of connection values that are used to make connections to an FDO provider that permits multiple connections.
    rpc GetConnectionPropertyValues (GetConnectionPropertyValuesRequest) returns (GetConnectionPropertyValuesResponse);
    // Gets a list of the available FDO providers together with other information such as the names of the connection properties for each provider
    rpc GetFeatureProviders (GetFeatureProvidersRequest) returns (GetFeatureProvidersResponse);
    // Gets the locked features.
    rpc GetLockedFeatures (GetLockedFeaturesRequest) returns (stream FeatureRecord);
    // Gets all available long transactions for the provider
    rpc GetLongTransactions (GetLongTransactionsRequest) returns (GetLongTransactionsResponse);
    // This method returns all of the logical to physical schema mappings for the specified provider and partial connection string
    rpc GetSchemaMapping (GetSchemaMappingRequest) returns (GetSchemaMappingResponse);
    // Gets a list of the names of all of the schemas available in the feature source
    rpc GetSchemas (GetSchemasRequest) returns (GetSchemasResponse);
    // Gets all of the spatial contexts available in the feature source
    rpc GetSpatialContexts (GetSpatialContextsRequest) returns (GetSpatialContextsResponse);
    // Inserts a new feature into the specified feature class of the specified Feature Source
    rpc InsertFeatures (InsertFeaturesRequest) returns (stream FeatureRecord);
    // Selects groups of features from a feature source and applies filters to each of the groups according to the criteria set in the aggregate query option supplied
    rpc SelectAggregate (SelectAggregateRequest) returns (stream DataRecord);
    // Selects features from a feature source according to the criteria set in the query options provided
    rpc SelectFeatures (SelectFeaturesRequest) returns (stream FeatureRecord);
    // Set the active long transaction name for a feature source
    rpc SetLongTransaction (SetLongTransactionRequest) returns (BasicResponse);
    // Connects to the Feature Provider specified in the connection string
    rpc TestConnection (TestConnectionRequest) returns (TestConnectionResponse);
    // Executes commands contained in the given command set
    rpc UpdateFeatures (UpdateFeaturesRequest) returns (UpdateFeaturesResponse);
    // Updates all features that match the given filter with the specified property values
    rpc UpdateMatchingFeatures (UpdateMatchingFeaturesRequest) returns (UpdateMatchingFeaturesResponse);
}

Running this service definition through the protoc compiler with grpc plugin gives us:
  • Auto-generated (and strongly-typed) protobuf classes for all the messages. ie: The request and response types for this service
  • An auto-generated FeatureService gRPC client ready to use in the language of our choice
  • An auto-generated gRPC server stub for FeatureService in the language of our choice ready for us to "fill in the blanks". For practical purposes, we'd generate this part in C++ and fill in the blanks by mapping the various service operations to their respective FDO APIs and its return values to our gRPC responses.
And at this point, we'd just need a simple C++ console program that bootstraps gRPC/FDO, registers our gRPC service implementation, start the gRPC server on a particular port and we'd have a functional Feature Service implementation in gRPC. Our auto-generated Feature Service client can connect to this host and port to immediately start talking to it.

The only real work is the "filling in the blanks" on the server part. Everything else is taken care of for us.

Extrapolate this to the rest of our services (Resource, Rendering, etc) and we basically have a gRPC-based MapGuide Server.

Also filling in the blanks is a conceptually simple exercise as well:
  • Feature Service - Pass down the APIs in FDO.
  • Rendering Service - Setup up FDO queries based on map/layers visible and pass query results to the Rendering/Stylization engine.
  • Resource Service - Read/write protobuf resources to some kind of persistent storage. It doesn't have to be something complex like DBXML, it can be as simple as a file system (that's what mg-desktop does for its resource service implementation btw)
  • Tile Service - It's just like the rendering service, but you're asking the Rendering/Stylization engine to render tile-sized content.
  • KML Service - Just like rendering service, but you're asking the Rendering/Stylization engine to render KML documents instead of images.
  • Drawing Service - Do we still care about DWF support? Well if we have to support this, it's just passing down to the APIs in DWF Toolkit.
  • Mapping Service - It's a mish-mash of tapping into the Rendering/Stylization engine and/or the DWF Toolkit.
  • Profiling Service - Just tap into whatever tracing/instrumentation APIs provided by gRPC.
Now because gRPC is cross-language, nothing says we have to use C++ for all the service implementations, it's just that most of the libraries and APIs we'd be mapping into are already in C++, so in practical terms we'd stick to the same language as well.

Front this with grpc-gateway, and we basically have our RESTful mapagent to build a map viewer against.

There's still a few unknowns:
  • How do we model file uploads/downloads?
  • Can server-side service implementations call other services?
Google's vast set of gPRC definitions for their various web services can answer the first part. The other part, will need more playing around.

The thought of a gRPC-based MapGuide Server is very exciting!