Make the wiki the source of truth for map data

[KockaAdmiralac]

Motivation

Our biggest pain point so far with our current map editing flow is how we need to have access to the latest TMX files in order to perform any map editing using Tiled. If someone edits a map using Tiled and doesn't share the TMX file, nobody else can edit that map using Tiled because they would overwrite the previous version. Worse, we've had plenty of editors change marker descriptions on the wiki itself, using the regular JSON editor (which is only good for editing marker descriptions and names, really).

Proposal

Given this is a collaborative map editing effort, it makes the most sense for the wiki itself to be a source of truth, without requiring any external file sharing. For this to become reality, we need the following:

1. DataMaps → Tiled conversion (the other direction already exists)
2. Metadata describing information such as which filenames to use for maps and images, map tile size, map names on other wikis, etc.
3. Interface that gives the user an option to pull the map from the wiki, and warns them if it finds an outdated local map
4. Mechanism which splits translation data to their respective wikis when publishing and reassembles the data when pulling

This pull request implements points 1, 3 and 4. Point 2 was already pushed to master beforehand.

Changes

Here are the practical changes that this pull request makes:

• Removes all maps and map images from the repository
• Upon opening a project for the first time, the extension pulls all maps and map images from the wiki and greets the user
• Introduces a File > Pull from wiki option which presents a dialog with all maps and map images that are out of sync with the wiki, and allows the user to select which ones to pull
• When editing a .mw-datamaps file, which is the format in which files pulled from the wiki appear in (as opposed to a TMX file, which is Tiled's native map format), publishing a map publishes it to all wikis at once
• When editing TMX files the flow remains mostly the same, though we will focus most of our efforts on improving the new flow

Limitations

Here are potential issues that we may run into in the future:

• If someone is editing maps through the wiki, it may lead to desync of some important information, most notably marker IDs. Current implementation shows some warnings when this happens, though resolution has to be manual.
• We can't have publishing to a single wiki anymore, because if a marker ID changes, or layers get reordered, all wikis need to have that information. If they don't, the translation data will fail to reassemble and users will get strange warnings which they don't know how to fix.
• We need to know which wikis to publish a map to, because some wikis may not need the map yet and it causes them unnecessary maintenance burden. We do this by checking which languages have any translations at all (markers, rectangles, layers...) and not publishing to languages with no translations. When editing TMX files, publishing to a language wiki with no translations will error.
• The method for getting all maps from a wiki might not return all maps if the combined size of all maps is too large, but that's a problem for a future me.
• The DataMaps format is still a lot more permissive than the subset we support for conversion. Wikis with existent maps might find it hard to use this extension if they used constructs that our conversion to Tiled doesn't support yet.
• The synchronization mechanism is still rudimentary - you can either pull all the maps and overwrite your local copy, or not pull any of the maps. It doesn't even detect whether the local copy is clean or dirty, and just reports that there is a difference. It would be good to show the user what exactly the differences are from the local copy, because sometimes the changes from upstream can be cleanly applied.
• Nesting layers is something the conversion from Tiled to DataMaps supports, but the other way doesn't yet. We never used this feature, but we might need to in the future?

Migration

To allow use of the new system, I made sure all of our TMX files are in sync with all wikis on the network, then published all of these TMX files to all the wikis. This ensures that metadata is in sync for the new system to take over.

Next steps

• This pull request introduces a lot of new properties. When users aren't given a clear interface for which properties can be edited, unusual map operations can become tedious (how do you translate a background name?). We need a proper interface for editing all of these properties.
• It's unclear how to create a new map. Legacy flow will work, but it's a bit unusual having to create a TMX file only to delete it. We need a map creation flow with map file upload.
• Moreover, it's unclear how to set up a new project for a new wiki which potentially wants to use the extension. A project creation flow would come in handy.
• Users no longer need to redownload the repo every time a map updates, which is great. But it would be even better if the extension itself also updated automatically, so they don't need to redownload the repo at all past the initial download. (This is already being worked on in the updater branch.)
• The extension is currently English-centric, and even more so with this update. We should probably allow setting a different primary language in case an English wiki doesn't exist or isn't the primary wiki.

[cloudflare-workers-and-pages]

Deploying maps with    Cloudflare Pages

Latest commit:	4cc9647
Status:	🚫  Build failed.

View logs