13 Oct 2017
New import system coming, replacing Flash
in Technical Blog by Kae Verens
We have imports in a number of places throughout the system, building each one separately. Some are better than others. The Jobs page, for example, has a nice importer which will let you upload a sheet of jobs in any reasonable format and let you "map" your own layout to our internal layout. You can even record that mapping so that you can use it again next time you need to do an import. I was asked to replicated that method throughout the site, and thought about it for a bit before diving in, so I have a few other ideas about it that will improve it further. File uploads in Web 2.0 systems have traditionally been done using Flash uploaders such as Uploadify, because up until recently, there was no good way to handle file uploads that didn't look clunky and Web 1.0. However, Flash is a deprecated technology and will be removed completely by Adobe in 2020. Some of the other paperless office software solutions are going to be hit really badly by this, as I hear that at least one of the larger workflow management system companies is built entirely on a Flash infrastructure. They're going to have to rebuild completely within the next two years! Google Chrome are hinting heavily at that as well, by the way that they disable Flash by default, to the point that when any of our clients say that they can't upload a file into our system, one of our things to check is "Have you enabled Flash?". It happens so often that we even made a video showing how to do it! So, Flash has got to go. Thankfully, the new HTML5 file uploaders are much better at handling uploads gracefully! Along with replacing flash as the uploader technology, we're also thinking about the actual flow of an import. When a file is imported into the Customers part of the customer relationship management software, for example, we will run the import and then tell you how many were imported. If your spreadsheet contained 2000 customers and we report that 1999 of them imported, what happened to the other one? Why did that not import? Well... because of how imports are currently done, it's very hard to pass error messages back to the browser, so no - we literally can't tell you what happened to that one, so you need to now go through your file and see for yourself what didn't upload. Ick! We are really sorry about that shoddiness (which is implicit in the technology, so hard to get around), and hopefully the new importer makes up for it. How we're fixing that in the new importer is that when you upload a file, we then send the contained data back to the browser, which then uploads the data again, but one at a time so it can keep tabs on the progress, and keep records of what errors popped up. After the import has finished, the browser then knows what rows failed to import, and offers you a download of those failed rows. Much neater! Depending on how busy we get with other stuff, I expect this to be all completed some time next week, and available to clients who are on our "bleeding edge" paperless office solutions server. Those on the "stable" servers will need to wait until the next public release of FieldMotion, which I expect will be in only a few months, as we're closing in on completing the major parts of it.