Last week, we worked on SFTP imports, where customer data was imported in CSV/XLS/XLSX format from the client’s SFTP server. This week, we’re working on the opposite – that when a customer is updated, the changes should be pushed out (exported) to a client’s SFTP server.
This is for fairly large clients that have their own internal customer relationship management software, that want any updates in our workflow management system to feed into theirs.
We were considering doing this in real-time, so that changes were exported the moment they were made, but this can cause network issues if there are a lot of changes. And when you think about it, if all the fields in a customer (name, address, email, phone, etc) are changed one by one, you’re really only interested in the end product – that one export of the whole customer after all changes are done.
So, we’re doing this on an hourly basis instead. If you have an export set up, then every hour, any customers that have been edited within the last hour will be exported from our job management software to your own server.
We also had to consider the end file – will it be a single large file containing all the changed customers, or lots of individual files? A big problem with the large file solution is that if you have an importer on the client server that is reading the file at the same time as we are writing it, then there is a potential loss of data there.
Instead, we will create individual files. That way the chance of read/write collisions is much smaller. One of the problems with paperless office solutions is that things happen so fast there are there “race conditions” that we start worrying about.
