Dev Diary 9: FAIMS3 Enters Beta!
Now supporting ever more notebooks and more features!
Earlier today we went through client acceptance tests on our April-July development cycle. We are now officially in beta!1 Contact us at info@faims.edu.au if you want to try out our beta software.
Dev Update
I’ve popped the release candidate tags on our github repositories and we’ve started regression testing preparatory to release! In this development period we achieved six major goals:
A conflict resolution interface
Attachment downloading controls2
A hierarchical vocabulary picker
Command-line field editing and record deletion/undeletion
CSV Export
We’ve also worked on bugfixing, increased development automation, and keeping up with the red queen’s race of Node patches.
Our conflict resolution interface
There comes a point in every collaborative online document where two people try to edit the same thing without synchronising their edits. This tends to happen rather more frequently when a product is designed to work offline. In this example, I induced a conflict by opening the same record in two browser profiles (one is called red, one is called purple) and hit save without allowing changes to sync from one browser to the other3.
While the usual case is that these two people will have edited different fields (and thus have their changes merged automagically…) there will be times in any record’s life where the same field has been edited by different people at the same time.4
Upon opening the conflicted record, I see:
A rather strongly worded: “Pretty please resolve your conflict!” (Which doesn’t stop a determined user from editing anyways if they really need to add data.) Clicking on “Why am I seeing this?” explains further:
Conflicts generally arise when two users have changed the same fields in a record, or if the user deleted a file while another user was modifying it. In these cases, FAIMS cannot automatically determine what is correct.
The record will be marked as conflicted. It is then the users' responsibility to resolve the conflict.
Users may continue to edit records whilst conflicts exist, but should be aware that doing so may create further conflicts, it is advisable to resolve all conflicts before editing.
Users can then navigate to our merge interface:
In this interface, they can choose which field to keep, and gradually resolve conflicts until one single record leaves. Unconflicted.
Hierarchical vocabulary picker
Given a vocabulary: https://vocabs.ardc.edu.au/viewById/313, I was able to render it into JSON5 and produce:
A picklist of arbitrary depth, allowing people to specify detailed controlled vocabularies without having a single dropdown 100-items long. It’s neat! We plan to implement our FAIMS 2-style picture gallery and hierarchical picture galleries as part of our August-November development period.
Exporters
We created a command line tool to allow for record editing and “deletion”6.
(The most interesting output is the list command, so you get to see that). This allows for external systems to interact with FAIMS notebooks and make changes to them. While in principle, this mechanism will support record creation … we’re not there yet.
I also wrote a generalised exporter that exports forms in CSV, json, and xlsx formats, the whole database in jsonlines format, and renames attachments to their record’s HRID. Each geometry also gets exported in its own geojson file as well as being embedded in the tabular data as WKT:
ls -R in an exported directory. My terminal is very purple.Now, folks can actually use their collected data. What a thought!
Where we’re at and where we’re going
The FAIMS team is now hard at work prioritising7 our August-November development season. This period will bring us to production ready with the ability to support researchers around the world in making notebooks, collecting data, and getting their well structured data out into a useful-to-them state. We’re also working up our sustainability plans for 2023, so if you know you’re going to want a notebook for yourself, your team, and8 your enterprise, let us know!
What I’m reading
Katy DeCorah, “In defense of magic” — on bridging the gaps between have and need.
Charity.wtf, “Advice For Engineering Managers Who Want To Climb The Ladder”
Paul E. Smaldino and Richard McElreath, “The natural selection of bad science”
Rands (Michael Lopp), “By design” — the necessity of clear communication in leadership
Mia Sato “The archive saving home sewing history from the trash” — a fascinating physical archive of sewing patterns
Linus Akesson “Partita Prelude” — a lovely performance of Bach on a commodore64
“Seafarer” — a little novella set in the “Human Altered” ‘verse.
If you’ve been paying attention, you may recall we have referred to private-beta and early-beta releases of the code. These were one step up from our alpha release back in June 2021 but not quite ready enough for actual field use, hence our hesitance to release it as a public beta. We are now ready. We are officially in beta.
Not much to show about these. By default, other folks’ downloads no longer sync onto your device until a toggle is enabled:
We do not support multiple tabs open within the same browser instance. It causes odd stuff to happen as a function of the “web service workers” we’re using to perform sync.
Manually at this point, but automagic imports from published vocabularies are absolutely in our backlog. Many things are in our backlog.
Delete=hide, not delete=expunge. Right now there is no way to expunge data from FAIMS, as keeping data safe is our highest priority.
Now more than 50 named features….
I can hope, right?