Siplicity Devlog

Devlog entry - Thursday, 7 November, 2024

Siplicity available in Early Access

What better day than World Digital Preservation Day to announce that Siplicity is now is now available to download, try, and buy in early access!

What is Siplicity? It’s a new application to make transferring digital files simpler, particularly when working in a digital preservation or digital curation context.

What does early access mean? It’s a term from the video game industry, and means that users can buy and use software while it is still being built. For users, this means they can access software earlier and engage with developers to help shape the final product. For developers, it means funding at a crucial stage and real world testing before the formal release.

Should you join the early access? Install and try it first. Although Siplicity is unfinished, the essential functionality is now in place, including the three principal working views (the workspace pane, the analysis pane, and the data entry pane). You can test all of that in the demo. A license is only required to unlock use of the commit button. If there’s already enough there to incorporate the tool in your workflows, or if it’s on the verge of that, please consider getting a license. To reward early access supporters, the early access license includes three year’s access to updates (rather than the normal license’s two years).

If you like what you see, but can’t yet justify the purchase, know that there’s a lot more still to come, including:

• cross-platform support
• multilingual support
• working with compressed and archive formats
• comprehensive, PREMIS-based audit trails
• reading and writing embedded metadata
• formulas/macros for metadata editing
• metadata import
• remote file locations
• support for multiple outputs including OCFL, EARK, and the various flavours of digital preservation system imports.

I’ll be posting my progress regularly on the devlog, and hope to have these pieces in place for a version one launch early next year.

Devlog entry - Sunday, 20 October, 2024

Actionable analytics

This week I finished the first draft of Siplicity’s analysis pane.

Here’s how it looks:

As soon as you add files to the to the workspace, an initial set of reports is automatically created, with visualisations for file sizes, dates modified, and file extensions.

Running actions over the files in your workspace can produce further reports:

• the siegfried action creates format ID, format class and mime-type reports
• any of the checksum actions will generate a duplicate report.

All of the reports can be used as the basis for queries to the worspace pane. So that, for example, you can more closely inspect files from a particular date range or of a particular format type.

This connection between the analysis and workspace panes can power workflows like:

• identifying and removing zero-byte files
• selecting duplicate files for removal
• blacklisting sets of file formats
• whitelisting file formats for selection to the output pane.

In other words, Siplicity’s analytics produce more than insights. They are actionable.

Devlog entry - Thursday, 3 October, 2024

Bake off - second attempt in the kitchen!

Two weeks ago I gave a first demonstration of Siplicity for the iPRES conference (the “bake off” stream). Many thanks to Micky, Dries, and the rest of the conference team for including me in the program.

My presentation was unfortunately a little undercooked. Many of the ingredients were there, but without enough time in the coding oven, and so I focused the presentation on Siplicity’s backend application (the GRPC server), which didn’t make for an appetizing meal.

OK, enough with the cooking puns! Here’s the demo of Siplicity I would have liked to have given:

In this screencast you can see that much of Siplicity’s basic functionality is now implemented in the frontend.

The screencast demonstrates:

• adding files and folders to an input graph
• manipulating the input graph (select, delete and move nodes to the output graph)
• running processes like file format identification and checksum calculation to enrich record metadata
• viewing record details and metadata
• manipulating the output graph and defining an output destination
• formatting the output graph as a bagit package
• committing the output graph to disk.

Devlog entry - Sunday, 15 September, 2024

Tech stack overview

Siplicity has a backend written in Go. I chose Go mainly because I am familiar with it but can justify the choice on the basis that it is cross-platform, reasonably performant, and enables native integration with Siegfried.

The frontend is Flutter. This framework is best known for mobile UIs but it now has good desktop and web support too. It is modern, performant, and native-feeling.

The backend and frontend communicate via gRPC. I’ve been using the Buf tooling for this.

I am focused on creating a local, client application but a benefit of this mix of go/flutter/grpc is that it opens the door to other potential deployment scenarios, including local client (native or browser) with a cloud backend (e.g. an AWS machine in the same region as your cloud storage).