Siplicity

Custom vocabularies

Thu, 02 Jul 2026 01:00:00 +0000

At the heart of Siplicity, there is a simple, but flexible, metadata model. Metadata, for files and folders, is stored as a tree of namespace/name/value nodes. Nodes can have children. Values are just strings.

There’s a special SCHEMA node in Siplicity that keeps track of the metadata terms being used in a particular session. There’s not much else special about it: it stores the schema in the same metadata structure described above, except without any values.

This looks like this:

          ns:name
            |
  ---------------------
  |         |         |
ns:name  ns:name  ns:name
            |         |
       ----------     --------------------    
       |         |    |          |       |
    ns:name  ns:name  ns:name  ns:name  ns:name

The SCHEMA node is updated automatically during a Siplicity session (e.g. if you run a checksum job, it might add the schema terms for checksum algorithm and checksum result). In the data entry pane, when you select fields to edit, you are selecting from the fields available on that SCHEMA node.

Over the past few months I’ve been working on a mechanism for end users to attach their own custom vocabulary terms to the SCHEMA node. To enable, for example, working with DC Terms or the PREMIS vocabulary.

TOML

End users can now add custom vocabularies by encoding their schema into a TOML file.

Here’s an example, showing the full complexity you can achieve:

namespace = "test" // namespaces are inherited, can be overriden below
name = "root" // name of root metadata node, if omitted, fields will sit at the top level of the schema
fields = [
    "simple",
    { name = "parent", fields = [
        "simple child",
        { name = "another", namespace = "distinct", fields = [
            "enough",
        ] },
    ] },
]

This translates to this tree of namespace/name pairs:

          test:root
            |
  ------------------
  |                |
test:simple  test:parent
                   |         
       ------------- 
       |           |
test:simple child  distinct:another
                   |
                   distinct:enough

It’s a little dense, I know, but there’s a simplified form that you can use for a flat schema where all the terms sit in a single namespace:

namespace = "AGRkMS"
name = "AGRkMS"
fields = [
  "Identifier",
  "Creator",
  "Date Created",
  "Title",
  "Protective Marking",
  "Disposal Class",
  "Rights",
]

Settings

Once you’ve created your TOML file, it needs to be saved within a “vocab” folder, within the “definitions” folder of your “siplicity” home directory.

On a Windows install, you can find the “siplicity” home directory within the hidden “AppData” directory, e.g.:

c:\Users\richardlehane\AppData\Local\siplicity

Activation

The next time you start Siplicity it will be loaded and the final step is to activate it. To do this, go into the “Definitions” tab of Siplicity’s settings menu, and activate the toggle for your vocabulary:

Using your new terms

Once that’s all done, you can start using your terms. When selecting fields for editing in the data entry pane, they will appear like this:

Siplicity available in Early Access

Wed, 06 Nov 2024 22:00:00 +0000

What better day than World Digital Preservation Day to announce that Siplicity is now is now available to download, try, and buy in early access!

What is Siplicity? It’s a new application to make transferring digital files simpler, particularly when working in a digital preservation or digital curation context.

What does early access mean? It’s a term from the video game industry, and means that users can buy and use software while it is still being built. For users, this means they can access software earlier and engage with developers to help shape the final product. For developers, it means funding at a crucial stage and real world testing before the formal release.

Should you join the early access? Install and try it first. Although Siplicity is unfinished, the essential functionality is now in place, including the three principal working views (the workspace pane, the analysis pane, and the data entry pane). You can test all of that in the demo. A license is only required to unlock use of the commit button. If there’s already enough there to incorporate the tool in your workflows, or if it’s on the verge of that, please consider getting a license. To reward early access supporters, the early access license includes three year’s access to updates (rather than the normal license’s two years).

If you like what you see, but can’t yet justify the purchase, know that there’s a lot more still to come, including:

cross-platform support
multilingual support
working with compressed and archive formats
comprehensive, PREMIS-based audit trails
reading and writing embedded metadata
formulas/macros for metadata editing
metadata import
remote file locations
support for multiple outputs including OCFL, EARK, and the various flavours of digital preservation system imports.

I’ll be posting my progress regularly on the devlog, and hope to have these pieces in place for a version one launch early next year.

July 2026 update: the early access period is continuing until v1.0 is released

Actionable analytics

Sun, 20 Oct 2024 00:00:00 +0000

This week I finished the first draft of Siplicity’s analysis pane.

Here’s how it looks:

As soon as you add files to the to the workspace, an initial set of reports is automatically created, with visualisations for file sizes, dates modified, and file extensions.

Running actions over the files in your workspace can produce further reports:

the siegfried action creates format ID, format class and mime-type reports
any of the checksum actions will generate a duplicate report.

All of the reports can be used as the basis for queries to the worspace pane. So that, for example, you can more closely inspect files from a particular date range or of a particular format type.

This connection between the analysis and workspace panes can power workflows like:

identifying and removing zero-byte files
selecting duplicate files for removal
blacklisting sets of file formats
whitelisting file formats for selection to the output pane.

In other words, Siplicity’s analytics produce more than insights. They are actionable.

Bake off - second attempt in the kitchen!

Thu, 03 Oct 2024 00:00:00 +0000

Two weeks ago I gave a first demonstration of Siplicity for the iPRES conference (the “bake off” stream). Many thanks to Micky, Dries, and the rest of the conference team for including me in the program.

My presentation was unfortunately a little undercooked. Many of the ingredients were there, but without enough time in the coding oven, and so I focused the presentation on Siplicity’s backend application (the GRPC server), which didn’t make for an appetizing meal.

OK, enough with the cooking puns! Here’s the demo of Siplicity I would have liked to have given:

In this screencast you can see that much of Siplicity’s basic functionality is now implemented in the frontend.

The screencast demonstrates:

adding files and folders to an input graph
manipulating the input graph (select, delete and move nodes to the output graph)
running processes like file format identification and checksum calculation to enrich record metadata
viewing record details and metadata
manipulating the output graph and defining an output destination
formatting the output graph as a bagit package
committing the output graph to disk.

Tech stack overview

Sun, 15 Sep 2024 02:00:00 +0000

Siplicity has a backend written in Go. I chose Go mainly because I am familiar with it but can justify the choice on the basis that it is cross-platform, reasonably performant, and enables native integration with Siegfried.

The frontend is Flutter. This framework is best known for mobile UIs but it now has good desktop and web support too. It is modern, performant, and native-feeling.

The backend and frontend communicate via gRPC. I’ve been using the Buf tooling for this.

I am focused on creating a local, client application but a benefit of this mix of go/flutter/grpc is that it opens the door to other potential deployment scenarios, including local client (native or browser) with a cloud backend (e.g. an AWS machine in the same region as your cloud storage).