Notes on consolidating snapshots/archives

  • Files could be duplicated between:

    • Snapshots

    • Archives

    • Backups

  • Wasting space by keeping zipped version of each

    • Option to delete the zip, show via icon

    • If deleted, will need to re-zip on download

  • Don't touch the backups

    • But can look there to extract if needed?

  • Take advantage of the PreviousUID to complete the chain

  • Think about -- how do we know which archives belong to what if they're all mixed up?

  • Redesign

    • main snapshots store NO archives

    • ALL archives in single _archives folder

    • Archives not zipped (zipped on demand)

    • When saving snapshot, don't offer archive options, just save the snapshot, and copy any archives that are not already in the single archive folder to it

    • When downloading snapshot, provide option to include a range of archives

    • On Snapshot List page, scan for missing archives and show a big Warning icon for any snapshots that are incomplete

    • On Archives page, show a list of archives that don't belong to any snapshot, with option to purge them

  • Endpoints

    • /list

      • gets list of snapshots only (not archives)

      • for each snapshot, check archives are present, any missing ones get listed in return list

      • also return "orphanedArchives" list

      • also return "currentArchivesNotInStore" list

      • Migration script to copy all existing archives into archive_store folder

    • /take

      • Normal snapshot save

      • Collect archives, copy any not present in archive store into it

      • Add code to populate totalFileSize of archive data that doesn't currently have it

    • /use

      • Same as current, give alert for any missing archives as it does now (but location will be different)

    • /upload

      • As current, but if it includes "archives", move them to archive store if not already there

    • /delete

      • As per, don't delete any archives

    • /purge-orphans

      • NEW: deletes any archives that don't belong to a snapshot

    • /download

      • takes an "archive range" parameter

      • zips on demand (no zips for normal snapshots stored)

      • keep a "zip cache" so requests with the same archive range can be re-downloaded without re-zipping.

      • Do zip as separate endpoint (or option of this one) so we can report progress in UI

      • Fix downloads to use the "correct" approach

    • /take-archive (NEW)

      • copies the current system archives to the snapshot archives store