View Ticket
Ticket UUID: feab8a3c9b040a2fea00b536f7a300d6256cc9fd
Title: Performance profiling
Status: Open Type: Test
Severity: UNSPECIFIED Priority: 2_Medium
Subsystem: Core Resolution: Open
Last Modified: 2017-04-12 14:33:24
Version Found In:
Write tools to:
  1. Test the raw throughput of a storage
  1. Test the snapshot speed with lots of small files (per-file overhead)
  1. Test the snapshot speed with large files (raw throughput)
  1. Test the speed of scanning directories trying to find changed files with the file-cache
  1. Test the extraction speed

...with standard test corpuses.

User Comments:
alaric added on 2017-04-12 14:33:24:

I've done some speed trials of a single 93MiB file from my laptop, locally and via ssh to anger.

I snapshotted the file twice, so the second time the blocks already exist.

Local, sqlite backend: 5.489s / 0.971s

Local, splitlog backend: 5.467s / 1.090s

Anger, sqlite backend, internal disk: 71.972s / 2.025s

Anger, splitlog backend, internal disk: 77.204s / 2.031s

Anger, sqlite backend, USB disk: 71.803s / 1.825s

Anger, splitlog backend, USB disk: 64.665s / 2.123s

Clearly, the big problem is going via ssh to anger. Let's try ssh to localhost and check it's ssh rather than anger.

Localhost, sqlite backend: 14.882s / 3.188s

Ah-hah. Seems anger is at fault!