amuck-landowner

Self-hosted Distributed Storage

Francisco

Company Lube
Verified Provider
Question for everyone tinkering with SyncThing - Does it sync files opened for writing?

I'm mostly curious if it syncs streaming data. Namely, you're WGET'ing a file to the FS,

is it syncing all of that on the fly or does it wait for the file to be complete before

blasting it off?

Francisco
 

drmike

100% Tier-1 Gogent
Question for everyone tinkering with SyncThing - Does it sync files opened for writing?


I'm mostly curious if it syncs streaming data. Namely, you're WGET'ing a file to the FS,


is it syncing all of that on the fly or does it wait for the file to be complete before


blasting it off?
Good question....   

I'll fire up a wget here in a few and see... big ISO maybe :)
 

drmike

100% Tier-1 Gogent
So @Francisco  SyncThing.....

Rescan Interval

30 s

Lowered that from 60s.

Ran Debian ISO of 4GB~ DVD.  Slow downloading (BW + VPN lag).

SyncThing on the remote end (this in datacenter where I am replicating to):

Global Repository 237 items, ~1.19 GiB  Local Repository 237 items, ~634 MiB  Out Of Sync

On the local SyncThing master (here on my desk):

Global Repository 237 items, ~1.24 GiB  Local Repository 237 items, ~1.24 GiB  Out Of Sync 0 items, 0 B

The local repository continues to group in GiB size:

Global Repository 237 items, ~1.25 GiB  Local Repository 237 items, ~1.25 GiB  Out Of Sync 0 items, 0 B

On the local master repository here on my desk:

Download Rate 61 bps (261 KiB)  Upload Rate 208 kbps (54.4 MiB)  Address -:22000  Synchronization 74%

The upload rate = current throughput and total transfer to remote since this session was initiated.   This will at times drop and reconnect and thus may be subject to reset of values.

So it *apears* the remote replication is happening while I download the file locally with wget.

I need to figure out where SyncThing stashes partial file transfers like this to confirm where we are.
 
Last edited by a moderator:

drmike

100% Tier-1 Gogent
The partial transfer file in SyncThing is given a period prefixed name:

.syncthing.debian-7.6.0-amd64-DVD-1.iso

= 70M

wget on the local server is up to about 750M currently....

Yes, my upload speeds are much slower than download speeds.

Will continue to watch and see how this does.
 

drmike

100% Tier-1 Gogent
That ISO is still chugging, slowly along on the replication.

Gets mega confusing if you have multiple web interfaces open for the end points in your SyncThing setup.  Cause on the master progress percentages aren't updated until the file is finished and complete on the other side.   In this instance I have like 1.6GB uploaded to remote and another 2GB+ to go on mirroring the ISO.

It appears to make some attempt to start/restart the files out of sync.   Waiting until done to see total bandwidth chewed up, but so far, looks right for file it is moving and not multiple retries or anything weird on full parts of files wasted/flushed...

I need to send up two ends on remote servers to pace this properly...  This week :)
 
Top
amuck-landowner