I would love (2) to be merged, because I did (1) ;)
We dealt with that by a simple rule (note: our EC policy is 12 data/3 parity):
if during a SYNC reconstruction we get 404_errors_from_fragments > ndata, we assume it's dark data, and we quarantine the local fragment
It's a pretty simple patch. The threshold might need to be more generic because in some config it would be dangerous (eg: 3 data/6 parity).
I would love (2) to be merged, because I did (1) ;)
We dealt with that by a simple rule (note: our EC policy is 12 data/3 parity): from_fragments > ndata, we assume it's dark data, and we quarantine the local fragment
if during a SYNC reconstruction we get 404_errors_
It's a pretty simple patch. The threshold might need to be more generic because in some config it would be dangerous (eg: 3 data/6 parity).