Capturing (or not) in provenance the order of processing

Data Management
#1

How much do we care about exact order in which things are processed? For example, say we are creating an object because we detected a new source in exposure A, then we detected a source for that object in exposure B, then C. Is it important to remember the order? This is not handled in the current baseline provenance design, and as far as I know we can’t reliably get the order from mysql unless we introduce some extra gimmicks. And I am asking because this information would somehow need to be kept per-source (or even forcedSource), which would have some non-negligible storage implications, unless we come up with a clever way.

J.

#2

I can’t think of any situations where this sort of tracking would be in the domain of the general provenance functionality. Instead, any time the order of operations can change the result, that will be baked into the algorithms (though it might be at a very high level), and a particular order will be guaranteed. And if it’s a situation where the order doesn’t matter, I don’t think there’s any need to track it.

#3

Can’t you back out this information from the processing provenance information of the relevant exposures and the source tables for those exposures? For example, If an object 1 is in the source table for exposure A then you can look up when source detection was run on exposure A, if object 1 is also in the source table for exposure B then you can look up when source detection was run on exposure B, and so on. So I don’t think provenance would need to be on a object-by-object basis.

#4

Not always, I think. What is exposure A and exposure B are processed in parallel on two machines at the same time. Then we glue it together into an object.

Luckily, if we go with what Jim said (it is up to the algorithms), it makes it simple from provenance perspective.