One thing I’ve been assuming we want in serialization the ability to save some object state in a more structured way that can be naturally expressed in some of the standardized file formats we might target (like FITS or HDF5). I think that would need a serialization library that is a bit more extensible than flatbuffers appears to be (at first glance, at least). I think that’s critical if we ever want to be able to describe the data model of any of our objects in a way that would allow them to be read without our code (I continue to be skeptical that we can do this for many objects, but I believe @rhl and @timj have expressed at least somewhat incompatible opinions).
The biggest flaw of our current afw::table::io
serialization library is that it requires all object state to be expressed as a set of structured tables, but it’s also an advantage, because it turns out most of our objects can be mostly represented quite well this way - they just have a small amount of extra state that doesn’t really fit into a table well (it just goes into a single-row table in our current system). I think we could do a lot better if we reimplemented this as a backend extension of a real third-party library, and save the structured data in tables (or images) without stuffing the less-structured stuff into them.
The top hit for “C++ serialization library” on google is Cereal, which looks like it was modeled heavily on Boost.Serialization but is header-only and relies on C++11 instead of Boost itself (much like the relationship between Boost.Python and pybind11). I think it’s worth a close look - I think Boost.Serialization is a good library that we used just a bit too naively last time around, but going back to it now just prolongs our inheritance of the Boost-wide dependency and inheritance problems. I certainly gained a tremendous amount of respect for it after the misadventure of writing my own serialization library.
As a side note, I find it extremely encouraging that reinventing good Boost libraries as standalone C++11 projects on GitHub seems to be becoming A Thing. IMO, that’s exactly what the legacy of Boost should be, and if it’s happened for other Boost libraries it could make it much easier for us to get away from Boost.