Also, I should note that I think the code as-is is pretty darn close to commit-ready, IMO.
I want to think more about if/how we might start fast-pathing all non-full reingests -- hopefully we can shard what we have and put pieces back together as compossible units without incurring too much overhead (extra UDF calls) or doing too much repeated work (XML re-parsing), while also making each part simpler. I'll try to find some time in the next couple weeks to look at that ... I'm hopeful, but not overly optimistic, that it's doable with restricted tuits.
Also, I should note that I think the code as-is is pretty darn close to commit-ready, IMO.
I want to think more about if/how we might start fast-pathing all non-full reingests -- hopefully we can shard what we have and put pieces back together as compossible units without incurring too much overhead (extra UDF calls) or doing too much repeated work (XML re-parsing), while also making each part simpler. I'll try to find some time in the next couple weeks to look at that ... I'm hopeful, but not overly optimistic, that it's doable with restricted tuits.