The second run of Vorta seems to be getting all the files, and it's uploading super fast. This would have taken at least a week with a standard backup service and it's almost done after a few hours.
I'm not sure how much I trust it though after the first run missing most of the data. I'll have to keep an eye on it. I'll give borgmatic a shot if I have further issues with Vorta.