@aral it's very definitely a yak, but i have gotten huge mileage out of this approach:
https://code.p1k3.com/gitea/brennen/bpb-kit/src/branch/inertia_intel_nuc/home/.sh_common#L69
...just an alias that brings up a fuzzy-finder for directory history recorded with:
https://code.p1k3.com/gitea/brennen/bpb-kit/src/branch/inertia_intel_nuc/home/.zshrc#L125
it's so useful once there's some history in it that i rarely type `cd` at all.