Also, fix the interpolation between visual and directional fractional
offsets. The interpolation now works visually in whatever direction the
result eventually gets resolved into.

Fixes #11847
Fixes #11357