How does a scanner flatten a curved book page without the cloud?

Photograph an open book and the text near the spine curves away from the camera — perspective correction alone can’t fix that, because the surface itself isn’t flat. Fixing it is a real computer vision problem, and it’s usually solved with a round trip to a server. Talpatra Scan solves it on the phone, using two different techniques stacked together: a classical geometry model and a small neural network, both running locally, with nothing ever leaving the device.

A curved book page isn’t a perspective problem, it’s a surface problem — no amount of “straighten the rectangle” fixes a page that physically isn’t flat.

Why do book pages come out curved in the first place?

Because a bound book doesn’t lie flat. Near the spine, the page lifts off the table and curves away from the camera in a shape that’s roughly cylindrical — flat side to side, curved along the axis running toward the binding. That’s a real 3D surface, not a flat sheet photographed at an angle.

Ordinary perspective correction (the four-corner “straighten this rectangle” transform every scanner app does) assumes the source is planar. It can compensate for a camera that’s tilted relative to a flat page, but it has no way to represent a page that bulges — there’s no 2D homography that unrolls a cylinder back into a flat sheet. Text near the spine stays compressed and curved no matter how you warp the outer quad. You need a model of the actual page surface, not just its four corners.

What’s the classical (non-ML) approach and where does it fall short?

The classical approach models the page as a parametric surface and solves for its shape from evidence in the image itself — most commonly, the way text lines curve. If you can detect where text baselines run across a curved page, their curvature tells you the surface geometry, and you can compute an unwarp that straightens both the lines and the geometry between them.

Talpatra’s classical dewarp path (BookGeometry plus a TextLineSpanDetector in the feature-imaging module) does exactly this: it looks for the horizontal span and curvature of text lines and fits a page-curvature model from that. It’s fast, fully explainable, and doesn’t need a neural network at all — good properties for a fallback path.

Where it falls short: it depends on having legible, sufficiently horizontal text lines to measure in the first place. Sparse text, large images/diagrams, unusual layouts, or a page angle where the curvature signal is weak all degrade the estimate. It’s a real geometry fit, but it’s only as good as the evidence available in that particular frame.

What does the neural half (UVDoc-style mesh dewarp) add, and why run it on-device instead of a server?

The neural half doesn’t need text lines to work at all. Talpatra’s feature-dewarp-ml module runs a UVDoc-style model — a compact ONNX network (8M parameters, from the SIGGRAPH Asia 2023 UVDoc paper) that takes the page image and directly regresses a dense unwarp grid: a mesh of control points describing how to remap every region of the curved image back onto a flat plane. Instead of inferring curvature from text geometry, it’s learned end-to-end from image pairs of warped and flattened documents, so it generalizes to pages where the classical text-line signal is weak — sparse text, mixed text and imagery, odd layouts.

Talpatra runs the classical and neural dewarp as a composited primary/fallback pair (CompositeDewarpEngine), gated by a curvature threshold (BookGeometry.CURVED_PAGE_THRESHOLD) so the extra inference only kicks in on pages that actually look curved. The model runs in fp32 — fp16 was measured 3.3x slower on the CPU execution provider on-device, so the “smaller number format should be faster” assumption didn’t hold here and was rejected on the actual measurement, not on paper.

The on-device part isn’t a footnote, it’s the point. A page you’re scanning might be a passport, a prescription, a contract, or student handwriting — sending it to a server for dewarping means that content leaves the device, even briefly, even to a well-intentioned backend. Running the same model locally means the flattening happens with the same data-never-leaves guarantee as the rest of the app: Talpatra Scan doesn’t hold the Android INTERNET permission at all, enforced by a build-time Gradle check that fails the build if the permission reappears in the merged manifest. A model that only runs on a server isn’t an option under that constraint — so the model had to be one that fits on a phone.

Why not just copy vFlat’s approach?

Because it’s patented, and because “copy the market leader” isn’t actually available here. vFlat’s dewarp (VoyagerX, patent family including US11145037) is a different kind of model: a CNN that looks at a small 192×144 frame and regresses a handful of geometric parameters — page curvature as Bezier terms, plus camera position and orientation — then applies that analytically at full resolution. It’s an elegant approach, built on top of Matt Zucker’s earlier page_dewarp geometry work, and it’s backed by a purpose-built data factory: roughly 100,000 ground-truth image pairs captured from books printed with physical marker grids and photographed at many angles. That data pipeline, more than the model architecture, is the real moat, and it’s not something a solo project can replicate.

UVDoc’s mesh-grid dense-flow approach is architecturally distinct from that parametric cylinder-plus-camera-pose regression — it doesn’t estimate a small set of geometric parameters at all, it directly predicts a full remapping grid. That difference isn’t incidental; it’s the reason Talpatra can ship real book dewarp without walking into vFlat’s patent claim. It also means the honest comparison isn’t “Talpatra invented book dewarp” — vFlat’s is real, and it’s backed by more engineering and more training data than a solo project can match. The claim that holds up is narrower and still true: Talpatra is a fully offline scanner with real book dewarp, composited from a classical geometry fit and a neural mesh model, running entirely on the phone that took the photo.