Image2Other Research prototype

Edit-Preserving Reconstruction of Scientific Figures

Cheng Chen

Tongji University

Working title · ongoing research

Interactive demo Code Research notes
Concept for edit-preserving figure reconstruction A flattened image is reconstructed into editable labels, shapes, and anchored connections. Moving one component should preserve its connection and leave another component unchanged. SOURCE IMAGE EDITABLE RECONSTRUCTION LOCAL EDIT Model A → Model B one flattened pixel layer Looks like a diagram. Edits like a photograph. Model A Model B EDITABLE LAYERS text · vector shapes · attached arrow Each element can be selected and revised. Model A Model B DESIRED BEHAVIOR connection follows · other regions stay put An evaluation target, not a claimed result.

Swipe to inspect the figure ↔

Figure 1. The research goal: recover editable structure from pixels, then test whether local edits preserve labels, connections, and untouched regions. Concept illustration · not a measured reconstruction result.

Abstract

Scientific figures are often available only as flattened images, making even a small revision difficult. Existing conversion workflows may recover visual similarity while leaving labels, shapes, and connections hard to edit independently. We study a stricter reconstruction goal: an editable scene whose text is legible, whose relationships remain valid after a local change, and whose untouched regions stay stable. Our current prototype combines reviewed OCR suggestions, vector tracing, scene-level editing, and render–compare feedback. These components define an engineering baseline and a candidate evaluation protocol; they do not yet establish a new trained method or measured research result.

Research direction

From pixels to an editable scene

The current system separates recognition, geometry, and relationships rather than treating the exported SVG as one traced image.

01

Recover text

OCR proposes labels and locations; users review the suggestions before glyph pixels become native SVG or PowerPoint text.

02

Rebuild artwork

Raster regions are traced into vector paths. The scene editor supports explicit shapes, lines, and manually assigned connector targets.

03

Render and compare

The exported scene is rendered back to pixels. A bounded local position search can reduce image error for selected layers.

04

Test local edits

Planned experiments will ask whether edits preserve text, connections, and untouched regions—and whether a compact model helps.

Engineering prototype

Try the current converter

Upload a figure, review OCR suggestions, trace artwork, edit scene layers, and export SVG or PowerPoint. The live tool also includes separate image and document converters.

Image2Other browser studio with editable layers and conversion controls
Figure 2. The current browser-based editor. Output quality varies with the input and requires review. Open the interactive demo ↗

Evaluation plan

Measure what survives an edit

A candidate benchmark would pair source figures with verified structure and controlled edit tasks. The axes below are a plan, not published results.

Visual fidelityRender the scene and compare it with the source, including local difference maps.
Text correctnessVerify labels, formulas, and placement against human-checked references.
Relation validityCheck whether connectors retain the intended endpoints after components move.
Edit localityQuantify unwanted changes outside the region deliberately edited by a user.

Pixel error alone cannot certify semantic correctness; held-out tasks and human-checked labels are needed before claiming improvement.

Project status

Available now: a browser converter, editable SVG/PPTX export, a reviewed text workflow, and an optional local PaddleOCR bridge.

Still open: fonts and semantics remain approximate; equations, handwriting, rotated labels, and automatically inferred connections are difficult. No trained model weights or benchmark dataset have been released. A paper and controlled evaluation are still in progress.

Interactive demo ↗Code and documentation ↗