Skip to main content
SCRAPERS & DATA

Gets data out of systems with no export button

One repeatable route from a screen you can only read to a clean, typed Excel sheet, used across four internal systems.

THE PROBLEM

What was in the way

An internal operations team needs a list that the system holding it will not hand over. The screen shows one record at a time, the export is missing or returns a dump nobody can work with, and what they need is spread across a nested tree, a week-by-week scheduling view and detail popups.

So somebody opens the app and retypes it. That takes days, it is stale by the time it is finished, and it has to happen again next quarter.

THE APPROACH

How it was built

The output contract stays fixed: one typed Excel sheet, real column widths, one row per record even when the source screen packs several into a single entry. Where a documented API exists, the script pages through it and lets the server join the related record. Where none exists, a browser walks the app the way a user would.

These systems sit behind single sign-on with conditional access, so a freshly launched browser fails before it starts. Four scripts, one technique applied four times. Each is small enough to read in an afternoon and change a column.

Locked screen to typed sheet
Locked screen to typed sheet
HIGHLIGHTS

How it works

The session is borrowed, not recreated

Each run copies the browser profile the user is already signed in with, token store included, into a temporary directory. Conditional access passes because it is the same signed-in state, and the script never touches a credential.

The walker learns the UI at runtime

Element ids change every session, so the tree walker clicks one icon, watches whether the row count grows, and learns from that which icons mean a category. A record counts as extracted only once the detail panel's title actually changes.

An interrupted run costs nothing

Rows are written and saved one at a time, keyed so a rerun recognises them. The next run picks up at the last exported date instead of walking the whole range again.

RESULTS

What it does now

349 variablespulled out of a config dialog with no export
resumes mid-exportevery row saved on write, reruns skip it
no fresh loginreuses the session that already passed SSO
STACK

What it is built on

Pythonscripts and entry points
Playwrightdrives the legacy UIs
openpyxlincremental Excel writing
Microsoft GraphAPI path where one exists
Pydantictyped config and row models

Got something in your workflow that looks like this?