A web cockpit over a Python PrestaShop → Shopify Plus toolkit: a 16-step dependency-ordered pipeline, live progress over SSE, and an entity browser that lets stakeholders approve data before it touches production. It boots against mock PrestaShop and Shopify servers, so the whole flow runs with no credentials and no store.
The step table
Every step names the steps it depends on. The engine topologically orders them, runs independent branches concurrently, and stops a branch without killing the run. Adding a step means adding a row — the sequence falls out of the dependencies rather than a hand-written order that drifts.
server/pipeline/steps.py
# Each step declares what it needs. The engine # resolves order and runs what it can concurrently. STEP_DEFINITIONS: list[StepDef] = [ StepDef("extract.products", "Fetch products", "extract"), StepDef("extract.combinations", "Fetch combinations & attributes", "extract", ("extract.products",)), StepDef("import.customers", "Create customers", "import", ("extract.summary",)), StepDef("import.companies", "Create B2B companies & catalogs", "import", ("import.customers",)), ] # B2B companies cannot exist before their contacts. # The dependency says so; nothing else has to.
What's Inside
Seven extract steps and nine import steps behind one surface: configure connections, run the sequence, inspect what came out, and hand a report to whoever asked for one.
demo-modeIn-process mock PrestaShop and Shopify servers boot alongside the app, generating multilingual catalog records. The full extract → import → report flow completes against them, so the tool can be demoed, screenshotted, or handed to someone without a store.
pipelineSixteen steps declare their prerequisites; the engine resolves the order, runs independent branches concurrently, and supports cancellation mid-run. B2B companies wait on customers because the step table says so.
sseThe run streams over server-sent events — per-step status, counts, and errors as they happen. Stakeholders watch a migration proceed instead of waiting for a summary email afterwards.
dual-importDirect GraphQL Admin import for speed, or a Matrixify workbook the client reviews in a spreadsheet and approves before anything is written. Same extracted JSON feeds both.
data-browserEvery extracted entity is browsable — products, variants, customers, pages, redirects — so a mapping mistake is caught while it is still a row in a table, not after it is live in a store.
reportsEach run produces a summary, ID mappings from source to target, and an error log, downloadable as files. The ID map is what makes a re-run safe rather than a duplicate-creation event.
Design Principles
The Python scripts worked before the cockpit existed. What they could not do was let a non-engineer watch a run, check what was extracted, and approve it — so the cockpit adds orchestration and visibility on top rather than replacing a pipeline that already ran.
Mock servers mean the tool is reviewable without production credentials — no staging store to provision, no client data in a screenshot. That constraint also keeps the engine honest: anything that only works against a real store is a coupling worth removing.
The Matrixify path exists because merchandising teams review catalogs in Excel, not in a GraphQL client. Meeting them there turns the risky step — first write to a live store — into something signed off in advance.
More Work