Hellmann Worldwide Logistics (WAG account)
Hellmann Worldwide Logistics - AI-Powered Shipment Booking And Load Manifest Automation Platform
Bitontree delivered an end-to-end platform that takes a supplier email from the inbox to a validated booking in Hellmann's SmartOrder, and then checks the loaded cargo against what was booked.
Hellmann Worldwide Logistics - AI-Powered Shipment Booking And Load Manifest Automation Platform
About The Client
Hellmann Worldwide Logistics is a global logistics provider. For one of its contract-logistics accounts, WAG (Worldwide Automotive Group), a South Africa-based automotive parts distributor, Hellmann manages inbound shipments of spare parts coming in from overseas manufacturers.
Every one of those shipments starts as an email. Suppliers send commercial invoices, packing lists, and house bills of lading as PDF, Word, and Excel attachments, each in a different layout. Hellmann's operations team then had to read them, match every line to the account's purchase-order master, key the data into Hellmann's SmartOrder system as a Vendor Shipment Booking, and later check that what was booked is what was actually loaded into the container.
Hellmann needed that chain turned into a controlled, auditable workflow: read the supplier emails automatically, extract and validate the shipment data, let a person review the exceptions, and send clean booking and load-manifest XML to SmartOrder.
| Project Detail | Information |
|---|---|
| Client | Hellmann Worldwide Logistics (WAG account) |
| Industry | Logistics, Contract Logistics, Freight Forwarding, Automotive Parts Import Operations |
| Location | TODO: Confirm Hellmann entity |
| Duration | TODO: Confirm duration |
| Project Scope | Operations console (frontend), backend API and database owner, and mail-processing and extraction service |
| Services Used | AI Automation Development, Document Intelligence, Custom Software Development, Workflow Automation, Logistics Integration, Cloud Engineering |
| Engagement | End-to-end product engineering across frontend, backend, mail automation, and SmartOrder integration |
| Live Link | https://prod.wag.logiai.info/ |
| Client Contact / Testimonial | TODO: Confirm if a testimonial or client quote can be included |
The Objective
Hellmann wanted to stop re-typing supplier paperwork into SmartOrder, and to catch discrepancies between what was booked and what was loaded before they became shipping problems.
The goal was not a generic OCR tool. The platform had to understand how each supplier actually sends documents, produce the exact XML SmartOrder expects, and give Hellmann's operations team a fast way to review and correct what the AI extracted.
The platform needed to:
- Monitor the account's shipping mailbox and pick out supplier invoice emails and packing-list / bill-of-lading emails
- Extract invoice, PO, item, quantity, weight, volume, container, and port data from PDF, Word, and Excel attachments
- Handle supplier-specific document layouts through supplier profiles rather than one brittle template
- Match every extracted line against the purchase-order item master and fill or flag missing item codes
- Generate Vendor Shipment Booking (VSB) XML and send it to SmartOrder
- Receive SmartOrder's booking confirmations (VEN number and carrier S-REF) through authenticated webhooks
- Ingest packing lists and house bills of lading when cargo is loaded, and generate Load Manifest (LM) XML
- Reconcile the booked shipment (VSB) against the loaded shipment (LM) line by line and surface every difference
- Let reviewers see the source document next to the extracted data, edit it, and approve before anything is sent
- Give managers a dashboard, activity feed, and full audit trail across every request
The Challenge
Inbound freight paperwork is messy in a way that generic automation does not survive. The same shipment passes through several documents, several suppliers, and two stages that must agree with each other.
The key challenges included:
- Suppliers use different invoice and packing-list layouts, languages, and file types, and some bills of lading arrive as scanned images
- Large invoices exceed what a single AI call can read reliably, so long documents had to be split without losing PO continuity or header context
- Excel documents come in many shapes, with total rows, merged headers, and multi-sheet batches that break naive parsing
- Item codes written by suppliers rarely match the account's PO master exactly, because of spacing, punctuation, and formatting differences
- One shipment can contain several invoices, and one invoice can span several purchase orders
- SmartOrder expects strictly structured XML, and a wrong value can delay a booking or a customs clearance
- SmartOrder answers asynchronously, first with a VEN number and later with a carrier booking reference, and those callbacks can arrive out of order, repeat, or arrive before the platform is ready
- The packing list and bill of lading arrive later and must be linked back to the original booking, with container numbers and seals that may not match the plan
- Booked and loaded quantities can differ, so the team needs a reliable comparison rather than a visual check
- Reviewer corrections must never be silently overwritten when the PO master is refreshed or a document is re-processed
- Several services share one database, so schema ownership, state transitions, and audit rules need to be explicit
- Mailbox processing must tolerate restarts and duplicates without creating duplicate shipments
Hellmann needed a system that handled the repetitive reading and matching, while keeping its people in control of the decisions that carry customs and cost risk.
Our Approach
Bitontree designed the platform as three parts: a Next.js operations console for Hellmann's reviewers, a FastAPI backend that owns the database and business workflows, and a Python mail-and-extraction service that runs the inbox poller and the AI pipelines.
The backend owns the schema, migrations, authentication, review workflow, outbound XML, webhooks, and reconciliation. The mail service owns Gmail/IMAP polling, attachment validation, blob upload, document extraction, and ingestion. A written backend contract defines which service may write to which table, which request states are valid, and which changes must be audited.
Our approach focused on:
- Treating the supplier email as the start of a tracked request, with a unique request ID that follows the shipment end to end
- Using AI for reading unstructured documents, while keeping XML generation, matching rules, state changes, and sending deterministic and reviewable
- Building supplier-aware profiles so each supplier family gets the right classification strategy and prompts
- Designing reconciliation as a stored, incrementally refreshed result so the queue stays fast under volume
- Protecting human edits as a first-class rule across every refresh and re-extraction path
- Keeping an explicit contract between services so a shared database stays safe
Core Features Developed
AI-Powered Mail Intake And Classification
We built a mail service that polls the shipping mailbox for unread messages with attachments, using the Gmail API with an Outlook client available behind the same interface.
Each message is classified as a VSB email (supplier commercial invoices) or an LM email (packing list and bill of lading), and the body is parsed for shipment hints such as port of loading, container type and count, PO references, and cargo-ready date. Body hints are preferred over defaults when they are present.
The poller records an audit event for every message, claims messages so two workers never process the same one, retries failures up to a limit, and deduplicates by message and by request, so restarts and re-deliveries do not create duplicate shipments.
Supplier-Aware Document Routing
Different suppliers need different handling, so the pipeline resolves a supplier profile before extraction.
A profile decides how attachments are classified (by content first or by filename first), which default supplier code applies, and which supplier-specific prompts are used. This lets a new supplier be supported by adding a profile and a prompt set instead of changing the pipeline.
Multi-Format Invoice Extraction (VSB)
We built the invoice extraction pipeline for PDF, Word, and Excel documents.
For PDF and Word, the document text is split into bounded chunks, and each chunk carries the header and previous-chunk context needed to keep PO and invoice continuity, while the model is told to emit items only from the current chunk. For Excel, AI is used to map columns and understand the layout, and rows are then extracted locally and deterministically, with spreadsheet total rows skipped. Multi-invoice shipments are processed as one bundle.
Extraction can run against several model providers, with configured timeouts, retries, and token limits, and a confidence consolidator reconciles the results into per-field confidence values.
Packing List And Bill Of Lading Extraction (LM)
We built a second pipeline for the loading stage. It reads packing lists in PDF, Word, and Excel and house bills of lading, including scanned bills that are transcribed first.
It extracts container numbers, seals, container types, packages, vessel, voyage, carrier, ports, and per-line order, item, quantity, weight, and volume. Supplier-specific prompts handle each supplier's packing-list layout, and source container data is used to fill gaps.
PO Master Matching And Item-Code Backfill
We built a purchase-order item master that is imported from Excel PO reports received by email, and a matching layer that links every extracted line to it.
Matching normalises case, whitespace characters, and punctuation, applies strict item-identity rules, and backfills missing or blank item codes from the master or from the matching VSB. Lines that cannot be matched are kept visible and separated into adjusted items and missing items, so nothing is silently dropped or invented.
Review Console With Side-By-Side Source Documents
We built a review workspace where a reviewer sees the original document next to the extracted data.
The console includes PDF, Word, and Excel viewers, field-level confidence chips, searchable extraction results, an editable table, bulk review for large line sets, and a jump-to-source action from a field to its place in the document. Reviewers can correct fields, fix item codes with a preview-then-apply flow, and approve PO master exceptions before the XML is generated.
Human Edit Protection
Reviewer corrections are tracked with field-level provenance and recorded as correction data.
When the PO master is refreshed or a request is re-processed, manually edited fields are preserved instead of overwritten, and the interface shows which edits were kept. This was important for trust: reviewers need to know that fixing a value once is enough.
VSB XML Generation, Validation, And Sending To SmartOrder
We built the generator for the Vendor Shipment Booking XML that SmartOrder expects.
The backend produces an XML preview, validates it against the outbound schema, supports download, and sends it to Hellmann's SmartOrder endpoint with the request ID embedded as the correlation number. Every send attempt is recorded with the response, duplicate uploads are blocked, and a notification email with the XML can be sent to the operations team. Automatic sending is configurable so the business can start with manual approval and relax it later.
Authenticated SmartOrder Webhooks
We built the inbound integration that receives SmartOrder's confirmations.
A VEN webhook stores the booking number against the right request, and an S-REF webhook stores the carrier booking reference. Both are protected by a static key header, accept the field-name variants the sender might use, and are idempotent, so repeated calls update the same state. Events that cannot be matched yet are acknowledged as pending and every attempt is stored, so late or out-of-order callbacks can be resolved without losing them.
Each booking moves through a clear lifecycle: sent, VEN received, S-REF received, and LM sent.
Load Manifest Workflow
When the packing list and bill of lading arrive, the platform links them back to the original booking, validates them, and prepares the LM XML.
Validation classifies each LM as ready, requiring container review, or requiring line-item review. Container mismatches between the plan and the documents have a dedicated review screen, and reviewers can select containers and reassign items between them. LM XML can be previewed, downloaded, sent manually, or sent automatically on a schedule once the data is ready.
Line-By-Line Reconciliation
We built reconciliation between what was booked in the VSB and what was loaded in the LM.
Every row is classified as unchanged, changed, removed, added, or incomplete, with totals for VSB and LM lines. Reviewers can resolve differences, apply their decisions, and regenerate the booking data, and the platform can send mismatch and summary notifications by email.
Stored Reconciliation Queue At Scale
Reconciliation results are stored as snapshots rather than recomputed on every page load.
Database triggers mark a summary stale whenever a related source row changes, whether the change came from the backend or the mail service. A background worker claims pending summaries safely across multiple API processes, refreshes them transactionally, and retries failures with a delay. The queue API is paginated and filterable by status, supplier, and urgency, shows an updating state instead of stale counts, and closes entries immediately once an LM is sent or manually closed.
Dashboard, Activity Feed, And Audit Trail
The console includes a dashboard with request summaries, daily volume, per-supplier metrics, and request-ID capacity, plus a live activity feed delivered over server-sent events with read tracking.
Email receipt, extraction, edits, sends, and webhook events are logged against the request, giving the team one place to see what happened to any shipment.
Supplier Onboarding
We built a supplier-template onboarding screen so new supplier layouts can be introduced into the workflow without redesigning the platform.
Secure Access Control
The backend provides login with short-lived access tokens and rotating refresh sessions, hashed passwords, failed-login rate limiting, role-based users, and protected console routes. The frontend talks to the backend through server-side route handlers so credentials and tokens stay out of the browser code.
Safe Deployment And Schema Ownership
The backend owns all migrations and ships a runbook for promoting changes through staging, UAT, and production in order, with a safe command to confirm the target database before any write. Database pool sizing guidance protects the shared database from connection exhaustion.
Technology Stack
| Layer | Technology |
|---|---|
| Frontend | Next.js 14, React 18, TypeScript, react-pdf / pdf.js, docx-preview, SheetJS, react-hook-form |
| Backend API | Python 3.12, FastAPI, Pydantic, SQLAlchemy 2, Alembic, Uvicorn |
| Mail And Extraction Service | Python, FastAPI, Gmail API, Outlook client, Jinja2 prompt templates, PyMuPDF, pypdf, python-docx, openpyxl, pandas |
| Database | PostgreSQL (SQLite for local and test runs) |
| AI / LLM | Google Gemini, Azure OpenAI (GPT-5.2), Grok via Azure, supplier-specific prompt sets, confidence consolidation |
| Observability | LangSmith tracing, structured logging, mail audit events |
| Integrations | Hellmann SmartOrder (XML upload and VEN / S-REF webhooks), HTML and plain-text notification email |
| Storage | Azure Blob Storage |
| Authentication | JWT access tokens, rotating refresh sessions, Argon2 / bcrypt password hashing, login rate limiting |
| Deployment | Azure Static Web Apps (frontend via Azure Pipelines), Azure-hosted backend, mail service as Linux systemd units (API and poller) |
| Testing And Quality | pytest, Ruff, mypy (strict), Jest, Testing Library, TypeScript checks |
Product Architecture
| System | Technology | Role |
|---|---|---|
| Operations Console | Next.js, React, TypeScript | Shipments, review workspace, reconciliation queue, dashboard, suppliers, PO master, activity feed |
| Backend | Python, FastAPI, PostgreSQL | Schema and migrations owner, auth, review APIs, outbound VSB and LM XML, SmartOrder send, webhooks, reconciliation, reporting |
| Mail Service | Python, FastAPI | Mailbox polling, classification, attachment validation, blob upload, VSB and LM extraction, PO report import, ingestion |
The Result
Bitontree delivered an end-to-end platform that takes a supplier email from the inbox to a validated booking in Hellmann's SmartOrder, and then checks the loaded cargo against what was booked.
The system brings mailbox intake, AI document extraction, PO master matching, human review, XML generation, SmartOrder webhooks, and reconciliation into one auditable workflow.
| Outcome | Result |
|---|---|
| Mail operations | Supplier invoice and packing-list emails converted into tracked requests automatically |
| Document extraction | PDF, Word, Excel, and scanned documents turned into structured shipment data |
| Supplier handling | Supplier-specific profiles and prompts for each supplier family |
| Master data | Extracted lines matched to the PO master, with missing and adjusted items shown separately |
| Review | Side-by-side source viewer, confidence chips, editing, and bulk review before sending |
| Booking | Validated VSB XML previewed, downloaded, and sent to SmartOrder with the request ID as the correlation number |
| SmartOrder callbacks | VEN and S-REF confirmations received through authenticated, idempotent webhooks |
| Load manifest | Packing lists and bills of lading linked to bookings, validated, and turned into LM XML |
| Reconciliation | Booked and loaded shipments compared line by line, with changes, additions, removals, and gaps surfaced |
| Scale | Stored reconciliation snapshots kept the queue fast with background refresh |
| Edit safety | Reviewer corrections preserved across PO master refreshes and reprocessing |
| Visibility | Dashboard, per-supplier metrics, live activity feed, and audit trail across every request |
| Platform foundation | Three-service architecture with an explicit database contract and a promotion runbook |
TODO: Add approved operational benchmarks if the client confirms them, such as documents processed per week, manual entry time before and after, or extraction accuracy on reviewed fields.
Business Impact
The platform gives Hellmann a controlled operating system for inbound shipment paperwork on this account.
Instead of re-keying supplier documents into SmartOrder and comparing booked and loaded quantities by eye, the operations team works through a structured queue where each shipment has a request ID, a source document, extracted data, a review state, and a full history.
The final solution helped Hellmann:
- Reduce manual effort in reading supplier invoices, packing lists, and bills of lading
- Reduce data-entry errors by matching extracted lines against the PO master before booking
- Send consistent, schema-validated booking and load-manifest XML into SmartOrder
- Keep a human review step on the decisions that carry customs and cost risk
- Detect booked-versus-loaded differences before they turn into shipping or clearance problems
- Track each shipment from email receipt through VEN, S-REF, and load manifest in one lifecycle
- Protect reviewer effort so corrections are not lost when master data changes
- Give managers visibility into volume, supplier mix, and queue status
- Reduce duplicate-shipment risk through message claiming, deduplication, and idempotent callbacks
- Onboard new suppliers through profiles and prompts instead of a rebuild
- Build a foundation that can be extended to other contract-logistics accounts and further automation, such as broader auto-send
Key Engineering Problems We Solved
Reading Long, Messy Invoices Reliably
Large invoices do not fit in a single reliable AI call. We built bounded chunking that carries header and previous-chunk context, instructs the model to emit items only from the current chunk, and merges the results, so PO and invoice continuity survives across chunks.
Using AI For Excel Without Trusting It With Every Row
For spreadsheets, we use AI to understand the layout and map columns, then extract rows locally and deterministically. This keeps the cost and risk of AI low on large sheets and avoids the model inventing or dropping rows.
Matching Supplier Item Codes To The PO Master
Supplier item codes differ from the master in spacing, punctuation, and formatting. We built normalisation and strict identity matching, with backfill from the master and from the booking, and a clear split between adjusted and missing items so reviewers see exactly what needs attention.
Correlating An Asynchronous SmartOrder
SmartOrder confirms bookings later and in separate steps. We embedded the request ID as the correlation number, made both webhooks idempotent and key-protected, stored every attempt, and acknowledged unmatched events as pending, so the system stays consistent even when callbacks are late, repeated, or out of order.
Keeping Human Edits Safe
Automation that overwrites corrections destroys reviewer trust. We tracked field-level edit provenance and enforced preservation across PO master refreshes, re-extraction, and reconciliation, and made the preserved edits visible in the interface.
Making Reconciliation Fast At Volume
Recomputing comparisons on every page load does not scale. We stored summaries, invalidated them with database triggers so changes from any service are caught in the same transaction, and refreshed them with a lock-skipping background worker that is safe across multiple API processes.
Handling Container Differences Between Plan And Reality
Container numbers, seals, and assignments often change between booking and loading. We built validation states, a container-mismatch review flow, and item reassignment between containers, so these differences are resolved by a person instead of corrupting the manifest.
Sharing One Database Across Services Safely
We wrote a backend contract that defines table ownership, valid request states, audit rules, and that schema changes start as backend migrations. This kept the mail service and the API from drifting apart.
Making Mail Automation Safe To Restart
Mailbox processing needed to survive crashes and re-delivery. We added message claiming with a time limit, bounded retry attempts, dedupe at message and request level, and audit events for every step.
Combining AI With Human Review
AI reads and extracts, but matching rules, XML generation, sending, and reconciliation decisions stay deterministic and reviewable. Auto-send is configurable, so the business controls how much autonomy to grant.
Development Timeline
| Phase | Deliverables |
|---|---|
| Phase 1 | Extraction prototypes for PDF, Word, and Excel invoices; chunking; confidence consolidation |
| Phase 2 | Backend foundation: schema ownership, authentication, request APIs; mail service structure, Gmail polling, blob storage |
| Phase 3 | VSB ingestion, supplier detection, review console with source viewers, activity feed, and lifecycle view |
| Phase 4 | VSB XML generation and validation, SmartOrder sending, notification emails, PO master import and matching |
| Phase 5 | LM extraction, packing-list and bill-of-lading pipelines, LM XML, container validation and review |
| Phase 6 | Reconciliation endpoints, stored snapshots with background worker, paginated queue, reviewed-resolution apply flow |
| Phase 7 | SmartOrder webhooks, supplier-family hardening, edit protection, deployment pipelines, migration runbook, test coverage |
TODO: Confirm phase boundaries and dates with the delivery team before publishing.
Why This Project Stands Out
Hellmann needed more than an OCR tool or a generic document parser. Inbound freight has its own rules: supplier-specific layouts, a PO master that never quite matches, a booking system that answers asynchronously, and a booked-versus-loaded comparison that has to be trusted.
Bitontree built the platform around that real workflow. AI does the reading, the system does the matching and the paperwork, and Hellmann's team keeps control of every decision that matters, with a complete history behind each shipment.