Document pipeline
Receive PDFs through a public form, extract their tables, normalize the data and store clean records in a database — with an S3 archive.
Invoices, statements, reports: documents arrive messy, data should land clean. This pipeline does the boring part.
The pattern
Form Input (file upload) → PDF Extractor → Pandas Processor → Database
└→ S3 Storage (original archive)Step by step
- Form Input (Public Interfaces) — a file field accepting PDFs; whoever needs to submit documents gets the public link.
- PDF Extractor (Data) — pulls the tables out of the PDF into structured rows.
- Pandas Processor (Data) — normalize: rename columns, fix types, drop empty rows, compute derived fields.
- Database (Data) — insert the clean records into your MongoDB collection.
- S3 Storage (Data) — parallel branch: archive the original file for audit.
Variations
- Add an LLM Agent after extraction to classify the document type or flag anomalies before saving.
- Batch mode: a Loop node processes multi-file submissions one by one.
- Close the loop with a Data Table public view so the team browses processed records.
Genius prompt: "Create a document pipeline: public form with PDF upload, extract tables, clean with pandas, save to database and archive the original in S3."