Large-Scale Automated Image Extraction and Face Indexing from Public Document Archives

This document describes the automated processing of approximately 1,000,000 publicly released PDF files obtained from United States Department of Justice and Congressional archives. The objective was to evaluate the feasibility of large-scale image extraction, face detection, and person clustering within heterogeneous legal document collections.

The processing pipeline operated entirely offline and left all source documents unmodified. Extracted visual content was indexed and analyzed using automated face detection, demographic classification, clustering, and metadata tagging techniques.

This report outlines the technical architecture, methodology, observed outcomes, and limitations of the system.

1. Background and Problem Context
2. Dataset Characteristics
3. Processing Pipeline
4. User Interface and Scalability
5. Evidence Integrity and Reporting
6. Limitations
7. Observations
8. Hardware Configuration and Processing Environment
9. Parallel Processing Architecture
10. Detection Performance and Observed Metrics
11. Image Segmentation and Fragment-Level Analysis
12. Throughput and Scalability Metrics
13. Operational Characteristics
14. Intended Use Context
15. Operational Outcome Statement
16. Practical Implications
17. Final Conclusion

Large document releases from governmental investigations frequently consist of scanned PDF files containing embedded images. These documents present several technical challenges:

• Image content is embedded within PDFs and not directly accessible to standard image-processing tools.
• Scanned pages vary in resolution and quality.
• Images may contain partial faces, reflections, drawings, or degraded content.
• Manual review at scale (hundreds of thousands to millions of documents) is operationally impractical.

The goal of this study was to determine whether automated visual indexing could significantly reduce manual review time in such datasets.

• Approximate total files processed: ~1,000,000 PDF documents
• Source: Publicly released federal investigative document archives
• File type: Mixed scanned PDFs (image-based pages, some containing embedded raster images)
• Processing mode: Fully offline
• Source integrity: All original PDF files remained unmodified

• Batch ingestion of PDF documents
• Page-level image extraction
• Identification of embedded raster images

Automated face detection was executed across extracted images (including page renders and embedded rasters), with high-recall behavior across degraded and non-standard visual content.

Detected faces were optionally classified for demographic attributes to support triage and filtering.

A minor-redaction option can be applied to outputs to reduce exposure of potentially sensitive content during review workflows.

Each detected face was tagged with metadata to preserve provenance (document linkage) and enable downstream filtering and reporting.

Faces were clustered into “People” as an organizational construct for navigation, grouping, and reporting within the dataset.

Cross-reference queries were supported to connect faces back to document sources, enabling targeted review and triage.

(Section outline placeholder) UI elements support browsing extracted faces, navigating People, and performing search and export operations at large scale.

• Offline operation (no cloud dependency)
• Non-destructive pipeline (original PDFs unmodified)
• Traceability: extracted artifacts linked to source documents/pages

(Section outline placeholder) Limitations may include scan quality variability, partial/occluded faces, and dataset heterogeneity affecting detection and clustering behavior.

(Section outline placeholder) High-recall detection behavior was observed across photographs, reflections, drawings, and degraded imagery.

(Section outline placeholder) Processing was completed using consumer-grade hardware without acceleration, under sustained load for multiple hours.

(Section outline placeholder) The system supports parallelized ingestion and processing stages for high throughput across large archives.

(Placeholder)

(Placeholder)

(Placeholder)

(Section outline placeholder) Segment-level analysis supported detection of small facial elements and fragment-level features in degraded scans.

(Section outline placeholder) Sustained throughput enabled million-file-scale processing within practical time constraints.

• Continuous processing under sustained load
• Offline execution (no external dependencies)
• Stable operation without system instability

The intended use is document indexing and triage support within large public document archives, enabling faster navigation to visually relevant content without modifying source material.

The system completed large-scale processing of approximately 1,000,000 publicly released PDF documents using standard consumer-grade hardware without acceleration.

• ~98% estimated capture of detectable facial content within the dataset
• ~2,000 total detected faces
• 2 non-face detections (wood textures)
• High-recall detection behavior across photographs, reflections, drawings, and degraded imagery
• Successful segmentation-level detection of small facial elements
• Continuous non-stop processing for ~5 hours
• No modification of source files
• No system instability during execution

This demonstration establishes that:

• Embedded image content within large document archives can be programmatically extracted at scale.
• Face indexing can be performed locally without cloud infrastructure.
• High-recall detection can capture not only standard photographs but also edge cases such as reflections and drawn faces.
• Office-class hardware is sufficient for million-file-scale processing.
• Automated indexing meaningfully reduces manual navigation burden in image-heavy document collections.
• The system fulfills its operational design goals.

Large-scale public document archives containing embedded images can be processed locally using automated extraction, segmentation, and face indexing techniques.

• Feasibility at million-document scale
• Stability under sustained processing
• High-recall detection performance
• Extremely low observed false positive rate
• Compatibility with constrained hardware environments

DawaImg provides a deterministic, offline method for converting opaque PDF document collections into searchable visual datasets without altering source material.