Benchmark Report
Total Runs: 15
Total Files Processed: 1284
Total Time: 6407.5 seconds
Framework Performance Summary
| Framework |
Avg Success Rate |
Avg Time (s) |
Avg Memory (MB) |
Files/sec |
| kreuzberg_sync |
82.0% |
0.19 |
0.0 |
15.00 |
| kreuzberg_async |
82.0% |
0.20 |
0.0 |
7.63 |
| docling |
83.9% |
9.29 |
0.0 |
0.16 |
| markitdown |
79.6% |
10.21 |
0.0 |
9.88 |
| unstructured |
90.0% |
8.05 |
0.0 |
2.34 |
Performance by Category
Tiny
| Framework |
Success Rate |
Avg Time (s) |
Memory (MB) |
Throughput |
| kreuzberg_sync |
57.1% |
0.03 |
0.0 |
34.84 |
| kreuzberg_async |
57.1% |
0.08 |
0.0 |
12.32 |
| docling |
71.4% |
4.13 |
0.0 |
0.24 |
| markitdown |
69.4% |
0.04 |
0.0 |
26.89 |
| unstructured |
85.7% |
0.17 |
0.0 |
6.03 |
Small
| Framework |
Success Rate |
Avg Time (s) |
Memory (MB) |
Throughput |
| kreuzberg_sync |
88.9% |
0.13 |
0.0 |
7.69 |
| kreuzberg_async |
88.9% |
0.13 |
0.0 |
7.92 |
| docling |
96.3% |
14.44 |
0.0 |
0.07 |
| markitdown |
88.9% |
0.37 |
0.0 |
2.72 |
| unstructured |
92.6% |
1.05 |
0.0 |
0.95 |
Medium
| Framework |
Success Rate |
Avg Time (s) |
Memory (MB) |
Throughput |
| kreuzberg_sync |
100.0% |
0.40 |
0.0 |
2.47 |
| kreuzberg_async |
100.0% |
0.38 |
0.0 |
2.65 |
| markitdown |
80.6% |
30.23 |
0.0 |
0.03 |
| unstructured |
91.7% |
22.92 |
0.0 |
0.04 |
Failure Analysis
| Error Type |
Count |
| MissingDependencyError |
114 |
| FileConversionException |
45 |
| ConversionError |
39 |
| ValidationError |
30 |
| OSError |
15 |
| UnsupportedFormatException |
9 |
| TesseractError |
9 |
| UnicodeDecodeError |
6 |
| TypeError |
3 |