SPARQLoscope DBLP Benchmark Results

We ran the SPARQLoscope DBLP evaluation on Fluree — 105 SPARQL queries against the DBLP computer science bibliography. We compared our results against published data for seven other engines from the SPARQLoscope project.

We did not run the other engines ourselves. Their numbers come from the SPARQLoscope evaluation published by the University of Freiburg. We ran only Fluree, on the same dataset and queries, and placed our results alongside theirs.

Download the full results: fluree-benchmark-results.csv

Replicate it: github.com/fluree/sparqloscope-benchmark — clone, run one command, compare.

---

Speed Rankings

Geometric Mean (seconds, lower is better)

Across all 105 queries, with penalty factor P=2 applied to timed-out queries (180s timeout × 2 = 360s penalty):

Rank	Engine	Geo Mean	vs Fluree	Failed Queries
1	QLever	0.176s	0.62× faster	0%
2	Fluree	0.282s	—	0%
3	Virtuoso	0.485s	1.72× slower	4.8%
4	Mdb	1.260s	4.5× slower	0%
5	GraphDB	5.796s	20.6× slower	0%
6	Blazegraph	13.350s	47.3× slower	14.3%
7	Jena	15.288s	54.2× slower	18.1%
8	Oxigraph	38.570s	136.8× slower	43.8%

QLever is a read-only engine — it cannot write, update, or delete data. Among engines that support writes, Fluree has the lowest geometric mean.

Arithmetic Mean (seconds, lower is better)

The arithmetic mean is sensitive to slow outliers and reflects total workload time:

Rank	Engine	Arith Mean	vs Fluree
1	Fluree	0.951s	—
2	QLever	1.243s	1.31× slower
3	Mdb	9.770s	10.3× slower
4	Virtuoso	20.851s	21.9× slower
5	GraphDB	26.229s	27.6× slower
6	Blazegraph	81.136s	85.3× slower
7	Jena	94.207s	99.1× slower
8	Oxigraph	177.355s	186.5× slower

Median (seconds, lower is better)

Rank	Engine	Median
1	QLever	0.181s
2	Fluree	0.203s
3	Virtuoso	0.536s
4	Mdb	2.522s
5	GraphDB	11.387s
6	Blazegraph	16.803s
7	Jena	33.044s
8	Oxigraph	81.883s

Query Completion

Fluree and QLever are the only two engines to complete all 105 queries within the 180-second timeout.

---

Results by Category

Joins

Query	Fluree	QLever	Virtuoso	GraphDB	Blazegraph	Jena
3-way chain join	0.202s	0.500s	1.650s	81.520s	96.650s	180s+
3-way star join	0.848s	1.130s	1.340s	24.580s	81.230s	63.160s
xlarge chain join	0.046s	0.490s	0.090s	0.010s	0.020s	0.010s
xlarge star join	0.045s	0.040s	0.190s	0.020s	0.100s	0.010s
2-way large-large	0.598s	0.180s	0.540s	15.540s	53.490s	38.940s
Multicolumn large	0.895s	2.010s	0.770s	14.060s	69.120s	36.540s

Aggregations (GROUP BY)

Query	Fluree	QLever	GraphDB	Blazegraph	Jena	Oxigraph
Complex aggregate	0.292s	0.660s	15.510s	18.430s	44.040s	180.9s
COUNT high-mult	0.063s	0.010s	13.940s	0.060s	180s+	43.570s
COUNT low-mult	2.338s	0.820s	19.050s	96.660s	139.58s	87.790s
Numeric AVG	0.082s	0.050s	1.800s	3.650s	1.480s	2.200s
String GROUP_CONCAT	4.077s	18.020s	40.360s	684.11s	123.65s	182.09s

Graph Patterns (OPTIONAL, MINUS, EXISTS, UNION)

Query	Fluree	QLever	Virtuoso	Blazegraph	Jena	Oxigraph
OPTIONAL 3-chain-1	0.201s	0.890s	2.340s	28.450s	124.98s	180.92s
MINUS 3-chain-1	0.190s	0.880s	1.500s	996.26s	31.410s	180.89s
EXISTS 3-chain-1	0.198s	4.380s	4.680s	482.87s	85.780s	180.89s
UNION large-join	1.126s	1.320s	1.140s	92.290s	168.59s	180.91s

String Operations and Regex

Query	Fluree	QLever	Virtuoso	GraphDB	Jena	Oxigraph
strstarts	0.099s	3.810s	1.520s	11.910s	33.520s	71.730s
strlen	5.707s	3.860s	1.590s	11.180s	33.250s	77.410s
regex-3	9.780s	6.820s	4.860s	12.960s	32.680s	83.010s
regex-prefix-1	0.173s	0.010s	1.650s	10.610s	31.320s	73.950s

Regex is Fluree's weakest relative category. Complex regex patterns take 6–10s. QLever and Virtuoso are faster here.

Transitive Paths

Query	Fluree	QLever	Virtuoso	Mdb	GraphDB	Jena
Large path+join	1.150s	13.470s	0s	39.050s	12.440s	180s+
Plus fixed-subject	0.049s	0.010s	2.400s	0s	0.010s	0.010s
Plus	0.189s	0.070s	0.010s	0.100s	0.320s	0.650s

Date and Numeric Operations

Query	Fluree	QLever	Virtuoso	GraphDB	Blazegraph	Jena
date-day	0.053s	0.100s	0.100s	4.350s	14.370s	3.330s
numeric-abs	0.085s	0.090s	0.070s	2.510s	3.620s	1.680s
filter-bin-50/50	0.050s	0.030s	0.030s	1.410s	3.780s	19.490s

Metadata Queries

Query	Fluree	QLever	Virtuoso	GraphDB	Blazegraph	Jena	Oxigraph
number-of-triples	0.049s	0.010s	0.400s	73.670s	194.83s	180s+	154.08s
number-of-subjects	0.159s	0.010s	174.39s	206.64s	67.580s	180s+	181.28s
number-of-objects	0.137s	0.010s	180.22s	210.13s	70.710s	180s+	181.70s

Several engines time out entirely on metadata queries. Fluree completes all of them under 0.2s.

---

About this benchmark

What SPARQLoscope is

SPARQLoscope is a SPARQL evaluation framework developed at the University of Freiburg (the team behind QLever), presented at ISWC 2025. The DBLP evaluation runs 105 queries against the DBLP computer science bibliography — real-world data, not synthetic.

The queries cover joins, aggregations, filters, graph patterns (OPTIONAL, MINUS, EXISTS, UNION), string operations, regex, transitive paths, date/numeric operations, and metadata queries.

What we ran

We ran Fluree on the SPARQLoscope DBLP evaluation using Fluree's default configuration (no special tuning). The config file is committed to the benchmark repo.

What we didn't run

We did not run the other engines. Their results are from the SPARQLoscope project's published evaluation data. We used those numbers as-is. This avoids any question about whether we configured a competitor's engine fairly.

Metrics

• Geometric mean (P=2): Timed-out queries (180s) penalized at 360s. Geometric mean computed across all 105 queries.
• Arithmetic mean (P=2): Same penalty. Sensitive to slow outliers.
• Median (P=2): Middle value across all 105 queries.
• Failed queries: Percentage that did not return results within the 180s timeout.

---

Caveats

• This is a single-user, read-only query benchmark. It does not test write throughput, concurrent load, or mixed workloads.
• DBLP is a medium-scale dataset (millions of triples). Performance at larger scales may differ.
• Regex is Fluree's weakest relative category (6–10s on complex patterns vs. QLever's 6–7s).
• Several products are not in the SPARQLoscope evaluation (Neptune, Stardog, MarkLogic, AllegroGraph, RDFox). We cannot make speed claims about engines we haven't tested.
• No benchmark replaces testing on your own data and queries.

---

Beyond query speed

This benchmark tests SPARQL query execution only. Fluree also provides capabilities that the benchmark does not measure and that most engines in the comparison do not have — immutable append-only storage with time travel queries, git-like branching and merging, content-addressed commits with W3C Verifiable Credentials, built-in HNSW vector search, BM25 full-text search, triple-level access control, AES-256-GCM encryption at rest, Apache Iceberg integration, and GeoSPARQL with S2 spatial indexing.

These are documented separately. See the Fluree developer documentation for details.

---

Replicate this yourself

One command

git clone https://github.com/fluree/sparqloscope-benchmark.git
cd sparqloscope-benchmark
./run-benchmark.sh

This pulls the Fluree Docker image, downloads the DBLP dataset, loads it, runs all 105 queries, and outputs results to results/fluree-results.tsv.

Requirements

• Docker 20.10+
• 32 GB RAM (64 GB recommended)
• 50 GB free disk

Manual steps

git clone https://github.com/fluree/sparqloscope-benchmark.git
cd sparqloscope-benchmark

# Download DBLP dataset
cd data && ./download-dblp.sh && cd ..

# Start Fluree
docker compose up -d

# Load data
./scripts/load-data.sh

# Run queries
./scripts/run-queries.sh

# Compute metrics
python scripts/compute-metrics.py results/fluree-results.tsv

# Compare to our published results
diff results/fluree-results.tsv reference-results/fluree-2026-03-30.tsv

Download the raw data

• Full CSV with all 105 query times — Fluree column plus published results for all other engines.
• Benchmark repo — scripts, config, and reference results.

On equivalent hardware, expect ±10–20% variance on individual queries and ±5% on aggregate metrics.

Running other engines

The SPARQLoscope project provides scripts for other engines. We didn't bundle them — running competitors yourself ensures fair configuration. See the QLever repo and the SPARQLoscope project documentation.