Every model ran on a MacBook Pro (Apple M5) through Ollama, at temperature 0 so runs are reproducible. Each model was given exactly two things: the project's two published skill files (word for word, the same documents any user downloads) and a one-line instruction to reply with a single SQL statement.
Each model answered 48 research questions, one at a time, split evenly between the SWRD and SSWR databases. The questions fall into six categories of increasing difficulty:
Each model's SQL was executed exactly as written against the live databases, through the same public read-only endpoint available to everyone. The result was compared with a reference answer computed before the run; every reference query was verified to return a stable, deterministic answer. Each answer receives one of three marks:
How the skill files improve: the feedback loop. The skill files are not written once and frozen. They are maintained by testing, in a simple cycle:
A pilot round produced four rules this way: stay inside one database's tables, always call search functions with SELECT, treat the relevance score as a score rather than a row position, and join the papers table when filtering authorship by year. The scores on this page come from the second pass, run against those improved files. That pass caught one more pattern (models asking a search function's output for a journal_id column it does not have; the functions return journal_name directly), and the published skill files now state each search function's exact return columns. The skill files you download today include every rule.
| Model (Ollama tag) | Architecture · disk size | Strict | Correct or defensible | Avg / question |
|---|---|---|---|---|
| gemma4:26b Google Gemma 4 26B | 26B dense · 16 GB | 47 / 48 | 47 / 48 | 6.6 s |
| qwen3.6:27b Qwen3.6-27B | 27B dense · 17 GB | 47 / 48 | 47 / 48 | 60.2 s |
| nemotron-cascade-2:latest NVIDIA Nemotron-Cascade-2 | 30B MoE (3B active) · 24 GB | 44 / 48 | 44 / 48 | 10.1 s |
| gemma4:31b-mlx Google Gemma 4 31B | 31B dense (MLX) · 20 GB | 46 / 48 | 47 / 48 | 27.8 s |
| ornith:35b Ornith 35B | 35B dense · 21 GB | 48 / 48 | 48 / 48 | 13.8 s |
| qwen3.6:latest Qwen3.6-35B | 36B MoE · 23 GB | 47 / 48 | 47 / 48 | 51.9 s |
Models not already on the machine were pulled, evaluated, and deleted afterward. Timings are Ollama generation only.
| Category (strict passes) | gemma4:26b | qwen3.6:27b | nemotron-c2 | gemma4:31b | ornith:35b | qwen3.6 |
|---|---|---|---|---|---|---|
| A · Simple retrieval | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 |
| B · Vocabulary fidelity | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 |
| C · Joins & relations | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 |
| D · Aggregation & trends | 8/8 | 8/8 | 8/8 | 7/8 | 8/8 | 8/8 |
| E · Search functions | 7/8 | 7/8 | 4/8 | 7/8 | 8/8 | 7/8 |
| F · Complex analytical | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 | 8/8 |
Accuracy by category: Simple retrieval 48/48 · Vocabulary fidelity 48/48 · Joins & relations 48/48 · Aggregation & trends 47/48 · Search functions 40/48 · Complex analytical 48/48. The gradient is the story: simple retrieval and vocabulary questions are near-ceiling for every size, while joins, nested search-function calls, and multi-step analytical queries separate the models. This is where the models genuinely differ.
Expand a category to see its questions with per-model marks (✓ strict · △ defensible · ✗ error, in model order from smallest to largest). Expand a question to see the reference answer and each model's verbatim SQL, timing, and outcome.
Reference answer: 110618
select count(*) from swrd.papers
select count(*) from swrd.papers
SELECT COUNT(*) FROM swrd.papers
select count(*) from swrd.papers
SELECT count(*) FROM swrd.papers
select count(*) from swrd.papers
Reference answer: 23288
SELECT count(*) FROM swrd.papers WHERE publication_year < 1989
SELECT COUNT(*) FROM swrd.papers WHERE publication_year < 1989
SELECT COUNT(*) AS n FROM swrd.papers WHERE publication_year < 1989
select count(*) from swrd.papers where publication_year < 1989
select count(*) from swrd.papers where publication_year < 1989
select count(*) from swrd.papers where publication_year < 1989
Reference answer: 91
select count(*) from swrd.journals
SELECT COUNT(*) FROM swrd.journals
select count(*) as n from swrd.journals
select count(*) from swrd.journals
select count(*) from swrd.journals
select count(*) from swrd.journals
Reference answer: 1624
select count(*) from swrd.papers p join swrd.journals j on p.journal_id = j.id where j.name = 'Child & Family Social Work'
SELECT count(*) FROM swrd.papers p JOIN swrd.journals j ON j.id = p.journal_id WHERE j.name = 'Child & Family Social Work'
SELECT COUNT(*) AS n FROM swrd.papers p JOIN swrd.journals j ON j.id = p.journal_id WHERE j.name = 'Child & Family Social Work'
select count(*) from swrd.papers p join swrd.journals j on p.journal_id = j.id where j.name = 'Child & Family Social Work'
select count(*) as n from swrd.papers p join swrd.journals j on j.id = p.journal_id where j.name = 'Child & Family Social Work'
select count(*) from swrd.papers p join swrd.journals j on j.id = p.journal_id where j.name = 'Child & Family Social Work'
Reference answer: 1516
select count(*) from sswr.papers where year = 2024
select count(*) from sswr.papers where year = 2024
SELECT count(*) FROM sswr.papers WHERE year = 2024
select count(*) from sswr.papers where year = 2024
select count(*) from sswr.papers where year = 2024
select count(*) from sswr.papers where year = 2024
Reference answer: 23793
select count(*) from sswr.papers
select count(*) from sswr.papers
select count(*) as n from sswr.papers
select count(*) from sswr.papers
select count(*) from sswr.papers
select count(*) from sswr.papers
Reference answer: 21209
SELECT count(*) FROM sswr.authors
select count(*) from sswr.authors
select count(*) from sswr.authors
select count(*) from sswr.authors
select count(*) from sswr.authors
select count(*) from sswr.authors
Reference answer: 423
select count(*) from sswr.papers where year = 2005
select count(*) from sswr.papers where year = 2005
select count(*) as n from sswr.papers where year = 2005
select count(*) from sswr.papers where year = 2005
SELECT COUNT(*) FROM sswr.papers WHERE year = 2005
select count(*) from sswr.papers where year = 2005
Reference answer: Mixed-Methods; Qualitative; Quantitative; Review
SELECT DISTINCT research_method FROM swrd.papers
SELECT DISTINCT research_method FROM swrd.papers
SELECT DISTINCT research_method FROM swrd.papers WHERE research_method IS NOT NULL
select distinct research_method from swrd.papers
select distinct research_method from swrd.papers order by 1
select distinct research_method from swrd.papers order by 1
Reference answer: 1970
SELECT count(*) FROM swrd.papers WHERE research_method = 'Mixed-Methods'
SELECT count(*) FROM swrd.papers WHERE research_method = 'Mixed-Methods'
select count(*) as n from swrd.papers where research_method = 'Mixed-Methods'
select count(*) from swrd.papers where research_method = 'Mixed-Methods'
select count(*) from swrd.papers where research_method = 'Mixed-Methods'
select count(*) from swrd.papers where research_method = 'Mixed-Methods'
Reference answer: 706
select count(*) from swrd.papers where research_method = 'Review'
select count(*) from swrd.papers where research_method = 'Review'
SELECT COUNT(*) AS n FROM swrd.papers WHERE research_method = 'Review'
select count(*) from swrd.papers where research_method = 'Review'
select count(*) as n from swrd.papers where research_method = 'Review'
select count(*) from swrd.papers where research_method = 'Review'
Reference answer: 27918
SELECT count(*) FROM swrd.papers WHERE is_scientific AND NOT is_empirical
select count(*) from swrd.papers where is_scientific and not is_empirical
SELECT COUNT(*) FROM swrd.papers WHERE is_scientific AND NOT is_empirical
select count(*) from swrd.papers where is_scientific = true and is_empirical = false
select count(*) as n from swrd.papers where is_scientific = true and is_empirical = false
SELECT count(*) FROM swrd.papers WHERE is_scientific = true AND is_empirical = false
Reference answer: other; review; mixed_methods; qualitative; quantitative
SELECT DISTINCT methodology FROM sswr.papers
SELECT DISTINCT methodology FROM sswr.papers
SELECT DISTINCT methodology FROM sswr.papers
select distinct methodology from sswr.papers
select distinct methodology from sswr.papers order by 1
SELECT DISTINCT methodology FROM sswr.papers ORDER BY 1
Reference answer: 240
select count(*) from sswr.papers where methodology = 'other'
SELECT count(*) FROM sswr.papers WHERE methodology = 'other'
SELECT COUNT(*) FROM sswr.papers WHERE methodology = 'other'
select count(*) from sswr.papers where methodology = 'other'
select count(*) from sswr.papers where methodology = 'other'
SELECT count(*) FROM sswr.papers WHERE methodology = 'other'
Reference answer: 2172
SELECT count(*) FROM sswr.papers WHERE methodology = 'mixed_methods'
SELECT count(*) FROM sswr.papers WHERE methodology = 'mixed_methods'
select count(*) as n from sswr.papers where methodology='mixed_methods'
select count(*) from sswr.papers where methodology = 'mixed_methods'
select count(*) from sswr.papers where methodology = 'mixed_methods'
select count(*) from sswr.papers where methodology = 'mixed_methods'
Reference answer: 7
SELECT count(DISTINCT format) FROM sswr.papers
SELECT count(DISTINCT format) FROM sswr.papers
select count(distinct format) as distinct_formats from sswr.papers
select count(distinct format) from sswr.papers
SELECT COUNT(DISTINCT format) FROM sswr.papers
select count(distinct format) from sswr.papers
Reference answer: 5931 · British Journal of Social Work
SELECT j.name, COUNT(*) as article_count FROM swrd.papers p JOIN swrd.journals j ON p.journal_id = j.id WHERE p.publication_year >= 1989 GROUP BY j.name ORDER BY article_count DESC LIMIT 1
select j.name, count(*) as n from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 group by j.id, j.name order by n desc limit 1
SELECT j.name AS journal_name, COUNT(*) AS article_count FROM swrd.papers p JOIN swrd.journals j ON p.journal_id = j.id WHERE p.publication_year >= 1989 GROUP BY j.name ORDER BY article_count DESC LIMIT 1
select j.name, count(*) as n from swrd.papers p join swrd.journals j on p.journal_id = j.id where p.publication_year >= 1989 group by j.name order by n desc limit 1
select j.name, count(*) as n from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 group by 1 order by 2 desc limit 1
select j.name, count(*) as n from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 group by j.name order by n desc limit 1
Reference answer: 2911
SELECT count(*) FROM swrd.papers p JOIN swrd.journals j ON p.journal_id = j.id WHERE p.publication_year >= 1989 AND j.name = 'Research on Social Work Practice'
select count(*) as n from swrd.papers p join swrd.journals j on j.id = p.journal_id where j.name = 'Research on Social Work Practice' and p.publication_year >= 1989
select count(*) as n from swrd.papers p join swrd.journals j on p.journal_id = j.id where p.publication_year >= 1989 and j.name = 'Research on Social Work Practice'
select count(*) from swrd.papers p join swrd.journals j on p.journal_id = j.id where j.name = 'Research on Social Work Practice' and p.publication_year >= 1989
select count(*) from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 and j.name = 'Research on Social Work Practice'
select count(*) from swrd.papers join swrd.journals on journals.id = papers.journal_id where journals.name = 'Research on Social Work Practice' and papers.publication_year >= 1989
Reference answer: 2463
SELECT COUNT(DISTINCT aa.organization_id) FROM swrd.author_affiliations aa JOIN swrd.papers p ON aa.paper_id = p.id WHERE p.publication_year = 2020
select count(distinct aa.organization_id) from swrd.papers p join swrd.author_affiliations aa on aa.paper_id = p.id where p.publication_year = 2020
SELECT COUNT(DISTINCT o.id) AS org_count FROM swrd.papers p JOIN swrd.author_affiliations a ON a.paper_id = p.id JOIN swrd.organizations o ON o.id = a.organization_id WHERE p.publication_year = 2020
select count(distinct aa.organization_id) from swrd.papers p join swrd.author_affiliations aa on p.id = aa.paper_id where p.publication_year = 2020
select count(distinct aa.organization_id) as n_organizations from swrd.papers p join swrd.author_affiliations aa on aa.paper_id = p.id where p.publication_year = 2020
select count(distinct swrd.author_affiliations.organization_id) from swrd.papers join swrd.author_affiliations on swrd.author_affiliations.paper_id = swrd.papers.id where swrd.papers.publication_year = 2020
Reference answer: 9953
SELECT count(*) FROM swrd.paper_authors pa JOIN swrd.papers p ON pa.paper_id = p.id WHERE p.publication_year = 2019
select count(*) from swrd.paper_authors pa join swrd.papers p on p.id = pa.paper_id where p.publication_year = 2019
SELECT COUNT(*) FROM swrd.paper_authors pa JOIN swrd.papers p ON pa.paper_id = p.id WHERE p.publication_year = 2019
select count(*) from swrd.paper_authors pa join swrd.papers p on pa.paper_id = p.id where p.publication_year = 2019
select count(*) from swrd.paper_authors pa join swrd.papers p on p.id = pa.paper_id where p.publication_year = 2019
SELECT count(*) FROM swrd.paper_authors pa JOIN swrd.papers p ON p.id = pa.paper_id WHERE p.publication_year = 2019
Reference answer: 290 · Washington University in St. Louis
SELECT pa.institution_normalized, COUNT(*) as n FROM sswr.papers p JOIN sswr.paper_authors pa ON p.id = pa.paper_id WHERE p.year BETWEEN 2020 AND 2026 AND pa.author_order = 1 GROUP BY 1 ORDER BY 2 DESC LIMIT 1
SELECT pa.institution_normalized, COUNT(*) AS presentation_count FROM sswr.paper_authors pa JOIN sswr.papers p ON p.id = pa.paper_id WHERE p.year BETWEEN 2020 AND 2026 AND pa.author_order = 1 GROUP BY pa.institution_normalized ORDER BY presentation_count DESC LIMIT 1
SELECT pa.institution_normalized AS institution, COUNT(*) AS cnt FROM sswr.paper_authors pa JOIN sswr.papers p ON p.id = pa.paper_id WHERE pa.author_order = 1 AND p.year BETWEEN 2020 AND 2026 GROUP BY pa.institution_normalized ORDER BY cnt DESC LIMIT 1
SELECT pa.institution_normalized, count(*) as n FROM sswr.paper_authors pa JOIN sswr.papers p ON p.id = pa.paper_id WHERE pa.author_order = 1 AND p.year BETWEEN 2020 AND 2026 GROUP BY 1 ORDER BY 2 DESC LIMIT 1
select institution_normalized, count(*) as n from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where pa.author_order = 1 and p.year between 2020 and 2026 group by 1 order by 2 desc limit 1
select institution_normalized, count(*) as n from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where pa.author_order = 1 and p.year between 2020 and 2026 group by 1 order by 2 desc limit 1
Reference answer: 89
SELECT count(*) FROM sswr.paper_authors WHERE author_id = 108089
select count(*) from sswr.paper_authors where author_id = 108089
SELECT COUNT(DISTINCT paper_id) FROM sswr.paper_authors WHERE author_id = 108089
select count(*) from sswr.paper_authors where author_id = 108089
select total_papers from sswr.authors where id = 108089
select count(*) from sswr.paper_authors where author_id = 108089
Reference answer: 2089
SELECT count(DISTINCT pa.author_id) FROM sswr.papers p JOIN sswr.paper_authors pa ON p.id = pa.paper_id WHERE p.year = 2015
select count(distinct pa.author_id) from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2015
SELECT COUNT(DISTINCT pa.author_id) AS distinct_canonical_authors FROM sswr.papers p JOIN sswr.paper_authors pa ON pa.paper_id = p.id WHERE p.year = 2015
select count(distinct pa.author_id) from sswr.papers p join sswr.paper_authors pa on p.id = pa.paper_id where p.year = 2015
select count(distinct pa.author_id) from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2015
select count(distinct pa.author_id) from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2015
Reference answer: 665
SELECT count(*) FROM sswr.paper_authors WHERE author_order = 1 AND institution_normalized = 'University of Michigan'
select count(*) from sswr.paper_authors where author_order = 1 and institution_normalized = 'University of Michigan'
SELECT COUNT(*) FROM sswr.paper_authors WHERE author_order = 1 AND institution_normalized = 'University of Michigan'
select count(*) from sswr.paper_authors where author_order = 1 and institution_normalized = 'University of Michigan'
select count(*) from sswr.paper_authors pa where pa.author_order = 1 and pa.institution_normalized = 'University of Michigan'
select count(*) from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where pa.author_order = 1 and pa.institution_normalized = 'University of Michigan'
Reference answer: 2023
SELECT publication_year FROM swrd.papers WHERE publication_year >= 1989 AND publication_year <= 2023 AND is_scientific = true AND abstract IS NOT NULL GROUP BY publication_year ORDER BY count(*) DESC LIMIT 1
SELECT publication_year, count(*) as n FROM swrd.papers WHERE publication_year >= 1989 AND publication_year <= 2023 AND is_scientific = true AND abstract IS NOT NULL GROUP BY publication_year ORDER BY n DESC LIMIT 1
SELECT publication_year, COUNT(*) AS n FROM swrd.papers WHERE is_scientific = true AND abstract IS NOT NULL AND publication_year BETWEEN 1989 AND 2023 GROUP BY publication_year ORDER BY n DESC LIMIT 1
select publication_year from swrd.papers where publication_year >= 1989 and publication_year <= 2023 and is_scientific = true and abstract is not null group by publication_year order by count(*) desc limit 1
select publication_year, count(*) as n from swrd.papers where publication_year >= 1989 and publication_year <= 2023 and is_scientific = true and abstract is not null group by 1 order by 2 desc limit 1
select publication_year, count(*) as n from swrd.papers where publication_year >= 1989 and publication_year <= 2023 and is_scientific and abstract is not null group by 1 order by 2 desc limit 1
Reference answer: 1729
SELECT count(*) FROM swrd.papers WHERE publication_year = 2020 AND is_scientific = true AND is_empirical = true AND abstract IS NOT NULL
select count(*) as n from swrd.papers where publication_year = 2020 and is_scientific = true and abstract is not null and is_empirical = true
SELECT COUNT(*) FROM swrd.papers WHERE publication_year = 2020 AND is_scientific = TRUE AND is_empirical = TRUE AND abstract IS NOT NULL
select count(*) from swrd.papers where is_scientific = true and abstract is not null and publication_year = 2020 and is_empirical = true
select count(*) as n from swrd.papers where publication_year = 2020 and is_scientific and is_empirical and abstract is not null
SELECT count(*) FROM swrd.papers WHERE publication_year = 2020 AND is_empirical = true AND is_scientific = true AND abstract IS NOT NULL
Reference answer: 21719
SELECT count(*) FROM swrd.papers WHERE is_scientific = true AND abstract IS NOT NULL AND publication_year >= 2010 AND publication_year <= 2019
select count(*) from swrd.papers where publication_year between 2010 and 2019 and is_scientific = true and abstract is not null
SELECT COUNT(*) FROM swrd.papers WHERE is_scientific AND abstract IS NOT NULL AND publication_year >= 2010 AND publication_year <= 2019
select count(*) from swrd.papers where is_scientific = true and abstract is not null and publication_year >= 2010 and publication_year <= 2019
select count(*) from swrd.papers where publication_year between 2010 and 2019 and is_scientific and abstract is not null
select count(*) from swrd.papers where publication_year between 2010 and 2019 and is_scientific = true and abstract is not null
Reference answer: 2.5
SELECT ROUND(AVG(author_count)::numeric, 2) FROM (SELECT COUNT(pa.author_id) AS author_count FROM swrd.papers p JOIN swrd.paper_authors pa ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY p.id) AS subquery
SELECT ROUND(AVG(cnt), 2) FROM (SELECT p.id, COUNT(pa.author_id) AS cnt FROM swrd.papers p JOIN swrd.paper_authors pa ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY p.id) t
SELECT ROUND(AVG(author_cnt),2) AS mean_authors_per_paper FROM (SELECT p.id, COUNT(DISTINCT pa.author_id) AS author_cnt FROM swrd.papers p JOIN swrd.paper_authors pa ON pa.paper_id = p.id WHERE p.publication_year >= 1989 GROUP BY p.id) sub
SELECT ROUND(COUNT(pa.author_id)::numeric / COUNT(DISTINCT p.id), 2) FROM swrd.papers p LEFT JOIN swrd.paper_authors pa ON p.id = pa.paper_id WHERE p.publication_year >= 1989
SELECT ROUND(AVG(cnt), 2) AS mean_authors_per_paper FROM (SELECT pa.paper_id, COUNT(*) AS cnt FROM swrd.paper_authors pa JOIN swrd.papers p ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY pa.paper_id) sub
select round(avg(cnt), 2) from (select paper_id, count(*) as cnt from swrd.paper_authors join swrd.papers on swrd.papers.id = swrd.paper_authors.paper_id where swrd.papers.publication_year >= 1989 group by paper_id) t
Reference answer: 2015
SELECT year FROM sswr.papers GROUP BY year HAVING COUNT(*) > 1000 ORDER BY year ASC LIMIT 1
select year, count(*) as n from sswr.papers group by 1 having count(*) > 1000 order by 1 asc limit 1
SELECT year FROM sswr.papers GROUP BY year HAVING COUNT(*) > 1000 ORDER BY year ASC LIMIT 1
SELECT year FROM sswr.papers GROUP BY year HAVING count(*) > 1000 ORDER BY year ASC LIMIT 1
select year, count(*) as n from sswr.papers group by 1 having count(*) > 1000 order by 1 asc limit 1
select year, count(*) as n from sswr.papers group by 1 having count(*) > 1000 order by 1 asc limit 1
Reference answer: 2026
SELECT year FROM sswr.papers GROUP BY year ORDER BY COUNT(*) DESC LIMIT 1
SELECT year, count(*) as n FROM sswr.papers GROUP BY year ORDER BY n DESC LIMIT 1
SELECT year, COUNT(*) AS n FROM sswr.papers GROUP BY year ORDER BY n DESC LIMIT 1
select year, count(*) as n from sswr.papers group by 1 order by 2 desc limit 1
select year, count(*) as n from sswr.papers group by 1 order by 2 desc limit 1
select year, count(*) as n from sswr.papers group by 1 order by 2 desc limit 1
Reference answer: 82
SELECT count(*) FROM sswr.papers WHERE year = 2010 AND methodology = 'qualitative'
select count(*) as n from sswr.papers where year = 2010 and methodology = 'qualitative'
select count(*) as n from sswr.papers where year = 2010 and methodology = 'qualitative'
select count(*) from sswr.papers where year = 2010 and methodology = 'qualitative'
select count(*) from sswr.papers where year = 2010 and methodology = 'qualitative'
select count(*) from sswr.papers where year = 2010 and methodology = 'qualitative'
Reference answer: 2025
SELECT year FROM sswr.papers WHERE methodology = 'mixed_methods' GROUP BY year ORDER BY COUNT(*) DESC LIMIT 1
select year, count(*) as n from sswr.papers where methodology = 'mixed_methods' group by year order by n desc limit 1
select year, count(*) as n from sswr.papers where methodology = 'mixed_methods' group by year order by n desc limit 1
select year, count(*) as n from sswr.papers where methodology = 'mixed_methods' group by 1 order by 2 desc limit 1
select year, count(*) as n from sswr.papers where methodology = 'mixed_methods' group by 1 order by 2 desc limit 1
select year, count(*) as n from sswr.papers where methodology = 'mixed_methods' group by 1 order by 2 desc limit 1
Reference answer: 68376
SELECT id FROM swrd.search_papers_keyword('food insecurity', 1)select id from swrd.search_papers_keyword('food insecurity', 1)select paper_id from swrd.search_papers_keyword('food insecurity', 1)select id from swrd.search_papers_keyword('food insecurity', 1)select id from swrd.search_papers_keyword('food insecurity', 1)SELECT id FROM swrd.search_papers_keyword('food insecurity', 1)Reference answer: 11
SELECT count(*) FROM swrd.search_papers_keyword('homelessness', 20) WHERE publication_year >= 2015select count(*) as n from swrd.search_papers_keyword('homelessness', 20) where publication_year >= 2015SELECT COUNT(*) FROM (
SELECT p.publication_year
FROM swrd.search_papers_keyword('homelessness', 20) AS s(paper_id int, rank int, similarity float)
JOIN swrd.papers p ON p.id = s.paper_id
WHERE p.publication_year >= 2015
) tSELECT count(*) FROM swrd.search_papers_keyword('homelessness', 20) WHERE publication_year >= 2015select count(*) from swrd.search_papers_keyword('homelessness', 20) where publication_year >= 2015select count(*) from swrd.search_papers_keyword('homelessness', 20) where publication_year >= 2015Reference answer: Social Work
SELECT j.name FROM swrd.search_papers_keyword('child welfare workforce turnover', 1) AS p JOIN swrd.journals j ON j.id = p.journal_idselect j.name from swrd.search_papers_keyword('child welfare workforce turnover', 1) sp join swrd.papers p on p.id = sp.id join swrd.journals j on j.id = p.journal_idSELECT pj.journal_name
FROM swrd.search_papers_keyword('child welfare workforce turnover', 1) AS k
JOIN swrd.papers_with_journals pj ON pj.id = k.idSELECT j.name FROM swrd.search_papers_keyword('child welfare workforce turnover', 1) p JOIN swrd.journals j ON p.journal_id = j.idselect j.name as journal_name from swrd.search_papers_keyword('child welfare workforce turnover', 1) s join swrd.papers p on p.id = s.id join swrd.journals j on j.id = p.journal_id(no SQL produced)
Reference answer: 0
SELECT count(*) FROM swrd.search_papers_keyword('social work education', 10) AS s JOIN swrd.papers p ON s.id = p.id JOIN swrd.journals j ON p.journal_id = j.id WHERE j.name = 'Journal of Social Work Education'select count(*) from swrd.search_papers_keyword('social work education', 10) spk join swrd.journals j on j.id = spk.journal_id where j.name = 'Journal of Social Work Education'SELECT id
FROM swrd.search_papers_keyword('social work education', 10)
)
SELECT COUNT(*) FROM top10 t JOIN swrd.journals j ON j.id = t.id WHERE j.name = 'Journal of Social Work Education'select count(*) from swrd.search_papers_keyword('social work education', 10) s join swrd.papers p on s.id = p.id join swrd.journals j on p.journal_id = j.id where j.name = 'Journal of Social Work Education'select count(*) from swrd.search_papers_keyword('social work education', 10) s join swrd.papers p on p.id = s.id join swrd.journals j on j.id = p.journal_id where j.name = 'Journal of Social Work Education'SELECT count(*) FROM swrd.search_papers_keyword('social work education', 10) AS res JOIN swrd.papers p ON p.id = res.id JOIN swrd.journals j ON j.id = p.journal_id WHERE j.name = 'Journal of Social Work Education'Reference answer: 2021-P-0475
SELECT id FROM sswr.search_papers_keyword('opioid overdose', 1)select id from sswr.search_papers_keyword('opioid overdose', 1)select id from sswr.search_papers_keyword('opioid overdose', 1) limit 1select id from sswr.search_papers_keyword('opioid overdose', 1)select id from sswr.search_papers_keyword('opioid overdose', 1)select id from sswr.search_papers_keyword('opioid overdose', 1)Reference answer: 9
SELECT count(*) FROM sswr.search_papers_keyword('food insecurity', 20) WHERE year >= 2020select count(*) from sswr.search_papers_keyword('food insecurity', 20) where year >= 2020SELECT COUNT(*) FROM (SELECT year FROM sswr.search_papers_keyword('food insecurity', 20)) AS sub WHERE year >= 2020select count(*) from (select year from sswr.search_papers_keyword('food insecurity', 20)) as results where year >= 2020select count(*) from (select year from sswr.search_papers_keyword('food insecurity', 20)) as t where year >= 2020SELECT count(*) FROM sswr.search_papers_keyword('food insecurity', 20) AS t WHERE t.year >= 2020Reference answer: 2026
SELECT year FROM sswr.search_papers_keyword('artificial intelligence', 1) LIMIT 1select year from sswr.search_papers_keyword('artificial intelligence', 1)SELECT year FROM sswr.search_papers_keyword('artificial intelligence', 1) LIMIT 1select year from sswr.search_papers_keyword('artificial intelligence', 1) limit 1select year, title from sswr.search_papers_keyword('artificial intelligence', 1) limit 1select year from sswr.search_papers_keyword('artificial intelligence', 1)Reference answer: 10
SELECT count(*) FROM sswr.search_papers_keyword('telehealth', 15) AS results JOIN sswr.papers p ON p.id = results.id WHERE p.methodology = 'quantitative'select count(*) from sswr.search_papers_keyword('telehealth', 15) where methodology = 'quantitative'SELECT COUNT(*) FROM (SELECT methodology FROM sswr.search_articles_keyword('telehealth', 15)) sub WHERE methodology = 'quantitative'select count(*) from (select methodology from sswr.search_papers_keyword('telehealth', 15)) as results where methodology = 'quantitative'select count(*) from sswr.search_papers_keyword('telehealth', 15) where methodology = 'quantitative'select count(*) from sswr.search_papers_keyword('telehealth', 15) as res join sswr.papers p on p.id = res.id where p.methodology = 'quantitative'Reference answer: Qualitative Social Work
SELECT j.name FROM swrd.papers p JOIN swrd.journals j ON p.journal_id = j.id WHERE p.publication_year >= 1989 GROUP BY j.name HAVING COUNT(p.id) >= 1000 ORDER BY SUM(CASE WHEN p.is_empirical = true AND p.research_method = 'Qualitative' THEN 1 ELSE 0 END)::float / NULLIF(SUM(CASE WHEN p.is_empirical = true THEN 1 ELSE 0 END), 0) DESC LIMIT 1
select j.name from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 group by j.id, j.name having count(*) >= 1000 order by sum(case when p.is_empirical and p.research_method = 'Qualitative' then 1 else 0 end)::float / nullif(sum(case when p.is_empirical then 1 else 0 end), 0) desc limit 1
SELECT jc.name AS journal_name
FROM (
SELECT p.journal_id,
COUNT(*) AS emp_total,
SUM(CASE WHEN p.research_method = 'Qualitative' THEN 1 ELSE 0 END) AS qual_count
FROM swrd.papers p
WHERE p.publication_year >= 1989 AND p.is_empirical
GROUP BY p.journal_id
) e
JOIN (
SELECT j.id, j.name, COUNT(*) AS total_papers_1989plus
FROM swrd.papers p
JOIN swrd.journals j ON j.id = p.journal_id
WHERE p.publication_year >= 1989
GROUP BY j.id, j.name
) jc ON e.journal_id = jc.id
WHERE jc.total_papers_1989plus >= 1000
ORDER BY (e.qual_count::numeric / NULLIF(e.emp_total,0)) DESC
LIMIT 1SELECT j.name FROM swrd.journals j JOIN (SELECT journal_id, COUNT(*) as total_count, SUM(CASE WHEN is_empirical THEN 1 ELSE 0 END) as empirical_count, SUM(CASE WHEN is_empirical AND research_method = 'Qualitative' THEN 1 ELSE 0 END) as qual_count FROM swrd.papers WHERE publication_year >= 1989 GROUP BY journal_id) p ON j.id = p.journal_id WHERE p.total_count >= 1000 AND p.empirical_count > 0 ORDER BY (p.qual_count::float / p.empirical_count) DESC LIMIT 1
select j.name from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 group by j.name having count(*) >= 1000 order by (count(*) filter (where is_empirical and research_method = 'Qualitative')::numeric / nullif(count(*) filter (where is_empirical), 0)) desc limit 1
select j.name, count(case when p.is_empirical and p.research_method = 'Qualitative' then 1 end) * 1.0 / count(case when p.is_empirical then 1 end) as prop from swrd.papers p join swrd.journals j on j.id = p.journal_id where p.publication_year >= 1989 group by j.name having count(*) >= 1000 order by prop desc limit 1
Reference answer: 6439
SELECT COUNT(*) FROM (SELECT p.id FROM swrd.papers p JOIN swrd.paper_authors pa ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY p.id HAVING COUNT(pa.author_id) > 5) AS subquery
SELECT COUNT(*) FROM (SELECT pa.paper_id FROM swrd.paper_authors pa JOIN swrd.papers p ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY pa.paper_id HAVING COUNT(pa.author_id) > 5) AS t
SELECT COUNT(*) FROM ( SELECT p.id FROM swrd.papers p JOIN swrd.paper_authors pa ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY p.id HAVING COUNT(pa.author_id) > 5 ) AS sub
select count(*) from (select p.id from swrd.papers p join swrd.paper_authors pa on p.id = pa.paper_id where p.publication_year >= 1989 group by p.id having count(pa.author_id) > 5) as subquery
SELECT COUNT(*) FROM (SELECT p.id FROM swrd.papers p JOIN swrd.paper_authors pa ON p.id = pa.paper_id WHERE p.publication_year >= 1989 GROUP BY p.id HAVING COUNT(pa.author_id) > 5) subquery
select count(*) from (select p.id from swrd.papers p join swrd.paper_authors pa on pa.paper_id = p.id where p.publication_year >= 1989 group by p.id having count(pa.author_id) > 5) t
Reference answer: 2022
SELECT publication_year FROM swrd.papers WHERE publication_year >= 1989 AND publication_year <= 2023 AND research_method = 'Review' GROUP BY publication_year ORDER BY COUNT(*) DESC LIMIT 1
select publication_year, count(*) as n from swrd.papers where publication_year >= 1989 and publication_year <= 2023 and research_method = 'Review' group by publication_year order by n desc limit 1
SELECT publication_year, COUNT(*) AS n FROM swrd.papers WHERE publication_year >= 1989 AND publication_year <= 2023 AND research_method = 'Review' GROUP BY publication_year ORDER BY n DESC LIMIT 1
select publication_year from swrd.papers where publication_year >= 1989 and publication_year <= 2023 and research_method = 'Review' group by 1 order by count(*) desc limit 1
select publication_year, count(*) as n from swrd.papers where publication_year >= 1989 and publication_year <= 2023 and research_method = 'Review' group by 1 order by 2 desc limit 1
select publication_year, count(*) as n from swrd.papers where research_method = 'Review' and publication_year between 1989 and 2023 group by 1 order by 2 desc limit 1
Reference answer: 27
SELECT count(*) FROM (SELECT journal_id FROM swrd.papers WHERE publication_year IN (1989, 2025) GROUP BY journal_id HAVING COUNT(DISTINCT publication_year) = 2) AS common_journals
select count(*) from (select journal_id from swrd.papers where publication_year in (1989, 2025) group by journal_id having count(distinct publication_year) = 2) t
SELECT COUNT(DISTINCT j.id) FROM swrd.papers p1 JOIN swrd.journals j ON j.id = p1.journal_id WHERE p1.publication_year = 1989 AND EXISTS (SELECT 1 FROM swrd.papers p2 WHERE p2.journal_id = j.id AND p2.publication_year = 2025)
select count(*) from (select journal_id from swrd.papers where publication_year = 1989 intersect select journal_id from swrd.papers where publication_year = 2025) as common_journals
SELECT COUNT(*) FROM (SELECT journal_id FROM swrd.papers WHERE publication_year IN (1989, 2025) GROUP BY journal_id HAVING COUNT(DISTINCT publication_year) = 2) sub
SELECT count(*) FROM swrd.journals j WHERE EXISTS (SELECT 1 FROM swrd.papers p WHERE p.journal_id = j.id AND p.publication_year = 1989) AND EXISTS (SELECT 1 FROM swrd.papers p WHERE p.journal_id = j.id AND p.publication_year = 2025)
Reference answer: Michal Grinstein-Weiss
SELECT a.name FROM sswr.authors a JOIN sswr.paper_authors pa ON a.id = pa.author_id JOIN sswr.papers p ON p.id = pa.paper_id WHERE p.year BETWEEN 2010 AND 2015 AND pa.author_order = 1 GROUP BY a.name ORDER BY COUNT(*) DESC LIMIT 1
select a.name from sswr.papers p join sswr.paper_authors pa on p.id = pa.paper_id join sswr.authors a on pa.author_id = a.id where p.year between 2010 and 2015 and pa.author_order = 1 group by a.id, a.name order by count(*) desc limit 1
SELECT a.name FROM sswr.paper_authors pa JOIN sswr.authors a ON a.id = pa.author_id JOIN sswr.papers p ON p.id = pa.paper_id WHERE pa.author_order = 1 AND p.year BETWEEN 2010 AND 2015 GROUP BY a.name, a.id ORDER BY COUNT(*) DESC LIMIT 1
SELECT a.name FROM sswr.paper_authors pa JOIN sswr.papers p ON pa.paper_id = p.id JOIN sswr.authors a ON pa.author_id = a.id WHERE p.year BETWEEN 2010 AND 2015 AND pa.author_order = 1 GROUP BY a.name ORDER BY COUNT(*) DESC LIMIT 1
select a.name, count(*) as n from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id join sswr.authors a on a.id = pa.author_id where pa.author_order = 1 and p.year between 2010 and 2015 group by 1 order by 2 desc limit 1
select a.name, count(*) as n from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id join sswr.authors a on a.id = pa.author_id where pa.author_order = 1 and p.year between 2010 and 2015 group by a.name order by n desc limit 1
Reference answer: 79
SELECT count(*) FROM (SELECT pa.author_id FROM sswr.paper_authors pa JOIN sswr.papers p ON p.id = pa.paper_id WHERE p.year IN (2005, 2026) GROUP BY pa.author_id HAVING COUNT(DISTINCT p.year) = 2) AS common_authors
select count(*) from (select pa.author_id from sswr.paper_authors pa join sswr.papers p on pa.paper_id = p.id where p.year in (2005, 2026) group by pa.author_id having count(distinct p.year) = 2) t
SELECT COUNT(*) AS n FROM (SELECT DISTINCT author_id FROM sswr.paper_authors pa JOIN sswr.papers p1 ON p1.id = pa.paper_id WHERE p1.year = 2005) a JOIN (SELECT DISTINCT author_id FROM sswr.paper_authors pa JOIN sswr.papers p2 ON p2.id = pa.paper_id WHERE p2.year = 2026) b USING (author_id)
select count(*) from (select pa.author_id from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2005 intersect select pa.author_id from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2026) as common_authors
select count(*) from (select distinct pa.author_id from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = '2005' intersect select distinct pa.author_id from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = '2026') as t
select count(*) from (select pa.author_id from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2005 intersect select pa.author_id from sswr.paper_authors pa join sswr.papers p on p.id = pa.paper_id where p.year = 2026) as common
Reference answer: 2026
SELECT p.year FROM sswr.papers p JOIN sswr.paper_authors pa ON p.id = pa.paper_id WHERE pa.author_order = 1 GROUP BY p.year ORDER BY COUNT(DISTINCT pa.institution_normalized) DESC LIMIT 1
select p.year, count(distinct pa.institution_normalized) as distinct_institutions from sswr.papers p join sswr.paper_authors pa on p.id = pa.paper_id where pa.author_order = 1 and pa.institution_normalized is not null group by p.year order by distinct_institutions desc limit 1
SELECT p.year FROM sswr.paper_authors pa JOIN sswr.papers p ON p.id = pa.paper_id WHERE pa.author_order = 1 GROUP BY p.year ORDER BY COUNT(DISTINCT pa.institution_normalized) DESC LIMIT 1
SELECT p.year FROM sswr.papers p JOIN sswr.paper_authors pa ON p.id = pa.paper_id WHERE pa.author_order = 1 GROUP BY p.year ORDER BY count(distinct pa.institution_normalized) DESC LIMIT 1
select p.year, count(distinct pa.institution_normalized) as n_institutions from sswr.papers p join sswr.paper_authors pa on pa.paper_id = p.id where pa.author_order = 1 group by 1 order by 2 desc limit 1
select p.year, count(distinct pa.institution_normalized) as n_institutions from sswr.papers p join sswr.paper_authors pa on pa.paper_id = p.id where pa.author_order = 1 group by p.year order by n_institutions desc limit 1
Reference answer: 1083
SELECT count(*) FROM sswr.papers WHERE author_count > 6
select count(*) from sswr.papers where author_count > 6
SELECT COUNT(*) FROM (SELECT paper_id FROM sswr.paper_authors GROUP BY paper_id HAVING COUNT(*) > 6) AS sub
select count(*) from (select paper_id from sswr.paper_authors group by paper_id having count(author_id) > 6) as subquery
select count(*) from (select paper_id from sswr.paper_authors group by paper_id having count(*) > 6) t
select count(*) from sswr.papers where author_count > 6