The Measurement Challenge
Retroactive funding promises to reward demonstrated impact, but measuring "impact" for public goods is notoriously difficult. Optimism has iterated significantly across rounds.
Evolution Across Rounds
RetroPGF 3 (January 2024)
30M OP to 501 projects
- Approach: Badgeholder voting with minimal structure
- Categories: OP Stack, Governance, Dev Ecosystem, End Users
- Challenges:
- 644 projects too many to evaluate thoroughly
- Cross-category comparison difficult
- Some gaming of profile presentation
RetroPGF 4 (June 2024)
10M OP with focused scope
- Approach: Narrowed to specific impact areas
- Improvements: Better category definition, clearer criteria
- Results: More consistent evaluation, still some subjectivity
RetroPGF 5 (Fall 2024)
8M OP with refined process
- Focus: Dev tooling and infrastructure
- Innovations:
- Impact metrics framework
- Badgeholder training
- Clearer evaluation rubrics
RetroPGF 6 (Active)
2.4M OP focused on governance
- Scope: Governance contributions only
- Approach: Narrow focus allows depth
- New: Algorithmic initial ranking
Key Learnings
- Scope matters: Narrower scope enables better evaluation
- Training helps: Badgeholder preparation improves consistency
- Metrics + judgment: Neither purely quantitative nor qualitative works alone
- Iteration required: Each round informs the next
Recommendations for Other Programs
- Start with narrow scope and expand
- Invest in evaluator training and support
- Build impact measurement infrastructure
- Plan for multi-round iteration