Scientific problem
Scientific problem
What is the problem, and why does it matter scientifically?
Specific field
Solvability
References & resources
Environment
Environment
What does the agent need to run this task?
Software and tools
Dataset & artifacts
Evaluation
Evaluation
How is success measured, and how is that measurement kept reproducible?
How will this task be evaluated?
What are the risks of cheating and leakage?
Contributor
Contributors
How you and any co-contributors will be credited, and how reviewers can reach you.
Review and submit
Review & submit
Submitting opens a GitHub Discussion linked to your GitHub identity, so sign in first. Nothing is sent until you press Submit — and nothing here is final, since you can edit the proposal later from its task page.
Checking sign-in…
Sign in with GitHub
Submit proposal
Copy as Markdown
Copied
Preview your proposal
Rendered
Markdown
Proposal submitted
Your proposal is now a GitHub Discussion, where reviewers will reply. To revise it later, open your task page and choose Edit proposal — changes there update both this site and the Discussion.
Domain review. Reviewers assess scientific value, verifiability and difficulty, and record the outcome in the Discussion.
Discussion. Questions about your proposal happen on Discord , where your task page links to its thread.
Revisions. Reviewers may ask for changes. Make them on your task page with Edit proposal ; the Discussion updates with it.
Task PR. An accepted proposal becomes a task specification and verifier in the benchmark repository. The contributor guide explains each step.
Validation. The task is run against agents and, once verified, enters a release.