Add ballot questions to maple

[fastfadingviolets]

Ballot Questions Feature

This PR delivers the MVP for the ballot questions feature, a new section of MAPLE that adds the concept of ballot questions to the site, mimicking the user experience already in-place for bills (i.e., read, contribute testimony/perspective, follow).

---

Data Model

The ballot question feature adds a thin /ballotQuestions collection that gives each petition its own URL and voter-facing metadata, without touching the existing structures for bills, hearings, or testimony.

interface BallotQuestion {
  id: string // petition number: "25-14"
  billId: string | null // H-bill in the existing bills collection: "H5004"
  title: string | null // human-readable title, currently recommended to be identical to the corresponding bill, minus "An Act" or "An Act Relative"
  court: number // General Court when referred: 194
  electionYear: number // Target election year: 2026
  type:
    | "initiative_statute"
    | "initiative_constitutional"
    | "legislative_referral"
    | "constitutional_amendment"
    | "advisory"
  ballotStatus: 
     | "expectedOnBallot" //Default status for ballot questions 
     | "failedToAppear" //Ballot question has been removed from the ballot for any reason 
     | "accepted" //ballot has received a majority of 'yes' votes at election
     | "rejected" //ballot has received a majority of 'no' votes at election
  ballotQuestionNumber: number | null // "Question 1" — null until assigned by Secretary of State
  relatedBillIds: string[] // admin-curated, format: "194/H1234"

  // Manually curated voter-facing content (all optional until ready)
  description: string | null // "What this question would do" — short voter-friendly prose
  atAGlance: { label: string; value: string }[] | null // "Key Details" bullet list
  fullSummary: string | null // "Summary generated by the Attorney General as required by law"
  pdfUrl: string | null // Link to the initiative petition PDF
}

Data Additions

Ballot question documents are defined as YAML files committed to the repo:

ballotQuestions/2026/
  25-14.yaml

A sync script must be manually called upon addition or edit of any YAML file. The script validates each YAML against the BallotQuestion type (functions/src/ballotQuestions/types.ts) before writing and will throw on malformed input.

# Local emulator
yarn firebase-admin -e local run-script syncBallotQuestions

# Staging
yarn firebase-admin -e dev run-script syncBallotQuestions

# Production (requires GOOGLE_APPLICATION_CREDENTIALS or gcloud ADC)
yarn firebase-admin -e prod run-script syncBallotQuestions

Firestore: Database Indexes & Rules

Composite Indexes (firestore.indexes.json)

Three new composite indexes were added to support the query patterns the feature relies on:

1. ballotQuestions — electionYear ASC + ballotStatus ASC (COLLECTION scope): powers the browse page's filter-by-year and filter-by-status queries.

2. publishedTestimony — ballotQuestionId ASC + publishedAt DESC (COLLECTION_GROUP scope): powers fetching all perspectives on a given ballot question, sorted by most recent.

3. publishedTestimony — billId ASC + court ASC + ballotQuestionId ASC (COLLECTION scope): extends the existing bill+court testimony index to also accommodate the ballotQuestionId field, supporting mixed-scope queries.

A field override was also added to declare ballotQuestionId as indexable on the publishedTestimony collection group, enabling the collection-group-scoped queries above.

Security Rules

A new top-level collection rule was added in firestore.rules:

match /ballotQuestions/{id} {
  allow read: if true;
  allow write: if false;
}

Ballot question documents are publicly readable but not client-writable. All writes go through the admin SDK.

---

Typesense Changes

Ballot questions themselves are not added to Typesense as a separate search collection — the existing Firestore-native query/filter approach is sufficient for the browse page.

However, because one of the new Firestore composite indexes requires ballotQuestionId to be present on publishedTestimony documents (Firestore excludes documents from composite indexes when an indexed field is absent), a backfill was required on the publishedTestimony collection. The script scripts/firebase-admin/backfillTestimonyBallotQuestionId.ts adds ballotQuestionId: null to all legacy testimony documents that predate this feature. This makes existing testimony visible in ballot-question-scoped index queries, and also applies to archivedTestimony. The script is idempotent and safe to re-run.

Additionally, ballotQuestionId was added as a field on BaseTestimony in components/db/testimony/types.ts, so newly submitted perspectives can be tagged to a ballot question when appropriate.

---

Back-End Changes

Database Service Layer (components/db/api.ts)

Two new methods were added to DbService:

• getBallotQuestion({ id }) — fetches a single ballot question by ID
• getBallotQuestions() — fetches all ballot questions (used by the browse page)

Notification Events (functions/src/notifications/populateBallotQuestionNotificationEvents.ts)

A Firestore-triggered Cloud Function was added on the /ballotQuestions/{id} document path. When a ballot question's ballotStatus changes, it creates or updates an entry in the notificationEvents collection. This integrates with the existing digest notification pipeline so that users who follow a ballot question receive email updates when its status changes or when new perspectives are submitted.

Digest Email Templates (functions/src/email/partials/ballotQuestions/)

Two Handlebars templates were added to the digest email system:

• ballotQuestion.handlebars — renders a single ballot question card showing its title, court, and perspective counts broken down by position (endorse / neutral / oppose)
• ballotQuestions.handlebars — wraps multiple ballot question cards; caps display at 4 with a "View All" link when there are more

---

Front-End Changes

Pages

• /pages/ballotQuestions/index.tsx — Browse page: a searchable, filterable grid of all ballot questions
• /pages/ballotQuestions/[id].tsx — Detail page: full single-question view with tabbed content and a sidebar perspective form

Components (components/ballotquestions/)

BrowseBallotQuestions — The browse grid, supporting:

• Full-text search across title and summary fields
• Filters for election year, General Court session, and ballot status
• Cards showing status badge, question number, title, summary excerpt, and testimony count breakdown (endorse / neutral / oppose)
• Responsive layout (3 columns → 2 → 1 on smaller viewports)

BallotQuestionDetails — The detail page shell, which:

• Renders the sidebar navigation and tab content area
• Implements a tabbed interface (currently, Overview and Perspectives tabs are active)

BallotQuestionHeader — The hero section at the top of each detail page, showing:

• Back link to the browse page
• A colored status dot and human-readable status label
• The ballot question number (or an explanatory tooltip if not yet assigned)
• Title
• Meta facts: type, election year, court session, document ID
• Right-hand sidebar with a Follow button (when notifications are enabled) and the YourTestimonyPanel

OverviewTab — Shows:

• The full official summary from the MA Attorney General
• Any committee hearing videos linked via a related bill

TestimoniesTab — Shows:

• Total perspective count with endorse / neutral / oppose breakdown and icons
• A small note directing users to the related bill to see additional testimony
• The full ViewTestimony component filtered to perspectives on this ballot question

YourTestimonyPanel — A conditional panel that:

• Shows the perspective write/edit form during expectedOnBallot phase
• Shows a "no longer accepting perspectives" message during terminal phases (failedToAppear, accepted, rejected)
• Includes an inline edit link when the signed-in user already has a published perspective

DescriptionBox, CommitteeHearing, BallotQuestionTabButton — Supporting display components used by the tabs and nav.

status.ts — Exports categorized sets of ballot statuses (activePhases, terminalPhases) used across the header and testimony panel to determine what UI state to render.

Tests

Unit and integration tests were added alongside the components:

• BrowseBallotQuestions.test.tsx
• YourTestimonyPanel.test.tsx
• BallotQuestionTabLinks.test.tsx
• CommitteeHearing.test.tsx
• tests/integration/ballotQuestions.test.ts — integration tests verifying YAML sync and composite index queries

---

Data Flow Summary

git repo: ballotQuestions/{year}/{id}.yaml
    │
    ▼  (syncBallotQuestions admin script)
Firestore: /ballotQuestions/{id}
    │
    ├─▶  DbService.getBallotQuestions()  →  /ballotQuestions (browse)
    ├─▶  DbService.getBallotQuestion()   →  /ballotQuestions/[id] (detail)
    │
    ├─▶  ballotStatus change  →  Cloud Function  →  /notificationEvents  →  digest email
    │
    └─▶  User submits perspective  →  publishedTestimony (with ballotQuestionId)
              │
              └─▶  Typesense indexes testimony for full-text search

Known Issues

• Ballot Question perspectives are displayed on the Browse Testimony page as though they are associated with the corresponding bill. The perspectives do not display on the bill's page, and clicking the individual testimony/perspective opens to the testimony detail page which shows it correctly only associated with the ballot question.
• Perspectives count on the individual ballot question page are unreliable - immediately after submitting a perspective, it sometimes does not increment even though the new perspective is rendered.

These issues are known and will be addressed on Monday :)

[Mephistic]

Looks good! None of my comments are blocking for this PR, but worth discussing for the follow-up.

Let's sync before we merge to make sure I've got all the items in the deploy checklist ready. AFAICT, it's just:

• Merge this + Deploy
• Run syncBallotQuestions
• Run backfillBallotQuestionTestimonyId
• Run backfillTestimonyBallotQuestionId

Happy to move this to DEV for testing (though I can discuss with Matt V tonight whether we should hide the new functionality behind a feature flag before deploying to PROD).

[Mephistic]

Is this the desired behavior? If a Ballot Question goes from state A -> state B -> state C, I would expect to see three notifications (one for A, one for B, and one for C), not a single notification that is modified from state A to B to C.

[LexaMichaelides]

In the current model of ballot question states, a question can't move through three states - they would all begin as expectedOnBallot and then all three additional states are terminal (failedToAppear, approved, rejected). A ballot question can't move from A -> B -> C.

In the future if additional states are added, I think multiple meaningful movements wouldn't happen within the timeframe of a single round of email notifications (a month, at most). Coherent additional states might be "legislature" and "confirmedOnBallot" imo, so the path would be:

• legislature -> expectedOnBallot -> confirmedOnBallot -> approved/rejected (terminal) OR
• legislature -> expectedOnBallot -> failedToAppear (terminal)

In that second case, I don't think it's meaningful to the user that a question was expected before it failed. But that's open for discussion, of course! I can add to the agenda to talk about if and when we add new states.

[Mephistic]

Given that we store counts on the Ballot Question docs themselves now, do we still need to handle this case?

[tripledoublev]

Good catch.
Now that counts are stored on the BQ doc, this is dead code.
I removed it in hyphacoop#52 and merged it here.

89227cf

[Mephistic]

Same Q here - do we still need the fresh count here now that we're keeping the counts in the Ballot Question itself?

[tripledoublev]

Same fix applied in hyphacoop#52

It reads directly from the BQ doc now.
89227cf

[LexaMichaelides]

Let's sync before we merge to make sure I've got all the items in the deploy checklist ready. AFAICT, it's just:
* Merge this + Deploy
* Run syncBallotQuestions
* Run backfillBallotQuestionTestimonyId
* Run backfillTestimonyBallotQuestionId
Happy to move this to DEV for testing (though I can discuss with Matt V tonight whether we should hide the new functionality behind a feature flag before deploying to PROD).

Definitely let's do DEV before PROD! And I like the feature flag - since we've got all this ready to go before legislature has failed to pass the bills, and we were aiming to release at the same time the ballot questions move out of legislature.

* Merge this + Deploy
* Run backfillTestimonyBallotQuestionId
* Run syncBallotQuestions

backfillBallotQuestionTestimonyId is maybe a typo in your comment? That's not an existing script, but I think you might mean backfillBallotQuestionTestimonyCounts. This script was used in dev as we were changing from one count storage to another when we had existing data in the db. This should not happen in prod so it does not need to be run.