JackCrawford

A political sentiment tracker that monitors media coverage of New Zealand's 2026 general election. It scrapes the major NZ news outlets, scores the sentiment of each article toward every party, and plots that against real polling data over time.

Tech Stack

• ·Frontend: React 18 with TypeScript on Vite, charts built with Recharts, and a guided tour via React Joyride. Routing is hash-based for easy embedding. Deployed on Vercel.
• ·Backend: ASP.NET Core on .NET 9 in C#, with Entity Framework Core. SQLite for local development, PostgreSQL in production (on Railway). Scheduled scraping runs on Quartz, and Swagger is exposed for API exploration.

How It Works

The pipeline runs on a schedule: Quartz fires every two hours, the scraper pulls articles from a set of NZ outlets (RNZ, Stuff, Newsroom, The Spinoff, Guardian NZ, Waatea) via a mix of RSS and HTML scraping, and each new article runs through scoring. Duplicate URLs are skipped so the job is idempotent.

Sentiment uses two independent models. The first is a rule-based dictionary scorer: a hand-built lexicon of weighted sentiment words with negation handling and intensifiers, so 'extremely praised' scores higher than 'praised' and 'not praised' flips negative. The second is an optional ML model (FinBERT via the HuggingFace inference API), which is good with financial and political language. When both are available I blend them into a composite score weighted 65% ML, 35% dictionary. The point of the dictionary scorer is that it works immediately with no API key and no external dependency; the ML model is an enhancement, and the whole thing falls back gracefully if the key's missing or the API is down.

Figuring out which party or politician an article is actually about is done with compiled word-boundary regex rather than NLP or naive substring matching. That distinction matters: it stops 'national park' registering as the National Party, or 'green energy' as the Greens. Politician names (with their aliases) auto-link back to their party. There's also a keyword-based topic classifier across ten categories like Economy, Housing, Healthcare and Climate.

Scored articles get aggregated into daily sentiment snapshots per party, and an insight generator flags things like sentiment shifts, volume spikes, and one party overtaking another. Polling percentages are scraped from Wikipedia's 2026 election polling page, with political events marked on the sentiment timeline as vertical lines. The frontend is deliberately focused on one strong chart — sentiment time-series per party — and keeps live polling to a minimal standings panel beneath it, with a dedicated polling-history page, a single-party page, and an embeddable version a click away. The read API is public; the admin endpoints (rescore the corpus, backfill history, re-fetch missing bodies) are protected with an API-key header.

Decisions

• ·Regex over NLP for entity detection: fast, predictable, and the word boundaries kill the obvious false positives.
• ·SQLite locally, Postgres in production: one set of EF Core migrations, zero-setup dev, scalable prod.
• ·Quartz on a 2-hour cron: the data keeps itself current without me touching it, and the same logic can be triggered manually for backfills.
• ·Targeted tests on the parts most likely to be wrong: the lexicon and the entity matcher, since those drive every number in the app.