Methodology: How We Find Opportunities

How does Opportunity Universe bring you the largest database of programmes worldwide?

This page documents how Opportunity Universe discovers, extracts and verifies the opportunities it publishes. The platform is built around an automated crawling and extraction pipeline that combines large language models with deterministic validation rules. The sections below describe each stage of that pipeline, from source acquisition to daily archiving, and report the figures the system currently produces.

1. Source acquisition

The crawler monitors three classes of source.

1. Established opportunity platforms. The public listing sites shown in the table further down, read on a fixed schedule.
2. Primary publishers. University, foundation, ministry and non-governmental organisation websites, read directly at the source.
3. Internet-wide search. A query layer that issues structured searches in 25 languages and surfaces programmes that appear on no listing site at all.

Because primary publishers are indexed directly, the database does not depend on any other platform remaining online, complete or accurate.

2. Extraction and classification

Every candidate page is passed to a language model that performs two tasks. It classifies the page, separating genuine calls for applications from news articles, listicles and closed announcements. It then extracts a structured record containing title, organiser, deadline, eligibility criteria, funding type, location and application links. The output is validated against a controlled vocabulary of types, subjects and regions, so that each record is machine-comparable rather than free text.

3. Deduplication

A single programme is frequently republished across many websites. The system resolves these into one canonical record with multiple attributed sources, using title similarity, organiser identity and deadline proximity. Recurring programmes are additionally linked into series. When an edition closes, the pipeline re-examines the series periodically, which allows the following edition to be detected as soon as it is announced.

4. Canonical source links

Each published record links to the website of the organiser and to the official application page. The platform does not link to other aggregators and does not route applicants through redirects or tracking intermediaries. Applications are therefore always submitted at the source.

5. Quality scoring

Every record carries three independent scores: extraction confidence, organiser prestige and legitimacy. Records below the confidence threshold are withheld from the public index and queued for review. Prestige and legitimacy inform ranking, so that well established and fully funded programmes appear before unverified listings.

6. Refresh cycle and archiving

Sources are synchronised on individual schedules that reflect how often they publish. High-volume platforms are re-read every few days, primary publishers weekly to monthly, and the internet-wide sweep runs monthly. Deadlines are evaluated daily and expired entries are archived automatically, so that the searchable index reflects opportunities that are genuinely open.

Current figures

Comparison with manual curation

Editorially curated platforms publish what a small team can read and write in a working day, which in practice limits them to a few entries per day, predominantly in English and predominantly from sources they already track. An automated pipeline reads those same sources, adds the primary publishers behind them, and operates continuously in every language it covers. This difference in throughput, rather than any difference in editorial judgement, explains the gap shown in the chart below.