Methods, quality assurance and limitations

How AIR records, classifies and compares evidence

This methodology makes the programme’s units, coding judgements, calculation rules, comparison permissions and limits of inference inspectable. AIR is an exploratory public-record study: its current releases support transparent description and hypothesis generation, not population incidence estimates, individual prediction or causal proof.

Methods version
1.0 effective 16 July 2026
Case data
v4.2.3 RC internal; public gate on hold
Context release
v1.5 4,141 versioned observations
Research status
Descriptive and hypothesis-generating

Scope, period and units of observation

The case database covers retained public-source records of terroristic attacks, thwarted terrorist plots and Coercive Ideological Violence Mobilisation (CIVM) incidents across Australia, Canada, New Zealand, the United Kingdom and the United States from 2000 to the current partial 2026 release. The case is the unit of observation: one row represents one active canonical incident, not one person, organisation or movement.

The social-context database uses a different unit. One row represents one released indicator–country–year observation. Context records describe national conditions; they are not attributes of an accused person, offender, victim or individual case.

Central interpretation rule

Case records are a systematically assembled public-record research corpus, not a census of all violence. Country-year context is ecological. Neither source can by itself establish individual exposure, motive, pathway or causation.

Three mutually exclusive primary incident categories

Every calculation-included case contributes to one primary incident category. Separate overlap fields may preserve analytically relevant features, but they do not place the same case into two primary totals.

Terroristic attack

An executed incident meeting the project’s element-by-element terrorism test: intentional lethal or other serious violence, hostage-taking or a credible qualifying threat; a political, religious, ideological or similarly consequential purpose; and an intended intimidation or coercive effect beyond the immediate victim. Actor scope, legal stage and statutory treatment remain separately recorded.

Thwarted terrorist plot

Preparatory, attempted or planned qualifying conduct disrupted before the intended attack was completed. Allegations, charges, convictions and final legal outcomes are recorded separately; disruption does not prove every alleged feature of the plan.

CIVM incident

A project-defined category for coercive, rights-infringing or violent ideological mobilisation that does not meet the project’s terrorism threshold. CIVM is not a legal offence and does not turn lawful dissent, unpopular expression or peaceful protest into violence.

Lawful activity

Lawful protest, advocacy, assembly and democratic participation are excluded. Where qualifying conduct occurs during a wider event, the record concerns a bounded actor or offending group supported by the sources—not the entire protest crowd or a wider identity community.

Minimum CIVM inclusion screen

A CIVM case requires reliable public evidence supporting all four elements:

  1. Completed or attempted interpersonal violence; a credible coercive threat; serious rights infringement; or an unlawful occupation, blockade or deliberate property-destruction episode causing substantial impairment. Peaceful civil disobedience, ordinary disruption and symbolic trespass do not satisfy this element by themselves.
  2. An ideological, identity-based, political, religious, conspiratorial or single-issue mobilisation context.
  3. An identifiable suspect, accused or convicted person, or a bounded offending group, rather than an undifferentiated crowd.
  4. Documented online or social-media activity, or a documented group or movement whose mobilisation and communications materially used online channels.

Arrest is allegation-stage evidence. It supports recording a suspect or bounded group where appropriate, but it is not treated as proof of guilt.

Terrorism and organisation-designation decision rule

The project test asks separately whether the conduct, purpose and wider intimidation or coercion elements are supported. Legal terminology differs across the five jurisdictions, so a project category is never represented as though it were a uniform statutory finding.

Under the current project-lead rule, involvement by an organisation that is currently listed by at least two Five Eyes governments is coded as terrorist-organisation involvement and the incident is assigned to a terrorism category rather than CIVM. This is a project classification convention, not a declaration of individual criminal liability. A versioned designation matrix, aliases, effective dates and official-list URLs must accompany the public release before this convention is treated as reproducible.

AIR is an evidence level, not a causal or harm score

The AIR evidence level is an unvalidated proxy for the strength and character of publicly documented digital-pathway involvement. It may combine evidence of exposure, communication, participation, production or offence-related dissemination. It is not a direct measure of recommendation-system exposure, psychological immersion or radicalisation, and it does not rank violence, culpability, ideological severity or dangerousness.

  1. Nil

    No qualifying algorithmic or social-media immersion under the project rule. Every incident before 1 January 2005 is Nil. Source-supported offline pathways also qualify. Operational internet, email or coded communication alone does not automatically establish immersion.

  2. Lower

    Limited or peripheral case-specific online contact, browsing, recruitment, communication or exposure, without evidence of sustained immersion.

  3. Moderate

    Substantive online context: relevant videos, propaganda, websites, chat groups or source-supported online organisation, but without case-specific evidence of sustained active participation at the Higher threshold.

  4. Higher

    Case-specific sustained or active online involvement, including content production, offence-related posting, manifesto publication, livestreaming, recruitment, group administration or documented self-directed online immersion. A post, manifesto or livestream proves digital activity; its timing and content must be assessed before it is interpreted as evidence of prior immersion.

For a post-2005 case, Nil requires affirmative source support for a predominantly offline pathway together with sufficiently informative source coverage; silence about online behaviour is not negative evidence. Unknown / Not Established remains the outcome when the reviewed public record does not support either an affirmative offline finding or a positive AIR level. A completed review does not convert undisclosed evidence into Nil.

Why 2005? The pre-2005 Nil boundary is a conservative project classification rule intended to avoid projecting the later social-media environment backwards. From 2005 onwards, a case becomes eligible for evidence-based coding; its date alone supplies no evidence of digital influence.

Source hierarchy, triangulation and expert judgement

Material facts are sought first from court decisions, charging documents, government designation lists, police statements, official inquiries and recognised statistical agencies. Peer-reviewed research and major academic publishers follow. Established international and national news organisations support independent corroboration; high-quality local reporting may supply incident detail. Partisan or advocacy sources are used only as leads or as evidence of what that actor claimed, not as sole confirmation of a contested fact.

Material facts should normally be triangulated between an official or primary source and at least one independent reputable source. Source and review metadata are retained in the master research infrastructure but are not part of the 22-field public case release. Where sources conflict, the underlying case record should preserve the disagreement, legal stage and uncertainty instead of collapsing them into an unqualified assertion.

What “expert coding” means

The project lead applies a published rubric to source-qualified evidence and records a reasoned professional analytical judgement. The programme sometimes describes this as a reasonable-grounds analytical standard. It is not a criminal-law burden of proof, a judicial finding or an assertion that every coded interpretation is an adjudicated fact.

How fact, judgement and inference are separated
  • Observed record: a value or characteristic represented in a released record or retained source.
  • Statistical description: a reproducible calculation from identified released records, with its denominator and data version.
  • Expert analytical judgement: a transparent application of the coding rubric where evidence requires interpretation.
  • Hypothesis or open question: a proposition that the present evidence does not establish and that requires further testing.

Fixed AIR Programme Insights apply a second, editorial classification to the resulting public claim: Observed finding, Statistical association, Informed hypothesis, or Methodological or contextual observation. An underlying record does not automatically become a finding, and an expert judgement does not become a statistical association without a disclosed analysis.

Authoritative scope and public denominator

The project lead-reviewed v4.2.3 master workbook contains 347 case records. The public denominator is the 346 records marked “yes” in the authoritative included_in_calculation field. The single record marked “no” remains in the master workbook for audit lineage and is excluded from the public dataset and all public case calculations.

Public KPIs, charts, percentages, cross-tabs, filtered exports and case–context comparisons use this 346-record release. The row-level payload and CSV contain only the 22 fields approved for publication; internal review, evidence-management and administrative fields are not exposed.

v4.2.3 authoritative and public scope
ScopeRecordsPublic use
Authoritative master workbook347Complete project record, including audit lineage
Marked “yes” for inclusion346Public case denominator
Marked “no” for inclusion1Retained only in the master workbook; not publicly released
Public case rows346Reviewed v4.2.3 release with 22 approved fields

The dataset does not treat a national movement, ideology, demographic category or organisation as a case. Multi-place episodes are separated only where sources support distinct incidents; duplicate representations do not enter the calculation denominator twice.

Unknown is not zero, and a partial year is not a full year

  • Recorded zero: the released record supports a numerical value of zero.
  • Missing or blank: no released value is available. It is never converted to zero, false, “none” or a negative finding.
  • Unknown / Not Established: review did not establish a defensible substantive value.
  • Suppressed: an aggregate value exists but is withheld under the small-cell rule.
  • Not applicable: the field or calculation does not apply to that unit.

The public case time field is the event date. Annual charts derive the event year from that date. The public release does not expose internal start or end date fields; users can consult the case summary where additional timing context is available.

The reviewed v4.2.3 workbook was supplied on 16 July 2026; this is not a claim that every eligible public source published by that date was found. The 2026 period is incomplete. Every affected chart, table and export identifies 2026 as partial and excludes it by default from complete-year rankings, rolling summaries and matched full-year change comparisons.

Comparison permissions are part of the data

Social Context v1.5 contains 4,141 versioned observations and a 79-entry runtime indicator registry derived from 76 source-master indicators. The registry, not the visual interface, determines which operations are permitted. Two legacy digital-time series account for 124 methodology-only observations; they remain available for audit but are not public analytical series.

Social-context dashboard classes
ClassEntriesPermitted use
Comparable with caveats33Levels, within-country trends and qualified same-year cross-country comparison; exploratory association only when explicitly permitted
Within-country trends only32Levels and change within the same country; no ranking, pooling or cross-country association
Snapshot or context only12Labelled value or context card; no trend claim, ranking, pooling or association
Methodology only2Audit documentation only; absent from public analytical selectors and charts

Every context visual must travel with its definition, unit, population or age base, source, observed coverage, dashboard class, derivation status and caveat. A value from one country does not become cross-country comparable merely because another country uses the same broad label.

Within-country and cross-country rules

Within-country trends require an indicator whose registry permits trend analysis. Lines show released observations, preserve gaps and mark partial years or documented breaks. They do not interpolate, carry values forward or select a nearest year.

Cross-country comparisons require the comparable with caveats class, the same governed period, compatible definitions, units and population bases, and visible coverage. Snapshot and within-country-only indicators cannot be ranked or pooled.

A common title or unit is not enough. Survey instruments, legal definitions, collection modes and denominator populations can make apparently similar measures substantively different.

Transformations must preserve the released values

Levels and change

Raw released values are the default. Absolute change, percentage-point change and relative percentage change require matched periods and a construct for which that operation is meaningful. Start and end values remain visible.

Rolling averages

A rolling value requires consecutive annual observations, an eligible trend series, no missing or partial year and no unresolved break within the window. The window and exclusions are stated; raw observations remain available.

Indexing and standardisation

Baseline indices require the same observed baseline year. Z-scores use a disclosed complete-case sample and report its mean, standard deviation and n. Standardised displays do not create a validated risk score.

Rates and percentages

Recorded case rates require a defensible same-country, same-year population denominator and are labelled “recorded cases per million residents”—not incidence, prevalence, threat or risk.

Unlike units are not placed on an unlabelled common axis. No missing value is interpolated by default. No nearest-year match, generic population weighting, automatic direction reversal or composite “AIR context score” is permitted.

Five Eyes totals and averages

Recorded cases use a Five Eyes total, not an average of national counts. A Five Eyes average is used for context only when all five countries have a value for the same governed period and the indicator, unit, population base and transformation are compatible. The default is an equally weighted arithmetic mean where equal country weighting answers the question.

When fewer than five countries are observed, the result is labelled available-country mean (n = x/5) and the absent countries are named. A pooled value is reported separately, with its numerator, denominator and formula; it is never presented as an unqualified average.

Cross-domain analysis is ecological and exploratory

Once an approved public case release exists, case and context data may join only through the explicit Five Eyes country crosswalk and exact calendar year. Aggregate analyses will use publication-approved, calculation-included cases. No nearest-year matching, interpolation or carry-forward is allowed when a same-country, same-year context value is unavailable.

Mandatory ecological caution. Country-year context describes the national environment in which recorded cases occurred. It does not measure an individual actor’s exposure, motive, pathway or causal response. Temporal correspondence and statistical association do not establish causation.

Matched trend panels and exact country-year tables are preferred to a single coefficient. Any exploratory correlation or lag must state the complete-case n, countries, years, missing pairs, method and lag direction. Five countries with repeated observations are not a large independent sample: clustering, autocorrelation, common time trends, rare counts and changing ascertainment can all produce misleading associations.

Exploratory views and AIR Programme Insights are different research products

The public platform provides two complementary analytical experiences. The AIR Research Laboratory lets a user choose permitted variables, countries, periods, transformations and displays. Its outputs are exploratory views generated from the active specification. They may identify patterns or motivate questions, but they do not become AIR programme findings simply because the software can draw them.

AIR Programme Insights are fixed, non-editable figures selected and reviewed by the research team. Each released insight must identify its evidence classification, data version, denominator, coverage, missingness, source lineage, figure version and last review date. Its explanatory panel must state what the figure shows, why it may matter, plausible and alternative explanations, what it does not prove, relevant AIR and external scholarship, limitations and questions for further research.

Evidence classifications for curated AIR Programme Insights
ClassificationMeaningInference boundary
Observed findingA reproducible descriptive result directly supported by the named released data.Description does not by itself establish mechanism or cause.
Statistical associationA relationship supported by a disclosed statistical analysis with its sample, method and uncertainty stated.Association is not a causal estimate unless a separate design justifies that claim.
Informed hypothesisA theoretically grounded AIR interpretation that remains open to empirical testing.The interpretation is not presented as an established fact.
Methodological or contextual observationA pattern that may reflect coverage, classification, reporting or data-generation conditions.The observation may qualify or complicate a substantive interpretation.

One figure may contain more than one kind of statement, but each statement must be labelled at the level it actually supports. A planned analysis receives no evidential classification until the analysis has been completed and reviewed. Curated figures that do not support an AIR hypothesis are eligible for publication on the same terms as those that do.

Quarterly figure review and version history

Curated insights are reviewed in January, April, July and October. Review does not require a visual to change. A figure is revised only when a new data release, corrected calculation, material change in interpretation or relevant scholarship warrants it. A major change receives a new figure version and a dated history entry; a retired figure remains identifiable in the archive.

The archive uses immutable figure-version records and an append-only lifecycle ledger. Each version preserves its exact data, explanatory panel, SVG, citation and file manifest. Current, superseded and retired status is derived from appended events; an earlier version is not edited to change its status. Repository-history checks, event hashes and bundle checksums make mutation detectable. They do not prevent an authorised hosting operator from bypassing the governed workflow.

Era bands and event markers are contextual annotations. They must remain visually subordinate to the observations, identify their source or governing rule, and must not be drawn or described as causal breakpoints. Dataset and methodology changes are distinguished from social or technological events.

Current release boundary

Context–context exploration and approved context-only insights may use the governed Social Context v1.5 public release. Context–case, case–case and case-derived curated figures remain unavailable until the separate case-data publication and figure-review gates pass.

Your Digital Use is not an individual assessment

The tool accepts only four time components, an entry-scope choice and optional reference choices. Its JavaScript prevents form submission and does not place entries in a URL, cookie, local storage, session storage, download or AIR dataset. The separate site-theme control may retain only the user’s light, dark or device preference. If JavaScript is unavailable, the calculation control remains disabled; if the governed reference file fails to load, personal arithmetic remains available and the comparison stays unavailable.

Personal arithmetic

Total minutes equal hours × 60 plus minutes. Social-media share equals social-media minutes divided by total entered minutes × 100. Social-media minutes cannot exceed total minutes, and neither value can exceed 1,440 minutes per day.

Measurement scope

A one-device entry describes the selected device report. An all-device entry is the user’s estimate of internet and social-media time across devices. Operating-system categories, simultaneous use, offline applications and uncaptured devices can make either value incomplete.

Contextual reference

The optional reference uses only the 2024 values for the audited GWI daily internet-time and social-media-time indicators in Social Context v1.5. The source population is internet users aged 16–64. A Five Eyes value is an unweighted mean of the five available country estimates, not a population-weighted average.

Comparison gate

A numeric difference from the reference is calculated only when the user selects an all-device estimate and confirms that the source population applies. A one-device report may be displayed beside the reference, but no difference is calculated because the constructs are not directly equivalent.

No output is labelled healthy, unhealthy, safe, dangerous, low, moderate or high. The tool provides no percentile, recommended limit, mental-health or loneliness screen, diagnosis, AIR evidence level, radicalisation inference or public-safety score. Its reflection questions distinguish time from content, purpose, platform design, reciprocal connection, displacement and perceived control.

Evidence boundary. Research on digital use and wellbeing is heterogeneous and much prominent work concerns adolescents. Association does not establish causation, youth findings do not automatically generalise to adults, and national estimates cannot determine an individual outcome.

Sensitive characteristics are descriptive, not predictive

Religion, migration, citizenship evidence, refugee or asylum pathway, sex, gender and age are descriptive fields and are not treated as motives, causes or population risk factors. The programme does not produce a demographic profile, predictive score or country danger ranking.

The project lead directed that a source description limited to “youth”, “teenager” or “minor” may be assigned age 16 for broad under-/over-25 grouping and aggregate age calculations. That is a project-derived imputation, not a source-reported age. The present public schema does not yet expose an age-evidence-status field; age means must therefore carry this limitation until the derived values are explicitly flagged.

Aggregate cross-tab cells involving sensitive personal characteristics are suppressed when the count is below five and displayed as <5. Complementary or secondary suppression is also applied where a row, column, total, chart mark, tooltip, text alternative or export would allow the hidden value to be reconstructed. The safeguard prevents sparse aggregate combinations from being presented as precise comparative evidence.

Per-record publication-minimisation gate. Before public release, every case record must carry a reviewed publication status covering name or pseudonym, youth and victim protection, legal restriction, sensitive identity fields, public-interest necessity, proportionality, source strength and the removal or correction route. Public availability alone is not sufficient ethical justification for republication. Until that gate is complete, no individual-record list is delivered to the public site.

Actor ethnicity, individual mental health, controlled platform use, urban or rural location and substantive citizenship or nationality are not approved governed comparative case variables in v4.2.3. The interface must not infer them from names, appearance, religion, addresses, narrative text or media shorthand.

Versioned releases, immutable inputs and visible corrections

  1. Preserve the research masters. Authoritative workbooks are retained as immutable inputs and identified by release version and cryptographic hash.
  2. Build governed public outputs. Explicit country crosswalks, controlled vocabularies, inclusion flags and indicator permissions are applied reproducibly.
  3. Validate structure and reconciliation. Release checks test source and output hashes, unique keys, chronology, required fields, aggregate reconciliation, partial-year flags, permissions, source retention and missing-value rules.
  4. Publish provenance with approved data. A public release is accompanied by an approved dictionary or indicator registry, source treatment and release metadata. A source URL is not republished merely because it exists internally.
  5. Correct through version control. A material factual, coding or methodological change receives a new version and validation record. It is not silently overwritten. Historical identifiers and exclusion lineage are retained where needed.

The reviewed v4.2.3 case release passes the public-projection validator: it reproduces the authoritative inclusion decision, contains 346 unique case identifiers, requires an event date, preserves chronological order and exposes exactly the approved 22 fields. Social Context v1.5 and Data Contract v1.0 retain their governed public validation status. Automated checks validate declared structure and transformations; they do not independently verify every substantive fact against external sources, establish inter-coder reliability or prove that every expert classification is uniquely correct.

What the present releases cannot establish

  • Public-record selection. Detection, prosecution, disclosure, archiving and media attention differ across time and jurisdictions. Recorded cases are not population-complete incidence.
  • Classification uncertainty. Terrorism, ideology, mobilisation, organisational attribution and online pathways can be contested or change as proceedings and evidence develop.
  • Uneven missingness. Unknown and not-recorded characteristics often reflect source limits, not the verified absence of a trait or pathway.
  • Changing regimes. Laws, statistical definitions, survey instruments, reporting practices and platform environments change over the 2000–2026 period.
  • Rare and heterogeneous outcomes. Small counts and influential incidents make annual differences unstable. CIVM, attacks and thwarted plots are not interchangeable outcomes.
  • No individual exposure measurement. National digital-use measures do not show what an actor viewed, how recommendation operated for that person or what counterfactual outcome would have occurred without it.
  • Ecological inference. National migration, religion, trust, crime, health or economic measures cannot be used to infer an individual case pathway.
  • Current indicator gaps. The released context data contain no harmonised Five Eyes loneliness, foreign-interference, platform-specific exposure, general mental-health or border-system performance panel suitable for the analyses those labels might imply.
  • No causal validation. Temporal overlap, visual association and descriptive correlation do not identify an algorithmic, social-media, policy, migration or contextual causal effect.

See how migration context, case evidence and border-policy claims are separated →

Completed quality assurance and work still required

The v4.2.3 workbook has completed the project lead’s coding review. The public release checks reconcile record counts, require unique case identifiers, validate event-date presence and ordering, enforce the approved field contract and prevent missing values from becoming zero. The public payload contains 346 records; one master-workbook record marked “no” for inclusion is omitted.

Row-level public publication is a separate gate. Every case, field group and source still requires a documented publication decision, including minimisation, second review and correction-route readiness. Until that gate is approved, no case records or case-source register are delivered to the browser or public download directory.

Important boundary. The reviewed release has not completed a formal independent inter-coder reliability study, external replication or causal validation of AIR. Structural checks and expert review must not be described as though they establish those results.

Planned empirical work includes:

  1. Freeze a detailed coding manual, decision rules and coder-training set before reliability testing.
  2. Independently double-code a pre-specified sample—or the full corpus where feasible—without access to the first coder’s decision.
  3. Report raw agreement and an appropriate chance-corrected reliability statistic for each central classification, with prevalence and sparse-category effects disclosed.
  4. Preserve disagreements, adjudication decisions and pre-adjudication results so reliability is not calculated only after consensus.
  5. Test sensitivity to alternative inclusion, AIR-level, temporal and source-quality rules, including null and contradictory findings.
  6. Seek external methodological review and replication using the released data, dictionaries, source registers and documented transformations.
  7. Use separately specified longitudinal or quasi-experimental designs for any future causal claim; do not infer causality from the present descriptive platform.

Reproduce and audit the public releases

The governed Social Context release is distributed as UTF-8 CSV files. Missing values remain blank, ISO dates are preserved, and the dataset is accompanied by definitions, sources and release metadata. A filtered or transformed export must retain the unaltered released value, identify its filters and formula, state the release version and report missing, excluded, partial and suppressed observations.

Open the Research LaboratoryReview AIR Programme InsightsBorder Security & Public SafetyExplore case dataExplore social contextYour Digital UseReview the AIR FrameworkGlossaryReferences