Methodology
This methodology is continuously refined based on empirical data. All changes are versioned and documented. Every text is assessed by the same yardstick — regardless of text type, length, topic, or origin.
How the Rating Works
Every analyzed article from a media outlet receives an individual score. Article scores are aggregated into an overall outlet rating. The entire process is fully automated — with no editorial influence or outlet-specific weighting.
1. Article Score: Four-Factor Model
Each text is analyzed for over 30 persuasion techniques across 9 categories. The score (0-100) consists of four weighted components:
Density
highest weightCaptures how frequently persuasion techniques occur. The measurement is length-neutral — the same number of techniques yields the same value regardless of text length.
Intensity
high weightAccounts for the severity of the techniques (low/medium/high) and the relevance of each category. A few strong techniques weigh more than many weak ones.
Variety
medium weightMeasures the breadth of techniques used. A text mixing many different categories is considered more layered than one repeating a single technique.
Combination
supplementary weightCaptures reinforcement effects when several different techniques co-occur in the same passage — for example, within the same sentence.
Calibration & Scale
The four factors first produce an internal raw signal. This is mapped to the 0–100 value through a fixed, versioned calibration curve. The curve is anchored so a text with no detected techniques scores 0, and assessments spread meaningfully across the full scale. The value is a comparability index (50 ≈ a typical text), not a percentage. Length differences are neutralized, and very short texts are flagged as “limited data”. Since an analysis can consist of one to six analysis steps, partial analyses are scaled to their expected full-analysis equivalent and enter the outlet rating at a reduced weight — so a more thorough analysis does not distort comparability. Within a methodology version, a text's assessment stays stable.
Module Scores & Contribution
The analysis is organized into up to six modules (e.g. core manipulation techniques, logical fallacies, framing). Each module receives two values: The module score (0–100) compares the module's signal strength with what texts typically show in that module — 50 corresponds to a typical text, 0 means no findings. Each module is calibrated against its own reference distribution; modules with a still-narrow data base use a shared, robust reference distribution and are automatically switched to their own calibration as the data base grows. The contribution (in %) shows the module's share of the total signal detected in this text — the contributions of all modules sum to 100%. Module scores and contributions serve as orientation within a single analysis; only the overall score enters the outlet rating.
2. Outlet Rating: Aggregation
Individual article scores are aggregated into an overall outlet rating:
Source Weighting
Each analysis receives a credibility weight based on its source: Registered users (1.0), Automated (0.8), Anonymous (0.7), Seed data (0.5).
Trimmed Mean
With 10 or more analyses, the top and bottom 10% of scores are excluded for outlier resistance. Below 10 analyses, all data points are used.
Inversion
The composite score is inverted: Composite = 100 − Average. Fewer detected techniques = higher score = better rating.
Preliminary Flag
Ratings based on fewer than 10 analyses are marked as "preliminary" to indicate the limited data basis.
3. Grade Scale
An outlet's composite value is mapped to a letter grade. Thresholds follow the actual distribution of rated outlets, so grades spread meaningfully from A to F. An average outlet sits in the C range.
| Grade | Composite | Avg. Techniques | Meaning |
|---|---|---|---|
| A+ | ≥ 82 | avg ≤ 18 | Markedly fewer techniques than most outlets |
| A | ≥ 75 | avg ≤ 25 | Very few techniques |
| A- | ≥ 68 | avg ≤ 32 | Few techniques |
| B+ | ≥ 62 | avg ≤ 38 | Below-average technique density |
| B | ≥ 57 | avg ≤ 43 | Slightly below average |
| B- | ≥ 52 | avg ≤ 48 | Near the average |
| C+ | ≥ 47 | avg ≤ 53 | Average technique density |
| C | ≥ 42 | avg ≤ 58 | Slightly above average |
| C- | ≥ 37 | avg ≤ 63 | Above-average technique density |
| D | ≥ 27 | avg ≤ 73 | Markedly above average |
| F | < 27 | avg > 73 | Very high technique density |
4. Trend Calculation
For each outlet, 7-day and 30-day trends are calculated. These compare the source-weighted average of recent analyses against the current composite score. A positive trend indicates improvement (fewer techniques detected), a negative trend indicates decline.
5. Neutrality Principles
- ✓Fully automated — no editorial influence on scores or ratings
- ✓No outlet-specific weighting — every analysis is processed identically
- ✓No assumptions — the methodology neither favors nor penalizes any outlet regardless of its orientation or reputation
- ✓Traceable — every rating can be traced back to its underlying individual analyses
- ✓Versioned — methodology changes are documented and published
6. Scientific Foundations
The scoring algorithm is based on established research approaches:
- Propaganda Analysis (Institute for Propaganda Analysis, 1937-1942) — Taxonomy of persuasion techniques
- Shannon Information Theory (1948) — Entropy-based diversity measurement
- Psycholinguistics (Weber-Fechner, Zipf) — Perception scaling
- Media Analysis (Herman & Chomsky, 1988) — Propaganda model and framing
- Composite indicators (OECD/JRC Handbook, 2008) — Normalization and distribution of metrics
- Rating-scale calibration (monotonic / percentile methods) — stable scales, comparable across texts
7. Recalibration
The current scale (v2.2) is calibrated to the actual distribution of texts analyzed so far. As the data base grows broader (more outlets, topics, and languages), the calibration is recomputed and published as a new methodology version with a complete changelog. Within a version, a text's assessment stays stable.