Skip to main content

Evaluations: scope, limits and rubrics

Summary

Every score in a Tech Assessment comes from an evaluation. An evaluation is a self-contained judgment: it does its own research and applies its own rubric to produce a score out of 100.

Twelve evaluations exist. An assessment template selects a subset of them and gives each one a weight. The default template (Tech Transfer Office) uses eight, so an assessment run on it shows eight scores. For which evaluations your template runs and how the weighting works, see Assessment templates: choosing, creating and editing.

No score is blended into another before you see it. The separation is deliberate. A technology can score well on market size and still be held back by a weak patent position, and you will see both numbers rather than one average that hides the second.

When to use

  • A score looks wrong and you want to know what the number was actually measuring.

  • You are comparing two assessments and they show different sets of scores.

  • You need to explain a result to an inventor, a licensing counterpart, or a committee.

  • You are deciding how much weight to put on a single number before you act on it.

  • You are choosing or building a template and need to know what the twelve options assess.

How it works

What an evaluation is

An evaluation answers one question about the technology. It gathers its own research, works through its own list of criteria, applies its own rubric, and reports a score out of 100 with the reasoning behind it.

Evaluations do not consult each other. Each one has a defined scope so it does not re-score what another evaluation already covers. Public disclosure timing, for example, belongs to one evaluation and stays out of the patent-position evaluation, so no single problem counts twice.

Averaging happens once, in the Overall Score, and only after every evaluation has reported on its own terms.

The twelve evaluations

These are the twelve a template selects from.

All twelve are available on both Basic and Premium. Your tier does not change which evaluations you can run or what each one reports back, including the written reasoning behind its score. For what the tier does change, see Basic and Premium: what changes.

Evaluation

What it asks

IP Viability

Is the invention novel and non-obvious against existing patents and publications, and how crowded is the patent field around it?

Market Size & Revenue

How large is the addressable market, is it growing, and what licensing revenue is realistic?

Competitive Positioning

How much better is this than the solutions already in use, and how defensible is that advantage?

Technology Readiness

How mature is the technology, and what time and investment remain before it is deployable?

Partner Fit & Licensing

How many credible licensees exist, and how well does the technology match what they are already working on?

Regulatory Complexity

What approvals, trials, or certifications stand between the technology and market, and how well established is that pathway?

Subject Matter Eligibility

Is this the kind of subject matter patent law permits at all, assessed separately from whether it is new?

Value Chain Positioning

Where does the technology sit commercially, how does it capture value, and how dependent is it on a small number of suppliers or channels?

Public Disclosure Status

Has the invention already been disclosed publicly, and what does that do to the filing window in the United States and in jurisdictions with no grace period?

Validation Evidence Quality

How rigorous is the evidence behind the claims: sample sizes, controls, statistical treatment, and independent replication?

Risk & Barriers

What commercial, funding, supply, and adoption barriers stand in the way, and are the mitigations evidenced or only claimed?

Innovation Level

How far does this advance the state of the art, and is the advance a predictable next step or a genuine departure?

A higher score is always the better reading

Several of the twelve assess a difficulty rather than a strength. The scale still runs the same way. On Regulatory Complexity, a high score means the pathway is short and well established. On Risk & Barriers, a high score means the risks are minor or already mitigated. On every evaluation, read the score as favorability.

How to read a score

Scores run out of 100, and they are worth reading as a band rather than an exact number. A 62 and a 66 are the same kind of result; a 62 and a 26 are not. Roughly:

Band

Reading

0 to 20

Very weak

21 to 40

Weak

41 to 60

Moderate

61 to 80

Strong

81 to 100

Exceptional

Each evaluation has its own Score Guide, which you reach by selecting Evaluation Details on that evaluation. It spells out all five bands in the language of that particular question, so you can read what your score actually means there.

The Score Guide for Public Disclosure Status, listing all five bands, each with its own description. A marker points out which band the score falls into.

The Score Guide for Public Disclosure Status, listing all five bands, each with its own description. A marker points out which band the score falls into.

The same band means different things on different evaluations

A band tells you how favorable a result is, not what it is about. Two evaluations at 85 are both strong, and they are strong at different things: on IP Viability an 85 says the patent position looks defensible, while on Market Size & Revenue it says the market is large. Neither implies the other. Each evaluation shows its own Score Guide, so read the description on the one in front of you rather than carrying a number across.

Zero and no score are different things

A score of 0 is a finding. It means the evaluation ran, examined the evidence, and concluded there are fundamental problems. It counts toward the Overall Score like any other number.

An evaluation with no score did not complete. The Overall Score excludes it rather than counting it as zero, so it neither helps nor harms the average. The switcher at the top of the result page marks it, and the remaining evaluations score normally.

A 0 answers the question. A blank leaves the question open, so you still need an answer on it from somewhere else.

Trust score

The Trust score tells you how stable a number is. It is available on Premium assessments only. FirstIgnite scores each evaluation five separate times and compares the five results.

  • High means all five closely agreed. You can generally take the score at face value.

  • Medium means they mostly agreed with some spread. The score is usable, and worth a second look before you rely on it.

  • Low Trust means they disagreed significantly. Treat the score as unstable and verify it yourself before acting on it.

Check the Trust score on any evaluation whose number is going to change a decision. A Low Trust score means the evidence does not yet support a firm answer on that evaluation.

A Basic assessment shows no Trust score. The evaluations and their scores work the same way on both tiers, so read a Basic score exactly as described above, without the stability reading beside it. See Basic and Premium: what changes.

An evaluation showing a Trust score of Medium beside the evaluation score.

An evaluation showing a Trust score of Medium beside the evaluation score.

Why the same technology scores differently on a re-run

Evaluations research live sources and reason over what they find. Two runs of the same input can retrieve different evidence and reach different conclusions, so scores move between runs.

On a Premium assessment, the Trust score tells you how much movement to expect. An evaluation that scored High is unlikely to move far. One that scored Low Trust was already unstable within a single run, so a re-run can land somewhere quite different.

Treat a re-run as a second opinion on the same input. Where the two runs disagree sharply, that disagreement is worth investigating before you rely on either number.

Tips

  • Read the individual evaluations before the Overall Score. The average tells you the shape of the result, and the individual evaluations tell you what to act on.

  • On a Premium assessment, check the Trust score on any evaluation you are about to quote to someone else.

  • When you compare two assessments, check they were scored on the same set of evaluations first. A different template means the two Overall Scores are not directly comparable.

  • Read a difficulty evaluation carefully before reacting to a low number. A low score on Regulatory Complexity describes the approval pathway rather than the technology.

  • If an evaluation matters to you and your template does not include it, use a template that does. See Assessment templates: choosing, creating and editing.

Common issues

An evaluation shows no score. That evaluation did not complete. The switcher at the top of the result page marks it, and the Overall Score leaves it out entirely rather than counting it as zero. Nothing is wrong with the other scores.

Two assessments of the same technology show different evaluations. They used different templates. A template selects which of the twelve to run. See Assessment templates: choosing, creating and editing.

A score of 85 on one evaluation and 85 on another seem to describe different situations. They do. The band is the same, but each evaluation has its own rubric inside it. Read each evaluation's Score Guide.

A high score on a risk or complexity evaluation looks wrong. A higher score is the more favorable reading on every evaluation, including the ones that assess difficulty.

The Overall Score looks low for a technology you rate highly. One weak evaluation pulls a weighted average down by design. Check which evaluation is low and what it was asking.

A re-run produced a noticeably different score. On a Premium assessment, check the Trust score on the evaluation that moved. A Low Trust score moves more between runs than a High one. A Basic assessment shows no Trust score, so compare the two results side by side instead.

Frequently asked questions

What does a score actually mean?

FirstIgnite scores every evaluation out of 100, read in five bands: 0 to 20 very weak, 21 to 40 weak, 41 to 60 moderate, 61 to 80 strong, and 81 to 100 exceptional. What a band describes differs by evaluation, which is why each one has its own Score Guide, behind Evaluation Details.

Is a score of 0 the same as no score?

No. A 0 means the evaluation ran and found fundamental problems, and it counts toward the Overall Score. An evaluation with no score did not complete, and the Overall Score excludes it rather than counting it as a zero.

How many evaluations will my assessment include?

That depends on the template, not on your tier. Twelve exist and all twelve are available on Basic and Premium. See Assessment templates: choosing, creating and editing.

Why are the scores kept separate instead of combined into one number?

Because a weak result on one question is usually the thing you need to act on, and an average hides it. Separate scores can show a technology as commercially attractive and legally exposed at the same time.

Can I add an evaluation to an assessment that has already run?

No. The set of evaluations is fixed when the assessment runs. Run a new assessment against a template that includes the evaluation you want. See Assessment templates: choosing, creating and editing.

Why does the same technology score differently when I run it again?

Evaluations research live sources and reason over what they find, so results move between runs. On a Premium assessment, the Trust score on each evaluation tells you how much movement to expect.

Does a Low Trust score mean the assessment failed?

No. It means five independent scoring runs disagreed on that evaluation. The assessment still reports the score. Verify it yourself before you rely on it.

Why does my assessment not show a Trust score?

The Trust score is a Premium feature. See Basic and Premium: what changes.

Where does the research behind an evaluation come from?

Did this answer your question?