Run the first pass yourself.

How to scope an AI system

Most failed AI projects were never scoped as systems — they were scoped as tools. This guide is the same first pass we run in discovery, written so you can run it yourself.

1. 01

   Name the workflow, not the technology

   Write down the process you want to change in one sentence: what enters, what leaves, who is accountable for the output. If you cannot write it without the word AI, the scope is not ready.

2. 02

   List every input

   Documents, emails, forms, models, drawings, database records, phone calls. Note the format and where it physically lives. Inputs define the engineering more than outputs do.

3. 03

   List every system already involved

   CRM, ERP, project systems, ticketing, file stores, spreadsheets. The system you build will have to live alongside these, not replace them on day one.

4. 04

   Mark the decisions

   For each step, ask: is this a rule, a judgement, or a rule with exceptions? Rules automate cleanly; judgement needs approval design; exceptions need escalation paths.

5. 05

   Write the acceptance test before the architecture

   Define what a correct output looks like on ten real cases. If you cannot define correctness, evaluation will be impossible and the project will drift.

6. 06

   Decide what stays human — explicitly

   Approvals, exceptions, customer-facing commitments and anything legally sensitive. Writing this down prevents the two classic failure modes: over-automation and an unused system.

Output: a one-page workflow map with inputs, outputs, systems, decisions, approvals and exceptions. That page is the specification every later design decision answers to.

Ready-made, productized or custom?

Arvanord offers three ways to buy a system. The honest version of how to choose between them, including when the answer is 'start with something smaller'.

1. 01

   Choose Ready-Made when the workflow is standard

   Meeting intelligence, inbox triage, document extraction, recurring reporting. These follow patterns we have already engineered; configuration and deployment take weeks, not quarters.

2. 02

   Choose Productized when you need a department-level layer

   Company AI OS, Sales Engine, Document Engine, Operations Engine, Support Engine. These have established architectures that are adapted to your systems, permissions and workflows.

3. 03

   Choose Custom when the work is genuinely unusual

   Proprietary engineering workflows, uncommon software, domain-specific rules, multi-system orchestration. This is where our systems engineering work concentrates — and where a template would only get in the way.

4. 04

   Do not confuse scope with ambition

   A large ambition usually starts with one constrained workflow. The first system should be the one where correctness is checkable and value is visible within weeks.

5. 05

   Budget for operation, not only construction

   Systems need monitoring, model updates, rule changes and knowledge maintenance. Arvanord Ops exists for this; plan for it from the start.

Rule of thumb: if you can describe the workflow as a pattern we list publicly, start ready-made or productized. If the description keeps breaking the template, that is the signal for custom engineering.

Data and document readiness

Before an engineering engagement starts, a surprising share of the work is simply finding out what exists. This checklist prevents the most common delay in AI system projects.

1. 01

   Pick representative documents, not the perfect ones

   For document work: 20–50 real examples including the messy ones — scans, signed copies, unusual layouts, older revisions. Clean samples produce clean demos and unreliable systems.

2. 02

   Establish where truth lives

   When documents and systems disagree, which one wins? Document the answer per data type. Systems engineering follows that rule; guessing it produces silent errors.

3. 03

   Inventory access, not just existence

   Who can read what? Which teams may see which data? Permission design happens at architecture time, so the constraints must be known before it.

4. 04

   Note the revision culture

   Are documents versioned, reviewed, marked up, re-issued? Revision workflow determines how extracted data stays trustworthy over time.

5. 05

   Decide the retention line

   What may be processed, stored, and for how long. This shapes storage architecture and the audit trail your system will need.

Output: a document and data inventory with ownership, access, revision practice and retention. Engineering starts from this inventory, not from discovery on the customer's time.

Evaluation design for AI systems

The difference between experimenting with AI and engineering with it is evaluation. If output quality cannot be measured, the system cannot be trusted in production.

1. 01

   Define desired behaviour in observable terms

   Not 'good summaries' — 'summaries list decisions, owners and dates; never invent names; flag unresolved questions'. Observable criteria can be tested.

2. 02

   Build a test set from real work

   Fifty to a few hundred real cases with expected outputs, including edge cases and known-hard examples. This set is an asset; keep and grow it.

3. 03

   Run, measure, find failure modes

   Score against the criteria, categorise mistakes with humans in the loop, and look for systematic patterns rather than isolated errors.

4. 04

   Improve the system, not the prompt alone

   Often the fix belongs in retrieval, validation, permissions or the workflow — not in wording. Evaluation tells you where the real fault is.

5. 05

   Monitor after deployment

   Distributions shift and business rules change. Production systems track output quality, exceptions and approval rates, and treat drift as a signal.

Output: an evaluation plan with criteria, datasets and a reporting rhythm. It becomes part of the system's operating documentation and part of Arvanord Ops.