Stock and orders
Allocate limited stock and resolve competing orders using the supplied rules.
Aprex tests AI on Norwegian work tasks with a fixed answer key. Explore the results and see how we check them.
Results updated
Advanced calibration 0.1 · 12 synthetic Bokmål tasks · one repetition per model. Eight HIGH routes (96 responses from two runs), plus a separate Qwen xhigh test.
Correct fields: 290/292. diagnostics · no partial credit.
anthropic/claude-haiku-5-5advanced-v01-2026-10-10-haiku · 2026-10-105f55d82430baa317bbd8dff30829b2d799f75b6be3f59476abe8455fd4e519fb3dea247270af42f7f0dddfaac4f7b861cd6309eb0a084315b28d5438ad2a2a3daf406048c8138dc0ce5c4bd763547929327e7b6e48f5b0998db0aef3517533240a4a93946c71ef58facb8b24599b94d7ee9468fdCorrect fields: 291/292. diagnostics · no partial credit.
anthropic/claude-opus-5-5advanced-v01-2026-10-10-opus · 2026-10-105f55d82430baa317bbd8dff30829b2d799f75b6b722c15b7dd27939c5b298d6efa6c7f49eb9a612ef70ab1fe3e726e36b7994678a2f2e872ee49b14d57e1c11b3354dbbd5e3f81271580302178fb2921c34e4fc716f082c0966955b427aad0d930c8c9b6e668129aa5c5d616a15354d183a30e6eCorrect fields: 288/292. diagnostics · no partial credit.
opencode-go/deepseek-v4.1-flashadvanced-v01-2026-10-10-deepseek · 2026-10-105f55d82430baa317bbd8dff30829b2d799f75b6b90b1a48c6548e487b81a1c802ad7c1ab6ed0d206edb8f0e58326b1f1f7a021ee7b079382e576759867a991e77bfc97c9e78e6be58b11553481d7be59027a4cae19c2f610da3d9b3c9b79b5e6cc46884401f8be4a347d230154ca3384ac142306Correct fields: 292/292. diagnostics · no partial credit.
openai/gpt-6-astraadvanced-v01-high-extension-2026-10-10-astra · 2026-10-10a230b80a22b962b1f69f39170b22950db657986dfccb8e875ae7fbc0546bd740eddc9c27a4a830bdb11414baed5964c34e1a311aa6e1ea40acd26fe488d294a8aaf9385bf20b4b70ad8a6cc0daaa3c4bfdefa76b50b516678c5c3db2397f923ade83ea3674956282fb6934b4751432fd0581fbadCorrect fields: 270/292. diagnostics · no partial credit.
codex/gpt-6-lunaadvanced-v01-2026-10-10-luna · 2026-10-105f55d82430baa317bbd8dff30829b2d799f75b6b544706915eea14f643d4bb929df1b47748519423db5f3c4ac5243401ff5a217471e4e55a203776ffd8ce4d0d2708d6a3e2cfd21958217bcf7e8ff31fae7baa9f078d9d0f541daad196d654abed9124978f9618340c34aadac5eeb3424807b87fCorrect fields: 292/292. diagnostics · no partial credit.
openai/gpt-6.1-soladvanced-v01-high-extension-2026-10-10-sol · 2026-10-10a230b80a22b962b1f69f39170b22950db657986d9ffb1f7958ea5f323cb0e4c340f6ccb3d3012a4d5230ab4cc36a0dcb9dee233bc555ade670f7d768f32e1ab7835411b329d6e61690bb3ed7649f962bb97b0a062e96682bb57236d544b6ed2027dfbc0d3399c39383f41bbf87aed20b185a5d98Correct fields: 292/292. diagnostics · no partial credit.
google/gemini-3.8-flashadvanced-v01-high-extension-2026-10-10-gemini · 2026-10-10a230b80a22b962b1f69f39170b22950db657986dbc4762926981f3674f92d986415fef29c50d1f4b550a0e2c48f3efe929f413a8172889dcbcb3d48f53b1c8f90d9357bcd03d2ffc2f64c604375880822b5d4004d31d7434cf37fc33733c85fa7ea62ffabd6a9214a6088b457327287717515a59Correct fields: 289/292. diagnostics · no partial credit.
meta-muse/muse-spark-1.3-contributoradvanced-v01-high-extension-2026-10-10-muse · 2026-10-10a230b80a22b962b1f69f39170b22950db657986d199d5ef3e6d9e75a45e6855d3dc090f390771dc905f61d7227da6a90dce56e331473ea0c803278eddbff91a53015d0eb6966b8fa36ecd514ad9574747aff456bd75dc632ce7a7aeab8ddc6da543b5390c1bfd4b6e3dba0074b0ff1af7e4b3281| Model | Stock and orders4 tasks · 80 fields | Invoice reconciliation4 tasks · 104 fields | Case flow and deadlines4 tasks · 108 fields |
|---|---|---|---|
| Claude Haiku 5.5 | 4/480/80 fields | 4/4104/104 fields | 3/4106/108 fields |
| Claude Opus 5.5 | 4/480/80 fields | 3/4103/104 fields | 4/4108/108 fields |
| DeepSeek V4.1 Flash | 4/480/80 fields | 4/4104/104 fields | 1/4104/108 fields |
| GPT-6 Astra | 4/480/80 fields | 4/4104/104 fields | 4/4108/108 fields |
| GPT-6 Luna | 3/479/80 fields | 1/496/104 fields | 0/495/108 fields |
| GPT-6.1 Sol | 4/480/80 fields | 4/4104/104 fields | 4/4108/108 fields |
| Gemini 3.8 Flash | 4/480/80 fields | 4/4104/104 fields | 4/4108/108 fields |
| Muse Spark 1.3 | 4/480/80 fields | 2/4101/104 fields | 4/4108/108 fields |
Separate xhigh run (excluded from HIGH numbers)
Correct fields: 292/292. diagnostics · no partial credit.
opencode-go/qwen3.8-maxqwen-xhigh-isolated-2026-10-10 · 2026-10-10a230b80a22b962b1f69f39170b22950db657986d75d73dca32bf2f5c9f4c94f4100e6a2787a5ab43fd67218d8dab3242f05a0ea435f4c3da16476b15e1f4e73ab5407fea6d72175f43a3e71e7663742d64eb311a4e2c0130fdf98dcc45858540f97bfa6e053b544b03df9f335b163e0117167d807a16849f6dff0cf0ea1af7ec7029b8919e34bdaf137492d060a6b56e8f663c545132dfd81cbb · scorer 4f26f9806b58 · audit ca60602200a7Aprex AS, Norwegian AI Barometer, advanced calibration 0.1, current audited selection October 10, 2026 (advanced-v01-audited-selection-2026-10-10). Link to this page and state the CSV/JSON date, the selection and the corrections.
Four tasks in each category. All documents and rules are fictional. The test measures precise answers, not free writing or legal expertise.
Allocate limited stock and resolve competing orders using the supplied rules.
Match invoices, payments and adjustments across several documents.
Apply exceptions and calculate deadlines from a fictional case history.
The published numbers are a calibration. The first full evaluation on reserved tasks is still ahead.
The program compares the response with a key computed from synthetic data. There is no AI judge.
A task is correct only when every required field is correct. Field scores help explain errors and give no partial credit.
The prompt, response, model setting and tool history are checked. We publish the tool and reasoning level with each result.
Read the full method: scoring, uncertainty and data format →
Businesses are adopting AI fast, but have little to go on when choosing a model for Norwegian tasks.
The well-known language model tests mostly measure English, coding and maths. They say little about a Norwegian invoice, a customer reply or minutes in Nynorsk.
The choice is often made on gut feeling or marketing. Numbers on Norwegian work tasks give a better basis, including for cost.
Aprex tests models when we build AI solutions. Now we do it systematically and publish the numbers with the method. The latest HIGH selection from October 10 is published with Gemini logs verified; earlier numbers are preserved as history. The first approved full run is pending.
Next phase: planned task types, models and public examples →
Previous reports stay available. Different test versions must not be read as a performance trend.
Earlier report versions are preserved.
Start with the task, not the model. Look at the score for the kind of work you will do, not just the total, and compare it with the cost. For simple tasks like sorting and extraction, a cheaper model may be good enough. Always test on a few of your own examples before deciding.
Yes, but settle a few things first: a data processing agreement with the provider, where the data is processed, whether the provider may train models on it, and that you send only what is needed. The GDPR also applies when an AI model processes the data. Read more →
It depends on the provider, product and agreement. Some offer processing in the EU or EEA, others do not. Check where data is stored and processed, and get it written into the agreement. Read more →
The barometer measures controlled structured tasks, not general fluency. The Nynorsk type has only controlled word-form and rule tasks. Check the scores once the first approved full run is published — the calibration above is only a development signal on 12 Bokmål tasks — and test on your own texts.
Through an API you pay per use, and prices vary a lot between models. Cost per 100 tasks will show the measured API price for tasks like those in the test. The subscription pilot reports no API cost; chat service licences have a fixed price per user. API and subscription are separate tracks and are never compared directly. The published calibration contains no cost or response-time measurements. Building and running the solution itself comes on top. Read more →
For employees who write, summarise and search, it often goes a long way. When a task must be done the same way many times, pull data from your own systems or run without someone chatting, you need an integration. Then the model behind it matters. The first pilot uses Codex and Claude Code with subscriptions. APIs and browser experiences are separate tracks; each measurement identifies product, model and version. Read more →
The EU AI Act sets requirements by risk. Common uses such as customer replies, summaries and document extraction are usually not high risk, but there are transparency duties, for example that customers should know when they are talking to an AI. Read more →
Not without checks. The barometer regulation type uses only supplied fictional rules, does not test legal expertise and is not legal advice. Use AI to find and summarise, and let a person check against the source before the answer is used.
Pick one workflow that takes a lot of time and has clear rules. Collect ten to twenty real examples, test two or three models on them, and measure time and errors before and after. It is what we do in the barometer, only on your tasks. Read more →
A barometer under development by Aprex AS (0.5-draft) for tightly scoped Norwegian work tasks with verifiable answer keys in Bokmål and Nynorsk: 48 tasks (12 public/12 calibration/24 holdout). Difficulty levels are author-graded, not empirically calibrated. The first approved full run is not published. The latest HIGH selection from October 10 (12 synthetic Bokmål tasks, 8 routes, 96 responses, plus separate Qwen xhigh with 12 responses) is published as a separate development signal. Full results will show scores per split with holdout-24 as the primary comparison, API cost and response time per model.
12 models from 8 providers are planned: GPT-6 Astra, Claude Opus 5.5, Gemini 3.1 Pro, Grok 4.7, Mistral Medium 3.5, DeepSeek V4 Pro, Qwen 3.8 Max, Muse Spark 1.3 as top models, and GPT-6 Luna, Claude Sonnet 5, Gemini 3.8 Flash, Mistral Large 3 as budget models. The list is not fixed; reference candidates are configured separately. The catalog has no results; the latest selection covers only the 8 actually run HIGH routes. Changes to the list are recorded in the changelog.
The target is every week. No approved full runs have been published yet; advanced calibration 0.1 is published as a separate development signal. Full results are published only once a run has passed the automatic checks and Aprex has reviewed the outliers.
Each task scores 100 or 0. The answer must be JSON with exactly the required fields. Every field must match after normalisation (dates, amounts, digits, categories, booleans). Null is valid only where the key explicitly allows it. There is no partial credit in the v0.5 ranking; share of correct fields is diagnostics only. Missing answers are listed separately without 0. No AI judge. The calibration uses the same 100/0 rule.
The 12 example tasks on the method page are public and may be known to a model. Calibration and holdout are not public: they are not shown on the site and are not in the website source or metadata. The repo is private without immunity to training, and a runner sends its assigned task text to the provider on each run. Holdout is reserved final evaluation. Results from public tasks and calibration are never combined into a claimed measurement on unseen tasks. The 12 advanced calibration tasks have been used for development and are no untouched final test; the 24 holdout tasks remain untouched.
Because the best model is not always the best choice. For many tasks a cheaper, faster model may be good enough. Cost is the measured API price from the provider and is never mixed into the score. The subscription pilot reports no API cost. API and subscription are separate tracks. The calibration reports no cost or response time.
Yes. The calibration numbers can be used now. Credit Aprex and link to this page. Every run gets a CSV file with all the numbers.
The latest selection covers 12 synthetic Bokmål tasks and 8 HIGH routes from two runs with different harnesses, plus separate Qwen xhigh with 12 responses, presented alphabetically with no ranking and no claim of general superiority. Full results will apply to these 48 controlled tasks and to the product and model versions available in the week they were measured. Difficulty levels are author-graded, not empirically calibrated. Free writing, tone and fluency are not measured. Aprex itself uses models from several of the providers. The providers have no say in the method and do not see results before they are published.
The barometer shows an average on controlled tasks. Tell us what you want to use AI for, and we will help you choose and test the right model on your own tasks.
Tell us what you want to test →