This text discusses how decision-making cna be divided to ”analysis” and ”value choice” phases and what the spread of AI might mean for the future of societal decision-making. Unpacking this division can, in turn, help us understand what AI — or politicians — are actually good for. And perhaps, taken further, how we should build governance structures and organizations amid the turbulence of AI and an increasingly complex world.
The most important decisions are multi-objective, and demand value choices
Let’s start from the fact that the most significant decision-making situations we face are multi-objective. This means the decision-maker doesn’t base the decision on a single goal but has to weigh the matter from several angles. A familiar everyday example is the quality-versus-price trade-off, or considering the size and location of a new apartment. Multi-objectivity can also be more subtle. Even if are the firmest believer in the notion that a company’s sole purpose is to maximize its profit, you cannot escape multiple objectives — this time along the dimension of time. Would you pursue quick wins or longer-term results? Even though NPV is just a single number, the discount rate used to calculate it is, in practice, a value choice about how to weigh different time periods. Change the discounting, and different solutions can suddenly look more favorable than under the original rate. You can, of course, derive the discount rate from something like the interest rate and thereby ”outsource” the value choice. Nothing actually stops you, though, from using some other rate based on your own weighting instead.
Pareto frontiers as a decision-making tool, and as a way of illustrating the division of labor
A key tool for handling multi-objectivity is the Pareto frontier. It lets us map out how different solutions affect several objectives at once, and the trade-offs between those objectives. Take an example. The accompanying chart shows a Pareto frontier for a hypothetical situation: deciding on the structure of a hospital network, with the simultaneous goals of pursuing both the best possible accessibility and the lowest possible cost level. The objectives are clearly in tension: accessibility improves by building more hospitals, but costs rise as a result — and vice versa.
In the chart, the blue curve represents the Pareto frontier: as the cost level rises (x-axis), accessibility (y-axis) improves, but the marginal benefit shrinks step by step — at low cost levels, additional investment brings a large improvement in accessibility, whereas at high cost levels, further improvements require proportionally more money. The grey squares represent inefficient solutions that fall below the frontier — for each of them, there’s always some point on the frontier that is both cheaper and more accessible. (Chart and caption generated with AI.)
If we could know the exact consequences of decisions in advance, a well-chosen solution would be part of the Pareto frontier, since only there are solutions ”efficient” — not needlessly leaving something achievable on the table. On the other hand, there are often numerous efficient solutions, and a choice still has to be made somehow, by weighing which objective matters more, and how much, in a given case. This is a value choice in its most concrete form. Which values we choose to emphasize, and what kinds of trade-offs we’re willing to make between them, determines which solution is ”most optimal” — that is, which outcome is best relative to a given set of values. This can’t be settled rationally, because no logic can answer the question ”what do I value most?”
AI and value choices — how do we steer the future?
But in order to make value choices — that is, to pick the winning solution from the Pareto surface — that surface first has to be created. And here, rational, logical, even mathematical thinking plays a central role. Without this stage, ”making value choices” is just a coin toss, wishful thinking, or populism.
This division — between constructing the Pareto frontier and using it to make value choices — also tells us something about which kinds of actors each stage belongs to. Take societal decision-making as an example: ideally, the job of the civil-service and research apparatus is to produce the range of solution options that ultimately make up the frontier, while the job of politicians is to choose where on that frontier — that is, which solution — to pick. In the workplace, making value choices on behalf of a company is the job of leadership (everyone is responsible for their own value choices individually). Analysts, engineers, researchers, and others are responsible for producing up-to-date information about the available options.
Today, we expect a great deal of help from AI in decision-making. Would you let AI make value choices? If you answered yes, you’ve given up on your own humanity and nothing remains except producing added value for shareholders and the tax authorities. If free will is real, value choices are precisely where we exercise it. If it isn’t, and we are merely sentient collections of cosmic dust, value choices are the point where individuality emerges and you ”act out your nature.”
So I adopt this maxim: ”AI must not cross the Pareto frontier.” Or in plain terms: ”AI must not make value choices.”
By contrast, using AI to construct the Pareto frontier is justified wherever it can be trusted and its route to its conclusions can be understood. Analyzing complex systems is demanding work and requires very complex computation — to the point that the human mind can genuinely benefit from capable tools. What matters is understanding exactly where analysis turns into value-choice-making, so that we don’t accidentally outsource our humanity. At the same time, it’s worth noting that even constructing the Pareto frontier itself involves value choices — above all, which objectives are considered in the analysis in the first place. In the example described above, for instance, if accessibility were left out of consideration, the obvious choice would simply be whichever option has the lowest cost.
Let’s close with a thought experiment — what might this mean, taken quite far, for how decisions come about? Imagine a world in which AI can give high-quality answers to very complex societal (or other) questions.
First, the number of analyzed decision options would likely grow in complex decision-making situations — there would be more alternatives with a fully worked-through estimate of their consequences. The quality of the analysis could also improve. This is simultaneously a good thing and a challenge. It’s certainly good that decisions could be made on better information and based on more analyzed alternatives. At the same time, though, understanding and internalizing more complex analysis and more numerous, possibly more divergent alternatives becomes more demanding.
At the same time, one might hope that shooting down flimsy justifications would become easier. Politicians’ talk of ”dynamic effects” might become easier to pin down concretely, and thus to assess whether there’s any substance behind the claims, or if they’re just word thrown around for justifying intuitive choices.
Making value choices would likely also become more challenging, since the number of alternatives grows and weighing the differences between them becomes harder. The focus of the human role thus shifts substantially — away from carrying out the analysis and toward directing it (what gets studied, how it’s studied), understanding the results, and doing the work of value choice. And doing these tasks carefully isn’t just ”the last job left for humans” — it is also absolutely critical if we are to retain the ability to steer our own future.
The conclusion is that AI can probably improve the quality of societal decision-making, but it won’t, on its own, resolve the major societal questions that are, at bottom, value choices (e.g., income distribution, responsibilities, rights, obligations, and so on). That’s still going to be our work as humans.