Thinking machines come up a lot in fiction, and lots of people expect them to be realized some time in the future (any moment now, according to Ray Kurzweil and his ilk).

A corollary of the thinking machine is the simulated or uploaded human, produced by scanning a human being at whatever is the required level of detail and then computing a simulated world in which the scanned representation is evolved in accordance with the laws of physics. The content of the simulation could be interfaced with the outside world in some way, or an entirely self-contained virtual reality.

A materialist view of the mind seems to suggest that the simulation will have a consciousness (whatever that is), and if the scan is a good scan and the model is very precise it will be a lot like the consciousness of the original person. So far so uncontroversial. Imagine the second kind of simulation, one which is self-contained. The simulated person can’t be aware of the real passage of time; their conscious experience would be the same no matter how frequently the simulation was updated. Their experience occurs only during simulated time, which is not connected to real time.

In fact none of the underlying details of the computation are visible to the simulated person - if the simulator works correctly, all their experiments in VR should produce indistinguishable results from similar experiments in plain old R. For a lot of singularitarians and AI people, this still sounds quite reasonable.

If the thought experiment holds up so far, we can consider the next step: computing the states of the simulation in the wrong order. Remember that the subject has no perception of the passage of real time, only simulated time, which is only really marked by consistency within a single frame of the computation. Consequently, the order in which the states are computed can’t affect the conscious experience of the subject; nor can the computation of other arbitrary states in between.

At this point we have arrived at an odd conclusion: consider the following simple program

while (true) : increment counter

Given a large memory for holding the counter, this program will eventually cycle through all available memory states; if the memory were big enough to store a hypothetical simulation of the sort mentioned, all of the possible conscious experiences in that simulation must have occurred during the execution of the loop.

Note that we can also parallelize the implementation of this program very well, by using 2^n machines to “compute” all the memory states simultaneously. The fact that this is happening on different machines concurrently remains invisible to the simulatee!

In fact, we can add one additional layer of computation to produce an even odder conclusion; if we design our machine with nbit to have a memory controller which applies some permutation to 2^nbefore storing the state into the memory, each possible memory state will correspond to a different part of the computation of the conscious experience; however this effect will still be invisible to the simulated person (remember, they cannot see the simulator’s implementation details).

Allowing for this, it seems like a single large memory register initialized to zero also immediately contains all the possible conscious experiences of a simulation which can fit in that register; after all, the simulated persons involved cannot know which machine they are running on - the one machine is effectively simulating all of the different permuted machines at once!

I’m not sure what the logical consequence of this idea are; it seems like one of:

1. The argument has a bug in it.
2. Consciousness is not contained within a person, but somehow in an interaction between persons (i.e. the closed VR contains no consciousness, but a simulator talking to a real person does contain consciousness). This seems to lack neatness, and doesn’t fit the experience of being conscious.
3. Simulations cannot be conscious, for some reason which is unclear.
4. The argument is correct, and everything is conscious in all possible ways concurrently, everywhere and forever, but all these permuted worlds are closed off to us.