If I were to ask you what space is, how would you respond? What do you think it is? We all seem to intuitively understand what it is, but what sort of magic tricks go on in our brains which give the impression that we’re immersed within it?
Before we get started, let’s first begin with a rather inspirational video. If you understand what space is, you can build machines which can navigate their way through it and wonderful things happen.
One thing I find puzzling about this sort of research is how there are so many different approaches to building machines which can navigate through space. For example, you can use lasers and radar type techniques, measuring the timing of reflected signal waves to build a 3D map of the world. That’s what’s going on with that spinning “can” at the top of the Google car. Or you can extract depth from a single colored image alone, analyzing how the colors in the image change. Or you can extract 3D spatial structure from changes in related images, such as in a film reel, where objects further in the distance move more slowly than those up close as the camera moves. You can even take two separate images from two nearby cameras, each with different focus settings, and extract the depth of the scene from that alone. There’s so many different approaches.
You can be conscious of space from your sense of touch, feeling objects and being conscious of the signals feeding in from your entire body. Or you can get a sense of space from sounds coming into your ears, the sound wave reaching one ear before the other. Your brain automatically can take the slight delay into account, and you can get a sense of space and where the sound is coming from.
All of these methods, and more, can extract the spatial structure of the world, but what do they all have in common? What property, quantity, or process are they all measuring? Whatever that is, that’s what I think space is. It’s a sort of information processing dynamic. A “closed” information flow within the observer. In our case, our brains. There’s different ways to create that information flow, but that flow itself is what space and time are. The real world has a casual structure to produce space (photons and light waves, physical matter, and so forth), but our simulations have different ways of doing it (polygons, textures, rendering algorithms, etc). There’s not just one way to get to “space”. Multiple approaches will get you there. That’s why I think space within a computer simulation is just as real as “real” space. Our simulations are now capable of making people feel immersed within space. As the digital world evolves and grows, eventually we’ll be immersing ourselves in sophisticated virtual realities, and I think those too can be considered true parallel universes.
But these days I’m rather obsessed with figuring out what the “real world’s” space is. I want to understand that common property between all the different methods we know of which can extract depth information. The past six months I’ve been studying mathematics and physics primarily, studying dynamics such as how light scatters from objects, how photons travel through space carrying spatial information, how space contracts in the direction of motion (Lorentz contraction), and am working tirelessly to unite the conception of space used by physicists with the type of space used in fields such as computer vision and cognitive neuroscience.
In some methods, in order to extract “space” from a sensory impression I need time as well, such as when I extract structure from motion. I need multiple sensory impressions and it’s a relation between them, a sort of complex information processing task, making certain heuristic assumptions about reality and assuming that reality has a constant structure which will remain throughout time. Other methods, such as extracting spatial depth from shading, seem to have no need of time or change. That alone baffles me.
The problem would be simpler if each method always worked, but that’s not the case. Different methods have different degrees of reliability. For example, I believe Ernst Mach concluded that the shape of an object is impossible to determine from the colors in an image alone because the laws governing light scattering from objects are too unpredictable. Computer vision scientists struggle with methods based on this approach because it relies on what’s called Lambertian scattering, which is an assumption that light scatters uniformly in all directions. In other words, nothing is shiny. Those algorithms fail to work when an object reflects light differently at different angles. Depending on the situation and your approach, you may or may not be able to extract spatial information.
I have all these thought experiments running through my head and I’m constantly working to learn more mathematics and physics so I can get closer to solving them. Of course there’s huge overlaps. It’s not like the computer vision algorithms use a different mathematics and physics. It’s just they’re so different in their approach, I consider them totally different subjects. All the physics I’ve done always assumes space exists and doesn’t bother trying to establish how we intuitively understand space. I calculate a trajectory, but in physics I don’t worry about how my brain processes the images of its flight and knows its moving and that time is progressing.
As I work on my philosophical conundrums of space, I may make some progress on some experiments, others I’ve had them on the back-burner for years. Trying to master all these different “subjective” spatial techniques (as I call them) from computer vision are quite involved and it’s a rapidly growing and changing science. The sort of space found in mathematics and physics changes less quickly, but it’s incredibly difficult to understand. I’ve been working on a notebook of thought experiments I want to solve eventually, most of them rooted in my imperfect understanding of relativity and quantum mechanics.
For example, I was never satisfied with Maxwell’s equations when I first studied them. I used to imagine two electromagnetic plane waves traveling side by side in the exact same direction, with the exact same polarity and phase, and I wondered if you could bring them infinitesimally close together without them interacting. Then I thought, “No, these equations must be an approximation. Surely they’d interact with each other at some point, even if not ‘perfectly’ touching.” I guessed that it was probably due to quantum physics and the Planck length or something. That’s one little thought experiment in my book, which I’ve largely solved.
Another example I’m thinking about. Space contracts in the direction of motion, so say I was to fly high in the sky over my hometown. Say I’m actually up in space, above my hometown, to make it a little simpler. Consider what would happen if I rapidly sped up to near light speed, 99.9999999999999….99999% light speed, and the Earth in front of me contracts to some infinitesimally thin plate. I only stay at this high speed for a moment, a near infinitesimal moment, and then I just as rapidly slow back down. The Earth in front of me “compresses” into the thin plate and then “decompresses” back into the Earth. Here’s the problem. Is it the same Earth? It’s similar to the problems of a singularity. The atoms would be compressed into a smaller and smaller space, and the plate which is the Earth is a sort of 2D singularity plate. I’m thinking weird effects of quantum physics must be taken into account and I’ve been wondering whether or not the Earth would decompress back into its original form, or would it be akin to transporting myself into a parallel universe?
Oftentimes they’re wild speculations, but I wonder about those sorts of things. It takes a lot of time and effort to solve all these experiments. My process generally goes like this. I study a bunch of things, think about the material over and over, make thought experiments in my notebooks, and then use them to direct my further inquiries and studies. Oftentimes I make no progress at all, and overall, the more complicated the problem, the longer it takes me to solve, if ever. They’ll sit in my notebooks for years before I get back to them, but I can’t get them out of my head. I hate aging. I want to continue working on these problems and solve them. I live an insignificant amount of time in a universe too complicated for my feeble brain to work out in the time it’s given to live. One of my professors once told me, “Jason, you’re not immortal.” I sort of stared at the floor and thought, “Yeah, unfortunately not.”