The past few days I’ve been either studying physics, or researching material on space and time, trying to think up an outline for my “Space-Time and the Mind” section for the site. Some of you may recall that one project I wanted to work on over the summer involved building a model of a 3D environment based on changing 2D images from a camera. When I finally get around to doing this, I plan to use OpenCV (Open Computer Vision) and the PCL (point cloud library). The more I learn about the work they’re doing at Willow Garage, the more I’m impressed. It turns out that they’re working directly on the exact same project I’ve been thinking of for years. They have two short videos explaining it, so take a look for yourself.
In this next video, they show how they’re storing the environment in an Oct-tree, which is a high-speed 3D graphics rendering technique for storing the polygons of the environment. It’s pure awesomeness.
I’m still blown away that they’ve managed to determine the 3D structure of an environment from the 2D images. You can see the robot looking around, with two cameras arranged in stereo like our eyes, and it builds a “point cloud” of the environment. Apparently, from what I can tell, their intelligence algorithms are sophisticated enough to distinguish the separate objects one from the other, and even has a separate storage area for high resolution versions of individual models. For all you folks at Willow Garage, I applaud you. That’s some amazing technology.
By accomplishing this great feat, scientists such as those working at Willow Garage, have reverse engineered one of the core processes of human thinking. To quote David Hume,
“All the objects of human reason or enquiry may naturally be divided into two kinds, to wit, Relations of Ideas, and Matters of Fact. Of the first kind are the sciences of Geometry, Algebra, and Arithmetic; and in short, every affirmation which is either intuitively or demonstratively certain. That the square of the hypotenuse is equal to the square of the two sides, is a proposition which expresses a relation between these figures. That three times five is equal to the half of thirty, expresses a relation between these numbers. Propositions of this kind are discoverable by the mere operation of thought, without dependence on what is anywhere existent in the universe. Though there never were a circle or triangle in nature, the truths demonstrated by Euclid would for ever retain their certainty and evidence.”
– David Human, An Inquiry Concerning Human Understanding
Many years ago I remember reading that text, and I sat in my room wondering what “thought” was. How was my brain coming to this conception of what we call “Geometry”. Why did the axioms of Euclid seem so intuitively certain? To quote Immanuel Kant,
Space is not an empirical concept which has been derived from outward experiences. For in order that certain sensations may relate to something outside me (that is, to something which occupies a different part of space from that in which I am); in like manner, in order that I may represent them not merely as outside of and next to each other, but also in separate places, the representation of space must already exist as a foundation. Consequently, the representation of space cannot be borrowed from the relations of external phenomena through experience; but, on the contrary, this external experience is itself only possible through the said antecedent representation.
– Immanuel Kant, Critique of Pure Reason
And another quote,
Space then is a necessary representation a priori, which serves for the foundation of all external intuitions. We never can imagine or make a representation to ourselves of the non-existence of space, though we may easily enough think that no objects are found in it. It must, therefore, be considered as the condition of the possibility of appearances, and by no means as a determination dependent on them, and is a representation a priori, which necessarily supplies the basis for external appearances.
– Immanuel Kant, Critique of Pure Reason
What is the a priori process by which we come to a sense of space? It seems to me that our brain performs an information processing algorithm of a similar nature to the computer algorithms these robots are using. This is truly amazing. I had always thought such questions couldn’t be answered. It was a complete mystery to me what the foundations of geometry were. But now, for the first time in human history, we have algorithms which can take 2D images, just like our eyes do, and produce a representation of “space”. Ladies and gentlemen, there you go.
But wait, our sense of space also comes from the feelings our body generates. Our arms, legs, fingers, toes, and so on, all send signals up to our brain, giving feedback and a sense of existing in the world. So what about proprioception? Have we reverse engineered that too? Yes, we have! How about we show another amazing robot? Meet Rollin’ Justin.
There’s one big difference between us and these robots — our human brains also generate consciousness while performing these operations. They’re performing the same sorts of computations but they also, in a sense I don’t understand yet, give rise to a conscious experience. The brain lets us experience the world outside of us, and each other. We’re not the ones performing these feats because when I study neuroscience texts, and read case studies of patients with brain damage in the these “space” areas, they can longer properly conceive and navigate space. Without this “a priori” module to perform the calculations and generate the conscious feeling of being in space, they no longer know what space is. I believe Kant was right, at least about subjective space. The brain is, in many senses, a computer, decoding away at the sense data coming into it. Nobody teaches a young child to understand space — it just does. As its brain develops, assuming normal healthy development, once those neurons form into the correct network, space comprehension is possible.
However, that is not the entire picture. Einstein showed us that this a priori conception of space is inadequate and that it is only an approximation to what true space and time are. His theory of general relativity tells us that space and time are both dependent on the distribution of mass, because this warps a rather strange concept physicists call “space-time”. I’ll talk more about that at another time. But what’s nice about the physicist’s conception of space and time is they detach it from its abstract metaphysical hinges. Space and time are mystical to many philosophers, incapable of being defined. But by showing the logic operations of a computer-brain, and reproducing these things in robots, and then showing the laws of physics governing all of this, everything ties together into a grand unity. Not only does physics describe things in space and time, but space and time itself as well. Now a new question remains. Physics depends on these numbers to represent quantities. What exactly are these numbers these equations are using? *Scratches head.* That… I still don’t fully understand. Apparently they’re a type of information as well because computers have no trouble with them. But defining them is very difficult, especially when you wonder about irrational numbers, like pi. Hopefully I’ll learn more about that this summer as well.
I hope you all are looking forward to my detailed space-time and the mind posts. I have a whole library of books I’m pillaging through. For example, I’d like to talk about experiments vision scientists and psychologists have done trying to discern exactly how our brain is representing objects we see. It’s really fascination. Yes yes, my desk is strewn with books at the moment. I’m wondering how to organize all this wonderful goodness. What is this? Riemann’s Foundations of Geometry, Mach’s The Principles of Dynamics, Einstein’s Geometry and Experience, Minkowski’s Space and Time, Poincare’s Space and Geometry, William James’ Principles of Psychology – The Perception Of Space… My my, good stuff awaits those who read my blog. Philosophy of space, time, the mind, consciousness, numbers, the universe — it’s all on the way.