A Physicist’s Ramblings
November 5, 2011
What drives you to do the things you do? What makes you get up in the morning? What’s your deepest passion? As for me, I crave a deeper understanding of what’s going on around me. I’ve set my aim on discovering something new about the universe before I die. I want to add some new novel insight into our knowledge of reality.
I’ve never had a subject draw me in like physics does. It seems like I have the same realization several times a day, “All of this is really going on. All of these equations, vibrating atoms, oscillating electromagnetic waves, nuclear fusion reactions, etc. That’s all happening and is the ultimate cause behind what I’m experiencing.”
Dr. Tegmark’s philosophy toward the universe reminds me a lot of John Locke’s primary qualities of objects. Tegmark views the world as ultimately comprised of mathematical equations and algorithms of some sort. Those sorts of thoughts have been running through my head lately as I’ve also been reading a book by Stephen Wolfram called A New Kind Of Science. It explores an idea that reality may ultimately consist of just a few lines of code. He explores simple algorithms and how they can produce very complicated output, and ties all of that in with the laws of our universe, chaos theory, how deterministic simple processes can produce unpredictable output, and so forth. I’m still reading it all, so I can’t comment on it yet. But anyways, Locke drew a distinction between what he called primary and secondary qualities of reality, and primary qualities were very similar to these ideas of Wolfram and Tegmark. Primary qualities are properties of objects that are independent of any particular observer. They include things like extension, motion, figure, and number. Secondary qualities deal with how we subjectively experience those objects, such as smell, taste, and color.
When I first read Locke’s Essay in my late teens, I immediately thought of 3D graphics simulations on computers. Primary qualities would be like the data structures you use to hold the simulation information. You might store the 3D environment as a tree of polygons, and so on. But take the person taking part in the simulation, viewing the computer monitor and navigating in the virtual world by watching the changing images on the computer screen. All they see is a 3D world, and they have no idea how computer is creating that world absent being able to look at the code. If they can’t see the code, the simulation engine could be programmed all sorts of different ways. There’s no way of knowing.
Take the old video game Doom for instance. When you play the game you see yourself immersed in a 3D environment as so.
But what about the code and data structures which actually produce that world you’re seeing? Does the game produce those images similar to how the real world produces the images falling on our eyes? No, not at all. There’s no light rays shooting around the room, interacting with surfaces, scattering the light, and reflecting back into the player’s eyes. In fact, the world isn’t even truly 3D. If you look at the actual code and data structures which the Doom engine uses, you’ll see that it actually stores that 3D room you’re seeing as tree of 2D polygons strung together.
Notice that it’s actually 2D polygons, each with an assigned texture and height. Textures are shared between polygons to save computer memory. Neat huh? Why do that? It saves computational cycles. Doom was written back in 1994, back when the top of the line computers were 486 66 Mhz, computers with like 16 MB of RAM. You had to do things elegantly to save resources.
These sorts of thought experiments are what make me think that space is a subjective experience created by our brain. True “reality” may well be something like Tegmark and Wolfram are alluding toward – an algorithm or mathematical structure of some sort, far different than how we perceive it. Our neocortex is arranged in a hierarchy and when it processes images from our eyes, it naturally finds patterns, and patterns within patterns, and that’s what space is. Our brain doesn’t care how the images were produced. It’ll find space in any image or sequence of changing images.
I’ve always found it strange that no “space” exists within the computer. When you’re playing Call of Duty on your XBox, there is no “space” inside the console. Where does that giant environment you’re playing within exist? It’s data on a disk. It’s code. It’s data structures. For example, when you’re “moving” your game character, nothing is moving. An internal variable stored within your character’s data structure, probably something like “WorldPosition.X, WorldPosition.Y, and WorldPosition.Z” are being changed. The game’s environment isn’t moving. It’s data on the system’s hard-drive, static and unchanging. Well, unless the environment is destructible, as is common these days. Then their vertex points change values, moving the block.
When I think of how the ultimate “reality” could be timeless and could contain all possibilities in some sort of infinite multi-verse, I think of this computer simulation analogy. It helps me a lot. For example, think of sitting in front of Microsoft Word. Now imagine hitting random keys on your keyboard, filling up page after page with text. Every possible book which could ever be written already exists. If your fingers just happened to hit the right keys in the right order, a book would emerge. The same applies to music. Take a mp3 file. Within that 10 megabytes of information, every possible song and remix already exists. If I wrote a program which cycled through every possible combination of 1s and 0s for that 10 megabytes on your disk, every possible song that could ever be played would be played. They all already exist. They’ve always existed. People never “create” anything.
Take the analogy a little further. Instead of source code on a computer, with 1s and 0s, think of the laws of physics. Instead of a hard-drive and memory, think of quantum bits of information stored in atoms. With just a little thought, you see that physics is the programming language of reality. You look at those equations and you see the “code” running our world.
But this way of viewing the world is missing something very important – it doesn’t handle subjective consciousness well. It’s like seeing the world as a giant computer running a program, which seems to be largely correct, but it just doesn’t seem to be the entire picture. The thing is, I’m immersed inside of this world. I don’t know if I’m actually capable of “doing” anything, and I don’t know if I even have free will. In fact, most evidence today seems to suggest the contrary. But I do know I’m currently experiencing reality, and it’s not just information flowing about. I’ve always struggled with a sort of dualism between those two ideas, objective reality and subjective reality. I think George Berkeley was right when he criticized Locke’s distinction between primary and secondary qualities way back in 1680-something. The same arguments apply today when looking at Tegmark or Wolfram’s ideas. Take an apple. If you strip it of all its subjective qualities, such as its redness, the way it feels in your hands, its taste, and so on, what’s left? What’s left for its “objective” reality? Locke seemed to conceive this sort of geometric shape, information related to its motion, such as inertia, and other “data” and numbers. Maybe that’s correct?
The ultimate problem is I can’t leave my body and see the world objectively, so how in the world can I know how things exist in and of themselves? If the true “reality” is data and numbers and mathematical expressions, how would you “experience” that? Meld with the math? When I think about the world’s objects, it’s my brain doing the thinking, and it got its information from sensory impressions. It then runs through its hierarchical process of pattern recognition and I call it “understanding”. But that doesn’t mean that time is actually flowing how my brain makes me feel it does. Things may not move how I think they move. Like in the simulator, I can “move” an object by just changing a few variables at certain memory addresses in the computer. That changes the images on the screen, which the person then experiences as an object flying across the room.
It is odd though. When I play Doom, no matter what keys I press, and how much I explore, I’m never going to be able to learn how the game was programmed. But in our world it’s different. There’s clues everywhere. There’s like this fractal structure within the information, baiting us to learn more. Our universe repeats the same simple processes, over and over and over. We’ve found a few simple equations which pretty much describe all of reality as we know it.
All the research I’ve been doing into machine vision and the brain are related to how that subjective sense of space is produced from sensory information and patterns. I’m hoping that by understanding how that works in more detail, maybe it’ll shed some light on how a particular reality can be experienced from unchanging math equations and so forth. It’s very vague and immaterial in my head. I still don’t “get” subjective space completely, but I seem to be moving toward something. I can write code which takes a series of images and the program will produce a “cloud” of 3D data points which resemble the 3D world on the screen. The points will all lie on the walls, and so forth. I’m not sure what sort of data structure to use to store the “walls” and rooms. How does the brain store that information? Well, it uses a hierarchy of six or so layers, storing patterns, and patterns within patterns, and patterns within those patterns, and so forth, up to six layers deep.
To me, that’s the bridge between subjective reality and objective reality as represented by math equations in physics. Figuring that problem out. I want to keep studying how that subjective space is built up, and keep studying more and more physics, learning how those equations work. I’ll keep working at roping the two together.
We’re rapidly approaching the day when we can immerse ourselves in virtual reality. If you think the world is nothing but what you actually experience, then those worlds are true parallel universes. You may say, “Oh no, that’s just data in the computer, and signals going through the brain, and so on.” But an experience is an experience. That space is space isn’t it? Why is it a different sort of “space” than “real” space? People are already interacting in it, having experiences together, and when we have true VR, will be fully immersed within it. How is that not a true parallel universe? It seems to me to be one. Why is our world the “real” one and the others “virtual”?
I think “space” can be created in many different ways. 3D graphics algorithms in computer programming captures one way to do so. It’s real space. It produces the proper changes in patterns of information which a brain, which is particular type of information processor, can process and turn into a subjective sense of existing within it. The code which runs electromagnetic waves, atoms, and how those waves scatter and interact with those groups atoms in physics “stuff” also contains those same patterns of information, which are processed into space. The light rays form images on our eyes, which in turn give us a sense of existing within space.
But does an “objective” space exist? I don’t fully understand that. Things bend and distort in weird ways as you increase your speed, especially near light speed. You have time dilation and the Lorentz contraction, and mass begins to increase. I’m still not a master of that level of physics, so I have to keep working at it. Our subjective “space” starts breaking down and not really working right. Once that’s gone, you don’t have any intuitive “tools” to help you understand what’s going on. The mathematical relationships in the equations are all you have. It’s not easy. I study it, get frustrated and wore out, study something else, and then go back to it, over and over and over again. Like in quantum mechanics, things aren’t as separate and can exist in multiple locations at once. The world of the very small is weird. It’s all too strange. This entry is getting too long.
Before I go, one last idea. The brain is like a hard-drive and computer processor built into one. It builds up a model of the world from sensory information fed into it. I speculate that there’s two separate “realities” taking place. If a reality is information and information processes, there’s the information structure within my brain, which is a certain flow of energy and information, and then there’s information flows outside my body. When an object moves, the movement that I perceive is based on how the data structures within my brain are changed and manipulated, but “objectively” that movement may be something akin to some variable changing in my simulators I program all the time. I think the whole “subjective” dynamic is based mostly in the fact that the brain is largely an internally connected information flow, only letting outside information flows in from sensory feeds.
It’s almost 6 AM and I’m dead tired. I’m typing this while half asleep, so keep that in mind. I’ll end it here. I typed this out as is. Very little proof-reading. This is how information just spews out of my head onto a page.
Popularity: 1% [?]
Topics: Philosophy | No Comments »
Goodbye Grandpa
October 22, 2011
Earlier today my grandfather passed away. It’s put me in a rather gloomy mood. Whenever I reflect on death I get really depressed. We live incredibly short lives. There’s never any guarantee of tomorrow.
Many years ago I spent a lot of time reading philosophy. I spend most of my time with science these days, but I used to read a lot of philosophy related to life, morals, and the mind. I always best related to existentialist philosophers like Jean Paul Satre. They don’t really try to offer an explanation for evil, nor all the terrible things that happen in this life. I don’t think there is any meaning in such events. Absurd things happen. Terrible things happen. Sad things happen. It’s just the way life is, and assuming we don’t work to change the world, things will stay that way.
Existentialists have a doctrine they call “absurdism”. I’m not a huge fan of using “isms”, but I went with the term anyway. I was trying to think about how best to define it, so I just looked it up on Wikipedia and liked their definition:
Absurdism, therefore, is a philosophical school of thought stating that the efforts of humanity to find inherent meaning will ultimately fail (and hence are absurd) because the sheer amount of information, including the vast unknown, makes certainty impossible.
- Wikipedia, Absurdism
The only tool us humans have available to use against this cold heartless universe is our powers of reason, but reason is very limited. Our brains forget the things we learn, never seem to have enough information, and can only process so much at a time. This leaves us vulnerable to all life’s complexities. We have to try to figure things out and adjust our actions in such a way as to produce the best outcomes both for ourselves and to those around us. However, the world is brutal, heartless, and near boundless in cruelty. We’re stuck with absurdity. I don’t think us humans will ever reach a level of absolute certainty, but things can be made better.
What does this have to do with death? Well, death is absurd, as are many things in life. We never know when or how we’re going to die. We never know when or how our loved ones will die. We never know when or how our friends will die. But they will die, I assure you. They’ll all die, and you will too. I’m not the harbinger of pessimism, this is reality.
I’ve always have liked Albert Camus, an existentialist author. He prescribed that we cope as best we can with all life’s absurdities.
… a person can choose to embrace his or her own absurd condition. According to Camus, one’s freedom – and the opportunity to give life meaning – lies in the recognition of absurdity. If the absurd experience is truly the realization that the universe is fundamentally devoid of absolutes, then we as individuals are truly free. “To live without appeal,” as he puts it, is a philosophical move to define absolutes and universals subjectively, rather than objectively. The freedom of humans is thus established in a human’s natural ability and opportunity to create his own meaning and purpose; to decide (or think) for him- or herself. The individual becomes the most precious unit of existence, as he or she represents a set of unique ideals which can be characterized as an entire universe in its own right. In acknowledging the absurdity of seeking any inherent meaning, but continuing this search regardless, one can be happy, gradually developing his or her own meaning from the search alone.
- Wikipedia, Absurdism
None of this makes me feel any better about death, but I don’t think there’s anything cheerful about it. Grandpa’s dead and I won’t ever get to see him again. From time to time I still find solace reading my Bible, though I’m not a religious person. I find Solomon, the author of the book of Ecclesiastes, to have been a wise man. Take this passage:
I also said to myself, “As for humans, [ ... ] they are like the animals. Surely the fate of human beings is like that of the animals; the same fate awaits them both: As one dies, so dies the other. All have the same breath; humans have no advantage over animals. Everything is meaningless. All go to the same place; all come from dust, and to dust all return. Who knows if the human spirit rises upward and if the spirit of the animal goes down into the earth?”
So I saw that there is nothing better for a person than to enjoy their work, because that is their lot. For who can bring them to see what will happen after them?
- The Holy Bible, Ecclesiastes Ch 3
Solomon gave the best advice as to how to live our lives during this stay on this planet:
Go, eat your food with gladness, and drink your wine with a joyful heart, [ ... ]. Always be clothed in white, and always anoint your head with oil. Enjoy life with your wife, whom you love, all the days of this meaningless life that God has given you under the sun—all your meaningless days. For this is your lot in life and in your toilsome labor under the sun. Whatever your hand finds to do, do it with all your might, for in the realm of the dead, where you are going, there is neither working nor planning nor knowledge nor wisdom.
- The Holy Bible, Ecclesiastes Ch 9
I had planned to write about my experiences with grandpa during my life, but I ended up writing about the thoughts bouncing around in my head as I’ve been out for my walks. Forgive me, I’m not really in a cheerful mood right now.
Popularity: 1% [?]
Topics: Personal | No Comments »
Swallowed Up By The Night
October 17, 2011
I believe I’ve mentioned this before, but I love to take long walks at late hours. As I was out tonight, I realized that after around 30 minutes or so, I enter into a trance. Maybe trance isn’t the right word. Maybe a state of intense concentration and focus? I’m only half-conscious as to what’s going on around me. During these walks, I’m in a state of half-daydream, half-reality.
Tonight I was reflecting on how the sky is like a giant canopy which only opens up at night. Though I find the daytime sky majestic, especially in the evenings, it also obscures the view of the vast universe of which we’re a part.
It’s best to find yourself a long winding country-road which leads out into the middle of nowhere. On a clear night when it’s nice out, just walk down that road and keep going. Hopefully you won’t encounter any cars and you can just keep walking and walking. Within a half-hour or so your eyes will adjust and become extremely sensitive to light. That’s when the sky above you will come alive. Stars will start to appear, one by one, until eventually the giant dome is filled with them. They’re everywhere and you’ll find it puzzling that you didn’t see them earlier. It’s not like they weren’t there before – you just didn’t see them. I don’t know why everyone goes to sleep when all of this becomes visible. It’s a shame. Too many people fail to remember that we live on a tiny mote of dust spiraling around just a single star in a universe of unfathomable proportion.
I think I realize what I love about the darkness: it’s a blank canvas to be painted on. My mind starts imagining all sorts of things and I soon lose myself. It’s the same reason I love writing simulations. I start with a blank canvas – a blank computer screen – and begin painting colors into forms, drawing whatever I can imagine. Unlike a painter, who is limited to a single canvas and does not have to worry about the rules behind the picture’s generation, I get full control over a fully interactive canvas. You can move through a virtual world and I control every parameter, painting countless pictures in succession, just like the real world does.
As I traversed the winding roads, I pondered my ideal universe and the rules behind beautiful forms. How could I create a virtual heaven? I always find myself inadequate for the task. For example, ask yourself whether or not you can even imagine a sky more beautiful than our own? I mean honestly, how can you find fault in this?
You know what’s best about the sky? It’s blue, my favorite color. I’m very fortunate in that outdoor scenery is painted in all my favorite colors. Greens, browns, grays, oranges, and blues. During the day the giant dome is sky blue, which is my second favorite color, next to navy blue. These beautiful fluid clouds float about, slowly twisting and flowing onward, like giant cotton balls suspended overhead. It’s different every single day, yet it’s still blue skies and white poofy clouds.
White is such a rich color. The sky looks pure and clean. I just want to get up there and fly around in those clouds, with my arms outstretched, breathing in that clean air. Pilots talk about how wonderful it is to fly small aircraft way up in the clouds. I’ve seen videos on youtube and it doesn’t even seem real.
Autumn is hard to beat too. As I walked, the leaves were shuffling about, many of them being lifted up into the air, twirling around in erratic, unpredictable patterns, carried by small gusts of wind. I had never noticed it before, but dead leaves can be scattered all over the ground and still look beautiful. They look like they belong there, blending wonderfully with the grass and small sticks on the ground. But you throw a small piece of litter beside them and it looks terrible, even if it’s the same color as the leaves. There’s more to a leaf’s perfection than its color. They’re not little brown pieces of paper. There’s little bends and curves in them, there’s veins running throughout them, and they have a beautiful fractal like symmetry, which matches the pattern of the trees. Nature has attention to detail. Your unconscious mind feels it and notices it, even if you’re not consciously able to identify exactly what it is you’re feeling with words.
I think I’ve been spending too much time looking at pictures created with Terragen. Have any of you heard of it? It’s a software program where you can build outdoor environments and render them. The images created are photo-realistic. They have a huge library of plant, tree, and flower models, as well as algorithms to produce rocks, canyons, mountains, weathering, erosion, flowing water, and so forth. This is the very sort of thing I imagine when I think about the future of virtual reality combined with advanced AI. We’ll each be given a huge portion of memory and computing power, and then be able to store whatever we want on it. In that space we’ll be able to create our own ideal home — our own personal universe. You can see pictures created with it here. Here’s an example of one picture created with it.
Take a look at this picture of a sky, created completely with software.
I love this form of digital art. Artists take 3d models of environments and mountains, and using these software tools, they create photo-realistic images of utopian worlds. When I’m walking in the darkness, the dark silhouettes of tree lines are filled in with flowers and grazing animals, peacefully meandering about in the fields dotted with color roses, carnations, and tulips. Castles appear off in the distance and beams of colored light penetrate the imagined foliage canopy.
My mind often drifts between what’s possible and what really is. I start to think about future generations a few hundred years from now, flying around in virtual reality, telling a super-advanced AI system, “I want to build an island here. I want these sorts of flowers, these sorts of trees.” The AI system replies, “As you command.” An island appears and then the person walks around it, “No. Change these purple flowers to red ones. I don’t like conifer trees. I want oak trees. Wait, show me all sorts of different trees.” A myriad of different trees start flashing before the person. “Eh’, I’ll go with that and some of those.” After finishing the outdoor environment, the person says, “I want to build a mansion over there. I’m looking for something similar to the Royal Pavillion. You know, Oriental style.” Then this structure appears in the flower garden.
It doesn’t seem real to me that future generations will experience that. Their brains may be removed from their bodies as young children and they may never know what it’s like to have a “normal” human body. If they interact with the real world, they “possess” advanced prosthetic bodies that feel just as real as ours but without all the pain and suffering. They may never know pain. Neural implants may keep them from ever experiencing loneliness or depression to any significant degree. They won’t have to learn from books, but will just download information into their brains. They’ll be able to race cars, fly any type of aircraft, immerse themselves in elaborate virtual adventures, bonding with one another. To them, after living in VR for hundreds of years, I don’t think they’ll even think in terms of what’s “real” and what’s “virtual”. An experience is an experience and they’ll be able to control their experiences to a degree far beyond what we can.
AI, neuroscience, quantum physics, software simulations, and virtual reality… Think about all of that too long and this is what happens to your brain: you get a strange man like me wandering down a country road at 2:00 AM, staring up at the stars, walking right past you, completely lost in thought, not even noticing you’re there.
Popularity: 2% [?]
Topics: Personal | No Comments »
How Is Everybody?
September 28, 2011
What is this? I haven’t posted anything on my blog for almost a month?! How irresponsible of me. If I said I’ve been too busy to post anything, I’d be lying. Really I’ve been immersed in some personal projects. Maybe I’ll talk with all of you about some of the things I’ve been up to. So let’s get right to it.
As some of you probably already know, my passion is understanding the mind and our world, and as of the past few years, I’ve been researching how to build intelligent machines which understand space. I want to better learn how our brain understands the world we see out of our eyes, and how it builds a model of the world. What sort of data structures are used within the human brain to hold onto spatial and object information? How is it accessed? How is that information processed and changed? How does our brain make predictions on what we will experience next, forming expectations of the world? In other words, I want to build a machine that has two camera eyes and can understand the world it sees through those eyes. I want it to be capable of driving or walking around, avoiding obstacles, identifying objects, having memories of past experiences with those objects, capable of predicting your next action, and so on.
I’ve been searching for algorithms that are modeled after the human brain, using simulated neurons and neural networks. After searching online for months and months, and reading textbooks I acquired after checking out MIT open-courseware’s Brain and Cognitive Sciences program (I bought the same textbooks they use for their neuroscience degree program at MIT, and have been reading them), eventually I found a jewel. Jeff Hawkins, president of a company called Numenta, and founder of Palm computing, has been traveling around the country giving introductory lectures on building intelligent machines and simulating the brain in computers. He’s developed an algorithm which he’s modeled after the neocortex. He calls it Hierarchical Temporal Memory (HTMs). It’s the exact sort of thing I’ve been searching for. So how does it work?
Let’s first pose a difficult question. How does your brain identify objects, say, a cat? Think of how different each cat or dog you see is from each other. One cat may be skinny, another fat. One may be white, another brown, while another black with white spots. One dog may be a beagle, while another is a golden retriever. Even so, you’re able to notice that they’re all dogs and easily can identify them as such. Small children simply look at a few pictures of dogs in a picture book and can then easily identify the animals they see in real life. How can the brain do things like that? Or better yet, how can I build a computer program that can look at a video feed and watch hours and hours of footage and then identify a particular type of animal when it comes on screen?
For example, when I was watching David Attenborough’s films, in the behind the scenes footage I saw their team having to stake out a bird’s nest for days, filming and filming, waiting for the bird to come back. Imagine if they could just leave their camera there, hidden in the brush, and film all the footage, and then have a computer “watch” all of that film, identifying the moments in time when the animal came back to the nest. AI could track the bird, moving the electronic tripod to keep the bird in view. That way the researchers wouldn’t have to sit and watch hours and hours and hours of footage.
Or say you wanted to make Youtube and Google far more intelligent. Instead of just applying intelligent machines to just visual information, say we applied it to audio information as well. Say you are wondering about the positions of a political candidate in an upcoming election. Has that candidate ever stated their position on such and such an issue? You could ask, “What does Ron Paul think of Medicare and Social Security?” And then AI algorithms would search Youtube and find particular clips of Ron Paul stating his positions on those topics. It could cut out small portions of much longer clips and bring them up to you for viewing.
Sounds neat doesn’t it? Well that’s what’s currently being developed these days and it’s amazing technology. That’s the sort of thing Jeff and his colleagues at Numenta are working on. But how does it work? First off, you can read their research papers here.
I was particularly drawn to them because I have always wondered how abstract thought took place within the human brain. I always wondered how the brain stored information about a generic “cat”. How would I write an algorithm that could identify a cat? I had no idea how the brain did that. But now I think I get it. Take a look at this picture below. This is the idea behind HTMs.
If you look at your brain’s neocortex, which is where this sort of thing happens, you’ll see that it is structured in layers. Though this is a gross simplification, the sensory organs, such as your eyes, feed into the bottom layers, which then process the information upward to higher and higher layers. Higher layers also feed back down to lower layers, but we’ll talk about that in a second. So what’s going on there?
Basically the brain starts with simple patterns, such as a direct image input from your eyes. The neurons then feed that information upward to the next layer up, which finds patterns in a small portion of the image. You can see that in the HTM image above. And then that feeds up to the next layer above it, which finds patterns in the patterns. Then the next layer up finds patterns within the patterns, within the patterns. And so on and so forth. A common very simple pattern algorithm is run over and over and over, passing the results upward in a pyramid of pattern information.
The same idea applies to audio information coming into your ears. You start with basic audio coming in from your left or right ear, which then feeds up to a higher level, and then another higher level. Each layer looks for patterns within the layer below it, and you end up with patterns of patterns of patterns of patterns.
Going back to our cat example, the information “cat” would be a higher level concept in this pyramid, and if you traced “downward” in the pyramid you would come to individual experiences with particular cats you’ve had contact with. So in one grand stroke, your brain is forming memories of the particular pet cat you’re playing with, but also forming generalized ideas about how cats behave in general, how they appear, and so forth. Your brain then comes to an understand, “Ah, so this is what a cat is like. This is how they behave.” And then in the future your brain can identify cats and have expectations about how they behave. For example, you’ll know to be careful when dangling your socks in front of their eyes as cats have an instinct to claw such interesting objects, possibly injuring your hand if you’re not paying attention.
Now let’s talk about the connections that feed downward. Your brain doesn’t just passively observe the world around it – it tries to make predictions about what will happen in the future. When I see my pet cat Meanus lying on my bed, I have had a lot of experiences with her. I know what to expect and when I go to rub her belly, I know what she’s going to do. My brain takes visual input from my eyes, which then triggers this pattern recognition process described above. If finds patterns, and then patterns within the patterns, and then patterns within the patterns within the patterns, and then matches that up with, “Oh, that’s meanus!” So those particular neurons fire and I become conscious of being in the room with my cat. Now at the same time, my brain is constantly comparing my present experience with experiences I’ve had in the past. Past memories of Meanus are being called up and accessed, being used to predict what she will do next. That’s what the feed downward links do. In particular they compare past experiences to the present, and if they’re not lining up the brain says, “Wooaaahhhh. Something new is going on here. Alert! Attention, attention, focus attention on this!”
For example, if I was here typing on my computer and then Meanus stood on her hind legs and started audibly singing, “Fly me too the mooonnn, let me plaaayyyyy among the starrrssss…” My head would spin, I’d be blown away, and then I’d think, “What the HELL IS GOING ON!” I’d lose interest in everything else and watch in silence as Meanus crooned me a toon. My brain would recognize the Meanus patterns but when it compared what Meanus is doing now to what she’s done in the past, it wouldn’t recognize the behavior, would consider it “weird” and out of the ordinary, and my attention would be drawn to it. That’s because this would be violating my brain’s current mental model of the world. Cats don’t sing! My brain would then have to start rethinking Meanus, such as, “How is she able to sing? Has she been possessed by spirits? Is she being controlled by aliens? How intelligent is she? Am I dreaming? Is this really happening?” I’d have to then change my relationship to that object and how I plan to respond to it in the future.
Before going on, I’d like to bring up something which I found fascinating about all of this. Many years ago I remember reading Wittgenstein’s Philosophical Investigations and he mistook this process for free will. Or maybe this is what free will is? He said we only notice our free will in action when our expectations are violated. For example, we may will to place an object down on the kitchen table, but as we lay it down and let go of it, it starts to fall over and roll toward the edge of the table. That’s when we scramble to grab it before it falls off and breaks. Our free will decided to place the object the table, but that decision was violated by reality, and then we had to make a new decision to grab for the object. Now, however, I understand that that’s just how the brain is structured to work — the HTM process. It’s how that pyramid hierarchy of information processing works. If something violates our expectations, attention is focused on that until the situation is brought under control. Most of it is an unconscious process though, and I don’t think it explains free will.
Ok, so how does this tie to how I’ve been spending my time? Well, first off, I’ve been studying neural computation, and how to model neural networks in software. I want to implement something like Numenta’s HTMs and then rig a computer up to go around, processing information from cameras. I want to store all the visual information in this HTM hierarchy and then train the system to identify objects. Next, I want to be able to go into this huge multi-terabyte database of visual information and run a program on it which can go into that HTM database and pull out 3D spatial information. I want to be able to say, “Computer, generate me a 3D model of my bedroom.” It then searches its database, finds that tree of information, processes downward through it, and then builds a 3D model of my bedroom on my screen, rendered in OpenGL. Then I can fly through it and look around.
I want to be able to walk around a place with a camera, filming things, and then show that film to my computer, let it parse in the video feeds, and build up an ever growing database of visual information. I could show it a video clip of me walking around a college campus and then say, “Computer, build a 3D model of the buildings you saw in that video.” It would then do so.
That’s the goal I’m working on. I want to fully understand how our subjective sense of space works and how our brain works with space and numbers, and logic and everything else. This HTM stuff is deeper than just space. It’s how language and abstract thought work. It’s how intelligence works. This is what intelligence is. It’s this process of finding patterns within patterns within patterns, and organizing them in a hierarchy, and making predictions with that information. At least, that’s what I’m currently thinking intelligence is.
From what I gather, algorithms similar to this are what are being used to build intelligent computer chips. Companies like IBM are wanting to build computer chips which process information in a way similar to these HTMs and then the computers can be intelligent. This doesn’t get rid of normal processors, but by changing the way information is being processed, sensory type information, such as sight and sound, can be processed far more effectively and easily.
However, during the past week or so I’ve had a long diversion from my research. I was a little burnt out from my brain research and had just taken two exams in class – math and physics. I wanted a break and needed some time to work on something different. It’s nice to do that here and there. I have just acquired a new development IDE and was reading through some programming books to learn the new features. That was nice. I was writing some “goof off” programs to test how things work. I ended up making a really stupid program with a paintbox which drew random lines within it. Once I finished that I leaned back in my chair, yawned, and went for a walk. It was a good day.
Once I got back from my walk I got to looking through my bookshelf and saw an old classic I hadn’t read in ages. It’s called The Black Art of 3D Game Programming: Writing Your own High-Speed 3D Polygon Video Games in C. Now there’s a gem for you! It was written in the early 90s and shows you how to write 3D games in DOS! Epic! Beyond Epic! Why is that? This is before the days of Windows, and DirectX, and OpenGL. This is back when you had to write directly to your video card’s memory buffer, writing your pixels for each dot on your screen. Memory address (X,Y) on your 320×200 screen set to some RGB value. I was thinking, “Ah, I remember this book. I love this book.” Then I got to reading the section on 3D game programming, the mathematics involved, the matrices, the vector mathematics to calculate collisions, and so forth. Then I got a wild idea. What if I wrote a 3D engine over the weekend, rendering to Windows paintbox? LOL. Software rendered 3D graphics engine using the code from this ancient book. So, that’s what I did!
I got to cranking and then made a simple Doom like game, where I was running around in 3D world. I wrote the code to render the picture, pixel by pixel. I had to write code to draw individual lines, to draw polygons and triangles, and to do lighting effects. LOL. I didn’t use any libraries of any kind. No help. I did it all from scratch. I coded my 3D points in 1 by 4 matrices, which I then multiplied by rotation and translation matrices, rotating my scene relative to the camera. I’d swing my mouse around and change my camera’s direction cosine angles and rotate my scene. That was cool.
I logged into MSN messenger and told one of my old friends how my Saturday had been one of the best days of my life. He didn’t seem to understand why writing a 3D graphics engine from scratch into a Windows paintbox was anything to be happy about. But to me, I went under the hood into how virtual reality and video games work, understanding how all the fancy 3D graphics and video games of today work. How do they render the walls, the textures, the lighting, and all of that? How do they make it look so real? Well, I know how it works, all the way down to drawing each individual pixel to the screen. That’s why it cool. I like knowing how stuff works. I especially like programming simulations.
In the video above, you find a ray tracer, which I would like to eventually find time to program. I programmed something similar, but much more simplistic. Mine also used complete software rendering, without using any libraries. That way I had full control over how things were rendered, the physics used, and so forth. In their simulation, they blast billions of light rays into a scene, which then bounce around, physics calculates how the light bends and reflects, and then it collides with the camera. It works how reality works. My simulation I wrote over the weekend uses a series of tricks and rotations, similar to how 3D games programmed today work. Ray tracing requires too much CPU power to handle in games today, but in 20 years, I guess it will be the norm as to how games are rendered. It produces photo-realistic graphics, as you can see.
I like video games and virtual reality in particular because in that world you’re God. Think of it this way — say you were made God for a day. You could change anything, and make anything work however you wanted. You could build your own world from scratch, to your every specification. How would your world work? Well, when you write your own video games, that’s essentially what you’re doing. You can make any experience for the player you can imagine. The real test is how creative you can be. When I’m out for walks, I try to notice every little detail of this world. I look at things both from the angle of the physicist, where everything is ordered and following laws, to the emotional and graphical, such as an artist’s perspective. I look at the sky, with the blues and oranges and reds, or the stars flickering in the sky. I notice specular highlights from the lights in the room shining on reflective objects, and light and soft shadows being cast from multiple light sources in the kitchen. I think to myself, “If I were God of my own virtual reality experience, how would I program my reality to work? What good things from my world would I keep, and what things would I change? What would people in my world experience?” I find that doing such an exercise makes me extremely happy because it makes me focus on everything that’s awesome about this world, and learn how those things work. Then I try to program those things into a computer, and I come to a very deep understanding of the things I love most. Also, when undergoing this process, I also have to search and find ways to bring those experiences to other people.
Richard Feynman once said, “What I cannot create I do not understand.” If you can’t create your own virtual reality similar to our own, you don’t understand the world you live in.
Oh yeah, and before I forget, you may have noticed that comments are closed. I started getting like 500 spam comments a day from bots. I got tired of cleaning them up. I don’t know how to stop them, so I just closed comments down entirely. I’ll try to work on fixing that sometime. The internet is such a sleazy place. Bots go around trying to create links to people’s sites, creating false sites full of viruses and spyware, all to help push some crappy website’s page ranking up in Google’s search results. Losers. Try writing real content and having a real website, and then maybe people would come to view your site without having to resort to tricks and lies. You’re like Newt Gingrich, hiring companies to make millions of fake twitter followers.
Popularity: 1% [?]
Topics: Personal | No Comments »
The Yin And Yang Of The Mind
September 5, 2011
I’d like to continue the discussion of yesterday and elaborate on a topic which is really interesting to me: the yin and yang of the mind. As I was reflecting on this idea today, I mainly considered how the mind is a swirl of yin and yang, like a lollipop.
Let’s start with being thankful versus being discontented. The same mental mechanisms are used for both, yet one is considered a virtue while the other a vice. The human mind has the ability to transcend space and time using its imagination. It can imagine and or remember things that don’t currently exist, and if you desire, it can compare that to what is currently in front of you. We do it all the time. It’s how we know that time is passing and that things are changing. It’s what allows us to understand the world, make decisions, and have a degree of control over our lives. It allows us to choose who we want to be. But like anything with great power, it’s a very dangerous tool as well.
If you compare “down”, we consider it being thankful. We think of all the nice things we have in our lives and how much worse off we could be. Using the exact same mental system, we can also look at our world and think about how much better everything could be. This is comparing “up”. Although what’s considered “up” and “down” (better and worse) are often relative to a person, you get the idea.
We’re often told to have big dreams and that we can accomplish great things. But what is a dream? It’s a state of discontentment. It’s an imagined reality which you want to bring into existence. It can be a dream to find a lover, to build a great company, or in some way change the world for the better. Either way, it’s a state of discontentment. The same system that allows you to better yourself is also what allows you to be discontent with your life. They work hand in hand, yin and yang. You can’t separate the two without destroying who you are, your dreams, and everything that you value.
I remember watching videos of a famous inventor talking about the future of nanotechnology, and how future humans will be able to immerse themselves in virtual reality, have any experience they desire, and also control the world all the way down to individual atoms. At the same time, I could tell the man wasn’t very happy with the world he lived in. He saw how much better things could be and that left him severely discontented. Why wasn’t he living in that wonderful world? The thought had to occur to him every single day. The man has ten PhDs and is one the most brilliant men on the planet. His mind has grown so large he can see possibilities far beyond most people. He sees where humankind is headed and it’s wonderful. But the same intelligence leaves him very unhappy.
Intelligence is generally considered a virtue. The more you perceive the laws of this universe, the more power you have to navigate between all the infinite possibilities to the one you desire. Every new scientific and engineering discovery we’ve made has allowed us more and more possibilities. Metaphorically speaking, it’s like we’re reuniting to God. Our drive to learn about the universe ultimately stems from this separation from the divine. We yearn to be better, to go farther, to be stronger. We want to explore, to learn, and to grow. As we grow and learn, we find and acquire new things to be thankful for, yet ultimately those things grew from a discontentment and a longing for a better world. If we weren’t bored and eager to explore, we never would have learned about all the wonderful sights we have to cherish in this world. We can’t be thankful for something we don’t know anything about.
The same applies to love and hate. The second you choose to love something, you also choose to hate something else. With every decision we make, we flee one thing and gravitate toward another. If you’ll love anything, you stand for nothing. If you’ll believe anything, you have no principles.
This intimate swirl between yin and yang is so embedded in our nature that I don’t see how this law could ever be removed without us losing our humanity. We can never be complete and satisfied. The mind is built to be discontent and continually desire bigger, better, and faster things. This universe of ours, this game of life we’re currently immersed in, isn’t something that can be “won”. We were constructed by a yin and yang dynamic. Our existence is not a static state, but is a flow, a process, a movement. And that movement is directed by the world’s feedback system – success and failure. We had replication with random mutations along with non-random selection, winners being decided by who was best suited to survive in the environment. That striving to be better, stronger, and endure is infused in us in every aspect of our being, consciously and unconsciously.
Popularity: 2% [?]
Topics: Philosophy | No Comments »