When I was younger, I used to get up early, or stay up late, to watch the major space achievements of the day. It is for the sense of moment.  Because while seeing the videos afterwards may still leave shivers, there is nothing more intensely real than “being there in the moment as an eye witness” to a history that is going to transcend our meager lives. Portugeuse exploration began between 1325 and 1357 under Alfonso IV.  This culminated in 1488 when Bartolomeu Dias rounded the southern tip of Africa, to which he gave the name “Cabo das Tormentas” – “Cape of Storms.”  In 1492 Christopher Columbus discovered the “New World.”  The first English Settlement in Virginia was 1607, Massachusetts 1620, discovery of Manhattan 1609.  If you’re keeping track, that’s a span of close to three hundred years.  And the point is that given the length of our lives, we are only privileged to “witness” a very few of the truly significant events.

So yes, if you’re wondering I did get up this morning to “watch” the European Space Agency’s space craft Rosetta rendezvous with Comet 67-P Churyumov-Gerasimenko.  More to the point at 4 am EDT, I watched this attractive, perky, English woman talk about it and stared at a computer screen at Mission Control  in Darmstadt, Germany waiting for the display to peak and then turn downwards – us scientists are easily satisfied!

But really, and most of all, I marveled at such images as this one taken on August 3 from 177 miles.  Truly what an amazing achievement.  As the perky, English woman said: “brilliant!”

When I was young and frequented New York’s amazing Hayden Planetarium, I just might have dreamed of such a thing, but then I put dreams aside for reality, and now they have become reality – which I suppose says something.  I thought this morning about photography and about the meaning of being there, when I am not really there and when the “there” is really not now because of the time lag.  This photograph and all the images and data that Rosetta has and will send back connect us not only to each other but really back to the time of the creation of the planet.  So to the team that dreamed and then spent a decade coaxing Rosetta to its destination, congratulations.  What a truly “brilliant” accomplishment.

Figure 1 – Lewis Hine, 1932, Worker on the Empire State Building. Created for the US Government WPA and in the public domain.

About a year ago, I posted about what I called “Morphin’ memes.”  This is the concept that the meaning or connotation of a photograph or the subject of a photograph can change with time. I took a set of photographs a triptych in 1968 of the Consolidated Edison Steam Power Plant on the Lower East Side of Manhattan and commented how it signified pollution but that five to twenty years earlier the same photographs would have symbolized power, strength, and national growth.  I started my last three blogs talking about the Empire State Building.  What could be more symbolic than that?  But in doing my research for those blogs, I noted something that struck me as pretty peculiar.

Consider Figure 1.  It was taken by Lewis Hines for the WPA.  It shows a construction worker building the Empire State Building.  It was after all built by men, tightening one bolt at a time, welding one joint at a time. It is pretty typical of the images of great construction projects at the time. Take a look at this set of images of the Empire State Building construction.  And you have, of course, also to look at this very iconic image by Charles C. Ebbets from 1932 showing workers having lunch on a beam during construction of the RCA Building.  The theme is men building and men creating.  Usually, this is set against the background of great height, but often enough the background is cloudy and overcast.  The men are the central theme.

Now consider images of today’s great construction projects.  They are typified by images such as those that I discussed in a previous blog about the construction of NYC’s Second Avenue Subway.  If you do a Google image search of the terms “Construction, World Trade Center,” you’ll see what I am talking about.  And so as not to belabor(sic) the point there are notable exceptions, one of my favorites being an image of a welder building the original twin towers in 1970.  The emphasis is structure, the emphasis is the power of machines.  It is a city without inhabitants.  You want brave and noble construction workers.  Take a look at Joel Meyerowitz images of the “deconstruction of the World Trade Center.”

I am probably overstating the point.  There are as many photographic views as there are photographic eyes and cameras.  But I do believe that in this context photography gives us a glimpse of ourselves and our deeper attitudes.  It’s meant to do that.  In the 1930’s we valued labor and our goal was to put men back to work during the depression.  The depression/recession of 2009-2013 what about that.  We spent a lot of time pointing fingers.  We no longer value work. Unions were to blame, everyone was to blame, except me. We glorified machines and monuments set against hollow cities.  After all, the really important question had become who could build the tallest building, the ultimate pinnacle of humanless success.

Figure 1 – The I Love You Wall, Le Mur Des Je TiAime with “I Love You” written in hundreds of languages, brainchild of Frederic Baron,  in Paris, March 28, 2011. From the Wikimediacommons and uploaded by Oh Paris to Flickr under Creative Commons Attribution License.

In the last two blogs we’ve gone from the Empire State Building to the concept that a photograph is a five dimensional object.  This merely states the obvious point that a photograph is a set pixels, which are laid out in a two-dimensional grid.  Each position requires two numbers to define it, and three numbers to define brightness and color, the amounts of red, green, and blue. So we define each pixel in the image as a vector (x,y,r,g,b). And here is where the fun begins.

Let’s imagine that we add the physicist’s favorite fourth dimension, time, to the mix.  So we wind up with a string of images, each taken at a different time.  Wait a minute that’s a movie, and each pixel in the movie as (x,y,r,g,b,t), where t can be expressed either as a true time, say in ms, or as a frame number.  A movie, which is really a type of photograph, is a 6-dimensional vector.

Well that’s a silent movie isn’t it?  What about sound.  We can certainly add sound to a movie.  We had to add time first, because human perception of sound requires time.  So do we just add sound as one more number to the mix.  Well obviously not.  Sound is not just intensity and more than light is, it’s also got frequency, which is a sound’s color.  Do we need three numbers or coordinates to describe sound? That is are there three primary sound frequencies?  Well like visual color, sound color is complicated and a mish-mash of psychology, physiology, and physics and way beyond what I want to discuss today.  But, at a minimum, we know that the concept of primary sounds cannot be completely devoid of validity, because an A is an A regardless of the octave, and all (western) music can be written as a sequence of the notes of the chromatic scale. So let’s just say that we have to add n more dimensions or numbers to define the sound in a movie.  So a sound movie is a form of photograph, which is 6+n dimensional.

For good measure, we can even make the movie 3D.  A simple way to do that is to make it stereoscopic, taken from two slightly shifted positions.  OMG this gives us a 7 + n dimensional photograph.

I know that all of this seems just a silly numbers game that restates in pretty terms the obvious.  But there’s one more piece to the puzzle that we have to speak about before we can understand the why.  This has to do with neuroplasticity, which is pretty much the ability of our brains, especially those of children to learn, adapt, and if necessary, to find new pathways.  I read an article once by someone who had lost his hearing over the course of a few short weeks.  To restore his hearing he had a cochlear implant,a device where the damaged parts of the ear are replaced by a microphone and electrodes that stimulate the neurons of the inner ear.  When he woke up from surgery there was a cacophany of sound.  He had to relearn how to hear because the new neuronal pathways, the circuits within his brain, were completely different than the ones that he previously used.  But his brain was plastic, aka moldable, enough to relearn.  A very similar things is going on with what are known as subretinal implants.   Like a cochlear implant, a subretinal implant consists of a silicon wafer containing light sensitive microphotodiodes.  These, generate electrical signals directly in response to light that directly stimulate the retinal cells.  Humans have tremendous neural plastic abilities.

There is a fascinating article in July’s Scientific American by Gershon Dublon and Joseph Paradiso of the MIT Media Lab, called Extra Sensory Perception. and I recommend it highly to you, as it gives us a glimpse of where we are going.

The computer and the crude network of computers that we call the internet has profoundly altered our lives.  Our children interact with a cadre of new devices with ease.  And I would argue that as a result of when they were exposed to it, that they are programmed differently than we of an older generation are.  Yet, we are all still pounding away at a key board, a single node in cyber space.  Our children just pound more efficiently. You’ve all heard about the new Google Glasses.  But just like our computer monitors whatever information we receive from these devices is reduced by our computer processor to some understandable two dimensional form – or as we have seen maybe five dimensional.  As I sit here and pound away, robotic eyes of the Mars Rover are beaming back data to me.  Well not really to me directly, because I do not have the right interface to receive these particular devices.  But at the same time I am surrounded by other signals.  I am bathed in them, in fact. Radios transmission, television transmissions, cell phone signals, motion detectors from my alarm system, WiFi signals from my computer and back from my internet service provider, GPS signals, etc., etc., etc.  It’s almost frightening.  Yet for every one of these signals, we need a dedicated device to interpret the signals.  What if we could assimilate all of this information directly, what if we could in essence see it?

Such a direct assimilation would profoundly alter our definition of the phrase and concept of “being there.”  How was it possible that we watched Neil Armstrong step onto the lunar surface in July of 1969.  I remember thinking at the time how strange it was.  It was as if I were actually there ahead of him.  Today we watch live pictures stream back from Mars and from the giant planets.  I had the same bizarre disconnect when I watched the very first images come back from the planet Jupiter.  My original understanding of being there was changed forever.  It became something so much profounder. In contrast to what will is, what will be is astounding – not even truly understandable yet.  We cannot yet really make sense of the ways in which we will assimilate all of this information into a greater extrasensory perception.

The photograph in its broadest sense, defines what we mean by being there.  First, we had still black and white photographs that capture an instant in time for ever.  Then we made pictures 3D.  Then we added color.  then we added time.  then we added sound.  You can visualize a little dimension counter adding new dimensions with each of the last five sentences.  We assimilated new information at each step and added new dimensions of perception. So when we ask what perception will mean in the hyperspace of the future, we have to at the same time ask what a photograph will be with these new dimensions added to it.

I like to think of the man sitting across the dinner table from a you woman with whom he is enamoured.  He struggles with the question of whether to say “I love you.” It poses such a risk, such an act of no return.  It occurs to him that perhaps he should say “Je t’aime” instead.  As an English speaker he believes that there might be less risk in saying it in French.  His five sense limit him.  With his eyes he looks across the table and studies the moistness and sparkle in her eyes, the flush in her cheeks (maybe just the wine) and the way that she looks back at him.  With his nose he sniffs for ambiguous pheromones hidden amidst the sweet smell of her perfume.  He listens for cues, a subtle change in her voice perhaps a or a quickening in her breathing.  He reaches across the table to touch her hand.  Does he feel the hairs rise?  He is not sure.  Whatever, the futrure holds, the magic will still be there and someone, he or she, must always take the risk and leap across the emotional abyss.

Figure 1 – NASA’s Gravity Probe Mission shown in a NASA graphic depicting the warp in spacetime caused by the planet Earth. From NASA and in the public domain.

We seem to have ended yesterday’s discussion with the conclusion that the Empire State Building is an object in a three dimensional world (space) rendered by the lens into a flat two dimensional object, called the photograph. But not so fast.  There is a bit more to a photograph.

What distinguishes the points in the photograph?  If it is a black and white photograph each pointy has to have an intensity associated with it. If you are happy to divide the range of intensities into 256 grey levels from 0 to 255, each point has an intensity associated with it.  So what we are really saying is that each point (x,y) on the photograph has an intensity and really needs to be denoted as (x,y,i), where i is the intensity.  So a black and white photograph really needs to be thought of not as a two-dimensional but three-dimensional object.

BTW – 256 grey levels is referred to as one computer byte of information.  If you are shooting in uncompressed raw black and white mode, there is a one-to-one correspondence between your number of pixels in your sensor and the number of byes required to store that image on your computer.  So I megapixels = 1 megabyte.

But what about a color image?  Every color imaginable (actually not quite) can be created or stored as a combination of the three colors: red, green, and blue.  So if you are shooting a color image you really have neither a two-dimensional photograph or a three-dimensional photograph, but really a five dimensional photograph.  Each point is described in a five dimensional space by five coordinates (x,y,r,g,b).  Where r, g, and b are the amounts of red, green, and blue respectively.  This is a hard thing to picture in your mind, or even on a graph.  But it is understandable, and really we have slipped seamlessly into what physicists and mathematicians refer to as a five-dimensional hyperspace. It’s almost Star Trekian! But, as Mr. Spock would say: “This all seems quite logical.”

Mr. Spock, of course, would have focused on a different form of hyperspace.  The four-dimensional space of space time, that Einstein used to describe gravity.  It is the stuff that warp-drives, worm holes, and time travel are made of.  I won’t go into all of that here.  But I will include as Figure 1 the wonderful graphic from NASA for its gravity probe mission.  You know, “to boldly go where no man has gone before.” 8<}

The point being that three dimensions is only sufficient to describe our world if it were merely everywhere either black or white, like a pencil drawing on white paper.  We need intensity and color to describe what we see.  Rather than shooting over our heads, the concept of hyperspace is really in our heads all along.  Our three dimensional vision- world is really six dimensional and photographs of it five dimensional.+

+And I guess, I should point out that to store a megapixels worth of color information I need not one megabyte, but a megabyte per color; so three megabytes in total.