New Experimental Website Converts Photos Into 3D Models

An artist might spend weeks fretting over questions of depth, scale and perspective in a landscape painting, but once it is done, what's left is a two-dimensional image with a fixed point of view. But the Make3d algorithm, developed by Stanford computer scientists, can take any two-dimensional image and create a three-dimensional "fly around" model of its content, giving viewers access to the scene's depth and a range of points of view.

"The algorithm uses a variety of visual cues that humans use for estimating the 3-D aspects of a scene," said Ashutosh Saxena, a doctoral student in computer science who developed the Make3d website with Andrew Ng, an assistant professor of computer science. "If we look at a grass field, we can see that the texture changes in a particular way as it becomes more distant."

The applications of extracting 3-D models from 2-D images, the researchers say, could range from enhanced pictures for online real estate sites to quickly creating environments for video games and improving the vision and dexterity of mobile robots as they navigate through the spatial world.

Extracting 3-D information from still images is an emerging class of technology. In the past, some researchers have synthesized 3-D models by analyzing multiple images of a scene. Others, including Ng and Saxena in 2005, have developed algorithms that infer depth from single images by combining assumptions about what must be ground or sky with simple cues such as vertical lines in the image that represent walls or trees. But Make3d creates accurate and smooth models about twice as often as competing approaches, Ng said, by abandoning limiting assumptions in favor of a new, deeper analysis of each image and the powerful artificial intelligence technique "machine learning."

Restoring the third dimension

To "teach" the algorithm about depth, orientation and position in 2-D images, the researchers fed it still images of campus scenes along with 3-D data of the same scenes gathered with laser scanners. The algorithm correlated the two sets together, eventually gaining a good idea of the trends and patterns associated with being near or far. For example, it learned that abrupt changes along edges correlate well with one object occluding another, and it saw that things that are far away can be just a little hazier and more bluish than things that are close.

To make these judgments, the algorithm breaks the image up into tiny planes called "superpixels," which are within the image and have very uniform color, brightness and other attributes. By looking at a superpixel in concert with its neighbors, analyzing changes such as gradations of texture, the algorithm makes a judgment about how far it is from the viewer and what its orientation in space is. Unlike some previous algorithms, the Stanford one can account for planes at any angle, not just horizontal or vertical. This allows it to create models for scenes that have planes at many orientations, such as the curved branches of trees or the slopes of mountains.

On the Make3d website, the algorithm puts images uploaded by users into a processing queue and will send an e-mail when the model has been rendered. Users can then vote on whether the model looks good, and can see an alternative rendering and even tinker with the model to fix what might not have been rendered right the first time.

Photos can be uploaded directly or pulled into the site from the popular photo-sharing site Flickr.

Although the technology works better than any other has so far, Ng said, it is not perfect. The software is at its best with landscapes and scenery rather than close-ups of individual objects. Also, he and Saxena hope to improve it by introducing object recognition. The idea is that if the software can recognize a human form in a photo it can make more accurate distance judgments based on the size of the person in the photo.

A paper on the algorithm by Ng, Saxena and a fellow student, Min Sun, won the best paper award at the 3-D recognition and reconstruction workshop at the International Conference on Computer Vision in Rio de Janeiro in October 2007.

For many panoramic scenes, there is still no substitute for being there. But when flat photos become 3-D, viewers can feel a little closer—or farther. The algorithm runs at http://make3d.stanford.edu.

Biologists Use Computers To Study Bacterial Cell Division

A group of computational biologists at Virginia Tech have created a mathematical model of the process that regulates cell division in a common bacterium, confirming hypotheses, providing new insights, identifying gaps in what is understood so far, and demonstrating the role of computation in biology.

The research, published in the January issue of PLoS Computational Biology, looks at the molecular machinery that governs replication of DNA and cell division in Caulobacter crescentus, an easily studied bacterium that is closely related to the bacteria that fix nitrogen in legumes and to the bacteria that cause brucellosis in cattle and Rocky Mountain spotted fever in humans.

"All share the same characteristic of asymmetric division; the daughter cells are different than the mother cell in some fashion," explains John Tyson, University Distinguished Professor of Biology at Virginia Tech and corresponding author of the PLoS article. "In C. crescentus, the mother cell attaches to a rock by a sticky stalk. If there is good eating, she divides and creates a daughter that can swim away. The stalked cell remains attached to the rock and the daughter--with a flagellum instead of a stalk--swims away, so that it does not compete with mama. After about 35 to 40 minutes, the daughter loses the flagellum, grows a stalk, and settles down to become a mother."

The Virginia Tech researchers are interested in the molecular machinery that governs replication of DNA and division of a cell into two different cell types. "A lot is known about genes that control this process. Much of the work was done in Lucy Shapiro's laboratory at Stanford," said Tyson.

"The mechanism is very complicated, involving dozens of genes and even more proteins. From experimental observations it is possible to construct a hypothetical 'wiring diagram' of how these genes and proteins interact."

But it is difficult to predict how cells will control their replication-division cycles from such a complicated hypothesis, he said. "Our goal is to convert the wiring diagram into mathematical equations that can be solved on a computer so that we can say with more confidence how the mechanism will govern cell growth, division, and differentiation."

The team's goal is also to demonstrate the role of computation in understanding biology. "We want to convert intuitive expectations into mathematical equations that can be tested more rigorously," Tyson said.

For example, models can be used to make testable predictions. A basic experiment is to create a mutant bacterium by knocking out a gene -- thus learning the role of the gene. This mutation can be simulated in the mathematical model to confirm the role of the gene in the wiring diagram. The mathematical model must agree with the observed behavior of all known mutants, and it can be used to compute the expected properties of mutants never before created in a lab, Tyson said. "If the prediction is confirmed by experiment, it promotes more confidence in the model. And sometimes you find that the model cannot reproduce the behavior of mutant bacteria, which suggests that the wiring diagram is incomplete and helps focus research on an improved understanding of the cell-division process."

In fact, there are known Caulobacter mutants that are not explained by the model described in the PLoS article, entitled "A Quantitative Study of the Division Cycle of Caulobacter crescentus Stalked Cells." "We knew our model was incomplete," Tyson said, "but we decided to publish at this stage because the model is good enough to illustrate the advantages of a computational approach. We have a new version of the model that fixes the problem and that accounts for the differentiation and development of swarmer cells as well as stalked cells."

So stay tuned.

"Computational biology is not much different from experimental biology -- you learn, publish, and keep working. There is always room for improvements. We would like to extend the model to the nitrogen-fixing and disease-causing cousins of C. crescentus," Tyson said.

Co-author Bruno Sobral, professor and executive and scientific director of the Virginia Bioinformatics Institute, remarked: "C. crescentus is a member of the alpha-proteobacteria, a group of diverse organisms whose members have successfully adopted different lifestyle and energy-yielding strategies in the course of evolution. It will be interesting to see if the molecular mechanism described in this study for control of the cell division cycle in C. crescentus is applicable to other species of this biologically important group of bacterial organisms."

Other authors of the PLoS article are Shenghua Li, a graduate student in Tyson's group; and Paul Brazhnik, research associate professor in biological sciences. The creation of a mathematical model of cell cycle regulation in C. crescentus is Li's Ph.D. research.

New Robotics Will Soon Revolutionize Industry And Services World

In the past few years industrial processes have monopolised practically all robotic developments and applications. However, the current tendency is marked by new robotics which will have a great impact in various spheres – from the industrial to services and including entertainment and care assistance for persons, amongst other applications.

This was the main topic analysed at the ‘Robot Business Opportunities’ (RBO) event, held the 24 and 25 January 2008 at Fatronik-Tecnalia Research Centre. This centre, located in the Basque province of Gipuzkoa, organised this event, within its strategy of constantly seeking new business opportunities for the enterprise sector, an event to which a selected number, in a position to form part of the new businesses that will be generated around robotics, were invited.

For the first time in Spain, Fatronik-Tecnalia has brought together the most prestigious experts in robotics worldwide, such as Tatsuo Arai from the University of Osaka, Herman Bruyninckx, co-ordinator of the European Excellence Network for Robotics Research, François Pierrot from the French CNRS or Todd Simonds from the North American company CTC and close collaborator with the North American Carnegie Mellon University, amongst others.

The new robotics

Conventional industrial robotics is a relatively mature market today. There are about 1.2 million industrial robots, in a market that has been going for more than four decades.

However, the industrial robots market is not growing. It dropped by 11% in 2006 and it is estimated that it will not recover for 2007. In reality, industrial robots are being displaced by non-conventional robotics which has three features:

Firstly, the ability of non-conventional robotics to co-exist with persons, services applications and advanced industrial applications in which robots work in direct cooperation with people, or share the same working environment with them. Secondly, its capacity to move around non-structured environments and, thirdly, its low cost.

When its lines of production are in full swing, robotics can reach new markets: robots can help us in cleaning chores, can transport loads within a workshop, can undertake monitoring tasks and can help us to overcome human physical difficulties or limitations. That is, its applications are very varied and numerous and undoubtedly are set to generate great opportunities in many sectors, both industrial and services.

We are beginning to see the first successful applications of robotics outside production lines. For example, the iRobot enterprise has sold more than 2.5 million units of its domestic vacuum cleaner robot while robot toys like Robosapien from WowWee have triumphed with over 2 million sales. Just two examples which, together, come to more than the total number of industrial robots in the world, which indicates that robotics is no longer restricted to the world of the industrial production line.

However, these examples are little more than the tip of the iceberg of an emerging market. According to all indicators, the new Robotics Society is to arrive this decade. Robots will be put on the market at an attainable price with applications in monitoring, construction, refuse and other collection, education, entertainment, personal assistance and much more.

New business opportunity

Given this future, the business world cannot stay still and get left behind. Today Spain is the fourth country in Europe and seventh worldwide with the greatest number of robots installed. Nevertheless, according to the Spanish Robotics Network (HispaRob), in which Fatronik-Tecnalia participates, although Spanish industry is one of the most robotised, it has high indiustrial-technological dependence (most robots installed are in the automobile industry and concentrated in the hands of a small number of multinational companies who have a closed technology and therefore make the incorporation of national technological business to a large extent difficult).

According to the article, “A robot in every home”, by Bill Gates in Scientific American (January-2007), the services robotics market will increase massively over the next few years. In the article Bill Gates argues that there are similarities, between the computers market in the 80s (few had access to it, it was reserved for those with great technical knowledge, it was expensive, and so on) and today’s services robotics market.

Fatronik-Tecnalia has undertaken an important commitment to this new, non-conventional robotics given that, although it is expected that this market will undergo a significant boom, the Gipuzkoa-based technological centre believes it to be fundamental that this field be invested, researched and developed as soon as possible in order to be in a position to take advantage of what it offers. .

Moreover, the new robotics is not only an opportunity for the manufacturers of the robots themselves but also for end users given that, for the first, it will enable the enhancing of their processes and, for the latter, their performances. It is a great opportunity for our industry given that, in order to develop these robotics, it is necessary to have the technological capacity such as our industrial environment has.

Robotics in Fatronik-Tecnalia

One of the work areas at Fatronik-Tecnalia is that of intelligent robotics.

In the industrial sector, examples would be in manufacture, assembly, handling, inspection, analysis, union, cleaning, etc. and applications in various industrial sectors. In the service sector, robotics will be used for entertainment and personal assistance, amongst others.

Examples of developed products at Fatronik-Tecnalia in this new robotics range from autonomous robotised modules aimed at carrying out a concrete task to high-speed handling robots for automating production lines or mobile platforms that work in direct cooperation with persons, or share the same working environment with them.

New Vaccine Against Deadliest Strain Of Avian Flu Successful In Mice

A vaccine against the most common and deadliest strain of avian flu, H5N1, has been engineered and tested by researchers at the University of Pittsburgh's Center for Vaccine Research and Novavax Inc. The vaccine produced a strong immune response in mice and protected them from death following infection with the H5N1 virus. The vaccine is being tested in humans in an early-phase clinical trial.

Recent outbreaks of avian flu around the world have prompted health officials to warn of its continued threat to global health and potential to trigger a flu pandemic. "While worldwide avian flu control efforts have been mostly successful, avian flu, like seasonal influenza, mutates year to year, creating new subtypes and strains that could easily and quickly spread among humans," said Ted M. Ross, Ph.D., lead author of the study* and assistant professor, Center for Vaccine Research, University of Pittsburgh. "To stem the spread of a potential pandemic, we need stockpiles of vaccines available that can be readily adapted to enhance the immune system's response to new strains."

A future flu pandemic is inevitable because of the virus's ability to continually reinvent itself and the lack of broad immunity in humans, according to Dr. Ross. Influenza pandemics have occurred three times throughout modern history with deadly consequences. The first, the Spanish Flu of 1918, caused more deaths than World War I.

Unlike other avian flu vaccines, which are partially developed from live viruses, the vaccine uses a virus-like particle, or VLP, that is recognized by the immune system as a real virus but lacks genetic information to reproduce, making it a potentially safer alternative for a human vaccine. Given the evolving nature of H5N1, the vaccine was engineered to encode genes for three influenza viral proteins to offer enhanced protection against possible new strains of the virus.

To test the vaccine, researchers administered it to mice in one-dose and two-dose regimens. Mice immunized twice with the vaccine developed protective antibodies against H5N1 and were protected from disease and death when directly exposed to the virus. The researchers also compared modes of vaccine administration by delivering the vaccine to the muscle or the nose. Both methods of vaccine administration were equally effective. However, mice injected with the vaccine through the muscle developed more antibodies in the blood, while mice that received the nasal administration had more antibodies in their lungs.

"VLPs may be advantageous over other vaccine strategies because they are easy to develop, produce and manufacture," said Dr. Ross. "Using recombinant technologies, within ten weeks, we could generate a vaccine most effective towards the current circulating strain of virus, making it a cost-effective counter-measure to the threat of an avian influenza pandemic."

*This study was published in the Jan. 30 issue of PLoS One.

The study was funded by Novavax located in Rockville, Md. Co-authors include Rick A. Bright, Ph.D., Niranjan M. Kumar, Ph.D., Peter Pushko, Ph.D., and Gale Smith, Ph.D., with Novavax; Donald M. Carter, M.S., Corey J. Crevar, B.S., Franklin R. Toapanta, M.D., Ph.D., Jonathan D. Steckbeck, B.S., M.B.A., and Kelly Cole, Ph.D., with the Center for Vaccine Research at the University of Pittsburgh; and Terrence M. Tumpey, Ph.D., with the Centers for Disease Control and Prevention.

How Does The Brain Attend To One Voice In A Noisy Room? New Findings On Selectively Interpreting Sounds

Scientists at Cold Spring Harbor Laboratory (CSHL) have reported new findings about how the mammalian brain interprets and fashions representations of sound that may help explain how we are able to focus on one particular sound among many in noisy environments such as offices or cocktail parties.

Neurons in the brain’s auditory cortex interpret incoming sound signals and send them to the rest of the nervous system, in the brain and spinal cord. Using rats, the CSHL team discovered that a very small minority of available auditory neurons react strongly when exposed to any specific sound.

“This finding challenges the standard model of sound representations in the auditory cortex, which predicts that neural representations of stimuli often engage a large fraction of neurons,” said Anthony Zador, Ph.D., CSHL professor and corresponding author of a new research paper.*

The researchers used a new technique called “in vivo cell-attached patch clamp recording” which measures the reaction of individual neurons. This recording technique samples neurons in a fair and unbiased way, unlike traditional approaches, which favored the largest and most active neurons. Using this technique, the team found that only 5% of neurons in the auditory cortex had a “high firing rate” when receiving a range of sounds of varying length, frequency, and volume. The experiment included white noise and natural animal sounds.

The team’s objective was to quantify the relative contributions of different sub-populations of neurons in response to the range of sounds. Most of what is known about the auditory cortex of the mammalian brain comes from studies of the anesthetized cortex. The results of the experiments reported today are important partly because they measure the response of neurons in rats that were not anesthetized. In animals that are awake, it’s possible to measure the response over an interval of time to one sound among many that are co-occurring.

This is the approach the Zador lab has taken to explain “selective attention,” or what Dr. Zador calls “the cocktail party problem.” Half of the neurons measured in the reported experiments showed no reaction at all to incoming stimuli. The researchers hypothesize that each neuron in the auditory cortex may have an “optimal stimulus” to which it is particularly sensitized.

“Your entire sensory apparatus is there to make successful representations of the outside world,” said Dr. Zador, who is director of the CSHL Swartz Center for Computational Neuroscience. “Sparse representations may make sensory stimuli easier to recognize and remember.” Recognizing the brain’s ability to distinguish “optimal stimuli” could help scientists find ways to improve how sounds are learned. Prior research has already yielded similar results when measuring sight, movement, and smell. This is the first evidence of a correlation between sparse representations and hearing.

“The goal of sensory processing is to take a signal, like a sound or a vision, from your environment and use it to drive behavior,” said Dr. Zador. “The brain needs to recognize and learn about these inputs in order to survive.”

*"Sparse Representation of Sounds in the Unanesthetized Auditory Cortex” appears in Public Library of Science: Biology on January 28. The authors are: Tomáš Hromádka, Michael R. DeWeese, Anthony M. Zador. (doi=10.1371/journal.pbio.0060016)

Adapted from materials provided by Cold Spring Harbor Laboratory.