leafsnap

Share this postShare on Facebook0Tweet about this on TwitterShare on Google+0Share on LinkedIn0Pin on Pinterest0Share on Reddit0Email this to someonePrint this page

This morning, I saw a twitter post on an interesting new free phone app.  Here is a brief video on the app posted at YouTube:

and here is what the developers say on their website:

Leafsnap is the first in a series of electronic field guides being developed by researchers from Columbia University, the University of Maryland, and the Smithsonian Institution.  These researchers are working on visual recognition software to help identify species from photographs. Leafsnap is the first in a series of electronic field guides being developed to demonstrate this new technology.  This free mobile app helps identify tree species from photographs of their leaves and contains beautiful high-resolution images of their flowers, fruit, petiole, seeds, and bark. Leafsnap currently includes the trees of New York City and Washington, D.C., and will soon grow to cover the trees of the entire continental United States.

Leafsnap turns users into citizen scientists, automatically sharing images, species identifications, and geo-coded stamps of species locations with a community of scientists who will use the stream of data to map and monitor the ebb and flow of flora nationwide.

The genesis of Leafsnap was the realization that many techniques used for face recognition developed by Professor Peter Belhumeur and Professor David Jacobs, of the Computer Science departments of Columbia University and the University of Maryland, respectively, could be applied to automatic species identification.Professors Jacobs and Belhumeur approached Dr. John Kress, Chief Botanist at the Smithsonian, to start a collaborative effort for designing and building such a system for plant species. Columbia and the University of Maryland designed and implemented the visual recognition system used for automatic identification. In addition, Columbia University designed and wrote the iPhone, iPad, and Android apps, the leafsnap.com website, and wrote the code that powers the recognition servers. The Smithsonian was instrumental in collecting the datasets of leaf species and supervising the curation efforts throughout the course of the project. As part of this effort, the Smithsonian contracted the not-for-profit nature photography group Finding Species, which collected and photographed the high-quality photos available in the apps and the website.

Once downloaded and opened, one can challenge oneself to one of a number of identification “Games”, “Browse” the current field guide of 184 plants, set particular “Options”, and take a picture using the phone’s built in camera that should automatically upload to the leafsnap database via “Snap It!”.  Once you have uploaded images, I assume you can view your own “Collection”.

Once you are browsing, you can view the species contained in the app using their common or scientific name, in a manner familiar to iPhone users and also used by National Geographic’s bird identification app.  By clicking on the image icon in the upper left-hand corner of this screen, the app toggles between displays of a representative leaf, flower, or fruit associated with each tree specimen, a nice feature that could help to confirm one’s hypothesis of identification.

   

Once you select a particular species, you can view a collection of detailed images of the trees leaves, flowers, fruits, seeds, and bark, or use the image icon in the upper right-hand corner to toggle to a dispaly basic information on the species being viewed.

 

The games section includes timed quizzes that test one’s ability to identify tree specimens by leaf, flower, or fruit in the “Leaf Challenge”.  The second image below is a screen shot of this game.  The name of the tree leaf to identify with a finger tap is noted, while the number correct and time left are noted in the upperleft and right-hand corners of the screen, respectively. 

In “Green Sweep” you are given four leaves to identify that are floating around the screen.  You identify each by moving them into their appropriately labeled square, with a touch and a drag.  If correct, it challenges you with another four leaves to identify, and continues in this manner until your time is up.  Each of the game categorizes your exepertise based on your score at the end of the game.

     

In using “Snap It!” the app uses the phone’s built in camera to take a picture which is then uploaded for identification by the recognition software described previously. 

Sadly, I have not been able to get this function of the app to work completely.   I went out a tried to take a picture of a redbud leave only to realize that your are supposed to take the picture of the leaf on a white background (I hadn’t read the top of the screen).  The app realized this though and informed me which was demonstrates its sophistication. 

After correcting this mistake, I had issues with the app crashing when I took a picture within the app.  So, I exited the app and took the picture within phone’s camera app, and selected to chose that image from within leafsnap.  This time, I experienced crashes while the app attempted to upload the image for identification.  I even changed the image resolution to small in the options setting to no avail.

With the number of people working on this app, I imagine that such problems will be solved in a short time.  I only wish they had a way to report bugs via the app or on their website.  If and when I get this aspect of the app working, I will edit the post and make a comment to report changes.

I look forward to using this app with my students!

Addendum:

Since the time of the orignal post, I e-mailed leafsnap, noticed that an update was posted on twitter, downloaded the update, and tried to upload an image of a Redbud from my back yard again.  This time it worked!

   

After uploading the image, it loads results to your phone.  The results are ordered, I assume by some probability of it being a match with the outline of the uploaded leaf image (you can see the top three results in the middle image above).  If you click on the “map” button at the top of the page it shows where the leaf was collected (dang, now you know where I live).

   

To verify a particular result, one finger swipes across the chosen selection to bring up the red label button (first image above), and selects the “label” button by touching it (second image above).  Once this is done, the labelled image now appears in your “Collection.”

Pretty cool! 

Now, I challenge you to be the second to add an image from the Kansas landscape!

Comments

comments

7 thoughts on “leafsnap

  1. @Lin – I look forward to seeing data from the Wichita area. I am glad this post may help you in your quest for iPads.

    If you would like to enhance any teaching of neurophysiology you might also be interested in checking out the spiker box at http://www.backyardbrains.com/Home.aspx. Their is a backyard brains app to use with the iPhone (I assume iPad as well), to capture electrical recordings from the neuromuscular impluses in the leg of a cockroach. You would have to invest in some spiker boxes and roaches too, but I think your students would find it quite an interesting mix of biology and technology.

    Otherwise, I enjoy using twitter on the iPad. I follow a number of science magazines, Ed Yong, Carl Zimmer, etc.. and the iPad is more condusive to reading an article or viewing a YouTube video, etc.. than the iPhone, although I only print from my computer.

    I would be interested in hearing other classroom uses of the iPad that you have discovered.

  2. I downloaded this app after seeing it here! I can’t wait to try it out this summer. No one in Wichita is using it yet. So, I will probably be first one.

    I am starting to work on a grant this summer for a cart of Ipads and intend to reference leaf snap as a valuable tool for the classroom. Thanks for the heads up on this one.

    ~Lin

  3. @Jesi – I am glad you like it as well. Did you have students use it?

    Their website has a map that it appears that you found. At the moment it doesn’t seem to be that sophisticated on that end. I image that it will continue to improve with time. Your comment on their only being one point in Manhattan is interesting, since I can see my collection points through my phone when I have added a leaf but I have yet to see the two spots that I have collected in Kansas show up on their website map.

    I can tell you that they are quite response to imput though. I sent an email to the the one I found on their website at: press@leafsnap.com. I received a response from these two addresses,1) from Peter Belhumeur at belhumeur@cs.columbia.edu, and the other from Neeraj Kumar at: me@neerajkumar.org

    I would definitely suggest that you forward your suggestion on being able to add notes to particular plants.

  4. Thank you for posting about this app! I think it’s fantastic.

    I haven’t been able to find where to view other user’s collections. I went to the “collectors” link, but I only see one point in Kansas, from Manhattan.

    One thing I think would be a useful addition to the app would be the ability to add notes to your collections. For instance, I’d like to add notes to the species that are host plants to certain insects.

  5. You could actually be the first from Kansas, since my upload was from Missouri. I will do my best to beat you though and upload another leaf once I get to school.

Comments are closed.