The Shuttleworth Foundation provides one year grants to individuals so they can focus fully on a project for one year.  This focus allows those individuals to move their projects forward and make significant progress.  I am applying for a fellowship, as well as continuing to follow my personal guideline of posting everything openly whenever possible. The application process requires submission of a video, a CV and to answer 4 questions about how things are currently and what one wants to do to create change. Find the answers to the questions and the video below, I’ve also uploaded a PDF copy of my resume.

The hard thing about posting this openly for me was the video. It is the first video of its kind that I’ve ever made completely on my own.

Opening OpenStreetMap

Describe the world as it is. (a description of the status quo and context in which you will be working)

OpenStreetMap was started with the goal of “Free editable map of the entire world(1).” Progress continues to this goal, but this is not without challenges. Currently there are over 600,000 OpenStreetMap accounts, but the majority of people that make an account don’t actually edit. This highlights basic usability issues with the software, which are even more prevalent in some culturally diverse regions. Additionally very few women ever contribute to the OpenStreetMap community.

A few major barriers within the OpenStreetMap community are arcane communication methods which are only friendly to technical people, the loud hostile voices of a few towards organizational change and cultural barriers that almost require fluency in English to really participate in the greater community. I believe these issues are what leads to new contributors leaving before they really got started and not very many women participating in OpenStreetMap.

The two main editors in OpenStreetMap are called JOSM and Potlatch2. JOSM is a desktop application and Potlatch2 is a Web based application, which appears when one clicks “edit” on the OpenStreetMap.org website. Both allow powerful geographic editing, but each has its own usability issues. Over the past year the Humanitarian OpenStreetMap Team has been working to create an OpenStreetMap community in Indonesia. When interns in that project were translating JOSM into Bahasa Indonesia one of them began laughing, the reason? They had to spend quite a bit of time researching the phrase “Orthogonalize Shape,” to translate it. These are final year students in a geography department that are fluent in Indonesian and English, translation of geographic software should be a relatively easy task for them. The majority of the features in JOSM and Potlatch2 are only used by a small amount of users; they manage to clutter the system and make it impossible for beginners to get started.

Fostering change in the OpenStreetMap community will be difficult. It is a large, highly technical community which operates in an informal manner and decentralized manner. My experience with the OpenStreetMap community over the past 3 years will be required to help with adoption of change. Though it will not be easy, I believe these difficult changes will be necessary to truly create a free map.

What change do you want to make? (a description of what you want to change about the status quo, in the world, your personal vision for this area)

Geographic information is powerful. OpenStreetMap exists for that very reason, that much data is either unavailable, out of date or inaccurate. I want to put this power in the hands of more people. One should not need to have a high level of technical proficiency to map their community. One should not have to speak English to map their community. One should not have to be male to map their community.

There have been multiple academic studies looking at the contributor discrepancies in OpenStreetMap. Such studies have approached OpenStreetMap at an academic research project manner, without the goal of fostering change in the environment being studied. Surveying the OpenStreetMap community can be challenging, best practices and workflows for collaborating with the community are not documented. Sharing of these methodologies could make such studies more effective. Community concerns regarding studies within OpenStreetMap are the information will not be shared openly back with the community and that those performing the studies are not participating in the community they are researching. As a current participant in OpenStreetMap and one that makes any information I create open, I believe I can alleviate these fears and use my studies to create change.

What do you want to explore? (a description of the innovations or questions you would like to explore during the fellowship year)

OpenStreetMap as it is today is a mix of social and technological questions. In many open-source projects the technology and the social interactions can be tightly tied together and OpenStreetMap is no different.

For creating a more usable map for everyone I would like to investigate multiple technological questions:

• What are the best ways to internationalize data?

• Are there other technology issues around adoption of OpenStreetMap that aren’t well known because they are specific to the way certain cultures relate to technology?

To help introduce people to the map there are social questions I think that must additionally be answered:

• What are ways that have been previously used to instigate change within open communities?

• Who are allies to change through technical contributions or to drive social movement?

• What are the social barriers to change? Are individuals or sub-communities creating those barriers? Are there ways to gain their cooperation?

Further questions are around the intersection of social and technology:

• How do different technical and geographic groups communicate differently within OpenStreetMap?

• When communication occurs between groups where does it take place?

Based on this discoveries I would like to create tools and documentation to help ease the path to OpenStreetMap adoption. I have already been involved in the creation of better learning tools both LearnOSM (2) and the free OpenStreetMap book (3). While these are a start, they are actions without heavy initial analysis or measurement of impact.

• Can more effective tools and guides be created with more understanding of the issues?

• Does new technology need to be created or can existing tools be utilized?

What are you going to do to get there? (a description of what you actually plan to do during the year)

Usability

I believe basic usability is one of the core issues causing the discrepancies within OpenStreetMap. There has been some work in this area previously. I would review this work as well as perform further studies to look at discrepancies between women and men utilizing OpenStreetMap software as well a culture disparate groups. A review of these types of studies in other open-source communities would also be conducted and I would see if it is possible to draw commonalities.

Surveys and Interviews

In addition to usability studies I will conduct surveys and interviews within the OpenStreetMap communities to investigate where their success lies. How did regional groups get started, what was the spark for them to continue to grow? I would also introduce new groups to OpenStreetMap and perform follow-up to determine why they either have or have not continued to contribute.

Change

Based on usability tests, surveys and interviews I will modify tools, create new training materials, host events and explore different types of communications platforms. Using these methods I will see if uptake in diverse OpenStreetMap contributors is influenced. Based on those evaluations I will expand some methodologies and try new ones. The goal to create a diverse free and open map.

(1) http://wiki.openstreetmap.org/wiki/About (2) http://learnosm.org/ (3) http://booki.flossmanuals.net/openstreetmap/_full

I hope my application is useful to others looking to apply and hopefully I’ll be reporting back here with good news!

When I heard DevelopmentSeed put in a proposal for the Knight News Challenge to make simple contribution tools for OpenStreetMap I was elated. I’ve long admired DevSeed for their beautiful maps and innovative uses of new technology.  They are already well involved in OpenStreetMap through support of Mapnik, attendance at development sprints and use of OpenStreetMap information in MapBox. I think there is a real opportunity here to improve the ability of average people to contribute to and utilize OpenStreetMap.  What is the key?  Make the tools simple and the cycle to use the data just as easy as contributing.

The Humanitarian OpenStreetMap Team probably spends more time than any other group in OpenStreetMap teaching the technology to non-technical users. Our work in Haiti and Indonesia has been with local people who are not programmers, nor GIS analysts. It can be a real struggle to ensure people can both enter data easily as well as utilize the information they have input. There are many great tools that help with this process, but it is far from smooth. Let’s look at the entire OpenStreetMap contribution cycle.

Gathering Data

I would say collecting data in the field is probably the easiest part of OpenStreetMap.  GPS units are relatively easy to use for example.  The menus on the Garmin Etrex units that HOT usually use do have some odd menus, but for the most part people are just pushing the button to mark a way point.  Though if I take a break from collecting data for a couple weeks I can never seem to find the menu item to delete those way points.  This problem is not too difficult though and most people get used to the workflow.

Walking Papers by far is the biggest improvement in field data collection in years.  My first job in mapping was creating field maps for mosquito control technicians.  If I had Walking Papers back then it would have been magical. I was constantly getting paper maps back with corrections on them, it is just natural for people to draw on maps to denote new features. I think in data collection OpenStreetMap is in a good place, the real areas for improvement are editing and data usage.

Entering Data

I mean no slight to the developers of OpenStreetMap editing tools with my critique here. Potlatch2 and JOSM are both tools that I frequently use, but I am a programmer with a Geography Degree.  How many people with my background are there on earth?  By percentage probably not many.

The biggest flaw with either of these tools is they try to allow you to do everything.  Most people however don’t need to do everything. In an average workshop that I lead key goals are to map roads, add some points of interest and map houses. We never touch relations, I don’t explain what a changeset is and someone help us if there is a conflict (which there always is).  Usually we are using JOSM, Potlatch2 has come a long way in helping make the data entry process easier.  Dragging icons over to the map to add them is far more natural than drawing an object and then clicking through a menu to define it as is done in JOSM.

Though the simple and advanced mode in Potlatch2 is a bit confusing and if you want to get the history of an item it is hard to find the right button. For example on the left is the advanced screen, can you guess where you would click to get the history of the selected item? Did you guess 30176850? Probably not. I think it is fair to say that looking at the object history is an advanced task, I don’t often go over it in the first workshop unless the hands on portion leads us there.  I don’t think however that it is obvious to click on the unique identifier to go to that page.

One other minor complaint would be that some of the buttons “look more button like.” It isn’t completely clear here to tab between the Simple and Advanced screens for example and to delete a tag you click on an “x” button but to add you click on the word “Add.” Using icons would make this more usable for those that don’t have a version of Potlatch2 translated into their native language.

Overall I really like how Potlatch2 has made editing much more simple.  If it worked offline we would consider teaching it instead of JOSM to beginners. It is a huge improvement over Potlatch2 and the team developing it has much to be proud. I think however there are ways to get even more simple.

Using Data

The biggest gap in OpenStreetMap is using the data. HOT has attempted to bridge that by teaching Quantum GIS, which has made huge strides in the past year in becoming an easy to use complete Desktop GIS.  That is just it though, you shouldn’t have to use Desktop GIS just to make a map. MapBox has greatly improved the accessibility of making web maps, but sometimes you just want to print. Speeding up the cycle of data usage would also assist.  The most frequent way we suggest people access the data is to download it from GeoFabrik, which is a great free service. The only thing lacking is speed, there is great joy in adding information and seeing it show up on the OpenStreetMap website virtually instantly, but some disappointment when you have to wait a day to be able to use it in specifically the way you want. There are ways to get around this, but they do require someone technical to set them up.

Simple Language

Normally when I post publicly about OpenStreetMap I make it a point to try to avoid the language of OpenStreetMap. It is a language people can use but is not immediately accessible to most people. My favorite example of obscure language in tools is when my interns Emir Hartato and Vasanthi were translating JOSM into Bahasa Indonesia and got to “Orthogonalize Shape.” Think about it, does the average person know what that means?  What about using language like straighten corners or something? By the way “Orthogonalize Shape” is “Ortogonalisasi Bentuk” in Bahasa Indonesia in case you were wondering.

What Do I Want?

So far I’ve pointed on flaws in OpenStreetMap, but what is my point? What do I really want?  Simple tools and simple language for the average person. If a person fluent in two languages has to stop and really think about how to translate something, then it is probably too complicated. Let’s concentrate on making work flows for the 90% of use cases that utilize the 10% of existing functionality, not try to cover every piece of possible functionality.

Rather than thinking about tasks by functions let’s think about them by themes.  Groups that are interested in mapping for disaster preparedness, groups that want to map the green spaces in their community, people that want to share a great hiking route with everyone.  In starting with the use-cases the needed functionality will become obvious.

More than anything, let’s make sure when someone puts data in they can use it.  Easily, without a struggle.

note: this post was written as my position paper for the 2012 Vespucci Institute . It seemed a shame to bother writing something and just a couple people read it

The power of providing spatial information to inform decision makers is without question. I believe the real question is “Who are decision makers?” I believe simply “everyone.”

Everyday people make decisions:

• Which route to take to work?

• Where to live?

• Which place to vacation?

• Which market has the best price for my produce?

Everyday communities make decisions as well:

• Are there adequate services where the poor people in our village live?

• What should the district area be for the new school?

• Will widening this road be enough to reduce the traffic?

Not everyone had access to the data and tools to make these decisions. Over the past year I’ve been working in Indonesia on a project to utilize OpenStreetMap to collect information on where people live for risk modeling.  A big portion of the task has involved finding people already doing mapping and helping them do it openly and digitally.  This has been the most exciting portion of the project because it has allowed me to work with community facilitators in Eastern Indonesia performing poverty mapping.

The Australian Community Development and Civil Society Strengthening Scheme (ACCESS) is an initiative between the Australia and Indonesia governments to strengthen civil society by helping create demand for better governance.  A portion of the project involves development of poverty maps so villages can see and discuss their situations. These maps are typically either handmade or created using CorelDraw with no geospatial aspect.  In July 2011 the Humanitarian OpenStreetMap Team (HOT) began teaching OpenStreetMap to the community facilitators to assist in their poverty mapping.  Eventually software was built to split the OpenStreetMap public information from the private poverty information and GIS software was used to make poverty maps. Now the community can quickly have different types of maps without the long process of redrawing and recoloring everything manually.  Instead they configure different themes in their GIS maps and can show them quickly.

The power of bringing geographic analysis to rural Indonesia to the people making decisions at a local level is huge. It reduces the time to update the maps and because of the flexibility of the data allows other uses. For example one of the groups trained mapped all of the local government projects and then made a dashboard for the local government so they could look at project costs. Another group has a plan to map all 90 villages in their regency, so far they’ve completed 11. What is amazing is all of the data that can be open is available in OpenStreetMap.  This allows other groups such as the National Disaster Agency of Indonesia (BNPB) to take that information and run exposure models when performing contingency plans.

When this project began it would have been impossible to determine all the possibilities. Through the use of Volunteered Geographic Information, simple GIS and cartography groups are empowered to make better decisions.  When more people are empowered it is difficult to determine what all the positive externalities are that will happen. That is why I believe it is so important to enable geographic decision by those who need it, which is everyone.

I have discussed the need for access to satellite information for crisis response often. When there is a large scale disaster imagery tends to become available relatively quickly, but in cases of smaller events or mapping for preparedness this is not always the case. In my work in Indonesia over the past year we have been fortunate to be able to purchase imagery of Padang City (OpenStreetMap login required) and additionally Microsoft has updated much of the imagery available through Bing Maps. This access has been enabled by the purchasing power of large organizations, AusAID and Microsoft respectively. Work continues to use similarly sized organizations to increase access in other areas, but the process is slow and there is no guarantee imagery will be available everywhere, every time. Yet. Currently the digitization of the imagery is performed by the crowd, what about enabling them to assist in the purchase?

Lack of Physical or Financial Access

The Public Laboratory for Open Science and Technology (PLOTS) has done much to enable the crowd-sourcing of inexpensive imagery. Unfortunately many times when crisis mapping occurs access to collect this type of imagery is impossible or extremely dangerous. Imagine flying a balloon in Syria today in order to collect information, the risks would be suicidal.  In other cases the physical terrain of the area may make it very difficult to access, but a small group may not have the financial means to purchase imagery by other means.

Enabling Collective Purchasing Power

Currently the Humanitarian OpenStreetMap Team utilizes a tool the Tasking Manager to coordinate digitization and verification of imagery analysis. What if this tool could also be utilized to feed into the purchase process of imagery. Individuals could choose to adopt squares and volunteer to pay for them, this would be an all or nothing proposition, if all the squares were pledge the project would go forward, otherwise it would not. This crowd-funding model would be very similar to Kickstarter, but perhaps instead of rewards pledgers could adopt squares to digitize once the imagery was purchased.

The image above is the overview map from the Tasking Manager of the task for Padang City.  It gives a quick overview of Total number of jobs, how many jobs are done, how many are validated and how many are currently being worked on.  What if there was an overview of squares that were funded?  The same as when someone takes a job to complete they could either select a random job or a specific one by clicking on the map.  The entire community would be able to see quickly if an area was near being available for purchase.

In Kickstarter it is possible for projects to collect more than they ask for initially.  If people were adopting specific squares I don’t see this being possible, but perhaps if you selected a random square they would not be assigned to the end.  That way it would be possible for more squares to be purchased than exist.  Sponsors of the imagery fundraising drive could decide on specific things that would happen in these cases (ahead of time to encourage the crowd), possible ideas would be:

• purchase of GPS units to create a bank for people living in those areas

• a HOT mission to train those locally how to map in OpenStreetMap

• Purchase of imagery of an expanded area

Can We Expand Access to Imagery?

I will continue to advocate for expansion of access to imagery. Schuyler Erle has previously stated that access to imagery in times of crisis is vital almost like water for mapping in response (Schuyler sorry I couldn’t find an exact quote). I fully agree with this assessment. I will continue to work with governments and large organizations with purchasing power that can help enable the crowd to respond.  If imagery is as vital as water though, I think we also have to take matters into our own hands as pursue every avenue possible.

disclaimer: I am not a lawyer, I’ve never pretended to be a lawyer and I’m not qualified to give legal advice. This is my interpretation of what the ODbL license means for usage in OpenStreetMap and what I am going to follow unless someone tells me otherwise.

Background

Though my work at the Humanitarian OpenStreetMap Team I am often asked questions about the OpenStreetMap license, which is in process to switch from CC-BY-SA 2.0 to ODbL 1.0. The OpenStreetMap Foundation has written much about the reason for the switch, so I’m not going to go deeply into that other than CC-BY-SA does not work for data globally. The reason for this blog post though is when I haven’t been deep in licensing for a while I tend to forget details, perhaps you have the same issue.  This is for both of us to have a summary. The information below is specifically related to the OpenStreetMap project, there has been much community discussion and guidelines produced leading up to the license change. The usefulness of this information for other projects may vary.

note: OpenStreetMap is switching fully to ODbL on April 1st, so these guidelines will best apply after that date.  They are however useful for planning now.

Summary

The basic premise of ODbL is that the license provides that anyone can use the released data for commercial or non-commercial use. This is providing that you give attribution and if you improve the database you release the resulting database under ODbL or a similar compatible license.

Part of ODbL is also the licensing of the data in the database, which requires the Database Contents License or DbCL. I’m not going into this in this post other than to state it is the licensing of the content in the database. Additionally OpenStreetMap is adding contributor terms with the license change, this is separate from the license though will make it easier in the future to change or upgrade the license as needed.

Important Terms

There are a series of terms that by understanding make the reading of the license much easier. They are defined in the database under “1.0 Definitions of Capitalised Words.” There are a few that I think are most important in understanding of the license.

• Database

• Derivative Database

• Produced Work

• Extraction

• Substantial: Refers to the data and the amount/quality of the data. In the OpenStreetMap wiki there are guidelines as to what is Insubstantial in reference to the OpenStreetMap project.

Community Guidelines

Items that are not included in the important terms above, but are important for usage of OpenStreetMap specific data.

• Insubstantial: Less than 100 Features. More than 100 Features if the extraction is non-systematic and clearly based on your own qualitative criteria. The features relate to an area of up to 1,000 inhabitants. (7)

• Trivial Transformations: Situations where a derived database is created but releasing it doesn’t really add value and the size of the database could possibly make it difficult to release.  Examples are rendering databases, routing databases and geocoding databases.

• Metadata Layers: Situations where other data is linked to OpenStreetMap data. For example a case where data e.g. restaurant reviews is linked to the OpenStreetMap data by IDs.

• Indexing: A situation where the data is indexed for example by a search engine.

Practical

Both CC-BY-SA(1) and ODbL(3) have a human-readable summary of the license as well as the full-license. Both explain at a high-level what you can do, but I think the issue with ODbL versus CC-BY-SA is people are more familiar with CC-BY-SA, meaning there are more examples of how to use it and what it means. The examples below are meant to clarify that.

note: I have borrowed heavily from the community guidelines for these examples for the full set please look at them on the OpenStreetMap wiki

Use of OpenStreetMap Data By Itself

There are many cases where the use of the OpenStreetMap data by itself is all that is needed. Examples include creating a map for a website, indexing the data for use in a routing engine and creating a geocoding database.

Creating a Map Image

If you create a map rendering (image) from the OpenStreetMap data you are free to distribute this produced work as you see fit.  The only requirements are you must attribute OpenStreetMap. The license on the produced work can be anything you want.  So if I wanted to create a map for my newsletter showing a park that was well mapped in OpenStreetMap I’m allowed to do that. I can license the picture with whatever license I want.

Often cartographers take a rendered image and then manually make improvements to it.  Usually this includes things like fixing label positioning, but may also include fixing errors. The resulting work of the artist can be released under any license they want as with the automated rendering, but the improvements for example the fixing errors must be released. It is suggested that the easiest way to do this maybe to simply add the fixes back into OpenStreetMap.

Using the OpenStreetMap Data for Geocoding

You have created a geocoder that using OpenStreetMap data by itself. Part of the process of geocoding is first indexing the data. There is a resulting derived work in this case which is the geocoding database.  According to the ODbL license either the resulting database, the modifications or the algorithm should be provided.

note: there is some disagreement about if these is how far OpenStreetMap would want to take this idea, but currently the interpretation of the license.

Search Engine Indexes OpenStreetMap Data

You have a search engine which crawls all the OpenStreetMap data. When a user searches for a specific restaurant on your search engine it provides a link to that restaurant on the OpenStreetMap website. This is considered fair use in the OpenStreetMap community guidelines, this is because it indexing the OpenStreetMap webpages and these are produced works.

Use of OpenStreetMap Data with Other Data

Geocoding Using OpenStreetMap

An organization has a list of members and wants to geocode that list to create a map. The resulting geocoded address is a derived database, but at what point do they need to release under a compatible license? In this situation the guideline of what is substantial(7) comes into effect. According to the community guidelines if more than 100 records were in the database it would need to be released under ODbL.

Improving Another Dataset

Let’s say you took the extract OpenStreetMap database and extracted all the restaurants.  You then manually went through all the restaurant to verify their locations and corrected those that were incorrect.  You then used this derived database to make an internal presentation to your marketing department. In this case you did make a derived database, but neither distributed the database or the produced work.  It is unnecessary to then release the data under ODbL.

After your presentation your boss came to you and said they wanted to use the information from your presentation in a product helping other marketers analyze restaurant locations. This would be distributed to all of your customers.  In this case your produced work or perhaps your derived database depending on the type of product are being distributed so you would need to release the information ODbL.

You reference IDs to the database from another database

The example from the OpenStreetMap wiki is you have a site that collects restaurant reviews and you link those to restaurants mapped in OpenStreetMap by the ID(10). It is stated “The “Fairhurst Doctrine” (previously discussed on the mailing list) solves this by saying that any mapping of machine-produced primary keys (e.g: OSM node IDs to review IDs), as it doesn’t “represent … significant investment”, doesn’t meet the definition of “Substantial”, therefore is not a Derivative Database.”

My personal example of this is through some work that the Humanitarian OpenStreetMap Team has done in Indonesia. We’ve been working with community facilitators to help them map poverty.  Building information goes directly into OpenStreetMap and the poverty information goes into a separate database linked by the OpenStreetMap ID. Originally this was done by typing the OSM ID directly into Excel, now there is an application that links the two.  When someone wants to make a poverty map they download the OpenStreetMap data as a Shapefile and link it to their spreadsheet. When we built this system my understanding was that there was not issue because the data was not going to be distributed.  The issue being that poverty data is private information and it isn’t fair to expect people to make it public.  According to the “Fairhurst Doctrine” it appears that we would be okay to distribute it to other groups without having to release it ODbL.

References

1. Creative Commons Attribution-ShareAlike 2.0 Generic Human-Readable License Summary

2. Creative Commons Attribution-ShareAlike 2.0 Legal Code

3. Open Data Commons Open Database License (ODbL) v1.0

4. Open Data Commons Open Database License (ODbL) Summary

5. OpenStreetMap Foundation Why CC BY-SA is Unsuitable

6. OpenStreetMap Wiki Entry for Open Database License

7. OpenStreetMap Wiki Open Data License/Substantial-Guideline

8. OpenStreetMap Wiki Open Data License/Community Guidelines

9. OpenStreetMap Wiki Open Data License/Trivial Transformation Guidelines

10. OpenStreetMap Wiki Open Data License/Metadata Layers Guidelines

11. OpenStreetMap Wiki Open Data License/Indexing Guidelines

12. OpenStreetMap Wiki Open Data License/Use Cases

There are many events in the Washington D.C. area related to geo. One is coming up which is sure to be educational and fun, FOSS4G North America. FOSS4G-NA is taking place April 10th-12th at the Walter E. Washington Convention Center in Washington D.C. Confession I’m on the program committee, but I’m not just saying you should go because of that. FOSS4G this year was in the United States in Denver, Colorado, it was the first year I attended and I both learned a lot and met others in the community I didn’t yet know. If you didn’t make it out to Denver you should definitely attend and if you did I doubt I have to convince you.

Attending

I’ve been heavily involved in FOSS4G related geo for about three years. Before that I worked for a company that used to say “what do you mean you can find an open-source solution for X.” That was more out of cheapness than actually wanted to be part of any community though. If you work for one of those companies maybe both you and your boss should attend so they can understand the real value of open-source. You’ll get to meet more of the community and perhaps learn about some tools you aren’t already using.

Speaking

Note: The Call for Speakers Ends on March 1st, so you better get your abstract in soon!

These types of conferences aren’t just about attending, they are also about sharing. Are you are creating a new project, improving an old one or have an interest implementation? Share your knowledge with the rest of us. I’m going to submit something about HOT’s work in Indonesia I just haven’t figured out which angle I’m going to take.  Teaching QGIS to community facilitators for poverty mapping? Using crowd-sourcing to create data for risk analysis? Or something else that will come to me 5 minutes before the submission deadline! Either way make sure to get your submission in before March 1st.

See you there!

I’ve begun blogging at wonderchook.com for anything non-geo related. Today I announced a “pay what you want” experiment. The idea is you can schedule my time to help with something and if you appreciate the help you can “pay what you want.” The reason for the experiment is I have my flexibility in my schedule since I consult for the Humanitarian OpenStreetMap Team, so I have time to test out wacky ideas.

The Experiment

1. I will publish office hours a week in advance

2. You can go sign-up for slots, including a description and how I will contact you. I’m open to any means of communication as long as they are free.

3. At the slotted time we will begin working on your problem. If it is a problem where it is be worked on asynchronously then I will work on it during the time-slot. If it is something that more requires discussion it will occur during the reserved time.

4. If you are satisfied with the help and would like to pay for it then you can pay me through Paypal, send me a gift, trade anything that seems appropriate to you.

There is more background on my idea around this experiment on my general blog. And you can reserve time through Google’s Appointment feature in Calendar.

Confession: When I studed Geography at George Mason University I only took one class that had anything to do with imagery of any type. Focused around manual interpretation of imagery mostly though we did use ERDAS Imagine once, for a lab.

I could have taken more classes on remote sensing, but I didn’t. My focus was more on cartography and statistical analysis.

Anyway, recently HOT bought some imagery. I naively thought that I could just run it through gdal2tiles.py and go skipping down the road to digitizing. Well, I was wrong.

I received a lot of advice on what to do with the imagery. For example changing it from 16 bit to 8 bit. Which I tried, oddly it worked on one image and not the other. This turns out was just dumb luck that the stretch was almost right when you did that. Other suggestions involved reordering the bands and things such as that. I tried a bunch of different things, none of them worked.

I did discover that if I applied a standard deviation stretch to the imagery in QGIS then it looked right. There wasn’t a way to save this stretch though directly to the GeoTIFF. I did a bunch of searching to see if there was a manual way to do this and I came across this post on using VRTs to add the minimum and maximum of the bands.

So here is my pretty manual way of doing things. Yes I could have done some automation but I figured it was just a couple images.

First I download the image and open it in QGIS. Then I right clicked on the layer and selected properties. In the properties dialog I selected “use standard deviation” and he “apply.” The image should now look like natural color.

Next I copied the Red, Green and Blue min/max values.  Now with the correct values the key is to feed them in GDAL correctly. Here you are building a VRT or Virtual Format.  So the actual processing doesn’t take place, you are describing it though.

First step is to stretch each band individually:

gdal_translate -b 1 -scale min max -ot Byte -of VRT your_image.tif 1.vrt

gdal_translate -b 2 -scale min max -ot Byte -of VRT your_image.tif 2.vrt

gdal_translate -b 3 -scale min max -ot Byte -of VRT your_image.tif 3.vrt

Next put the bands back together (virtually):

gdalbuildvrt -separate b123.vrt b1.vrt b2.vrt b3.vrt

Now actually create your GeoTiff

gdal_translate -of gtiff b123.vrt my_beautiful_tif.tif

After that you can tile your tif or do anything else you might want to do with it. In my case I ran gdal2tiles.py

gdal2tiles.py my_beautiful_tif.tif

I suspect there may be a better way of doing things than this.  Though after a lot of researching I didn’t find one.  So this is the process that works for me.  Any suggestions?  Please, please, please leave them in the comments!

Over the past week I’ve been at Google in Mountain View working on a book sprint for OpenStreetMap. I had no idea when the week started that we were going to write a book in a week, nor that it was possible to do that. I also wasn’t familiar with FLOSS Manuals which operates under the premise that not only should software be open and free but the documentation should be as well.

The premise of the week was to bring individuals from open-source projects together as well as others specializing in documentation in as well to help. A combination of an unconference and then intensive documentation writing ensued. The OpenStreetMap group included Ian Dees, Shaun McDonald and myself from the project. We additionally had Anne Goldenberg, Anne Gentle, Tomi Toivio and Nóirín Plunkett. Having so many people from outside OpenStreetMap was extremely valuable. They were able to help us be far more clear about the materials and see past assumptions we may have made about topics. At the end we had a published book, which can be purchased from Lulu.com. Having a completed work at the end of the sprint was a great feeling. Often after hackathons that is missing for me and it ends up feeling like we just flogged ourselves with some code for a set period of time.

In going through this process I started to think about other topics that could benefit from this type of approach. I started thinking about the topics that we go over repeatedly within the Humanitarian Open-Source, CrisisMappers and Volunteer Technology Communities (for lack of a better name). I really think the community could benefit from a generic guide to issues within the humanitarian technology realm. Topics such as standards, licensing and workflows could be documented. It is impossible to say what exactly the book would cover, because that is determined as part of the book sprint process. What I am proposing though is that it would be greatly beneficial to gather for this process.

Typically a book sprint would happen over roughly a week time with five to ten individuals with different experience on the topic. I’m proposing the same thing. Let’s plan a book sprint and figure out how to make it happen. The idea would be to gather about ten people with different experience, everything from technologists to humanitarian responders to specialists in documentation. This group would get together for an intensive week long book sprint, the goal being to create a manual of the basics of humanitarian technology. Topics that come to mind for me are data licensing, standards, security, privacy and workflows. I’m only one individual and I’m sure topics would change and mold to the importance decided on by the group. The point though would be to create a resource useful within our community.

Does this sound interesting to you? Contact me in the comments or by email and let’s make it happen!

This week is the first expert meeting regarding Space Based Technology and Crowd-Sourcing being hosted by UN-SPIDER. I am not attending, but from the Twitter stream (#spidercrowd) it appears there has been a lively discussion on using satellite information and crowd-sourcing for disaster response. What I believe is often missing from these types of conversations is preparedness. Maybe it is presumptuous of me to discuss preparedness when imagery isn’t available freely every time for every disaster.

I’m typing this email from Padang, Indonesia in Western Sumatra. Yesterday I hosted a workshop at the University of Andalas to teach them about OpenStreetMap as part of the Humanitarian OpenStreetMap Team’s project in Indonesia. This is the fifth workshop I’ve done at a University in Indonesia. Things went well, much like all of the other workshops I’ve given. Students were excited and picked up the tools very quickly. The one difference? Available imagery.

This region of Indonesia is especially vulnerable to earthquakes and tsunamis, the last major earthquake in Padang was in 2009. In preparing for this workshop series I investigated each area we were going to be working to check for available imagery. For those not familiar OpenStreetMap is allowed to use Bing’s imagery to trace information, this is one of the major sources of imagery for OpenStreetMap. In checking I looked at Jakarta, Bandung, Yogyakarta and Surabaya. All four of these cities had really night, half-meter or so imagery. In Padang? It appears to just be Landsat. I think this highlights the need for imagery to be available beforehand though. If something major happened here I think it is likely imagery would become freely available. Though creating a detailed map through crowd-sourcing? That would take a week or two and be a huge effort from people that don’t know the area as well as locals. So why not map ahead of time? Imagine the power if after an event the map just had to be updated, instead of created?

There are many different ways imagery could be made available. Providers could donate it, I’m not asking for the shot the day before, fancy brand new imagery. What about imagery that is a year old or so? People purchasing imagery are usually looking for something more recent, the older stuff isn’t as valuable. An imagery fund could be created for preparedness. What about doing a highschool style fundraiser where everyone purchases and adopts a square kilometer? When purchasing imagery those doing the purchasing could negotiate for better terms on it. For example sometimes you can add additional licensees without increasing the cost. What if everyone purchasing imagery at least asked if they could license the imagery also for OpenStreetMap to use to trace vectors? Or an even more liberal license if possible.

I’m interested to hear what others think about this and if there are other ideas. Also don’t worry specifically about HOT’s project in Padang, we were fortunate to be able to purchase imagery. As I suggested about licensing above, we also made sure to extend the license to the government and our partners.